Data Readers¶

LSL Developer Primitives¶

LWA Development Primitives - A set of utilities that should make developing new analysis software easier. These functions wrap the nitty gritty of the file reading and unpacking behind Python objects.

Data format objects included are:

DRXFile
DRX8File
DRSpecFile
TBXFile
CORFile

Also included is the LWADataFile function that takes a filename and tries to determine the correct data format object to use.

Changed in version 1.2.0: Added support for LWA-SV ADP data

Changed in version 3.0.4: Added support for DRX8 data

class lsl.reader.ldp.CORFile(filename=None, fh=None, ignore_timetag_errors=False, buffering=-1)¶

Class to make it easy to interface with a COR file. COR data contain full polarization complex visibility data from each baseline pair in the array. Each channel has a bandwidth of f_C (25 kHz) and there may be up to 792 channels within a single recording. The stand numbering for the baseline pair is based on the input into the digital system rather than the stand number in the array.

Methods defined for this class are:

get_info - Get information about the file’s contents
get_remaining_frame_count - Get the number of frames remaining in the file
offset - Offset a specified number of seconds into the file
read_frame - Read and return a single lsl.reader.cor.Frame instance
read - Read in the capture and return it as a numpy array

offset(offset)¶: Offset a specified number of seconds in an open COR file. This function returns the exact offset time.

Note

The offset provided by this function is relatively crude due to the structure of COR files.

Changed in version 1.2.4: Offsets are now relative to the current location in the file rather than to the start of the file

read(duration=None, time_in_samples=False)¶

Read and return the entire COR capture. This function returns a three-element tuple with elements of: 0) the actual duration of data read in, 1) the time tag for the first sample, and 2) a 5-D Numpy array of data.

The time tag is returned as seconds since the UNIX epoch as a lsl.reader.base.FrameTimestamp instance by default. However, the time tags can be returns as samples at lsl.common.ndp.fS if the `time_in_samples’ keyword is set.

The sorting order of the output data array is by baseline.

read_frame()¶: Read and return a single lsl.reader.cor.Frame instance.

class lsl.reader.ldp.DRSpecFile(filename=None, fh=None, ignore_timetag_errors=False, buffering=-1)¶

Class to make it easy to interface with a DR Spectrometer file. DR Spectrometer data contain DRX data that has been transformed to the Fourier domain, converted to power, and integrated on-the-fly by the data recorder. These data can have various integration times, channel counts, and polarization products stored.

Methods defined for this class are:

get_info - Get information about the file’s contents
get_remaining_frame_count - Get the number of frames remaining in the file
offset - Offset a specified number of seconds into the file
read_frame - Read and return a single lsl.reader.drspec.Frame instance
read - Read a chunk of data in and return it as a numpy array

offset(offset)¶: Offset a specified number of seconds in an open DR spectrometer file. This function returns the exact offset time.

read(duration, time_in_samples=False)¶

Given an open DR spectrometer file and an amount of data read in in seconds, read in the data and return a three-element tuple of the actual duration read in, the times at the beginning of each stream, and the data as numpy array.

The time tag is returned as seconds since the UNIX epoch as a lsl.reader.base.FrameTimestamp instance by default. However, the time tags can be returns as samples at lsl.common.ndp.fS if the `time_in_samples’ keyword is set.

Note

This function always returns a 3-D array with the first dimension indexing over data product, the second over time and the third over frequency channel.

read_frame()¶: Read and return a single lsl.reader.drspec.Frame instance.

class lsl.reader.ldp.DRX8File(filename=None, fh=None, ignore_timetag_errors=False, buffering=-1)¶

Class to make it easy to interface with a DRX8 file. DRX8 data consist of a time series of complex data a variable sample rate of up to 19.6 MHz from the beamformer.

Methods defined for this class are:

get_info - Get information about the file’s contents
get_remaining_frame_count - Get the number of frames remaining in the file
offset - Offset a specified number of seconds into the file
read_frame - Read and return a single lsl.reader.drx.Frame instance
read - Read a chunk of data in and return it as a numpy array
estimate_levels - Estimate the n-sigma level for the absolute value of the voltages

estimate_levels(nframe=100, sigma=5.0)¶

Estimate the n-sigma level for the absolute value of the voltages. Returns a list with indicies corresponding to:

Tuning 1, X pol.

Tuning 1, Y pol.

Tuning 2, X pol.

Tuning 2, Y pol.

Note

The returned list always has four items, regardless of whether or not the input DRX8 file has one or two tunings.

offset(offset)¶: Offset a specified number of seconds in an open DRX8 file. This function returns the exact offset time.

read(duration, time_in_samples=False, return_ci8=False)¶

Given an open DRX8 file and an amount of data to read in in seconds, read in the data and return a three-element tuple of the actual duration read in, the time for the first sample, and the data as numpy array.

The time tag is returned as seconds since the UNIX epoch as a lsl.reader.base.FrameTimestamp instance by default. However, the time tags can be returns as samples at lsl.common.ndp.fS if the `time_in_samples’ keyword is set.

If return_ci8 is True then the data are returned will contain lsl.reader.base.CI8 data instead of numpy.complex64. The two dimensions are input by samples.

Note

This function always returns an array with the first dimension holding four elements. These elements contain, in order:

Tuning 1, polarization X

Tuning 1, polarization Y

Tuning 2, polarization X

Tuning 2, polarization Y

read_frame(return_ci8=False)¶: Read and return a single lsl.reader.drx8.Frame instance. If return_ci8 is True then the frame will contain lsl.reader.base.CI8 data instead of numpy.complex64 data.

class lsl.reader.ldp.DRXFile(filename=None, fh=None, ignore_timetag_errors=False, buffering=-1)¶

Class to make it easy to interface with a DRX file. DRX data consist of a time series of complex data a variable sample rate of up to 19.6 MHz from the beamformer.

Methods defined for this class are:

get_info - Get information about the file’s contents
get_remaining_frame_count - Get the number of frames remaining in the file
offset - Offset a specified number of seconds into the file
read_frame - Read and return a single lsl.reader.drx.Frame instance
read - Read a chunk of data in and return it as a numpy array
estimate_levels - Estimate the n-sigma level for the absolute value of the voltages

estimate_levels(nframe=100, sigma=5.0)¶

Estimate the n-sigma level for the absolute value of the voltages. Returns a list with indicies corresponding to:

Tuning 1, X pol.

Tuning 1, Y pol.

Tuning 2, X pol.

Tuning 2, Y pol.

Note

The returned list always has four items, regardless of whether or not the input DRX file has one or two tunings.

offset(offset)¶: Offset a specified number of seconds in an open DRX file. This function returns the exact offset time.

read(duration, time_in_samples=False, return_ci8=False)¶

Given an open DRX file and an amount of data to read in in seconds, read in the data and return a three-element tuple of the actual duration read in, the time for the first sample, and the data as numpy array.

The time tag is returned as seconds since the UNIX epoch as a lsl.reader.base.FrameTimestamp instance by default. However, the time tags can be returns as samples at lsl.common.ndp.fS if the `time_in_samples’ keyword is set.

If return_ci8 is True then the data are returned will contain lsl.reader.base.CI8 data instead of numpy.complex64. The two dimensions are input by samples.

Note

This function always returns an array with the first dimension holding four elements. These elements contain, in order:

Tuning 1, polarization X

Tuning 1, polarization Y

Tuning 2, polarization X

Tuning 2, polarization Y

read_frame(return_ci8=False)¶: Read and return a single lsl.reader.drx.Frame instance. If return_ci8 is True then the frame will contain lsl.reader.base.CI8 data instead of numpy.complex64 data.

lsl.reader.ldp.LWADataFile(filename=None, fh=None, ignore_timetag_errors=False, buffering=-1)¶: Wrapper around the various LWA data reader classes defined here that takes a file, determines the data type, and initializes and returns the appropriate LDP class.

class lsl.reader.ldp.TBXFile(filename=None, fh=None, ignore_timetag_errors=False, buffering=-1)¶

Class to make it easy to interface with a TBT/TBS file. TBX data are a complex frequency-domain product that contains blocks of channels from all antennas in the array. Each channel has a bandwidth of f_C (25 kHz) and there may be up to 3584 channels within a single recording. The stand ordering is based on the input into the digital system rather than the stand number in the array.

Methods defined for this class are:

get_info - Get information about the file’s contents
get_remaining_frame_count - Get the number of frames remaining in the file
offset - Offset a specified number of seconds into the file
read_frame - Read and return a single lsl.reader.tbx.Frame instance
read - Read in the capture and return it as a numpy array

offset(offset)¶: Offset a specified number of seconds in an open TBX file. This function returns the exact offset time.

read(duration=None, time_in_samples=False, return_ci8=False)¶

Given an amount of data to read in in seconds, read in the data from a TBT/TBS capture. This function returns a three-element tuple with elements of:

the actual duration of data read in,

the time tag for the first sample, and

a 3-D Numpy array of data.

The time tag is returned as seconds since the UNIX epoch as a lsl.reader.base.FrameTimestamp instance by default. However, the time tags can be returns as samples at lsl.common..ndp.fS if the `time_in_samples’ keyword is set.

The sorting order of the output data array is by digitizer number - 1.

Note

This function always returns a 3-D array with the first dimension indexing over input, the second over frequency channel and the third over time.

Note

To load in an entire file pass None for the duration. This is useful for small TBT dumps.

read_frame(return_ci8=False)¶: Read and return a single lsl.reader.tbx.Frame instance. If return_ci8 is True then the frame will contain lsl.reader.base.CI8 data instead of numpy.complex64 data.

Python module that contains various helpers for the lsl.reader module.

New in version 2.0.0.

class lsl.reader.utils.FilePositionSaver(fh)¶: Simple class to save the current location in a file and return to that position when we are done with it.

class lsl.reader.utils.SplitFileWrapper(filenames, sort=True, buffering=-1)¶: Class to allow seamless access to a file that has been split into parts.

class lsl.reader.base.FrameTimestamp(si=0, sf=0.0)¶

Class to represent the UNIX timestamp of a data frame as an integer number of seconds and a fractional number of seconds.

property astropy¶: Timestamp as an astropy.time.Time instance.

property datetime¶: Timestamp as a naive datetime.datetime instance in UTC.

property dp_timetag¶: Timestamp as a DP timetag (ticks of a 196 MHz clock since UTC midnight on January 1, 1970).

classmethod from_astropy(t)¶: Create a new FrameTimestamp from a astropy.time.Time object.

classmethod from_dp_timetag(value, offset=0)¶: Create a new FrameTimestamp instance from a raw DP timetag with an optional offset.

classmethod from_mjd_mpm(mjd, mpm)¶: Create a new FrameTimestamp from a MJD/MPM (milliseconds past midnight) pair.

classmethod from_pulsar_mjd(mjd, mjd_frac, sec_frac)¶: Create a new FrameTimstamp from a three-element tuple of integer number of MJD days, integer seconds since 0h on that day, and fractional seconds.

property jd¶: UTC JD as a floating point value.

property mjd¶: UTC MJD as a floating point value.

classmethod now()¶: Create a new FrameTimestamp instance for the current time as determined from time.time().

property pulsar_mjd¶: UTC MJD as three-element tuple of integer number of MJD days, integer number of seconds since 0h on that day, and fractional seconds.

property tai_mjd¶: TAI MJD as a floating point value.

property unix¶: UNIX timestamp as a floating point value.

property utc_datetime¶: Timestamp as a time zone-aware datetime instance in UTC.

Low-Level TBX¶

New in version 4.0.

Python module to reading in data from TBX files. TBX data are a complex frequency-domain product that contains blocks of up to 16 channels from all antennas in the array. Each channel has a bandwidth of f_C (25 kHz) and there may be up to 224 different blocks of channels within a single recording. The stand ordering is based on the input into the digital system rather than the stand number in the array.

This module defines the following classes for storing the TBX data found in a file:

Frame

object that contains all data associated with a particular TBX frame. The primary consituents of each frame are:

FrameHeader - the TBX frame header object and

FramePayload - the TBX frame data object.

Combined, these two objects contain all of the information found in the original TBX frame.

The functions defined in this module fall into two classes:

convert a frame in a file to a Frame object and
describe the format of the data in the file.

For reading in data, use the read_frame function. It takes a python file- handle as an input and returns a fully-filled Frame object.

New in version 3.1.0.

class lsl.reader.tbx.Frame(header=None, payload=None, valid=True)¶

Class that stores the information contained within a single TBX frame. It’s properties are FrameHeader and FramePayload objects.

property channel_freqs¶: Convenience wrapper for the Frame.FrameHeader.channel_freqs property.

property frame_id¶: Convenience wrapper for the Frame.FrameHeader.frame_id property.

property is_tbx¶: Convenience wrapper for the Frame.FrameHeader.is_tbx property.

property nchan¶: Convenience wrapper for the Frame.FrameHeader.nchan property.

property nstand¶: Convenience wrapper for the Frame.FrameHeader.nstand property.

property time¶: Convenience wrapper for the Frame.FramePayload.time property.

class lsl.reader.tbx.FrameHeader(frame_id=None, frame_count=None, second_count=None, first_chan=None, nstand=None, nchan=None)¶

Class that stores the information found in the header of a TBX frame.

property channel_freqs¶: Return a numpy.float32 array for the center frequencies, in Hz, of each channel in the data.

property is_tbx¶: Function to check if the data is really TBX. Returns True if the data is TBX, false otherwise.

class lsl.reader.tbx.FramePayload(timetag=None, fDomain=None)¶

Class that stores the information found in the data section of a TBX frame.

property time¶: Function to convert the time tag from samples since the UNIX epoch (UTC 1970-01-01 00:00:00) to seconds since the UNIX epoch as a lsl.reader.base.FrameTimestamp instance.

lsl.reader.tbx.get_channel_count(filehandle)¶: Find out the total number of channels that are present by examining the first 1000 TBX records. Return the number of channels found.

lsl.reader.tbx.get_first_channel(filehandle, frequency=False, all_frames=False)¶: Find and return the lowest frequency channel in a TBX file. If the frequency keyword is True the returned value is in Hz. If all_frames is True then the lowest frequency in each unique TBX frame is returned as a list.

lsl.reader.tbx.get_first_frame_count(filehandle)¶: Find and return the lowest frame count encountered in a TBX file.

lsl.reader.tbx.get_frame_size(filehandle)¶: Find out what the frame size in a file is at the current file location. Returns the frame size in bytes.

lsl.reader.tbx.get_frames_per_obs(filehandle)¶: Find out how many frames are present per time stamp by examining the first 2500 TBX records. Return the number of frames per observation.

lsl.reader.tbx.read_frame(filehandle)¶: Function to read in a single TBX frame (header+data) and store the contents as a Frame object.

lsl.reader.tbx.read_frame_ci8(filehandle)¶: Function to read in a single TBX frame (header+data) and store the contents as a Frame object.

Note

This function differs from read_frame in that it returns a lsl.reader.tbx.FramePayload that contains a 4-D numpy.int8 array (channels by stands by polarizations by real/complex) rather than a 3-D numpy.complex64 array.

New in version 2.1.3.

Low-Level COR¶

New in version 1.2.

Python module to reading in data from COR files. COR data contain full polarization complex visibility data for 132 channels from each baseline pair in the array. Each channel has a bandwidth of f_C (25 kHz) and there may be up to six different contiguous blocks of channels within a single recording. The stand numbering for the baseline pair is based on the input into the digital system rather than the stand number in the array.

This module defines the following classes for storing the COR data found in a file:

Frame

object that contains all data associated with a particular COR frame. The primary consituents of each frame are:

FrameHeader - the COR frame header object and

FramePayload - the COR frame data object.

Combined, these two objects contain all of the information found in the original COR frame.

The functions defined in this module fall into two class:

convert a frame in a file to a Frame object and
describe the format of the data in the file.

For reading in data, use the read_frame function. It takes a python file- handle as an input and returns a fully-filled Frame object.

Changed in version 1.2.1: Updated for the switch over to 72 channels, complex64 data, and no data weights

New in version 1.2.0.

class lsl.reader.cor.Frame(header=None, payload=None, valid=True)¶

Class that stores the information contained within a single COR frame. It’s properties are FrameHeader and FramePayload objects.

property adp_id¶: Convenience wrapper for the Frame.FrameHeader.adp_id property.

property channel_freqs¶: Return a numpy.float32 array for the center frequencies, in Hz, of each channel in the data.

property gain¶: Convenience wrapper for the Frame.FrameHeader.gain property.

property id¶: Convenience wrapper for the Frame.FramePayload.id property.

property integration_time¶: Convenience wrapper for the Frame.FramePayload.integration_time property.

property is_cor¶: Convenience wrapper for the Frame.FrameHeader.is_cor property.

property time¶: Convenience wrapper for the Frame.FramePayload.time property.

class lsl.reader.cor.FrameHeader(adp_id=None, frame_count=None, second_count=None, first_chan=None, gain=None)¶

Class that stores the information found in the header of a COR frame. All three fields listed in the DP ICD version H are stored as well as the original binary header data.

property is_cor¶: Function to check if the data is really COR. Returns True if the data is COR, false otherwise.

class lsl.reader.cor.FramePayload(timetag=None, navg=None, stand0=None, stand1=None, vis=None)¶

Class that stores the information found in the data section of a COR frame.

property id¶: Return a tuple of the two stands that contribute the this frame.

property integration_time¶: Return the integration time of the visibility in seconds.

property time¶: Function to convert the time tag from samples since the UNIX epoch (UTC 1970-01-01 00:00:00) to seconds since the UNIX epoch as a lsl.reader.base.FrameTimestamp instance.

lsl.reader.cor.get_baseline_count(filehandle)¶: Find out the total number of baselines that are present by examining the first several COR records. Return the number of baselines found.

lsl.reader.cor.get_channel_count(filehandle)¶: Find out the total number of channels that are present by examining the first several COR records. Return the number of channels found.

lsl.reader.cor.get_first_channel(filehandle, frequency=False, all_frames=False)¶: Find and return the lowest frequency channel in a TBX file. If the frequency keyword is True the returned value is in Hz. If all is True then the lowest frequency in each unique TBX frame is returned as a list.

lsl.reader.cor.get_frame_size(filehandle)¶: Find out what the frame size in a file is at the current file location. Returns the frame size in bytes.

lsl.reader.cor.get_frames_per_obs(filehandle)¶: Find out how many frames are present per time stamp by examining the first several COR records. Return the number of frames per observation.

lsl.reader.cor.read_frame(filehandle)¶: Function to read in a single COR frame (header+data) and store the contents as a Frame object.

Low-Level DRX¶

Python module to read in DRX data. DRX data consist of a time series of complex data a variable sample rate of up to 19.6 MHz from the beamformer. The data are divided into packets of 4096 samples per beam per tuning per polarization.

This module defines the following classes for storing the DRX data found in a file:

Frame

object that contains all data associated with a particular DRX frame. The primary constituents of each frame are:

FrameHeader - the DRX frame header object and

FramePayload - the DRX frame data object.

Combined, these two objects contain all of the information found in the original DRX frame.

ObservingBlock object that stores a collection of Frames for all beams/tunings/ polarizations for a particular time.

The functions defined in this module fall into two class:

convert a frame in a file to a Frame object and
describe the format of the data in the file.

For reading in data, use the read_frame function. It takes a python file- handle as an input and returns a fully-filled Frame object. The readBlock function reads in a (user-defined) number of DRX frames and returns a ObservingBlock object.

For describing the format of data in the file, two function are provided:

get_beam_count: read in the first few frames of an open file handle and return how many beams are present in the file.
get_frames_per_obs: read in the first several frames to see how many frames (tunings/polarizations) are associated with each beam.
..versionchanged:: 2.1.3: Added a new read_frame_ci8 function that returns 8-bit+8-bit complex integers instead of numpy.complex64
..versionchanged:: 1.2.0: Dropped support for ObservingBlock since the lsl.reader.buffer modules does a better job.

Changed in version 0.4.0: Switched over from pure Python readers to the new C-base Go Fast! readers.

lsl.reader.drx.FILTER_CODES = {1: 250000, 2: 500000, 3: 1000000, 4: 2000000, 5: 4900000, 6: 9800000, 7: 19600000}¶: List of filter codes and their corresponding sample rates in Hz

lsl.reader.drx.FRAME_SIZE = 4128¶: DRX packet size (header + payload)

class lsl.reader.drx.Frame(header=None, payload=None, valid=True)¶

Class that stores the information contained within a single DRX frame. It’s properties are FrameHeader and FramePayload objects.

property central_freq¶: Convenience wrapper for the Frame.FramePayload.central_freq property.

property filter_code¶: Convenience wrapper for the Frame.FrameHeader.filter_code property.

property id¶: Convenience wrapper for the Frame.FrameHeader.id property.

property is_drx¶: Convenience wrapper for the Frame.FrameHeader.is_drx property.

property sample_rate¶: Convenience wrapper for the Frame.FrameHeader.sample_rate property.

property time¶: Function to convert the time tag from samples since the UNIX epoch (UTC 1970-01-01 00:00:00) to seconds since the UNIX epoch as a lsl.reader.base.FrameTimestamp instance.

class lsl.reader.drx.FrameHeader(frame_count=None, drx_id=None, second_count=None, decimation=None, time_offset=None)¶

Class that stores the information found in the header of a DRX frame. All six fields listed in the DP ICD version H are stored as well as the original binary header data.

property filter_code¶: Function to convert the sample rate in Hz to a filter code.

property id¶: Parse the DRX ID into a tuple containing the beam (1 through 4), tunning (1 and 2), and polarization (0 and 1).

property is_drx¶: Function to check if the data is really DRX and not DRX8 by examining the drx_id field. Returns True if the data is DRX, false otherwise.

property sample_rate¶: Return the sample rate of the data in samples/second.

class lsl.reader.drx.FramePayload(timetag=None, tuning_word=None, flags=None, iq=None)¶

Class that stores the information found in the data section of a DRX frame. All three fields listed in the DP ICD version H are stored.

property central_freq¶: Function to set the central frequency of the DRX data in Hz.

lsl.reader.drx.get_beam_count(filehandle)¶: Find out how many beams are present by examining the first 32 DRX records. Return the number of beams found.

lsl.reader.drx.get_frames_per_obs(filehandle)¶: Find out how many frames are present per beam by examining the first 32 DRX records. Return the number of frames per observations as a four- element tuple, one for each beam.

lsl.reader.drx.get_sample_rate(filehandle, nframes=None, filter_code=False)¶

Find out what the sampling rate/filter code is from a single observations. By default, the rate in Hz is returned. However, the corresponding filter code can be returned instead by setting the FilterCode keyword to true.

This function is included to make easier to write code for TBN analysis and modify it for DRX data.

lsl.reader.drx.read_frame(filehandle, gain=None)¶: Function to read in a single DRX frame (header+data) and store the contents as a Frame object. This function wraps readerHeader and readData.

lsl.reader.drx.read_frame_ci8(filehandle, gain=None)¶: Function to read in a single DRX frame (header+data) and store the contents as a Frame object. This function wraps readerHeader and readData.

Note

This function differs from read_frame in that it returns a lsl.reader.drx.FramePayload that contains a 2-D numpy.int8 array (samples by real/complex) rather than a 1-D numpy.complex64 array.

New in version 2.1.3.

Low-Level DRX8¶

New in version 4.0.

Python module to read in DRX8 data. DRX8 data consist of a time series of complex data a variable sample rate of up to 19.6 MHz from the beamformer. The data are divided into packets of 4096 samples per beam per tuning per polarization.

This module defines the following classes for storing the DRX8 data found in a file:

Frame

object that contains all data associated with a particular DRX8 frame. The primary constituents of each frame are:

FrameHeader - the DRX8 frame header object and

FramePayload - the DRX8 frame data object.

Combined, these two objects contain all of the information found in the original DRX8 frame.

ObservingBlock object that stores a collection of Frames for all beams/tunings/ polarizations for a particular time.

The functions defined in this module fall into two class:

convert a frame in a file to a Frame object and
describe the format of the data in the file.

For reading in data, use the read_frame function. It takes a python file- handle as an input and returns a fully-filled Frame object. The readBlock function reads in a (user-defined) number of DRX8 frames and returns a ObservingBlock object.

For describing the format of data in the file, two function are provided:

get_beam_count: read in the first few frames of an open file handle and return how many beams are present in the file.
get_frames_per_obs: read in the first several frames to see how many frames (tunings/polarizations) are associated with each beam.
..versionchanged:: 1.2.0: Dropped support for ObservingBlock since the lsl.reader.buffer modules does a better job.

Changed in version 0.4.0: Switched over from pure Python readers to the new C-base Go Fast! readers.

lsl.reader.drx8.FILTER_CODES = {1: 250000, 2: 500000, 3: 1000000, 4: 2000000, 5: 4900000, 6: 9800000, 7: 19600000}¶: List of filter codes and their corresponding sample rates in Hz

lsl.reader.drx8.FRAME_SIZE = 8224¶: DRX8 packet size (header + payload)

class lsl.reader.drx8.Frame(header=None, payload=None, valid=True)¶

Class that stores the information contained within a single DRX8 frame. It’s properties are FrameHeader and FramePayload objects.

property central_freq¶: Convenience wrapper for the Frame.FramePayload.central_freq property.

property filter_code¶: Convenience wrapper for the Frame.FrameHeader.filter_code property.

property id¶: Convenience wrapper for the Frame.FrameHeader.id property.

property is_drx8¶: Convenience wrapper for the Frame.FrameHeader.is_drx8 property.

property sample_rate¶: Convenience wrapper for the Frame.FrameHeader.sample_rate property.

property time¶: Function to convert the time tag from samples since the UNIX epoch (UTC 1970-01-01 00:00:00) to seconds since the UNIX epoch as a lsl.reader.base.FrameTimestamp instance.

class lsl.reader.drx8.FrameHeader(frame_count=None, drx_id=None, second_count=None, decimation=None, time_offset=None)¶

Class that stores the information found in the header of a DRX8 frame. All six fields listed in the DP ICD version H are stored as well as the original binary header data.

property filter_code¶: Function to convert the sample rate in Hz to a filter code.

property id¶: Parse the DRX8 ID into a tuple containing the beam (1 through 4), tunning (1 and 2), and polarization (0 and 1).

property is_drx8¶: Function to check if the data is really DRX8 and not DRX by examining the drx_id field. Returns True if the data is DRX8, false otherwise.

property sample_rate¶: Return the sample rate of the data in samples/second.

class lsl.reader.drx8.FramePayload(timetag=None, tuning_word=None, flags=None, iq=None)¶

Class that stores the information found in the data section of a DRX8 frame. All three fields listed in the DP ICD version H are stored.

property central_freq¶: Function to set the central frequency of the DRX8 data in Hz.

lsl.reader.drx8.get_beam_count(filehandle)¶: Find out how many beams are present by examining the first 32 DRX8 records. Return the number of beams found.

lsl.reader.drx8.get_frames_per_obs(filehandle)¶: Find out how many frames are present per beam by examining the first 32 DRX8 records. Return the number of frames per observations as a four- element tuple, one for each beam.

lsl.reader.drx8.get_sample_rate(filehandle, nframes=None, filter_code=False)¶

Find out what the sampling rate/filter code is from a single observations. By default, the rate in Hz is returned. However, the corresponding filter code can be returned instead by setting the FilterCode keyword to true.

This function is included to make easier to write code for TBN analysis and modify it for DRX8 data.

lsl.reader.drx8.read_frame(filehandle, gain=None)¶: Function to read in a single DRX8 frame (header+data) and store the contents as a Frame object. This function wraps readerHeader and readData.

lsl.reader.drx8.read_frame_ci8(filehandle, gain=None)¶: Function to read in a single DRX8 frame (header+data) and store the contents as a Frame object. This function wraps readerHeader and readData.

Low-Level DR Spectrometer¶

New in version 0.5.

Python module to read in DR spectrometer data. DR Spectrometer data contain DRX data that has been transformed to the Fourier domain, converted to power, and integrated on-the-fly by the data recorder. These data can have various integration times, channel counts, and polarization products stored.

This module defines the following classes for storing the spectra found in a file:

Frame

object that contains all data associated with a particular spectrometer frame. The primary constituents of each frame are:

FrameHeader - the spectrometer frame header object and

FramePayload - the spectral data object.

Combined, these two objects contain all of the information found in the original spectrometer frame.

The functions defined in this module fall into two class:

convert a frame in a file to a Frame object and
describe the format of the data in the file.

For reading in data, use the read_frame function. It takes a python file- handle as an input and returns a fully-filled Frame object For describing the format of data in the file, three function are provided:

get_sample_rate - get the sample rate in the file

getFRAME_SIZE - get the total (header + data) frame size

get_ffts_per_integration - get the number of FFT windows per integration

get_transform_size - get the FFT length

get_integration_time - get the integration time

Note

This reader works with the most current version of the DR spectrometer data format as specified in the version 1.7. To read data created with previous versions of the DR spectrometer, use LSL version 0.5.2.

Changed in version 1.0.1: Added in new functions to help better describe the contents of a DR spectrometer file.

lsl.reader.drspec.FILTER_CODES = {1: 250000, 2: 500000, 3: 1000000, 4: 2000000, 5: 4900000, 6: 9800000, 7: 19600000}¶

List of filter codes and their corresponding sample rates in Hz. .. note:

These are just the DRX filter codes

class lsl.reader.drspec.Frame(header=None, payload=None, valid=True)¶

Class that stores the information contained within a single DR spectrometer/ DRX frame. It’s properties are FrameHeader and FramePayloadLinear/FramePayloadStokes objects.

Changed in version 0.6.0: By default the data contained with in a frame is normalized by the number of fills (header.fills parameter). For data products that are a function of more than one primary input, i.e., real(XY*) or I, the minimum fill of X and Y are used for normalization.

property central_freq¶: Convenience wrapper for the Frame.FramePayload.central_freq property.

property data_products¶: Convenience wrapper for the Frame.FrameHeder.data_products property.

property ffts_per_integration¶: Conveinence wrapper for the Frame.FrameHeader.ffts_per_integration property.

New in version 1.0.1.

property filter_code¶: Convenience wrapper for the Frame.FrameHeader.filter_code property.

property id¶: Convenience wrapper for the Frame.FrameHeader.id property.

property integration_time¶: Return the integration time for data in seconds.

New in version 1.0.1.

property is_linear¶: Convenience wrapper for the Frame.FrameHeder.is_linear property.

property is_stokes¶: Convenience wrapper for the Frame.FrameHeder.is_stokes property.

property sample_rate¶: Convenience wrapper for the Frame.FrameHeader.sample_rate property.

property time¶: Function to convert the time tag from samples since the UNIX epoch (UTC 1970-01-01 00:00:00) to seconds since the UNIX epoch as a lsl.reader.base.FrameTimestamp instance.

property transform_size¶: Find out what the transform size is.

New in version 1.0.1.

class lsl.reader.drspec.FrameHeader(beam=0, format=0, decimation=None, time_offset=None, nints=None)¶

Class that stores the information found in the header of a DR spectrometer/DRX frame.

property data_products¶: Return a list of data products contained in the file.

New in version 0.6.0.

property ffts_per_integration¶: Return the number of FFT windows per integration.

New in version 1.0.1.

property filter_code¶: Function to convert the sample rate in Hz to a filter code.

property id¶: Return the beam the frame corresponds to.

property is_linear¶: Return whether or not the frame contains linear polarization products or not.

New in version 0.6.0.

property is_stokes¶: Return whether or not the frame contains Stokes polarization parameters or not.

New in version 0.6.0.

property sample_rate¶: Return the sample rate of the data in samples/second.

class lsl.reader.drspec.FramePayload(timetag=None, tuning_words=None, fills=None, errors=None, saturations=None)¶

Class that stores the information found in the data section of a DR spectrometer/ DRX frame.

Changed in version 0.5.3: Added the saturations field to keep up with saturation counts.

Changed in version 0.6.0: The attributes that store the data are not defined until a frame is read in order to account for the fact that spectrometer files can store either linear or Stokes data.

property central_freq¶: Function to set the central frequency of the DRX data in Hz.

lsl.reader.drspec.get_data_products(filehandle)¶: Find out the data products contained in the file by looking at a frame.

lsl.reader.drspec.get_ffts_per_integration(filehandle)¶: Find out what the number of FFT windows per integration is at the current file location.

New in version 1.0.1.

lsl.reader.drspec.get_frame_size(filehandle)¶: Find out what the frame size in a file is at the current file location. Returns the frame size in bytes.

lsl.reader.drspec.get_integration_time(filehandle)¶: Find out what the integration time is at the current file location.

lsl.reader.drspec.get_sample_rate(filehandle, nframes=None, filter_code=False)¶: Find out what the sampling rate/filter code is from a single observations. By default, the rate in Hz is returned. However, the corresponding filter code can be returned instead by setting the FilterCode keyword to true.

lsl.reader.drspec.get_transform_size(filehandle)¶: Find out what the transform size in a file is at the current file location.

lsl.reader.drspec.is_linear(filehandle)¶: Find out if the file contains linear polarization products or not.

lsl.reader.drspec.is_stokes(filehandle)¶: Find out if the file contains Stokes parameters or not.

lsl.reader.drspec.read_frame(filehandle, gain=None)¶: Function to read in a single DR spectrometer/DRX frame (header+data) and store the contents as a Frame object.

Low-Level VDIF¶

Python module to read in VDIF data. This module defines the following classes for storing the VIDF data found in a file:

Frame

object that contains all data associated with a particular DRX frame. The primary constituents of each frame are:

FrameHeader - the VDIF frame header object and

FramePayload - the VDIF frame data object.

Combined, these two objects contain all of the information found in the original VDIF frame.

The functions defined in this module fall into two class:

convert a frame in a file to a Frame object and
describe the format of the data in the file.

For reading in data, use the read_frame function. It takes a python file- handle as an input and returns a fully-filled Frame object. The readBlock function reads in a (user-defined) number of VDIF frames and returns a ObservingBlock object.

For describing the format of data in the file, two function are provided:

get_thread_count: read in the first few frames of an open file handle and return how many threads are present in the file.

class lsl.reader.vdif.Frame(header=None, payload=None, valid=True)¶

Class that stores the information contained within a single VDIF frame. It’s properties are FrameHeader and FramePayload objects.

property central_freq¶: Convenience wrapper for the Frame.FrameHeader.central_freq property.

property extended_user_data¶: Convenience wrapper for the Frame.FrameHeader.extended_user_data property.

property id¶: Convenience wrapper for the Frame.FrameHeader.id property.

property sample_rate¶: Convenience wrapper for the Frame.FrameHeader.sample_rate property.

property time¶: Convenience wrapper for the Frame.FrameHeader.time property.

class lsl.reader.vdif.FrameHeader(is_invalid=0, is_legacy=0, seconds_from_epoch=0, ref_epoch=0, frame_in_second=0, version=1, nchan=0, frame_length=0, is_complex='C', bits_per_sample=0, thread_id=0, station_id=0, extended_data_1=None, extended_data_2=None, extended_data_3=None, extended_data_4=None, sample_rate=0.0, central_freq=0.0)¶

Class that stores the information found in the header of a VDIF frame. Most fields in the VDIF version 1.1.1 header are stored.

property extended_user_data¶: Parse the extended user data if it was included with the reader. The data is returned as a dictionary.

property id¶: Return a two-element tuple of the station ID and thread ID.

Note

The station ID is always returned as numeric.

property time¶: Function to convert the time tag to seconds since the UNIX epoch as a lsl.reader.base.FrameTimestamp instance.

class lsl.reader.vdif.FramePayload(data=None)¶: Class that stores the information found in the data section of a VDIF frame.

Note

Unlike the other readers in the lsl.reader module the data are stored as numpy.float32 values.

lsl.reader.vdif.get_frame_size(filehandle, nframes=None)¶: Find out what the frame size is in bytes from a single observation.

lsl.reader.vdif.get_frames_per_second(filehandle)¶: Find out the number of frames per second in a file by watching how the headers change. Returns the number of frames in a second.

lsl.reader.vdif.get_sample_rate(filehandle)¶: Find and return the sample rate in Hz by looking at how many frames there are per second and how many samples there are in a frame.

lsl.reader.vdif.get_thread_count(filehandle)¶: Find out how many thrads are present by examining the first 1024 records. Return the number of threads found.

lsl.reader.vdif.has_guppi_header(filehandle)¶: Determine if a VDIF file has a GUPPI header or not.

New in version 2.0.0.

lsl.reader.vdif.read_frame(filehandle, sample_rate=0.0, central_freq=0.0)¶: Function to read in a single VDIF frame (header+data) and store the contents as a Frame object. This function wraps the _readerHeader and _readData functions.

lsl.reader.vdif.read_frame_i8(filehandle, sample_rate=0.0, central_freq=0.0)¶

Function to read in a single VDIF frame (header+data) and store the contents as a Frame object. This function wraps the _readerHeader and _readData functions.

Note

This function differs from read_vdif in that it returns a lsl.vdif.FramePayload containing numpy.int8 array rather than a numpy.float32 array for real data. Complex data is always returned as numpy.complex64.

Note

The unpacking to integer does not match that for float for 2-, 4-, and 8-bit data. Specifically:

For 2-bit data: (-3.3359, -1, 1, 3.3359) -> (-3, -1, 1, 3)

For 4-bit data: (-8/2.95, -7/2.95, …, 7/2.95) -> (-8, -7, …, 7)

For 8-bit data: (-255/256, -253/256, …, 255/256) -> (-128, -127, … 127)

New in version 2.1.3.

lsl.reader.vdif.read_guppi_header(filehandle)¶: Read in a GUPPI header at the start of a VDIF file from the VLA. The contents of the header are returned as a dictionary.

Reader Error Codes¶

Module that contains the error classes for the DRX, TBN, and TBW readers. These errors are currently meant to deal with file I/O problems.

exception lsl.reader.errors.BaseReaderError(strerror, errno='-1', filename=None)¶: Base class for file I/O problem during numpy.fromfile calls and out-of- sync Mark5C headers.

exception lsl.reader.errors.EOFError¶: Extension to the base class for dealing with EOF errors. The error code is 1.

exception lsl.reader.errors.SyncError(type='Mark5C', location=None, sync1=None, sync2=None, sync3=None, sync4=None)¶: Extension to the base class for dealing with Mark 5C header sync word problems. If the sync word doesn’t match what is expected. The error code is 2.

lsl.reader.errors.list_error_codes(errno=None)¶: Function to provide a list of errors defined in this file. It alternatively takes an error code using the ‘errno’ keyword and returns its description.

Reader Ring Buffer¶

Buffer for dealing with out-of-order/missing frames.

Changed in version 0.5: Removed support for DRX FrameBuffers since they shouldn’t be needed.

Changed in version 0.6: Removed support for TBW FrameBuffers since they didn’t really work.

Changed in version 1.0.0: Added back in the DRX FrameBuffers since they might be useful. Dropped TBW support from within the FrameBufferBase class.

Changed in version 1.0.2: Added support for the LWA-SV ADP DRX8 mode

Changed in version 1.1.4: Dropped support for the LWA-SV ADP DRX8 mode

Changed in version 1.2.1: Added support for the LWA-SV TBF and COR modes

Changed in version 3.0.2: Added support for the DRX8 mode (again).

class lsl.reader.buffer.DRX8FrameBuffer(beams=[], tunes=[1, 2], pols=[0, 1], nsegments=20, reorder=False)¶

Bases: FrameBufferBase

A sub-type of FrameBufferBase specifically for dealing with DRX8 frames. See lsl.reader.buffer.FrameBufferBase for a description of how the buffering is implemented.

Keywords:

beams: list of beam to expect packets for
tunes: list of tunings to expect packets for
pols: list of polarizations to expect packets for
nsegments: number of ring segments to use for the buffer (default is 20)
reorder: whether or not to reorder frames returned by get() or flush() by stand/polarization (default is False)

The number of segements in the ring can be converted to a buffer time in seconds:

Segments	DRX8 Filter Code
Segments	1	2	3	4	5	6	7
10	0.16	0.08	0.04	0.02	0.01	0.004	0.002
20	0.33	0.16	0.08	0.04	0.02	0.008	0.004
30	0.49	0.25	0.12	0.06	0.03	0.013	0.006
40	0.66	0.33	0.16	0.08	0.03	0.017	0.008
50	0.82	0.41	0.20	0.10	0.04	0.021	0.010
100	1.64	0.82	0.41	0.20	0.08	0.042	0.021

create_fill(key, frameParameters)¶: Create a ‘fill’ frame of zeros using an existing good packet as a template.

get_figure_of_merit(frame)¶

Figure of merit for sorting frames. For DRX it is:: <frame timetag in ticks>

get_max_frames()¶: Calculate the maximum number of frames that we expect from the setup of the observations and a list of tuples that describes all of the possible stand/pol combination.

class lsl.reader.buffer.DRXFrameBuffer(beams=[], tunes=[1, 2], pols=[0, 1], nsegments=20, reorder=False)¶

Bases: FrameBufferBase

A sub-type of FrameBufferBase specifically for dealing with DRX frames. See lsl.reader.buffer.FrameBufferBase for a description of how the buffering is implemented.

Keywords:

beams: list of beam to expect packets for
tunes: list of tunings to expect packets for
pols: list of polarizations to expect packets for
nsegments: number of ring segments to use for the buffer (default is 20)
reorder: whether or not to reorder frames returned by get() or flush() by stand/polarization (default is False)

The number of segements in the ring can be converted to a buffer time in seconds:

Segments	DRX Filter Code
Segments	1	2	3	4	5	6	7
10	0.16	0.08	0.04	0.02	0.01	0.004	0.002
20	0.33	0.16	0.08	0.04	0.02	0.008	0.004
30	0.49	0.25	0.12	0.06	0.03	0.013	0.006
40	0.66	0.33	0.16	0.08	0.03	0.017	0.008
50	0.82	0.41	0.20	0.10	0.04	0.021	0.010
100	1.64	0.82	0.41	0.20	0.08	0.042	0.021

create_fill(key, frameParameters)¶: Create a ‘fill’ frame of zeros using an existing good packet as a template.

get_figure_of_merit(frame)¶

Figure of merit for sorting frames. For DRX it is:: <frame timetag in ticks>

get_max_frames()¶: Calculate the maximum number of frames that we expect from the setup of the observations and a list of tuples that describes all of the possible stand/pol combination.

class lsl.reader.buffer.FrameBufferBase(mode='DRX', stands=[], beams=[], tunes=[], pols=[], chans=[], threads=[], nsegments=6, reorder=False)¶

Bases: object

Frame buffer for re-ordering TBN and DRX frames in time order. This class is filled with frames and a returns a frame list when the ‘nsegments’ starts filling. In that case, the oldest segment is returned.

The buffer also keeps track of what has already been read out so that tardy frames are just dropped. For buffers that are read out, missing frames are replaced with frames filled with zeros.

Note

Due to the nature of the buffer, it is possible that there will still be ‘nsegments’-1 segements in the buffer that are either full or partially full. This can be retrieved using the buffer’s ‘flush()’ function.

append(frames)¶: Append a new frame to the buffer with the appropriate time tag. True is returned if the frame was added to the buffer and False if the frame was dropped because it belongs to a buffer that has already been returned.

abstract create_fill(key, frameParameters)¶

Create a ‘fill’ frame of zeros using an existing good packet as a template.

This will be overridden by sub-classes of FrameBufferBase.

property filled¶: Indicates whether or not the ring buffer is full or not.

New in version 1.2.4.

flush()¶: Generator to return all of the remaining frames in the buffer from buffers that are considered ‘full’. Afterwards, delete all buffers. This is useful for emptying the buffer after reading in all of the data.

Note

It is possible for this function to return list of packets that are mostly invalid.

get(key_to_return=None)¶: Return a list of frames that constitute a ‘full’ buffer. Afterwards, delete that buffer and mark it as closed so that any missing frames that are received late are dropped. If none of the buffers are ready to be dumped, None is returned.

abstract get_figure_of_merit(frame)¶

Figure of merit for storing/sorting frames in the ring buffer.

This will be overridden by sub-classes of FrameBufferBase.

abstract get_max_frames()¶

Calculate the maximum number of frames that we expect from the setup of the observations and a list of tuples that describes all of the possible stand/pol combination.

This will be overridden by sub-classes of FrameBufferBase.

is_empty()¶: Determine if there is anything in the buffer or not. Returns False if there is, True if there is not.

property overfilled¶: Indicates whether or not the ring buffer has too many segements or not.

New in version 1.2.4.

peek(require_filled=True)¶

Peek into the buffer to see what the next key to be retruned by a call to get() will be. Returns None if the buffer is not full and

require_filled is True.

New in version 1.2.4.

put(*args, **kwds)¶: Synonymous with ‘append’.

reset()¶: Emtpy the contents of the buffer and reset it to a clean state.

status()¶

Print out the status of the buffer. This contains information about:

The current buffer fill level
The numer of full and partial buffer dumps preformed
The number of missing frames that fill packets needed to be created for
The number of frames that arrived too late to be incorporated into one of the ring buffers

class lsl.reader.buffer.TBXFrameBuffer(chans, nsegments=25, reorder=False)¶

Bases: FrameBufferBase

A sub-type of FrameBufferBase specifically for dealing with TBX frames. See lsl.reader.buffer.FrameBufferBase for a description of how the buffering is implemented.

Keywords:

chans: list of start channel numbers to expect data for
nsegments: number of ring segments to use for the buffer (default is 25)
reorder: whether or not to reorder frames returned by get() or flush() by start channel (default is False)

The number of segements in the ring can be converted to a buffer time in seconds:

Segments	Time
10	0.0004
25	0.001
50	0.002
100	0.004
200	0.008

create_fill(key, frameParameters)¶: Create a ‘fill’ frame of zeros using an existing good packet as a template.

get_figure_of_merit(frame)¶: Figure of merit for sorting frames. For TBF this is: frame.payload.timetag

get_max_frames()¶: Calculate the maximum number of frames that we expect from the setup of the observations and a list of tuples that describes all of the possible stand/pol combination.

class lsl.reader.buffer.VDIFFrameBuffer(threads=[0, 1], nsegments=10, reorder=False)¶

Bases: FrameBufferBase

A sub-type of FrameBufferBase specifically for dealing with VDIF frames. See lsl.reader.buffer.FrameBufferBase for a description of how the buffering is implemented.

Keywords:

threads: list of thread IDs to expect data for
nsegments: number of ring segments to use for the buffer (default is 10)
reorder: whether or not to reorder frames returned by get() or flush() by stand/polarization (default is False)

create_fill(key, frameParameters)¶: Create a ‘fill’ frame of zeros using an existing good packet as a template.

get_figure_of_merit(frame)¶: Figure of merit for sorting frames. For VIDF this is: seconds_from_epoch * 1000000 + frame_in_second

get_max_frames()¶: Calculate the maximum number of frames that we expect from the setup of the observations and a list of tuples that describes all of the possible stand/pol combination.

Data Readers¶

LSL Developer Primitives¶

Low-Level TBX¶

Low-Level COR¶

Low-Level DRX¶

Low-Level DRX8¶

Low-Level DR Spectrometer¶

Low-Level VDIF¶

Reader Error Codes¶

Reader Ring Buffer¶

Table of Contents

Previous topic

Next topic

This Page