luv_finder.data

Measurement-set I/O and visibility-domain operations.

Visibilities are held per (field, spectral window) in a Chunk: the weighted visibilities X = w * V and the weight of every visibility w (0 where flagged) as (n_chan, n_row) arrays, and what is constant along an axis (u, v, time, baseline and scan per row, frequency per channel) once. The weights are corrected on reading, per scan and channel, by the noise measured in the data (reweighted()).

Positions are offsets in one sky frame per dataset: arcsec east and north of Metadata.ref, the model’s (dra, ddec) convention. Each chunk carries its field’s phase centre in that frame, so the same (dra, ddec) means the same sky position in every pointing.

CASA (casatools) is imported lazily inside the functions that read a measurement set, so the NPZ path (DataHandler.from_npz()) works in an environment without CASA.

Module Attributes

CHUNK_ARRAYS

Arrays stored per chunk in an NPZ, under {field}_{spw}_{name}.

READ_BLOCK

Target size of one block of rows read from a measurement set, in visibility-channels.

MAX_GAP

Largest time between two integrations of a baseline that measure the weights' noise together, s.

CHANNEL_BLOCK

Channels handled at once when the weights are measured.

MAX_LAG

Longest channel lag channel_noise() measures, and the significance a lag's correlation needs to count.

LAG_SIGNIFICANCE

Longest channel lag channel_noise() measures, and the significance a lag's correlation needs to count.

FREQUENCY_FRAMES

casacore's MFrequency reference codes (MEAS_FREQ_REF of an MS's SPECTRAL_WINDOW).

Functions

channel_noise(jackknife)

Of one window's jackknife (Chunk.jackknife()): the noise power per channel over the weights, sum |X|^2 / 2 sum w, and the channels' correlation (1, rho_1, ...) up to the first lag consistent with 0.

check_frame(msfile, codes)

Warn unless every window is in LSRK: the search, the catalogue and the fit keep the MS's frame.

fields_in(path)

Target fields of a measurement set, an NPZ or a list of NPZs, sorted; an NPZ is not read beyond its index.

iter_chunks(msfile, ref[, fields, spws, ...])

Read target visibilities one spw at a time over all fields, weights corrected, offsets from ref.

load(path[, fields])

The visibilities of fields (default: all) from a measurement set, an NPZ or a list of one dataset's NPZs.

pair_weight(a, b)

Inverse variance of (x_a +- x_b) / 2 from the weights of x_a and x_b; 0 if either is.

pointing(path, field)

A field's dataset metadata, phase centre (arcsec, the data's sky frame) and lowest frequency (Hz), from a measurement set, an NPZ or a list of NPZs, without reading its visibilities.

reweighted(chunks[, ws])

The chunks of one window, weights divided by ws (default weight_scale()) per scan and channel.

sky_direction(offset, ref)

(RA, Dec) in rad of the (east, north) offset in arcsec from ref, the inverse of sky_offset().

sky_offset(direction, ref)

(east, north) offset in arcsec of direction on the tangent plane at ref, both (RA, Dec) in rad.

stokes_i(data, weight, flag)

Average the parallel hands [0, -1] into (vis, w); weight per row or per channel, w 0 if flagged.

target_selection(msfile)

Field and spectral-window ids observed with the target intent.

weight_scale(chunks)

sum w z^2 / 2 sum w per (scan, channel) over a window's integrations of a baseline <= MAX_GAP apart.

write_npz(path, metadata, chunks)

Write an NPZ one array at a time, so chunks may be a generator that never sits in memory whole.

Classes

ChannelNoise(scale, rho)

Noise of each channel over its weights, scale (1 where they hold), and between channels, rho.

Chunk(field, spw, offset, freq, u, v, time, ...)

Visibilities of one (field, spectral window), channel-major.

DataHandler([msfile, chunks, metadata, ...])

Visibilities of a measurement set, one Chunk per (field, spectral window).

Metadata(dish_diameter, ref, ...)

Quantities of the whole dataset, the same whichever fields or windows are loaded.

WeightScale(scans, scale, check, error)

Noise power over the weights per scan and channel, scale, its robust check and that one's error.

luv_finder.data.CHANNEL_BLOCK = 16

Channels handled at once when the weights are measured.

luv_finder.data.CHUNK_ARRAYS = ('offset', 'freq', 'u', 'v', 'time', 'baseline', 'scan', 'X', 'w')

Arrays stored per chunk in an NPZ, under {field}_{spw}_{name}.

class luv_finder.data.ChannelNoise(scale: np.ndarray, rho: np.ndarray)[source]

Noise of each channel over its weights, scale (1 where they hold), and between channels, rho.

rho: ndarray

Alias for field number 1

scale: ndarray

Alias for field number 0

class luv_finder.data.Chunk(field: int, spw: int, offset: ndarray, freq: ndarray, u: ndarray, v: ndarray, time: ndarray, baseline: ndarray, scan: ndarray, X: ndarray, w: ndarray)[source]

Visibilities of one (field, spectral window), channel-major.

X = w * V is zero where w is (flagged), so a weighted sum over visibilities is a plain sum over X.

Attributes:
offset(2,) arcsec, the field’s phase centre in the dataset’s sky frame
freq(n_chan,) Hz, ascending
u, v(n_row,) m
time(n_row,) s
baseline(n_row,) int, identifies the antenna pair
scan(n_row,) int64, the scan, OBSERVATION_ID << 32 | SCAN_NUMBER
X(n_chan, n_row) complex, weighted visibilities
w(n_chan, n_row) float32, inverse variance of each visibility, 0 where flagged
binned(n: int, rho=None) → tuple[Chunk, ndarray][source]

The channels averaged in runs of n (the last shorter unless n divides them) and the binned channels’ correlation.

X and w add, an inverse-variance average at each run’s mean frequency. Under the channels’ correlation rho ((1, rho_1, ...)) a run’s variance is sum_kl rho_|k-l| times the independent one: the weights are divided by it, and neighbouring runs correlate by the rho spanning them.

jackknife() → Chunk[source]

A signal-free noise realisation: consecutive integrations of every baseline differenced and halved.

Each baseline’s integrations are paired in time order, its first with its second, third with fourth, …, whatever the other baselines do, so a mosaic field seen once per visit (or a baseline missing from some integrations) still pairs nearly every row; an odd last integration is left out. A signal constant over a pair cancels; the noise stays, with weight pair_weight (2w for equal weights).

The split is by integration, never random: a random sign flip of individual visibilities destroys the baseline structure and does not give the observation’s white-noise level.

longer_than(length: float) → Chunk[source]

Only the rows of baselines at least length (m) long.

phase(dra: float, ddec: float) → ndarray[source]

exp(-2 pi i (u dra + v ddec)): moves the phase centre to (dra, ddec) arcsec in the sky frame.

scaled(factor) → Chunk[source]

X and w times factor (per visibility, or broadcast): weights rescaled, data as they were.

uv_waves() → tuple[ndarray, ndarray][source]

u and v in wavelengths, (n_chan, n_row).

property vis: ndarray

Visibilities in Jy, 0 where flagged.

class luv_finder.data.DataHandler(msfile: str | None = None, chunks: list[Chunk] | None = None, metadata: Metadata | None = None, fields=None, reweight: bool = True)[source]

Visibilities of a measurement set, one Chunk per (field, spectral window).

Parameters:
msfilestr, optional

Measurement set to read; fields selects a subset of its target fields, and reweight=False keeps its weights (a noiseless simulation; see iter_chunks()).

chunks, metadataoptional

Ready-made data, e.g. from from_npz() or a test.

channel_noise(jackknife: DataHandler | None = None) → dict[int, ChannelNoise][source]

channel_noise() per spectral window, pooled over its fields: {spw: ChannelNoise}; on jackknife if already made (default: jackknife(), a chunk at a time).

property freqs: ndarray

Channel frequencies in Hz, every chunk in order.

classmethod from_npz(path: str, fields=None) → DataHandler[source]

Load an NPZ written by to_npz() or luv-export; fields loads only those.

jackknife() → DataHandler[source]

Signal-free noise realisation of every chunk (see Chunk.jackknife()).

spectrum(dra: float = 0.0, ddec: float = 0.0, model=None) → ndarray[source]

Weighted mean visibility per channel, phase-shifted to (dra, ddec) arcsec.

With model (a source component) the mean is over its visibilities on the data’s uv coverage and weights instead.

to_npz(path: str) → None[source]

Save the chunks and metadata to an NPZ (CASA-free format).

luv_finder.data.FREQUENCY_FRAMES = ('REST', 'LSRK', 'LSRD', 'BARY', 'GEO', 'TOPO', 'GALACTO', 'LGROUP', 'CMB')

casacore’s MFrequency reference codes (MEAS_FREQ_REF of an MS’s SPECTRAL_WINDOW).

luv_finder.data.LAG_SIGNIFICANCE = 5.0

Longest channel lag channel_noise() measures, and the significance a lag’s correlation needs to count.

luv_finder.data.MAX_GAP = 450.0

Largest time between two integrations of a baseline that measure the weights’ noise together, s.

luv_finder.data.MAX_LAG = 8

Longest channel lag channel_noise() measures, and the significance a lag’s correlation needs to count.

class luv_finder.data.Metadata(dish_diameter: float, ref: tuple[float, float], central_frequency: float, max_uvwave: float)[source]

Quantities of the whole dataset, the same whichever fields or windows are loaded.

Attributes:
dish_diameterm
ref(RA, Dec) in rad that sky offsets are measured from (the first target field)
central_frequencyHz
max_uvwavelongest baseline in wavelengths
classmethod from_ms(msfile: str) → Metadata[source]

The measurement set’s metadata (read once per process: it scans every row’s UVW).

minresolution() → float[source]

Angular resolution in arcsec from the longest baseline.

luv_finder.data.READ_BLOCK = 25000000

Target size of one block of rows read from a measurement set, in visibility-channels.

class luv_finder.data.WeightScale(scans: np.ndarray, scale: np.ndarray, check: np.ndarray, error: np.ndarray)[source]

Noise power over the weights per scan and channel, scale, its robust check and that one’s error.

at(scan: ndarray) → ndarray[source]

scale of rows in scan, (n_chan, n_row).

check: ndarray

Alias for field number 2

deviation() → ndarray[source]

check / scale - 1 in units of its counting error: about N(0, 1) for Gaussian noise.

error: ndarray

Alias for field number 3

scale: ndarray

Alias for field number 1

scans: ndarray

Alias for field number 0

luv_finder.data.channel_noise(jackknife: Iterable[Chunk]) → ChannelNoise[source]

Of one window’s jackknife (Chunk.jackknife()): the noise power per channel over the weights, sum |X|^2 / 2 sum w, and the channels’ correlation (1, rho_1, ...) up to the first lag consistent with 0.

luv_finder.data.check_frame(msfile: str, codes) → None[source]

Warn unless every window is in LSRK: the search, the catalogue and the fit keep the MS’s frame.

luv_finder.data.fields_in(path) → list[int][source]

Target fields of a measurement set, an NPZ or a list of NPZs, sorted; an NPZ is not read beyond its index.

luv_finder.data.iter_chunks(msfile: str, ref, fields=None, spws=None, reweight: bool = True, scales: dict | None = None) → Iterator[Chunk][source]

Read target visibilities one spw at a time over all fields, weights corrected, offsets from ref.

Autocorrelations, flagged rows and rows with no unflagged channel are dropped. Every window’s weights are divided by the noise measured in it (weight_scale()), so the peak memory is one window over all its fields; reweight=False keeps the measurement set’s weights, for a noiseless simulation, which has no noise to measure. A scales dict receives each window’s WeightScale by spw.

luv_finder.data.load(path, fields=None) → DataHandler[source]

The visibilities of fields (default: all) from a measurement set, an NPZ or a list of one dataset’s NPZs.

luv_finder.data.pair_weight(a, b) → ndarray[source]

Inverse variance of (x_a +- x_b) / 2 from the weights of x_a and x_b; 0 if either is.

luv_finder.data.pointing(path, field: int) → tuple[Metadata, ndarray, float][source]

A field’s dataset metadata, phase centre (arcsec, the data’s sky frame) and lowest frequency (Hz), from a measurement set, an NPZ or a list of NPZs, without reading its visibilities.

luv_finder.data.reweighted(chunks: Sequence[Chunk], ws: WeightScale | None = None) → list[Chunk][source]

The chunks of one window, weights divided by ws (default weight_scale()) per scan and channel.

luv_finder.data.sky_direction(offset, ref) → ndarray[source]

(RA, Dec) in rad of the (east, north) offset in arcsec from ref, the inverse of sky_offset().

luv_finder.data.sky_offset(direction, ref) → ndarray[source]

(east, north) offset in arcsec of direction on the tangent plane at ref, both (RA, Dec) in rad.

luv_finder.data.stokes_i(data: ndarray, weight: ndarray, flag: ndarray)[source]

Average the parallel hands [0, -1] into (vis, w); weight per row or per channel, w 0 if flagged.

luv_finder.data.target_selection(msfile: str)[source]

Field and spectral-window ids observed with the target intent.

luv_finder.data.weight_scale(chunks: Sequence[Chunk]) → WeightScale[source]

sum w z^2 / 2 sum w per (scan, channel) over a window’s integrations of a baseline <= MAX_GAP apart.

The integrations pair in their own field, so a pair sees one sky; a (scan, channel) with no such pair (a mosaic field seen once per visit) pairs across fields. A pair that spans two scans counts in both, so a scan of one integration is measured too.

luv_finder.data.write_npz(path: str, metadata: Metadata, chunks: Iterable[Chunk]) → None[source]

Write an NPZ one array at a time, so chunks may be a generator that never sits in memory whole.