eat.hops package

Submodules

eat.hops.util module

HOPS utilities

class eat.hops.util.ControlFile(cf=[])

Bases: object

actions()

evaluate(cond, station=None, baseline=None, source=None, scan=None, dropmissing=False)

filter(station=None, baseline=None, source=None, scan=None, dropmissing=False)

open(cf)

class eat.hops.util.PDF(pdfdata)

Bases: object

eat.hops.util.adhoc(b, pol=None, window_length=None, polyorder=None, snr=None, baseline=None, ref=None, prefix='', timeoffset=0.0, roundrobin=True, bowlfix=False, secondorder=True, p=None, tcoh=None, alpha=1.6666666666666667, ap=None, timetag=None)

create self-corrective ad-hoc phases from fringe file (type 212) assume a-priori phase bandpass and fringe rotation (delay) has been applied use round-robin training/evaluation to avoid self-tuning bias some SNR-based selection of averaging timescale is done to tune Sav-Gol filter compensate for delay-rate rotator bias for frequencies away from reference frequency allow for generation of corrective phases outside of data range (using chan_ids) check for -1 bad flag data in type212 and interpolate over (not done)

Parameters:

• b – numpy array of shape (nap, nchan), of fringe filename, or mk4fringe object from which visibs will be read if fringe filename or object is sent, auxiliary information will be provided for control file
• pol – will filter for pol if multiple files or pattern given
• window_length – odd integer length of scipy.signal.savgol_filter applied to data for smoothing (default=7)
• polyorder – order of piecewise smoothing polynomial (default=3)
• snr – manually set SNR to auto determine window_length and polyorder, else take from fringe file
• ap – accumulation period ap if no params
• baseline – baseline string, default is pulled from fringe file (if available) else AB
• timetag – need to specify if no params available
• ref – reference station (default is same as data) for adhoc phases, corrective phases apply to REM station
• prefix – add prefix to adhoc_filenames (e.g. source directory) as described in control file string
• timeoffset – add timeoffset [units of AP] to each timestamp in the adhoc string
• roundrobin – True to use frequency-slicing round-robing training to avoid self-tuning
• bowlfix – fix bowl effect in 2017 data (must send fringe file for parameters)
• secondorder – fix the second-order effect from delay change over time
• p – custom params, if chan_ids is available adhoc corrections are generated for all chan_ids
• tcoh – coherence timescale in seconds, if no AP is available it is assumed to be 1.0s
• alpha – structure function index

Returns:

visibility vector from which adhoc phase is estimated (not normalized) vcorr: corrected visibility (adhoc phase removed) phase: estimated baseline phase in radians code: params code (params and control file statements returned if sent mk4 object) days: params days vector (note this does not account for fourfit adhoc indexing bug) scan_name: params scan_name timetag: params timetag filename: filename of adhoc phase file cfcode: fourfit control file lines string: adhoc file contents as string

Return type:

v

eat.hops.util.align(bs, snrs=None, tint=5.0)

eat.hops.util.chaninfo(b=None)

eat.hops.util.closefringe(a)

eat.hops.util.compare_alist_v6(alist1, baseline1, polarization1, alist2=None, baseline2=None, polarization2=None)

Compare two alist data in pandas.DataFrame.

This function pick up rows at common datetimes in two input alist data and at specified sets of the baseline(s), polarization(s), and concatenate two alist data into one pandas.DataFrame. The keys for the first and second alist data will be “xxx1” (e.g. “snr1”, “amp1”) and “xxx2” (e.g. “snr2”, “amp2”), respectively.

Parameters:

• alist1 (pandas.DataFrame) – the first alist data read by eat.io.hope.read_alist_v6
• baseline1 (string) – a baseline of the first alist data to be compared
• polarization1 (string) – a polarization of the first alist data to be compared
• alist2 (optional, pandas.DataFrame) – the second alist data in pandas.DataFrame read by eat.io.hope.read_alist_v6 (default: same to alist1)
• baseline2 (optional, string) – a baseline of the second alist data to be compared (default: same to baseline1)
• polarization2 (optional, string) – a polarization of the second alist data to be compared (default: same to polarization1)

Returns:

A pandas.DataFrame object includes two alist data when the datetimes of two data sets at the specified baselines and polarizations are overlapped.

eat.hops.util.dcloseplot(p, baselines=None, vlines=[])

eat.hops.util.delayplot(df, site, offs={}, vlines=[])

eat.hops.util.delayscan(fringefile, res=4, dt=1, df=None, delayrange=(-10000.0, 10000.0), pol=None, fix_outliers=True, kind=120)

eat.hops.util.doff2cf(row, nchan=32, ref='L')

doff2cf: convert delay offsets table row to control file codes

Parameters:

• row – delay offsets code
• nchan – number of channels to define

Returns:

control file codes (str)

eat.hops.util.drclose(a, index=['ref_freq', 'expt_no', 'scan_id', 'scan_no', 'source', 'timetag', 'baseline'])

eat.hops.util.expmean(x, s=8, n=4)

eat.hops.util.findfringe(fringefile=None, kind=None, res=4, showx=6, showy=6, center=(None, None), dt=2, df=None, ni=1, ret=False, showhops=False, delay_off=0.0, rate_off=0.0, flip=False, segment=(None, None), channels=(None, None), pol=None, unrotate_212=True, replacedata=None, cf=None, aspect=None)

eat.hops.util.fixsqrt2(df)

eat.hops.util.fixstr(s)

eat.hops.util.flip(df, flipidx)

eat.hops.util.fplot(b=None, pol=None, filelist=True, noauto=False)

eat.hops.util.fringegroups(bls, baselines=False)

eat.hops.util.getfringefile(b=None, filelist=False, pol=None, quiet=False)

eat.hops.util.getpolarization(f)

eat.hops.util.hilo_segmented(a1, a2, restarts={}, boundary=19, index=['expt_no', 'scan_id', 'source', 'timetag'], values=['mbd_unwrap', 'mbd_err', 'snr', 'path'], idcol='ref_freq')

hilo_segmented: general hi-lo delay statistics given alist data

Parameters:

• a1 – dataframe from alist file with rates and delays, delay errors are added if missing
• b2 – dataframe from alist file with rates and delays, delay errors are added if missing
• restarts – special times in which to segment certain stations
• boundary – hour boundary for daily segments (default 19h)

eat.hops.util.hiloplot(p, baselines=slice(None, None, None), polarizations=slice(None, None, None))

eat.hops.util.mk4time(time)

eat.hops.util.nextpow2(x)

eat.hops.util.params(b=None, pol=None, quiet=None, cf=None)

extract many basic parameters from fringe file for easy access

The parameters attempt to follow metadata stored in fringe files. This function should be moved to a class with better structure and lazy evaluation.

Parameters:

• b – filename, filename pattern, or mk4fringe structure
• pol – polarization to look for if filename pattern is given (see getfringe)

Returns:

fringe file path from datadir ref_freq: reference frequency in MHz nchan: number of channels (processed) nap: number of accumulation periods (processed) nspec: number of spectral points per channel in type_230 (usually double the original number) nlags: number of lags from type_202 (also probably double the original FX data) code: ffit_chan_id’s for the channels in order (from type_205) a, b, c, … expt_no: HOPS 4-digit experiment number pol: baseline polarization product label e.g. LL sbd: single band residual delay average [us] mbd: multi band residual delay fitted value [us] delay: unwrapped mbd [us] rate: residual delay rate [us/s] amplitude: correlation coefficient estimate [10^-4] snr: signal-to-noise of scan-average visibility amplitude T: processed time [s] ap: accumulation period spacing [s] days: days since Jan 1 00:00:00 UT of each processed AP center time dtvec: a time vector with zero at frt (time reference for scan) dfvec: a frequency vector by visib type with zero at ref_freq trot: time rotators to rotate out fringe solution fedge: frequency edge of each channel flip: 1 for USB (vs fedge) and -1 for LSB for each channel bw: bandwidth of all channels as derived as best as possible from sample frequency info foffset: offset of middle of channel from channel edge baseline: baseline code source: source name start: start datetime of processed segment stop: stop datetime of processed segment (marks end of last segment) startidx: start idx into type120 of processed APs stopidx: stop idx into type120 of processed APs (not inclusive) apfilter: boolean indexer for type120 of processed time frt: fourfit reference time frtoff: frt offset from scan start time (scantime) utc_centeral: from fringe file, probaby marks reference time for delay rate compensation scan_name: name of scan (various conventions) scantime: scantime, generally start time of scan timetag: scantime in timetag format scantag: scantime + VEX start offset in timetag format (does not include fourfit start offset)

Return type:

name

eat.hops.util.pickref(df, nosma=True, nopdb=True, threshold=6.0, tcoh=6.0, full=False)

eat.hops.util.plotfringe(ns, showx=6.0, showy=6.0, center=(None, None), showhops=False, kind=230)

eat.hops.util.pop120(b=None, pol=None, fill=0)

eat.hops.util.pop210(b=None, pol=None)

eat.hops.util.pop212(b=None, pol=None, weights=False)

eat.hops.util.pop230(b=None, pol=None)

eat.hops.util.rcloseplot(p, baselines=slice(None, None, None), vlines=[])

eat.hops.util.rl_segmented(a, site, restarts={}, boundary=19, index=['ref_freq', 'expt_no', 'scan_id', 'scan_no', 'source', 'timetag', 'baseline'])

eat.hops.util.rlplot(p, corrected=True, wrap=True, vlines=[])

eat.hops.util.rrll_segmented(a, restarts={}, boundary=19, index=['ref_freq', 'expt_no', 'scan_id', 'scan_no', 'source', 'timetag'], aggfunc='first')

rrll_segmented: general RR-LL delay statistics given alist data

Parameters:

• a – dataframe from alist file with rates and delays, delay errors are added if missing
• restarts – special times in which to segment certain stations
• boundary – hour boundary for daily segments (default 19h)

eat.hops.util.rrllplot(p, baselines=slice(None, None, None), wrap=False, vlines=[])

eat.hops.util.sbdmbd2cf(row)

sbdmbd2cf: convert MBD-SBD table row to control file codes for ALMA

Parameters:row – delay offsets code Returns:control file codes (str)

eat.hops.util.set_datadir(datadir, expt_no='expt_no', scan_id='scan_id')

eat.hops.util.setparity(df)

eat.hops.util.short2int(short)

eat.hops.util.spectrum(bs, ncol=4, delay=None, rate=None, df=1, dt=1, figsize=None, snrthr=0.0, timeseries=False, centerphase=False, snrweight=True, kind=120, pol=None, grid=True, do_adhoc=True, precorrect=False, cf=None, channels=(None, None), ret=False, figwidth=8.0, pad=0)

eat.hops.util.stackfringe(b1, b2, d1=0.0, d2=0.0, r1=0.0, r2=0.0, p1=0.0, p2=0.0, coherent=True, **kwargs)

eat.hops.util.timeseries(bs, dt=1, pol=None, kind=212, cf=None, delay=None, rate=None, ret=False)

eat.hops.util.trendplot(df, site='', offs={}, col='sbdelay', vlines=None, bltrans=<function <lambda>>, rem=True, tag=True, **kwargs)

eat.hops.util.uvplot(df, source=None, color=None, threshold=6.5, bltrans=<function <lambda>>, flip=True, col=None, ulthreshold=None, cmap=<sphinx.ext.autodoc.importer._MockObject object>, vmin=None, vmax=None, log=True, tag=None, markerleg=True, allscans=None, year=None)

eat.hops.util.vecphase(ff, doadhoc=True, dt=30, df=4, pol=None, almaref=True, replacedata=None, bowlfix=True)

eat.hops.util.vecplot(vs, dtvec, dfvec, delay, rate, ref_freq, dt=1, df=1)

eat.hops.util.wavg(x, sigma=1.0, col=None, w=None, robust=10.0)

wavg: weighted average with error propagation

Parameters:

• x – data values
• sigma – std error on values, scalar or vector
• col – return Dict labels, for automatic naming of pandas columns under conversion col=(mean, [err], [chisq_r], [n_outliers])
• w – optional weights for averaging (otherwise 1/sigma**2)
• robust – if nonzero, remove [robust]-sigma outliers from median value before averaging

Returns:

OrderedDict of col=(mean, [err], [chisq_r], [n_outliers])

Module contents

EHT HOPS related tools and utilities