Welcome to aimfast’s documentation!

An Astronomical Image Fidelity Assessment Tool

Contents:

aimfast

An Astronomical Image Fidelity Assessment Tool

Introduction

Image fidelity is a measure of the accuracy of the reconstructed sky brightness distribution. A related metric, dynamic range, is a measure of the degree to which imaging artifacts around strong sources are suppressed, which in turn implies a higher fidelity of the on-source reconstruction. Moreover, the choice of image reconstruction algorithm and source finder also affects the estimate on-source brightness distribution.

Fidelity Matrices

Image dynamic range

Dynamic range is a measure of the degree to which imaging artifacts around strong sources are suppressed, which in turn implies a higher fidelity of the on-source reconstruction. Here we determine it in three ways: Obtaining the quotient of - highest peak flux (\(flux_{peak}\)) and the absolute of the minimum flux (\(flux_{min}\)) around the peak in the residual image. - highest peak flux (\(flux_{peak}\)) and the rms flux (\(flux_{local_rms}\)) around the peak in the residual image. - highest peak flux (\(flux_{peak}\)) and the rms flux (\(flux_{grobal_rms}\)) in the residual image.

(1)\[DR = \frac{flux_{peak}}{\left | {flux_{min}} \right | } (1) DR = \frac{flux_{peak}}{\left | {flux_{local_rms}} \right | } (2) DR = \frac{flux_{peak}}{\left | {flux_{global_rms}} \right | } (3)\]

Statistical moments of distribution

The mean and the variance provide information on the location (general value of the residual flux) and variability (spread, dispersion) of a set of numbers, and by doing so, provide some information on the appearance of the distribution of residual flux in the residual image. The mean and variance are calculated as follows respectively

(2)\[MEAN = \frac{1}{n}\sum_{i=1}^{n}(x_{i}) (4)\]

and

(3)\[VARIANCE = \frac{1}{n}\sum_{i=1}^{n}(x_{i} - \overline{x})^2 (5)\]

whereby

(4)\[STD\_DEV = \sqrt{VARIANCE} (6)\]

The third and fourth moments are the skewness and kurtosis respectively. The skewness is the measure of the symmetry of the shape and kurtosis is a measure of the flatness or peakness of a distribution. This moments are used to characterize the residual flux after performing calibration and imaging, therefore for ungrouped data, the r-th moment is calculated as follows:

(5)\[m_r = \frac{1}{n}\sum_{i=1}^{n}(x_i - \overline{x})^r (7)\]

The coefficient of skewness, the 3-rd moment, is obtained by

(6)\[SKEWNESS = \frac{m_3}{{m_2}^{\frac{3}{2}}} (8)\]

If there is a long tail in the positive direction, skewness will be positive, while if there is a long tail in the negative direction, skewness will be negative.

Figure 1. Skewness of a distribution.

The coefficient kurtosis, the 4-th moment, is obtained by

(7)\[KURTOSIS = \frac{m_4}{{m_2}^{2}} (9)\]

Smaller values (in magnitude) indicate a flatter, more uniform distribution.

Figure 2. Kurtosis of a distribution.

Furthermore, there is median absolute deviation which is a measure of how distributed is the residual data with regards to the median. This can be compared with the standard deviation to verify that the residuals are noise-like (and Gaussian).

Installation

Installation from source, working directory where source is checked out

$ pip install .

This package is available on PYPI, allowing

$ pip install aimfast

Command line usage

Get the statistics of the residual image

$ aimfast --residual-image cube.residual.fits

Get the residual image stats and dynamic range in one step where argument -af is a factor to multiply the beam area to get target peak area:

$ aimfast --residual-image cube.residual.fits --restored-image cube.image.fits -af 5

or using sky model file (e.g. tigger lsm.html):

$ aimfast --residual-image cube.residual.fits --tigger-model model.lsm.html -af 5

NB: Outputs will be printed on the terminal and dumped into fidelity_results.json file. Moreover if the source file names are distinct the output results will be appended to the same json file.

$ cat fidelity_results.json
$ {"cube.residual.fits": {"SKEW": 0.124, "KURT": 3.825, "STDDev": 5.5e-05,
                          "MEAN": 4.747e-07, "MAD": 5e-05},
       "cube.image.fits": {"DR": 35.39, "deepest_negative": 10.48,
                           "local_rms": 30.09, "global_rms": 35.39}}

Additionally, normality testing of the residual image can be performed using the D’Agostino (normaltest) and Shapiro-Wilk (shapiro) analysis, which returns a tuple result, e.g {‘NORM’: (123.3, 0.1)}, with the z-score and p-value respectively.

$ aimfast --residual-image cube.residual.fits --normality-model normaltest

Furthermore, a comparison of residual images can be performed as follows: To get random residual flux measurements in residual1.fits and residual2.fits images

$ aimfast --compare-residuals residual1.fits residual2.fits --area-factor 2 -dp 100

where –area-factor is the number to multiply the beam size to get area and -dp is the number of data points to sample. In case the beam information is missing from the image header use –psf-image | -psf, the point spread function file or psf size in arcsec, otherwise a default of 5 arcsec will be used. To get on source residual flux measurements in a residual1.fits and residual2.fits images

$ aimfast --compare-residuals residual1.fits residual2.fits --tigger-model model.lsm.html

where –tigger-model is the name of the model or catalog file to locate exact source residuals. For random or on source residual noise comparisons, the plot on the left shows the residuals on image 1 and image 2 overlayed and the plot on the right shows the ratios. The colorbar shows the distance of the sources from the phase centre.

Figure 3. The random/source residual-to-residual/noise ratio measurements

Moreover aimfast allows you to swiftly compare two (input-output) model catalogs. Currently source flux density and astrometry are examined. It returns an interactive html correlation plots, from which a .png file can be easily downloaded.

$ aimfast --compare-models model1.lsm.html model2.lsm.html -tol 5

where -tol is the tolerance to cross-match sources in arcsec. Moreover -as flag can be used to compare all source irrespective of shape (otherwise only point-like source with maj<2” are used). Access to (sumss, nvss,) online catalogs is also provided, to allow comparison of local catalogs to remote catalogs.

$ aimfast --compare-online model1.lsm.html --online-catalog nvss -tol 5

In the case where fits images are compared, aimfast can pre-install source finder of choice (pybdsf, aegean,) to generate a catalogs which are in turn compared:

$ aimfast --compare-images image1.fits image1.fits --source-finder pybdsf -tol 5

After the first run attempt one of the outputs is source_finder.yml file, which provide all the possible parameters of the source finders. Otherwise this file can be generated and edited prior to the comparison:

$ aimfast -gd my-source-finder.yml
$ aimfast --compare-images image1.fits image2.fits --config my-source-finder.yml -sf pybdsf -tol 5

For Flux density, the more the data points rest on the y=x (or I_out=I_in), the more correlated the two models are.

Figure 4. Input-Output Flux model comparison

For astrometry, the more sources lie on the y=0 (Delta-position axis) in the left plot and the more points with 1 sigma (blue circle) the more accurate the output source positions.

Figure 5. Input-Output Astrometry model comparison

Lastly, if you want to run any of the available source finders, generate the config file and edit then run:

$ aimfast -gd my-source-finder.yml
$ aimfast source-finder -c my-source-finder.yml -sf pybdsf

aimfast

aimfast modules

amifast.aimfast module

aimfast.aimfast.compare_models(models, tolerance=0.2, plot=True, all_sources=False, closest_only=False, prefix=None, flux_plot='log')

Plot model1 source properties against that of model2

models : dict

Tigger formatted model files e.g {model1: model2}.

tolerance : float

Tolerace in detecting source from model 2 (in arcsec).

plot : bool

Output html plot from which a png can be obtained.

all_source: bool

Compare all sources in the catalog (else only point-like source)

closest_only: bool

Returns the closest source only as the matching source

flux_plot: str

The type of output flux comparison plot (options:log,snr,inout)

prefix : str

Prefix for output htmls

results : dict

Dictionary of source properties from each model.

aimfast.aimfast.compare_residuals(residuals, skymodel=None, points=None, inline=False, area_factor=None, prefix=None, fov_factor=None)

aimfast.aimfast.create_logger()

Create a console logger

aimfast.aimfast.fitsInfo(fitsname=None)

Get fits header info.

fitsname : fits file

Restored image (cube)

fitsinfo : dict

Dictionary of fits information e.g. {‘wcs’: wcs, ‘ra’: ra, ‘dec’: dec, ‘dra’: dra, ‘ddec’: ddec, ‘raPix’: raPix, ‘decPix’: decPix, ‘b_size’: beam_size, ‘numPix’: numPix, ‘centre’: centre, ‘skyArea’: skyArea}

aimfast.aimfast.generate_default_config(configfile)

Generate default config file for running source finders

aimfast.aimfast.get_aimfast_data(filename='fidelity_results.json', dir='.')

Extracts data from the json data file

aimfast.aimfast.get_argparser()

Get argument parser.

aimfast.aimfast.get_box(wcs, radec, w)

Get box of width w around source coordinates radec.

radec : tuple

RA and DEC in degrees.

w : int

Width of box.

wcs : astLib.astWCS.WCS instance

World Coordinate System.

box : tuple

A box centred at radec.

aimfast.aimfast.get_detected_sources_properties(model_1, model_2, area_factor, all_sources=False, closest_only=False)

Extracts the output simulation sources properties.

models_1 : file

Tigger formatted or txt model 1 file.

models_2 : file

Tigger formatted or txt model 2 file.

area_factor : float

Area factor to multiply the psf size around source.

all_source: bool

Compare all sources in the catalog (else only point-like source)

closest_only: bool

Returns the closest source only as the matching source

(targets_flux, targets_scale, targets_position) : tuple

Tuple of target flux, morphology and astrometry information

aimfast.aimfast.get_model(catalog)

Get model model object from file catalog

aimfast.aimfast.get_sf_params(configfile)

aimfast.aimfast.get_source_overlay(sources1, sources2)

Get source from models compare for overlay

aimfast.aimfast.get_src_scale(source_shape)

Get scale measure of the source in arcsec.

source_shape : lsm object

Source shape object from model

(scale_out_arc_sec, scale_out_err_arc_sec) : tuple

Output source scale with error value

aimfast.aimfast.image_dynamic_range(fitsname, residual, area_factor=6)

Gets the dynamic range in a restored image.

fitsname : fits file

Restored image (cube).

residual : fits file

Residual image (cube).

area_factor: int

Factor to multiply the beam area.

DR : dict

DRs - dynamic range values.

aimfast.aimfast.json_dump(data_dict, filename='fidelity_results.json')

Dumps the computed dictionary results into a json file.

data_dict : dict

Dictionary with output results to save.

filename : str

Name of file json file where fidelity results will be dumped. Default is ‘fidelity_results.json’ in the current directory.

If the fidelity_results.json file exists, it will be append, and only repeated image assessments will be replaced.

aimfast.aimfast.main()

Main function.

aimfast.aimfast.measure_psf(psffile, arcsec_size=20)

Measure point spread function after deconvolution.

psfile : fits file

Point spread function file.

arcsec_size : float

Cross section size

r0 : float

Average psf size.

aimfast.aimfast.model_dynamic_range(lsmname, fitsname, beam_size=5, area_factor=2)

Gets the dynamic range using model lsm and residual fits.

fitsname : fits file

Residual image (cube).

lsmname : lsm.html or .txt file

Model .lsm.html from pybdsm (or .txt converted tigger file).

beam_size : float

Average beam size in arcsec.

area_factor : float

Factor to multiply the beam area.

DR : dict

DRs - dynamic range values.

aimfast.aimfast.noise_sigma(noise_image)

Determines the noise sigma level in a dirty image with no source

noise_image : file

Noise image (cube).

noise_std : float

Noise image standard deviation

aimfast.aimfast.normality_testing(data, test_normality='normaltest', data_range=None)

Performs a normality test on the image data.

data : numpy.array

Residual residual array. i.e. fitsio.open(fitsname)[0].data

test_normality : str

Perform normality testing using either shapiro or normaltest.

data_range : int

Range of data to perform normality testing.

normality : dict

dictionary of stats props. e.g. {‘NORM’: (123.3, 0.012)} whereby the first element is the statistics (or average if data_range specified) of the datasets and second element is the p-value.

aimfast.aimfast.plot_aimfast_stats(fidelity_results_file, units='micro', prefix='')

Plot stats results if more that one residual images where assessed

aimfast.aimfast.plot_astrometry(models, label=None, tolerance=0.2, phase_centre=None, all_sources=False)

Plot model-model positions from lsm.html/txt models

models : dict

Tigger/text formatted model files e.g {model1: model2}.

label : str

Use this label instead of the FITS image path when saving data.

tolerance: float

Radius around the source to be cross matched.

phase_centre : str

Phase centre of catalog (if not already embeded)

all_source: bool

Compare all sources in the catalog (else only point-like source)

aimfast.aimfast.plot_photometry(models, label=None, tolerance=0.2, phase_centre=None, all_sources=False, flux_plot='log')

Plot model-model fluxes from lsm.html/txt models

models : dict

Tigger/text formatted model files e.g {model1: model2}.

label : str

Use this label instead of the FITS image path when saving data.

tolerance: float

Radius around the source to be cross matched (in arcsec).

phase_centre : str

Phase centre of catalog (if not already embeded)

all_source: bool

Compare all sources in the catalog (else only point-like source)

aimfast.aimfast.plot_residuals_noise(res_noise_images, skymodel=None, label=None, area_factor=2.0, points=100)

Plot residual-residual or noise data

res_noise_images: dict

Dictionary of residual images to plot {res1.fits: res2.fits}.

skymodel: file

Skymodel file to locate on source residuals (lsm.html/txt)

label : str

Use this label instead of the FITS image path when saving data.

area_factor : float

Factor to multiply the beam area.

points: int

Number of data point to generate in case of random residuals.

aimfast.aimfast.residual_image_stats(fitsname, test_normality=None, data_range=None, threshold=None, chans=None, mask=None)

Gets statistcal properties of a residual image.

fitsname : file

Residual image (cube).

test_normality : str

Perform normality testing using either shapiro or normaltest.

data_range : int, optional

Range of data to perform normality testing.

threshold : float, optional

Cut-off threshold to select channels in a cube

chans : str, optional

Channels to compute stats (e.g. 1;0~50;100~200)

mask : file

Fits mask to get stats in image

props : dict

Dictionary of stats properties. e.g. {‘MEAN’: 0.0, ‘STDDev’: 0.1, ‘RMS’: 0.1,

‘SKEW’: 0.2, ‘KURT’: 0.3, ‘MAD’: 0.4, ‘MAX’: 0.7}

If normality_test=True, dictionary of stats props becomes e.g. {‘MEAN’: 0.0, ‘STDDev’: 0.1, ‘SKEW’: 0.2, ‘KURT’: 0.3, ‘MAD’: 0.4, ‘RMS’: 0.5, ‘SLIDING_STDDev’: 0.6, ‘MAX’: 0.7, ‘NORM’: (123.3,0.012)} whereby the first element is the statistics (or average if data_range specified) of the datasets and second element is the p-value.

aimfast.aimfast.source_finding(sf_params, sf=None)

aimfast.aimfast.targets_not_matching(sources1, sources2, matched_names, flux_units='milli')

Plot model-model fluxes from lsm.html/txt models

sources1: list

List of sources from model 1

sources2: list

List of sources Sources from model 2

matched_names: dict

Dict of names from model 2 that matched that of model 1

flux_units: str

Units of flux density for tabulated values

target_no_match1: dict

Sources from model 1 that have no match in model 2

target_no_match2: dict

Sources from model 2 that have no match in model 1

amifast.tests.test_aimfast module

License

This project is licensed under the GNU General Public License v3.0 - see license for details.

Contribute

Contributions are always welcome! Please ensure that you adhere to our coding standards pep8.

Contact us

Athanaseus Ramaila (aramaila@ska.ac.za)

Indices and tables

• Index
• Module Index
• Search Page