rail.core.utilStages module

Stages that implement utility functions

class rail.core.utilStages.ColumnMapper(args, comm=None)

Bases: RailStage

Utility stage that remaps the names of columns.

1. This operates on pandas dataframs in parquet files.

2. In short, this does: output_data = input_data.rename(columns=self.config.columns, inplace=self.config.inplace)

Configuration Parameters: output_mode [str]: What to do with the outputs (default=default) chunk_size [int]: (default=100000) columns [dict]: (required) inplace [bool]: (default=False)

config_options = {'chunk_size': 100000, 'columns': <class 'dict'>, 'inplace': False, 'output_mode': <ceci.config.StageParameter object>}

inputs = [('input', <class 'rail.core.data.PqHandle'>)]

name = 'ColumnMapper'

outputs = [('output', <class 'rail.core.data.PqHandle'>)]

run()

Run the stage and return the execution status

class rail.core.utilStages.Dereddener(args, comm=None)

Bases: RailStage

Utility stage that does dereddening

Configuration Parameters: output_mode [str]: What to do with the outputs (default=default) bands [str]: (default=ugrizy) mag_name [str]: (default=mag_{band}_lsst) band_a_env [list]: (default=[4.81, 3.64, 2.7, 2.06, 1.58, 1.31]) dustmap_name [str]: (default=sfd) dustmap_dir [str]: (required) copy_cols [list]: (default=[])

config_options = {'band_a_env': [4.81, 3.64, 2.7, 2.06, 1.58, 1.31], 'bands': 'ugrizy', 'copy_cols': [], 'dustmap_dir': <class 'str'>, 'dustmap_name': 'sfd', 'mag_name': 'mag_{band}_lsst', 'output_mode': <ceci.config.StageParameter object>}

fetch_map()

inputs = [('input', <class 'rail.core.data.Hdf5Handle'>)]

name = 'Dereddener'

outputs = [('output', <class 'rail.core.data.Hdf5Handle'>)]

run()

Run the stage and return the execution status

class rail.core.utilStages.LSSTFluxToMagConverter(args, comm=None)

Bases: RailStage

Utility stage that converts from fluxes to magnitudes

Note, this is hardwired to take parquet files as input and provide hdf5 files as output

Configuration Parameters: output_mode [str]: What to do with the outputs (default=default) bands [str]: (default=ugrizy) flux_name [str]: (default={band}_gaap1p0Flux) flux_err_name [str]: (default={band}_gaap1p0FluxErr) mag_name [str]: (default=mag_{band}_lsst) mag_err_name [str]: (default=mag_err_{band}_lsst) copy_cols [dict]: (default={}) mag_offset [float]: (default=31.4)

config_options = {'bands': 'ugrizy', 'copy_cols': {}, 'flux_err_name': '{band}_gaap1p0FluxErr', 'flux_name': '{band}_gaap1p0Flux', 'mag_err_name': 'mag_err_{band}_lsst', 'mag_name': 'mag_{band}_lsst', 'mag_offset': 31.4, 'output_mode': <ceci.config.StageParameter object>}

inputs = [('input', <class 'rail.core.data.PqHandle'>)]

mag_conv = 0.9210340371976182

name = 'LSSTFluxToMagConverter'

outputs = [('output', <class 'rail.core.data.Hdf5Handle'>)]

run()

Run the stage and return the execution status

class rail.core.utilStages.RowSelector(args, comm=None)

Bases: RailStage

Utility Stage that sub-selects rows from a table by index

1. This operates on pandas dataframs in parquet files.

2. In short, this does: output_data = input_data[self.config.start:self.config.stop]

Configuration Parameters: output_mode [str]: What to do with the outputs (default=default) start [int]: (required) stop [int]: (required)

config_options = {'output_mode': <ceci.config.StageParameter object>, 'start': <class 'int'>, 'stop': <class 'int'>}

inputs = [('input', <class 'rail.core.data.PqHandle'>)]

name = 'RowSelector'

outputs = [('output', <class 'rail.core.data.PqHandle'>)]

run()

Run the stage and return the execution status

class rail.core.utilStages.TableConverter(args, comm=None)

Bases: RailStage

Utility stage that converts tables from one format to anothe

FIXME, this is hardwired to convert parquet tables to Hdf5Tables. It would be nice to have more options here.

Configuration Parameters: output_mode [str]: What to do with the outputs (default=default) output_format [str]: (required)

config_options = {'output_format': <class 'str'>, 'output_mode': <ceci.config.StageParameter object>}

inputs = [('input', <class 'rail.core.data.PqHandle'>)]

name = 'TableConverter'

outputs = [('output', <class 'rail.core.data.Hdf5Handle'>)]

run()

Run the stage and return the execution status