Preprocessing API#

Raw CCD frames to preprocessed images (IM files): bias/dark subtraction, variance initialisation, bad-pixel masking, and cosmic-ray removal. See the IM stage theory for background.

Make IM (Image) Module

This module handles the preprocessing of raw astronomical data to create IM files. It performs the following steps: 1. Copy raw file to IM file 2. Mark bad pixels 3. Subtract bias and process overscan 4. Subtract dark frame (optional) 5. Create and initialize variance HDU 6. Apply long-slit flat field (optional) 7. Remove cosmic rays (optional)

Based on the Fortran MAKE_IM routine from 2dfdr.

class kspecdr.preproc.make_im.MakeIM(verbose: bool = True)[source]#

Bases: object

Handles preprocessing of raw astronomical data to create IM files.

This class implements the equivalent of the Fortran MAKE_IM routine, performing all necessary preprocessing steps to convert raw instrument data into calibrated image files with proper variance estimates.

process_raw_to_im(raw_filename: str, im_filename: str | None = None, use_bias: bool = False, bias_filename: str | None = None, use_dark: bool = False, dark_filename: str | None = None, use_glow_pca: bool = False, glow_pca_filename: str | None = None, dc_rate_filename: str | None = None, glow_n_components: int = 5, glow_fit_rows: list | None = None, use_lflat: bool = False, lflat_filename: str | None = None, bad_pixel_mask: str | None = None, bad_pixel_mask2: str | None = None, mark_saturated: bool = True, cosmic_ray_method: str = 'NONE', **kwargs) → str[source]#

Process raw file to create IM file with all preprocessing steps.

Parameters:

  • raw_filename (str) – Path to the raw input file

  • im_filename (str, optional) – Path for the output IM file. If None, will be derived from raw_filename

  • use_bias (bool, optional) – Whether to subtract bias frame

  • use_dark (bool, optional) – Whether to subtract dark frame

  • dark_filename (str, optional) – Path to dark frame file

  • use_glow_pca (bool, optional) – Whether to subtract glow using PCA templates instead of (or in addition to) simple dark subtraction.

  • glow_pca_filename (str, optional) – Path to the PCA glow cube FITS file (glow_pca_cube.fits). Must contain a PRIMARY HDU with the mean glow pattern and extensions named PC01…PCxx with individual PCA component images.

  • dc_rate_filename (str, optional) – Path to a FITS file with the per-pixel dark-current rate [e-/s]. If None, the DC-current contribution is assumed to be zero and only the glow (mean + PCA) model is subtracted.

  • glow_n_components (int, optional) – Number of PCA components to use in the glow fit (default: 5).

  • glow_fit_rows (list of (int, int), optional) – Row ranges (axis=0) to include when fitting the glow model, e.g. [(0, 500), (800, 1300)]. Pixels outside these ranges are excluded from the least-squares fit but the model is still evaluated and subtracted everywhere. If None, all finite pixels are used.

  • use_lflat (bool, optional) – Whether to divide by long-slit flat field

  • lflat_filename (str, optional) – Path to long-slit flat field file

  • bad_pixel_mask (str, optional) – Path to bad pixel mask file

  • bad_pixel_mask2 (str, optional) – Path to second bad pixel mask file (e.g., cosmic ray mask)

  • mark_saturated (bool, optional) – Whether to mark saturated pixels as bad

  • cosmic_ray_method (str, optional) – Method for cosmic ray removal (‘NONE’, ‘LACOSMIC’, ‘BCLEAN’, ‘PYCOSMIC’)

  • **kwargs – Additional keyword arguments

Returns:

Path to the created IM file

Return type:

str

kspecdr.preproc.make_im.make_im(raw_filename: str, im_filename: str | None = None, use_bias: bool = False, use_dark: bool = False, dark_filename: str | None = None, use_glow_pca: bool = False, glow_pca_filename: str | None = None, dc_rate_filename: str | None = None, glow_n_components: int = 5, glow_fit_rows: list | None = None, use_lflat: bool = False, lflat_filename: str | None = None, bad_pixel_mask: str | None = None, bad_pixel_mask2: str | None = None, mark_saturated: bool = True, cosmic_ray_method: str = 'NONE', verbose: bool = True, **kwargs) → str[source]#

Convenience function to process raw file to IM file.

Parameters:

  • raw_filename (str) – Path to the raw input file

  • im_filename (str, optional) – Path for the output IM file. If None, will be derived from raw_filename

  • use_bias (bool, optional) – Whether to subtract bias frame

  • use_dark (bool, optional) – Whether to subtract dark frame (simple exposure-time-scaled subtraction)

  • dark_filename (str, optional) – Path to dark frame file

  • use_glow_pca (bool, optional) – Whether to subtract glow using PCA templates. Can be used instead of or in addition to use_dark.

  • glow_pca_filename (str, optional) – Path to the PCA glow cube FITS file (PRIMARY = mean glow, extensions PC01…PCxx = PCA component images).

  • dc_rate_filename (str, optional) – Path to a FITS file with the per-pixel dark-current rate [e-/s]. If None, no DC-current term is included in the glow model.

  • glow_n_components (int, optional) – Number of PCA components to use in the glow fit (default: 5).

  • glow_fit_rows (list of (int, int), optional) – Row (axis=0) ranges used for the glow fitting, e.g. [(0, 500), (800, 1300)]. Pixels outside these ranges are excluded from the fit but the model is subtracted everywhere. None → all pixels.

  • use_lflat (bool, optional) – Whether to divide by long-slit flat field

  • lflat_filename (str, optional) – Path to long-slit flat field file

  • bad_pixel_mask (str, optional) – Path to bad pixel mask file

  • bad_pixel_mask2 (str, optional) – Path to second bad pixel mask file

  • mark_saturated (bool, optional) – Whether to mark saturated pixels as bad

  • cosmic_ray_method (str, optional) – Method for cosmic ray removal

  • verbose (bool, optional) – Whether to print verbose output

  • **kwargs – Additional keyword arguments

Returns:

Path to the created IM file

Return type:

str

Preprocessing and Reduction Routines for KSPEC.

This module implements the high-level reduction functions similar to 2dfdr: - combine_image: Combine multiple images - reduce_bias: Process bias frames - reduce_dark: Process dark frames - reduce_lflat: Process long-slit flat frames - reduce_fflat: Process fiber flat frames - reduce_arc: Process arc frames

kspecdr.preproc.preproc.combine_image(input_files: List[str], output_file: str, method: str = 'MEDIAN', adjust_levels: bool = True, level_method: str = 'SCALE', sigma: float = 5.0, **kwargs) → str[source]#

Combine multiple images into a single image.

Parameters:

  • input_files (List[str]) – List of input IM filenames

  • output_file (str) – Output combined filename

  • method (str) – Combination method (‘MEDIAN’, ‘MEAN’, ‘SIGMA_CLIP’). Default is ‘MEDIAN’.

  • adjust_levels (bool) – Whether to adjust levels (normalize) before combining. Default is True.

  • level_method (str) – Method for level adjustment when adjust_levels is True. ‘SCALE’ (default): multiplicative scaling using median of central region. ‘OFFSET’: additive offset using median of central region.

  • sigma (float) – Sigma value for sigma clipping (used only if method=’SIGMA_CLIP’). Default is 5.0.

  • **kwargs – Additional arguments

Returns:

Path to the created combined file

Return type:

str

kspecdr.preproc.preproc.reduce_arc(raw_files: List[str], output_file: str = 'ARCcombined.fits', bias_filename: str | None = None, dark_filename: str | None = None, lflat_filename: str | None = None, **kwargs) → str[source]#

Reduce arc frames: Make IM files, combine, and prepare for extraction.

Parameters:

  • raw_files (List[str]) – List of raw arc filenames

  • output_file (str) – Output combined arc filename

  • bias_filename (str, optional) – Master bias file

  • dark_filename (str, optional) – Master dark file

  • lflat_filename (str, optional) – Master lflat file

Returns:

Path to the combined arc file

Return type:

str

kspecdr.preproc.preproc.reduce_bias(raw_files: List[str], output_file: str = 'BIAScombined.fits', bias_type: str = 'MASTER', method: str | None = 'SIGMA_CLIP', sigma: float | None = 4.0, adjust_levels: bool = True, level_method: str = 'OFFSET', **kwargs) → str[source]#

Reduce bias frames: Make IM files and combine them.

Parameters:

  • raw_files (List[str]) – List of raw bias filenames

  • output_file (str) – Output master bias filename

  • bias_type (str) – Type of bias frame (‘MASTER’, ‘OVERSCAN’, ‘OTHER’)

  • method (str, optional) – Combination method (‘MEDIAN’, ‘MEAN’, ‘SIGMA_CLIP’). Default is ‘SIGMA_CLIP’.

  • sigma (float, optional) – Sigma value for sigma clipping (used only if method=’SIGMA_CLIP’). Default is 4.0.

  • adjust_levels (bool, optional) – Whether to adjust levels (normalize) before combining. Default is True.

  • level_method (str, optional) – Method for level adjustment when adjust_levels is True. ‘SCALE’ (default): multiplicative scaling using median of central region. ‘OFFSET’: additive offset using median of central region.

Returns:

Path to the master bias file

Return type:

str

kspecdr.preproc.preproc.reduce_dark(raw_files: List[str], output_file: str = 'DARKcombined.fits', bias_filename: str | None = None, method: str | None = 'MEDIAN', sigma: float | None = None, **kwargs) → str | List[str][source]#

Reduce dark frames: Make IM files (subtract bias) and combine them.

Parameters:

  • raw_files (List[str]) – List of raw dark filenames

  • output_file (str) – Output master dark filename

  • bias_filename (str, optional) – Master bias file to subtract

  • method (str, optional) – Combination method (‘MEDIAN’, ‘MEAN’, ‘SIGMA_CLIP’). Default is ‘MEDIAN’.

  • sigma (float, optional) – Sigma value for sigma clipping (used only if method=’SIGMA_CLIP’). Default is None.

Returns:

Path to the master dark file(s). If multiple exposure times are present, returns a list of filenames.

Return type:

str or List[str]

kspecdr.preproc.preproc.reduce_fflat(raw_files: List[str], output_file: str = 'FFLATcombined.fits', bias_filename: str | None = None, dark_filename: str | None = None, lflat_filename: str | None = None, **kwargs) → str[source]#

Reduce fiber flat frames: Make IM files, combine, and prepare for extraction.

Note: Full reduction of fiber flats typically involves extraction (Tramline Map) and normalization, which may be handled by subsequent steps or modules. This function currently performs the image-level preparation and combination.

Parameters:

  • raw_files (List[str]) – List of raw fflat filenames

  • output_file (str) – Output combined fflat filename

  • bias_filename (str, optional) – Master bias file

  • dark_filename (str, optional) – Master dark file

  • lflat_filename (str, optional) – Master lflat file

Returns:

Path to the combined fflat file

Return type:

str

kspecdr.preproc.preproc.reduce_lflat(raw_files: List[str], output_file: str = 'LFLATcombined.fits', bias_filename: str | None = None, dark_filename: str | None = None, **kwargs) → str[source]#

Reduce long-slit flat frames: Make IM files (sub bias/dark) and combine.

Parameters:

  • raw_files (List[str]) – List of raw lflat filenames

  • output_file (str) – Output master lflat filename

  • bias_filename (str, optional) – Master bias file

  • dark_filename (str, optional) – Master dark file

Returns:

Path to the master lflat file

Return type:

str