Core Reader

This page documents the main MERTISDataPackReader workflow that used to live in nbs/00_core.py.

Import the Reader

from pathlib import Path

import mertisreader as mr

Locate the Repository Data

candidate_roots = [Path.cwd().resolve(), Path.cwd().resolve().parent, Path.cwd().resolve().parent.parent]
repo_root = next(root for root in candidate_roots if (root / "data").exists())

input_path = repo_root / "data/bcmer_tm_all_START-20200409T000000_END-20200410T000000_CRE-20240717T132010-ParamEventBootSciHK-short/cal"
output_path = Path("/tmp/")

print(f"Repo root: {repo_root}")
print(f"Input path exists: {input_path.exists()}")

Repo root: /home/kidpixo/work/esa/mertis_data_management/mertisreader
Input path exists: True

Build the Reader

ms_reader = mr.MERTISDataPackReader(input_dir=input_path, output_dir=output_path, log_level="INFO")

print(ms_reader)

2026-06-11 11:45:51,025|2181462|INFO|input_dir=PosixPath('/home/kidpixo/work/esa/mertis_data_management/mertisreader/data/bcmer_tm_all_START-20200409T000000_END-20200410T000000_CRE-20240717T132010-ParamEventBootSciHK-short/cal')

MERTISDataPackReader(
  input_dir=/home/kidpixo/work/esa/mertis_data_management/mertisreader/data/bcmer_tm_all_START-20200409T000000_END-20200410T000000_CRE-20240717T132010-ParamEventBootSciHK-short/cal
  output_dir=/tmp
  log_level=INFO
  file counts: {
    hk_default: 1
    hk_extended: 1
    sc_tis: 1
    sc_tir: 0
    sc_tis_ql: 0
    sc_tir_ql: 0
  }
)

Inspect the Input Files

ms_reader.show_files()
ms_reader.listfiletypes()

All files in input_dir :

Counter({'.dat': 2, '.lblx': 3, '.fits': 1})

All files in input_dir matching old pattern <v0.2.6 (\d{8}_\d{8}):

Counter()

All files in input_dir matching new pattern >=v0.2.6 (mer_cal_sc_tis_YYYYMMDD_1-...):

Counter({'mer_cal_sc_tis': 1})

{
    'hk_default': ['mer_cal_hk_default_20200409_1-0651130766-12538__0_1.dat'],
    'hk_extended': ['mer_cal_hk_extended_20200409_1-0651130766-12595__0_1.dat'],
    'sc_tis': ['mer_cal_sc_tis_20200409_1-0651130819-21186__0_1.fits'],
    'sc_tir': [],
    'sc_tis_ql': [],
    'sc_tir_ql': []
}

{'hk_default': 1, 'hk_extended': 1, 'sc_tis': 1, 'sc_tir': 0, 'sc_tis_ql': 0, 'sc_tir_ql': 0}

Assemble the Data

ms_reader.data_collector()
ms_reader.data_assembler(verbose=True)

print(f"Frames: {list(ms_reader.frames.keys())}")
print(f"Wavelengths: {list(ms_reader.wavelengths.keys())}")
print(f"Metadata tables: {list(ms_reader.mertis_tis_metadata.keys())}")

Reading filetype: hk_default from mer_cal_hk_default_20200409_1-0651130766-12538__0_1
Processing label: /home/kidpixo/work/esa/mertis_data_management/mertisreader/data/bcmer_tm_all_START-20200409T000000_END-20200410T000000_CRE-20240717T132010-ParamEventBootSciHK-short/cal/mer_cal_hk_default_20200409_1-0651130766-12538__0_1.lblx
Now processing a Table_Delimited structure: MERTIS_CAL_DEFAULT_HOUSEKEEPING_DATA_TABLE
Reading filetype: hk_extended from mer_cal_hk_extended_20200409_1-0651130766-12595__0_1
Processing label: /home/kidpixo/work/esa/mertis_data_management/mertisreader/data/bcmer_tm_all_START-20200409T000000_END-20200410T000000_CRE-20240717T132010-ParamEventBootSciHK-short/cal/mer_cal_hk_extended_20200409_1-0651130766-12595__0_1.lblx
Now processing a Table_Delimited structure: MERTIS_CAL_EXTENDED_HOUSEKEEPING_DATA_TABLE
Reading filetype: tis from mer_cal_sc_tis_20200409_1-0651130819-21186__0_1

Reading filetype: tis from
/home/kidpixo/work/esa/mertis_data_management/mertisreader/data/bcmer_tm_all_START-20200409T000000_END-20200410T000
000_CRE-20240717T132010-ParamEventBootSciHK-short/cal/mer_cal_sc_tis_20200409_1-0651130819-21186__0_1.fits

n_wav=40 # generic wavelengths : not precise enough for scientific analysis!
|    | tis_stem                                        |   finite(geo) |   geo.size |
|---:|:------------------------------------------------|--------------:|-----------:|
|  0 | mer_cal_sc_tis_20200409_1-0651130819-21186__0_1 |           672 |      10500 |
Indices of measurements targets (HK_STAT_TIS_DATA_ACQ_TARGET):
space_index_merged.shape=(21,)
bb7_index_merged.shape=(0,)
bb3_index_merged.shape=(0,)
planet_index_merged.shape=(0,)
Collected data statistics:
Number of TIS files: 1
Number of HK files: 2
Number of TIR files: 0
Number of TIS QL files: 0
Number of TIR QL files: 0
Frames: ['mer_cal_sc_tis_20200409_1-0651130819-21186__0_1']
Wavelengths: ['mer_cal_sc_tis_20200409_1-0651130819-21186__0_1']
Metadata tables: ['mer_cal_sc_tis_20200409_1-0651130819-21186__0_1']

Reuse the Cached Frames

original_frames = ms_reader.get_original_frames()
first_key = next(iter(original_frames))

print(f"Original frame shape for {first_key}: {original_frames[first_key].shape}")

Original frame shape for mer_cal_sc_tis_20200409_1-0651130819-21186__0_1: (40, 100, 21)

The code above mirrors the old notebook flow, but the published site now stays in Quarto while the API is documented separately from docstrings.