PDS4 Validation

This page covers the label validation helpers from nbs/03_pds4_validation.py.

Import the Helpers

from pathlib import Path

from mertisreader.pds4_validation import (
    PDS4_TOOLS_AVAILABLE,
    extract_label_metadata,
    extract_mertis_hk_columns,
    get_csv_field_metadata,
    validate_pds4_label,
)

print(f"pds4_tools available: {PDS4_TOOLS_AVAILABLE}")

pds4_tools available: True

Locate a Sample Label

candidate_roots = [Path.cwd().resolve(), Path.cwd().resolve().parent, Path.cwd().resolve().parent.parent]
repo_root = next(root for root in candidate_roots if (root / "data").exists())
sample_dir = repo_root / "data/bcmer_tm_all_START-20200409T000000_END-20200410T000000_CRE-20240717T132010-ParamEventBootSciHK-short/raw"

hk_files = list(sample_dir.glob("*_hk_*.dat")) if sample_dir.exists() else []
print(f"Sample directory exists: {sample_dir.exists()}")
print(f"HK files found: {len(hk_files)}")

Sample directory exists: True
HK files found: 2

Extract Metadata When Available

if hk_files:
    metadata = extract_label_metadata(hk_files[0])
    print(metadata)

    columns = extract_mertis_hk_columns(hk_files[0])
    print(columns[:5])

    field_meta = get_csv_field_metadata(hk_files[0])
    print(field_meta.head())
else:
    print("No HK sample file available in the current checkout.")

Processing label: /home/kidpixo/work/esa/mertis_data_management/mertisreader/data/bcmer_tm_all_START-20200409T000000_END-20200410T000000_CRE-20240717T132010-ParamEventBootSciHK-short/raw/mer_raw_hk_default_20200409_1-0651130766-12538__0_1.dat
{'product_line': None, 'label_version': None, 'processing_level': None, 'acquisition': {}, 'instrument': {}, 'error': 'The requested PDS4 label file does not appear contain valid XML: /home/kidpixo/work/esa/mertis_data_management/mertisreader/data/bcmer_tm_all_START-20200409T000000_END-20200410T000000_CRE-20240717T132010-ParamEventBootSciHK-short/raw/mer_raw_hk_default_20200409_1-0651130766-12538__0_1.dat'}
Processing label: /home/kidpixo/work/esa/mertis_data_management/mertisreader/data/bcmer_tm_all_START-20200409T000000_END-20200410T000000_CRE-20240717T132010-ParamEventBootSciHK-short/raw/mer_raw_hk_default_20200409_1-0651130766-12538__0_1.lblx
Now processing a Table_Delimited structure: MERTIS_RAW_DEFAULT_HOUSEKEEPING_DATA_TABLE
['TIME_UTC', 'TIME_OBT', 'TimeStamp', 'HK_STAT_SID', 'PAD']
Processing label: /home/kidpixo/work/esa/mertis_data_management/mertisreader/data/bcmer_tm_all_START-20200409T000000_END-20200410T000000_CRE-20240717T132010-ParamEventBootSciHK-short/raw/mer_raw_hk_default_20200409_1-0651130766-12538__0_1.lblx
Now processing a Table_Delimited structure: MERTIS_RAW_DEFAULT_HOUSEKEEPING_DATA_TABLE
          name     unit                            description  \
0     TIME_UTC     None                UTC time of acquisition
1     TIME_OBT     None            OnBoard Time of acquisition
2    TimeStamp  seconds     Instrument timestap of acquisition
3  HK_STAT_SID     none    Structure Identifier for HK packets
4          PAD     none  Parameter is used for as filling byte

                 data_type
0  ASCII_Date_Time_YMD_UTC
1             ASCII_String
2               ASCII_Real
3            ASCII_Integer
4            ASCII_Integer

Notes

The helper functions prefer pds4_tools when it is installed and fall back to XML parsing when needed.