Example: Inspecting a NeXus file

Example: Inspecting a NeXus file#

To demonstrate that the h5rdmtoolbox works with NeXus files, we download a file and inspect it. Note that by “works” we don’t mean that the data model of NeXus is understood by the toolbox. Rather, we want to demonstrate that we can identify specific data in the file, plot it, and apply a minimal example of convention and layout validation that we will quickly design here.

import h5rdmtoolbox as h5tbx

1. Get the file#

import requests
import pathlib

if not pathlib.Path('file.nxs').exists():
    r = requests.get('https://github.com/nexusformat/exampledata/raw/master/Soleil/hdf5/file_1.nxs')
    with open('file.nxs', 'wb') as f:
        f.write(r.content)

2. Visit (Dump) the file#

h5tbx.dump('file.nxs')

/home/docs/checkouts/readthedocs.org/user_builds/h5rdmtoolbox/checkouts/v2.2.1/h5rdmtoolbox/_repr.py:594: RuntimeWarning: Cannot decode attribute value for units
  warnings.warn(f'Cannot decode attribute value for {name}', RuntimeWarning)

/(1)
- HDF5_Version: 1.6.5
- NeXus_version: {Version of NeXus API used in writing the file}
- creator: Synchrotron SOLEIL
- file_name: {File name of original NeXus file}
- file_time: {Date and time of file creation}
- file_update_time: {Date and time of last file change at close}
- NDX3_00021_ND__DLPC__00001(23)
  - NX_class: NXentry
  - name: {Entry Name}
  - SWING(59)
    - NX_class: NXinstrument
    - name: {Name of instrument}
    - ANS-C11__EI__C-U20__#1(1)
      - NX_class: NXinsertion_device
      - name: {Name of insertion device}
    - ANS__CA__MACHINESTATUS__#1(6)
      - NX_class: NXsource
      - name: source
      - LifeTime(2)
        NX_class: NXtechnical_data
        name: {Data name}
        
        data (1) [float64]
        description: Data values
        long_name: {Title of technical data}
        timestamp: 2008-10-23T15-06-53.962
        units: h
        
        description: [|S82]
        description: Data description
    - Aviex(15)
      - NX_class: NXdetector
      - name: {Name of detector bank}
      - Exposure(2)
        NX_class: NXtechnical_data
        name: {Data name}
        
        data (1) [float64]
        description: Data values
        long_name: {Title of technical data}
        timestamp: 2008-10-23T15-09-38.425
        units: ms
        
        description: [|S79]
        description: Data description
      - Frames(2)
        NX_class: NXtechnical_data
        name: {Data name}
        
        data (1) [int32]
        description: Data values
        long_name: {Title of technical data}
        timestamp: 2008-10-23T15-09-38.441
        units:
        
        description: [|S87]
        description: Data description
      - ROI1Xmax(1)
        NX_class: NXtechnical_data
        name: {Data name}
        
        data (1) [int32]
        description: Data values
        long_name: {Title of technical data}
        timestamp: 2008-10-23T15-09-38.464
      - ROI1Xmin(1)
        NX_class: NXtechnical_data
        name: {Data name}
        
        data (1) [int32]
        description: Data values
        long_name: {Title of technical data}
        timestamp: 2008-10-23T15-09-38.458
      - ROI1Ymax(1)
        NX_class: NXtechnical_data
        name: {Data name}
        
        data (1) [int32]
        description: Data values
        long_name: {Title of technical data}
        timestamp: 2008-10-23T15-09-38.476
      - ROI1Ymin(1)
        NX_class: NXtechnical_data
        name: {Data name}
        
        data (1) [int32]
        description: Data values
        long_name: {Title of technical data}
        timestamp: 2008-10-23T15-09-38.470
      - Saturation(2)
        NX_class: NXtechnical_data
        name: {Data name}
        
        data (1) [float64]
        description: Data values
        long_name: {Title of technical data}
        timestamp: 2008-10-23T15-09-38.432
        
        description: [|S13]
        description: Data description
      - ShutterCloseDelay(2)
        NX_class: NXtechnical_data
        name: {Data name}
        
        data (1) [float64]
        description: Data values
        long_name: {Title of technical data}
        timestamp: 2008-10-23T15-09-38.498
        units: ms
        
        description: [|S36]
        description: Data description
      - ShutterMode(2)
        NX_class: NXtechnical_data
        name: {Data name}
        
        data (1) [int32]
        description: Data values
        long_name: {Title of technical data}
        timestamp: 2008-10-23T15-09-38.494
        
        description: [|S92]
        description: Data description
      - Trigger(2)
        NX_class: NXtechnical_data
        name: {Data name}
        
        data (1) [int32]
        description: Data values
        long_name: {Title of technical data}
        timestamp: 2008-10-23T15-09-38.447
        units:
        
        description: [|S125]
        description: Data description
      - UseROI(2)
        NX_class: NXtechnical_data
        name: {Data name}
        
        data (1) [uint8]
        description: Data values
        long_name: {Title of technical data}
        timestamp: 2008-10-23T15-09-38.453
        
        description: [|S69]
        description: Data description
      - Xbin(2)
        NX_class: NXtechnical_data
        name: {Data name}
        
        data (1) [int32]
        description: Data values
        long_name: {Title of technical data}
        timestamp: 2008-10-23T15-09-38.482
        
        description: [|S42]
        description: Data description
      - Ybin(2)
        NX_class: NXtechnical_data
        name: {Data name}
        
        data (1) [int32]
        description: Data values
        long_name: {Title of technical data}
        timestamp: 2008-10-23T15-09-38.489
        
        description: [|S40]
        description: Data description
    - I11-C-C00__CA__SAI.B__#1(3)
      - NX_class: NXsai_controller
      - name: {controller name}
      - Frequency(2)
        NX_class: NXtechnical_data
        name: {Data name}
        
        data (1) [float64]
        description: Data values
        long_name: {Title of technical data}
        timestamp: 2008-10-23T15-06-53.999
        units: Hz
        
        description: [|S29]
        description: Data description
      - IntegrationTime(2)
        NX_class: NXtechnical_data
        name: {Data name}
        
        data (1) [float64]
        description: Data values
        long_name: {Title of technical data}
        timestamp: 2008-10-23T15-06-54.002
        units: ms
        
        description: [|S23]
        description: Data description
      - TriggerNumber(2)
        NX_class: NXtechnical_data
        name: {Data name}
        
        data (1) [int32]
        description: Data values
        long_name: {Title of technical data}
        timestamp: 2008-10-23T15-06-54.005
        units:
        
        description: [|S148]
        description: Data description
    - I11-C-C00__CA__SAI.C__#1(3)
      - NX_class: NXsai_controller
      - name: {controller name}
      - Frequency(2)
        NX_class: NXtechnical_data
        name: {Data name}
        
        data (1) [float64]
        description: Data values
        long_name: {Title of technical data}
        timestamp: 2008-10-23T15-06-54.021
        units: Hz
        
        description: [|S29]
        description: Data description
      - IntegrationTime(2)
        NX_class: NXtechnical_data
        name: {Data name}
        
        data (1) [float64]
        description: Data values
        long_name: {Title of technical data}
        timestamp: 2008-10-23T15-06-54.031
        units: ms
        
        description: [|S23]
        description: Data description
      - TriggerNumber(2)
        NX_class: NXtechnical_data
        name: {Data name}
        
        data (1) [int32]
        description: Data values
        long_name: {Title of technical data}
        timestamp: 2008-10-23T15-06-54.041
        units:
        
        description: [|S148]
        description: Data description
    - I11-C-C00__CA__SAI.D__#1(3)
      - NX_class: NXsai_controller
      - name: {controller name}
      - Frequency(2)
        NX_class: NXtechnical_data
        name: {Data name}
        
        data (1) [float64]
        description: Data values
        long_name: {Title of technical data}
        timestamp: 2008-10-23T15-06-54.058
        units: Hz
        
        description: [|S29]
        description: Data description
      - IntegrationTime(2)
        NX_class: NXtechnical_data
        name: {Data name}
        
        data (1) [float64]
        description: Data values
        long_name: {Title of technical data}
        timestamp: 2008-10-23T15-06-54.068
        units: ms
        
        description: [|S23]
        description: Data description
      - TriggerNumber(2)
        NX_class: NXtechnical_data
        name: {Data name}
        
        data (1) [int32]
        description: Data values
        long_name: {Title of technical data}
        timestamp: 2008-10-23T15-06-54.096
        units:
        
        description: [|S148]
        description: Data description
    - I11-C-C03__OP__MONO__#1(2)
      - NX_class: NXmonochromator
      - name: name of monochromator
    - I11-C-C04__EX__FENT_AL1-MT_D__#1(3)
      - NX_class: NXpositioner
      - name: {Name of positioner}
    - I11-C-C04__EX__FENT_AL1-MT_D__#2(3)
      - NX_class: NXpositioner
      - name: {Name of positioner}
    - I11-C-C04__EX__FENT_AL1-MT_I__#1(3)
      - NX_class: NXpositioner
      - name: {Name of positioner}
    - I11-C-C04__EX__FENT_AL1-MT_I__#2(3)
      - NX_class: NXpositioner
      - name: {Name of positioner}
    - I11-C-C04__EX__FENT_AL1-MT_O__#1(3)
      - NX_class: NXpositioner
      - name: {Name of positioner}
    - I11-C-C04__EX__FENT_AL1-MT_O__#2(3)
      - NX_class: NXpositioner
      - name: {Name of positioner}
    - I11-C-C04__EX__FENT_AL1-MT_U__#1(3)
      - NX_class: NXpositioner
      - name: {Name of positioner}
    - I11-C-C04__EX__FENT_AL1-MT_U__#2(3)
      - NX_class: NXpositioner
      - name: {Name of positioner}
    - I11-C-C05__OP__KB.1-MT_Rx__#1(3)
      - NX_class: NXpositioner
      - name: {Name of positioner}
    - I11-C-C05__OP__KB.1-MT_Tz__#1(3)
      - NX_class: NXpositioner
      - name: {Name of positioner}
    - I11-C-C05__OP__KB.2-MT_Rz__#1(3)
      - NX_class: NXpositioner
      - name: {Name of positioner}
    - I11-C-C05__OP__KB.2-MT_Tx__#1(3)
      - NX_class: NXpositioner
      - name: {Name of positioner}
    - I11-C-C06__EX__FENT_AL2-MT_D__#1(3)
      - NX_class: NXpositioner
      - name: {Name of positioner}
    - I11-C-C06__EX__FENT_AL2-MT_D__#2(3)
      - NX_class: NXpositioner
      - name: {Name of positioner}
    - I11-C-C06__EX__FENT_AL2-MT_I__#1(3)
      - NX_class: NXpositioner
      - name: {Name of positioner}
    - I11-C-C06__EX__FENT_AL2-MT_I__#2(3)
      - NX_class: NXpositioner
      - name: {Name of positioner}
    - I11-C-C06__EX__FENT_AL2-MT_O__#1(3)
      - NX_class: NXpositioner
      - name: {Name of positioner}
    - I11-C-C06__EX__FENT_AL2-MT_O__#2(3)
      - NX_class: NXpositioner
      - name: {Name of positioner}
    - I11-C-C06__EX__FENT_AL2-MT_U__#1(3)
      - NX_class: NXpositioner
      - name: {Name of positioner}
    - I11-C-C06__EX__FENT_AL2-MT_U__#2(3)
      - NX_class: NXpositioner
      - name: {Name of positioner}
    - I11-C-C07__DT__MI_DIODE.5(5)
      - NX_class: NXintensity_monitor
      - name: {monitor name}
      - AverageIntensity(2)
        NX_class: NXtechnical_data
        name: {Data name}
        
        data (1) [float64]
        description: Data values
        long_name: {Title of technical data}
        timestamp: 2008-10-23T15-09-49.500
        units: A
        
        description: [|S91]
        description: Data description
      - Gain(2)
        NX_class: NXtechnical_data
        name: {Data name}
        
        data (1) [float64]
        description: Data values
        long_name: {Title of technical data}
        timestamp: 2008-10-23T15-09-49.490
        units:
        
        description: [|S46]
        description: Data description
      - bias(2)
        NX_class: NXtechnical_data
        name: {Data name}
        
        data: [|S37]
        description: Data values
        long_name: {Title of technical data}
        
        description: [|S49]
        description: Data description
      - offset(2)
        NX_class: NXtechnical_data
        name: {Data name}
        
        data (1) [float64]
        description: Data values
        long_name: {Title of technical data}
        timestamp: 2008-10-23T15-09-49.503
        units: V
        
        description: [|S26]
        description: Data description
    - I11-C-C07__EX__FENT_AD3-MT_D__#1(3)
      - NX_class: NXpositioner
      - name: {Name of positioner}
    - I11-C-C07__EX__FENT_AD3-MT_D__#2(3)
      - NX_class: NXpositioner
      - name: {Name of positioner}
    - I11-C-C07__EX__FENT_AD3-MT_I__#1(3)
      - NX_class: NXpositioner
      - name: {Name of positioner}
    - I11-C-C07__EX__FENT_AD3-MT_I__#2(3)
      - NX_class: NXpositioner
      - name: {Name of positioner}
    - I11-C-C07__EX__FENT_AD3-MT_O__#1(3)
      - NX_class: NXpositioner
      - name: {Name of positioner}
    - I11-C-C07__EX__FENT_AD3-MT_O__#2(3)
      - NX_class: NXpositioner
      - name: {Name of positioner}
    - I11-C-C07__EX__FENT_AD3-MT_U__#1(3)
      - NX_class: NXpositioner
      - name: {Name of positioner}
    - I11-C-C07__EX__FENT_AD3-MT_U__#2(3)
      - NX_class: NXpositioner
      - name: {Name of positioner}
    - I11-C-C08__EX__FENT_AD4-MT_D__#1(3)
      - NX_class: NXpositioner
      - name: {Name of positioner}
    - I11-C-C08__EX__FENT_AD4-MT_D__#2(3)
      - NX_class: NXpositioner
      - name: {Name of positioner}
    - I11-C-C08__EX__FENT_AD4-MT_I__#1(3)
      - NX_class: NXpositioner
      - name: {Name of positioner}
    - I11-C-C08__EX__FENT_AD4-MT_I__#2(3)
      - NX_class: NXpositioner
      - name: {Name of positioner}
    - I11-C-C08__EX__FENT_AD4-MT_O__#1(3)
      - NX_class: NXpositioner
      - name: {Name of positioner}
    - I11-C-C08__EX__FENT_AD4-MT_O__#2(3)
      - NX_class: NXpositioner
      - name: {Name of positioner}
    - I11-C-C08__EX__FENT_AD4-MT_U__#1(3)
      - NX_class: NXpositioner
      - name: {Name of positioner}
    - I11-C-C08__EX__FENT_AD4-MT_U__#2(3)
      - NX_class: NXpositioner
      - name: {Name of positioner}
    - I11-C-C08__EX__TAB.1-MT_TX.2__#1(3)
      - NX_class: NXpositioner
      - name: {Name of positioner}
    - I11-C-C08__EX__TAB.1-MT_TZ.2__#1(3)
      - NX_class: NXpositioner
      - name: {Name of positioner}
    - I11-C-C08__EX__TAB.1-MT_Tx__#1(3)
      - NX_class: NXpositioner
      - name: {Name of positioner}
    - I11-C-C09__DT__DTC_CCD.A(1)
      - NX_class: NXdetector
      - name: {Name of detector bank}
      - Exposure(2)
        NX_class: NXtechnical_data
        name: {Data name}
        
        data (1) [float64]
        description: Data values
        long_name: {Title of technical data}
        timestamp: 2008-10-23T15-09-38.425
        units: ms
        
        description: [|S79]
        description: Data description
    - I11-C-C09__DT__DTC_CCD.A__#1(1)
      - NX_class: NXdetector
      - name: {Name of detector bank}
      - Exposure(2)
        NX_class: NXtechnical_data
        name: {Data name}
        
        data (1) [float64]
        description: Data values
        long_name: {Title of technical data}
        timestamp: 2008-10-23T15-06-55.540
        units: ms
        
        description: [|S79]
        description: Data description
    - I11-C-C09__DT__MI_DIODE.8a(5)
      - NX_class: NXintensity_monitor
      - name: {monitor name}
      - AverageIntensity(2)
        NX_class: NXtechnical_data
        name: {Data name}
        
        data (1) [float64]
        description: Data values
        long_name: {Title of technical data}
        timestamp: 2008-10-23T15-09-50.060
        units: A
        
        description: [|S91]
        description: Data description
      - Gain(2)
        NX_class: NXtechnical_data
        name: {Data name}
        
        data (1) [float64]
        description: Data values
        long_name: {Title of technical data}
        timestamp: 2008-10-23T15-09-49.963
        units:
        
        description: [|S46]
        description: Data description
      - bias(2)
        NX_class: NXtechnical_data
        name: {Data name}
        
        data: [|S38]
        description: Data values
        long_name: {Title of technical data}
        
        description: [|S50]
        description: Data description
      - offset(2)
        NX_class: NXtechnical_data
        name: {Data name}
        
        data (1) [float64]
        description: Data values
        long_name: {Title of technical data}
        timestamp: 2008-10-23T15-09-50.174
        units: V
        
        description: [|S26]
        description: Data description
    - I11-C-C09__DT__MI_DIODE.8a__#1(4)
      - NX_class: NXintensity_monitor
      - name: {monitor name}
      - AverageIntensity(2)
        NX_class: NXtechnical_data
        name: {Data name}
        
        data (1) [float64]
        description: Data values
        long_name: {Title of technical data}
        timestamp: 2008-10-23T15-06-55.642
        units: A
        
        description: [|S91]
        description: Data description
      - Gain(2)
        NX_class: NXtechnical_data
        name: {Data name}
        
        data (1) [float64]
        description: Data values
        long_name: {Title of technical data}
        timestamp: 2008-10-23T15-06-55.631
        units:
        
        description: [|S46]
        description: Data description
      - bias(2)
        NX_class: NXtechnical_data
        name: {Data name}
        
        data: [|S38]
        description: Data values
        long_name: {Title of technical data}
        
        description: [|S50]
        description: Data description
      - offset(2)
        NX_class: NXtechnical_data
        name: {Data name}
        
        data (1) [float64]
        description: Data values
        long_name: {Title of technical data}
        timestamp: 2008-10-23T15-06-55.646
        units: V
        
        description: [|S26]
        description: Data description
    - I11-C-C09__DT__MI_DIODE.9__#1(4)
      - NX_class: NXintensity_monitor
      - name: {monitor name}
      - AverageIntensity(2)
        NX_class: NXtechnical_data
        name: {Data name}
        
        data (1) [float64]
        description: Data values
        long_name: {Title of technical data}
        timestamp: 2008-10-23T15-06-55.668
        units: A
        
        description: [|S91]
        description: Data description
      - Gain(2)
        NX_class: NXtechnical_data
        name: {Data name}
        
        data (1) [float64]
        description: Data values
        long_name: {Title of technical data}
        timestamp: 2008-10-23T15-06-55.657
        units:
        
        description: [|S46]
        description: Data description
      - bias(2)
        NX_class: NXtechnical_data
        name: {Data name}
        
        data: [|S37]
        description: Data values
        long_name: {Title of technical data}
        
        description: [|S49]
        description: Data description
      - offset(2)
        NX_class: NXtechnical_data
        name: {Data name}
        
        data (1) [float64]
        description: Data values
        long_name: {Title of technical data}
        timestamp: 2008-10-23T15-06-55.671
        units: V
        
        description: [|S26]
        description: Data description
    - I11-C-C09__EX__BST.3-MT_Tx__#1(3)
      - NX_class: NXpositioner
      - name: {Name of positioner}
    - I11-C-C09__EX__BST.3-MT_Tz__#1(3)
      - NX_class: NXpositioner
      - name: {Name of positioner}
    - I11-C-C09__EX__BST.6-MT_Tx__#1(3)
      - NX_class: NXpositioner
      - name: {Name of positioner}
    - I11-C-C09__EX__BST.6-MT_Tz__#1(3)
      - NX_class: NXpositioner
      - name: {Name of positioner}
    - I11-C-C09__EX__SPT.1-MT_Ts__#1(3)
      - NX_class: NXpositioner
      - name: {Name of positioner}
    - I11-C-C09__EX__SPT.2-MT_Tx__#1(3)
      - NX_class: NXpositioner
      - name: {Name of positioner}
    - I11-C-C09__EX__SPT.2-MT_Tz.1__#1(3)
      - NX_class: NXpositioner
      - name: {Name of positioner}
  - Sample(1)
    - NX_class: NXsample
    - name: {name of the sample}
    - temperature_env(1)
      - NX_class: NXenvironment
      - name: {Name of sample environment}
      - I11-C-C08__EX__THERM_BATH.1__#1(6)
        NX_class: NXsensor
        name: {Name of sensor}
        
        measurement: [|S11]
        description: what we measure "temperature | pH | magnetic_field | electric field | conductivity | resistance ...
        
        model: [|S17]
        description: Sensor identification code/model number
        
        name: [|S25]
        description: Name for the sensor
        
        externalTemperature(2)
        NX_class: NXtechnical_data
        name: {Data name}
        
        data (1) [float64]
        description: Data values
        long_name: {Title of technical data}
        timestamp: 2008-10-23T15-06-55.523
        units: b'\xb0C'
        
        description: [|S26]
        description: Data description
        
        internalTemperature(2)
        NX_class: NXtechnical_data
        name: {Data name}
        
        data (1) [float64]
        description: Data values
        long_name: {Title of technical data}
        timestamp: 2008-10-23T15-06-55.412
        units: b'\xb0C'
        
        description: [|S26]
        description: Data description
        
        setPointTemperature(2)
        NX_class: NXtechnical_data
        name: {Data name}
        
        data (1) [float64]
        description: Data values
        long_name: {Title of technical data}
        timestamp: 2008-10-23T15-06-55.525
        units: b'\xb0C'
        
        description: [|S50]
        description: Data description
  - User(7)
    - NX_class: NXuser
  - comment_conditions(2)
    - NX_class: NXnote
    - name: {name of note}
  - experimental_frame(2)
    - NX_class: NXnote
    - name: {name of note}
  - image#0(1)
    - NX_class: NXdata
    - name: {Name of data block}
  - image#1(1)
    - NX_class: NXdata
    - name: {Name of data block}
  - image#2(1)
    - NX_class: NXdata
    - name: {Name of data block}
  - image#3(1)
    - NX_class: NXdata
    - name: {Name of data block}
  - image#4(1)
    - NX_class: NXdata
    - name: {Name of data block}
  - image#5(1)
    - NX_class: NXdata
    - name: {Name of data block}
  - image#6(1)
    - NX_class: NXdata
    - name: {Name of data block}
  - image#7(1)
    - NX_class: NXdata
    - name: {Name of data block}
  - image#8(1)
    - NX_class: NXdata
    - name: {Name of data block}
  - image#9(1)
    - NX_class: NXdata
    - name: {Name of data block}
  - sample_info(1)
    - NX_class: NXsample
    - name: {name of the sample}
    - comment_sample(2)
      - NX_class: NXnote
      - name: {name of note}

3. Find and plot data#

obj = h5tbx.database.FileDB('file.nxs').find_one({'$name': {'$basename': 'image#5'}},   
                                                 recursive=True)
obj

<LGroup "/NDX3_00021_ND__DLPC__00001/image#5" in "file.nxs">

obj.data[()].plot()  # note, we can access children (e.g. dataset "data") of the (lazy) group

<matplotlib.collections.QuadMesh at 0x731dbef319c0>

../_images/29ebd1f05592b6f9963e76ae4740d368f7b8af2dea42e9eaa74450e9234471ec.png

4. Write a standard attribute for timestamp#

If you carefully check the dump output, you will find, that all numeric (!) datasets have the attribute “timestamp”. All of them? Well, let’s write a convention, which defines that providing the attribute “timestamp” with every numeric dataset is required. The timestamp value must have a specific format.

Let’s create the standard name first:

4.1 Standard Name “timestamp”#

from h5rdmtoolbox.convention import StandardAttribute

from pydantic import BaseModel, ValidationError
from pydantic.functional_validators import AfterValidator, Annotated
from datetime import datetime
from typing import Union

def parse_datetime(v: Union[str, datetime]) -> str:
    """validation function used by the pydantic model class"""
    _format = '%Y-%m-%dT%H-%M-%S.%f'
    if isinstance(v, datetime):
        return v.strftime(_format)
    return datetime.strptime(v, _format).strftime(_format)

class NexusTimeStampObject(BaseModel):
    """The standard name table of the convention."""
    value: Annotated[str, AfterValidator(parse_datetime)]

NexusTimeStamp = StandardAttribute('timestamp',
                                   validator=NexusTimeStampObject,
                                   target_method='create_dataset',
                                   description='Timestamp of writing the data',
                                  default_value='$EMPTY')

4.2 Build the convention#

from h5rdmtoolbox.convention import Convention

cv = Convention(name='nexus', contact=h5tbx.__author__)

cv.add_standard_attribute(NexusTimeStamp)
cv.register()

h5tbx.use(cv)

using("nexus")

First test the convention on a temporary file:

with h5tbx.File() as h5:
    h5.create_dataset('test', data=4, timestamp='2008-10-23T15-09-39.175')
    h5.dump()

/(1)

# should raise an error due to NO timestamp
with h5tbx.File() as h5:
    try:
        h5.create_dataset('test', data=4)
    except h5tbx.errors.StandardAttributeError as e:
        print(e)

Convention "nexus" expects standard attribute "timestamp" to be provided as an argument during dataset creation.

# should raise an error due to a wrong timestamp
with h5tbx.File() as h5:
    try:
        h5.create_dataset('test', data=4, timestamp='2008 10 23 15-09-39.175')
    except h5tbx.errors.StandardAttributeError as e:
        print(e)

Validation of "2008 10 23 15-09-39.175" for standard attribute "timestamp" failed.
Expected fields: {'value': FieldInfo(annotation=str, required=True, metadata=[AfterValidator(func=<function parse_datetime at 0x731dbef8f130>)])}
Pydantic error: 1 validation error for NexusTimeStampObject
value
  Value error, time data '2008 10 23 15-09-39.175' does not match format '%Y-%m-%dT%H-%M-%S.%f' [type=value_error, input_value='2008 10 23 15-09-39.175', input_type=str]
    For further information visit https://errors.pydantic.dev/2.10/v/value_error

4.3 Validate the NeXus file based on the convention:#

We find, that two datasets in fact have no timestamp!

invalid_objs = cv.validate('file.nxs')
invalid_objs

[MissingAttribute(/NDX3_00021_ND__DLPC__00001/duration, timestamp),
 MissingAttribute(/NDX3_00021_ND__DLPC__00001/run_number, timestamp)]

Let’s convince ourselves:

obj = h5tbx.database.FileDB('file.nxs').find_one({'$name': invalid_objs[0].object_name})
obj.attrs.get('timestamp', 'missing!!!')

'missing!!!'

h5tbx.use(None)

using("h5py")

5. Validating the file structure with a layout definition#

The structure or layout defines which groups, datasets and attributes need to exist and in which arrangement. This cannot be defined/checked during file creation (using standard attributes).

Let’s define part of a potential layout for the NeXus file in words first and then write the code:

Ten datasets starting with “image#” and ending with a number between 0 and 9
The group user must exist, which has the datasets “name” with the attribute “description”

The layout is defined by specifying queries, which correspond to what we are expecting in the file. If it is not found, the layout specification failed:

from h5rdmtoolbox import layout

lay = layout.Layout()

# 1.) 10 images:
spec_10images = lay.add(h5tbx.database.FileDB.find,
                      flt={'$name': {'$regex': '.*/image#[0-9]$'}},
                      n=10)

# 2.) User group. Identify by attribute "NX_class":
spec_userGroupExists = lay.add(h5tbx.database.FileDB.find,
                               flt={'NX_class': b'NXuser'},
                               n=None)

# within the User group, find group "name":
spec_UserName = spec_userGroupExists.add(
    h5tbx.database.ObjDB.find_one,
    flt={'$name': {'$basename': 'name'}},
    n=1
)

# within the Name group find dataset "name" with a certain "description":
spec_UserNameDescription = spec_UserName.add(
    h5tbx.database.ObjDB.find_one,
    flt={'$name': {'$basename': 'name'}, 'description': b'Name of user responsible for this entry'},  # !binary string!
    n=1
)

Validate the file:

res = lay.validate('file.nxs')
res.is_valid()

True