DIALS core meeting 2022-09-29

Previous Actions

• ND to review: TOFBeam - A time-of-flight beam in dxtbx cctbx/dxtbx#492 (prev)
• std::string support for messagepack. This is used in XFEL module, should try to fix. dials/dials#1858.
  • Is using pickle until this is fixed, so probably good to do

Agenda

Dropping Data Blocks

https://github.com/cctbx/dxtbx/pull/504

• GW PR
• After PR/PRs, ASB to contact e.g. Mike Wall about this, too.
• GW thinks reasonable to merge when these concerns addressed
• Waiting on AB now, will get sorted as soon as can
• Progress!
  • DataBlock in Lunus is one of the outstanding issues
    • AB Identified scripts, Mike Wall agreed that they should be deprecated
    • This is Done! Was entirely in deprecated scripts that weren’t being used any more
  • XFEL repos are still WIP
    • ls49
    • cxid9114 (this should be fixed)
• Planned time to work on this stuff in October

LCLS XFEL Updates

• cctbx/dxtbx#517
• GW suggested Ben to have a look
• Action: Add logging to the except Exception: so it doesn’t silently ignore errors
• ASB to turn on tests now dials-data changes below merged
• ASB address DP comments in dials/dials#2110 (I think)
• Probably being treated in future planned hackathon

cctbx/dxtbx#538 and Zenodo data

• Talking about https://github.com/cctbx/dxtbx/pull/538 and associated data to go into dials-data
• The incident_wavelength field is derived by (potentially non-trivial) operations on the wavelength spectra, for each image
• Nexus Variants is a mechanism for accessing “Old” version of data for which the current data is derived/refined
• Aaron wants to use this to store the chain of “Derived incident_wavelength” but the original 2D dataset spectra (which is also a valid value for incident_wavelength itself)
• Aaron also wants to access this chain of data, to get the spectra that was used to derive the incident_wavelength, to do separate calculations (but the rest of the code should access the derived version of the field)
• (ND interpretation) Essentially: we need a canonical way to access variants in NXmx. This could potentially apply to any field
• Had code issues implementing
  • Seems related to NXmx _start() function opening HDF5, reading intial data, then closing link to H5 file
  • If data is read, it’s re-opened and then that file handle is Cached
  • Maybe we should just cache the initial handle?
• Merged with a view to come back later. Mark as done.

AOB?

“Equipment component”

• “Equipment component” is active part of of Nexus that we want to add in
  • In CBF can collapse levels of the hierarchy together using an axis term called “equipment component”. Lets you combine several transformations together so that hierarchy has fewer layers. Between AB and JMP was put into original Nexus code - reading EuXFEL code would get four-level hierarchy. FormatNXMX gives you a six-level. Marked as such in nexus file so expected it to appear. However! Surprise! It didn’t seem to have actually made it into the nexus specification…
  • Changes number of levels
  • Makes it easier to do per-equipment parametrisation
  • https://github.com/nexusformat/definitions/pull/1182
  • https://github.com/nexusformat/definitions/pull/1182#issuecomment-1227616985
• Going to create a PR over next couple of weeks for this.

XFEL Regression tests running

• https://github.com/dials/dials/issues/2214
• Is a bit confusingly displayed. The Jobs are run, if everything else passed. If the other steps ran but failed, the XFEL regression tests will not run; but will show as having passed. This is less than idea.
• Action: Nick: Investigate getting psana tests running on the DIALS xfel-regression testing

FullCBF Trusted range

• https://github.com/cctbx/dxtbx/issues/559
• Discussion ongoing, needs AB, GW checks to see what this affects.

Raster fix for dials.stills_process

• https://github.com/dials/dials/pull/2128
• Action: JBE going to run tests on this to see if this affects
• Several wary of changing the dials_import assumption to always start at 0-based indices.

Fix beam center and intensity readout of pixel coordinates

• https://github.com/dials/dials/pull/2194
• Action: AB to complete coordinate updates

CBFlib/PyCBF

• What’s the deal, ND is confused
• AB will make enquiries (longer term)

Next meeting

Thursday, October 13th, 4pm (BST), 8am (PDT)