DIALS core meeting 2022-08-18

Previous Actions

• ND to review: TOFBeam - A time-of-flight beam in dxtbx cctbx/dxtbx#492 (prev)
• std::string support for messagepack. This is used in XFEL module, should try to fix. dials/dials#1858.
  • Is using pickle until this is fixed, so probably good to do

Agenda

Dropping Data Blocks

https://github.com/cctbx/dxtbx/pull/504

• GW PR
• After PR/PRs, ASB to contact e.g. Mike Wall about this, too.
• GW thinks reasonable to merge when these concerns addressed
• Waiting on AB now, will get sorted as soon as can
• Progress!
  • DataBlock in Lunus is one of the outstanding issues
    • AB Identified scripts, Mike Wall agreed that they should be deprecated
    • This is Done! Was entirely in deprecated scripts that weren’t being used any more
  • XFEL repos are still WIP
    • ls49
    • cxid9114 (this should be fixed)

LCLS XFEL Updates

• cctbx/dxtbx#517
• GW suggested Ben to have a look
• Action: Add logging to the except Exception: so it doesn’t silently ignore errors
• ASB to turn on tests now dials-data changes below merged
• ASB address DP comments in dials/dials#2110 (I think)
• Probably being treated in future planned hackathon

dials.find_bad_pixels

ASB: would like to discuss dials.background and noisy pixels -> park on agenda for next meeting. TL;DR - noisy pixels at EU-XFEL, wanted to compare and contrast with find bad pixel code. Confusion: Aaron meant dials.find_bad_pixels. MPI enabling, much lower thresholding. Think already uses future parallels? Adjusting threshold could be useful too.

• Shadow goniometer masking is good, but specialised to one use case
• Need more generic so that we can implement in the dxtbx nexus format class
  • ND: Does this mean NXMX?
    • NXMX does have per image mask, but is a per image mask, no way to generate a dynamic mask
• Goniometer masker is only implementation here
• At EuXFEL used LPD detector
  • Detector rotates between four memory banks. Each memory bank has different noise characteristics depending on the gain mode.
  • Would like a way to dynamically calculate a mask depending on bank and gain mode.
• Under active research, can be dropped for now

AOB?

cctbx/dxtbx#538 and Zenodo data

• Talking about https://github.com/cctbx/dxtbx/pull/538 and associated data to go into dials-data
• The incident_wavelength field is derived by (potentially non-trivial) operations on the wavelength spectra, for each image
• Nexus Variants is a mechanism for accessing “Old” version of data for which the current data is derived/refined
• Aaron wants to use this to store the chain of “Derived incident_wavelength” but the original 2D dataset spectra (which is also a valid value for incident_wavelength itself)
• Aaron also wants to access this chain of data, to get the spectra that was used to derive the incident_wavelength, to do separate calculations (but the rest of the code should access the derived version of the field)
• (ND interpretation) Essentially: we need a canonical way to access variants in NXmx. This could potentially apply to any field
• Had code issues implementing
  • Seems related to NXmx _start() function opening HDF5, reading intial data, then closing link to H5 file
  • If data is read, it’s re-opened and then that file handle is Cached
  • Maybe we should just cache the initial handle?

Next meeting

Thursday, September 1st, 4pm (BST), 8am (PDT)