DIALS meeting 2020-09-30
New flex data types
It looks like the institutional knowledge about how flex array innards work has dried up. Consequently adding new supported types is difficult.
Could we only keep the functions exposed to C++ rather than Python?
NKS may have the needed knowledge. A question to cctbxbb hasn’t evoked any replies yet.
JBE bitshuffling
Change dxtbx get_raw_data functions to use bitshuffle directly, as we have more knowledge about the data format than generic HDF5 calls do. (dxtbx #219)
This would put performance at par with Durin.
Takes 20-30% off read decompression time, plus avoids needs for object conversion/copying. Opens up parallel chunk processing as we don’t need to go through HDF5 chokepoint.
proper NeXus support
NearlyNexus should go away.
NearlyNexus is not tolerant. Anything not matching the exact layout will fail, eg. must contain the string Eiger in a magic place.
We also don’t support any non-trivial geometry (dxtbx #200), some hacking towards that was made in dxtbx #206, but that is not merge-ready by any measure.
BW and DGW to follow up after meeting. Test data exist in dials/data-files — see PR dials/data-files#12 and PR dials/data#161. A test seems to have been introduced in PR dxtbx#164.
fp3 (Faster parallel processing pipeline)
This is “fast-dp powered by dials”, PR is dials-scratch#5. It is already much faster than xia2. Would benefit from parallel integrator and any improvements in data handling should be very visible, too.
DGW: Can we use this to bootstrap a more careful DIALS processing run?
Separate new package? Probably yes, because separate documentation and use-case.
Merge into fast_dp? Not at this time.
2D detector projection
This would fix the image viewer for single panel detectors. Obviously not useful for metrology and/or I23.
If useful it could go into the detector module. Or a projection mode in the image viewer. And in the report.
We need to throw different detector images at it, to see how it fares.
Maths questions: Randy and PHZ
Q: Are we interested in detecting different spot classes early on in processing?
This is relevant for order-disorder (OD) structures.
AP: GW to sort out conference call
image viewer
Discuss next time.