DIALS core meeting 2026-03-12

Previous Actions

• ND Investigate getting psana tests running on the DIALS xfel-regression testing
• ND: Move dials.diamond.ac.uk to a redirect to dials.github.io
  • Caveat: Sub-path links either work or not, do not get redirected

  • Agenda

IUCR DIALS Workshop final agenda

Final IUCr DIALS Workshop agenda:

• 8am: welcome and intro to DIALS and data processing (talk/demo) (Graeme/Aaron)
• 9am: hands on tutorial 1 (all)
• 10am: multi-crystal and trouble shooting (talk/demo) (Nick D)
• 10:30 coffee
• 11am: hands on tutorial 2 (all)
• 11:30am: How to expand DIALS into other use cases (10 min flash talks)
  • Rick Hewitt (Laue, confirmed)
  • Aaron Finke (neutron, confirmed)
  • David Mittan-Moreau (electron, confirmed)

2026-03-12:

• We had asked for multiple rooms for parallel streams. Offered four rooms!
• Original plan:
• Mostly agreed to go back to this, will be passed to IUCR

Berlin Diffraction Methods

• Dan from LBL going
• Discussion about NOBUGS
  • Nexus NIAC Fri-Sun
  • Possibilty of weighting more general interest into first day of satellite meetings onto Fri morning session
  • https://indico.xfel.eu/event/2/overview

Other CI Issues

• Installers failing on our internal CI machines. Down to launch logic in https://github.com/dials/dials/blob/e91f5f66368ec546b160ccfd2fbd30b2926a65de/installer/create_installer.py#L36-L52 that prefers python2.7 if available - looks like conda-pack switched to requiring python 3.
• Probably easiest way for now is to just use python/python3 and remove the fallbacks.
• 31/36 xfel-regression tests failing with errors like “/bin/sh: 1: cctbx.xfel.time_varying_refinement: not found”. So some build issue there? Help please?
  • Dan reports working again now

Proposal for dials issue categorisation/management. (JBE)

JBE: I was thinking about how to organise the github issues and would like to give people chance to raise objections/feedback before going ahead. Discussed at Diamond Dials meeting, general agreement there.

• Using github issues labels and/or types, categorise tickets into one of 3 categories: bug, enhancement/feature, or discussion/question. Discussion/question issues can optionally be converted to github discussions (but does anyone look at these)? Bugs are software problems/crashes. Features are feature requests/suggestions. Discussions are user questions or nuanced long discussions on algorithms.
• Have a github project board each for bugs and features.
• On the project boards, classify each ticket using the idea of a 3x3 impact-effort matrix. Impact classification: high, medium, low. Effort classification: difficult (1>week), medium (~day), low (<1hr).

How we would use this.

• After 2+ weeks of a ticket being opened (to give time for initial discussion/external user input) we categorise and classify any new uncategorised issues (At the Tuesday Diamond DIALS meeting and/or Thursday dials-core).
• JBE volunteers to go through historical backlog to classify, or can organise an issue triage meeting(s).
• Use the impact-effort ratings to bring up selected issues for discussion at regular dials meetings with a view to reducing the backlog of most critical issues over time. Helps us to prioritise fixing real bugs.
• Issue fixing will continue to remain a voluntary process: issues are not going to be blindly assigned to people, we all have other priorities.

dials.github.io

• Mac installer link broken
• ND to look into, currently macos tests failing causing the development build not to be deployed
• https://github.com/cctbx/cctbx_project/issues/1119
• Fixed in Feburary 2026 release 2026.1
• ND: Make sure this fixes issue
• Mac breaking for other reasons, OS too old for conda-forge builds, need to retire x86 build.

Metrics

• 2025-09-25 - dxtbx-side merged in, nothing yet in DIALS side to push it into the mtz history https://github.com/cctbx/dxtbx/pull/816
• Needs to be work to put in on DIALS side
• Work to do:
  • Need to package history into MTZ, but MTZ history not the right place. Decided MTZ-appendix is the right place to put this in, but work not started yet
  • Aaron has offered Yang’s skills as his work should cover this area
  • Write integrate and scale history to MTZ #2924
  • David to dig relevant information out
• MTZ Appendix: Some controversy
  • Phenix/DIALS not included in discussions
  • Fundamental technological issues
    • Mixing MTZ/CIF
• Gemmi supports
• AB met with BP/DGW/Oleg/Dorothy and talking about MTZ appendix issue
  • Conclusion: Nobody completely happy, AB to approach Clemens and get conversation going
• Dan Paley found issue with profiling and MTZ appendix https://github.com/cctbx/dxtbx/pull/867
  • Agreed looks reasonable
  • Requested David to have a look
  • All agreed, merge after newsfragment
• AB: To make meeting with Clemens to try to find agreement that we probably need to just go with this

Hamburg Work

• Hamburg: imageset, sequences, laziness so it works on 100,000 image HDF5 files
• Did some work in a branch to get stills_process to use imagesequences
• Nick did some work to eliminate need for check_format
• All of this is in branches that have not been revisited
• Discussion about still goniometer dogma

ImageSequence vs ImageSet

• Marko is working on an NXmx-like format class for SerialED, where the beam position changes each image
• It seems not possible to use ImageSequence for this, as it does not allow different models per image. Is this correct?
• ImageSet works, but import and viewing images is much slower
• Is there a long term plan to converge to just one of these classes, with all performance and feature issues fixed?
• Suggestion to look at DMcD properties table
• Discussion about serialization/deserialization overhead, brief discussion of using another form e.g. hdf5 or sqlite (for Experiment Lists).
• Discussion of existing uses of mixin inheritance
• Marko raises issue where ArgumentParser processes diffs twice, possibly this affects only ImageSet

JSON Parsing

• Some reports that JSON parsing itself can be a bottleneck for large experiments
• People to keep an eye out for good examples of it being slow

Shoebox object size

• DW raises performance issues, particularly with dials.integrate
  • Found issues The Shoebox struct is overweight
  • WIP: https://github.com/dials/dials/pull/3119

New issues for discussion

• https://github.com/dials/dials/issues/3110 dials.index with max_lattices >= 2: rejection criteria.
  • We think related to known symmetry
  • Wait until Graeme here
• Long discussion about integration memory usage and sample datasets.

AOB

• Cylindrical detectors
  • DW raises old documentation on how to handle mathematically from Gerard, to share
  • Daresbury Laue has cylindrical support

Next meeting

Due to time zone changes the normal meeting time must change:

Thursday, March 26th, 3pm (GMT), 8am (PDT), 10am (CDT)