DIALS core meeting 2023-06-29

Previous Actions

• ND: PR to remove std::string support for messagepack from dials code
• ND Investigate getting psana tests running on the DIALS xfel-regression testing
• Dropping Data Blocks (#504 #570)
  • ND: Look at DIALS failures and work out what needs to be migrated
• AB: Check test case on Multidimensional Images dxtbx#612 is reasonable
• DP: Add msgpack “hot” path to include environment in cctbx (prev)
  • Work in https://github.com/cctbx/cctbx_project/commit/03a9a6797044126984a69ff30d00be6df408e43a, pending approval/merge
• [not until august at least] JBE: Solve “multiplex: Duplicate Batch offsets detected” xia2#430 (prev)
• AB: Check if RG test cases on dxtbx#612 “Multidimensional Images” are appropriate
  • A dependency of dxtbx#615 NX Writer (prev) so promote this to agenda again when done
• AB: Moving forward serialtbx changes (prev). cctbx#872 seems immediate blocker.
• ND: Merge Imageset Slicing PRs: https://github.com/cctbx/dxtbx/pull/634 https://github.com/dials/dials/pull/2411 (prev)

Agenda

Procrunner

• CCTBX removal process merged, a few downstream libraries needed to be updated but seems to have gone smoothly.

dials/2320 - Cosym applies incompatible reindexing operator

• Marked as action for RG
• RG is leaving
• Some movement on ticket, what is status of this
• RG: Investigate dials.cosym applies an incompatible reindexing operator (dials/2320) (prev)
  • Needs a fundamentally different approach to matching algorithm to get unit cells into consistent settings.
  • Needs research. Possibly applicable is upcoming Andres, Bernstein, Sauter paper.

bit_depth_imageset

• Forthcoming NIAC meeting, GW proposal to add should pass
• Pragmatism!
• Current Nexus code camp: To be discussed and hopefully agreed this week
• Constitutional crisis! No quorum to make changes.
• We are going to continue on assumption that this was merged.

DIALS memory usage

• Previously AOB
• Aaron - this should be a top priority, wants to start conversation on how we do this
• David McD made a simple_integrate script that does the basics
  • Would be good as a form of testbed, see what is missing, what needs to be added. Try out different approaches without massive concern for performance.
• GW: We should accept that we are going to need a full dials_integrate rewrite
• Discussion about shoeboxes, original motivation

AOB?

• CHESS SSX Workshop upcoming https://indico.classe.cornell.edu/event/2248/overview

Cannot view reciprocal lattice using dials.reciprocal_lattice_viewer

• https://github.com/dials/dials/issues/2443
• Not aware of this issue
• DGW has not seen, but not tested recently

Dropping Python 3.8

• https://github.com/dials/dials/pull/2445
• https://github.com/cctbx/dxtbx/pull/642
• Following NEP-29, the larger scientific python ecosystem is very actively dropping 3.8 now.
• Several dependencies of upstream packages have dropped (e.g. pint), numpy and scipy actively dropping (or dropped). Seeing dependency issues.

Note on DLS Jungfrau Commissioning?

• Spun onto discussion on MPI at diamond, serial merging and xfel merge - approaches to multi-node processing.

DIALS-support

• Error importing .cbf files using DIALS
  • ND has agreed to take a look at this also

Next meeting

Thursday, July 13th, 4pm (BST), 8am (PDT)