DIALS core meeting 2023-06-15

Previous Actions

• ND: PR to remove std::string support for messagepack from dials code
• ND Investigate getting psana tests running on the DIALS xfel-regression testing
• Dropping Data Blocks (#504 #570)
  • ND: Look at DIALS failures and work out what needs to be migrated
• RG: Investigate dials.cosym applies an incompatible reindexing operator (dials/2320) (prev)
• AB: Check test case on Multidimensional Images dxtbx#612 is reasonable
• DP: Add msgpack “hot” path to include environment in cctbx (prev)
  • Work in https://github.com/cctbx/cctbx_project/commit/03a9a6797044126984a69ff30d00be6df408e43a, pending approval/merge
• JBE: Solve https://github.com/xia2/xia2/issues/430

Agenda

NX Writer! (prev, prev)

• https://github.com/cctbx/dxtbx/pull/615
• Dependent on dxtbx#612 “Multidimensional Images”
  • Richard added basic test case -> Aaron is ok?

Removing circular xfel dependencies with serialtbx

• Set of PRs to implement
  • CCTBX: https://github.com/cctbx/cctbx_project/pull/872 - contains descriptions of all changes made
  • dxtbx: https://github.com/cctbx/dxtbx/pull/627
  • dials: https://github.com/dials/dials/pull/2404
  • iota: https://github.com/ssrl-px/iota/pull/5
  • xfel_regression https://gitlab.com/cctbx/xfel_regression/-/merge_requests/10
• We all need to look at things
  • Schedule some time next week at DLS to look over these PRs
• ND reviewed, looks OK , please can we hold off from merging until the dust settles from scan and polybeam changes.
• ASB happy for this to hold off a couple weeks for non-technical reasons

NXMX

• Discussion about dynamic masking in NXmx.
• Defer until DP back, recollection on this discussion is fuzzy
• Possibly:
  • https://github.com/dials/nxmx/pull/17
  • https://github.com/cctbx/dxtbx/pull/629
• Action: RG to have look at
  • Suggested alternate NXmx change https://github.com/dials/nxmx/pull/21
• Discussion concluding that “Static” is probably okay for “Precaculated” masks, even if it changes from frame-to-frame, and “Dynamic” would imply calculated at anaylsis.
• General agreement that this is fine, as long as we clarify the meaning of static on the function docstring
• https://github.com/cctbx/dxtbx/blob/842d45939160d47829e03776a98f671f15ae16a2/src/dxtbx/format/FormatMultiImage.py#L17
• Discussed motivations, AB & DP going to go back and validate that it fulfills the requirements of the data processing that was intended by these changes
• Re-address when ready for submission again

xia2.multiplex fails because “Duplicate batch offsets detected”

• Longstanding (>3 years) issue: https://github.com/xia2/xia2/issues/430
• Now getting reports from real user data
• Potential solution suggested to never share Scan objects cctbx/dxtbx#152
• implement non-sharing of Scan objects - no logic for doing so, since readonly?
• JBE: Do this

Imageset slicing broken

• https://github.com/cctbx/dxtbx/issues/633
• Fix now ready at https://github.com/cctbx/dxtbx/pull/634 and https://github.com/dials/dials/pull/2411
• Anyone else want to look?
• When merged, will require reversion of changes to azure-pipelines YML files to checkout specific dxtbx branch
• DGW thinks this one can be merged without contention - needs sync’d repo merge
• rebase this branch on top of Scans branch & run tests
• Tie off loose ends and merge internally
• ND Merge tomorrow

bit_depth_imageset

• Forthcoming NIAC meeting, GW proposal to add should pass
• Pragmatism!
• Current Nexus code camp: To be discussed and hopefully agreed this week

AOB?

• CHESS SSX Workshop upcoming https://indico.classe.cornell.edu/event/2248/overview

• Aaron Finke was asking Graeme if we had considered how to address the memory requirement issues.

  • ND: Don’t think you could refactor, think would need to re-approach the problem
  • AB: New grant application mentions this issue, would potentially have resources to assign people to the problem

Next meeting

Thursday, June 29th, 4pm (BST), 8am (PDT)