DIALS core meeting 2021-01-28

Previous Actions

• MG+ND: write a proposal for ‘overall architecture discussion’ and ‘new installer’
  • MG+ND: Come up with a proposal to move away from all code being in header files and consolidate into a single library
• ND: update CONTRIBUTING to include a soft-imposition of stable master, dials#1353.
  • MG to look into and see if we can make a reasonable change otherwise we can drop it
• MG: organise a typing intro lecture, likely to happen in Q2.
• cbflib conda-forge package
  • ND to prepare an agenda for the HDRMX meeting in March
  • ASB to organise a side-meeting next to the HDRMX workshop
  • ASB to explore what should happen to cbflib maintenance in the future
• ND to set up a cbflib_adaptbx pull request for MAR detector fixes. Verify with labelit_regression.
• GW: write dxtbx rewrite project proposal.
• DLS: review Multiple stills view (#1463) by next meeting. If not happened, merge straight in pending news.
• MG/ND/ASB: merge is_stills PR (#1508) with MG to refactor test, ND to add newsfragment, ASB to merge. Proviso to please add a ticket that we will review this as part of the dx2 work.

Agenda

Per-image analysis-related PRs

Bug fix (critical): PR dxtbx#289 Ensure format instance is cached when first read

• Format classes constructed directly by the PIA service do not take advantage of the format cache, causing construction twice. On subsequent images, still always constructed the format class to identify (and then used cache for actual spotfinding)
• Also fixes dxtbx#245 (format_instance caching leads to wrong data when reading the same file twice)
• ASB: will review today

Performance: PR dxtbx#279 NeXus: avoid unnecessarily walking the full tree

Housekeeping - not so critical:

• PR dxtbx#286 DataFactory: if VDS don’t also include individual external datasets
• PR dxtbx#288 Construct Experiments directly rather than via DataBlocks
  • ASB to run xfel-regression tests as per comment

Soft-stable master (dials#1560)

The process we agreed in dials#1353 was to go soft-stable master for a month, then reevaluate whether to make it a requirement.

PR dials#1560 changes the contribution guidelines to that effect.

PR dxtbx#287 is a half-move towards stable master in dxtbx.

Outcome: Both can go in.

Drop Python 2 idioms and compatibility fixes (dials#1567)

There is no further benefit to keeping old constructs around, so this pull request cleans out a couple 1000 lines of old code.

Outcome:

• ASB to add xfel test instructions to the PR (xfel-regression, ucmetrics, simtbx, etc.), for GW to test.
• If tests pass according to instructions PR can be merged.

dx2 proposal

• Move proposal into a .md in the dx2 repository?
• MG’s migration proposal:
  • Install dx2 next to, and in addition to dxtbx.
  • Eventually dxtbx and dx2 need to be able to co-exist anyway.
  • Remove C++ code from dx2 for now, as that fixes conflicts with flex types. Then we can get started with dx2 right away.
  • dx2 can import the C++ bits from dxtbx in the first instance
  • Allows us to figure out how to build C++ in dx2 with dx2 being a normal python package.
  • Tricky bit comes in once we need to diverge in C++ land. I guess it might help that we want to base data structures primarily off numpy rather than flex?

Conversation on this is still ongoing on the mailing list

• Overall the plan is to leave dxtbx in place so that existing software doesnt have to be changed.
• There is no plan to drop format class support.
• ASB to reply on list
• GW to merge all the different discussion strains together

dials.index: create new experiments

not discussed in this meeting

dials.index should (or should not) create new experiments with crystals in rather than modifying in place existing experiments as proposed in this issue

Next meeting

Thursday, February 11th, 4pm UK time, 8am PDT.