DIALS core meeting 2022-03-03

This is a future meeting, please see the WIP agenda at hackmd.io

Previous Actions

• dxtbx src/ layout [prev]
  • Nick: Outstanding issues with dispatcher re-export on windows
  • This is a higher priority now - David feeling pain of missing
  • Also, seeing related errors “in the wild
  • Nick: Action before 3.9
• Removal of DataBlock [prev, prev]
  • Nick: More visible deprecationwarning (cctbx/dxtbx#448)
  • Merged 23rd November. Do we want to set any plans for further?
  • Action: GW Make a PR/branch to remove this so that we can run things against it
    • Graeme will do this on 18th March
• DXTBX/pycbf [prev]
  • ND: conda-forge pycbf: Make new release to use dials-data directly for tests
  • Nick: Add lots more attribution
  • dxtbx: Now 3.8.0 our can move forwards conda-forge/staged-recipes#16846
• Nick: Make sure that HKLviewer (prev) works at Diamond on module load
• Aaron: Reduce zenodo dataset for contiguous nexus
  • Put on Dan Paley backburner also - time is short
• (*) Nick: dxtbx and read-only base

Agenda

DXTBX read-only base

• Read only base by default would be the ideal
• Defer for two meetings - pinning on DIALS 3.2/virtualenv script working around for now

Dropping Data Blocks

https://github.com/cctbx/dxtbx/pull/480

Drops data blocks from radial average. All good -> merge. We should enumerate where data blocks are actually used. DIALS, dxtbx, xia2, … we should make that list.

• ASB to contact e.g. Mike Wall about this, too.
  • Blocked behind GW PR

DXTBX conda package

This we believe now exists and works. Does that mean that pycbf got in there with many attributions as we previously discussed?

Add option for dxtbx’s libtbx_refresh to not modify conda_base

• Issue cctbx/dxtbx#444
• ND: Add environment variable to do this
• Add this as default in the CCTBX
  • Couple of false starts
  • Currently awaiting feedback cctbx/dxtbx#474
• do for DIALS, iota, dxtbx, xia2 when accepted
• Top Priority

This is queued behind catching up with Nick - it turned out to be involved.

Contiguous Nexus

• “Contiguous Nexus” cctbx/dxtbx#356
  • Non-draft state is pending checks against issues @dwpaley raised, and checking for non-contiguous cases [prev]
• AB wants this to be applied against new nexus writing
• RG: Probably useful to mock up an example in the nxmxtests
• Action: Aaron Reduce down zenodo dataset used in the issue (cctbx/dxtbx#93) so that we can include it in dials-data.
• Afterward: Coordinate with RG to make sure the ideas in this are integrated into new nexus reading

PRs to remove std::string from messagepack

• dials/dials#1982 DO NOT MERGE
• Discussion about how this is used in XFEL module
• Action: Actually try to fix this (https://github.com/dials/dials/issues/1858)

Commentary: ASB and crew have 5 experiments over the next few weeks; so yeh a bunch of stuff is gonna get bumped.

• Realistically not before April

Still process

• Going to be a PR for “Diffbragg stage 1 refinement”
  • Uses pixel models of image to refine crystal using polychromotic spectra as per Mendez 2020 paper.
  • Implements an override in stills_process

ASB wants to put diffbrag stage 1 into dials.stills_process as a refinement engine. Depends on simtbx. This is very much dependent on GPU. Can use OMP etc.

• Derek Mendez going to send changes to Aaron

Discussion

What should we do with “experimental” matters? Should new experimental programs go in? Trying to dig out legacy potatoes was a nightmare, so there is an argument for adding it on a live branch where you can merge main in from time to time.

• Discussion spawned by James explorations on dials.potatoes
• New work goes in as a branch, early on, kept up-to-date over main

CCP4 / PHENIX

Release now held up by a program being broken, but only one. So therefore something of an unknown. Would bring CCP4 pretty much up to date.

PHENIX - depending on the readonly dxtbx stuff - until that is fixed DIALS will be pinned at 3.2 or similar. They also have a hard stop at Python 3.7, due to a slowly changing dependecy.

Add reading and writing of flux to full CBF files

• https://github.com/cctbx/dxtbx/pull/493
• Add compatibility for current CCTBX release by calling twice with TypeError fallback
• Then merge

Proposal for a time-of-flight beam in dxtbx.

• TOFBeam https://github.com/cctbx/dxtbx/pull/492
• Also scan object required to properly use neutron data in DIALS
• Discussion on motivation and fundings
• Add ND as a reviewer on the commit

Mac Dispatchers

• Briefly discussed, participants not directly familiar with issue. Will continue discussing in tickets.

Next meeting

• 13th March California clocks move forward
• 27th March UK clocks move forwards

so:

Thursday, March 17th, 3pm UK (GMT), 8am PDT.

or

Thursday, March 17th, 4pm UK (GMT), 9am PDT.

Conclusion: Skip March 17th. Next meeting will be 31st March, 4pm UK (BST), 8am PDT.