DIALS core meeting 2022-02-17

Previous Actions

• dxtbx src/ layout [prev]
  • Nick: Outstanding issues with dispatcher re-export on windows
  • This is a higher priority now - David feeling pain of missing
  • Also, seeing related errors “in the wild
  • Nick: Action before 3.9
• Removal of DataBlock [prev, prev]
  • Nick: More visible deprecationwarning (cctbx/dxtbx#448)
  • Merged 23rd November. Do we want to set any plans for further?
  • Action: GW Make a PR/branch to remove this so that we can run things against it
• DXTBX/pycbf [prev]
  • ND: conda-forge pycbf: Make new release to use dials-data directly for tests
  • Nick: Add lots more attribution
  • dxtbx: Now 3.8.0 our can move forwards conda-forge/staged-recipes#16846
• Nick: Make sure that HKLviewer (prev) works at Diamond on module load
• Aaron: Reduce zenodo dataset for contiguous nexus
• (*) Nick: dxtbx and read-only base

Agenda

Dropping Data Blocks

https://github.com/cctbx/dxtbx/pull/480

Drops data blocks from radial average. All good -> merge. We should enumerate where data blocks are actually used. DIALS, dxtbx, xia2, … we should make that list.

• ASB to contact e.g. Mike Wall about this, too.

DXTBX conda package

This we believe now exists and works. Does that mean that pycbf got in there with many attributions as we previously discussed?

Add option for dxtbx’s libtbx_refresh to not modify conda_base

• Issue cctbx/dxtbx#444
• ND: Add environment variable to do this
• Add this as default in the CCTBX
  • Couple of false starts
  • Currently awaiting feedback cctbx/dxtbx#474
• do for DIALS, iota, dxtbx, xia2 when accepted
• Top Priority

This is queued behind catching up with Nick - it turned out to be involved.

Contiguous Nexus

• “Contiguous Nexus” cctbx/dxtbx#356
  • Non-draft state is pending checks against issues @dwpaley raised, and checking for non-contiguous cases [prev]
• AB wants this to be applied against new nexus writing
• RG: Probably useful to mock up an example in the nxmxtests
• Action: Aaron Reduce down zenodo dataset used in the issue (cctbx/dxtbx#93) so that we can include it in dials-data.
• Afterward: Coordinate with RG to make sure the ideas in this are integrated into new nexus reading

PRs to remove std::string from messagepack

• dials/dials#1982 DO NOT MERGE
• Discussion about how this is used in XFEL module
• Action: Actually try to fix this (https://github.com/dials/dials/issues/1858)

Commentary: ASB and crew have 5 experiments over the next few weeks; so yeh a bunch of stuff is gonna get bumped.

Merging of tables/experiments

• Make merging of tables and resolution of experiment identifiers easier Action: JBE - work out what we need to handle and how to do this
  • https://github.com/dials/dials/pull/1994
  • Aaron: Write issue describing usage https://github.com/dials/dials/issues/1983
• Remove existing usage of std::string in XFEL modules

Looks good, for JBE to squash and merge.

Still process

• Going to be a PR for “Diffbragg stage 1 refinement”
  • Uses pixel models of image to refine crystal using polychromotic spectra as per Mendez 2020 paper.
  • Implements an override in stills_process

ASB wants to put diffbrag stage 1 into dials.stills_process as a refinement engine. Depends on simtbx. This is very much dependent on GPU. Can use OMP etc.

Discussion

What should we do with “experimental” matters? Should new experimental programs go in? Trying to dig out legacy potatoes was a nightmare, so there is an argument for adding it on a live branch where you can merge main in from time to time.

New business -

https://github.com/cctbx/dxtbx/pull/484

PSANA does not work like reading CBF files; it aggregates information into a data source, so gives you an API for reading data which is on disk somewhere. So, there are locator files on disk. -> FormatXTC + a few sibclasses. Phil file does all the “how we find things” stuff. This PR adds “get_spectrum”.

Changes appear limited to FormatXTC, adds ability to return a spectrum object.

Some tests failed, that’s weird. Something about HTTP versions.

Image Viewers

GW made a napari image viewer. It was slow. Flumpy helped some. It did appear to work. Need to decide where we take this. Folks want to get added to the image viewer working discussion stuff.

CCP4 / PHENIX

Release now held up by a program being broken, but only one. So therefore something of an unknown. Would bring CCP4 pretty much up to date.

PHENIX - depending on the readonly dxtbx stuff - until that is fixed DIALS will be pinned at 3.2 or similar. They also have a hard stop at Python 3.7, due to a slowly changing dependecy.

Deferred to next meeting

Things which depended on Nick :-)

Next meeting

Thursday, March 3rd, 4pm UK (GMT), 8am PT.