DIALS core meeting 2024-10-31

Previous Actions

• ND Investigate getting psana tests running on the DIALS xfel-regression testing (see Github Actions)
• ND: Sync new releases with current state of dials/cbflib and main branch. Keep synced.

Agenda

CBFlib

• Aaron wants to switch forks for cbflib to make dials fork the main cbflib. Has been discussed with Herbert.
• Questions around cmake build - Johan working on this? Graeme thinks already been done before?
• Aaron to set up call for subgroup of interested people on cbflib switch and build (principally including Nick and Herbert).

PS2 processing discussion

Previous:

• Users had issues processing PS2 data taken at Diamond. Graeme suggested setting up dedicated zoom chat.

Status:

• Still outstanding - Graeme spoke to user in-person at APS.

Github Actions

Previous:

• ND in progress of migrating CI to github actions
• Getting XFEL-regression working again main TODO, will be able to have as explicit step (less confusing than current)
• Should relieve pressure on azure pipelines limits, make easier to see what is going on
• I like Github Actions! Much nicer to use than Azure/Gitlab

Status: Done.

• Now done including XFEL-regression. No reported issues with new actions.
• Aaron would still like to have a walkthrough of new actions scripts at some point

Handle the case of very few reflections for a given dataset in weighted cosym analysis

Previous:

• https://github.com/dials/dials/pull/2741
• Broke xfel-regression
• Test actually failed in CI, but showed green tick -> Should fix with Github Actions rewrite
• James will run tests and check fix

Status: Done

• James investigated failing test, suggested fix.
• Aaron updated reference, issued fixed.

MCD Outlier Rejection

Previous:

• AB PR: https://github.com/dials/dials/pull/2676
• Relatively happy with, touches xia2.ssx so wanted JBE to look over
• Adds lots of new settings, does not change current defaults but thinks it should do
• JBE will have a look at it
  • JBE Has had a look! Has feedback/suggestions
  • AB will have a look, check next time

Status: Still outstanding

Fast-feedback indexer

Previous:

• Adding the fast-feedback-indexer to DIALS
• https://github.com/dials/dials/pull/2717
  • Conda-forge upstream PR still in progress, HC currently on leave.

Status: Still outstanding, DGW will try to chase.

Status of James’ HDF5 work

Previous:

• JBE: H5 on-disk experiment: review practicalities of using this & welcome people looking at James’s work above. round trip example files could be useful - exists in dials.export.
• https://github.com/dials/dials/pull/2677
• Graeme and Nick need to have a look over
  • Had a look this week, a few changes then look again
  • This solves some problems we have been having
• We need a persuasive argument why we are not using NXreflection
• AB thinks should be NXReflections
  • NXReflections is not an application definition; is a base class
  • Does not want to create a new file format
• GW emphasises that should match the internal representation because is not intended as interchange, storage file should match what we want to read/write
• Long discussion without agreement, parties agree to defer until future discussions.

Status:

• JBE done work on this since last time. Now include a NXreflections group that references the data through references and hdf5 regionreferences for multi-dimensional data (eg. miller index), so get NXReflections conforming fully to standard and working dials format exacty how we want.
• Aaron happy now with changes.
• Suggestion from Graeme to add documentation on this as part of PR - dials wiki etc. James to do.
• Call files .refl (users don’t need to know).
• Will not be the default storage format for now once merged in.
• Aside, Aaron notes that we are very close to a relational DB model. Should discuss further in future, point raised often by Herbert? Is it actually useful for us? Main issue of primary key.
• Plan to merge PR next week once final updates made.

Laue-dials tests

Outstanding:

• Rick wanted to run these as part of dials test suite - follow up.

PyCBF on PPC

Previous:

• Derek Mendez queried about getting DIALS build for PPC platform. PyCBF seemed to have issues.
• Autobuild is broken, in process of moving to plain setuptools
• ND: Not had time to continue working on this

Status:

• Graeme now messaging Derek on slack about this. Aaron says we should support if Derek needs this to run on supercomputers at facility/institute.

PR: NXmx: read data_scale_factor

Previous:

• https://github.com/cctbx/dxtbx/pull/756
• Needed nxmx release
  • ND: Make nxmx release and push through to conda-forge.
  • This is done
  • CI errors appear to be legit

Status: done (PR merged).

“Add the ellipsoid profile model as an option in dials.stills_process”

Previous:

• https://github.com/dials/dials/pull/2615

Status:

• waiting on Aaron to review, not pressing so likely not in immediate future.

AOB

Workshops

• Yearly workshop at 6 light sources in US - Aaron wants to teach at two a year. Funds available from R24 for others to teach, including from DIALS UK.

Request for Nick: Please add Central US time to meeting time on agenda.

Other PRs

Aaron dials web-page PRs

https://github.com/dials/dials/pull/2528/ https://github.com/dials/dials/pull/2780/

• 2528 drops sourceforge mailing list. Decided to be ok because better to point to making github issues, less likely that something is missed. Aaron wants link to slack channel on webpage. Graeme says sign up links usually limited to 90 days by default so would need to work out how to do this.

• 2780 Adds R24 to webpage. Discussion on changes. Aaron to update statement about specifics of board rather than ‘coming soon’. Graeme suggests more description of ongoing activities. Aaron thinks should probably add more xfel papers to publications list in due course. Aaron requests more workshops be added to list in future. Aaron needs to merge asap so will address immediate issues and then can address rest of these points.

Graeme workshop bug hunt

• Tqdm chart on import; https://github.com/cctbx/dxtbx/pull/768 Small change, no new depencies, Graeme asks for Nick’s review.

• dials image viewer problems observed at workshop. Needs making better for multi-crystal (e.g. dropdown to select sweep). DGW seeing issues with slider on linux, not seen by others yet. Need to decide on long term future of image viewer. DGW has investigated viewers; says bottleneck is getting slices out of hdf5 - limited to ~10 frames/sec. Aaron notes working group with M Fuchs a couple of year ago, one of work items in R24 is build new image viewer. Likely to be one of Dan Paley’s job’s. Suggestion of Napari being a potential good starting point.

Next meeting

Thursday, November 14th, 4pm (GMT), 8am (PST)