DIALS core meeting 2025-02-13

Previous Actions

• ND Investigate getting psana tests running on the DIALS xfel-regression testing
• ND: Move dials.diamond.ac.uk to a redirect to dials.github.io

Agenda

Welcome Yang Ha as a new member of Aaron’s team for a six month period.

Python 3.10 and psana

• psana repo: https://github.com/lcls-psana/psana
• cctbx environment for running: https://github.com/cctbx/cctbx_project/tree/master/xfel/conda_envs
• Azure yml: https://github.com/cctbx/cctbx_project/blob/master/.azure-pipelines/xfel/unix-psana-build.yml
• Request to build on anaconda channel lcls-i have been made, undetermined timescales before this is built for python 3.10
• Wait until 3.10 version is available, add to xfel-regression tests, then address long-term situation.
• Modern builds now! Up to 3.11, 3.12 has API issues
• xtc2 daq2 psana2 will be coming, xfel will need to handle these at some point
• See above action item regarding integrating into CI tests, this should be relatively simple now.
• RESOLUTION: posted a build for 3.11, Aaron & co could remove pins - now in good shape. Had a good meeting with psana devs and asked if they could please continue to support new versions. xfel-regression tests still need to test “psana stuff” - tests for formatxtc in particular

“FormatXTC: add wavelength_fallback parameter”

• https://github.com/cctbx/dxtbx/pull/781
• Was added as PR to dials/dxtbx back in March ‘24
• I moved it across to cctbx/dxtbx
• Still wanted?
• Set to automerge when tests pass
• merged, can close out item

Aaron dials web-page PRs

• Aaron wants link to slack channel on webpage
  • Link on front of DIALS README still works, should add this
  • ND: Action: Replace dials-support link with slack link
  • GW to review https://github.com/dials/dials/pull/2843

General README overhaul

• New grants not listed!
  • Probably an Aaron action - done
• README somewhat crusty and out of date
• Contact link to slack should be more prominent
• Action: tidy up

Github Actions

• Aaron would still like to have a walkthrough of new actions scripts at some point
  • Reserve back half of one of these at some point
  • Billy talking github actions too, so now is good

MCD Outlier Rejection

Previous:

• AB PR: https://github.com/dials/dials/pull/2676
• Relatively happy with, touches xia2.ssx so wanted JBE to look over
• Adds lots of new settings, does not change current defaults but thinks it should do
• JBE will have a look at it
  • JBE Has had a look! Has feedback/suggestions https://github.com/dials/dials/pull/2750
  • AB will have a look
• Probably April
• Drop for now, pick up again in future

“Add the ellipsoid profile model as an option in dials.stills_process”

• https://github.com/dials/dials/pull/2615
• Status: waiting on Aaron to review, not pressing so likely not in immediate future.
• Probably April
• Drop for now, pick up again in future

Collaboration Meeting?!

• Sleep this item until End of Feb
• Graeme requested to add
• AB: Dials workshop in association with a user meeting? Two days workshop two days DIALS perhaps? Discuss!
• GW: Kick off conversation in dials-support about this
  • Discussion started
  • Could do something at Cornell?
  • R24 team: ~4 people
  • UK team: 10
  • hosting in UK could make more sense?
  • Dan P doing CHESS / DIALS @ user meeting early June

AOB

GM/CA workshop

Dates: June 23rd through 30th, 2025 (on-site), June 2nd and 16th (virtual) Location: Advanced Photon Source, Argonne National Laboratory, Argonne (near Chicago), Illinois, USA

Someone from DIALS-R24 to help out? 23-26 data processing stuff?

Dials-support stuff

• Aaron wants to deprecate sourceforge dials-support and move to something with an archive
• TODO: Nick/Graeme work out how to set up on Jiscmail
• Aaron wants a Dials User Group!
  • Name the new mailing list this
  • ND/GW do this for next time

Metrics

• Checked PDB, very little proportion of datasets are reported as being DIALS
• ND remembers this discussion at cross-developer meetings from several years ago (GRC?) and metadata gets often lost across processing
• Irakli is working on properly generating metadata for deposition
• Aaron has a new person starting who will also be interested in this
• dials.report_usage call_home=true id=”diamond mx”
• dials.report_usage call_home=true id=”graeme@ne-cat.edu”
• dials.report_usage call_home=false
• https://github.com/dials/dials/issues/2844
• hosting: LBL can host an end point

James found a Bug

Issue here: https://github.com/dials/dials/issues/2851

Deserves more thought before presenting properly but looks to be overestimating the parameters - which is a biggie for MX processing for mosaic samples

AOB more

Diffrview is very awesome - could do with seeing.

PSANA and FormatXTC

This came up in a meeting again. Aaron feels bad that this isn’t usable for xia2.ssx. Cannot do dials.image_viewer on xtc stream, because stream not random access. Do have some offsets to allow random access to data. dials.import has to read the whole file to get all the metadata, to get the wavelength per shot so has to stream the whole data set. 1,500 image stream import took 30s to import, this is much smaller than normal ones. Fix by switching to image sequence… but won’t solve the wavelength-per-image problem. Would be good to have an answer here. On HDF5 file it is much easier because reading the data are structured. FormatXTC can be created in different modes… idx mode vs. some other mode (smd): the latter pulls the metadata from the small.data file. Why do we need to care? Fold it into format class.

JUNGFRAU gain dependent on wavelenth, because pixels in keV. In dials.stills_process just having a per-image gain. May still be wrong in the integrate step… JF going to MFX over the next 6 mo, will be interesting to find out what happening there. LCLSII is MHz source => they have problems coming 🎉

Next meeting

Thursday, February 27th, 4pm (GMT), 8am (PST), 10am (CST)