DIALS core meeting 2020-09-03
- ND: rewrite contribution guidelines (as PR) to include towncrier/newsfragments
- MG: make Python 3.8 the new default version (→ #1373)
- MG: remove DIALS 1.14 tutorials from website (→ #1359)
Action: DWP will have a look over them and email comments to ND
Overall architecture discussion
full conda or alternatives?
- move from libtbx dispatchers to normal python dispatchers
- ND: I don’t see this possible while we are using libtbx/scons
- allow regular python commands access to the libtbx-modules namespaces
- bigger issue, needs more discussion and figuring out how to do this and how to migrate
- Should we remove the DIALS installer from cctbx bootstrap?
- We don’t advertise or test or use cctbx bootstrap any more. Anyone using it may end up with an unsupported environment.
- When cctbx becomes a conda-forge installable package then there is strong interest from the DIALS side to use that
- GW: Can we have a proposal that includes what the impact, benefits, and costs are.
- Action: MG & ND to do that
Following from the overall architecture discussion: How should a new installer look like?
stable master branch
- Report result from 1+ month evaluation on xia2
- Actual value was observed in xia2
- How would we distinguish with pull requests that should be reviewed and pull requests that can be merged as soon as the tests have passed?
- MG: labels may be an option? GitHub plans to introduce an auto-merge feature for pull requests later this year. We could probably do this right now using github actions.
- GW: pull requests can be moved to draft status to express “hang fire, I would like to look at”
- MG: A label saying “merge as soon as tests pass” may alleviate concerns from ND and myself about pull requests with a very short shelf-life
- ASB: Sounds like something that should be mentioned in
- ASB: cctbx is running CI jobs in branches in batch mode. May want to consider that for DIALS/dxtbx
- Fundamentally it’s possible that tests fail because of an upstream repository change (eg. cctbx/scitbx/constants breaking xia2)
- May have to monitor this. This could be resolved by using semantically versioned upstream releases (for cctbx_project) and stable build mechanism (for dxtbx) but comes with other drawbacks.
- Agreement: update
CONTRIBUTINGto include a soft-imposition in a (separate) pull request.
- Action for ND
dxtbx json/msgpack performance
(not discussed beyond adding the action below)
ASB reports json being problematic when importing e.g. 50,000 experiments, as everything is stored in a single file, so everything needs to be read at once.
- GW: cf. Load before heat death (dxtbx#118). At the core of it are bad design choices in dxtbx.
- Future discussion topics:
- treatment of large serial datasets
- using an HDF5 backend for reflection files
- ASB: I want
check_formatto go away.
- GW: discuss in a separate meeting early US, late UK time
- Action GW: set up this meeting
What do we want to do?
DIALS documentation build
Should the DIALS documentation build outside a cctbx environment?
- Currently you can only build the DIALS documentation inside a cctbx environment. This means a remote documentation build needs to build cctbx first, so we can’t easily run the documentation build on every commit. When things break they are difficult to fix. We can’t use readthedocs.
- MG: I believe the main obstacles are three cctbx Sphinx plugins that may need to be extracted or removed from DIALS documentation, and some phil parsing logic.
master branch →
- Nothing much will happen on this any time soon - either way we’ll wait for GitHub support on this one
- General assent that this is useful
- from 1st of October new GitHub repositories will have
mainas the default
- MG: Suggest turning this into an issue and removing from agenda until GitHub provides a migration path for existing repositories.
Any other business
- We are only referencing
cbflib_adaptbxhas a MSVC-ancient-dependency. Could copy the functions into
dxtbx_extand remove the dependency.
- We are only referencing
September 17th, 4pm UK time, 8am PDT.