View on GitHub

DIALS // knowledgebase

community resources, howtos, meeting minutes

DIALS core meeting 2020-09-17


Previous Actions


In this meeting we will only discuss the agenda items on typing and dxtbx json/msgpack performance.

Typing: mypy

This is a pre-commit together with a Github action (#1357). Mypy only checks explicit type declarations. To deploy this we need to do minor cleanups as some existing type declarations are wrong. At an absolute minimum this would guarantee that all type declarations we make are correct.

The mypy action checks the entire codebase, but only flags issues where we have conflicting type declaration/usage. The pre-commit hook only checks modified files.

This can be added relatively quickly, existing warnings must be addressed before enabling mypy type checking.

Discussion outcome:

Typing: pytype

Separately from mypy we can also add a pytype action (#1364). This does do type inference, so can go beyond (far beyond) existing type declarations. However it only covers a smaller part of the codebase, as it will not process any file that depends directly or indirectly on any file that contains pytype warnings/errors. Currently in DIALS this currently excludes 75% of all python files, but as we start addressing issues this percentage is expected to fall.

Pytype is also much more expensive to run, so I would not recommend running this as a pre-commit hook. I believe this could be very useful as a pull request GitHub action though. I have implemented a proof-of-concept action that only fails whenever new errors are introduced.

This can be added straightaway, no code changes required.

Discussion outcome:

dxtbx json/msgpack performance

ASB reports json being problematic when importing e.g. 50,000 experiments, as everything is stored in a single file, so everything needs to be read at once.

Discussion outcomes:

DIALS documentation build

(to be discussed in the following meeting)

Should the DIALS documentation build outside a cctbx environment?

renaming master branch → main

(to be discussed in the following meeting)

cbflib_adaptbx dependency

(to be discussed in the following meeting)

We are only referencing compress and uncompress functions, and cbflib_adaptbx has a MSVC-ancient-dependency. We could copy the functions into dxtbx_ext and remove the dependency.

Any other business

(to be discussed in the following meeting)

Next meeting

October 1st, 4pm UK time, 8am PDT.