The “Shifts” project launches!

We are pleased to announce the launch of the Shifts Project — an international collaboration of academic and industrial researchers dedicated to studying distributional shift in real-world applications.

Distributional shift, or the mismatch between training and deployment conditions, is one of the biggest obstacles to the wide-spread adoption of and impact of AI across all industries. Distributional shift is ubiquitous in real-world ML applications and can cause a degradation in model performance. This is especially critical in applications with strict safety requirements, such as autonomous driving and medical diagnostics.

Ideally, machine learning models should perform reliably across a broad range of conditions and, if they fail to do so, should indicate that via estimates of uncertainty. The lack of large and diverse datasets with examples of ‘in-the-wild’ distributional shifts from a variety of real-world applications makes it difficult to develop such models. The primary obstacle that needs to be overcome to obtain such data is the limited communication between researchers studying distributional shift and practitioners working on applications affected by distributional shift.

The Shifts Project has three primary goals. First, we wish to build a cross-disciplinary international community, bringing together researchers who study distributional shift and practitioners who work on tasks affected by distributional shift. Second, we aim to collect new datasets sourced from industry that feature ‘in-the-wild’ distributional shift and make them freely available. This will enable better research in the area with outcomes that can be directly applied in-production. Finally, we will be organizing events, challenges and workshops to popularize these new datasets and raise awareness of the problem of distributional shift among the general ML community.

Currently the Shift Project members are the organizing team of researchers from the Universities of Basel, Lausanne, HES-SO Valais and Cambridge, as well as DeepSea and Yandex Research. We hope that this list will grow over time as the community grows and expands. Our desire is that the Shifts Project evolves into a self-sustaining community of researchers and ML practitioners who will drive the development of robust, reliable and safe ML systems.

Our first major event will be the Shifts Challenge 2022, which will last from September 15th to April 1st 2023 — find out more here! We are also planning a host of other events later on throughout the year. To join our community and stay tuned — follow us on Twitter, join our mailing list and drop by on Discord. If you would like to contribute, don’t hesitate to get in touch!

Sincerely,
The Shifts Project