Challenges

This challenge will be the second iteration of Shifts, which successfully debuted last year at NeurIPS 2021. This year, the competition will consist of two new tracks, each corresponding to a high-risk application affected by distributional shift: marine cargo vessel power estimation and White Matter Multiple Sclerosis lesion segmentation in 3D Magnetic Resonance Imaging of the brain. Each application is associated with a unique data modality and predictive task. Both tasks are strongly affected by distributional shift and have strict requirements on robustness. Furthermore, both applications are socially relevant: the former in terms of lowering carbon emissions of supply chains and the latter in terms of alleviating a widespread, incurable and degenerative illness.

In 2021, the first Shifts Challenge debuted at NeurIPS. The challenge consisted of three tracks, each corresponding to a particular data modality and task: tabular weather prediction data, machine translation data and self-driving car (SDC) for the task of vehicle motion prediction. These data modalities and tasks are affected by distributional shift and pose interesting challenges with respect to uncertainty estimation. Fifty participants from 13 countries took part in the development phase. From these 50 participants, 17 teams and individual participants later took part in the evaluation phase of the Shifts Challenge. At NeurIPS 2021, we ran an online Shifts Challenge breakout workshop and announced the winners, who later presented their solutions at the poster session.