Shifts Challenge 2022 begins!
The Shifts Project is Happy to announce the launch of the Shifts Challenge 2022! This will be the second iteration of the Shifts Challenge, which successfully debuted last year at NeurIPS 2021.
This year, the competition will consist of two new tracks, each corresponding to a high-risk application affected by distributional shift: marine cargo vessel power estimation and White Matter Multiple Sclerosis lesion segmentation in 3D Magnetic Resonance Imaging of the brain. Each application is associated with a unique data modality and predictive task. Both tasks are strongly affected by distributional shift, have strict requirements on robustness and are of high social relevance. The competition, running from September 15th until April 1st, will comprise two-phases. In the first phase, participants are asked to focus on model development and training. In the second phase, they will be able to evaluate their models on heldout datasets. Final rankings will be decided by position on the evaluation-set leaderboard.
Maritime transport delivers around 90% of the world’s traded goods, emitting almost a billion tonnes of CO2 annually and increasing. Energy consumption varies greatly depending on the chosen routes, speeds, operation and maintenance of ships. The complex underlying relationships between are not fully known or taken into account at the time these decisions are made, leading to significant fuel waste. Weather and sea conditions that affect vessel power consumption are highly variable based on seasonality and geographical location and cannot all be fully measured, further complicating planning of vessel operations. Training accurate power consumption estimation models for route optimisation can help significantly reduce costs and emissions. However, significant distributional shifts can be expected to occur between the real use cases of models and the data used to train and evaluate them. Inaccurate power prediction and the resultant errors in fuel planning and route optimisation can be considerably costly and potentially hazardous. Thus, the development of uncertainty-aware and robust models is essential to enable the effective deployment of this technology to reduce the carbon footprint of global supply chains.
Multiple Sclerosis (MS) is a debilitating, incurable and progressive disorder of the central nervous system that negatively impacts an individual’s quality of life. Estimates claim that every five minutes a person is diagnosed with MS, reaching 2.8 million cases in 2020 and that MS is two-to-four times more prevalent in women than in men. Magnetic Resonance Imaging (MRI) plays a crucial role in the disease diagnosis and follow-up. However, manual annotations are expensive, time-consuming, and prone to errors. Automatic, ML-based methods may introduce objectivity and labor efficiency in the tracking of MS lesions. However, the availability of training images for machine learning methods is limited. No publicly available dataset fully describes the heterogeneity of the pathology. Furthermore, changes in MRI scanner vendors, configurations, imaging software and medical personnel leads to significant variability in the imaging process. These differences, which are exacerbated when considering images collected from multiple medical centers, represent a significant distributional shift for ML-based MS detection models, reducing the applicability and robustness of automated models in real-world conditions. The development of robust MS lesion segmentation models is necessary to bring improvements in the quality and throughput of the medical care available to the growing number of MS patients.
The Shifts Challenge 2.0 aims to draw attention to these two tasks and drive the development of robust, uncertainty-aware machine learning methods for these tasks. To join our challenge, register on Grand Challenge, join the Discord, follow us on Twitter and sign up to our mailing list.
