Weak supervision for knee MRI: 58 gold-labelled studies, 4,407 reports
An ongoing entry to the 2026 RSNA Knee Abnormality Detection competition on Kaggle: radiology reports teach an EfficientNetV2 and DINOv2 ensemble that must read knee MRI from the images alone.
- Reportsevery training study has a free-text radiology report
- Labelsonly 58 of 4,407 studies carry the 12 gold labels, so labels are read from the reports
- Image modela 2.5D EfficientNetV2 and DINOv2 ensemble, 12 outputs; evolved from a timm and BiGRU baseline
- Image-only inferenceno report at test time; read against 58 held-out gold studies
The underlying scans are patient data and cannot be shown, so this page shows aggregate results and model outputs only.
The hypothesis
The 2026 RSNA Knee Abnormality Detection competition is a research code competition on Kaggle: multimodal knee-MRI classification of a defined set of clinically important abnormalities. It is still running, so this page reports a method, its status and my own held-out validation. There is no medal or leaderboard position to report.
My entry rests on a hypothesis about labels. Only 58 of the 4,407 training studies carry the 12 gold labels, which is far too few to train an image model on directly. But every training study comes with a free-text radiology report. If the reports can be turned into labels, the image model can learn from thousands of studies instead of dozens, and still run on the images alone when it is scored. The 58 gold studies are then worth more as a test than as training data.
What I did
- Preprocessed about 500 GB of raw DICOM in shards on Kaggle, writing windowed, rescaled uint8 arrays at 384 pixels, and synced the result to a GPU cluster.
- Laid the pipeline out as four stages: report, labels, image model, image-only inference.
- Built a first baseline over 2.5D slice stacks: a timm slice encoder feeding a BiGRU, then attention pooling, series aggregation and 12 logits, trained with DistributedDataParallel on two Quadro GV100 GPUs with binary cross-entropy, mixed precision, a cosine schedule and AUC logged per label.
- Moved from that baseline to the current model, a 2.5D ensemble of EfficientNetV2 and DINOv2, trained on 3× A100 80GB over more than 300 GPU-hours.
- Split the data with GroupKFold at patient level, so validation never sees a patient the model trained on.
- Held out the 58 image-verified gold studies and read the ensemble against them, label by label. The readout below is that comparison.
Decisions that mattered
- The report is the teacher
- With 58 gold-labelled studies the supervised route is too thin to trust. Every study has a report, so the report supplies the training signal and the image model is the student. At inference the report is gone: the model that is scored sees the MRI and nothing else.
- Preprocess once, in shards
- About 500 GB of raw DICOM is windowed and rescaled to uint8 arrays at 384 pixels, shard by shard on Kaggle, and the result is synced to the GPU cluster. The training loop reads the prepared arrays.
- Start with a baseline, then replace it
- The first model was a timm slice encoder with a BiGRU reading the slices in order. It proved the data path and the per-label logging. The approach then evolved into an ensemble of two different image encoders, EfficientNetV2 and DINOv2, still over 2.5D slice stacks.
- Split by patient, before anything else
- GroupKFold at patient level, the same discipline as in my MSc work on prostate MRI. A split that lets one patient sit on both sides makes every later number unreliable, so it was settled first.
- Look at every label, study by study
- There are 12 labels and very few gold examples of each. A single averaged score would hide a label the model has not learned, so each label is tracked on its own and the readout shows whole studies, misses included.
- 2.5D stacksneighbouring slices grouped from the 384 px uint8 shards
- Slice encodera timm backbone encodes each stack
- BiGRUreads the encoded slices in order, in both directions
- Attention poolingweighs the slices within a series
- Series aggregationcombines the series of one study
- 12 logitsone per gold label, trained with binary cross-entropy
What the model sees in a knee
A 2.5D EfficientNetV2 and DINOv2 ensemble, read against 58 image-verified, held-out gold studies. For each study the ensemble gives a confidence for each of the twelve abnormalities; at or above 0.50 it calls the abnormality present. Three studies are set out below against the radiologist’s ground truth.
Case A. Advanced degenerative knee
11 of 12 labels correct at the 0.50 threshold
Case B. Mixed presentation, a hard read
9 of 12 labels correct at the 0.50 threshold
Case C. Near-normal, isolated meniscus tear
12 of 12 labels correct at the 0.50 threshold
- Across all 58 gold studies
- The ensemble calls 83.5% of label cells correctly at 0.50. It slips most on synovitis and effusion.
- Compute
- 3× A100 80GB, more than 300 GPU-hours.
- Standing
- The competition is still running. This is held-out validation on my side: no leaderboard position, no medal.
Evidence so far
The competition is ongoing and I publish no leaderboard position or medal. What can be stated is my own held-out validation and what exists.
| Part | State, September 2026 | Context |
|---|---|---|
| Data engineering | done | about 500 GB of raw DICOM, preprocessed in shards on Kaggle to uint8 at 384 px, synced to a GPU cluster |
| Baseline model | built, superseded | timm slice encoder, BiGRU, attention pooling, series aggregation, 12 logits; two Quadro GV100 GPUs |
| Current model | trained | 2.5D EfficientNetV2 and DINOv2 ensemble; 3× A100 80GB, more than 300 GPU-hours |
| Held-out validation | 83.5% of label cells correct | 58 image-verified gold studies, 12 labels each, threshold 0.50; weakest on synovitis and effusion |
| Leaderboard result | none | competition still running; no position, score or medal is claimed |
Limitations and status
Fifty-eight studies is a small test set, and the three cases in the readout are illustrations, not a sample. Label-cell accuracy at a fixed threshold is a blunt measure. Most labels are absent in most knees, so a model earns a good share of its cells by saying no, and 83.5% should be read with that in mind. The ensemble slips most on synovitis and effusion. None of this is a competition score: the competition is still running, and there is no leaderboard position or medal to report.
What I learned
Weak supervision moves the difficulty from the images to the reports.
The question stops being how to train on 58 studies and becomes how far a label read from a report can be believed. The data engineering had to come first: until about 500 GB of DICOM is in a form the GPUs can consume, there is no model to discuss. The second lesson carried over from the MSc: the split comes first, because nothing measured afterwards can be trusted without it. The third is that a model is easier to judge one study at a time than through one averaged number.
Competition and profile
The competition page on Kaggle (opens in a new tab) describes the task and the data. My Kaggle profile (opens in a new tab) is public. The training repository is private.
Inspect
External links open in a new tab.