For every method, in the same order: ① its original pretrained backbone, ② our fine-tuned DINO dropped into that method’s pipeline, and ③ the method’s SAE version, and ④ our fine-tuned DINO with the SAE, trained. Frames, resolution, layer and pooling stay fixed within a table. Each scene retrieves its ten nearest scenes from other driving logs, and we score how closely their next 4 s match the query’s.
Not done — interim version. Our method has all four rows (①–④, from four matched training runs, final). DrivoR: the matched “stock DINOv2 + SAE” run is now final (all 25 epochs). DrivoR’s ④ is not shown yet — it is queued for training; its first run was invalid (it auto-resumed a 30-step smoke test’s checkpoint and inherited a one-epoch learning-rate schedule) and was restarted cleanly on 2026-09-29. Drive-JEPA ④ is not started. Rows marked OLD SAE are the earlier hard-deletion SAE, kept only as a reference.
Rows inside one table are directly comparable; rows in different tables are not. ADE is the mean error between the query’s 4 s future and each neighbour’s, in their own t0 ego frames. Δ ADE is paired: the same queries, scored with each backbone, differenced per query against ①, with a bootstrap 95% interval — green when the whole interval is below zero, red when above. Wins is the share of queries where that backbone’s neighbours drive closer to the query than ①’s. The best of ①–③ is highlighted; reference rows below the divider are there to interpret them, not to rank.
Four matched training runs in Mehdi’s trainer and architecture (CLS + 4×6 pooled patch tokens), identical except trunk and SAE: ① stock, ② our fine-tuned DINO, ③ stock + his updated learned-gate SAE on block 11, ④ fine-tuned + the same SAE (trajectory teacher, keep budget 0.5, hard gate at half the run, 8 epochs). The trunk stays frozen; the gate is what trains. Planner validation ADE: ① 0.871 · ② 0.863 · ③ 1.010 · ④ 0.931 m — the SAE costs the stock trunk +0.139 but the fine-tuned trunk only +0.068. Unlike DrivoR’s gate, these gates close live codes: 1531 and 1540 of 3,072, carrying 49% and 53% of code magnitude, with only 52 and 43 of the 100 strongest codes kept — in 8 epochs the budget forced the cut before the teacher could choose it. The fine-tuned dictionary also reconstructs worse (FVU 0.150 vs 0.098).
| backbone | trajectory ADE [95% CI] | Δ ADE vs ① [95% CI] | wins vs ① | speed err | direction /10 | score gap |
|---|---|---|---|---|---|---|
| ① Original backbone: stock DINOv2 ViT-S/14 | 4.65 m [4.54, 4.79] | baseline | — | 2.18 | 8.02 | 0.203 |
| ② Our fine-tuned DINO | 3.63 m [3.52, 3.74] | -1.03 m [-1.16, -0.90] | 66% | 1.70 | 8.56 | 0.197 |
| ③ SAE way: stock + updated SAE — gate sits on block 11, so it cannot reach CLS | 4.65 m [4.53, 4.79] | -0.00 m [-0.01, -0.00] | 1% | 2.17 | 8.02 | 0.203 |
| ④ Our fine-tuned DINO + updated SAE — same: CLS untouched by construction | 3.63 m [3.52, 3.73] | -1.03 m [-1.16, -0.90] | 66% | 1.70 | 8.57 | 0.197 |
| reference rows — to read the three above | ||||||
| ref · OLD SAE (hard deletion, pre-update): stock + sky/nature/structures deleted | 4.38 m [4.27, 4.50] | -0.27 m [-0.38, -0.16] | 52% | 1.99 | 8.00 | 0.204 |
| ref · OLD SAE, reconstruction only (nothing deleted) | 4.68 m [4.56, 4.82] | +0.03 m [-0.02, 0.08] | 47% | 2.19 | 8.10 | 0.205 |
| backbone | trajectory ADE [95% CI] | Δ ADE vs ① [95% CI] | wins vs ① | speed err | direction /10 | score gap |
|---|---|---|---|---|---|---|
| ① Original backbone: stock DINOv2 ViT-S/14 | 4.64 m [4.52, 4.78] | baseline | — | 2.13 | 8.11 | 0.207 |
| ② Our fine-tuned DINO | 3.99 m [3.88, 4.10] | -0.65 m [-0.75, -0.55] | 60% | 1.88 | 8.47 | 0.204 |
| ③ SAE way: stock + Mehdi's updated learned-gate SAE | 4.70 m [4.57, 4.82] | +0.05 m [-0.02, 0.13] | 47% | 2.13 | 8.05 | 0.205 |
| ④ Our fine-tuned DINO + updated SAE | 4.22 m [4.11, 4.33] | -0.43 m [-0.54, -0.32] | 55% | 1.97 | 8.41 | 0.209 |
| reference rows — to read the three above | ||||||
| ref · OLD SAE (hard deletion, pre-update): stock + sky/nature/structures deleted | 4.61 m [4.50, 4.72] | -0.04 m [-0.14, 0.07] | 49% | 2.07 | 8.18 | 0.212 |
| ref · OLD SAE, reconstruction only (nothing deleted) | 4.63 m [4.51, 4.76] | -0.01 m [-0.08, 0.06] | 48% | 2.10 | 8.12 | 0.208 |
DrivoR trains LoRA r32 on DINOv2-S/14-reg4; its base weights are bit-identical to timm’s release, so ① is exactly the pretrained backbone. ② runs our fine-tuned DINO through DrivoR’s own preprocessing. Our DINO has no register tokens, so the stock-DINOv2 reference row separates that architecture difference from the effect of our fine-tune.
| backbone | trajectory ADE [95% CI] | Δ ADE vs ① [95% CI] | wins vs ① | speed err | direction /10 | score gap |
|---|---|---|---|---|---|---|
| ① Original backbone: pretrained DINOv2-S/14-reg4 | 4.51 m [4.39, 4.64] | baseline | — | 2.10 | 8.03 | 0.205 |
| ② Our fine-tuned DINO | 3.69 m [3.59, 3.80] | -0.82 m [-0.94, -0.69] | 63% | 1.73 | 8.58 | 0.201 |
| ③ SAE way: DrivoR + SAE gate | 3.18 m [3.09, 3.27] | -1.33 m [-1.45, -1.20] | 70% | 1.49 | 8.69 | 0.192 |
| reference rows — to read the three above | ||||||
| ref · stock DINOv2 + SAE, trained in DrivoR (matched to ④) (final, epoch 25/25) | 3.25 m [3.16, 3.35] | -1.25 m [-1.38, -1.13] | 69% | 1.55 | 8.72 | 0.198 |
| ref · stock DINOv2 ViT-S/14 (our architecture, not fine-tuned) | 4.56 m [4.44, 4.70] | +0.06 m [-0.02, 0.13] | 48% | 2.15 | 8.00 | 0.205 |
| ref · DrivoR without the SAE | 3.33 m [3.23, 3.44] | -1.17 m [-1.29, -1.05] | 70% | 1.55 | 8.57 | 0.195 |
| backbone | trajectory ADE [95% CI] | Δ ADE vs ① [95% CI] | wins vs ① | speed err | direction /10 | score gap |
|---|---|---|---|---|---|---|
| ① Original backbone: pretrained DINOv2-S/14-reg4 | 4.59 m [4.48, 4.71] | baseline | — | 2.12 | 8.10 | 0.207 |
| ② Our fine-tuned DINO | 3.98 m [3.88, 4.09] | -0.60 m [-0.71, -0.49] | 61% | 1.87 | 8.47 | 0.206 |
| ③ SAE way: DrivoR + SAE gate | 3.04 m [2.96, 3.13] | -1.54 m [-1.66, -1.43] | 75% | 1.43 | 8.75 | 0.193 |
| reference rows — to read the three above | ||||||
| ref · stock DINOv2 + SAE, trained in DrivoR (matched to ④) (final, epoch 25/25) | 3.11 m [3.03, 3.20] | -1.47 m [-1.59, -1.36] | 74% | 1.47 | 8.74 | 0.197 |
| ref · stock DINOv2 ViT-S/14 (our architecture, not fine-tuned) | 4.60 m [4.48, 4.73] | +0.02 m [-0.07, 0.10] | 50% | 2.11 | 8.07 | 0.206 |
| ref · DrivoR without the SAE | 3.53 m [3.43, 3.64] | -1.05 m [-1.15, -0.95] | 70% | 1.61 | 8.51 | 0.203 |
① is Drive-JEPA’s released V-JEPA pretraining (vitl_merge_3dataset_e50.pt). ② is our fine-tuned DINO on the same two frames with the same crop, at 504×252 (DINOv2 needs multiples of 14) and with ImageNet normalisation, which DINOv2 was trained with and V-JEPA was not.
| backbone | trajectory ADE [95% CI] | Δ ADE vs ① [95% CI] | wins vs ① | speed err | direction /10 | score gap |
|---|---|---|---|---|---|---|
| ① Original backbone: pretrained V-JEPA ViT-L (driving video) | 2.80 m [2.73, 2.87] | baseline | — | 1.19 | 8.80 | 0.197 |
| ② Our fine-tuned DINO | 4.05 m [3.94, 4.17] | +1.26 m [1.16, 1.35] | 26% | 1.86 | 8.46 | 0.203 |
| ③ SAE way: Drive-JEPA + SAE gate | 2.22 m [2.16, 2.27] | -0.58 m [-0.64, -0.52] | 68% | 0.94 | 8.91 | 0.186 |
| reference rows — to read the three above | ||||||
| ref · stock DINOv2 ViT-S/14 (our architecture, not fine-tuned) | 4.68 m [4.56, 4.82] | +1.88 m [1.77, 2.00] | 21% | 2.12 | 8.14 | 0.205 |
| ref · Drive-JEPA without the SAE (released) | 2.14 m [2.09, 2.19] | -0.65 m [-0.71, -0.60] | 71% | 0.92 | 8.93 | 0.184 |
For each of the 2,000 queries, count how many of its ten retrieved scenes have a 4 s future within 2 m ADE of the query’s — the gallery’s green border — then average. At least one is the share of queries where any of the ten qualifies. Neighbours are chosen by scene features alone; the trajectory is only used afterwards to grade them.
| backbone | green tiles per query [95% CI] | queries with at least one |
|---|---|---|
| Our method · Backbone alone · CLS token | ||
| ① Original backbone: stock DINOv2 ViT-S/14 | 2.95 / 10 [2.84, 3.07] | 77% |
| ② Our fine-tuned DINO | 3.96 / 10 [3.83, 4.08] | 87% |
| ③ SAE way: stock + updated SAE — gate sits on block 11, so it cannot reach CLS | 2.95 / 10 [2.84, 3.08] | 77% |
| ④ Our fine-tuned DINO + updated SAE — same: CLS untouched by construction | 3.96 / 10 [3.84, 4.09] | 87% |
| ref · OLD SAE (hard deletion, pre-update): stock + sky/nature/structures deleted | 3.16 / 10 [3.05, 3.28] | 81% |
| ref · OLD SAE, reconstruction only (nothing deleted) | 2.96 / 10 [2.85, 3.08] | 76% |
| Our method · Backbone alone · patch mean | ||
| ① Original backbone: stock DINOv2 ViT-S/14 | 3.05 / 10 [2.94, 3.18] | 77% |
| ② Our fine-tuned DINO | 3.59 / 10 [3.47, 3.71] | 85% |
| ③ SAE way: stock + Mehdi's updated learned-gate SAE | 2.99 / 10 [2.87, 3.11] | 76% |
| ④ Our fine-tuned DINO + updated SAE | 3.30 / 10 [3.19, 3.42] | 84% |
| ref · OLD SAE (hard deletion, pre-update): stock + sky/nature/structures deleted | 2.95 / 10 [2.85, 3.06] | 80% |
| ref · OLD SAE, reconstruction only (nothing deleted) | 3.02 / 10 [2.90, 3.14] | 77% |
| DrivoR · Backbone alone · CLS token | ||
| ① Original backbone: pretrained DINOv2-S/14-reg4 | 2.99 / 10 [2.87, 3.12] | 76% |
| ② Our fine-tuned DINO | 3.85 / 10 [3.73, 3.97] | 87% |
| ③ SAE way: DrivoR + SAE gate | 4.42 / 10 [4.29, 4.55] | 88% |
| ref · stock DINOv2 + SAE, trained in DrivoR (matched to ④) (final, epoch 25/25) | 4.37 / 10 [4.24, 4.51] | 88% |
| ref · stock DINOv2 ViT-S/14 (our architecture, not fine-tuned) | 2.95 / 10 [2.84, 3.07] | 75% |
| ref · DrivoR without the SAE | 4.33 / 10 [4.20, 4.47] | 86% |
| DrivoR · Backbone alone · patch mean | ||
| ① Original backbone: pretrained DINOv2-S/14-reg4 | 3.01 / 10 [2.90, 3.15] | 77% |
| ② Our fine-tuned DINO | 3.56 / 10 [3.44, 3.68] | 85% |
| ③ SAE way: DrivoR + SAE gate | 4.61 / 10 [4.48, 4.75] | 89% |
| ref · stock DINOv2 + SAE, trained in DrivoR (matched to ④) (final, epoch 25/25) | 4.49 / 10 [4.36, 4.63] | 89% |
| ref · stock DINOv2 ViT-S/14 (our architecture, not fine-tuned) | 3.04 / 10 [2.92, 3.16] | 76% |
| ref · DrivoR without the SAE | 4.04 / 10 [3.92, 4.18] | 85% |
| Drive-JEPA · Backbone alone · token mean | ||
| ① Original backbone: pretrained V-JEPA ViT-L (driving video) | 4.71 / 10 [4.61, 4.84] | 92% |
| ② Our fine-tuned DINO | 3.51 / 10 [3.40, 3.64] | 83% |
| ③ SAE way: Drive-JEPA + SAE gate | 5.69 / 10 [5.57, 5.82] | 95% |
| ref · stock DINOv2 ViT-S/14 (our architecture, not fine-tuned) | 3.04 / 10 [2.93, 3.17] | 77% |
| ref · Drive-JEPA without the SAE (released) | 5.86 / 10 [5.74, 5.99] | 95% |
| Our method · Planner readout (scene latent) | ||
| ① Stock DINOv2 trunk | 4.30 / 10 [4.17, 4.43] | 88% |
| ② Our fine-tuned DINO trunk | 4.44 / 10 [4.30, 4.57] | 89% |
| ③ Stock trunk + updated SAE | 3.97 / 10 [3.85, 4.10] | 88% |
| ④ Our fine-tuned DINO + updated SAE | 4.26 / 10 [4.13, 4.39] | 89% |
| DrivoR · Planner readout (16 scene tokens) | ||
| DrivoR without the SAE | 4.28 / 10 [4.15, 4.42] | 88% |
| DrivoR + SAE gate | 4.76 / 10 [4.62, 4.90] | 90% |
| ref · stock DINOv2 + SAE, trained in DrivoR (final, epoch 25/25) | 4.83 / 10 [4.70, 4.97] | 90% |
| Drive-JEPA · Planner readout (projector tokens) | ||
| Drive-JEPA without the SAE (released) | 5.92 / 10 [5.81, 6.05] | 96% |
| Drive-JEPA + SAE gate | 5.86 / 10 [5.73, 5.98] | 95% |
| Reference — no backbone pair | ||
| Past motion only (no image) | 5.58 / 10 [5.45, 5.71] | 93% |
| Past motion + command (no image) | 6.50 / 10 [6.38, 6.63] | 97% |
| Our shipped model · readout (+ motion & command) | 8.44 / 10 [8.34, 8.53] | 99% |
| Our shipped model · readout (image only) | 3.92 / 10 [3.79, 4.04] | 86% |
| chance — ten random scenes from other logs | 0.80 / 10 | 52% |
The layer a planner reads is created by training it, so a pretrained backbone or our DINO has nothing to put here without training a new planner. These tables compare only the trained models.
| backbone | trajectory ADE [95% CI] | Δ ADE vs ① [95% CI] | wins vs ① | speed err | direction /10 | score gap |
|---|---|---|---|---|---|---|
| ① Stock DINOv2 trunk | 3.34 m [3.25, 3.44] | baseline | — | 1.56 | 8.78 | 0.194 |
| ② Our fine-tuned DINO trunk | 3.26 m [3.16, 3.36] | -0.09 m [-0.17, -0.01] | 53% | 1.52 | 8.81 | 0.195 |
| ③ Stock trunk + updated SAE | 3.50 m [3.41, 3.59] | +0.16 m [0.08, 0.23] | 43% | 1.62 | 8.71 | 0.199 |
| ④ Our fine-tuned DINO + updated SAE | 3.45 m [3.35, 3.56] | +0.11 m [0.02, 0.19] | 48% | 1.63 | 8.68 | 0.199 |
| backbone | trajectory ADE [95% CI] | Δ ADE vs ① [95% CI] | wins vs ① | speed err | direction /10 | score gap |
|---|---|---|---|---|---|---|
| DrivoR without the SAE | 3.42 m [3.32, 3.52] | baseline | — | 1.60 | 8.62 | 0.195 |
| DrivoR + SAE gate | 3.02 m [2.93, 3.10] | -0.40 m [-0.48, -0.32] | 57% | 1.45 | 8.82 | 0.195 |
| reference rows — to read the three above | ||||||
| ref · stock DINOv2 + SAE, trained in DrivoR (final, epoch 25/25) | 2.90 m [2.82, 2.98] | -0.51 m [-0.59, -0.44] | 61% | 1.38 | 8.82 | 0.191 |
| backbone | trajectory ADE [95% CI] | Δ ADE vs ① [95% CI] | wins vs ① | speed err | direction /10 | score gap |
|---|---|---|---|---|---|---|
| Drive-JEPA without the SAE (released) | 2.11 m [2.06, 2.17] | baseline | — | 0.92 | 8.94 | 0.185 |
| Drive-JEPA + SAE gate | 2.15 m [2.10, 2.20] | +0.04 m [0.00, 0.07] | 46% | 0.94 | 8.91 | 0.188 |
Same queries and scoring, but no matched backbone to pair with, so they are not comparable to the tables above. The controls use no image at all.
| representation | trajectory ADE | speed err | direction /10 | score gap |
|---|---|---|---|---|
| Past motion only (no image) | 2.27 m | 0.71 | 7.53 | 0.203 |
| Past motion + command (no image) | 1.83 m | 0.55 | 10.00 | 0.202 |
| Our shipped model · readout (+ motion & command) | 1.20 m | 0.46 | 9.32 | 0.186 |
| Our shipped model · readout (image only) | 3.62 m | 1.71 | 8.69 | 0.203 |
Choose pipelines to draw. Inside each, ①②③ are stacked in order, with reference rows dimmed beneath them. Under every neighbour, its 4 s future is drawn bold in teal, with the query’s as a faint dashed grey reference, each as 8 waypoints at 0.5 s spacing, with the ADE between them. A green border marks a neighbour whose future is within 2 m ADE of the query’s — one that actually drives the same way. The lateral axis is stretched about 3× so turns stay visible; ADE is unaffected.
| Backbone, within one pipeline original vs method-trained vs +SAE, everything else fixed | yes, headline | the only rows that are directly comparable |
| Backbone-alone vs planner readout same backbone, two layers of the same model | yes | shows whether training changes the backbone or only what sits on it |
| With vs without the SAE gate DrivoR and Drive-JEPA | yes | what a gate removes |
| Trajectory ADE query's 4 s future vs each neighbour's | yes, headline | separates every encoder; paired difference + 95% CI over the same 2,000 queries |
| Win rate share of queries where this backbone's ADE is lower than the original's | yes | paired, so it ignores scene difficulty |
| Speed error |speed difference| at t0 | yes | tracks ADE, coarser |
| Direction driving command matches | weak | saturates near 8-9 of 10 |
| Score gap |PDMS difference| to neighbours | weak | 0.19-0.21 for all vs 0.24 chance |
| Mean cosine similarity of the retrieved ten | no | each space has its own scale |
| Rows across pipelines e.g. DrivoR CLS vs Drive-JEPA token mean | no | different frames, resolution, layer and pooling -- listed only as orientation |
| Our shipped model's readout no matched backbone pair exists for it | reference | kept below the pairs, labelled as such |
| navhard frames where both SAE papers report their gains | yes, next | navtest only here |
| Neighbour gate failures does a neighbour fail the same PDMS gate | yes | sub-scores exist |
Our method. Trunk rows: DINOv2 ViT-S/14 on the cached 504×280 t0 frame. Readout rows: the scene aggregator of the trunk 2×2 models (DINOWaypointCLS, navtrain, 4,000 steps, seed 42), cell C with the stock trunk and cell D with the fine-tuned one — cell D’s trunk is bit-identical to the fine-tuned trunk in the rows above. Past motion joins only after the readout, so the readout is image-only.
DrivoR. Rebuilt from the shared checkpoints; the original backbone is the same LoRA model with the LoRA branch set to zero, verified against timm’s release (174/175 tensors identical, the last being timm’s own pos_embed resample, equal bit-for-bit). The +SAE model applies its gate inside the forward pass at blocks[10], as it runs. Readout = mean of the 16 scene tokens the planner reads.
Drive-JEPA. Original = vitl_merge_3dataset_e50.pt from the Drive-JEPA release; released and +SAE models from the shared checkpoints; the +SAE gate is applied on the projector as it runs.
Queries: 2,000 evenly spaced over the corpus; 30 display queries stratified by direction and speed. Neighbours always from other driving logs. Generated 2026-09-30T05:06:28Z.