DACE: Dense Flow from Adaptive Correspondence (NeurIPS 2026)

TL;DR Optical flow methods estimate motion at fixed grid points. DACE learns which pixels to match together: it groups pixels into adaptive tokens, matches the tokens across frames, and passes their motion back to every pixel. The flow is sharper at motion boundaries and more robust where no match is visible.

2.10/3.11

Sintel Final / KITTI EPE

lowest zero-shot error among published methods, with fewer MACs than most

15–19%

lower error where no match is visible

than WAFT, zero-shot on Sintel and KITTI

9–15%

lower error near motion boundaries

than WAFT, even though WAFT matches a 2× finer grid

6 / 6

metrics better than fixed grids

at DACE’s resolution and 2× finer

Abstract

Dense optical flow estimates where every pixel moves between two images. It must be precise and robust. Both properties depend on two choices: which units to match, and what evidence to match them with. Small units localize precisely but carry weak evidence. Large units carry richer evidence but can mix motions across boundaries. Recent work has strengthened the evidence with better features, cost volumes, attention, and generative priors. The unit, however, is still a fixed grid point. We propose DACE: Dense flow from Adaptive CorrespondencE, which learns the units while keeping the evidence dense. A differentiable assignment groups pixels into tokens, builds their descriptors from dense image features, and maps token motion back to pixels as dense flow. Tokens can be large where motion is coherent and small near boundaries and fine detail. Grouping and matching are learned jointly from flow supervision: grouping makes matching easier, and matching shapes grouping. DACE achieves the lowest zero-shot Sintel Final and KITTI endpoint errors among published methods, with fewer MACs than most. Controlled comparisons with fixed grids show that the gain comes from the adaptive units, which make DACE both more precise near motion boundaries and more robust where no match is visible, such as at occluded or out-of-frame pixels.

The idea

Improve the unit, not just the evidence

Flow methods match units between two frames using image evidence. For years, progress has come from better evidence while the unit stayed a fixed grid point. DACE changes the unit.

fixed grid cells

Prior work · 2018–2025

Better evidence

Match the same fixed grid points with stronger evidence.

2018PWC-Net cost volumes
2020RAFT all-pairs correlation
2022GMFlow, FlowFormer transformers
2023DDVM diffusion priors
2025WAFT feature warping

adaptive tokens

This work · 2026

Better units

Learn which pixels to match together, and keep the evidence dense.

Small tokens at motion boundaries: precise
Large tokens where motion is coherent: robust
Grouping is learned from motion, jointly with matching

An idealized scene: a faintly textured object moves right over a still background, and part of it leaves the frame. Hover over or tap a unit to see where it is matched.

Examples

Recovering motion where no match is visible, preserving fine structure

Zero-shot predictions on Sintel and KITTI, after training on synthetic data. WAFT matches a 2× finer fixed grid; DACE matches adaptive tokens.

Pixels with no visible match in frame 2 — No visible match in frame 2

WAFT DACE (ours)

Drag to compare WAFT and DACE. Hover to magnify; the panels above mark the same point.

How to read these images

Hue gives the direction of motion and color intensity its speed; white means no motion. The flow maps of one example share a color scale. In the no-visible-match map, blue marks pixels that are occluded or leave the frame, gray the other labeled pixels, and black pixels without labels. Black in the ground truth also marks missing labels; KITTI's sparse labels are enlarged for display.

Method

Group pixels into tokens, match the tokens, broadcast motion back

Every step is differentiable, so grouping and matching are learned together, end to end.

DACE pipeline: feature extraction, pixel memberships, token descriptors, token matching and refinement, and broadcasting back to pixels

Soft memberships decide what each token covers

Tokens sit on a coarse lattice, but the image decides which pixels each token covers. Each pixel \(x\) belongs softly to the nine tokens around it, with weights from a Gaussian it predicts (mean \(\mu(x)\), inverse covariance \(G(x)\)) over the anchor offsets \(\delta_i\):

\[ a_i(x) = -\tfrac12\big(\delta_i-\mu(x)\big)^{\!\top} G(x)\,\big(\delta_i-\mu(x)\big),\qquad A_i(x)=\frac{\exp a_i(x)}{\sum_{j=1}^{9}\exp a_j(x)}. \]

Try it: one pixel's memberships

Drag the orange point \(\mu(x)\) to move the Gaussian.

Spread Elongation Orientation

How spread out the memberships are

one tokenshared by many

Gaussian scores over the nine neighboring anchors define the memberships — **Memberships.** Gaussian scores over the nine nearest anchors, normalized.

Pixel-to-token pooling — **Pooling.** A token's descriptor is the membership-weighted mean of its pixels' features.

Token-to-pixel broadcasting — **Broadcasting.** A pixel's motion blends its tokens' motions with the same memberships.

Because a token passes its motion to all of its pixels, a pixel with no visible match inherits motion from the visible part of its token.

Grouping is learned from motion alone

No segmentation labels or appearance losses. Three losses on ground-truth flow train the whole model:

Token motion

Each token predicts its motion, supervised by the ground-truth flow pooled over its pixels.

Assignment

Each pixel learns to favor the token whose motion best explains its own, which sharpens grouping at motion boundaries.

Flow consistency

Ground-truth flow pooled into tokens and broadcast back should reproduce the original flow, so each token keeps to one motion.

Benchmarks

Lowest zero-shot endpoint error on Sintel Final and KITTI

Every model is trained on the synthetic FlyingChairs and FlyingThings3D datasets, then tested on Sintel and KITTI, which it has never seen.

Error versus compute

Lower is better. Left: error versus compute for one 540×960 image pair; the dashed line links the most accurate method at each budget. Right: all methods, ranked.

Show the numbers

Precise and robust at once

Pixels with no visible match are only 6–14% of all pixels but cause about half of WAFT's error. On KITTI, DACE has the lowest error on both axes. On Sintel Final, it is close to SAMFlow, which needs 12× the compute. Methods with public checkpoints trained on the same data.

Why adaptive

The gain comes from the adaptive units

To isolate the units, we compare DACE with fixed-grid models that share everything else.

The setup: the same model with different units

WAFTa 2× finer fixed grid, one unit per 2×2 pixels

Fixed grid, same resolutionone unit per 4×4 pixels+ contextualization

DACEtokens on the same lattice, shaped by the image+ contextualization

Contextualizationeach unit also reads the dense features in the dashed window around it

All models share the backbone, its dense features (one per 2×2 pixels), matching by feature warping and refinement, and the training data and recipe.

Adaptive tokens have lower error than both fixed grids on every metric

lower is better

WAFT, 2× finer gridFixed grid, same resolutionDACE

Error of the three models in the setup above. DACE is lowest on all six metrics.

Show the numbers

Adaptive tokens gain more from the same contextualization

lower is better

Fixed grid, same resolutionDACEwithout → with contextualization

Error on pixels with no visible match, without and with the same contextualization; the numbers give the change. All panels share one scale. DACE’s error drops more on every dataset.

Conclusion

DACE learns the units of correspondence. Matching adaptive tokens instead of fixed grid points makes optical flow more precise at motion boundaries and more robust where no match is visible. Learning what to match complements learning how to match it.

Citation

@inproceedings{hsieh2026dace,
  title     = {Dense Flow from Adaptive Correspondence},
  author    = {Hsieh, Cheng-Lin and Yu, Stella X.},
  booktitle = {Advances in Neural Information Processing Systems (NeurIPS)},
  year      = {2026}
}

Acknowledgements

This project was supported in part by NSF grant 2313151, with additional computing resources provided through NAIRR Pilot CIS240421 and CIS240431. The authors thank SpixelFCN and WAFT for sharing their code.