Flow matching & rectified flow

At 3 a.m., a generation endpoint is spending four seconds making one image. The model is not necessarily slow. It may simply be calling the model 50 times, once for each denoising step. If one call takes 80 milliseconds, that is already 4 seconds before the VAE decode, text encoder, network overhead, and queueing have joined the party.

Diffusion models accept this cost because each small denoising move is easy to learn. But the path from noise to an image is often longer and more winding than it needs to be.

Flow matching trains a neural network to predict a velocity field: the direction and speed that should move a point at a particular location and time. Generation starts with noise, follows that field, and arrives at the data distribution by solving an ordinary differential equation, or ODE.

The key idea is:

Choose a path from noise to data, teach the model the velocity along that path, and make the path simple enough for a numerical solver to follow cheaply.

Start with one point

Take a noise point

n = [0, 0]

and a target data point

y = [3, 4]

A straight path between them is

x_t = (1 − t)n + t y

Here t is time, normalised from 0 to 1. At t = 0, the point is noise; at t = 1, it is data.

Because n is [0, 0] and y is [3, 4]:

• at t = 0.25, x_t = [0.75, 1]
• at t = 0.5, x_t = [1.5, 2]
• at t = 0.75, x_t = [2.25, 3]
• at t = 1, x_t = [3, 4]

The velocity is the derivative of position with respect to time:

u_t = dx_t / dt = y − n = [3, 4]

It is constant, with speed 5: the length of [3, 4].

A tiny Euler solver follows this path:

import numpy as np

noise = np.array([0.0, 0.0])
data = np.array([3.0, 4.0])
velocity = data - noise

x = noise.copy()
steps = 4

for step in range(steps):
    dt = 1.0 / steps
    x = x + dt * velocity
    print(f"step {step + 1}: x={x.round(2)}")

step 1: x=[0.75 1.  ]
step 2: x=[1.5 2. ]
step 3: x=[2.25 3.  ]
step 4: x=[3. 4.]

Euler is exact here because the velocity never changes. Real generation must provide the right velocity at every point in a high-dimensional space. The model does not repeatedly ask, “What noise should I remove next?” It asks, “Which way should this current point move?”

A field, not a memorised line

One training pair gives one line. A generator needs a vector field, assigning a velocity to every location x and time t:

vθ(x, t)

The network receives an intermediate state and time, then returns its direction and speed. Generation solves

dx_t / dt = vθ(x_t, t)

from a random noise sample at t = 0 to t = 1.

The field is shared across the distribution; it does not memorise a separate line for each image. This creates the central difficulty. If different noise-data paths cross at the same location and time but require incompatible next directions, a deterministic network cannot output both. Squared-error regression learns their conditional average, which can produce a curved or unhelpful trajectory.

The pairing matters. Independent noise and data samples are simple and often effective, but their paths may criss-cross. An optimal-transport coupling tries to create lower-cost, more organised pairings. Exact transport is expensive at dataset scale, so implementations use minibatch approximations or other coupling strategies.

How the network learns the field

Training turns transport into supervised regression. For each data example y:

1. Draw noise n and a time t between 0 and 1.
2. Construct the intermediate point x_t.
3. Compute its known target velocity u_t.
4. Regress the network prediction vθ(x_t, t) toward that target.

For the straight path:

x_t = (1 − t)n + t y
u_t = y − n

The loss is

L = E[ ||vθ(x_t, t) − u_t||² ]

The expectation averages over sampled data, noise, and time. The network does not need to observe complete trajectories; it sees random points from many of them.

import torch

def flow_matching_loss(model, data):
    noise = torch.randn_like(data)
    t = torch.rand(data.shape[0], device=data.device)

    broadcast_shape = (data.shape[0],) + (1,) * (data.ndim - 1)
    t_broadcast = t.view(broadcast_shape)

    xt = (1.0 - t_broadcast) * noise + t_broadcast * data
    target_velocity = data - noise

    predicted_velocity = model(xt, t)
    error = predicted_velocity - target_velocity

    return error.square().flatten(1).mean()

This is conditional flow matching: the target is conditional on one sampled noise-data pair. Under squared error, the ideal prediction at a location is

v*(x, t) = E[u_t | x_t = x]

That conditional mean explains both the method and its failure mode. When paths are aligned, it is a useful direction. When incompatible paths collide, it is a compromise.

The straight path is not mandatory. A broader setup defines a probability path, a sequence of distributions changing from noise to data. One common form is

x_t = α(t)y + σ(t)ε

where ε is Gaussian and α(t), σ(t) control the data-noise mixture. Differentiating gives the target velocity. Thus flow matching still has a schedule, endpoints, scaling, and often a time-sampling strategy; it simply trains directly on velocity rather than using traditional diffusion targets.

Sampling is numerical integration

At inference, the network is queried by an ODE solver. Euler uses

xnext = x + Δt · vθ(x, t)

A real field changes with x and t, so Euler assumes its beginning-of-step velocity remains constant. Smaller steps reduce this error but require more network evaluations. Midpoint and Runge–Kutta methods use extra field evaluations to estimate curvature more accurately. Nonuniform or adaptive time grids spend steps where the field changes quickly.

Fewer steps are possible when the learned field is smooth and the chosen path is easy to integrate. They are not guaranteed by the words “flow matching.”

Classifier-free guidance can make the field sharper: it combines conditional and unconditional predictions to increase prompt adherence. A path that integrates easily without guidance may need more steps after strong guidance.

Flow matching and diffusion are neighbours

Both methods transform a simple base distribution, usually Gaussian noise, into data.

Traditional diffusion commonly defines a forward noising SDE, or stochastic differential equation, then reverses it using a score-related model. A score function is the gradient of log density, pointing toward more probable regions. Diffusion also has a deterministic probability-flow ODE with the same time-dependent marginals as the corresponding diffusion process when the score is exact.

The practical distinction is that diffusion usually chooses a noise-injection path and predicts noise, a score-related quantity, or a close equivalent. Flow matching chooses a probability path and predicts its velocity directly. Both need a large network, time conditioning, and a solver. Flow matching can use Gaussian diffusion-like paths, while diffusion can use an ODE.

A straighter trajectory is easier for a coarse solver: it has less curvature to miss and usually fewer conflicting pairwise directions.

What rectified flow adds

Rectified flow methods aim to make transport paths straighter. A common technique is reflow: train an initial flow, use it to generate trajectories and endpoints, then train another flow on those model-generated pairs. Repeating this can reduce bending under the model’s own transport map.

This does not guarantee one-step generation. Mapping a complicated high-dimensional noise distribution to detailed images in one evaluation may sacrifice texture, diversity, or prompt fidelity. Rectification is an efficiency-quality trade-off, not a law that all paths become perfectly straight.

Straightness also depends on coordinates. Image models often transport VAE latents rather than pixels. A path straight in latent space can decode to a visually curved pixel-space change; that is acceptable because the model and loss are defined in latent space.

The modern production pattern

Many image and video generators use this latent recipe:

1. A VAE encodes an image into a latent.
2. Noise is sampled in latent space.
3. A transformer predicts velocity at a chosen time.
4. Conditions such as text are injected.
5. An ODE solver transports the latent toward data.
6. The VAE decodes the result.

Stable Diffusion 3 and FLUX are publicly described as using rectified-flow or flow-matching ideas. Video models use related transport methods because reducing evaluations matters even more for many-frame samples.

The backbone still determines per-call cost. Eight expensive transformer calls can cost more than 30 cheap U-Net calls. Compare end-to-end latency and quality at a fixed budget, not step counts alone.

Failure modes you can actually see

The image is washed out at low step counts

Four steps may produce vague shapes while 16 restore edges and text. The field is changing too quickly for the Euler grid, or guidance has sharpened it. Increase steps, use the checkpoint’s supported solver and schedule, or reduce guidance.

Samples run away from the data

Saturated values or exploding latents often indicate a sign or time-direction error. With t = 0 as noise and t = 1 as data, the straight-path target is data − noise. Also verify the integration interval and the sign of Δt.

Training loss falls, but generations are blurry

The field may be averaging incompatible directions caused by crossing paths. Inspect fixed-time samples or low-dimensional field projections. Improve the coupling, add disambiguating conditioning, use an optimal-transport approximation, or apply rectification. More capacity cannot recover information removed by ambiguous pairing.

The middle looks fine, but endpoints fail

Random time sampling does not ensure equal performance at both endpoints. Track loss by time bins and check time embeddings, endpoint parameterisation, loss weighting, and latent normalisation.

The model works in training but not after swapping the VAE

VAE scaling, latent standard deviation, channel order, and data normalisation define the coordinate system in which the flow was learned. Keep the encoder, scaling, decoder, and flow model as one tested pipeline.

The honest trade-off

Flow matching is most attractive when inference calls dominate cost and you can train a large model on a suitable path. It is less compelling when the model is small, the data distribution resists simple transport, maximum quality matters more than latency, or the checkpoint expects diffusion-specific conventions.

It moves complexity rather than removing it: you still choose a path, time sampling scheme, solver, step schedule, guidance scale, latent representation, and endpoint convention. A large transformer with fewer evaluations is not automatically cheaper than a small model with more.

Quick check

Next

Flow matching is a different path through the same generative landscape as diffusion. It often runs in a latent-diffusion pipeline. For the broader map of generative methods, see the generative-models overview. To compare quality and speed, use generative evaluation.