When operator learning beats per-instance training

Part 8 has presented operator learning as a paradigm-shifting toolkit. We now confront the honest cost-benefit question: WHEN does it pay off, and when do classical methods (FDTD, FSM, per-instance PINNs) remain the right choice? The answer is a single inequality.

The crossover formula

Let:

• $T_{\mathrm{op}}$: cost of pretraining the operator network ONCE.
• $T_{\mathrm{per}}$: cost of solving ONE problem instance from scratch (e.g., training a per-instance PINN, or running FDTD).
• $T_{\mathrm{inf}}$: cost of ONE operator inference (a single forward pass through the trained network).
• $N$: number of distinct problem instances we want to solve.

Total cost for each strategy:

Operator wins when $\mathrm{cost}${\mathrm{op}}(N) < \mathrm{cost}{\mathrm{per}}(N), which gives

For seismic operators $T_{\mathrm{inf}}$ is typically $10^{-4}\text{–}10^{-2}$ s and $T_{\mathrm{per}}$ is $10\text{–}10^4$ s, so the speedup ratio $T_{\mathrm{per}} / T_{\mathrm{inf}}$ is enormous. The crossover $N^*$ depends primarily on how cheap the problem already is per-instance vs how expensive pretraining is. Common ranges:

• 1-D toy problems (this textbook): $T_{\mathrm{per}} \sim 1$ s, $T_{\mathrm{op}} \sim 5$ s. Crossover at $N^* \sim 5$.
• 2-D acoustic FWI: $T_{\mathrm{per}} \sim 60$ s (PINN per source), $T_{\mathrm{op}} \sim 1$ hr. Crossover at $N^* \sim 60$ sources.
• 3-D elastic wave propagation: $T_{\mathrm{per}} \sim 1$ hr (FDTD shot), $T_{\mathrm{op}} \sim 1$ week (F-FNO training, Lehmann et al 2024). Crossover at $N^* \sim 168$ shots.

For full survey-scale FWI projects with thousands of shots and many velocity-model iterations, all three regimes are deeply in the operator-wins regime.

Try it: measure your own crossover

The widget runs in three timed phases: (1) pretrain a DeepONet on the §8.5-style 7-parameter heat-equation family, (2) measure inference time over 100 forward passes for tight statistics, (3) fit a small per-instance MLP to ONE specific instance via supervised regression. After all three measurements, the cost-vs-N chart shows where operator learning beats per-instance for THIS browser, on THIS machine, with THESE problem sizes. The N slider lets you place yourself at any working point and read off the speedup factor.

Beyond N*: the qualitative arguments

Crossover is the quantitative argument. There are four qualitative arguments that reinforce it:

• Bayesian-friendly forward model. Sampling from a posterior $p(c \mid \mathrm{data})$ via MCMC needs $O(10^4\text{–}10^6)$ forward evaluations. With FDTD this is impractical at scale; with an operator surrogate, a few hours of MCMC suffices. This unlocks UNCERTAINTY QUANTIFICATION on FWI results — the central topic of Part 9.
• Differentiable end-to-end. Operator networks are differentiable through their inputs (initial conditions, velocity model). For inverse problems formulated as gradient descent on the velocity model, the operator provides $\partial T / \partial c$ via auto-diff. FDTD requires manual adjoint-state implementation per equation type.
• Real-time interactivity. §8.5's parameter sliders. With FDTD, a designer waits seconds-to-minutes per parameter change; with an operator surrogate, design happens at 60 fps.
• GPU-efficient inference. Operator networks pack many forward passes into a single GPU kernel call. A typical TensorRT or ONNX deployment of an FNO does 1000 forward passes in 100 ms — far faster than 1000 separate FDTD invocations would manage.

When operator learning loses

Three scenarios where classical solvers and per-instance PINNs still win:

• One-off problems with N = 1. If you have a single legacy survey to analyse and never need to re-do it, pretraining an operator network is wasted effort. Just run FDTD or train one PINN.
• Out-of-distribution problems. Operator networks trained on Marmousi-class velocity models will not generalise to volcanic basement structures or strong-anisotropy salt domes. The training distribution is the operating envelope; outside it, predictions silently fail. For exotic case studies, classical solvers handle out-of-distribution inputs trivially.
• Verifying classical-solver baselines. Even with a deployed operator surrogate, important production runs typically include at least one FDTD verification for trust calibration. The operator surrogate is the workhorse; FDTD is the safety net.

Hybrid architectures: best of both

Modern production seismic-imaging pipelines often use hybrid architectures that combine operator pretraining with PINN fine-tuning:

• Operator warm-start for PINN. Pretrain an operator surrogate on a velocity-model family. For a NEW velocity model, evaluate the surrogate to get an initial guess, then refine with a per-instance PINN initialised from that guess. The PINN converges in 10× fewer epochs because it starts from a good answer.
• Operator + classical FWI gradient correction. Use the operator for the wave-equation forward solve in FWI; combine its gradient with a small classical FDTD correction to reduce out-of-distribution errors. Saves $\sim 90$ of compute.
• Operator + Bayesian UQ. Cheap operator forwards enable HMC/Stein VI over the posterior, with a final classical-FDTD posterior-mean simulation for trust verification.

These are the architectures Part 9 will build on. Operator surrogates are not a replacement for PINNs and FDTD; they are a NEW LAYER in the toolkit, slotted in where amortisation pays off, and bypassed where it does not.

Out-of-distribution detection: a critical gap

The single biggest engineering risk of operator-based seismic workflows is SILENT FAILURE on OOD inputs. A network trained on smooth gradient velocity models will produce confident-looking predictions for a velocity model containing a salt dome — and those predictions can be ARBITRARILY WRONG without any in-band warning. This is fundamentally different from FDTD failure modes, which are usually loud (numerical instability, NaN propagation, etc.).

Production OOD detection techniques:

• Likelihood under the training distribution. Compute $\log p(c \mid \mathrm{train})$ for each new input. If below a threshold, flag and revert to FDTD.
• Model ensembles. Train several operator networks with different seeds; high disagreement on a new input indicates OOD.
• PDE-residual check. Apply the network to predict $T(x)$, then check if $T_{\mathrm{NN}}$ satisfies the eikonal residual. If not, fall back to FSM/FDTD.
• Posterior uncertainty (Bayesian Operators). Train operator networks with weight-space uncertainty (BNN, MC-Dropout). High predictive variance signals OOD inputs. Implementation cost: 1.5-2× normal training but provides a direct uncertainty signal.

Part 9 will revisit OOD detection in the broader UQ context.

Part 8 wrap-up

Part 8 covered the operator-learning paradigm end-to-end:

• §8.1 Per-instance vs operator framing — the conceptual pivot, with a worked DeepONet on the antiderivative operator demonstrating amortised inference.
• §8.2 DeepONet architecture deep-dive — the branch + trunk decomposition as a learned-basis representation of the operator, visualised on the Poisson BVP.
• §8.3 Fourier Neural Operators (FNO) — spectral-convolution layers, resolution-invariance, and a single-layer FNO that recovered the heat-equation operator to machine precision.
• §8.4 Learned wave-equation propagators — time-stepping with FNO, the eigenvalue-stability constraint that determines whether rollouts stay bounded, and structural-stability tricks (β=-1 freeze, α clamp) that mirror production-code techniques.
• §8.5 Real-time parametric explorers — the amortisation made tactile, with 7-D heat-operator family explored at 60 fps after a 5-second pretraining.
• §8.6 (this section) The crossover analysis: when operator learning amortises, when classical methods still win, and the hybrid architectures that combine the strengths of both.

The reader who completes Part 8 has the toolkit to BUILD an operator surrogate for any new PDE family, EVALUATE the cost-benefit decision honestly, and DEPLOY hybrid PINN-operator architectures in production seismic-imaging workflows. Part 9 takes operator learning to the next step: hybridise it with classical FWI, add Bayesian uncertainty quantification on top, and confront the unique challenges of seismic-inverse problems at production scale.

References

• Lu, L., Jin, P., Pang, G., Zhang, Z., Karniadakis, G.E. (2021). DeepONet. Nat. Mach. Intell. 3(3), 218–229.
• Li, Z., Kovachki, N., Azizzadenesheli, K., et al. (2020). Fourier Neural Operator for Parametric Partial Differential Equations. ICLR 2021. arXiv:2010.08895.
• Lu, L., Meng, X., Cai, S., et al. (2022). A comprehensive and fair comparison of two neural operators. CMAME 393, 114778. Empirical comparison of DeepONet and FNO across multiple problem families.
• Lehmann, F., Gatti, F., Bertin, M., Clouteau, D. (2024). F-FNO 3D elastic-wave propagation. CMAME 420, 116718. Production-scale operator-based seismic surrogate.
• Hendrycks, D., Gimpel, K. (2017). A baseline for detecting misclassified and out-of-distribution examples in neural networks. ICLR 2017. Foundational OOD-detection paper.