NO. 44 · Computational & Data

Deep Learning for Wave Physics

From a single neuron to a PINN that converges, with every failure mode met on purpose. Networks, losses, architectures, and the training pathologies that separate a published result from a pretty loss curve.

You can build and train a neural network from first principles, put a PDE residual into its loss and know why that works, choose an architecture for a wave problem deliberately, and diagnose the convergence failures that stop most physics-informed models.

19 competencies · 6 interactive widget challenges · 9 to 15 hours of guided study

For geoscientists who want to train networks that respect physics

Draws on: Physics-Informed Neural Networks for Exploration Seismology

Neural networks from zero

Neurons, layers, and depth

A neuron is a weighted vote and depth is composition; universal approximation says the pair can represent your wavefield, not that training will find it.

Reads: What is a neuron? Layers, depth, and universal approximation

Activations and the loss landscape

The activation decides what the network finds easy, and the loss landscape decides where the optimizer can actually go.

Reads: Activation functions and inductive bias Loss functions and the loss landscape

Gradient descent and backpropagation

One optimizer trains everything in this book; backprop is just the chain rule with receipts, and you should do it by hand once.

Reads: Gradient descent by hand The chain rule and backpropagation

Autodiff and the training loop

Automatic differentiation is why PINNs are possible at all: the same graph that trains the network also evaluates the PDE.

Reads: Auto-differentiation as a computational graph Training loops and optimisers

Spectral biaswidget challenge

Networks learn low frequencies first, and seismic signals are not low frequency; this single fact explains half of every failed wavefield fit.

Reads: Spectral bias and why physics needs it fixed

The PINN idea

Why supervised learning cannot solve PDEs

Data-only networks interpolate; physics extrapolates. The PDE residual in the loss is what buys you the second power.

Reads: Why supervised ML alone cannot solve PDEs The PINN idea: PDE residual as a loss term

Burgers' equation, end to endwidget challenge

The hello-world of PINNs, trained honestly from scratch: one nonlinear PDE, one network, every step visible.

Reads: 1D Burgers’ equation, end to end

Boundary conditions, soft and hard

Soft penalties negotiate with the boundary; reparameterisation enforces it. Choosing wrong is the most common silent PINN error.

Reads: Boundary and initial conditions: soft enforcement Hard-constraint enforcement via reparameterisation

The canonical gallery

Heat, wave, Poisson, Navier-Stokes: seeing the same recipe succeed across the gallery is what makes the method feel like a method.

Reads: The canonical PINN gallery

Architectures

MLPs and Fourier featureswidget challenge

A plain MLP cannot see high frequencies; Fourier feature embeddings are the standard cure, and tuning their scale is a real skill.

Reads: Vanilla MLPs and their limits Fourier feature embeddings

SIREN and hard-constrained nets

Sinusoidal activations speak the wave equation's native language, and hard-constrained architectures make boundary errors structurally impossible.

Reads: SIREN, sinusoidal representation networks Hard-constrained architectures

Multi-scale nets and the decision tree

Real velocity models carry structure at every scale; the architecture decision tree turns folklore into a checklist.

Reads: Multi-scale and multi-resolution networks Picking an architecture: a decision tree

Training pathologies

Why most beginner PINNs failwidget challenge

The pathology gallery and the loss-balance crisis: nine out of ten stuck PINNs are stuck for one of the reasons on this page.

Reads: Why most beginner PINNs do not converge The loss-balance crisis: data + PDE + BC weights

NTK and adaptive weighting

Balancing loss terms by hand does not scale; NTK-based and self-adaptive weights are the principled versions of the knob you were already turning.

Reads: NTK-balanced weighting Gradient pathologies and adaptive weights

Causality and curriculum

Time-domain PINNs cheat by fitting the future first; causality weighting and curricula force the network to learn physics in the order physics happens.

Reads: Causality weighting for time-domain PINNs Curriculum and multi-stage training

Adaptive sampling and domain decompositionwidget challenge

Put collocation points where the residual lives, and split domains the residual cannot span: the two scaling moves for hard problems.

Reads: Adaptive collocation point sampling (RAR/RAD) Domain decomposition: XPINN, cPINN, FBPINN

Waves

The acoustic wave equation, 1D and 2D

The PDE this whole path has been training toward; in one and two dimensions it is demanding enough to test everything you built.

Reads: The 1D acoustic wave equation The 2D acoustic wave equation

Time versus frequency

Time-domain or Helmholtz is the first architecture-level decision of any wave PINN, and it changes which pathologies you will meet.

Reads: Picking a formulation: time vs frequency

Capstone

Capstone: reproduce an arXiv paperwidget challenge

The researcher's rite of passage: take a current PINN paper and make its central figure appear on your own screen, failure modes and all.

Reads: Reproducing a current arXiv PINN paper end-to-end