IC-Anchored Projection for PINNs: Mitigating Gradient Conflict in Initial Value Problems

Abstract

Physics-Informed Neural Networks (PINNs) for time-dependent initial value problems often exhibit long training plateaus and strong sensitivity to initialization. For such IVPs, the initial condition is trajectory-defining: errors near t=0 propagate forward and accumulate, degrading the solution at later times. On the one-dimensional Allen-Cahn equation, we empirically observe persistent gradient conflict and scale imbalance between the initial-condition (IC) loss and the PDE residual, which can yield update directions that are destructive to IC satisfaction under naive gradient summation. We propose IC-Anchored Projection (ICap), a simple geometry-driven gradient operator that designates the IC gradient as an anchor and projects each non-IC gradient onto the half-space with a nonnegative inner product with the IC gradient. This removes only IC-opposing components while preserving IC-compatible information from the PDE and boundary losses.

Across 4/6/8-layer tanh PINNs trained for 10^6 Adam iterations and evaluated over 20 random seeds per depth, ICap consistently improves final relative L^2 error and robustness, shortens plateau-like regimes, and yields more stable time-resolved trajectory errors. Anchor-direction ablations further show that BC- or PDE-anchored variants can converge to locally residual-small yet IC-violating trajectories, highlighting that the anchor choice is crucial for collocation-trained IVPs. These results suggest that anchor-aware gradient design is a practical complement to sampling- and weighting-based strategies for stabilizing time-dependent PINN training.