Tie the weights, and let the depth run
A deep network is a finite stack of layers. Make every layer the same map — tie the weights — and you can ask what happens as the stack grows without end.
The limit, when it exists, is a deep equilibrium: a function of the input alone, with no layer count left in it. That phantom layer on the right is not computed by any finite network. It is the asymptotic output state.
Theorem (Alva, Dueñez, Iovino, Walton.
Deep equilibria: existence and computability.)
If the layer states space L of a neural network and a weight-tied layer map T : L → L both satisfy suitable compactness hypotheses (L is realcompact and T is confined), then the successively iterated deep computations (T, T², T³, …, Tⁿ, …) admit an idempotent deep equilibrium (DEq) T̃ : L → L.
Proof: Via the Ellis–Numakura lemma.
The ability to explicitly compute the DEq is the focus of the paper.
Push it right and watch the stack run off the edge toward the limit.