A useful way to think about deep neural networks is not as static stacks of layers, but as evolving systems.
In residual networks, this is especially clear, since each layer takes the form
If you look at this update through a mathematical lens, it is exactly the forward Euler discretisation of the following ordinary differential equation
From this perspective, depth plays the role of time, and a deep network can be interpreted as a discrete approximation of a continuous flow.
This viewpoint is more than a curiosity. It provides a conceptual framework for understanding how information moves through a network. Instead of thinking in terms of dozens of layers, we can think in terms of trajectories evolving under a vector field. Questions about stability, sensitivity to perturbations, and robustness naturally translate into properties of the underlying dynamics.
In dynamical systems, small changes in the initial condition can either remain bounded or grow rapidly, depending on the system's structure. The same intuition applies to neural networks. This is why notions such as Lipschitz continuity, contractivity, and stability, which are familiar from differential equations, play an essential role in modern machine learning.
In the video embedded below, I walk through this interpretation step by step, showing how residual networks connect to ordinary differential equations and why this point of view is useful both conceptually and practically.