Neural network, efficient network of the future

The concept of “always loaded neural network”

We nowadays process the giant neural networks but through a narrow nozzle of processor, slowly waiting for the tank of water to be drained or processed from the narrow tap to be useful. All processors today can access vast amounts of information, but not simultaneously process vast amount of information.

Normally, in a neural network, while processing on a processor, only a single layer is active, but what we are looking for is more natural brain like hardware, in which among all the active brain regions, everything is active all the time. That is, multiple layers (or all) are processing time based continuous information (such as video stream, audio, thoughts, etc.).
So the goal is to load or make the entire neural network on the hardware at once, (much like TPU loads and processes tensors at once), so entire tank of water is drained at once (maybe not so good analogy), and processing is made faster, or more realtime, to its peak processing capability in this 3D-t universe.

This is possible, by physically implementing the entire network, but would require fast production of silicon for making iterative designs, or once the design is final and ready, entire network could be produced in hardware. although ,this may not be programmable in traditional sense, and, any sort of ‘programming edit’ would have to follow the hardware manufacturing process, this is where the turnout-time of matter to its designed hardware is useful.