this in combination with some of the modern neural operator (fourier, wavelet) methods are really going to be the norm for most computational physics in industry that use continuum models I think
So cool, thank you, professor! I have a possibly naive question: does this mean that MLPs are inherently unable to fully model such systems, no matter the complexity or depth of their architecture, because they will always lose a system's symmetry relations?
least action seems like the only way you can actually ground a neural net if you want it stay in reality, even abstract from physics since at some point it needs to come back to reality where there is thermodynamics governing always. Seems like a necessary core for neural nets in general to adopt.
I do long for an agent that can serf all models tuned and weighted although I'm sure it will be a while before we the people really get tomorrow's access today like that .
Excellent video! About the intrinsic coordinates problem for HNNs, can't you use an auto-encoder to "discover" the correct (q,p) pairs from your input data like Greydanus et al do for the pendulum video example in the HNN paper? It seems like the added cost of computing the Hessian could be a significant bottleneck for more realistic, high-dimensional datasets.
Can I clarify something? Does the NN just give the updated position and velocity at the next time step? And then you repeatedly use the NN to integrate the system to find its full time evolution? You can't use this sort of architecture (at least for simple problems) to find the state at an arbitrary point in time in a single NN calculation?
This is a good question. I interpreted that you get back the accelerations i.e. enough to make a timestep from your (input) initial values. In either case (Baseline or Lagrangian NN) it seems the accelerations are the training data - it is just how they are used. For predictions, the difference is how the accelerations for the next step are generated - the autodifferentiated L version being superior, presumably being more constrained by the Lagrangian rather than simply being some sort of ML interpolator of the training observations. While I can see this is potentially interesting for deducing underlying dynamics from observations, I am curious how it useful it is for chaotic systems. Consider the training data as samples of positions and velocities in the phase space (e.g. 4 dimensional for the double pendulum), then of course velocities (given) and accelerations indicate the tangent to a trajectory at each of these phase space points. However, given that trajectories of initially close points diverge in chaotic systems, how realistic is this for marching forwards?
In his older videos he used a PINN that was was an integrator to show that they are much better at long term predictability. I imagine it is exactly the same. These are local methods, not global. Since they are discrete methods they can't do any global derivations which would be symbolic. It is likely an impossible problem to have a global solver. Effectively you are then asking to be able to compute an exact timestep to get where you want with zero loss. This would, at the very least, require one to have the symbolic description of the system rather than just discrete sample points.
Good eveny sir and tank you for yours videos Please Can WE use or those méthodes(all that you have présent) in a epidemylogycal model ? ( Driving by ODE or PDE system)
Problem (input), reaction (hidden), solution (output) -- the Hegelian dialectic! Your mind (concepts) is the reaction or anti-thesis to the outside world (perception). Input vectors are dual -- contravariant is dual to covariant -- dual bases, Riemann geometry. Concepts are dual to percepts -- the mind duality of Immanuel Kant. "Always two there are" -- Yoda. Neural networks are based upon the Hegelian dialectic! Lagrangians are dual to Hamiltonians.
Hello sir ! Is it possible to make mathematical model/transfer function of Diode /Thyristor so that we can predict output of diode just by convolution of diode and input sine wave ?
I think this Lagrangian neural network would be good at most classical physical simulations and has applications like being used in a physics engine. There is the Lagrangian of the standard model so it should be possible to also replicate particle physics excluding gravitational effects.
any proof that it will find the right lagrangian? in my case the qpp values fit, however the lagrangian is completly off. seems like there is no 1:1 relationship.
To all those who read my comment: I want to apply a Lagrangian Neural Network (NN) to approximate or model a temporal signal, such as temporal traffic flow. However, I don't know where to start. Could you guide me on whether it is possible and where I can find related python-code? I would also be happy to learn if anyone has applied LNN to the MNIST features as an embedded layer in a neural network.
![Neural Implicit Flow (NIF) [Physics Informed Machine Learning]](http://i.ytimg.com/vi/y-s1oECkbuU/mqdefault.jpg)
![Neural Implicit Flow (NIF) [Physics Informed Machine Learning]](/img/tr.png)
![Hamiltonian Neural Networks (HNN) [Physics Informed Machine Learning]](http://i.ytimg.com/vi/AEOcss20nDA/mqdefault.jpg)
![The moment we stopped understanding AI [AlexNet]](/img/n.gif)
this in combination with some of the modern neural operator (fourier, wavelet) methods are really going to be the norm for most computational physics in industry that use continuum models I think
So cool, thank you, professor! I have a possibly naive question: does this mean that MLPs are inherently unable to fully model such systems, no matter the complexity or depth of their architecture, because they will always lose a system's symmetry relations?
Great content
So we have Lagrangian & Hamiltonian Neural Networks, I think the question is obvious: do we have Hamilton-Jacobi Neural Networks?
least action seems like the only way you can actually ground a neural net if you want it stay in reality, even abstract from physics since at some point it needs to come back to reality where there is thermodynamics governing always. Seems like a necessary core for neural nets in general to adopt.
Integrating chaotic systems: when Runge-Kute can be called naive
Very interesting! ❤️
I do long for an agent that can serf all models tuned and weighted although I'm sure it will be a while before we the people really get tomorrow's access today like that .
Excellent video! About the intrinsic coordinates problem for HNNs, can't you use an auto-encoder to "discover" the correct (q,p) pairs from your input data like Greydanus et al do for the pendulum video example in the HNN paper? It seems like the added cost of computing the Hessian could be a significant bottleneck for more realistic, high-dimensional datasets.
Can I clarify something? Does the NN just give the updated position and velocity at the next time step? And then you repeatedly use the NN to integrate the system to find its full time evolution? You can't use this sort of architecture (at least for simple problems) to find the state at an arbitrary point in time in a single NN calculation?
This is a good question. I interpreted that you get back the accelerations i.e. enough to make a timestep from your (input) initial values. In either case (Baseline or Lagrangian NN) it seems the accelerations are the training data - it is just how they are used. For predictions, the difference is how the accelerations for the next step are generated - the autodifferentiated L version being superior, presumably being more constrained by the Lagrangian rather than simply being some sort of ML interpolator of the training observations. While I can see this is potentially interesting for deducing underlying dynamics from observations, I am curious how it useful it is for chaotic systems. Consider the training data as samples of positions and velocities in the phase space (e.g. 4 dimensional for the double pendulum), then of course velocities (given) and accelerations indicate the tangent to a trajectory at each of these phase space points. However, given that trajectories of initially close points diverge in chaotic systems, how realistic is this for marching forwards?
In his older videos he used a PINN that was was an integrator to show that they are much better at long term predictability. I imagine it is exactly the same. These are local methods, not global. Since they are discrete methods they can't do any global derivations which would be symbolic.
It is likely an impossible problem to have a global solver. Effectively you are then asking to be able to compute an exact timestep to get where you want with zero loss. This would, at the very least, require one to have the symbolic description of the system rather than just discrete sample points.
I suggest a "transcendental neural network". Can I publish?
Good eveny sir and tank you for yours videos
Please Can WE use or those méthodes(all that you have présent) in a epidemylogycal model ? ( Driving by ODE or PDE system)
Do we use single (or same) optimizer for both grandient descent and ascent in these Lagrangian NN or separate one ??
Could you please do a video explanation one for Implicit Neural Representations with Periodic Activation Functions? Thank you!
Problem (input), reaction (hidden), solution (output) -- the Hegelian dialectic!
Your mind (concepts) is the reaction or anti-thesis to the outside world (perception).
Input vectors are dual -- contravariant is dual to covariant -- dual bases, Riemann geometry.
Concepts are dual to percepts -- the mind duality of Immanuel Kant.
"Always two there are" -- Yoda.
Neural networks are based upon the Hegelian dialectic!
Lagrangians are dual to Hamiltonians.
Hello sir ! Is it possible to make mathematical model/transfer function of Diode /Thyristor so that we can predict output of diode just by convolution of diode and input sine wave ?
I think this Lagrangian neural network would be good at most classical physical simulations and has applications like being used in a physics engine.
There is the Lagrangian of the standard model so it should be possible to also replicate particle physics excluding gravitational effects.
sir..can u make a video on tensor for a physics major.??????we will be grateful if u make one....
any proof that it will find the right lagrangian?
in my case the qpp values fit, however the lagrangian is completly off. seems like there is no 1:1 relationship.
To all those who read my comment:
I want to apply a Lagrangian Neural Network (NN) to approximate or model a temporal signal, such as temporal traffic flow. However, I don't know where to start. Could you guide me on whether it is possible and where I can find related python-code? I would also be happy to learn if anyone has applied LNN to the MNIST features as an embedded layer in a neural network.
excellent
An easy introduction/refresher on Newtonian, Hamiltonian, and Lagrangian mechanics here: th-cam.com/video/0DHNGtsmmH8/w-d-xo.html
❤
I know, no one care, but please tell me when that Perceptron BS ended on this channel, and I can finnaly enjoy real math as half year or so ago.
The videos, went from being really instructive, towards, just skimming the surface unfortunately. The content has changed and not for the better..
I so want to dub a version of this video to ZZTops 'LaGrange.'