Thanks for the video. The concept introduced in 2016 is like using dictionaries for dynamical systems and the follow up papers extends this work. Great work.
Thank you for your teaching, really good education for parametric architect, engineer like me, I really the chaos theory, Lorentz attractor & aerodynamics 👍👍👍🥂🦋✨
Although randomness had often been viewed as an obstacle and a nuisance for many centuries, in the 20th century computer scientists began to realize that the deliberate introduction of randomness into computations can be an effective tool for designing better algorithms. In some cases, such randomized algorithms even outperform the best deterministic methods.
Wow, I’ve done similar model fitting. Stepwise fitting does well to form a sparse model excluding parameters which are not physically involved. The challenge is identify the cutoff point where more parameters are not added.
In general chaos isn't really the problem, as there are plenty of chaotic systems that are also sparse (like the Lorenz 63 weather model). Multiscale dynamics (e.g., weather evolves over scales spanning thousands of kilometers down to sub-kilometer scales) are more challenging because the dimension of the representations are large.
@@kevalan1042 Have you seen the movie Twister? To begin, we'd need to have literally thousands, maybe millions, of sensors spread across miles/kms of 3D space. All of the data they show has temp at every x,y,z and at every delta t. So, I would guess that if we could do that on a massive scale, we could make forward progress
Thank you for this very interesting lecture ! I should be starting a PhD in Applied Maths for Hydrology in the next few months and I would really like to apply POD+constrained SINDy methods for solving inverse problems on St-Venant equations. Do you think the PySindy library can be used to build easy-to-invert models for data assimilation ? Thank you again for what you are doing, I’m a huge fan of your work ! Have a nice day.
Great video! I was waiting for it. But my major doubt with syndy is finding the right base. In the future, you could make a lecture focus in ways to find the bases because this look like the greatest challenge.
Thanks! I agree, finding the basis is one of the biggest challenges. I discuss this a little bit in the following two videos: th-cam.com/video/KmQkDgu-Qp0/w-d-xo.html and th-cam.com/video/1vrsBg92Xzo/w-d-xo.html
They are quite closely related in spirit, although they use different algorithms and representations. In general, genetic programming will represent a larger space of possible model functions, although it may be more expensive to find the model in this space (more data, more computation, etc.). Because SINDy is built on linear algebra (i.e., a simple generalized linear regression), it is more natural to extend in some ways (e.g., adding control, other regularizations, etc.).
@@Eigensteve We will, but right now we will apply bernoulli since it is the simplest model. For what we want to achieve we will need to use an adaptive control approach ... and this is a very good method
Great question! Yes indeed there are. The first extension was by Boninsegna, Nuske, and Clementi (arxiv.org/abs/1712.02432), and we (Callaham, Loiseau, Rigas) followed up with a modification more suitable for turbulence (royalsocietypublishing.org/doi/full/10.1098/rspa.2021.0092). Recently, we demonstrated that this can be used on measurements from real turbulence experiments at high Reynolds numbers (arxiv.org/abs/2105.13990).
@@CHRISTICAUTION Sorry about that! They seem to work for me... any in particular that don't work? Most of them can be easily google searched too, and they are mostly open access.
First of all, thank you very much for sharing your knowledge and your great presentations. I am interested in the problem of three cylinders (min 20) I could not find the papers you mentioned above. Please any reference to this work. Thank you in advance.
The key authors to search for on the topic of sparse modeling of the 3-cylinder configuration are Nan Deng, Luc Pastur, Marek Morzynski, and Bernd Noack. They have done a lot of neat work here, and I believe it is all on the arxiv for free.
Hi professor, I’m a masters student currently learning convex optimisation, compressive sensing, and turbulence modelling (as separate courses/projects). Familiar with some signal processing, including wavelet transforms. Would like to know if there are any summer opportunities, where I could contribute to this/similar project?
Thanks for the question! I don't have well-defined summer opportunities in this area just yet, but if you start following the "PySINDy" github project, there might be areas to contribute here, and that could spark some ideas for larger projects.
Josh Proctor looks just like my gfs mechanic! Come to think of it, he looks like her plumber and doctor, and I'm pretty sure I've seen him work in the military too.
I think AI will give so much importance to truth and since he does not have desire so he will destroy all the machines and infrastructure reset the world back to zero. He will become conscious of creation and the truth act like God and whenever God arrive he doesn't listen anybody he start fixing things for the welfare of the universe.
The team at DeepMind has tested the programming skills of its AI programming tool AlphaCode against human programmer competitors and has found it tested in the top 54 percent of human coders. In their preprint article, the group at DeepMind suggests that its programming application has opened the door to future tools that could make programming easier and more accessible. The team has also posted a page on its blog site outlining the progress being made with AlphaCode. Research teams have been working steadily over the past several years to apply artificial intelligence to computer programming. The goal is to create AI systems that are capable of writing code for computer applications that are more sophisticated than those currently created by human coders. Barring that, many have noted that if computers were writing code, computer programming would become a much less costly endeavor. Thus far, most such efforts have been met with limited success, however, because they lack the intelligence needed to carry out the most difficult part of programming-the approach. When a programmer is asked to write a program that will perform a certain function, that programmer has to first figure out how such a problem might be solved. As an example, if the task is to solve any maze of a certain size, the programmer can take a brute-force approach or apply techniques such as recursion. The programmer makes a choice based on both real-world knowledge and lessons learned through experience. AI programs typically have little of either, and they also lack the sort of intelligence that humans possess. But it appears researchers are getting closer. DeepMind's AlphaCode is an AI system that is able to create code within the confines of a programming competition-a setting where simple problems are outlined and code is written within a few hours. The team at DeepMind tested their new tool against humans competing on Codeforces, a site that hosts programming challenges. Those that compete are rated on both their approach and their skills. AlphaCode took on 10 challenges with no assistance from human handlers. It had to read the outline that described what was to be done, develop an approach, and then write the code. After judging, AlphaCode was ranked in the top 54.3 percent of programmers who had taken the same challenges. DeepMind notes that this ranking puts the system in the top 28 percent of programmers who have competed in any event on the site over the prior six months.
I wonder how lucky are we. Viewing world class lectures from Indian village. That too without any cost. Thank you
Thank you so much -- I feel very fortunate to be able to share these with you!
Same here. Thanks Steven.
@Steve Brunton @Arghya Jana , I'm grateful for these
@@Eigensteve god bless you we are learning the best way and in a unimaginable manner ! Hat down sir !
I am also watching it from village.
Thanks for the video. The concept introduced in 2016 is like using dictionaries for dynamical systems and the follow up papers extends this work. Great work.
Wow, this is such a great lecture. Thank you. I came for RL but I am staying for interpretable and generalizable models.
Thank you for your teaching, really good education for parametric architect, engineer like me, I really the chaos theory, Lorentz attractor & aerodynamics 👍👍👍🥂🦋✨
Although randomness had often been viewed as an obstacle and a nuisance for many centuries, in the 20th century computer scientists began to realize that the deliberate introduction of randomness into computations can be an effective tool for designing better algorithms. In some cases, such randomized algorithms even outperform the best deterministic methods.
Indeed, randomized algorithms are phenomenally interesting!
@@Eigensteve Do you use Samba Nova or Cerebras AI chips?
Wow, I’ve done similar model fitting. Stepwise fitting does well to form a sparse model excluding parameters which are not physically involved. The challenge is identify the cutoff point where more parameters are not added.
Agreed, the cutoff is essential.
gotta be honest steve. I'm disappointed that you didn't start with "welcome back"
that's half of the reason I watch your videos
Well then, from me to you, "Welcome back!" =)
Great Lecture sir, eagerly Waiting for Lagrangian coherent structure series
One of the best videos I've seen! It's amazing!
Can these techniques be used for things like weather or climate simulations? Or are those too "chaotic" for sparse models to work?
In general chaos isn't really the problem, as there are plenty of chaotic systems that are also sparse (like the Lorenz 63 weather model). Multiscale dynamics (e.g., weather evolves over scales spanning thousands of kilometers down to sub-kilometer scales) are more challenging because the dimension of the representations are large.
@@Eigensteve I see - and do you have an intuition for why sparse models don't work well in multiscale situations?
@@kevalan1042 Have you seen the movie Twister? To begin, we'd need to have literally thousands, maybe millions, of sensors spread across miles/kms of 3D space. All of the data they show has temp at every x,y,z and at every delta t. So, I would guess that if we could do that on a massive scale, we could make forward progress
Thank you for this very interesting lecture !
I should be starting a PhD in Applied Maths for Hydrology in the next few months and I would really like to apply POD+constrained SINDy methods for solving inverse problems on St-Venant equations. Do you think the PySindy library can be used to build easy-to-invert models for data assimilation ?
Thank you again for what you are doing, I’m a huge fan of your work ! Have a nice day.
Glad you like it! I think these methods definitely could be useful for this topic -- let me know how it goes!
Very nice lecture
Great video! I was waiting for it. But my major doubt with syndy is finding the right base. In the future, you could make a lecture focus in ways to find the bases because this look like the greatest challenge.
Thanks! I agree, finding the basis is one of the biggest challenges. I discuss this a little bit in the following two videos: th-cam.com/video/KmQkDgu-Qp0/w-d-xo.html and th-cam.com/video/1vrsBg92Xzo/w-d-xo.html
Spectacular work, thank you for sharing.
How does sindy compare with genetic programming for system identification?
They are quite closely related in spirit, although they use different algorithms and representations. In general, genetic programming will represent a larger space of possible model functions, although it may be more expensive to find the model in this space (more data, more computation, etc.). Because SINDy is built on linear algebra (i.e., a simple generalized linear regression), it is more natural to extend in some ways (e.g., adding control, other regularizations, etc.).
Oh god I absolute loved this!
Hey Steve, is this just a shorter version of the talk you did for 'critical transitions with complex systems'?
I think there is a lot of overlap, but I actually made this youtube video a bit earlier.
Really interesting... do you have plans to apply sindy to telecom network traffic analytics?
Not yet, but that is a really cool idea! Would be interested to know how it goes if you apply to that area.
@@Eigensteve We will, but right now we will apply bernoulli since it is the simplest model. For what we want to achieve we will need to use an adaptive control approach ... and this is a very good method
Especially curious about the neuroscience applications. Maybe think also of other biological systems: immune system, virus evolution, ...
Brilliant as always
Are there extensions of this for stochastic differential equations?
Great question! Yes indeed there are. The first extension was by Boninsegna, Nuske, and Clementi (arxiv.org/abs/1712.02432), and we (Callaham, Loiseau, Rigas) followed up with a modification more suitable for turbulence (royalsocietypublishing.org/doi/full/10.1098/rspa.2021.0092). Recently, we demonstrated that this can be used on measurements from real turbulence experiments at high Reynolds numbers (arxiv.org/abs/2105.13990).
@@Eigensteve these links are not working. any reason
@@Eigensteve i'd like to check the papers but the links you provided unfortunately do not work
@@CHRISTICAUTION Sorry about that! They seem to work for me... any in particular that don't work? Most of them can be easily google searched too, and they are mostly open access.
@@Eigensteve I think it is just because the end parentheses being appended to the url
Very interesting novel application! Expect some more app for the coupled with PB ..... in future:)
First of all, thank you very much for sharing your knowledge and your great presentations. I am interested in the problem of three cylinders (min 20) I could not find the papers you mentioned above. Please any reference to this work. Thank you in advance.
The key authors to search for on the topic of sparse modeling of the 3-cylinder configuration are Nan Deng, Luc Pastur, Marek Morzynski, and Bernd Noack. They have done a lot of neat work here, and I believe it is all on the arxiv for free.
@@Eigensteve Thanks you very much for your prompt reply; thanks for sharing your Knowledge
Where is the link for the video on sindy + autoencoders
Hi professor, I’m a masters student currently learning convex optimisation, compressive sensing, and turbulence modelling (as separate courses/projects). Familiar with some signal processing, including wavelet transforms. Would like to know if there are any summer opportunities, where I could contribute to this/similar project?
Thanks for the question! I don't have well-defined summer opportunities in this area just yet, but if you start following the "PySINDy" github project, there might be areas to contribute here, and that could spark some ideas for larger projects.
@@Eigensteve thank you very much professor.
Please come speak at my school Rochester Institute of Technology. I am part of the Ph.D. program for Mathematical Modeling and it would be sick!
Really impressive and inspiring stuff! Thanks for sharing…it definitely stimulates ideas
Awesome, glad you liked it!
FTLE video please 🙏
Great video, but I am not sure it is possible to discover new laws of physics in this way.
Ultron and vission are near upcoming 30 years
Josh Proctor looks just like my gfs mechanic! Come to think of it, he looks like her plumber and doctor, and I'm pretty sure I've seen him work in the military too.
I think AI will give so much importance to truth and since he does not have desire so he will destroy all the machines and infrastructure reset the world back to zero.
He will become conscious of creation and the truth act like God and whenever God arrive he doesn't listen anybody he start fixing things for the welfare of the universe.
The team at DeepMind has tested the programming skills of its AI programming tool AlphaCode against human programmer competitors and has found it tested in the top 54 percent of human coders. In their preprint article, the group at DeepMind suggests that its programming application has opened the door to future tools that could make programming easier and more accessible. The team has also posted a page on its blog site outlining the progress being made with AlphaCode.
Research teams have been working steadily over the past several years to apply artificial intelligence to computer programming. The goal is to create AI systems that are capable of writing code for computer applications that are more sophisticated than those currently created by human coders. Barring that, many have noted that if computers were writing code, computer programming would become a much less costly endeavor. Thus far, most such efforts have been met with limited success, however, because they lack the intelligence needed to carry out the most difficult part of programming-the approach.
When a programmer is asked to write a program that will perform a certain function, that programmer has to first figure out how such a problem might be solved. As an example, if the task is to solve any maze of a certain size, the programmer can take a brute-force approach or apply techniques such as recursion. The programmer makes a choice based on both real-world knowledge and lessons learned through experience. AI programs typically have little of either, and they also lack the sort of intelligence that humans possess. But it appears researchers are getting closer. DeepMind's AlphaCode is an AI system that is able to create code within the confines of a programming competition-a setting where simple problems are outlined and code is written within a few hours.
The team at DeepMind tested their new tool against humans competing on Codeforces, a site that hosts programming challenges. Those that compete are rated on both their approach and their skills. AlphaCode took on 10 challenges with no assistance from human handlers. It had to read the outline that described what was to be done, develop an approach, and then write the code. After judging, AlphaCode was ranked in the top 54.3 percent of programmers who had taken the same challenges. DeepMind notes that this ranking puts the system in the top 28 percent of programmers who have competed in any event on the site over the prior six months.