“Every so often, we get that cheerful log message: ‘Solver skipping solution as problem is nonconvex.’ It’s like the solver just took a look, shrugged, and said, ‘Yeah, nope, not today.’
This is a great video! I'm a physicist. I've had to do some challenging control theory and optimization stuff in the past. I wish this would have been covered in grad school. It's such a great way to exercise the mathematical muscles and it's a lot of fun! Getting this to run in real time in a noisy nonlinear non constant delayed environment is so hard... but oddly satisfying! Subscribed!
I will do. I just need to find a bit of time to start. Maybe I will start during the Christmas holiday, but I can't promise. But they will eventually come.
Optimizing is the art of making the right decision using math to find it. Thank you for sharing the video. I hope I get the chance to explore this topic more closely in the future. In the meantime, I'll keep an eye on this channel.
The real art of optimization is making a good enough decision very very quickly, very very cheaply. Certain specifically trained LLMs can give you really close three body problem estimates in a very short amount of time with very small amounts of energy. You can do things now like gravity capture orbits without spending months trying to backwards work out the answer. Edit: dyslexia problems
At 13:14 does the convexified cost function have to be larger than the original at every point? I guess it makes sense for hard constraints but what about the soft ones?
You can do both. You can take the convex upper envelope, like in the video and this will give you a minimum which is not necessarily as good as the minimum of the original function. Or you can take the convex lower envelope which lies below the original function. In this case the minimum of the convexified problem may not be achievable by the original function. In other words, you can get an upper bound and a lower bound depending on how you convexify. A similar discussion holds for the constraints, but usually in that case you want to take the inner convex set, so that you know that the obtained point is feasible also for the original problem. Of course this will give a conservative solution (an upper bound on the minimum).
Excellent review of a fascinating subject. I have often thought it would make a great video game: the user controls thrust and direction as functions of t (which define a path) and tries to minimize fuel consumption for launching or landing. Most trials will result in failure to reach orbit or crash landings, but with continual fine tuning, success can be achieved. The successful paths can then be compared to the optimal path.
Hello ! Can you recommend a good textbook on these convex optimisation problem ? In particular about the methods for the convexification of a non-convex problem ? Thank you for the nice video !
I suggest starting with "Convex Opmization" by Vandenberghe and Boyd. You can download the book from the website of Stephen Boyd. For more advanced tools (non-convex problems), check more material from Boyd, there is a lot available. For implementation I advise CVXPY, but you need to know a bit of Python. There are other versions of CVX, but I personally prefer the Python version.
I doubt he could recommend anything useful, inasmuch his video diagrams demonstrate that he doesn't know the difference between convexity and concavity. 3:10
You are confusing a convex shape with a convex function. First figure here en.m.wikipedia.org/wiki/Convex_function If you wonder why that function is called convex even though it may look otherwise is because a convex function f(x) is called convex because its epigraph is convex, that is it curves outward. See second figure at the link.
Thank you Dr Scarciotti, mathematics isnt my strongsuit by any means but your presentation was very enjoyable. Obviously optimization is a robust and significant component. But how does a falcon 9 for instance "know" its orientation in 3d space in order to provide data to the algorithms? And does optimization apply to guidance in zero g environments the same way it operates in the earths atmosphere where more dynamic forces are at play?
Many thanks! The formulations in the video are very simplified versions of what's going on. For each aspect there are teams of people working on that aspect, and the number of constraints is definitely higher. Regarding your first question, the rocket knows its orientation by applying data fusion techniques based on multiple sensor measurements. I touched upon this briefly in my previous video. Regarding the second question, yes optimization is always applied, what changes is the formulation of the problem, so that the equations take into account the specific situation in which the rocket is.
Very interesting and fun to know and spread this… I taught most were non-convex… Since many equations are in incomplete or unsolvable or simplification (that could have chaotic variations if over simplification)
Yes, many problems are non-convex, which is why I explain the idea of convexification. But yes, those are approximations which have their limitations. That said, convex opmization plays a big role, and there are rumours, although not officially confirmed, that they actually use CVXPY specifically as tool.
Well, mathematical problems are also useful if someone can show they can't be solved fast enough... Which would mean they need to think about a different approach, or bigger margins, or at the end of it all, that it can't be done.
I agree, but maths problems don't even need to be useful in the first place to be interesting. There is a quote from "Mathematician's Apology" by G.H. Hardy that goes "the real mathematics of real mathematicians is almost wholly useless". I am not endorsing this statement, but I think it's an interesting one and the book is a fun read if you don't take it too seriously.
It really is an amazing achievement of a long standing thermodynamical image only dreamed of. This application has uncountable uses. Seeking eqaul measure above all else absorbing complexity along the way triangulating deterministism in a frame of reference of space is even hard for people. Historically speaking, they get burned alive by society like a William tyndale in 1531 Brussels. The path to get here was first in man, then machine now computational beast of burden robot horsepower utility cpu serfdom. Lol A long time coming, Such fine-tuned controll of explosions will also have scaling usages in long distance astronomical uses as well.
How on _Earth_ could the guy confuse surface concavity for convexity?, something we were taught in HS: "A concave surface _has the side _*_caved-in_* ". The inside of the bowl he is describing ( 3:05 ) is a _concavity,_ plain and simple.
I did not... You are confusing a convex shape with a convex function. First figure here en.m.wikipedia.org/wiki/Convex_function If you wonder why that function is called convex even though it may look otherwise is because a convex function f(x) is called convex because its epigraph is convex, that is it curves outward. See second figure at the link.
@@RonJohn63 Accepted. I was really just pointing out that the most significant optimisation is that of the reuse of hardware. It's gone from impossible to everyday. Look at what it has enabled SpaceX to achieve: from launch cadence, through to launch cost.
@@TontNZ how much of that is "the math of convex optimizations", and how much of it is "computer hardware (including cameras and other sensors) has finally become advanced enough that reusable boosters finally became possible", and SpaceX were the young turks audacious enough to implement it?
All space agencies, all aircraft companies, all car companies, all industrial robotics companies, all power generation, transmission and distribution companies and agencies, all FAANG companies and you can go on forever, use maths and optimization. In fact, any AI tool currently in use is designed using optimization… for instance during training, via SGD, or hyperparameter optimization. Everything is attempted to be optimized at industrial scale, because if it not, you are not making as much money as you can. This also includes better sensors and hardware. For instance by minimizing surface used on the wafer or reducing power consumption, thus making them smaller and more efficient. So why SpaceX? Because it's something a lot of people relate to at the moment.
“Every so often, we get that cheerful log message: ‘Solver skipping solution as problem is nonconvex.’ It’s like the solver just took a look, shrugged, and said, ‘Yeah, nope, not today.’
Or "Problem does not follow DCP rules" if you use CVXPY
Yes, very interested, please do make the in-depth lectures. We love the technical stuff, the more detail the better! ⚙️ 📈 ✨️
Agreed!
This is a great video! I'm a physicist. I've had to do some challenging control theory and optimization stuff in the past. I wish this would have been covered in grad school. It's such a great way to exercise the mathematical muscles and it's a lot of fun! Getting this to run in real time in a noisy nonlinear non constant delayed environment is so hard... but oddly satisfying! Subscribed!
I am interested in convex optimization and look forward to your series of lectures. Thank you 😊
I will do. I just need to find a bit of time to start. Maybe I will start during the Christmas holiday, but I can't promise. But they will eventually come.
Please make more detailed lectures. Looking forward to it eagerly!
Optimizing is the art of making the right decision using math to find it. Thank you for sharing the video. I hope I get the chance to explore this topic more closely in the future. In the meantime, I'll keep an eye on this channel.
Great description of what Opmization is! Thank you!
The real art of optimization is making a good enough decision very very quickly, very very cheaply.
Certain specifically trained LLMs can give you really close three body problem estimates in a very short amount of time with very small amounts of energy.
You can do things now like gravity capture orbits without spending months trying to backwards work out the answer.
Edit: dyslexia problems
Great. Thanks for sharing
At 13:14 does the convexified cost function have to be larger than the original at every point? I guess it makes sense for hard constraints but what about the soft ones?
You can do both. You can take the convex upper envelope, like in the video and this will give you a minimum which is not necessarily as good as the minimum of the original function. Or you can take the convex lower envelope which lies below the original function. In this case the minimum of the convexified problem may not be achievable by the original function. In other words, you can get an upper bound and a lower bound depending on how you convexify. A similar discussion holds for the constraints, but usually in that case you want to take the inner convex set, so that you know that the obtained point is feasible also for the original problem. Of course this will give a conservative solution (an upper bound on the minimum).
@ Makes sense, thank you!
New KSP tutorial just dropped
Would love a detailed video on closed-loop powered explicit guidance.
Thank you for this suggestion. I will consider this topic for a future video.
Excellent review of a fascinating subject. I have often thought it would make a great video game: the user controls thrust and direction as functions of t (which define a path) and tries to minimize fuel consumption for launching or landing. Most trials will result in failure to reach orbit or crash landings, but with continual fine tuning, success can be achieved. The successful paths can then be compared to the optimal path.
Where can someone find the mathematical details of this?
Hello !
Can you recommend a good textbook on these convex optimisation problem ? In particular about the methods for the convexification of a non-convex problem ?
Thank you for the nice video !
I suggest starting with "Convex Opmization" by Vandenberghe and Boyd. You can download the book from the website of Stephen Boyd. For more advanced tools (non-convex problems), check more material from Boyd, there is a lot available. For implementation I advise CVXPY, but you need to know a bit of Python. There are other versions of CVX, but I personally prefer the Python version.
I doubt he could recommend anything useful, inasmuch his video diagrams demonstrate that he doesn't know the difference between convexity and concavity. 3:10
You are confusing a convex shape with a convex function. First figure here en.m.wikipedia.org/wiki/Convex_function If you wonder why that function is called convex even though it may look otherwise is because a convex function f(x) is called convex because its epigraph is convex, that is it curves outward. See second figure at the link.
@@Prof_Gio thank you for the quick answer!
@@axnj2314that's my favorite book. If you're interested in control theory you're going to want to look at different books/literature.
I would like to see how two factors are taken into account: atmospheric resistance, and attitude control
Thank you Dr Scarciotti, mathematics isnt my strongsuit by any means but your presentation was very enjoyable. Obviously optimization is a robust and significant component. But how does a falcon 9 for instance "know" its orientation in 3d space in order to provide data to the algorithms? And does optimization apply to guidance in zero g environments the same way it operates in the earths atmosphere where more dynamic forces are at play?
Many thanks! The formulations in the video are very simplified versions of what's going on. For each aspect there are teams of people working on that aspect, and the number of constraints is definitely higher. Regarding your first question, the rocket knows its orientation by applying data fusion techniques based on multiple sensor measurements. I touched upon this briefly in my previous video. Regarding the second question, yes optimization is always applied, what changes is the formulation of the problem, so that the equations take into account the specific situation in which the rocket is.
@@Prof_Gio thank you, i look forward to your next video!
Very interesting and fun to know and spread this…
I taught most were non-convex…
Since many equations are in incomplete or unsolvable or simplification (that could have chaotic variations if over simplification)
Yes, many problems are non-convex, which is why I explain the idea of convexification. But yes, those are approximations which have their limitations. That said, convex opmization plays a big role, and there are rumours, although not officially confirmed, that they actually use CVXPY specifically as tool.
Excellent, I will watch your video with interest. I am still struggling with Lambert Solvers and they are not necessarily efficient.
based KSP pfp
Yes! Interested in the gory maths!
From a Senior in aerospace; we’re interested
I’ve always wondered.
Well, mathematical problems are also useful if someone can show they can't be solved fast enough... Which would mean they need to think about a different approach, or bigger margins, or at the end of it all, that it can't be done.
I agree, but maths problems don't even need to be useful in the first place to be interesting. There is a quote from "Mathematician's Apology" by G.H. Hardy that goes "the real mathematics of real mathematicians is almost wholly useless". I am not endorsing this statement, but I think it's an interesting one and the book is a fun read if you don't take it too seriously.
14-19 launches to refuel in orbit for the moon is the definition of optimization.
if you want to send 150 tones to it I guess.
Let's see Paul Alan's payload capacity.
Still cheaper than one SLS engine
We are interested dr 🙏
Calculations depend on 'the tool' used..
Yea nothing is perfect or it could be replaced by pen and paper.
It really is an amazing achievement of a long standing thermodynamical image only dreamed of. This application has uncountable uses.
Seeking eqaul measure above all else absorbing complexity along the way triangulating deterministism in a frame of reference of space is even hard for people. Historically speaking, they get burned alive by society like a William tyndale in 1531 Brussels.
The path to get here was first in man, then machine now computational beast of burden robot horsepower utility cpu serfdom. Lol
A long time coming,
Such fine-tuned controll of explosions will also have scaling usages in long distance astronomical uses as well.
How on _Earth_ could the guy confuse surface concavity for convexity?, something we were taught in HS: "A concave surface _has the side _*_caved-in_* ".
The inside of the bowl he is describing ( 3:05 ) is a _concavity,_ plain and simple.
I did not... You are confusing a convex shape with a convex function. First figure here en.m.wikipedia.org/wiki/Convex_function If you wonder why that function is called convex even though it may look otherwise is because a convex function f(x) is called convex because its epigraph is convex, that is it curves outward. See second figure at the link.
Don't _all_ space launch organization use such optimization?
If you consider throwing their bosters away after a single use, then yes. Otherwise no.
@@TontNZ they certainly have to optimize for launch and for where and when to fire retrorockets so as to land capsules in the desired location.
@@RonJohn63 Accepted. I was really just pointing out that the most significant optimisation is that of the reuse of hardware. It's gone from impossible to everyday. Look at what it has enabled SpaceX to achieve: from launch cadence, through to launch cost.
@@TontNZ how much of that is "the math of convex optimizations", and how much of it is "computer hardware (including cameras and other sensors) has finally become advanced enough that reusable boosters finally became possible", and SpaceX were the young turks audacious enough to implement it?
All space agencies, all aircraft companies, all car companies, all industrial robotics companies, all power generation, transmission and distribution companies and agencies, all FAANG companies and you can go on forever, use maths and optimization. In fact, any AI tool currently in use is designed using optimization… for instance during training, via SGD, or hyperparameter optimization. Everything is attempted to be optimized at industrial scale, because if it not, you are not making as much money as you can. This also includes better sensors and hardware. For instance by minimizing surface used on the wafer or reducing power consumption, thus making them smaller and more efficient. So why SpaceX? Because it's something a lot of people relate to at the moment.