Hi, can you please explain your question a little bit? Do you want to add an additional time-dependent source/force inside the ODE? In this case no special changes have to be made: Just stick with the descretization and add the pointwise evaluation of your time-dependent source.
In the code given by you, the final desired end effector position is (x,y)=(2,2). Instead of this, if I want to make the end effector to follow a sinusoidal trajectory, say x=0.5sin(wt), y= 0.5cos(wt), what change should I make in the code.@@franksmathematics9779
Hi, i suppose you talk about the Hamilton-Jacobi-Bellmann equation? Thats an interesting topic/question for sure. I have dealt with these equations before and I will definitely put this on the list. At the moment I am preparing a series about optimal control problems and how to solve them. The HJB-equations would definitely fit there!
How could we solve this problem in a reduced order? Since all state variables are a function of the control input u, we could just solve the optimization problem to find the optimum value of u and, consequently, the optimum state values will be derived from the state equations. To this end, the system must have the controllability criterion, which is the case here.
Hi, in the end most of this problems work in the way as you described: Getting rid of the state variable y and reduce this problem to an optimization problem which only depends on the control u. However to do so there is a lot of theory (especially theory of ODEs like solvability, reachability, controllability and so on) which is skipped in this video, as the focus was on the numerical implementation. If you think of a more basic approach (do not use the magic black box fmincon) like a (semismooth) Newton method you have an update step of the form u_{k+1} = u_k + s(u_k) where s is a (more or less complex) function. And here now comes the fun part: To compute s(u_k) you have to solve the ODE, or a linearized form of it - sometimes even multiple times. Thats one way to solve it in a reduced order, I hope this helps. Edit: In my video about the Semi-smooth Newton Method I did a reformulation of a constrained optimization problem (and nothing more is this problem in the video) to a fixed point equation, which is solved by a Newton method. A similar approach can be done here: th-cam.com/video/zOIMH6fNFOA/w-d-xo.html
@@franksmathematics9779 Thanks for your comprehensive reply. I would assume that solving the reduced order problem with fmincon (if, for the sake of simple implementation, my choice is that at the moment) is only possible once we just have the input constraints and not state constraints. In this case, the state equation can be easily integrated into the objective function recursively (I think this approach works). In other cases where we have state constraints or state-dependant constraints on the input, we must also include the states as the decision variables. Am I right?
Yes this is correct. (depending on the type of constraints) Without state constraints your approach should work. You just need a straightforward implementation of a ODE solver (this could be some recursion of course). Are you going to implement this? If so, keep me updated! State constraints, especially inequalities in ODE/PDE optimization are always ugly. As far as I know there is no direct solver or technique how to include them into a reduced formulation. I once wrote a paper how to tackle them with some sort of augmented Lagrange function, which however is again some sequence of optimization problems...
@@franksmathematics9779 I work with LTI state equation and would like to consider different objectives with and without state constraints. I will share the results.
@@franksmathematics9779 Feasible Direction Algorithm handles state-dependant input constraints in the reduced format. The effort is needed to convert state constraints to state depending input constraints
Yeah, maybe you are right - from an educational point of view. However I had this video in my head for quite a while and I really want to do a more complex example. Also, hopefully this video helps some people to tackle complex problems, like how to treat a nonlinear examples and dealing with more than one dimension. Best, Frank
I hope I will finish this video. You have started well.
Thanks! I hope you made it through the whole video! Best, Frank
What change should be made in the code if the reference input is a time-varying signal?
Hi, can you please explain your question a little bit? Do you want to add an additional time-dependent source/force inside the ODE? In this case no special changes have to be made: Just stick with the descretization and add the pointwise evaluation of your time-dependent source.
In the code given by you, the final desired end effector position is (x,y)=(2,2). Instead of this, if I want to make the end effector to follow a sinusoidal trajectory, say x=0.5sin(wt), y= 0.5cos(wt), what change should I make in the code.@@franksmathematics9779
hi, thank you could you please make a video on solving hjb equation in case of optimal control problem (numerically)
Hi, i suppose you talk about the Hamilton-Jacobi-Bellmann equation? Thats an interesting topic/question for sure. I have dealt with these equations before and I will definitely put this on the list. At the moment I am preparing a series about optimal control problems and how to solve them. The HJB-equations would definitely fit there!
@@franksmathematics9779 sure thank you
How could we solve this problem in a reduced order? Since all state variables are a function of the control input u, we could just solve the optimization problem to find the optimum value of u and, consequently, the optimum state values will be derived from the state equations. To this end, the system must have the controllability criterion, which is the case here.
Hi,
in the end most of this problems work in the way as you described: Getting rid of the state variable y and reduce this problem to an optimization problem which only depends on the control u. However to do so there is a lot of theory (especially theory of ODEs like solvability, reachability, controllability and so on) which is skipped in this video, as the focus was on the numerical implementation.
If you think of a more basic approach (do not use the magic black box fmincon) like a (semismooth) Newton method you have an update step of the form
u_{k+1} = u_k + s(u_k)
where s is a (more or less complex) function. And here now comes the fun part: To compute s(u_k) you have to solve the ODE, or a linearized form of it - sometimes even multiple times. Thats one way to solve it in a reduced order, I hope this helps.
Edit: In my video about the Semi-smooth Newton Method I did a reformulation of a constrained optimization problem (and nothing more is this problem in the video) to a fixed point equation, which is solved by a Newton method. A similar approach can be done here:
th-cam.com/video/zOIMH6fNFOA/w-d-xo.html
@@franksmathematics9779 Thanks for your comprehensive reply. I would assume that solving the reduced order problem with fmincon (if, for the sake of simple implementation, my choice is that at the moment) is only possible once we just have the input constraints and not state constraints. In this case, the state equation can be easily integrated into the objective function recursively (I think this approach works). In other cases where we have state constraints or state-dependant constraints on the input, we must also include the states as the decision variables. Am I right?
Yes this is correct. (depending on the type of constraints)
Without state constraints your approach should work. You just need a straightforward implementation of a ODE solver (this could be some recursion of course). Are you going to implement this? If so, keep me updated!
State constraints, especially inequalities in ODE/PDE optimization are always ugly. As far as I know there is no direct solver or technique how to include them into a reduced formulation. I once wrote a paper how to tackle them with some sort of augmented Lagrange function, which however is again some sequence of optimization problems...
@@franksmathematics9779 I work with LTI state equation and would like to consider different objectives with and without state constraints. I will share the results.
@@franksmathematics9779 Feasible Direction Algorithm handles state-dependant input constraints in the reduced format. The effort is needed to convert state constraints to state depending input constraints
Next time, start with easier example
Yeah, maybe you are right - from an educational point of view. However I had this video in my head for quite a while and I really want to do a more complex example. Also, hopefully this video helps some people to tackle complex problems, like how to treat a nonlinear examples and dealing with more than one dimension.
Best, Frank