First of all, great series of videos, you describe complex designs in a great and very understandable way. Thank you! I believe that you should clarify in the video that you talk about "Deterministic" FSM vs BT. I would like to know your opinion about non-deterministic FSM vs BT. Which are the pros and cons in that comparison?
Hi Christian. Thanks for your question. The reactivity needed to handle interrupts is built into BTs (as long as you are using BTs that include the running return status, not the odd version that has only Success/Failure). So, for instance you might have the following BT: ((In Safe Area) ? (Goto Safe Area)) -> (Do other stuff) This would constantly be checking if the agent is in the "Safe Area". If so it would "Do other stuff", but as soon as the agent is not in the "Safe Area" it would immediately execute "Goto Safe Area", resulting in "In Safe Area" returning true at some point, and the agent returning to "Do other stuff". An example scenario might be when the fire alarm turns on, then a whole building might be considered unsafe, and the agent would stop doing what it did and exit the building, before doing something else. You might want to check out this video: th-cam.com/video/DCZJUvTQV5Q/w-d-xo.html (at 3:15) where reactivity is discusses.
@@petterogren7535 thank you. Do you need then to keep a reference to the current running action to abort it in a nice fashion. For example if eating the banan and the alarm goes off. Then I can't just stop eatin the banan I first have to drop it before reacting to the alarm. And if I was eating an apple, I first have to drop that? For reference I was watching this talk about Behavior Tree th-cam.com/video/Qq_xX1JCreI/w-d-xo.html
@@TodiloDaSloth Yes, good question! This might be a good idea, depending on circumstances. A biped robot that is running cannot just switch to speaking to a human passing by, without terminating the run with a proper slow down and stop action. But you might be able to stop eating an apple and leave the house with the apple in the hand. The whole issue of "cleaning up" after activities might need to be considered. If an agent is eating breakfast to satisfy hunger, does it make sense to put the juice back in the fridge before running off to catch the bus? Is it a goal in itself to have a "tidy" house? Or should needs just be taken care of as they appear.
@@petterogren7535 Thanks for the input, I will continue my search on the web for more Behavior Tree examples. Thanks for your videos, much informative!
We stopped using bt in our game for decision making, and implemented hfsm for that. We start use bt for list of commands, that need to be executed at some state(goto, play anim, deal damage, recover, etc), and that solve our problems with AI.
I am not sure how to interpret the part about "FSM doesn't depend on internal state". Is this a theoretical statement? Is it something to try to achieve in production code? This is clearly not true from what I have seen. Trivial example: When being attacked usually a unit will fight back, unless it is returning for repairs, in which case it will just continue its route...
Hi Denis. Thanks for your question. I think you refer to the slide at 6:36 where I say that "State feedback is optimal... it does not depend on the previous action (the internal FSM-state)". So, in your example we want a unit to fight back when being attacked. This is probably true no matter if the unit was resting, marching, eating or training. Thus, this transition (to "fighting" when "being attacked") has to be added to all these four different states (resting, marching etc). This is an invitation to making errors. If we just miss this transition in one place (say "resting"), we will have an agent that keeps resting, even though it is being attacked. In a BT solution you would just have one check of "being attacked" and the response "fight back", that would take care of this no matter what your agent was currently doing. So FSMs do depend on the internal state (i.e. which of the FSM states it is currently executing) and this can be a source of problems. Please let me know if I understood your questions correctly.
@@petterogren7535 Your argument doesn't hold for his example. He wants it to fight unless it is returning for repairs. In the BT solution you would have to remember that there is such a state as "returning for repairs" and add that as an exception to your check, right? If we just miss this exception when writing the check, you would end up with the robot fighting while he shouldn't. In FSMs however you would write out the reaction to all events in all states, so you are less likely to choose the wrong behavior for "being attacked" while "returning for repairs".
@@MauricevanderPot As "state" has two meanings in this context I will use the terminology "world-state" to describe the state the world is in (including the agent), usually denoted by 's' in reinforcement learning and 'x' in control theory, and "FSM-state" to describe which of the "actions/behaviors" the FSM is in. I think the key thing is that your choice of action should be based on the world-state, not the FSM-state. So if you are returning for repairs (FSM-state) and not able to fight (world-state), you should not fight because of the world-state. But if you are returning for repairs (FSM-state) in terms of some scheduled maintenance (world-state) (oil change perhaps) you can and probably should fight. Does this clarify what I was trying to say?
I never had problem programming without State Machine or Behavior Trees, even for complex programming, I find it very unnecessary, people defend FSM by saying it saves you from writing many Booleans to execute something to not override other execution, so they use state machine, but at the end you will use each one of those Booleans the only difference is they are separated, for example if you are in a falling state, you can go from Falling to Walking , Idle , Swimming , Sliding , Attacking, so at the end you will end up with the same amount of Booleans, I've tried to use State Machine in a complex character controller, but at the end I deleted it all because it just add complexity and extra work, for people doing it on youtube and teach you how to make a state machine they all use Walk Idle Jump as example, which can work perfectly with any state, but later when adding like multiple states it will be a nightmare.
The advantage of FSMs is that you don‘t have to manage boolean valus. You only need to store the current state and then add the transitions to each state. E.g., if my player is in the state „on_wall“, the only transitions you need to add to it are „falling“ and „wall_jump“. There is no need to set any boolean values which makes FSMs much less error-prone and more scalable than manually handling boolean values.
Edit: Regarding multiple states: you simply build another state machine. For example, you could have one state machine for movement and another state machine for combat. Each state machine knows the current state of the other state machines. Thus, for example, if the player is in the state „on_wall“, the combat state machine is not allowed to transition to an attack state.
Internally screaming after seeing all those lines float up and connect each action to each other action. That looks like a hell of a lot of work.
The comparison and criticism about RL are just amazing. Looking forward to seeing the improvement of RL methods with behavior trees.
This was a great lecture series. Thank you for making it!
I'm glad you liked it! /Petter
Very nicely explained. Thanks.
First of all, great series of videos, you describe complex designs in a great and very understandable way. Thank you!
I believe that you should clarify in the video that you talk about "Deterministic" FSM vs BT.
I would like to know your opinion about non-deterministic FSM vs BT. Which are the pros and cons in that comparison?
Grating voice and ponderous delivery.
How do you solve interrupts /abort in a behavior tree? So if you are walking to you goal but get interrupted before you reach your move goal?
Hi Christian. Thanks for your question. The reactivity needed to handle interrupts is built into BTs (as long as you are using BTs that include the running return status, not the odd version that has only Success/Failure). So, for instance you might have the following BT:
((In Safe Area) ? (Goto Safe Area)) -> (Do other stuff)
This would constantly be checking if the agent is in the "Safe Area". If so it would "Do other stuff", but as soon as the agent is not in the "Safe Area" it would immediately execute "Goto Safe Area", resulting in "In Safe Area" returning true at some point, and the agent returning to "Do other stuff".
An example scenario might be when the fire alarm turns on, then a whole building might be considered unsafe, and the agent would stop doing what it did and exit the building, before doing something else.
You might want to check out this video: th-cam.com/video/DCZJUvTQV5Q/w-d-xo.html (at 3:15)
where reactivity is discusses.
@@petterogren7535 thank you. Do you need then to keep a reference to the current running action to abort it in a nice fashion. For example if eating the banan and the alarm goes off. Then I can't just stop eatin the banan I first have to drop it before reacting to the alarm. And if I was eating an apple, I first have to drop that?
For reference I was watching this talk about Behavior Tree th-cam.com/video/Qq_xX1JCreI/w-d-xo.html
@@TodiloDaSloth Yes, good question!
This might be a good idea, depending on circumstances. A biped robot that is running cannot just switch to speaking to a human passing by, without terminating the run with a proper slow down and stop action. But you might be able to stop eating an apple and leave the house with the apple in the hand. The whole issue of "cleaning up" after activities might need to be considered. If an agent is eating breakfast to satisfy hunger, does it make sense to put the juice back in the fridge before running off to catch the bus? Is it a goal in itself to have a "tidy" house? Or should needs just be taken care of as they appear.
@@petterogren7535 Thanks for the input, I will continue my search on the web for more Behavior Tree examples. Thanks for your videos, much informative!
We stopped using bt in our game for decision making, and implemented hfsm for that. We start use bt for list of commands, that need to be executed at some state(goto, play anim, deal damage, recover, etc), and that solve our problems with AI.
I am not sure how to interpret the part about "FSM doesn't depend on internal state".
Is this a theoretical statement? Is it something to try to achieve in production code?
This is clearly not true from what I have seen.
Trivial example: When being attacked usually a unit will fight back, unless it is returning for repairs, in which case it will just continue its route...
Hi Denis. Thanks for your question. I think you refer to the slide at 6:36 where I say that "State feedback is optimal... it does not depend on the previous action (the internal FSM-state)". So, in your example we want a unit to fight back when being attacked. This is probably true no matter if the unit was resting, marching, eating or training. Thus, this transition (to "fighting" when "being attacked") has to be added to all these four different states (resting, marching etc). This is an invitation to making errors. If we just miss this transition in one place (say "resting"), we will have an agent that keeps resting, even though it is being attacked. In a BT solution you would just have one check of "being attacked" and the response "fight back", that would take care of this no matter what your agent was currently doing.
So FSMs do depend on the internal state (i.e. which of the FSM states it is currently executing) and this can be a source of problems.
Please let me know if I understood your questions correctly.
@@petterogren7535 Your argument doesn't hold for his example. He wants it to fight unless it is returning for repairs. In the BT solution you would have to remember that there is such a state as "returning for repairs" and add that as an exception to your check, right? If we just miss this exception when writing the check, you would end up with the robot fighting while he shouldn't. In FSMs however you would write out the reaction to all events in all states, so you are less likely to choose the wrong behavior for "being attacked" while "returning for repairs".
@@MauricevanderPot As "state" has two meanings in this context I will use the terminology "world-state" to describe the state the world is in (including the agent), usually denoted by 's' in reinforcement learning and 'x' in control theory, and "FSM-state" to describe which of the "actions/behaviors" the FSM is in. I think the key thing is that your choice of action should be based on the world-state, not the FSM-state. So if you are returning for repairs (FSM-state) and not able to fight (world-state), you should not fight because of the world-state. But if you are returning for repairs (FSM-state) in terms of some scheduled maintenance (world-state) (oil change perhaps) you can and probably should fight.
Does this clarify what I was trying to say?
Is there anywhere we can discuss things with you like discord?
Hi Tyler. My webpage is here www.kth.se/profile/petter/. If you send me an email, we can take it from there.
I never had problem programming without State Machine or Behavior Trees, even for complex programming, I find it very unnecessary, people defend FSM by saying it saves you from writing many Booleans to execute something to not override other execution, so they use state machine, but at the end you will use each one of those Booleans the only difference is they are separated, for example if you are in a falling state, you can go from Falling to Walking , Idle , Swimming , Sliding , Attacking, so at the end you will end up with the same amount of Booleans, I've tried to use State Machine in a complex character controller, but at the end I deleted it all because it just add complexity and extra work, for people doing it on youtube and teach you how to make a state machine they all use Walk Idle Jump as example, which can work perfectly with any state, but later when adding like multiple states it will be a nightmare.
The advantage of FSMs is that you don‘t have to manage boolean valus. You only need to store the current state and then add the transitions to each state.
E.g., if my player is in the state „on_wall“, the only transitions you need to add to it are „falling“ and „wall_jump“. There is no need to set any boolean values which makes FSMs much less error-prone and more scalable than manually handling boolean values.
Edit: Regarding multiple states: you simply build another state machine.
For example, you could have one state machine for movement and another state machine for combat.
Each state machine knows the current state of the other state machines.
Thus, for example, if the player is in the state „on_wall“, the combat state machine is not allowed to transition to an attack state.
It kind of sounds like BTs are the perfectly refactored form of an optimal FSMs…
The volume is low and it ruins the video. Please take care of this issue.