One awesome thing we can do is to skip certain parts if you knew / don't care about them. I wish I could do this in real life with real professors (lol)
Intro => Most of the lecture, we're going to concentrate on their steady-state behavior. meaning, we're going to look at what does a Markov chain do, if it has run for a long time. What can we say about the probabilities of the different states? So what I would like to repeat is a statement I made last time that Markov Chain is very very useful class of models.Pretty much anything in the real world can be approximately modeled by a Markov chain provided that you set your states in the proper way.
@24:38 When the professor mentions that the system is singular, why does he say it is because (0,0,..0) is a solution of the system. Doesn’t a non-trivial nullspace of matrix A make the equation Ax=0 singular?
Because of two assumptions above, 1.) we have a single or no recurrent class 2.) No periods in the recurrent class This ensure (because n-> infinity) case 1: if 0 recurrent classes than staying at the same state is not possible case 2: if 1 recurrent class no periods so the event has to occur to pull out from recurrent chain So, in both case pi0 is 0, (it does not stay at same state after infinite transitions) by looking at the recurring equation pi0 to pin are all zeros hence, solution is (0,0....0), cause of one row being 0 in the matrix.
39:58 I don't follow this logic. Why do the number of upward transitions need to match the number of downward transitions? Won't it depend on the values of p_i and q_(i+1)?
Can someone what is the relevance of the last plot to Markov process, I see it as the number of person at the queue when time tends to infinity. But how can we relate that kind of definition to a markov process, where we just need the probability of any state ?
"The memory of where you started washes out..". Restatement of mathematical equations like this makes these videos a gem. Thank you, MIT!
MIT is really the hope of human !
spread education all over the world !
Intuitive Ideas are actually helpful to understand the concept. How simply he has explained it.
The most intuitive lecture for Markov Chain that I've ever seen.
Thank you MIT opencourseware, it helps a lot!
Thanks Sir ! I m so glad i ve had the chance to listen to your lecture its so clear and it made me advance in short time
46:47 "It's a very popular model for modeling people who are drunk" lol
Outstanding lecture from Prof. Tsitsiklis
Excellent courses by professor Tsitsiklis .Ευχαριστώ πολύ κύριεΤσιτσικλή
Great lecture. One of my favorite professors.
Thanks for you good work sir, this is the technique which every civilian should keep in their tool bag.
Thanks for MIT. It does help a lot.
One awesome thing we can do is to skip certain parts if you knew / don't care about them. I wish I could do this in real life with real professors (lol)
excellent teaching! it makes complex things easier, break-down the points, better than other online class!.
My exam is the day after tomorrow. Thanks MIT
Absolutely beautiful stuff ...
Very helpful video thank you MIT!!!
Brilliant lecturer! Thanks a mill
I really like his lectures!
Respect to MIT.
very clear !
48:50 1/(1+rho) ~ 1-rho
R K What?
Intro => Most of the lecture, we're going to concentrate on their steady-state behavior.
meaning, we're going to look at what does a Markov chain do, if it has run for a long time.
What can we say about the probabilities of the different states?
So what I would like to repeat is a statement I made last time that Markov Chain is very very useful class of models.Pretty much anything in the real world can be approximately modeled by a Markov chain provided that you set your states in the proper way.
@24:38 When the professor mentions that the system is singular, why does he say it is because (0,0,..0) is a solution of the system. Doesn’t a non-trivial nullspace of matrix A make the equation Ax=0 singular?
Because of two assumptions above,
1.) we have a single or no recurrent class
2.) No periods in the recurrent class
This ensure (because n-> infinity)
case 1: if 0 recurrent classes than staying at the same state is not possible
case 2: if 1 recurrent class no periods so the event has to occur to pull out from recurrent chain
So, in both case pi0 is 0, (it does not stay at same state after infinite transitions)
by looking at the recurring equation pi0 to pin are all zeros
hence, solution is (0,0....0), cause of one row being 0 in the matrix.
Thanks much, this was really helpful
very helpful videos...
39:58 I don't follow this logic. Why do the number of upward transitions need to match the number of downward transitions? Won't it depend on the values of p_i and q_(i+1)?
this time-stamped section clarified my confusion: th-cam.com/video/XKYpKYspe1w/w-d-xo.html
Thanks for iploading
wonderful
Can someone what is the relevance of the last plot to Markov process, I see it as the number of person at the queue when time tends to infinity. But how can we relate that kind of definition to a markov process, where we just need the probability of any state ?
how did he calculate E[X] in Birth-Death Process? , E[X]=rho/(1-rho) ?
Prakash Besra by using the expectation of a geometric random variable minus 1
NO account of twins in Birth-Death