I was playing around with arima for weeks and never got any progress. Thank you so much for this. It's really important and thankful to have a guide along the way!!
Amazing Video! I sometimes play with ARMA models and stock prices to see how it works and indeed after some dense exploratory data analysis oen could combine some "features" in the decision making which increases a lot a the result!! Super cool man, if you could do that more of these videos would be amazin!
As example it's fine, but not for real trading, which has spread between best buy and best sell price, commissions for deals and impossibility to get strict open market price and close price. I spent much time to find something stable method to trade - Q-learn, random forest, ARIMA, cointegration, simple moving averages and their combinations... But it seems that trading is more about risk control, when you simultaneously have several positions, each of specific size depending on volatility and correlation between stock in your portfolio. But now I look at fundamental data like companies financial reports and P/E and so on... The less trading the better results as for me ) I think high frquency trading is something possible to earn with models of order book behaviour, but you need huge amounts of money to get profit (making thousands deals every day with minimal profit each one) and build trading architecture like computers near exchenge and super optimized, well tested code.
To compare the performance of the models you can check the information criterias (bic, Aic) of each model, select the lowest, check the noise level, check the normality of the distribution, p- value of each coefficient, plot the predicted values against real ones, use Error metrics, try arima to integrate the original time serie...some basic stuff I learnt during my DS training I wonder why you didn't do it, or simply suggest it to keep your video short Nevertheless, it was a nice introduction for complete beginners in the field
You know in this model you picked a down trend right i.e. a bear market? It's very difficult for any buying scheme to work in a down market :-) My guess is that the ar(5) model happened to work because it bought less often than the other models. When you buy less, the variance is much greater and you happened to get lucky on the days you bought. Run the same ar(5) on a different stock and it will likely lose in a bear market as well. Which brings me to one big issue. When choosing historical data points, it's obvious you get to choose whether you were in a bull or bear or trending market. If you had happened to choose a bull market, then all of these models would have likely made money. So the key to stock trading is not so much the model, it's when you put your money in :-) Peace
This. ritvikmath is clearly a bright guy, but either he knows very little about the stock market or he chose a misleading use-case for the ARIMA model to get more views. If it was possible to accurately predict the stock market, he would be a very wealthy man and probably not spend time teaching ARIMA on youtube. If it sounds too good to be true, it probably is too good to be true.
Nice video! Is the model doing well partly because we're backtesting it on the same set we ran the acf plot on? Presumably you'd have to split your data to estimate its performance out of sample?
This is really interesting! Instructive but also practical. Love it! In addition, I'm learning time series analysis and is confused about how to determine the orders for ARMA model❓ The way I am doing it now is just trying out different combinations of the orders, with no idea what I am doing. Hope there is a video explaining it
Great video! I have just 1 remark. The language of selected language of autogenerated subtitles is selected incorrectly. It is really helpful for people that learn English using youtube videos and sometimes don't hear exactly your words.
Stock prices follow geometric browning motion process. Btw, just because you have poor returns does not mean you had a "bad" model. Bad in what context? Making money, or bad in terms of your residual not being normally distributed and having p values of your ARIMA above 0.05? P.S I like your chanel and the scientific content you produced previously such as those ARIMA model videos you have mentioned in the video, but I feel videos like this is just a way to get more views for the chanel and catch audience attention. Anyway, I wish you all luck beating the market and if you did, make a proposal to join Renaissance Technologies or compete with them.
Very interesting, good job! However, do you always have to buy and sell every day? Did you also try it for different time periods? If so, how did it go?
Thanks so much for the video!!! I just come across a problem when running the code. When the type of order is tuple, the graphs plotted always show that return=0 and there is no red or green window in the graph. However, when I change the order type to float or change "pred>thresh" in simulation part to "pred
Just curious, why don't you check the model fitting before you applying ARMA(5,5). It's also possible that the model doesn't fit well. And it seems that you use the same model to predict all the coming values, I always wondering if it's more reasonable to update the data when time goes and use the updated model to predict the nearest data point. But that would be not efficient as using the same model.
You should really use very long-term (i.e. several years) rolling origin cross validation. Your current results are likely very highly skewed by randomness.
Thanks your super videos, I am a real fan of all :). I saw in many videos that we produce returns so as to have stationary data. I think in this case we get a white noise which has no autocorrelation (I checked it with Durbin -Watson test getting around 2) and no use for predicting. Can you help if it is true, or if I am wrong?
surely overfitting the model here would lead to fantastic returns as you are testing it on the data you used to make the model? Overfitting generally only leads to problems when you go outside the training data no?
This should not be the case since for each time step he is fitting an ARIMA on data from day 0 to day (step - 1) in his run_simulation function. This is still not the best practice though as each training set is gradually getting larger from 14 days up to the length of his data set, typically you would want to use a fixed size rolling/sliding window with sufficient length to ensure the time series models fit at each window are receiving the same amount of data which will lead to a more informed comparison of the model parameters. He should also be using a validation time period in which the training window will slide to train each time step of an ARIMA structure. This scheme should be used when optimizing the ARIMA parameters before leading up to a final test of the best ARIMA parameters using an out of sample test set which begins after the last datapoint on the validation window.
I know this isn't the point of the video, I'm confused about why the random model would have reliably performed so badly. Was this a period where the entire stock market was in decline? Otherwise it seems like if you did this on average, and didn't account for transaction fees (which it looks like you don't), your return would tend to be slightly positive (since the stock market usually goes up over time). Also wouldn't a simpler baseline be 'for each of the N stocks in our list, put 1/N of our cash into it on day one, then sell them all on the last day'?
hi thanks for the explanation in this great video.. by the way is it possible to combine volume change (stationary data) besides price variable? so that it will be multivariete i hope you could make another video about it..
I absolutely love your videos on time series, but in these plots you really put in a tripwire for the colorblind. Those nuances of red and green are impossible for me to distinguish.
Hey I'm sorry about that and I appreciate you pointing that out. I'll be more conscious of color choices in the future. If you happen to have any references on accessible color schemes I'd love to learn, but will also research this myself.
@@ritvikmath I have not done any research about it, but I know that there are color schemes for color blind used in gaming etc. I am sure you can find them at least as good as I could. Thanks again for your awesome work. You are making really complicated topics immensely clear and digestible.
Can I know how this code helps forecasting the stock? It seems like it is used to analyzed the past stock instead. Sorry for my insufficient knowledge in advance.
In my understanding, ARMA and AR are both built upon linear transformations, right? So why don't we apply some non-linear tech on it to achieve super cool performance such as RNN, LSTM, ATTENTION-LSTM... So the question is whats differences between DL methods and time series models?
Hi thanks for the explanation, However, I have a question, what does it mean AR(5)? Does it mean, if the predicted returns is bigger than the threshold then we will buy the stock the next day and sell it on the following next 5 days?
Okay but if you have a working model, by making a video you’ve lost your edge, and if this isn’t legit then why watch. Either way I’m not gaining much from this.
I am thinking of using scipy.fft() for decomposing the entire stock data and use the base frequencies for the exog of SARIMAX, do you think it would help minimize the mean error?
Really appreciate your effort to teach/demystify all these models! And always going straight to the point. Keep them coming 🙏
Who votes for this has the coolest video on trading with python? I definitely do.
I was playing around with arima for weeks and never got any progress. Thank you so much for this. It's really important and thankful to have a guide along the way!!
Started watching these videos for my Financial Econometrics exam, now hooked on to the practicality of it. Thanks a lot!!!
Great to hear!
Very insightful! Especially the part about overfitting. Definitely something I might’ve overlooked.
you're a great teacher man! really appreciate all the material
I appreciate the kind words!
Great job ritvik! Always a pleasure to watch your videos
Nice, in my senior project we used ARIMA and Prophet to predict stock and bond prices.
Did you make some profits?)
Curious to hear if successful
pls say something bart senpai
As usual great video! Really love your explanation and passion in these!
Glad you like them!
Amazing Video! I sometimes play with ARMA models and stock prices to see how it works and indeed after some dense exploratory data analysis oen could combine some "features" in the decision making which increases a lot a the result!! Super cool man, if you could do that more of these videos would be amazin!
As example it's fine, but not for real trading, which has spread between best buy and best sell price, commissions for deals and impossibility to get strict open market price and close price. I spent much time to find something stable method to trade - Q-learn, random forest, ARIMA, cointegration, simple moving averages and their combinations... But it seems that trading is more about risk control, when you simultaneously have several positions, each of specific size depending on volatility and correlation between stock in your portfolio. But now I look at fundamental data like companies financial reports and P/E and so on... The less trading the better results as for me ) I think high frquency trading is something possible to earn with models of order book behaviour, but you need huge amounts of money to get profit (making thousands deals every day with minimal profit each one) and build trading architecture like computers near exchenge and super optimized, well tested code.
Great and concise video. Trying to implement a similar model using a non-parametric Garch Model.
To compare the performance of the models you can check the information criterias (bic, Aic) of each model, select the lowest, check the noise level, check the normality of the distribution, p- value of each coefficient, plot the predicted values against real ones, use Error metrics, try arima to integrate the original time serie...some basic stuff I learnt during my DS training
I wonder why you didn't do it, or simply suggest it to keep your video short
Nevertheless, it was a nice introduction for complete beginners in the field
Just saw you made many videos on time series. You probably mentioned the model selection methods in on of them.
I keep watching
You know in this model you picked a down trend right i.e. a bear market? It's very difficult for any buying scheme to work in a down market :-) My guess is that the ar(5) model happened to work because it bought less often than the other models. When you buy less, the variance is much greater and you happened to get lucky on the days you bought. Run the same ar(5) on a different stock and it will likely lose in a bear market as well. Which brings me to one big issue. When choosing historical data points, it's obvious you get to choose whether you were in a bull or bear or trending market. If you had happened to choose a bull market, then all of these models would have likely made money. So the key to stock trading is not so much the model, it's when you put your money in :-) Peace
This. ritvikmath is clearly a bright guy, but either he knows very little about the stock market or he chose a misleading use-case for the ARIMA model to get more views. If it was possible to accurately predict the stock market, he would be a very wealthy man and probably not spend time teaching ARIMA on youtube. If it sounds too good to be true, it probably is too good to be true.
Nice video! Is the model doing well partly because we're backtesting it on the same set we ran the acf plot on? Presumably you'd have to split your data to estimate its performance out of sample?
You are 100% correct, this video demonstrates the process, it does not prove a simple ARMA will get you 10% return
Hitting the like button wasn't enough... awesome video! thanks!
this was a really interesting video. can you do more applied videos, as a recommendation.
Amazing video, I will start with ARMA model but I want my main model to be NN model, can not wait to start losing money
This is really interesting! Instructive but also practical. Love it!
In addition, I'm learning time series analysis and is confused about how to determine the orders for ARMA model❓ The way I am doing it now is just trying out different combinations of the orders, with no idea what I am doing. Hope there is a video explaining it
May worth looking at not only long the stock but also short it
Very nice and concise implementation. I will implement in my trading
You are the GOAT
Well planned and presented!!
Could you explain Hierarchical Risk Parity in a video and give an example pls? Congratulations for the great job
Great video! I have just 1 remark. The language of selected language of autogenerated subtitles is selected incorrectly. It is really helpful for people that learn English using youtube videos and sometimes don't hear exactly your words.
Thanks for this informed video.
Stock prices follow geometric browning motion process.
Btw, just because you have poor returns does not mean you had a "bad" model. Bad in what context? Making money, or bad in terms of your residual not being normally distributed and having p values of your ARIMA above 0.05?
P.S I like your chanel and the scientific content you produced previously such as those ARIMA model videos you have mentioned in the video, but I feel videos like this is just a way to get more views for the chanel and catch audience attention.
Anyway, I wish you all luck beating the market and if you did, make a proposal to join Renaissance Technologies
or compete with them.
Awesome! Is there any particular reason why you take the data after the 14th period? 00:31
Very interesting, good job! However, do you always have to buy and sell every day? Did you also try it for different time periods? If so, how did it go?
Thanks so much for the video!!! I just come across a problem when running the code. When the type of order is tuple, the graphs plotted always show that return=0 and there is no red or green window in the graph. However, when I change the order type to float or change "pred>thresh" in simulation part to "pred
I have the same issue
Very nice and tested fine. Windows 11, latest Python version.
Just curious, why don't you check the model fitting before you applying ARMA(5,5). It's also possible that the model doesn't fit well. And it seems that you use the same model to predict all the coming values, I always wondering if it's more reasonable to update the data when time goes and use the updated model to predict the nearest data point. But that would be not efficient as using the same model.
You should really use very long-term (i.e. several years) rolling origin cross validation. Your current results are likely very highly skewed by randomness.
Thanks
Did you do training-test split on the data?
How can you have a 1,000 runs if we only have the luxury to observed one realization of apple stock time series?
Nice Video, It was very clear. I was wondering, have you tried to run a simulation with only a MA(5) model ?
Nice work!
since you made it stationary by using 1st difference, can we say that this is an ARIMA model with I = 1 ?
really appreciate thanks a lot
in forex trading ARIMA/GARCH does not work, becouse all ACF and PACF samples, excluding zero lag, is equal to zero
Thanks your super videos, I am a real fan of all :). I saw in many videos that we produce returns so as to have stationary data. I think in this case we get a white noise which has no autocorrelation (I checked it with Durbin -Watson test getting around 2) and no use for predicting. Can you help if it is true, or if I am wrong?
surely overfitting the model here would lead to fantastic returns as you are testing it on the data you used to make the model? Overfitting generally only leads to problems when you go outside the training data no?
This should not be the case since for each time step he is fitting an ARIMA on data from day 0 to day (step - 1) in his run_simulation function. This is still not the best practice though as each training set is gradually getting larger from 14 days up to the length of his data set, typically you would want to use a fixed size rolling/sliding window with sufficient length to ensure the time series models fit at each window are receiving the same amount of data which will lead to a more informed comparison of the model parameters. He should also be using a validation time period in which the training window will slide to train each time step of an ARIMA structure. This scheme should be used when optimizing the ARIMA parameters before leading up to a final test of the best ARIMA parameters using an out of sample test set which begins after the last datapoint on the validation window.
I know this isn't the point of the video, I'm confused about why the random model would have reliably performed so badly. Was this a period where the entire stock market was in decline? Otherwise it seems like if you did this on average, and didn't account for transaction fees (which it looks like you don't), your return would tend to be slightly positive (since the stock market usually goes up over time).
Also wouldn't a simpler baseline be 'for each of the N stocks in our list, put 1/N of our cash into it on day one, then sell them all on the last day'?
nice
hi thanks for the explanation in this great video..
by the way is it possible to combine volume change (stationary data) besides price variable? so that it will be multivariete
i hope you could make another video about it..
I absolutely love your videos on time series, but in these plots you really put in a tripwire for the colorblind. Those nuances of red and green are impossible for me to distinguish.
Hey I'm sorry about that and I appreciate you pointing that out. I'll be more conscious of color choices in the future. If you happen to have any references on accessible color schemes I'd love to learn, but will also research this myself.
@@ritvikmath I have not done any research about it, but I know that there are color schemes for color blind used in gaming etc. I am sure you can find them at least as good as I could.
Thanks again for your awesome work. You are making really complicated topics immensely clear and digestible.
Can I know how this code helps forecasting the stock? It seems like it is used to analyzed the past stock instead. Sorry for my insufficient knowledge in advance.
Why aren't you considering the first lags of ACF and PACF?
Nice! Collab link?
Linked now!
Hey @ritvikmath, where can I download the notebook?
Linked now!
In my understanding, ARMA and AR are both built upon linear transformations, right? So why don't we apply some non-linear tech on it to achieve super cool performance such as RNN, LSTM, ATTENTION-LSTM... So the question is whats differences between DL methods and time series models?
Great question. Also good timing. I'm planning some videos on exactly this topic soon.
Cool！
any tips for adding a sharpe ratio to the output of the simulations?
If you take the ln of prices does that sorta kinda represent growth?
It might help the stationarity assumption of the data, but it also increases the difficulty when you try to interpret the final model.
Hi thanks for the explanation,
However, I have a question, what does it mean AR(5)?
Does it mean, if the predicted returns is bigger than the threshold then we will buy the stock the next day and sell it on the following next 5 days?
AR(5) indicates an autoregressive model with 5 lags.
what is indicated by the width of the green and red bars in the graphs
Can we fit ARMA to non stationary data though?
What’s threshold
You didn't look at p-values tho...
Would it make sense to do the opposite if the ARMA model shows us losing money, given that if we do the opposite, then we should be making money?
Transaction fees eat up all of those profit already
Hahahaha
i make more percentage gains using simple technical analysis...this mumbo jumbo is making things only complicated
Okay but if you have a working model, by making a video you’ve lost your edge, and if this isn’t legit then why watch. Either way I’m not gaining much from this.
Good job bro, JESUS IS COMING BACK VERY SOON; WATCH AND PREPARE
I am thinking of using scipy.fft() for decomposing the entire stock data and use the base frequencies for the exog of SARIMAX, do you think it would help minimize the mean error?