NOTE: at 4:20, pandas data reader has been deprecated!! So, the logic when obtaining your data will change. Thus, I have updated the Jupyter notebook here: github.com/SpencerPao/Pairs_Trading At 9:40 2 standard deviations are 95%. 1.25 standard deviations is ~90%
Should note: you can have a cointegrated pair with zero correlation. Correlation does not imply cointegration or rather an underlying relationship which influences the process of each time series in a distinct way. The relationship is what we are trading, or in other more specific terms, the statistical deviation from a cointegrated spread.
Yes, people do use returns for calculating hedging ratios in real life.. but I guess the reason he used "Price" is it is easier to create visualizations using Pricing data.
What is your rationale for doing TH-cam instead of developing a trading model? Do you view career or TH-cam as a more predictable way to increase earnings or do you doubt that it's possible to develop such model? Ignore question confines if needed, any thoughts appreciated.
An excellent question -- I actually never really put that much thought into creating quant models; I am more into Long only trades with longer time horizons. It may be an interesting venue to explore. It essentially boils down to time -- I only have so many hours in the day, and I recently became very, very busy. There's a lot more to that story but those are the high level points.
Hi Spencer, this is a fantastic channel! Your explanations are some of the clearest I've seen. When you have the buying and selling signals, that's on the specific asset you declared you were going to be long or short one, correct? I would love a video on how to implement the actual buying/selling part of the strategy and backtesting if you're taking recommendations! Thanks again Spencer
To answer your first question -- Yes indeed! I'm glad that you enjoyed this video. :) I'll dabble in the back testing / actual implementation of the algorithm via a brokerage. and see if I can make a video quality walkthrough. Note that the back testing is an art form; many of the testing algorithms are in house. I'm sure that there are some open source backtesting algo's to use.
Hey Spencer, thankyou so much for the video!! This is my first video on pairstrading and I finally understood the idea. However I have a few doubts especially at 8:55 , when you calculate the mean and then find your price signals. When you do that does that not cause a look ahead bias for older signals? Like we really never knew it was the mean at the beginning of the datapoint right? I'm sorry for the dumb question, but if you can clarify it, that would help me a lot! Thankyou!
There are no dumb questions! :) There are inherent assumptions when you go and utilize the pair trading algorithm. (i.e what time horizon you want to choose) Bias is an inherent factor with most if not all time-series particular model. There are some mays to mitigate that bias: 1) you can have normalized models in relation to the market over differing economic trends (recession, depression, growth etc..) and see how that compares. 2) Have additional features that is more representative of the data you are seeing now 3) Data augmentation etc. Hopefully that answered your question?
Spencer, very interesting video. Please note that there is an “overfitting” problem in your concept. You are using a prior knowledge regarding the correlation of these two instruments. In other words, the fact that these two stocks are correlated in the period of time that you are analyzing is not necessarily going to be true in the future (and you are analyzing the performance of your method only for the past signal… that was used to determine if the stocks are correlated in the first place). Take care
You're absolutley correct! This is an assumption that there is an inherent dependency in this method and your positions will inherently become more risky the longer you hold onto a specific position.
I talk with others who are in the field. Honestly, the 'best' way to become a quant trader is to do it yourself. And, if you want to join a huge fund, you will have an edge over everyone else because you were an actual practioner. You learn by failing over and over.
Sir, In pars trading can one trade spread between SP500 & Nasdq 100 based just on market sentiment of broad market/major indices.When sentiment is bullish,Nasdq100 would outperform SP 500 index.And in bearish scene vice versa.Same may be applied for pairs trading SP500 & Dow 30.In weak market Dow 30 would hold better & go down less compared to SP500/nasdq100.We seek your wisdom.Thank you.
Sir, Have you tried pair trading forex using rsi7 ,rsi14,rsi30 (add them up for comparison) say on hourly chart & selling strong pair & buying weak pair--pairs have to be highly correlated(eg aud/usd and nzd/usd OR dow30 & sp500).One can do this on any correlating underlying stocks/commodities/futures/crypto/bonds. Trading on hourly charts there would be tons of opportunities all year around.
Any thoughts on rolling z-scores or z-z-scores (z-scores of z-scores when distributions may be not as mean reverting) and testing for effective lookback periods without data mining and information creep?
You most definitley can do a rolling z-score of z scores approach. (In fact that approach can be an extension of what I did in this video.) In terms of back testing, that is definitley more of an artform that I am not 100% super familiar with. Many companies have their own built in backtesting. But, you can look at it from the point of view of specific time horizons in the stock exchange to judge how well your model will fare in economic upturns or dowturns. You don't have to have real time artificially generated data that represents a specific trend. At the end of the day, if you get good predictions, then by all means go for it! :)
Goodnight ! Do you have any video on your channel about the Pair Trading strategy but instead of long x short, does it work in the form long & long or short & short? Note: congratulations for the channel.
Hello! I do not have a video on that topic. However, what you are stating is quite similar to the pair trading algorithm I discussed. Just switch the logic. I.e if asset A and asset B were to decrease (high correlation), you can spread your bets between the 2 asset classes.
Sir, In pair trading correlated currency pairs(say AUD/USD & NZD/USD) do you compare RSI value & sell overpriced pair & buy the weaker? OR do you compare CCI of each pair? Do you suggest using correlated cross currencies as pair trades(eg EUR/JPY & GBP/JPY)? On 4H chart sum up RSI 3, RSI 7,RSI 14, & compare added value on each pair & go. Thank you.
You can try any of those -- whichever has the best signals, go for it. Regardless of which relationships you seek, the pairs trading principle applies to any.
@@ronaknathwani5142 Hmm. Those are ticker symbols I assume? You can run any tool that you are comfortable with. (by tools do you mean software?) Check out code in C. Those are typically the fastest language in the quant world. If you mean which theories to use, you should test whichever one in your overall strategy to get your edge. Perhaps even use a combination of them or some other strategy that I am unfamiliar with.
@@ronaknathwani5142 Sir, On one chart plot RSI 3, RSI 7, RSI 14 ( say using MT4) & write values of each & add the numbers for ES NQ & YM--and compare.Please sell overpriced & buy the underpriced. System Programs use RSI 14, but as you know RSI 3 is the best guide to price & RSI 7 is more recent than RSI 14--so we have better picture of strength/weakness mtf.You need no program--just paper & pencil.
Ah LQE? Few folks are aware of that haha. Estimating position(s) is actually incredibly interesting based on uncertain features. That would be an entirely different video altogether (and not necessarily related to Pairs Trading) I have thought about using Kalman filter on an ARIMA model, and after reading a paper, the filter does provide an incremental improvement. I might do a video on the topic but not in the near future.
Hey great video Spencer. I have a doubt at the end when we get the buy and sell signals, so say the buy signal is the trigger we got, now do we buy BRK or buy MSFT? how do we decide? Also how is it a pair strategy, we are only buying or selling only one stock at a time?
Mathematically and theoretically, you can go long on X and short Y. The direction of either one depends on what the model suggests. Also, from the pairs strategy (X,Y), you would typically standardize the pair i.e X would be buy and Y would be sell
Hi Spencer thanks for the video. why did you choose 1.25 sd ? 1 sd from the mean covers 68% of the data I agree, but If you want to cover 95% of the data, you should choose 2 sd, considering a normal distribution. Thnaks
I tried to run it but sadly, the signal is not correct. As you can see on the graph u presented, the signal is false as well. Any other way to detect peak and bottom of a graph ?
I am curious about something, aren't we having a lookahead bias here when doing the tests to determine the pairs to trade, like say in one 1, we wont have info from the future right to tell us if this pair is cointegrated. How do you think we can tackle this?
Bias is huge. And, I don't talk about penalization. One way to do this is through continuous backtesting and revalidating the model (reweighting and reupdating the model) OR even having a different model altogether.
Yes! I was essentially using the ratio between two different assets based on the historical data. And, based on the base model: rolling averages, I depend on the historical data to have a prediction in the future. (In essence, using in sample data for predictions) It is a basic model where the typical machine learning principles are not at play.
Hi ! I tried to use your code however there is an error. From what I understand, it can't access the data from yahoo finance correctly. Are you experiencing the same issue ?
Yes indeed! Thank you for letting me know! Yahoo finance has changed their backend so the functions I go over in this video are deprecated. However, I have updated my notebook to adjust for this. You can check it out on my Github here: github.com/SpencerPao/Pairs_Trading
It depends on your use case. If speed is not an issue, you can resort to scraping data from Yahoo Finance with Python/R. Those languages should suffice.
Most likely. If you are able to query data from the Binance APi without much limit, you can do the pairs trading algorithm. It's just a matter of latency at that point.
Algo system? Not really. For deployments for trading algorithms, you can look into using interactive brokers via their api. Plug in your model and stream their data. They are quite hefty in costs though.
@@amriksingh5003 Hmm. Well, unfortunatley everything has some elment of code. Though, you could just run my notebook locally from github and plug in your pair of public companies (for example) manually purchase/sell equity on some exchange.
Hey, thanks for the amazing content, great video. I have one python-related technical question: by using pandas 1.2, the statsmodels doesn't work. I get this error when trying to import statsmodels.tsa.stattools: "module 'pandas' has no attribute 'Panel'". It seems the panel has been removed from Pandas module 0.25.0 onwards. How did you overcome this problem? Thanks in advance. Keep up the good work!
@@ВиталийДягилев-ж5я ooh. I think you're referring to scraping financial asset data? I have not attempted to scrape from binance, so I am not familiar with the tooling, but you can set up price scraping mechanisms on Yahoo Finance's API. I did a similar video on this topic: th-cam.com/video/GGgNM7WanK8/w-d-xo.html Or this video (includes Yahoo finance data): th-cam.com/video/wlaLmM_LvWg/w-d-xo.html
dude, you gotta get rid of this line in your code "Cointegration test: A technique used to find a potential correlation in a time series (long term)". This is *absolutely* not what it is. But you have mistaken correlation for causality the entire way through this video so I'm not surprised. Cointegration actually establishes a causal link from one time series to another" Peace bro. But please don't say the purpose of cointegration is to find correlations. Nothing is further from the truth :-)
Hi Spencer, this is pure gold. Thank you for sharing so much knowledge with us for free :)
A rather complicated concept made concise and very comprehensive. Excellent!!
Great video, and with Python code too!!
Thank you for making this clear explanation, and for making it available to your viewers
Very well explained thanks!
NOTE: at 4:20, pandas data reader has been deprecated!! So, the logic when obtaining your data will change. Thus, I have updated the Jupyter notebook here: github.com/SpencerPao/Pairs_Trading
At 9:40 2 standard deviations are 95%. 1.25 standard deviations is ~90%
i think i found a rare gem, thanks for the video, keep up the good work
Highly underrated channel!
I love the support :')
Should note: you can have a cointegrated pair with zero correlation.
Correlation does not imply cointegration or rather an underlying relationship which influences the process of each time series in a distinct way.
The relationship is what we are trading, or in other more specific terms, the statistical deviation from a cointegrated spread.
You could use the returns instead of price. And then the ratio between retturns of both assets
Yes, people do use returns for calculating hedging ratios in real life.. but I guess the reason he used "Price" is it is easier to create visualizations using Pricing data.
i love this video Spencer!
I would also look at cash indices, i quite like DAX vs Nasdaq. Especially as a spreadbet in which u are not exposed to CCY flutuations.
Brilliant!
What is your rationale for doing TH-cam instead of developing a trading model? Do you view career or TH-cam as a more predictable way to increase earnings or do you doubt that it's possible to develop such model? Ignore question confines if needed, any thoughts appreciated.
An excellent question -- I actually never really put that much thought into creating quant models; I am more into Long only trades with longer time horizons. It may be an interesting venue to explore. It essentially boils down to time -- I only have so many hours in the day, and I recently became very, very busy. There's a lot more to that story but those are the high level points.
Hi Spencer, this is a fantastic channel! Your explanations are some of the clearest I've seen. When you have the buying and selling signals, that's on the specific asset you declared you were going to be long or short one, correct? I would love a video on how to implement the actual buying/selling part of the strategy and backtesting if you're taking recommendations! Thanks again Spencer
To answer your first question -- Yes indeed!
I'm glad that you enjoyed this video. :)
I'll dabble in the back testing / actual implementation of the algorithm via a brokerage. and see if I can make a video quality walkthrough.
Note that the back testing is an art form; many of the testing algorithms are in house. I'm sure that there are some open source backtesting algo's to use.
@@SpencerPaoHere that would be fantastic, thank you! Thanks again for the great content - really appreciate all the effort
Awesome video l. Thanks
Amazing Video!!!
great work man! great work!
Michael, your tactics are incredible Which one are you recommend for the beginners?
Rich in content
You are making it sound sooo easy hahah
Hey Spencer, thankyou so much for the video!! This is my first video on pairstrading and I finally understood the idea. However I have a few doubts especially at 8:55 , when you calculate the mean and then find your price signals. When you do that does that not cause a look ahead bias for older signals? Like we really never knew it was the mean at the beginning of the datapoint right? I'm sorry for the dumb question, but if you can clarify it, that would help me a lot! Thankyou!
There are no dumb questions! :)
There are inherent assumptions when you go and utilize the pair trading algorithm. (i.e what time horizon you want to choose)
Bias is an inherent factor with most if not all time-series particular model. There are some mays to mitigate that bias: 1) you can have normalized models in relation to the market over differing economic trends (recession, depression, growth etc..) and see how that compares. 2) Have additional features that is more representative of the data you are seeing now 3) Data augmentation etc.
Hopefully that answered your question?
Spencer, very interesting video. Please note that there is an “overfitting” problem in your concept. You are using a prior knowledge regarding the correlation of these two instruments. In other words, the fact that these two stocks are correlated in the period of time that you are analyzing is not necessarily going to be true in the future (and you are analyzing the performance of your method only for the past signal… that was used to determine if the stocks are correlated in the first place). Take care
You're absolutley correct! This is an assumption that there is an inherent dependency in this method and your positions will inherently become more risky the longer you hold onto a specific position.
thank you alot
Awesome Video! How did you gain the knowledge that you have? What's the best way for an aspiring Quant Trader to gain all these knowledge from basics?
I talk with others who are in the field.
Honestly, the 'best' way to become a quant trader is to do it yourself. And, if you want to join a huge fund, you will have an edge over everyone else because you were an actual practioner. You learn by failing over and over.
Sir, In pars trading can one trade spread between SP500 & Nasdq 100 based just on market sentiment of broad market/major indices.When sentiment is bullish,Nasdq100 would outperform SP 500 index.And in bearish scene vice versa.Same may be applied for pairs trading SP500 & Dow 30.In weak market Dow 30 would hold better & go down less compared to SP500/nasdq100.We seek your wisdom.Thank you.
very good video
Conceptually wrong. You should check if the pairs are cointegrated or not. Correlation does not imply cointegration.
Can you explain it more
Sir, Have you tried pair trading forex using rsi7 ,rsi14,rsi30 (add them up for comparison) say on hourly chart & selling strong pair & buying weak pair--pairs have to be highly correlated(eg aud/usd and nzd/usd OR dow30 & sp500).One can do this on any correlating underlying stocks/commodities/futures/crypto/bonds. Trading on hourly charts there would be tons of opportunities all year around.
I have not! You can definitely find many correlations -- the trick perhaps is to find causailty.
Buying what ? And selling what ??
Any equities !
are these trading algorithms "a nice to have" or something that you would ned to properly lidentify pairs?
Are trading past if you have not rolled Z scores. You may have false signal with heteroskedasticity even in a pair of two non homoskedastic series.
Any thoughts on rolling z-scores or z-z-scores (z-scores of z-scores when distributions may be not as mean reverting) and testing for effective lookback periods without data mining and information creep?
You most definitley can do a rolling z-score of z scores approach. (In fact that approach can be an extension of what I did in this video.)
In terms of back testing, that is definitley more of an artform that I am not 100% super familiar with. Many companies have their own built in backtesting. But, you can look at it from the point of view of specific time horizons in the stock exchange to judge how well your model will fare in economic upturns or dowturns. You don't have to have real time artificially generated data that represents a specific trend.
At the end of the day, if you get good predictions, then by all means go for it! :)
Goodnight ! Do you have any video on your channel about the Pair Trading strategy but instead of long x short, does it work in the form long & long or short & short?
Note: congratulations for the channel.
Hello! I do not have a video on that topic. However, what you are stating is quite similar to the pair trading algorithm I discussed. Just switch the logic. I.e if asset A and asset B were to decrease (high correlation), you can spread your bets between the 2 asset classes.
Sir, In pair trading correlated currency pairs(say AUD/USD & NZD/USD) do you compare RSI value & sell overpriced pair & buy the weaker? OR do you compare CCI of each pair? Do you suggest using correlated cross currencies as pair trades(eg EUR/JPY & GBP/JPY)? On 4H chart sum up RSI 3, RSI 7,RSI 14, & compare added value on each pair & go. Thank you.
You can try any of those -- whichever has the best signals, go for it. Regardless of which relationships you seek, the pairs trading principle applies to any.
Hi Spencer what tools do I need to run the similar test as above , I’m trading ES NQ and YM correlated pairs and like to see the stats for divergence?
@@ronaknathwani5142 Hmm. Those are ticker symbols I assume? You can run any tool that you are comfortable with. (by tools do you mean software?) Check out code in C. Those are typically the fastest language in the quant world.
If you mean which theories to use, you should test whichever one in your overall strategy to get your edge. Perhaps even use a combination of them or some other strategy that I am unfamiliar with.
@@ronaknathwani5142 Sir, On one chart plot RSI 3, RSI 7, RSI 14 ( say using MT4) & write values of each & add the numbers for ES NQ & YM--and compare.Please sell overpriced & buy the underpriced. System Programs use RSI 14, but as you know RSI 3 is the best guide to price & RSI 7 is more recent than RSI 14--so we have better picture of strength/weakness mtf.You need no program--just paper & pencil.
Thx for the content!
Do you plan by any means to make a video of pair trading but with the use of kalman filter instead of moving averages?
Ah LQE? Few folks are aware of that haha.
Estimating position(s) is actually incredibly interesting based on uncertain features.
That would be an entirely different video altogether (and not necessarily related to Pairs Trading)
I have thought about using Kalman filter on an ARIMA model, and after reading a paper, the filter does provide an incremental improvement.
I might do a video on the topic but not in the near future.
Sir can you help me with adftest for pair trading
Hello, just wanted to know how you calculate the spread mine is all weird. Great video
The spread (which I believe your are mentioning at 6:17) is calculated by BRK_B - MSFT which is the difference between adjusted closing prices.
Hey great video Spencer. I have a doubt at the end when we get the buy and sell signals, so say the buy signal is the trigger we got, now do we buy BRK or buy MSFT? how do we decide? Also how is it a pair strategy, we are only buying or selling only one stock at a time?
Mathematically and theoretically, you can go long on X and short Y. The direction of either one depends on what the model suggests.
Also, from the pairs strategy (X,Y), you would typically standardize the pair i.e X would be buy and Y would be sell
Hi Spencer thanks for the video. why did you choose 1.25 sd ? 1 sd from the mean covers 68% of the data I agree, but If you want to cover 95% of the data, you should choose 2 sd, considering a normal distribution. Thnaks
Hi! Yeah me choosing 1.25 sd was arbitrary. It does make a lot of sense to do 2 sd.
@@SpencerPaoHere ok :) and a z-score of 1.25 is at the ~90th percentile right, not 95th, as you mentioned. Thats why I got confused.
I tried to run it but sadly, the signal is not correct. As you can see on the graph u presented, the signal is false as well. Any other way to detect peak and bottom of a graph ?
I am curious about something, aren't we having a lookahead bias here when doing the tests to determine the pairs to trade, like say in one 1, we wont have info from the future right to tell us if this pair is cointegrated.
How do you think we can tackle this?
Bias is huge. And, I don't talk about penalization. One way to do this is through continuous backtesting and revalidating the model (reweighting and reupdating the model) OR even having a different model altogether.
Good explanation ... but you use In Sample data? Really more complex to apply in real world trading when you need to use Out Sample data.
Yes! I was essentially using the ratio between two different assets based on the historical data. And, based on the base model: rolling averages, I depend on the historical data to have a prediction in the future. (In essence, using in sample data for predictions)
It is a basic model where the typical machine learning principles are not at play.
Hi ! I tried to use your code however there is an error. From what I understand, it can't access the data from yahoo finance correctly. Are you experiencing the same issue ?
Yes indeed! Thank you for letting me know!
Yahoo finance has changed their backend so the functions I go over in this video are deprecated. However, I have updated my notebook to adjust for this. You can check it out on my Github here: github.com/SpencerPao/Pairs_Trading
@@SpencerPaoHere that’s great, thank you !!!
At 9:40, doesn't 95% of the data lie between 2 standard deviations, not 1.25?
Yes. You are correct. I've updated my pinned comment and it was also asked in an earlier comment as well.
Thanks. Any good tool or means to check stock co-integration?
It depends on your use case. If speed is not an issue, you can resort to scraping data from Yahoo Finance with Python/R. Those languages should suffice.
Do you think this can be done using AUDUSD vs. EURAUD?
Hi! Most definitely! Although, there are a lot of other features that are associated with forex, Pairs Trading can apply to any sort of asset classes!
Pairs Trading belongs to Mean Reversion or Statistical Arbitrage?!!!
Mean reversion.
This is more in line with stat arb. th-cam.com/video/TNJNCJe9fDY/w-d-xo.html
Wouldnt this work better with forex pairs?
Not sure. I haven't tried. Though, I bet a lot of folks may use a version of pairs trading in any type of financial market.
Any chance to have the same written for crypto In Binance API?
Most likely. If you are able to query data from the Binance APi without much limit, you can do the pairs trading algorithm. It's just a matter of latency at that point.
can get this code?
I uploaded the notebook here: github.com/SpencerPao/Pairs_Trading
audio quality is not good
Hi do you have the algo system for pair trading I can use. How much it costs ?
Algo system? Not really.
For deployments for trading algorithms, you can look into using interactive brokers via their api. Plug in your model and stream their data.
They are quite hefty in costs though.
@@SpencerPaoHere thanks for your prompt response. I m not programming guy.
@@amriksingh5003 Hmm. Well, unfortunatley everything has some elment of code. Though, you could just run my notebook locally from github and plug in your pair of public companies (for example) manually purchase/sell equity on some exchange.
Hey, thanks for the amazing content, great video.
I have one python-related technical question: by using pandas 1.2, the statsmodels doesn't work. I get this error when trying to import statsmodels.tsa.stattools: "module 'pandas' has no attribute 'Panel'". It seems the panel has been removed from Pandas module 0.25.0 onwards. How did you overcome this problem?
Thanks in advance.
Keep up the good work!
Yeah. I'd recommend setting up a separate environment for that specific version of pandas and other dependencies for this specific overview.
Update: with the latest anaconda version (released in May 2022), everything works perfectly, there was no need to install a specific pandas version.
hi tell me how to do it for the cryptocurrency market?
You could think of it in terms of replacing the assets with coins. Same process.
@@SpencerPaoHere I can’t load data market from binance
it doesn't work there
@@ВиталийДягилев-ж5я ooh. I think you're referring to scraping financial asset data? I have not attempted to scrape from binance, so I am not familiar with the tooling, but you can set up price scraping mechanisms on Yahoo Finance's API. I did a similar video on this topic: th-cam.com/video/GGgNM7WanK8/w-d-xo.html
Or this video (includes Yahoo finance data): th-cam.com/video/wlaLmM_LvWg/w-d-xo.html
@@SpencerPaoHere oh thanks, I think I found it
dude, you gotta get rid of this line in your code "Cointegration test: A technique used to find a potential correlation in a time series (long term)". This is *absolutely* not what it is. But you have mistaken correlation for causality the entire way through this video so I'm not surprised. Cointegration actually establishes a causal link from one time series to another" Peace bro. But please don't say the purpose of cointegration is to find correlations. Nothing is further from the truth :-)
Why so complicated ? Make money is easy .its simpl .
There's a reason you make TH-cam videos instead of working in the industry. Your implementation and programming is garbarge.