I've watched tons of math/data science videos before but always find something missing..finally settled on this channel! continue the great work Ritvik!
Man… you are amazing! Please never stop teaching this way… this is pure didactic… Could you explore more videos on how to treat the outliers after detection as you showed with the mean approach from the previous video? Or the mean calc would be just enough to remove the outlier?
As a data scientist, these techniques I found very helpful, because in real data you need to deal these issues, i would request him to add more videos from machine laerning from his experince , because his way is explain to so simple, I binge watched complete playlist.
Nice video! In this example it might be best to do a multiplicative decomposition by taking a log first, which should get rid of the heteroscedasticity you see in the seasonal and residual components.
Absolute Gold content.. Thank you for making this video. Also, can you please make a video on how LOESS work? It will help me in understanding what is going on behind the scene. Thanks, Bhuvanesh
Great video. One clarification - When creating the lower and upper limit thresholds, wouldn't you want to use the mean and standard deviation of the residuals UP TO (but not including) each point. Otherwise you are using future information, which wouldn't allow you to determine if an event is an outlier in real time.
Hello RitvikMath. Those thing seems so easy for you ..I envy you!! I have one big question for you. Is there a way to use Time series to pick the most "suitable" outcome in a set of finite possible outcomes we have. Not predict but find.
Briliant. I am new to data science and timeseries analysis in general, and i find this video very useful. My question is, how can I quantify or assess the 'strength' of my data's seasonality? more specifically, I want to write a code that automatically detects if my data is seasonal or not. an example of a dataset i want to work on is hourly temperature fluctuations (high temp during the day, low temp during the night). How can I automate a test that checks whether this data follows a seasonal trend or not, and if so at what frequency? Thanks!
Thank you so much for the video! But I have a question on this. What if I want to make a daily anomaly detection check program? Should I calculate the standard deviation for the entire period every time? Or should I set a window size to calculate for the specific amount of time period?
Thanks for wonderful Video on STL . I just want to ask you, I am working on real world dataset.When I try to decompose time series into trend and seasonal components, I need to specify period value, but you didn't do that. what is that?
Once you've identified these anomalies would you consider doing anything with them as they could affect your forecast? Would you consider removing the data points? Or maybe replacing them with something else?
Amazing work bro! I got a quick question. If you want to account for future data, and you want to set an anomaly detection threshold, how can you do it? You can't really use lower and upper from the code as lower gives u a negative value. How can we get the actual numbers from the data? Thx!
is small residuals (remainder) a good thing? I sometimes found stl, x11, seats retrurn different scalse of seasonality and remainders. How to evaluate a decomposition model? Thanks
How do we configure to detect the Anomalies in real time (as soon as possible).. Will there be a delay in identifying if say yesterday's data had an anomaly for a daily frequency dataset
I often get confuse when interpreting the seasonal_decompose plot. How can we certainly know if the seasonality is daily, weekly, monthly or yearly? I would appreciate a lot your answer
@@komalsaini5668 I used my own dataset so so I skip this part, but maybe you could calculate the frequency with a groupby().count() instead of infer_freq
Hey, ice_cream_interest = ice_cream_interest.asfreq(pd.infer_freq(ice_cream_interest.index)) this set the column `interest` to NaN Do not know what I'm doing wrong.
@johnysmith1375 , this will work: ice_cream_interest = pd.read_csv('./data/ice_cream_interest.csv') ice_cream_interest["month"] = pd.to_datetime(ice_cream_interest["month"]) # add this line ice_cream_interest.set_index('month', inplace=True) ice_cream_interest = ice_cream_interest.asfreq(pd.infer_freq(ice_cream_interest.index))
very few teachers can be addictive and u r one among them... completed the whole playlist in just 2 days... kudos to u...
I've watched the whole 36 videos of this playlist and they were really really helpful..THANK YOU SO MUCH SIR !!
Agree with everyone else, absolute gold. Keep it on, cheers!
Thanks!
I've watched tons of math/data science videos before but always find something missing..finally settled on this channel! continue the great work Ritvik!
You are actually the man, I wish you good fortune my friend
my favorite video about outliers. Absolutely simple, amazingly useful and so easy to implement! Thank you so much!
Man… you are amazing! Please never stop teaching this way… this is pure didactic…
Could you explore more videos on how to treat the outliers after detection as you showed with the mean approach from the previous video? Or the mean calc would be just enough to remove the outlier?
As a data scientist, these techniques I found very helpful, because in real data you need to deal these issues, i would request him to add more videos from machine laerning from his experince , because his way is explain to so simple, I binge watched complete playlist.
Glad it was helpful!
This is Gold GOLD, I take it back it's Platinum
Lovely small video but worth a million likes
thanks!
Big fan of your videos! Keep it up~
Thanks!
Nice video! In this example it might be best to do a multiplicative decomposition by taking a log first, which should get rid of the heteroscedasticity you see in the seasonal and residual components.
Absolute Gold content..
Thank you for making this video. Also, can you please make a video on how LOESS work? It will help me in understanding what is going on behind the scene.
Thanks,
Bhuvanesh
Thanks! And yes I am planning to make that LOESS video soon.
Such a great video, thanks for sharing it! It would be lovely if you can continuing creating this content, and explain LOESS.
Thank you! Will do!
Great video. One clarification - When creating the lower and upper limit thresholds, wouldn't you want to use the mean and standard deviation of the residuals UP TO (but not including) each point. Otherwise you are using future information, which wouldn't allow you to determine if an event is an outlier in real time.
yes great point !
@Shikamaru23... Can you explain how to do this
A quick search shows Ben and Jerry's released a limited batch of Chocolate Cherry Garcia ice cream in late 2016.
Great video! Would love to see a LOESS explanation :)
Thanks for the video, especially the visuals. Though, do you know any similar command in Stata?
Hello RitvikMath. Those thing seems so easy for you ..I envy you!! I have one big question for you. Is there a way to use Time series to pick the most "suitable" outcome in a set of finite possible outcomes we have. Not predict but find.
Amazing explanation !!
Briliant. I am new to data science and timeseries analysis in general, and i find this video very useful. My question is, how can I quantify or assess the 'strength' of my data's seasonality? more specifically, I want to write a code that automatically detects if my data is seasonal or not. an example of a dataset i want to work on is hourly temperature fluctuations (high temp during the day, low temp during the night). How can I automate a test that checks whether this data follows a seasonal trend or not, and if so at what frequency? Thanks!
Great video, great information, appreciate your help
You are a genius! Thanks for the insight.
i need this video but done in R, great video thank you
Can't find any info on how to set "sesonal" and "period" parameters
Good video, make one for cyclic decomposition
Woow its so clear now!!!!. THANK YOU!!!
Thank you so much for the video! But I have a question on this. What if I want to make a daily anomaly detection check program? Should I calculate the standard deviation for the entire period every time? Or should I set a window size to calculate for the specific amount of time period?
Thanks for wonderful Video on STL . I just want to ask you, I am working on real world dataset.When I try to decompose time series into trend and seasonal components, I need to specify period value, but you didn't do that. what is that?
How about time series in which the more general trend changes, goes up for 6 months, goes down for 4 months, then up again for 5 months, etc?
This video is amazing...
Crystal clear...thanks for the video...can you please make a video of LOESS theory part? Thanks in advance
Once you've identified these anomalies would you consider doing anything with them as they could affect your forecast? Would you consider removing the data points? Or maybe replacing them with something else?
This is cool thanks. Is there a way to get the cyclicity as well in this package?
Does STL decomposition works as an additive model ?
Hi @ritvikmath, do you trade stocks using time series analysis?
Yes! I'm planning to soon make some videos on how I choose stocks.
@@ritvikmath awesome!
@@ritvikmath hi Ravikant , I really learnt good stuff from your previous videos. I really love watching time series .
@@ritvikmath Can't wait!
Amazing work bro! I got a quick question. If you want to account for future data, and you want to set an anomaly detection threshold, how can you do it? You can't really use lower and upper from the code as lower gives u a negative value. How can we get the actual numbers from the data? Thx!
could u pls cover robust pca for anomaly detection in timeseries, heard its really effective at large scale
is small residuals (remainder) a good thing? I sometimes found stl, x11, seats retrurn different scalse of seasonality and remainders. How to evaluate a decomposition model? Thanks
How do we configure to detect the Anomalies in real time (as soon as possible).. Will there be a delay in identifying if say yesterday's data had an anomaly for a daily frequency dataset
Bro.. Any help would be great!
I often get confuse when interpreting the seasonal_decompose plot. How can we certainly know if the seasonality is daily, weekly, monthly or yearly? I would appreciate a lot your answer
Amazing, thank you so much
Excellent
I used it in a production delivery, lol
@@komalsaini5668 I used my own dataset so so I skip this part, but maybe you could calculate the frequency with a groupby().count() instead of infer_freq
Hahaha nice event called "ice cream is gay"
bro your videos are super easy to understand pls marry me no homo
Ice-cream is gay )))) . Not the central point of the video but anyways, also an anomaly.
can I make my data stationary by removing the trend part for instance? I mean if my data is a called df the do: df - df.trend (in a pseudo code)
when i use plt.plot() to print the base plot it says unhashable type: 'numpy.ndarray'
Hey,
ice_cream_interest = ice_cream_interest.asfreq(pd.infer_freq(ice_cream_interest.index))
this set the column `interest` to NaN
Do not know what I'm doing wrong.
Hi, I am facing the same issue. Did you find a solution?
@@somyatripathi6164 Try loading the csv with "ice_cream_interest = pd.read_csv('ice_cream_interest.csv',parse_dates=[0], index_col=0)".
@johnysmith1375 , this will work:
ice_cream_interest = pd.read_csv('./data/ice_cream_interest.csv')
ice_cream_interest["month"] = pd.to_datetime(ice_cream_interest["month"]) # add this line
ice_cream_interest.set_index('month', inplace=True)
ice_cream_interest = ice_cream_interest.asfreq(pd.infer_freq(ice_cream_interest.index))