Multivariate Time Series Modeling using Facebook Prophet
ฝัง
- เผยแพร่เมื่อ 22 ก.ค. 2024
- #timeseries #datascience #facebookprophet
For univarate time series on this dataset check this video - • Time Series Modeling u...
Link to my TIme Series playlist - • Time Series Modelling ...
Prophet is a procedure for forecasting time series data based on an additive model where non-linear trends are fit with yearly, weekly, and daily seasonality, plus holiday effects. It works best with time series that have strong seasonal effects and several seasons of historical data. Prophet is robust to missing data and shifts in the trend, and typically handles outliers well - วิทยาศาสตร์และเทคโนโลยี
Beautiful! Clear, concise and most perfect explanation out there on multivariate fbprophet out there. Thank you so much!
Thanks for these excellent videos on fb prophet. Your clear explanations on the process and how to interpret the output are extremely helpful.
this is one of the best lectures.....you're a Genius...
great Job !!
this should be featured more in the search results, great level of depth. thanks lots
Thank you, this made it so clear. Excellent tutorial !
Thank you for the video a very good one!
I only see one problem with the CV process, you have a leaking problem. When you use the dummy variable for the temperature you are using the info that is t+1 so obviously, you will have a lower estimation error. As you say, it is better to simulate this point using a Montecarlo, but it is not going to resolve this problem.
Best.
Thanks for informative video. I am clear why you created summer temp, but could you explain a little bit more why you created month bin?
Thanks a lot for this!! In the future prediction, for the multivariate case, are you still predicting the demand values?
Thanks a lot for this video .Is it possible to perform multistep in Fbprophet like in LSTM windowing? So that desired intervals can be set and forecasted
Great explanation!! Thanks.
hi ..thanks for the nice video..one doubt..can we deploy ml models on cloud foundry as well?
Hi,
For Multivariate, do we need future values to predict with prophet?
Thanks for this tutorial. Please did you later teach on Monte Carlo Simulation?
@AIEngineering, how to explain what feature is how much impactful?
what does extra regressive chart explains about. Please say something
Can you please tell me if fbprophet is impacted by mutlicollinearity among variables?
Very good, thank you!
Thank you, very useful.
Thank you for the excellent video! Found it quite informative. I have a question: the data that you used is NYC electricity consumption (1 city). What if my dataset has multiple cities with similiar-ish weather, e.g. I have a column that specifies the cities in addition to the demand, precip and temp? When I feed the dataset to Prophet, should the dataset be split by the different cities first?
Yes. You are into multiple time series problem. You might have to build individual time series for each city. You can check my Multiple time series video on how you can accomplish it easily
Hello dear , i tried all steps following the tutorial but unfortunately faced the below error without a clear resolution even after searching the internet a lot. C
error
ufunc 'isnan' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''
Does the binning process necessary?
Can u please do video for multivariate using DeepVAR in gluonTS
Hi, can we use fbprophet for forecasting #defects by categories (like hue parameter/color in tableau) instead of total #defects per day ?
yes you can so your are here talking about multiple time series as each category will be a time series. Check my multiple time series video and see if it helps you
Thanks for the explanation. I had one query, how can we give numeric values as an input to the model without converting them into binary numbers?
You mean for additional regressors. You can still pass it directly
Hey ! thanks for the vid ! What if we wanna train multiple time series at the same time ? Like, if we wanna predict the temperature in different parts of a city! The same way the Random Forest models, for example !
I have lot of videos on multiple time series. We can create one model for each time series and I have mentioned it here - th-cam.com/video/AO8pCAQ0sPw/w-d-xo.html
Also have videos on deepAR in bbn my channel that allows one to build single global model
where can i find python code file?
Hi Srivatsan,
This is really great content. Thanks for your efforts.
As a data science professional, am expected to build more explainable multivariate time series models. Could you please make videos on ARIMAX & SARIMAX to explain more about multivariate explainable models? That world really help. There's no good content related to these on TH-cam rn as per my exploration
Have you checked my VAR model which covers it for multivariate - th-cam.com/video/TpQtD7ONfxQ/w-d-xo.html
ARIMAX I do not have it yet and will do it in future
Yeah thanks, gone through the same.Just a suggestion, if you can cover ARIMAX/SARIMAX, that would add more value for this well curated time series playlist.
Yes Pavan.. will do it. That is the only one missing in my playlist. I wanted to cover it end to end and was looking for good dataset as well. But will be doing it
Thank you!
Thank you for such wonderful video, will Prophet handles better if we have less data points as well? I have roughly 35 data points monthly and i need a monthly forecast rather daily so can I use prophet to do that?
Mouli.. If the data has all trends, pattern, seasonality covered in 35 data points, it might work. Said that more data the better forecast. You can use less data points to maybe perform one forecast at a time by iterating, rather predicting too long in future. I would say give it a try and see
Kindly do video on VAR model
Thank you very much for the wonderful video.
I have 4 columns (X variables), in that 1 is date, 2nd and 3rd are String values, 4th is Float value and prediction is Y is Float
How should I handle 2nd and 3rd columns which are in string.
Girish.. Depends on nature of string data and its importance you can convert to numerical encoding or any other encoding and pass it as addition regressors
Hello Girish, did you find the solution?
Hi,
Quite an informative video! I had a question that is : Why did we feed in the summer_temp and bins values as regressors and not the original temp values as regressors?
Was that done to tackle seasionality or to give a rough estimate of temp and phase of year?
Thank You!
Deyashini.. Main reason is if you have seen the graph most of the year the average temperature is stable and our uni-variate model was not able to model extreme temperature. From EDA in the video you might have noticed the extreme points are temp > 80 and also in specific month. Since prophet treats the regressor as additive linear model it might not be able to model temp variable directly and hence breaking it down
@@AIEngineeringLife Thank you so much for the explanation, makes total sense!
I also wanted to ask if we can use SARIMAX or VAR with exogenous variables for the same problem. I can take my questions to another platform if you'd prefer that.
Thank You!
Yes you can model it as exogenous variable. It should work on this data as this data is more towards stationary and does not have trend
thank you for the video.
Does Prophet work with categorical variable as well? I have a time series where daily sales is highly depends on number of articles in inventory, weekday and the the inventory locations ( let say different cities). If I make dummy variables for weekdays and locations then can I implement Prophet to make a sale forecast?
Yes it will. You can have encoded categorical variables as additional regressors. In this video as well I am using summer temp which is a categorical value
@@AIEngineeringLife after performing one hot encoding I have around 400 columns. can I pass all of them as additional regressors? if yes do I need to manually add each column or is there any way to take all the columns as additional regressors in one go?
@@anjan700 400 i would not recommend.. see if these 400 contribute really to outcome or try out other techniques like feature hashing and add
@@AIEngineeringLife do yo have a separate video for feature hashing? can I bring the COVID scenario as a indicator for my 2021 prediction. if yes, how do I do it and how can I evaluate my model accuracy?
Sir,
I want to use FB Prophet for forecasting in milliseconds(Network traffic data analysis), does it performs well? or is it poor on millisecond level ? Can I get the link to code used in this video ?
If the data has predictive pattern based on past then it must be able to handle millisecond level as well. Said that is you see at millisec level data does not have lot of variance it is better to resample it to higher time
My git repo is here - github.com/srivatsan88/End-to-End-Time-Series
Thanks alot for the video and effort and time it require for making the content.it would be great if you can clear my query :
1)As we have divided years in three parts and assigning them 0,1,2 so my quest is wont model give more priority to the months which are assigned value as 2 in comparison to 0 an 1 ?
2)If we derive further features from already existed feature ,like we did with temp wont it cause overfitting???
Kindly help me to clear these doubt and pls suggest how to further enhance the learning in timeseries.
Coming to your second question first. In my case demand was only variable that was fed into model, so temperature is fine. If I am adding engineered feature say demand again then I need to see if that variable will create spurious relationship
In this case priority is fine as I want to weight months based on higher demand to model. But for use cases where it has to be treated equal it is good to have one hot encoded done
Can u please tell how to install prophet in anaconda
I am trying conda install - c conda-forge fbprophet it is taking huge time.
Can you please specify how Can I do the pip or conda installation or any requirement for this
Zaid.. what you did is right way?. I do not see any other way of doing it. I would say if conda install is taking time switch to conda env and just do pip install for this package alone
Hi Srivatsan,
This is the best series on Time Series Forecasting on the Internet. Thanks a lot!
I have a problem that I hope to get some guidance.
I have some additional regressors that I want to use to forecast my main time series using FB Prophet.
The issue is that I don't have future values for those.
Can you please let me know some possible directions I can go from here. I have tried forecasting the regressors as univariate first and then using them for my main time series but, it doesn't make sense to estimate something and then use it to estimate some other thing. Kindly shed some light.
Ali.. Typically and unfortunately that is the way to go. You typically get or predict future value of regressor and then use in regular model. Now if you need to predict all together you can look at VAR model. You can check my video on VAR model as well
Thanks sir, I didn’t go for VAR because I can not lose seasonality. Its business requirement. Thank you for your guidance. I can safely pursue forecasting the regressors now 👍👍👍
Can u share the files and code
Hi i am trying to use prophet to predict more than one output variable, but they affect each other, so in your case, it would be like predicting demand and temperature together. is the possible in fbprophet?
Jordon.. I doubt there is a way in prophet. Say in my case you have to build 2 models one that predicts temp and use output from that model to predict demand. Multivariate time series reason is hard as more variable the more models we have to build to predict each future variable. There are ways in LSTM to predict multiple variable but the output may have to backtested to make sure it is reliable
Thanks for the reply.. I will look up some RNN, LSTM for this
@@jordon603 you can check my lstm time series videos. I do not have exact same problem but it can be easily extended
can you please share the Notebook? looks great Video
Hello sir,Can we fit ARIMA model on multivariate time series?
You can use ARIMAX models for it
@15.06 my_df_final['summer']= ((my_df_final.temp>80)+0)
Better than. adding a function and calling the apply function .
Hi, I have a usecase where i need to forecast sales, profit and revenue all together for the given timeline(lets say for next 12 months) i have monthly dataset for last 5 years. can i use fbprohpet for my usecase ?
You might have to create multiple models here. I would suggest you go for VAR model where the model takes care of creating multiple equations for you
@@AIEngineeringLife thanks for responding but I need clarification whether fb-prophet is designed for such task ? I’ve been asked to do it use prophet 😜
I think I got my answer from your other videos comments, I have one quick question what will be the alternatives for VAR & VARMA for the above case ?
@@manikandanjeyabal3389 .. Rather designed for that task, is it possible?.. Yes.. Even VAR does nothing but multiple equation
In fb prophet you have your target prediction column in list, iterate and forecast that column. Problem here is each model is assumed separate. I may be not aware of other way though if one is there
In some models you can do multi target prediction at once but in fb prophet above is what I am aware
@@AIEngineeringLife thanks @AIEngineering.
Please add a link to the notebooks.
You can find link to all my repo here - github.com/srivatsan88
This one against my time series course
Can we also add more than 2 variables?
Yes you can add as many variables. You can also check my VAR model video where I cover tests to highlight if a time series variable will be useful or not
Can we get link to your github for the code??
All video related code are here - github.com/srivatsan88
Hi Sir, have you worked on optimal reconciliation?
Can you elaborate the scenario here to understand better please
@@AIEngineeringLife hierarchical time series!! When there are different levels of aggregation you need to forecast...
Eg: Sales
Sales at national level
Sales at state level
Sales at District level
Did you set up fbprophet in windows? if yes how did you resolve dll error?
Mouli, Nope I have only tried on Linux not windows
@@AIEngineeringLife it seems installing it in windows is cumbersome unless if you use conda.
@@moulich7372 yes and maybe because of stan library that is C underneath
@@AIEngineeringLife yes it requires lots of c native environments and compilers
Can you please share the link to your notebook?
Here you go - github.com/srivatsan88/End-to-End-Time-Series/blob/master/Time_Series_using_Prophet_for_Multivariate.ipynb
@@AIEngineeringLife thanks
you didn't showed how to predict future data, model trained and tested well on the dataset given how to predict future data points?
Mouli.. I have covered it in my univariate videos and that is same for all.. In this video I am showing prediction but just that i did not have out of time test but process is same
@@AIEngineeringLife ok , for multivariate time series model while predicting future values how will we know regressor values ? We need to pass those values right?
@@moulich7372 .. Yes and thats where the challenges come in. I am covering it in one of my upcoming videos as well. So if say the future values are externally available like temperature, inflation etc then we can pull from alternate providers but if it is internal then one has to build separate model to forecast regressors and use it in main model. Now forecasted regressor might not be perfect and this is where monte carlo or other simulations come into play to get close to accurate forecast
@@AIEngineeringLife thank you for your detailed response, i am eagerly waiting for the upcoming videos, lets say if those values were external then in order to predict next 3 months of forecast how good those values if we pull from alternate providers?
@@moulich7372 say If u r pulling weather from outside and you belive it might fluctuate 20% either sides then u r running simulations on this weather +_ 20% and getting confidence band for your main model forecast. And this you have to do each variable combo and among them. Reason TS is hard to model
The title of this video is a bit misleading. Here you are simply including multiple variables as regressors. I clicked on this title because I wanted to know about forecasting when there is more than one dependent variable (vector time series regression).
Micheal.. if you are looking multi output prediction you can check my VAR model video here - th-cam.com/video/TpQtD7ONfxQ/w-d-xo.html
I am sorry if this title is misleading as typically if you see many literature online as well many consider multiple input variables as multivariate. So I had both variant of it :)
@@AIEngineeringLife no worries, its a good video none-the-less!
th-cam.com/video/4Mkne5AmlgY/w-d-xo.html
Totally confused video, why making everything so complex