Man, if it was not for your guide I may not be able to complete my final year project, many thanks for the guide on Custom ARIMA model, I will make sure to give credits on your channel!!
Could you please share the csv datra format so that I can have similar data format that I can work? I did search atmospheric air composition but have different format and could not able to fit those data and follow the steps. Sample of your dataset would be great help for me. Thanks
Your videos are quite detailed. I'm checking if my objectives for my research project are achievable. I'm using ARIMA model to predict drought pattern in an area and my data is in pixel(raster) format not point format, Can the model handle pixel data. Kindly assist
@Tech Know How i greatly appreciate your all videos you have uploaded for all of us. lots of work and time dedicated just to help us. thank you. i was able to re-produce your entire code in to R-studio however the results of the forecast generated is same for different datasets . how is it possible that ARIMA model of forecasting giving us same results when we run the code on two seperate data sets. something is really wrong with your code or either how i am re-produced code is not correct. i am sending you the code in the post
Nothing wrong with my code used in the video - they are used at thousands of colleges and universities throughout the world (including Harvard, MIT, Stanford, Johns Hopkins, Cambridge University, etc.). I use these processes myself all the time. As a matter of fact, I just used this ARIMA process yesterday for an executive at my company. BHU.csv is one of your datasets. We would have to look at your data. Also, I am not sure what results you are speaking of? Graphs of forecast? Well, RStudio will only hold so many graphs before it gets stuck on one. Might want to try clearing out the graphs if that is the case. If it's the accuracy or forecast results then there is something wrong with your process. There is no way 2 different datasets will give exact same scores for MAPE and other results. Also, what was the auto arima pdq values, p values and end accuracy values? A lot of the code and values are missing (~ 90%)? Be sure and follow the videos and do it step by step. If you want me or one of my many subscribers to look at this for you when we have some extra time, be sure and place the data files where we can download them (kaggle, github, etc.)
Also, as any good data scientist knows, there is good data and there is bad data. Bad data can have lots of bias, duplicate records, bad analogies, missing data, etc...You cannot just take any dataset and run it through processes like this without first knowing a little about the data. Where was it collected from? Is it updated? When was the last time it was updated? Checked for issues? From the little bit I can see in the code you did post, you are trying to forecast 572 periods (h=572 below). That is a heck of a long forecast. Here is your code with the forecast periods: #forecast new fit model fit2 for h=30 periods (30 days) fcast
Hello I have a dataset with hourly data for the last 5 years i.e. 8760 points of electricity prices for each year, of the format dd-mm-yyyy hh:mm:ss. Could you tell me how i can what line of code should i use to convert it into the proper date time format and move forward with the arima modeling?
Great question. You have several choices. R is very easy to work with and change formatting of dates and other data types. First, you can always just reformat the data in Excel and re-import it. If you want to stay in R, you can use the format() function or Lubridate. Check out this webpage here that shows you how to change formatting in all sorts of ways. www.aridhia.com/technical-tutorials/working-with-dates-in-r/ Here's another interesting package called anytime which does just that. It allows you to convert any format to the correct date and/or time format you want. dirk.eddelbuettel.com/code/anytime.html
What does it mean if for my data, the cleaned data, the monthly MA and weekly MA are all the same when graphed? They are literally on top of one another and I’m not sure why. Is that supposed to happen?
That means you have zero variance over time which is not realistic. Or if it is at one point in time, it is a point of major inflection or change. Usually right after a point like that there is a major up or downswing in the graph and all related metrics. Think beginning point or a major recession, major bull run, etc.
Great content @Tech Know How!! I was wondering what is the reason for creating moving average data? can we not use the original? And on another note, would it not have been more fitting to use a SARIMA method? Instead of removing the seasonality.
@@techknowhow4802 Hello, please i'm using ARIMA and other statistical model to make price forecast and i'm having some issues, can you give me your contact so that i discuss this with you?
Man, if it was not for your guide I may not be able to complete my final year project, many thanks for the guide on Custom ARIMA model, I will make sure to give credits on your channel!!
Could you please share the csv datra format so that I can have similar data format that I can work? I did search atmospheric air composition but have different format and could not able to fit those data and follow the steps. Sample of your dataset would be great help for me. Thanks
i realize it's kind of off topic but does anyone know a good site to watch newly released tv shows online?
@Ishaan Ayden i would suggest Flixzone. Just search on google for it =)
Your videos are quite detailed. I'm checking if my objectives for my research project are achievable. I'm using ARIMA model to predict drought pattern in an area and my data is in pixel(raster) format not point format, Can the model handle pixel data. Kindly assist
Hi, the x axis doesn’t represent the actual date. This continues trough the hole series. How can I change that? Thank you sir
@Tech Know How i greatly appreciate your all videos you have uploaded for all of us. lots of work and time dedicated just to help us. thank you.
i was able to re-produce your entire code in to R-studio however the results of the forecast generated is same for different datasets . how is it possible that ARIMA model of forecasting giving us same results when we run the code on two seperate data sets. something is really wrong with your code or either how i am re-produced code is not correct. i am sending you the code in the post
Nothing wrong with my code used in the video - they are used at thousands of colleges and universities throughout the world (including Harvard, MIT, Stanford, Johns Hopkins, Cambridge University, etc.). I use these processes myself all the time. As a matter of fact, I just used this ARIMA process yesterday for an executive at my company. BHU.csv is one of your datasets. We would have to look at your data. Also, I am not sure what results you are speaking of? Graphs of forecast? Well, RStudio will only hold so many graphs before it gets stuck on one. Might want to try clearing out the graphs if that is the case. If it's the accuracy or forecast results then there is something wrong with your process. There is no way 2 different datasets will give exact same scores for MAPE and other results. Also, what was the auto arima pdq values, p values and end accuracy values? A lot of the code and values are missing (~ 90%)? Be sure and follow the videos and do it step by step. If you want me or one of my many subscribers to look at this for you when we have some extra time, be sure and place the data files where we can download them (kaggle, github, etc.)
Also, as any good data scientist knows, there is good data and there is bad data. Bad data can have lots of bias, duplicate records, bad analogies, missing data, etc...You cannot just take any dataset and run it through processes like this without first knowing a little about the data. Where was it collected from? Is it updated? When was the last time it was updated? Checked for issues? From the little bit I can see in the code you did post, you are trying to forecast 572 periods (h=572 below). That is a heck of a long forecast.
Here is your code with the forecast periods:
#forecast new fit model fit2 for h=30 periods (30 days)
fcast
Hello
I have a dataset with hourly data for the last 5 years i.e. 8760 points of electricity prices for each year, of the format dd-mm-yyyy hh:mm:ss. Could you tell me how i can what line of code should i use to convert it into the proper date time format and move forward with the arima modeling?
Great question. You have several choices. R is very easy to work with and change formatting of dates and other data types. First, you can always just reformat the data in Excel and re-import it. If you want to stay in R, you can use the format() function or Lubridate. Check out this webpage here that shows you how to change formatting in all sorts of ways. www.aridhia.com/technical-tutorials/working-with-dates-in-r/
Here's another interesting package called anytime which does just that. It allows you to convert any format to the correct date and/or time format you want.
dirk.eddelbuettel.com/code/anytime.html
@@techknowhow4802 Thank you for this really helpful tip!
What does it mean if for my data, the cleaned data, the monthly MA and weekly MA are all the same when graphed? They are literally on top of one another and I’m not sure why. Is that supposed to happen?
That means you have zero variance over time which is not realistic. Or if it is at one point in time, it is a point of major inflection or change. Usually right after a point like that there is a major up or downswing in the graph and all related metrics. Think beginning point or a major recession, major bull run, etc.
Great content @Tech Know How!!
I was wondering what is the reason for creating moving average data? can we not use the original?
And on another note, would it not have been more fitting to use a SARIMA method? Instead of removing the seasonality.
Where to get this data from?
It probably also tells you where in the video. Most of the videos I tell you where the dataset is from if it is publicly available.
I wasted hours of time searching for this data. Please provide the site where I can find it as well.
Google the keywords "atmospheric air composition dataset" or similar. There are many, many sites where you can download datasets like this.
@@techknowhow4802 Hello, please i'm using ARIMA and other statistical model to make price forecast and i'm having some issues, can you give me your contact so that i discuss this with you?