@Jeff Heaton Sir, I enjoy learning through your videos. You're a pioneer. Thank you so much. I would like to correct one line of code above. In the LSTM code, print("Expected classes: {}", y) is the correct one instead of print("Expected classes: {}", predicted classes) since the variable "predicted_classes" holds the predicted results. Expected classes will be the values in "y". Thank you.
Dear Jeff, thank you for this video. I have one important question regarding the number of units you chose for the input layer. Are there any considerations for choosing the number of units for each LSTM layer or not?
Actually it would be nice if you would explaint he model.add(LSTM(num_units, recurrent dropout, ......)), I dont understand how the num_units is applied to the graph you showed in the video. also the recurrent dropout and other params too. Thanks!
I'm a computer science university student from Italy. This video helped me a lot in understanding how to use LSTM from a practical point of view. I have to use LSTM for battery capacity estimation with time series. I have a question, how could I predict only one value at the end of the sequence? In my case I have a series of measurements of Voltage, Temperature and Current and after let's say 5000 measurements I have a calculated capacity value Q. How can I be able to take in count all the measurements to see how they affect the Q and at the same time how can I be able to not give a label to all time steps but the last (which will have the calculated capacity)? You are clear and effective, thank you very much for sharing your expertise with us.
If you still need an answer to this... I would look into "multivariable" time series predictions. If you remember the sun spot's example in the video, he had two variables of data: spot count and time. But he used just the spot count! This is "single variable" time series predictions. To start you off, I'd go here: machinelearningmastery.com/multivariate-time-series-forecasting-lstms-keras/ This is an awesome tutorial detailing a multivariable time series prediction with weather data. BTW, your battery capacity problem is really cool! If you have a github repository for that, I'd love to see it.
Just a tip on for the car example on transforming the integer label vector : you can use y2=to_categorical(y, num_classes=max_features) ( after “ from keras.utils import to_categorical)
Respected sir can you please explain is their is any possibility where we can create a Bi-GRU Model which accepts multiple channel inputs like Multi-channel CNN. I mean we send him part of speech embedding on one channel and word embedding on other channel. Thank you
Hii How to handle persistent model problem. While doing time series analysis i get the output which seems to be one time step ahead of the actual series. How to rectify this problem?? This thing i am getting with several ML, DL, and as well as with statistical algos. Please do reply??
Hi! just a question, does lstm predict on sequences of FEATURES in ONE SAMPLE or sequences of SAMPLES (outputs) in ONE BATCH? For eg. I need to predict next number as many to one. I fit first sample as x1=1, x2=2 and output y=3, next sample x1=4, x2=5 y=6. NOW Does the model look on sequence of features (x1,x2) or sequence of samples (y, which are output of the model)
Doesn't have too! The 128 value is the amount of NODES in the LSTM network. This is an esoteric value and, typically, they like to keep those values in multiples of 16 or 32 or 64 etc. In simple terms, the more NODES you have, the "smarter" the network. Now, be careful with this, because two many nodes may be wasteful and create a network the eats up too many resources. The best network only utilizes what it needs, or, is "just smart enough". Play around with this variable to see how your network performs!
@@jackmead7292 Thanks for the explanation. I was looking into this the other day, and I think I understand it better now. As far as I can tell, the 128 is the length of the vector that is output from every LSTM unit, not the number of LSTM units (as I originally thought).
thank You infinity Time for your videos, I want to ask you what happens if you have a data frame where each row of observations on how to convert a data frame of observation to sequence? Thank s for advance.
@@HeatonResearch I understand, that‘s comprehensible. I used Encog MLP back in 2009 and just re-discovered. Was much faster than my own MLP implementation :-)
Dear Mr. Heaton, first off, thank you for providing the useful resource. I do have a question for you and that is about evaluating the time series model. In my opinion, I do not think that the correct way to evaluate the model is how you are doing. You took the ground truth values from the dataset to generate the test set. Shouldn't the test set be generated by the model and then compare the values with the ground truth because that's how you will correctly measure the model's performance. For example: train_set = [x1, x2, x3, ..., xn] was used to train the model (m) and the last 10 sequence [xn-10, xn-9, ..., xn] will be fed to the model to predict [xn+1] and further [xn-9,xn-8,...xn+1] will be used to predict [xn+2] and so on... And once you do that you will have a set of [xn+1, xn+2,...xn+z] (where z is the future time steps you wanna predict) that will be used to evaluate the model. Let me know if my question makes sense.
I would say it depends on how far into the future you need to really predict. For example, a high frequency trading prediction algorithm would only predict a few minutes to an hour into the future. To go further into the future you would do exactly as you are suggesting, and feed the model's output back into the input. You would have to design a very custom objective function to really capture that so that the neural network is being optimized for that future state.
The code should work on TensorFlow 2.0. Now be aware that I created the Jupyter notebooks during 2.0 beta. Now that the release is out, I am going to do one big pass over the whole course at the end of the semester in December. I don't switch versions mid-sememster.
how to add initial weight and bias into LSTM from csv file into below code model = Sequential() model.add(LSTM(250, input_shape=(train_X.shape[1], train_X.shape[2]),activation='relu')) model.add(Dropout(0.2)) model.add(Dense(1)) opt = Adam() model.compile(loss='mae', optimizer=opt)#,metrics=['accuracy'])#mean_squared_error # Fit the model history = model.fit(train_X, train_y, epochs=20,validation_data=(test_X, test_y),batch_size=24,verbose=2,shuffle=False)
Dude, thanks for sharing but, please, diagrams illustrating what you're talking about :) Just verbally describing concepts that are often new for the audience isn't enough to create a mental model. Visuals would have made me stick with this presentation, but as such I feel after 3 minutes I don't have the foundation to continue the video which is a shame as you obviously have a wealth of knowledge to share. Peace!
Jesus Christ you are so good explaining these concepts. Thank you so much Mr. Heaton!
Amen! Are you a christian ?
So glad I found your videos. Great explanation of RNN’s. Best I have seen! Thank you!
And is modern too! everything is relevant and up to date 2020!
@@AhmedIsam Yes other channel videos are 3, 4 years old
yes, this is a great video. Thanks for sharing.
Thank you for the educational videos
Informative video. Thanks for sharing prof
@Jeff Heaton Sir, I enjoy learning through your videos. You're a pioneer. Thank you so much. I would like to correct one line of code above. In the LSTM code, print("Expected classes: {}", y) is the correct one instead of print("Expected classes: {}", predicted classes) since the variable "predicted_classes" holds the predicted results. Expected classes will be the values in "y". Thank you.
Dear Jeff, thank you for this video. I have one important question regarding the number of units you chose for the input layer. Are there any considerations for choosing the number of units for each LSTM layer or not?
Thank you! best greetings from collegues (Germany)
Never knew that default scientific calculator had those functionalities! :)
Thank you for sharing your knowledge with us hopefuls! Subscribed.
Great video, love the discussion on literature + real application. Much clearer than my masters program, why do I bother lol
Glad you enjoyed it!
Great explanation. Thanks.
Great content. Please provide links to the next topic as you stated (using LSTMs with CNNs)
Actually it would be nice if you would explaint he model.add(LSTM(num_units, recurrent dropout, ......)), I dont understand how the num_units is applied to the graph you showed in the video.
also the recurrent dropout and other params too.
Thanks!
I'm a computer science university student from Italy. This video helped me a lot in understanding how to use LSTM from a practical point of view.
I have to use LSTM for battery capacity estimation with time series. I have a question, how could I predict only one value at the end of the sequence? In my case I have a series of measurements of Voltage, Temperature and Current and after let's say 5000 measurements I have a calculated capacity value Q. How can I be able to take in count all the measurements to see how they affect the Q and at the same time how can I be able to not give a label to all time steps but the last (which will have the calculated capacity)?
You are clear and effective, thank you very much for sharing your expertise with us.
If you still need an answer to this... I would look into "multivariable" time series predictions. If you remember the sun spot's example in the video, he had two variables of data: spot count and time. But he used just the spot count! This is "single variable" time series predictions. To start you off, I'd go here: machinelearningmastery.com/multivariate-time-series-forecasting-lstms-keras/ This is an awesome tutorial detailing a multivariable time series prediction with weather data. BTW, your battery capacity problem is really cool! If you have a github repository for that, I'd love to see it.
Thank you, Mr. Jack. Very informative.
I have a question. Why did you set the SEQUENCE_SIZE value to 10? Any reason?
Just a tip on for the car example on transforming the integer label vector : you can use y2=to_categorical(y, num_classes=max_features) ( after “ from keras.utils import to_categorical)
Great Video! Thanks so much - subscribed!
Respected sir can you please explain is their is any possibility where we can create a Bi-GRU Model which accepts multiple channel inputs like Multi-channel CNN. I mean we send him part of speech embedding on one channel and word embedding on other channel. Thank you
Hii
How to handle persistent model problem. While doing time series analysis i get the output which seems to be one time step ahead of the actual series. How to rectify this problem?? This thing i am getting with several ML, DL, and as well as with statistical algos. Please do reply??
Hi! just a question, does lstm predict on sequences of FEATURES in ONE SAMPLE or sequences of SAMPLES (outputs) in ONE BATCH? For eg. I need to predict next number as many to one. I fit first sample as x1=1, x2=2 and output y=3, next sample x1=4, x2=5 y=6. NOW Does the model look on sequence of features (x1,x2) or sequence of samples (y, which are output of the model)
Could you explain the Keras LSTM model? I don't quite understand how the 128 units match up with the input data.
Doesn't have too! The 128 value is the amount of NODES in the LSTM network. This is an esoteric value and, typically, they like to keep those values in multiples of 16 or 32 or 64 etc. In simple terms, the more NODES you have, the "smarter" the network. Now, be careful with this, because two many nodes may be wasteful and create a network the eats up too many resources. The best network only utilizes what it needs, or, is "just smart enough". Play around with this variable to see how your network performs!
@@jackmead7292 Thanks for the explanation. I was looking into this the other day, and I think I understand it better now. As far as I can tell, the 128 is the length of the vector that is output from every LSTM unit, not the number of LSTM units (as I originally thought).
@@BrettClimb Right! Yeah, that's another way of looking at it.
thank You infinity Time for your videos,
I want to ask you what happens if you have a data frame where each row of observations on how to convert a data frame of observation to sequence?
Thank s for advance.
sorry i found the solution at 24:11 thank's again
how to predict target column value after dividing whole dataset into independent and dependent parameters using LSTM?
Use a softmax function instead of a sigmoid. Softmax function activation gives a sequence output as a probability distribution
Thanks for the great video! Do you plan to integrate LSTM to your Encog Core?
I really do not support Encog anymore... Now that the major tech companies all have neural network engines, there is just not much point.
@@HeatonResearch I understand, that‘s comprehensible. I used Encog MLP back in 2009 and just re-discovered. Was much faster than my own MLP implementation :-)
Dear Mr. Heaton, first off, thank you for providing the useful resource. I do have a question for you and that is about evaluating the time series model. In my opinion, I do not think that the correct way to evaluate the model is how you are doing. You took the ground truth values from the dataset to generate the test set. Shouldn't the test set be generated by the model and then compare the values with the ground truth because that's how you will correctly measure the model's performance.
For example: train_set = [x1, x2, x3, ..., xn] was used to train the model (m) and the last 10 sequence [xn-10, xn-9, ..., xn] will be fed to the model to predict [xn+1] and further [xn-9,xn-8,...xn+1] will be used to predict [xn+2] and so on...
And once you do that you will have a set of [xn+1, xn+2,...xn+z] (where z is the future time steps you wanna predict) that will be used to evaluate the model.
Let me know if my question makes sense.
I would say it depends on how far into the future you need to really predict. For example, a high frequency trading prediction algorithm would only predict a few minutes to an hour into the future. To go further into the future you would do exactly as you are suggesting, and feed the model's output back into the input. You would have to design a very custom objective function to really capture that so that the neural network is being optimized for that future state.
How well did the network predict the sunspot values? Plots?
The RMSE was 22. So able to guess within 22 of the actual count of spots(i.e. predicting 500 when actual was 522)
Great video. Am I correct to assume that this is using tensorflow 1.0?
The code should work on TensorFlow 2.0. Now be aware that I created the Jupyter notebooks during 2.0 beta. Now that the release is out, I am going to do one big pass over the whole course at the end of the semester in December. I don't switch versions mid-sememster.
Very helpful. Thank you for the reply and the great courses
how to add initial weight and bias into LSTM from csv file into below code
model = Sequential()
model.add(LSTM(250, input_shape=(train_X.shape[1], train_X.shape[2]),activation='relu'))
model.add(Dropout(0.2))
model.add(Dense(1))
opt = Adam()
model.compile(loss='mae', optimizer=opt)#,metrics=['accuracy'])#mean_squared_error
# Fit the model
history = model.fit(train_X, train_y, epochs=20,validation_data=(test_X, test_y),batch_size=24,verbose=2,shuffle=False)
Dude, thanks for sharing but, please, diagrams illustrating what you're talking about :) Just verbally describing concepts that are often new for the audience isn't enough to create a mental model. Visuals would have made me stick with this presentation, but as such I feel after 3 minutes I don't have the foundation to continue the video which is a shame as you obviously have a wealth of knowledge to share. Peace!
Sinquence? haha
Какой объем видеопамяти лучше для ltsm ? 🇷🇺🇷🇺🇷🇺🐻🐻🐻🇷🇺🇷🇺🇷🇺
Не менее 12 гигабайт.