Kaggle Earthquake Prediction Challenge
ฝัง
- เผยแพร่เมื่อ 17 ก.พ. 2019
- Data Science coding challenge time! The popular Data Science competition website Kaggle has an ongoing competition to solve the problem of earthquake prediction. Given a dataset of seismographic activity from a laboratory simulation, participants are asked to create a predictive model for earthquakes. In this video, I'll attempt the challenge as a way to teach 3 concepts; the Data Science mindset, Categorical Boosting, and Support Vector Regression models. I'll be coding this using python from start to finish in the online Google colab environment. Enjoy!
Code for this video:
github.com/llSourcell/Kaggle_...
Please Subscribe! And Like. And comment. Thats what keeps me going.
Want more education? Connect with me here:
Twitter: / sirajraval
instagram: / sirajraval
Facebook: / sirajology
Join us at the School of AI:
theschool.ai/
More learning resources:
www.kaggle.com/c/LANL-Earthqu...
www.analyticsvidhya.com/blog/...
blog.griddynamics.com/xgboost...
towardsdatascience.com/catboo...
accio.github.io/machinelearni...
kernelsvm.tripod.com/
www.saedsayad.com/support_vec...
/ support-vector-regress...
scikit-learn.org/stable/modul...
Join us in the Wizards Slack channel:
wizards.herokuapp.com/
Please support me on Patreon:
www.patreon.com/user?u=3191693
Signup for my newsletter for exciting updates in the field of AI:
goo.gl/FZzJ5w
Hit the Join button above to sign up to become a member of my channel for access to exclusive content! Join my AI community: chatgptschool.io/ Sign up for my AI Sports betting Bot, WagerGPT! (500 spots available):
www.wagergpt.co
I really appreciate these kaggle competitions you do cause it’s helping me gain confidence to do my own, which in turn is adding projects to my portfolio!
The best video you have made to date.
Cutting down the meme, rap, special effects really helped me focus more.
Keep up the good work!
haha, totally agreed! :) Thanks Siraj for this coding video :)
very good!!!
hm noted
@@SirajRaval I like the memes, rap and special effects :D
can anyone help me doing this in xgboost algorithm???
Thanks Siraj! Study ML for 3 weeks already and want you to shoot more episodes with more focus on data-preparation and all that stuff with features because when scikit-learn has all algorithms made for us how could we get the max power from them tuning hypersparams and so on. I think you get what I mean. This episode seems pretty easy so I want to see you really making art from it!! Greets from Russia
Glad you re-did this. More to the point and cleaned up. Keep it up! Papers with code - super cool!
Make sure you join the competition in Kaggle. Otherwise, kaggle download command will fail with forbidden error.
Siraj, this video was gold. Thanks for everything that you do.
@Siraj What's super exciting is hearing what you have to say! Whenever I watch your videos I know I'll not get disponited. You should do more videos about you & what you're up to etc.
Would love to see a video by you on SLAM algorithms! Keep up the good work and keep inspiring!
Dude you should make a video on Lingvo (a framework for sequence-sequence modeling) I'd love to hear your thoughts ~
With this video we now know how nervous Siraj is when coding hahah
Thank you so much Siraj!! You are the man! Cheers
Hey Dude,
I'm a DS Practitioner from India and I really appreciate 🙌 your work. A year ago, I was working in Mumbai and heard about a few things that got viral about you. I cannot choose to ignore that but I personally feel that you've been a strong influencer in the whole data science community. Just be yourself and keep doing what you do.. with citations and references, of course. :D #LetsMakeIndiaAGreatNation 🙌
dude u first go learn datastructures and algorithms
Wow man, you made my day!! Thanks a lot!
the code in step 4 gives me an invalid syntax on the last line
return pd.Series.(strain)
What am I doing wrong?
Amazing Video Siraj, thanks for sharing.
Waouwh TH-cam videos can be shown on Jupyter :O That is so cool :) Siraj you're just the best !
what else can i say......YOU ARE GOOD BRO
I think you did a live session with this problem. Did you delete it? I couldn't find that on your channel. That was also a great video.
Pls pls keep up what you are doing with kaggle. You helped me a lot
This is really a helpful video,siraj!!! Helped me a lot... Thanks a lot
really high quality video Siraj.
Thanks for the "tutorial" Siraj. Great video and has sure stimulated some new ideas on prediction modeling.
Siraj great video man... really helpful
Thank you, Siraj! I have two questions: 1. Does catboost perform better than LGB and XGBoost? 2. Could you please do a video talking about the whole process of feature engineer? I found feature is more important than models but I don’t know how to create fantastic features based on the original data.
I like these Kaggle videos, but it would be nice to see even more advanced videos
Do a lesson on creating your own deep learning framework. please or tell me where it can be studied
can this type of data be classified as segmented data?
Hi Siraj,
In step 4 you use chunksize=150_000 in read_csv. Can you explain how does 150_000 work?
Thanks
Thanks for the all your Videos Siraj, its really awesome ! Could you please demonstrate how should we approach Multivariate Time Series Prediction?
Don't know about earthquakes, but this guy's hand-waving (both literally and allegorically) must have triggered the tornado in Alabama.
Thank u for this awesome video
Ahhaha, "Time to Rap" . Wanting on the2019 AI Rap ... : )
Idea - Could a cause of a vulcano eruption be that not wanted material got under into the lava? Like CO2 for instance. So a eruption has some very important meaning and is not just a physical effect or random noise event. I assume the lava_model() probably iterates like a sun_model(). So faults in the iteration is not wanted and the lava_model() needs to get rid of them.
So wonder if you can insert a removable problem in a lava_model() and let the model find it and then move it.
// Per Lindholm
can you explain how [24] works because it gives
TypeError: string indices must be integers
thank you man great explanation !
Great video! Could you do a video about genetic algorithms for hyperparameter tuning? maybe compare to other methods like gridsearchcv.
Very nice !
I want to ask if we can tweak this AI /ML Algo. to predict "Crimes Probability at any place" and "People's gathering at any event" ???
There is already software which predicts crime locations and the probabilities for crime
i did not understand your command for uploading file
Best takeaway of this video is "from IPython.display import TH-camVideo"
Nice video! Shouldn't you split the training data again into two sets for parameter optimization?
Kaggle API would work to fetch the list of competitions one has signed up for. So, every person who is trying to fetch a kaggle competition dataset to Colab, will first need to signup for the competition and then use the api.
@Siraj: perhaps if you include this somewhere in the video, for someone new to kaggle competitions following your video will be easier. I had not signed up for the competition at first and tried fetching the list of datasets from kaggle (following your live stream) and got stuck there for a while as it kept on throwing error, until I did some digging and finally signed up for the competition. It is a small detail that could save one a lot of time and trouble.
Thanks for the great videos. :)
Yay!
The older hand intro?????
Siraj make some videos on solving Numerai Challenge
What a day I found your video. I just didnt get the code to confirm your claims.
THANK YOU SO MUCH !!
I am new to Machine Learning and Deep Learning. I have learned some basic and done some small experiments. Now the school task is to join the challenge of research in Kaggle. I have no idea how to start. Any suggestions? I do appreciate your response. Thanks in advance.
Hey, i'd like to learn more about stateful LSTMs
i am not able to download the dataset following error is shown
Warning: Your Kaggle API key is readable by other users on this system! To fix this, you can run 'chmod 600 /root/.kaggle/kaggle.json'
403 - Forbidden
You have to join the kaggle competition first. This is why you are getting the forbidden error
Siraj, can you make tutorial about prediction keywords with high volume or will be booming in 1/2 months from google trends data using machine learning? it would be awesome. Thanks
HEY SIRAJ WE need kaggle for beginner
I Learn a lot from you boss, learn a lot.
Awesome! :)
Great job
Can you please make video on neural network application on neuroscience
search siraj math of neuroscience
Can you please make videos on probabilistic graphical models from very basics.
Next, VSB Power Line Fault Detection?
9:38 so, the data was downloaded to the 'cloud'? and we cannot see the data locally? how to check the data in the cloud? where is the data exactly?
! ls
My Visualization graphs are different from yours while I'm implementing your code with same data, I'm a complete beginner (rather just started learning data science). Please help me in this regard.
i just dont usually see you use deep learning/tensorflow to solve problems like this, why? is that not fit to solve kaggle cases?
not enough data
please do vedios on kaggle for bigginers,
thank your
Is there any chanels like Siraj's?
Thanks buddy🕺
I'm unable to import the LANL-Earthquake-Prediction dataset by following the code from the video. I get the error "403 - Forbidden". How to fix this?
have same issue
Did you enroll in the competition?
Also there is a typo error, in the video he wrote ~/kaggle/kaggle.json, but file has to be placed in ~/.kaggle/kaggle.json.
The github code is cleaner:
github.com/llSourcell/Kaggle_Earthquake_challenge/blob/master/Earthquake_Challenge.ipynb
Hi!~It's a great video. Help me a lot. I try to do the same step to learn,
but i stuck in step 4. I try to debug but it's still error about " module 'numpy' has no attribute 'quantile' ".
I already try install higher numpy. As numpy==1.15.1 and upgrade.
But error still the same. I see you video has no this problem. why it happen?
!pip install kaggle
!pip install numpy==1.16.4
!pip install catboost
@Siraj help....
When trying to download earthquake data, it shows forbidden 403 error in Google colab
You need to join the competition on the kaggle website to have access to download the data. On the kaggle website do a search for LANL Earthquake Prediction then click the join the competition button.
@@randykat thanks but i already did enrolled.
anyways i figured it out :D
i was running out of storage space :P
Another Great Video!
can you make one for real earthquakes? just for fun, maybe use IRIS earthquake database
Understanding those faults movements will get to nothing because there is always new ones appearing...understanding the cause's of those movements will be something...😊😊
Make a video on twitter sentiment analysis in python with dataset
Can someone explaine why this error is occuring:
Warning: Looks like you're using an outdated API Version, please consider updating (server 1.5.6 / client 1.5.4)
403 - Forbidden
TIA
This might have happened when you tried to use the kaggle API for downloading the data. You have to first accept the rules of the competition (on Kaggle) in order to use the API .
Finally..!
thanks siraj
awesome
You make this so clear! But... using Colab, I'm getting this outdated API message, and a '403 Forbidden' message Any ideas on fix/workaround?
Saving kaggle.json to kaggle.json
User uploaded file "kaggle.json" with length 68 bytes
Warning: Looks like you're using an outdated API Version, please consider updating (server 1.5.6 / client 1.5.4)
Got it! I joined the competition.
Zarah effect random is the best earthquake forecast in world it can predicted minutea before it happen in epicentrum.i challenge kaggle.
First time you made a proper video. This really helps to learn.
bro while i'm trying to download datset it's giving me "403 forbidden" help me bro
You need to join the competition on the kaggle website to have access to download the data. On the kaggle website do a search for LANL Earthquake Prediction then click the join the competition button.
@@randykat thanks
hi there is a gpt2 video??? (by siraj)
blog.openai.com/better-language-models/
I’ve never seen him so uncomfortable lol.
Great video nevertheless I learnt a lot. Thank you so much sir.
Earthquake prediction is possible .. time and magnitude .. with 1% error .. Prediction Formula is generated .. and passed testing and verification phase ..
estamoh en primer lugar somos el mejor pais de chile
WHAT IS A PERSON IN DATA SCIENCE CALLED A QUANT
IOError: Could not find kaggle.json. Make sure it's located in /root/.kaggle. Or use the environment method 😢
neither when I typed by myself nor copied uploading kaggle.json and next steps from github :(
btw Siraj has same error mv: target '/root/kaggle/kaggle.json' is not a directory however his code is worked.
yikes, I missed second '&&' before chmod >_
well it's like financial time series; when i was young i learned things, and i seriously doubt one can find out smtg useful.....
Hi Siraj, I doubt the pseudocode you shown in 22:13 was AdaBoost, rather than gradient boosting. Or if I misunderstood the concept?
Check this link out www.quora.com/What-is-the-difference-between-gradient-boosting-and-adaboost
Nice video by the way. Helps me a lot.
♥️♥️🔥
!kaggle competitions download -c LANL-Earthquake-Prediction
I am getting 403 - Forbidden error while i am trying to download the dataset from kaggle
Narendra: Go to Kaggle and sign up for the competition from the competition’s page. This needs to be done manually first, before kaggle let’s you access the competition’s data through the api. Good luck. 👍
@@SUGATORAY Thank u ,it helped
@@SUGATORAY I did it but it didn't work , get the same error
@@wasgeht2409 Follow the instructions as in this Github gist: gist.github.com/sugatoray/9244867c89fd3868ec8e5a5ba7341901#file-kaggle_import_dataset_public-ipynb
just want to know whether is it legal to discuss an ongoing challenge??
Please provide your data .. and I will predict the next coming earthquake .. time-wise and magnitude ..
Just slap a GPS collar on to my ex wife. The closer she gets to your location, an earthquake is a comin. Bam, project done! No ML needed.
why making a video twice?
watching this makes realize how much of a rookie I am...
Viva Chile weon
He should have told that he is actually using google colab and not jupyter
I was thinking he used Kaggle kernel. but he is taking about google colab around 5:10.
hey bro i'm the first viewer woho!!!!!!!!!!
Lol we live on a flat earth right
first
No offense. I don’t see the value that you added descriptive statistics in one single value to all rows as features. You must do a prank or make yourself like a fool.
does the magic. He iterates over chunks, not single rows. I was wondering the same.
Useless information that explained everything unclearly ! it shows that you just memorized it and didn't understand it from the depth. I'm just wasted my time watching this.
how does a womens vajaja muscle predict earthquakes?