Nvidia H100 GPU on Lambda labs is just $2/hr, I am using it for past few months unlike $12.29/hr on AWS as shown in the slide. I get it, it's still not cheap but just worth mentioning here
You are right, we reported the AWS price there as it's hte most popular option and it was not practical to show all the pricing of all the vendors. But yes you can get them for cheaper elsewhere like from Lambda, thanks for pointing it out
@@rankun203 They are available only in specific region mine is in Utah, I don't think they have expanded it plus there is no storage available in this region meaning if you shut down your instance, all data is lost
Cool video. If I want to fine-tune it on a single specific tassk (keyword extraction), should I first train an instruction-tuned model, and then train that on my specific task? Or mix the datasets together?
at 51:30 he says don't repeat the same prompt in the training data. What if I am fine-tuning the model on a single task but with thousands of different inputs for the same prompt?
It will cause overfitting. It would be similar to training an image classifier with a 1000 pictures of roses and only one lilly, then asking it to predict both classes with good accuracy. You want the data to have a normal distribution around your problem space.
It really depends on the dataset. Ludwig has also an early stopping mechanism where you can specify the number of epochs (or steps) without improvement before stopping, so you could set epochs to a relatively large number and have the early stopping take care of not wasting compute time
when I run the code in Perform Inference, I frequently receive ValueError: If `eos_token_id` is defined, make sure that `pad_token_id` is defined. what should I do?
![[Webinar] LLMs for Evaluating LLMs](http://i.ytimg.com/vi/jW290vZThgw/mqdefault.jpg)
![[Webinar] LLMs for Evaluating LLMs](/img/tr.png)
Very helpful! Already trained llama-2 with custom classifications using the cookbook. Thanks!
Amazing, can’t wait to play and train my first model 🎉
Well this was simply excellent, thank you 🙏🏻
Very informative. Direct and to the point content in a easily understandable presentation.
I like to kindly request @DeepLearningAI to prepare such hands-on workshop on fine-tunning Source Code Models.
Don't miss our short course on the subject! www.deeplearning.ai/short-courses/finetuning-large-language-models/
@@Deeplearningai , Wow thanks.
One of the most complete videos. Must watch
Great content, well presented!
Excellent xtal clear surgery on GPU VRAM utilization...
Thank you!
🖖alignement by sectoring hyperparameters in behaviour, nice one
Excellent coverage, thankyou.
Great video, thank you!
Really helpful. Thank you 👍
Nvidia H100 GPU on Lambda labs is just $2/hr, I am using it for past few months unlike $12.29/hr on AWS as shown in the slide.
I get it, it's still not cheap but just worth mentioning here
You are right, we reported the AWS price there as it's hte most popular option and it was not practical to show all the pricing of all the vendors. But yes you can get them for cheaper elsewhere like from Lambda, thanks for pointing it out
Last time I tried it, H100s are out of stock on Lambda
@@rankun203 They are available only in specific region mine is in Utah, I don't think they have expanded it plus there is no storage available in this region meaning if you shut down your instance, all data is lost
together AI is $1.4/hr on your own fine tuned model :)
@@Abraham_writes_random_code Predibase is cheaper than that
amazing video
What's the music in the beginning, can't shake it off
Clear and informative, thanx.
28 zoom notifications! Travis working too hard
Very helpful. Thanks.
Amazing Content of fine tuning LLM
And I was under the delusion that I would be able to fine-tune the 70B param model on my 4090. Oh well...
I got a 40b model working on a 4090
@@iukeay Did you fine tune it, or just inference?
70B param? hahaha.
Great job, thumbs up!
@pieromolino_pb -Is Ludwig allows to locally download and deploy the fine-tuned model?
Cool video. If I want to fine-tune it on a single specific tassk (keyword extraction), should I first train an instruction-tuned model, and then train that on my specific task? Or mix the datasets together?
also working on keyword extraction! I was wondering if you'd had any success fine tuning?
at 51:30 he says don't repeat the same prompt in the training data. What if I am fine-tuning the model on a single task but with thousands of different inputs for the same prompt?
It will cause overfitting. It would be similar to training an image classifier with a 1000 pictures of roses and only one lilly, then asking it to predict both classes with good accuracy. You want the data to have a normal distribution around your problem space.
@PickaxeAI Did you come across a solution for this?
Can you give an example for the task? I'm trying to understand in what situation you'd need different completions for the same prompt
Amazing ❤
I ran Colab T4 and still got into “RuntimeError: CUDA Out of memory”. Any thing else I can do please?
Eh, c'était super. Merci beaucoup!
Please can you provide a link to the slides?
How long did the entire training process take?
Depends on your hardware, dataset, and hyper parameters you’re manipulating. The training process is the longest phase in developing a model.
Thankyou
epochs=3, since we are fine tunning, would epochs=1 would suffice?
It really depends on the dataset. Ludwig has also an early stopping mechanism where you can specify the number of epochs (or steps) without improvement before stopping, so you could set epochs to a relatively large number and have the early stopping take care of not wasting compute time
Hello everyone, I would be so happy if the recorded video have caption/subtitles.
Right
it does, you just have to enable it! 😂
@@dmf500now it is enabled😂
Cool! ❤
can we have the slides plz ?
can you share the slide, please?
❤❤❤
This seems to make a case for Apple Silicon for training. The M3 Max performs close to an RTX 3080, but with access to up to 192GB of memory.
Did you try on Apple silicon M1 Max?
when I run the code in Perform Inference, I frequently receive ValueError: If `eos_token_id` is defined, make sure that `pad_token_id` is defined.
what should I do?
This is now fixed on Ludwig master!