Custom Fine-tuning 30x Faster on T4 GPUs with UnSloth AI
ฝัง
- เผยแพร่เมื่อ 25 ก.ค. 2024
- Unclothe AI is a new AI startup that promise to have 30x faster training of the open source LLMs compared to the Huggingface Transformers implementation.
Want to Follow:
🦾 Discord: / discord
▶️️ Subscribe: www.youtube.com/@engineerprom...
Want to Support:
☕ Buy me a Coffee: ko-fi.com/promptengineering
|🔴 Support my work on Patreon: / promptengineering
Need Help?
📧 Business Contact: engineerprompt@gmail.com
💼Consulting: calendly.com/engineerprompt/c...
Join this channel to get access to perks:
/ @engineerprompt
LINKS:
UnslothAI: unsloth.ai/introducing
Github: github.com/unslothai/unsloth
Google Colab: tinyurl.com/566nhsxr
Timestamps:
[00:00] Introduction to Unsloth AI
[00:34] How Unsloth AI Achieves Performance Increase
[01:26] Unsloth AI's Different Plans
[01:55] Performance Analysis of Unsloth AI
[03:40] How to Use Unsloth AI Package
[06:08] Training the Model with Unsloth AI
[08:05] Post-Training Analysis and Statistics
[09:13] Making Predictions with the Trained Model
[10:-7] Conclusion and Final Thoughts
All Interesting Videos:
Everything LangChain: • LangChain
Everything LLM: • Large Language Models
Everything Midjourney: • MidJourney Tutorials
AI Image Generation: • AI Image Generation Tu... - วิทยาศาสตร์และเทคโนโลยี
Thanks for sharing Unsloth - highly highly appreciate it and great work on the video! I'm the engineer behind Unsloth - so if anyone has any questions - ask away!
Thanks for making this package. Would love to experiment more with it on the multiGPU setup.
Hi Dan! Can unsloth be used for over training a model to achieve grokking? My partner and i are planning to use unsloth for fine tuning
Learnt a lot! Thanks for the great video!
Hello, im having trouble get the inference part work, can you make a part 2 for inference? it,ll help alot please reply and love your videos.
thanks for sharing nice packages VLLM and Unsloth
Do you think they'll make a simpler version for end users? This is so complicated.
Thank you, i did it on Colab T4, and training takes about 8min, but dont understand why the last step inference takes more than 15min? Where can we take the Tuned Model to do inferencing, like Ollama, or LM studio?
You should be able to save the tuned model, then reload it through your favourite package - we are actively working on making inference faster!!
From my understanding, it seems to be the unquantized model right?
If it can only be fine tuned on the Alpaca dataset, then that's a big setback. It means we can't use other HF datasets?
No, you just need to put the data in the alpaca format. You can use any dataset you want. Just need to be reformatted which is very common
We're working on a version making any dataset (csv, excel, pdf, word etc) allowed to be finetuned!
The training is done without LORA. Do you think that to train a Llama2 model that specialises in translating between 2 languages like French and English it would be a good idea to use training without LORA because Llama2 has little pre-training data in French and it is better to do a harder training than LORA?
I would assume so, but you will need much larger dataset to do it.
Does anyone know if OpenAI assistant functionality is offered by any other company using open-source LLM (with RAG and tools)?
How to finetune an llm using a pdf ? #rather than QA
You need to create your own dataset, alpaca format is good format for that. I've build this QA generator myself, not sure if there are some opensource available.
You can fine tune in raw with lora, then you fine tune that adapter with sft, like this guy said alpaca format. Use a script to convert your pdf to raw text. Then train with that. Then on top of that train with maybe 1-2k alpaca formatted sft questions so the model works.
@@Nick_With_A_Stick is there any references, that i can folllow? any video? github links? article?
We're working on a system where any text is allowed!! I'll comment here again once it's done!!
@@danielhanchen do you have a github for this? I’d love to check it out, I HATE converting pdf’s to text.
hello brother your video is amazing but they are boring as well. I am trying to do some constructive criticism. You can add your face video to make it interesting
Be nice bro I like the format and cutting the entertainment crap out we should be grateful for the information 😅🎉 Ty for sharing and updating us on such important topics always my goto never mind a video from prompt engineering
I am here for the learning and not for the show. Always right to the point, very clear explanation, good pace to provide a ton of info in a concise manner, no distracting music, no silly intros.
Thank you for sharing your knowledge
@@opusmas7909 True. But brilliant people don't make their presentation boring. I feel it, I expressed it. Thanks.
💡 Suggestion:
Increase the play ▶️🔙 Speed⏭️
⚙️ Adjust the Speed from "NORMAL" Speed ⏯️
To either
1️⃣.2️⃣5️⃣❌ Speed ⏩
𝙊𝙍
1️⃣.5️⃣❌✖️ Speed ⏭️
🔔 Technique commonly used to listen👂 to audio books 📚
🔔 play ▶️🔙 Speed increases⏫ the tempo in sounds 🗣️💬-̳͟͞͞➪ which may temporarily help increase attention to what is being said