Amazing !!! I have red the book -"Generative AI on AWS" today and learnt all the concepts of quantization, PEFT, LoRA, QLoRA and you have uploaded the video for the same!! Thanks a lot!!
Thank you for the video. The main issue I face from these tutorials is the custom dataset preparation part. Here also the dataset is loaded from HF. I have a tabular NLP classification dataset in my local. Let's say sentiment analysis dataset. How should I prepare the dataset and run the llm finetuning locally? Thank you again for this tutorial. I hope you'll show us the implementation of actual local, own dataset finetuning. Also, there's a paper called TabLLM, which uses LLM on numeric tabular datasets. Making a video on that one would be so much helpful regarding implementing it on the custom private dataset. Thank you again, and keep bringing good content as always
Amazing video Krish, Can you also make a video on how to build RAG based LLM for Q&A over multiple documents where we can actually compare between two or more documents.
Can you please upload videos indepth of how different prompting techniques like chain of thought, self consistency, knowledge generation etc were practically used with which the outputs of the models based on use cases are getting improved
Please also make a video on mathematical concepts and the intuition behind the LLMs. Already subscribed and liked the video, as you are doing an amazing job.
Could you please explain the parameters, what does they mean, what is the effect of these in the model performance and what is the significance etc. ?? In interviews, they ask such questions only, because almost all the candidates has a lot of project from various online resources, but we need the underneath principle behind the working of finetuning with different params.
Hi everyone, I have a question about data preparation and fine tuning of LLMs. What should the data format look like in the fine tuning process? On the one hand, it can be pure text to add special knowledge to the LLM. On the other hand, the data set can be structured in question and answer / prompt and answer format. What do you think? Do you have any recommendations for me? Thank you and best regards!
May I ask a question? I used your code to fine-tune llama2 7b-chat on my data and the code works perfectly, but for some reason my new LLM can't predict the EOS token. So, every time I ask the model to generate text, it will generate tokens until it reaches the max_length. I think there is something wrong with the way Lora is using this EOS token. Do you have any idea how to fix this? By the way, amazing video. Thanks.
Krish sir, can you tell me for fine tunning llama 3 why most of the people are using alpaca format, is this a straight away rule, or it just like it works well with alpaca format for fine-tunning.
Sir this is a great video, but Please give a method that dont use hugging face to download the model, because we may have to train the same model again with new data. Also in the video we have download the model but how to load the model if it is locally available ?
Thank you for this amazing video. Can u also explain how to create custom dataset in Q/A format from the raw text and fine tune it and should we fine tune or use RAG if we want reponse from a particular domain only.Thanks
Your google colab code has conflict of packages and can not be run. I suppose just recently something has changed with the hidden package versions and so the problem has raised, or?
I didn't understand "NousResearch/Llama-2-7b-chat-hf" this part. The original version of this model belongs to "meta-llama/Llama-2-7b-chat-hf". Since you are reducimg the precision, are you using this "NousResearch" model or what's the use in using this model. Also, what's the difference between these two models?
Finetuning happens when you cannot train the model for each user or each use case. If you want the model to work in a specific native language, the Llama2 model should have been trained in the same native language. Remember you are training the model with the data and the current Llama2 model is training on the English dataset.
@Krish Can you load the fine-tuned model and then test/check it on the test data? In last code snippet I guess you are using the base model to get the results. Please correct me if I am wrong. Coping that line of code here which I am doubtful of. pipe = pipeline(task="text-generation", model=model, tokenizer=tokenizer, max_length=200)
You do not need an LLM model for it. Wedding possibility is a classification problem and it is very easy to make a model if you know logistic regression or decision tree algorithm. These are week learners with a low accuracy but your use case is solvable by the said algorithms.
Hi folks, can anyone help? he had taken from the hugging-face for the final demonstration of the output but we need to test the fine-tuned model right?
Hi Krish, Hi , I want to fine tune a code generator model with our organisational data specific to embedded Software. code generated should be specific to the chipset we are using. I was thinking of using starcoder/CodeLLAMA as a base model and fine tune with QLORA. But I dont have much clarity of the format in which I should prepare the custom data set. Can you please help on this. Will joining the group with subscription will help to get some 1:1 guidance
looks like a typo!! Are we not supposed to be using "new_model" instead of "model" while testing the fine tuned model? I'm referring to this line-----> pipeline(task="text-generation", model=model, tokenizer=tokenizer, max_length=200). I think correct argument should be --> model=new_model instead of model=model ??
RuntimeError: Found no NVIDIA driver on your system. Please check that you have an NVIDIA GPU and installed a driver. I'm facing this error in google colab while running the GPU compatibility part. What can be the solution?
How can I fine-tune the LLAMA 3 8B model for free on my local hardware, specifically a ThinkStation P620 Tower Workstation with an AMD Ryzen Threadripper PRO 5945WX processor, 128 GB DDR4 RAM, and two NVIDIA RTX A4000 16GB GPUs in SLI? I am new to this and have prepared a dataset for training. Is this feasible?
Hi krish sir! I have worked on this script 2 days back, and I am eagerly waiting for your explanation about this llama. And my doubt is that, I think you didn't gone through this cell, "Reload the fp16 and merge it with lora weights " Explain this code cell and how it will merge and where it could be stored. For this particular block error : I'm getting out of the memory issue . And waiting for math's behind peft and theriotical knowledge... I hope this comment you will read,, And hope for response to my question!!! Thank you🌹
I want to integrate my database to the llm model , is it possible to finetune that. Can you show demo for integrating databses and fine tuning the llm model based on it.
Hey! Facing this issue with Langcahin loader: AttributeError: module 'numpy.linalg._umath_linalg' has no attribute '_ilp64' Literally dying to resolve this. Please help
it seems you are following any blog or article to show us how to finetuned, it is not worth. I wish, if you can create your own datasets and then ask any question to that model, would be more practice and useful, I tried doing that but failed, in self made datasets it won't work, but following the pre built datasets works.
Apologies, but upon viewing your video, I found the explanation lacking. It's crucial to have a strong initial explanation, especially when presenting more intricate concepts like fine tuning.
Does anyone retrain LLma 2 on monolingual data (let's say some data in a sepecific low resource language XX) and then finetune it on parallel data en - XX ?
Hi Krish, Thanks for the video. What is the purpose of developing the model using PEFT? Is the objective is to mimic CHATGPT where you ask questions and you get the answer?
Can you tell me what dataset templates should be used for fine-tuning? What fields should be there? If I need the model to answer questions in chat mode, like a first-line support bot, make a summary of the text I insert - are these different sets and as a result different models? That is, it turns out that I need already 2 models, each solves a specific task? If, for example, there is a production department and a financial department in the company, then is it better to use 2 separate small models and they are tailored to a separate knowledge sphere or use one large one? Show how to fine-tune on a local computer in the VSC environment on an RTX4090 video card
Amazing !!! I have red the book -"Generative AI on AWS" today and learnt all the concepts of quantization, PEFT, LoRA, QLoRA and you have uploaded the video for the same!! Thanks a lot!!
Can you please share the link of book if you have
Book name??
@@ViveksCodes He literally told the book name. Are you guys blind?
have some manners
@@mysticlunala8020
Can U please send me the book or link for it
I started my fine tuning journeys ,
hope it would be something interesting
Thank you for the video. The main issue I face from these tutorials is the custom dataset preparation part. Here also the dataset is loaded from HF.
I have a tabular NLP classification dataset in my local. Let's say sentiment analysis dataset.
How should I prepare the dataset and run the llm finetuning locally?
Thank you again for this tutorial. I hope you'll show us the implementation of actual local, own dataset finetuning.
Also, there's a paper called TabLLM, which uses LLM on numeric tabular datasets. Making a video on that one would be so much helpful regarding implementing it on the custom private dataset. Thank you again, and keep bringing good content as always
Amazing video Krish, Can you also make a video on how to build RAG based LLM for Q&A over multiple documents where we can actually compare between two or more documents.
This guy's excitement for NLP is adorable but man needs to get out more, the real world is calling!
Make theocratical videos on PEFT, LoRA, QLoRA, how quantization work, how quantize a model and Mixture of experts works
Actually this the video i want to ask you but you read my mind before I ask that why I am saying now Krish sir is mind reader
Please make vidoes on theoretical concepts such as LLM model internals, Mixture of Experts, RLHF and so on.
👍
+1
+1
Amazing how did you know all this sir😢😢😢😢
Can you please upload videos indepth of how different prompting techniques like chain of thought, self consistency, knowledge generation etc were practically used with which the outputs of the models based on use cases are getting improved
Mistral's medium posts helped me a ton, then found enterprise for hands on work
does mistral have a medium page?
but finding it
Wow thanks for breaking it down step by step.
Very Amazing video. Please make vidoes using json or csv file as a dataset
thanks for the video , it would be better if u can show documentation side by side with ur testing plz
Please also make a video on mathematical concepts and the intuition behind the LLMs.
Already subscribed and liked the video, as you are doing an amazing job.
Yes, please make a theoretical video as well on all open source llms
Sir do a video on how to transfer the customer data into q&a format for fine tuning to llms
thank you sir for this video, please make videos on theortical concepts needed to understand this fine tuning process. It will mean alot thanks sir
Can we do fine tuning on unsupervised data?
Could you please explain the parameters, what does they mean, what is the effect of these in the model performance and what is the significance etc. ?? In interviews, they ask such questions only, because almost all the candidates has a lot of project from various online resources, but we need the underneath principle behind the working of finetuning with different params.
need a theory content also sir . I would help in making the foundations stronger
what modification i need to do i i wanted to fine-tune this llama model on text-Summarization task... ?
Hi everyone,
I have a question about data preparation and fine tuning of LLMs. What should the data format look like in the fine tuning process? On the one hand, it can be pure text to add special knowledge to the LLM. On the other hand, the data set can be structured in question and answer / prompt and answer format.
What do you think? Do you have any recommendations for me?
Thank you and best regards!
May I ask a question? I used your code to fine-tune llama2 7b-chat on my data and the code works perfectly, but for some reason my new LLM can't predict the EOS token. So, every time I ask the model to generate text, it will generate tokens until it reaches the max_length. I think there is something wrong with the way Lora is using this EOS token. Do you have any idea how to fix this?
By the way, amazing video. Thanks.
Thanks for the video. Could you please create an end-to-end implementation video where you use Streamlit and local CPU?
Amazing video! Can you please make a video on the theoretical aspects as well?
it's working as charm wow.
Krish sir, can you tell me for fine tunning llama 3 why most of the people are using alpaca format, is this a straight away rule, or it just like it works well with alpaca format for fine-tunning.
Please conduct sessions on running AI models on cloud platforms like AWS, Azue and Google
Sir this is a great video, but Please give a method that dont use hugging face to download the model, because we may have to train the same model again with new data. Also in the video we have download the model but how to load the model if it is locally available ?
Amazing! Can you please make video on how to use fine tuned model in RAG.
how is Lora fine-tuning track changes from creating two decomposition matrix? How the ΔW is determined?
Thank you for this amazing video. Can u also explain how to create custom dataset in Q/A format from the raw text and fine tune it and should we fine tune or use RAG if we want reponse from a particular domain only.Thanks
Boss the code link provided in the description is not working
Hi Krishnaik, can you please create a Series on securing LLM responses, and Guardrails as it is burning topic now a days. Sincere Request.
Krish Naik please discuss how to evaluate the model ?
You are a gem ❤❤
Your google colab code has conflict of packages and can not be run. I suppose just recently something has changed with the hidden package versions and so the problem has raised, or?
Hi Krish, I had a doubt:
Will quantization decrease the accuracy of the whole model? Will that mean that we will get less accurate results?
Can you please make a video on DPO fine tuning method and its implementation.
I didn't understand "NousResearch/Llama-2-7b-chat-hf" this part. The original version of this model belongs to "meta-llama/Llama-2-7b-chat-hf". Since you are reducimg the precision, are you using this "NousResearch" model or what's the use in using this model. Also, what's the difference between these two models?
I m not surw if you said. But what is the avg hardware configuration?
Can we train
this model locally by creating a virtual environment (e.g. conda) ?
It is possible but you need a very high VRAM to process.
Thanks for uploading! how to create the dataset from your HTML/PDF content to train the model?
Can you please upload a video on how to finetune LLM model to work on or understand local language given a dataset on local languages
Finetuning happens when you cannot train the model for each user or each use case. If you want the model to work in a specific native language, the Llama2 model should have been trained in the same native language. Remember you are training the model with the data and the current Llama2 model is training on the English dataset.
I have already run this script one month ago, but this model cannot provide accurate answers as on custom data on which this llama2 model is trained .
Hi Sir, Thanks for the information. Could you please share that pdf(Parameter - Efficient Transform learning for NLP).
Thanks Krish
Can we use this for other languages such as Arabic Thanks a lot!!
Yes u can
@Krish Can you load the fine-tuned model and then test/check it on the test data? In last code snippet I guess you are using the base model to get the results. Please correct me if I am wrong.
Coping that line of code here which I am doubtful of.
pipe = pipeline(task="text-generation", model=model, tokenizer=tokenizer, max_length=200)
I have the same doubts. It seems it is picking the old base model and not the fined-tune one. Correct me if I am wrong
Why you are train for 1 epoch only? What will be the optimal number of epochs?
@Krish- After done fine tuning of the model, how can I run the fine-tuned model on local machine
Sir can you create videos for evaluation of LLMs
Please make a vido with demo that robot perform task using LLM
from where did you learn ? can you share some resources , so that i can learn all of these from one place ?
waiting for the ollama video buddy
Hello sir ❤
Can you make a welding detection project using AI
Is it possible to you or not
If you make then please make
You do not need an LLM model for it. Wedding possibility is a classification problem and it is very easy to make a model if you know logistic regression or decision tree algorithm. These are week learners with a low accuracy but your use case is solvable by the said algorithms.
@@ashishmehra5143 you can do this project then please make a video
Hi folks, can anyone help?
he had taken from the hugging-face for the final demonstration of the output but we need to test the fine-tuned model right?
When i load the model. I facing error config.json not appear. And my model saved adapter_config.json
Please provide solution.......
i created my owndata set with my company data.It is showing wrong answers why?
Hello krish could you please make videos on Auto gen?
Can we finetune llama3 model for machine translation task
Hi Krish, Hi ,
I want to fine tune a code generator model with our organisational data specific to embedded Software. code generated should be specific to the chipset we are using. I was thinking of using starcoder/CodeLLAMA as a base model and fine tune with QLORA. But I dont have much clarity of the format in which I should prepare the custom data set. Can you please help on this. Will joining the group with subscription will help to get some 1:1 guidance
How can I save the fine tuned model locally?
looks like a typo!! Are we not supposed to be using "new_model" instead of "model" while testing the fine tuned model? I'm referring to this line-----> pipeline(task="text-generation", model=model, tokenizer=tokenizer, max_length=200). I think correct argument should be --> model=new_model instead of model=model ??
RuntimeError: Found no NVIDIA driver on your system. Please check that you have an NVIDIA GPU and installed a driver. I'm facing this error in google colab while running the GPU compatibility part. What can be the solution?
How can I fine-tune the LLAMA 3 8B model for free on my local hardware, specifically a ThinkStation P620 Tower Workstation with an AMD Ryzen Threadripper PRO 5945WX processor, 128 GB DDR4 RAM, and two NVIDIA RTX A4000 16GB GPUs in SLI? I am new to this and have prepared a dataset for training. Is this feasible?
hey Krish, why I'm not able to see course request form on TechNeuron and why there is no new content?
downloaded the dataset and entered my own one prompt template by replacing other, and was not able to achieve the result for it, Kindly help me.
Good tutorial!
Hi krish sir!
I have worked on this script 2 days back, and I am eagerly waiting for your explanation about this llama.
And my doubt is that,
I think you didn't gone through this cell,
"Reload the fp16 and merge it with lora weights " Explain this code cell and how it will merge and where it could be stored. For this particular block error : I'm getting out of the memory issue .
And waiting for math's behind peft and theriotical knowledge...
I hope this comment you will read,,
And hope for response to my question!!!
Thank you🌹
you should change your google Colab memory runtime
@@SayaliYadav-h7xi have worked with it's T4 gpu.. Do i need to change to another runtime?
I want to integrate my database to the llm model , is it possible to finetune that. Can you show demo for integrating databses and fine tuning the llm model based on it.
Hello Krish, May I know how can we deploy this as an app in a kubernetis enviroment. thanks
I have a 50k sample dataset i want to fine tune the model.. can i do with this code ??
How can i get my model in gguf format?
Can you please update the link to the code? The one given in the description does not work anymore
Hi krish, If I have client data and don't want to load from huggingface then How can I do this?
Please make video on oneDNN
Can we use LLAMA for urdu language applications?
Will the same code work for llama 13b chat . If not can you share Collab for fine tuning llama 13b .
Sir, can I run the code on my local vs code?
Yes if your local compute has enough processing power to run it.
need a whole playlist on llms sir
How i can pass my model to .gruff ?
buy 4bit quantization don't u think we will be loosing information ?
There's some reduction in performance but we get low latency and reduced model size. Its trade off.
We are still waiting for theoretical part
why dont you explain the theory also
Next video
At the end of the day js permanent 😂
Hey! Facing this issue with Langcahin loader: AttributeError: module 'numpy.linalg._umath_linalg' has no attribute '_ilp64'
Literally dying to resolve this. Please help
it seems you are following any blog or article to show us how to finetuned, it is not worth. I wish, if you can create your own datasets and then ask any question to that model, would be more practice and useful, I tried doing that but failed, in self made datasets it won't work, but following the pre built datasets works.
This video makes my LoRA hard
Apologies, but upon viewing your video, I found the explanation lacking. It's crucial to have a strong initial explanation, especially when presenting more intricate concepts like fine tuning.
He said it's an overview. Just a rundown before starting the video:
CaSUal nhi sir ji....CaUSal...!!!
The video was just to show off your knowledge, it wasn't a tutorial to anything
Great video - thanks! Quick question: How do you actually get the finetuned model off of Colab and use it in other projects?
why we set fp16=False,
bf16=False in training_arguments = TrainingArguments() ?
Hi Your video is amaizing but I am not able to assess the code can provide me the github link I can utilize it.
Please please............
Does anyone retrain LLma 2 on monolingual data (let's say some data in a sepecific low resource language XX) and then finetune it on parallel data en - XX ?
Hi Krish,
Thanks for the video.
What is the purpose of developing the model using PEFT? Is the objective is to mimic CHATGPT where you ask questions and you get the answer?
great tutorial
Can you tell me what dataset templates should be used for fine-tuning? What fields should be there? If I need the model to answer questions in chat mode, like a first-line support bot, make a summary of the text I insert - are these different sets and as a result different models? That is, it turns out that I need already 2 models, each solves a specific task? If, for example, there is a production department and a financial department in the company, then is it better to use 2 separate small models and they are tailored to a separate knowledge sphere or use one large one? Show how to fine-tune on a local computer in the VSC environment on an RTX4090 video card