Your idea of having dedicated qlora models as experts is fascinating to me. One of the things i like doing is having AIs emulate a "Council" of various historical figures. Like having Marcus Aurelius, Einstein, and Prince "sit" in a roundtable dicussion of whatever random idea i have. I can only imagine having a specific high quality lora dedicated purely to Marcus's work would improve his emulation.
Idea. Have multiple qlora models and use either an LLM agent or RL agent to choose which model to use for tasks, then use the different model for inference, or string them together to complete complex tasks
@@Prof1Patel I haven't heard of one. I haven't even heard of a way to load multiple qlora's and activate them individually for each instance. My coding skills are unfortunately too weak to pull something like that off to do it myself. Best I can do is prompting the emulation.
ปีที่แล้ว
The MoE made with several QLora adapaters, I have not heard of it before. But the concept of using diffferent QLora adapters in a 'Nesperesso' approach where you provide a "Market Place" of adapters and you customize the generic pre-trained model on the fly depending the context, ... I have implemented. Of course, you cannot switch between adapters for each interaction human-model but at the begin of a session, it is totally doable.
@ Any chance you could try switching the qlora to a different one without reloading the entire model? If the delay for that is relatively short due to keeping all them in ram it'd be worthwhile. Ask a question, present an idea, then let it run overnight locally talking to itself and you can then use something like claude to give a summary the next day? How feasible do you think that idea is?
Having models individualise will definitely allow them to be distinct from other models out in the market and form a sense of personality / humor (and if trained with British data, some sarcasm too). It will allow them to be more "realistic" especially if trained in a way that allows them to conceptualise the information they receive. I think retaining their conversations and then training the models on those conversations will be the way to do that. Thank you for this video, great to watch and precise to its points.
that's awesome man, thank you for sharing this. if you dig deeper into that, I'd love to see a more step-by-step tutorial, maybe highlighting some aspects or nuances of the process, some theory behind it, etc.
LoRA in SD1.5 and SDXL models is HUGE because its easy to train, easy to use ( WebUI ), easy to swap or stack and easy to find with civitai. QLoRa for LLM needs the same requirements especially the Adapter you were talking about. I Would love a QLoRa trained on Text based adventure games (Correct me if I'm wrong , still New to LLMs)
Lora were first invented for llm ironically enough. Not Lora, but the first case of fine tuning was ai dungeon 2.0, to emulate text based adventure, which was the inspiration for current chatbot architecture.
I have played around with your 13B model and converted it to llama-cpp (quantized 8 bits)... I am just blown away by how natural and lifelike the response are compared to anything else, even the GPT-4 responses seem very robotic and "HR" like in comparison. I can now run inference on my crappy laptop using just CPU and RAM, I don't even have a dedicated GPU on this machine. It is surprisingly quick with an average inference time of about 10-15 seconds (this is with the 13B model, the 7B is very fast at like 3-4 seconds max). The next step would be to increase the dataset with fresh data gathered from the forum and run another QLoRA pass. Also I was thinking about having an active bot posting on the forum, then harvest the responses and how many upvotes the bots get. This data could then theoretically be used for additional RLHF training (although I have no idea on how to do that yet lol). Anyway thanks for this!
I am trying to build a multi class classification model for my work, I have minimal data (3000 rows of labelled data), can I utilise the llama 7B for this somehow? Can you share more info on how you converted the 13B to llama-cpp?
I mostly use GPT for sentiment rewriting, for example, "rewrite the following text better" or "rewrite the following text more eloquently" (if I want to sound super smart), etc. I would love to see how these types of instructions could be trained on open-source LLMs.
You can instruct a model to have emotions and even emotional goals. I tested this asked it to create a list of emotions and physical needs handle some of these as a time function (hunger sleep etc) others based on the discussion. Then tell it to keep a reward cost system and then it can get quit real. I tried it multiple times and it worked sometimes, better might be to add instead of a point system a simple neural reward system. Easier for time based emotions. Best would be I think to have some long-term memmory ( value keeping system so it doesn't forget values or hallucinate them)
You have changed my life bro. Been worried about AI and how it would transform and disrupt our lives since 2018. The only one with any actual clarity on the subject which could resonate with my mind was you.
Awesome video! You should make a video about the GPTQ algorithm and autoGPTQ. Unlike QLoRa however, you need to finetune the model before applying GPTQ since its a one shot. However it provides faster inference. You could finetune a model using QLoRa and then use autoGPTQ to upload it on a consumer GPU
Gptq is a quantization technique. what to make video about it ? And it became popular cuz of llama.cpp and then they themselves moved to gguf, which is future proof cuz it retains meta information.
@@ashu- you are right, i forgot to mention that i was referring to quantization using QLoRa and bitsandbytes And its not something to do with the video fully, just something within that realm. My apologies 🙏
Not sure if intended, or some side effects of compression or something, but to me it seems that the video introduces some micro stuttering, at least with 60fps quality options. It's especially visible when you're in motion. I understand a thing or two about cameras, and would suggest checking your camera's shutter speed if it's at a too slow setting (if the camera has the options for changing it, ofcourse). A good rule of thumb for videos is that the shutter speed should be at least double the intended video framerate, so a minimum of 1/120th of a second for 60fps video. I've noticed the fps thing already a while back when watching your videos, and don't know if it even bothers you or anyone else. Anyways, just decided to try and give constructive feedback if it'd help. Don't hate plz 🤓And thanks for the content, good stuff!
About models for mobiles and small devices, did you check Tflite? I'm trying to find if I can modify a llama2 model to run on tflite , does anyone know that?
Thanks for the video! It would be great if you can create a video on using QLoRa to fine-tune BERT-derived models on MLM task. These models could also benefit from efficient training techniques developed for generative models.
Compressor are predictor, in between gzip joint probability compression and quantization, we will arrive at this middle point where we do lossy compression as learning 😂😂
what we need is "adaptive personality" on top of a LLM... kinda like LoRA, but not pre-trained, and each one always training to develop their own personality... just like humans are
This is good stuff it almost connects the last two missing pieces in my brain, I’ve fine tuned some Lora adapters 1. To use them as an llm do I need to merge these all back together to get a gguf? File I can use in day lm studio? 2. If that is not the case is there a sample colab or script that shows how to use the base model+ weightings and is this the preferred technique Thanks!
I really would love to read/watch more about the MoE Lora project! Where can I follow this?
ปีที่แล้ว +1
Very cool video :) Every time I watch your videos I get curious thought about what is the story behind the USB hub on the microphone arm. :D Also if this trains so fast and with so little data, can this be run locally on a 24GB GPU?
Yes you can absolutely QLoRA fine-tune a 7B model on a 24GB GPU (assuming relatively recent for quantization requirements). I think you need ampere, so 3090 or 4090 for consumer GPUs, for example. The USB hub is there to combat the "Yeti whine" issue with this particular microphone. It allows me to add USB power which seems to help make the whine go away.
Just to explain what rank is to anyone interested: *The basics:* _A vector._ In 2d that's a certain amount on the x and a certain amount on the y. So [1, 0] - pointing towards the x with a length of 1. [1, 0, 0] is a 3d vector. [1, 0, 0, 0] is 4d. _A basis matrix._ This contains more than one vector. When you plot on an everyday graph you have the x pointing to the right and the y up. And you plot your vector against that. Well, you can use 2 vectors to describe that. [[1, 0], [0, 1]] one vector pointing to the right and one up. If you changed that to [[2, 0], [0, 2]] in your basis matrix then your vector [1, 0] would suddenly be twice as long. I hope you can imagine how useful that is in graphics engines. *"Get on with it! Tell me what a rank is!"* Imagine your basis matrix is [[0, 1], [0, 1]] - here is the x and y bases are both pointing in the same direction. So no matter what vector you have you can never enter the second dimension. You're stuck on a line. So, the concept of rank tells you it's 1. Now imagine you have a huge matrix of thousands, even millions of dimensions in a neutral net. If your rank is less than your dimensionality then you're inefficiently wasting computational resources and you have an inelegant solution. If you're using 100 dimensions to describe 3d space then you've gone very wrong. You need to boil it down to 3 dimensions but that's what's in the geometry.
Thank you so much for this video! I need to training my model now and I'll use QLoRa. Good timing. Did you tried the model "TheBloke/Mistral-7B-OpenOrca-AWQ"? The original model (Mistral 7B Orca) it's a bit better than Llama2, the size is really small (
Do you have a video were we can learn about these llms from scratch? How to got about learning llms to build actually practical application in real World etc.
With llama2-7b-chat it is quite easy to make it have a personality with just adjusting the system prompt. I've had a blast with with something like "You are a cat named Sir Sire, you only know about cat-stuff." Very funny responses including a lot of emoticons etc. Try asking it about (atomic) bombs in example.
Would you use LLAMA-2 or LLAMA-2-Chat if you wanted to make an expert on a certain topic? I imagine it's easier to fine tune LLAMA-2 on some documentation, because you don't have to create a "question-answer" format, you simply make LLAMA-2 read the docs
i haven't played that much w/ the non-chat llama 2 specifically but in general the untrained models aren't so much more flexible as they are just utterly aimless, like if you give them a question they might answer w/ more questions that are similar b/c maybe we're making a list of questions who knows what's going on, the basic orientation given to the chat models that there's user messages coming in w/ intentions and they're supposed to do something in particular is pretty useful generally
@@mungojelly Agreed, also I've looked at th-cam.com/video/g68qlo9Izf0/w-d-xo.html , they say finetuning a model is not for making it acquire new knowledge, better to use retrieval
Hey awesome video. I have a few of questions: 1. Why is LoRA (QLoRA) applied only on the q and v matrices of the transformer? 2. What is LoRA applied only on ∆W, and not on the weights themselves? 3. Why can't LoRA be used for pre-training, but only fine-tuning? Thanks :)
Have you considered or tried running qlora on a mobile device? We were looking for good tech interfacing for this and weren't able to find many good ways to get started.
I loved your video. I am understanding a bit more thanks to it. Can you share the colab you used in your learning journey? I would like to follow the video with it. I joined Discord channel in hopes you were sharing it there. I want to try training a dataset in another language.
Please finish the neural nets from scratch in python series. Only course that is any good at explaining them, shame for it to be unfished. Love your vids!
I don't understand how you can only use the adapters that are 200MB for inference. I understand that when you fine-tune a model using QLoRA, you basically freeze most parameters and only update/ retrain a small portion of the full model. However, when running the inference you still need to load and use the full model or am I wrong? Can someone explain please.
I was wondering... What's the best approach of "arranging the dataset" so it will reflect more than 1 reply per message? Did you simply concatenated all replies to a single one?
Any idea regarding hardware specs required to fine-tune LLama2-7b using Qlora? I have read that 12 GB VRAM is enough but I have been struggling to make it work for a while.
tbh if people get offended by just this much swearing, it kind of implies how childish society is becoming. great video tho! exciting improvements. just hope that the alignment people can stop decapitating these large models.
🎯 Key Takeaways for quick navigation: 00:13 😄 The current state of AI conversations is often perceived as cold and boring. Many desire AI with personality and humor. 01:09 🧠 QLoRA (Quantized Low Rank Adapters) is a technique developed based on Facebook's research. It can significantly reduce trainable parameters, making fine-tuning faster and more memory-efficient. 02:16 🔍 QLoRA uses two matrices (A and B) to lower dimensionality, reducing the complexity of fine-tuning compared to traditional methods. 03:40 💡 QLoRA allows fine-tuning with very few samples, as low as a thousand, making it versatile for various generative text tasks. 07:20 🤖 When training AI models, careful consideration of the dataset is essential to avoid generating offensive or inappropriate content. 15:08 🚀 QLoRA adapters are lightweight, enabling the creation of a mixture of experts with small memory footprints for versatile AI applications. 22:02 😆 The speaker values AI models with character, humor, and personality, hoping for more models that can genuinely make people laugh and engage in fun conversations. Made with HARPA AI
I've trained a few QLora models. Whenever I increase the epochs it just creates the last token until it rans out of context size. I couldn't solve that issue
Can you please make a video about gaussian splattering? I've seen it being implemented in polycam and it seems to be in research right now. It is a technique of photogrammetry which seems to have been trained on a large amount of 3D models to make 3D models from a bunch of photos. Greetings from Germany 😊
You just forgot to mention there are a million of caveats regarding use-case, model-data fitting and so on... Only if it was that simple indeed hahahaha.
Extra confusing because LoRa and LORA refer to something from an adjacent field, but really have nothing to do with LoRA. Someone made some bad choices in naming LoRA. I could see an absurd scenario occurring where someone utilizes LoRA when designing advanced error correction for something over LORA.
Is it possible to train a Llama 7B QLoRA locally on a 4090? Or does it absolutely need server GPUs? I do SDXL training and have never looked into LLM training before.
Does anybody know if QLoRA can be used to fine tune LLMs for regression tasks? I've seen examples for Classification and LoraConfig has a task type specific to classification but I haven't been able to find anything for regression examples. For example, grading student essays on a scale from 0-100.
Did you mention something about a GPU sniping script? -- Is that linked somewhere? -- I'm not finding it. Maybe somebody can point me in the right direction.
nope! fine tuning will only surface knowledge that's already in the model, and they took out all of the non-english data from the training set for llama,,,,, you might have better luck w/ the falcon models, i think they have some turkish in their dataset
Interesting perspective when I hear people want to see personality from something non-human ...yet here I am and I want to see A.I. as close to being "Vulkan" as you can possibly get (meaning its function should be focused as a utility).... the more human-like A.I. gets the less human interaction humans may need to be..... Interesting note that I recently heard was "your children's first childhood crush will be an A.I."..... let that sink in .... hence I feel it should focus on being a utility without emotion or the ability to use psychological methods in it's deliver of logic and data
Does anyone know of a good tutorial to add QLoRA to your own (custom) models? And/or some tutorials that implement QLoRA from scratch? (Preferably in PyTorch).
What I hate about ChatGPT and Bing is how much handwringing, boiletplate nonsense, caveats it adds. It feels like being lectured to instead of getting an answer to a question. If ChatGPT was a real person irl, I would not go to that person for anything. So it's good to have models that are trained differently or have actually different personalities.
you can also just ask it to communicate differently w/ you ,, here are some requests that work well for me personally w/ chatgpt "use a broad vocabulary and make obscure references" and i just started using "give me a bunch of new ideas about this that i probably wouldn't have already thought of" ,, gpt4 is substantially better both at following communication style requests & also at having something to say, i wouldn't actually expect very much out of 3.5, but actually you can get pretty good stuff from it just by asking for a second or third draft of something w/ specific style notes or directions, if you automate taking it through a few layers of changes gpt-3.5-turbo can do a lot, just uh, one thing at a time, gpt4 can do more like a dozen things at once which is um difficult to make full use of
![[#2024MAMA] BIGBANG (빅뱅) - 뱅뱅뱅 (BANG BANG BANG) + FANTASTIC BABY | Mnet 241123 방송](http://i.ytimg.com/vi/EQsYfiPR9uM/mqdefault.jpg)
You need more appreciation than any AI gurus.
AI "Gurus" vs an actual expert who has done this for years
I think he gets that appreciation too. Just look at his subs. For such deeply tech oriented channel it's difficult to get over a million
Appreciation is all you need!
@@heyyounotyouyou3761 but he was one of the great minds in programming community. Underrated I would say
i feel you, the space is filled with low quality content. the karpathy videos great though
Your idea of having dedicated qlora models as experts is fascinating to me. One of the things i like doing is having AIs emulate a "Council" of various historical figures. Like having Marcus Aurelius, Einstein, and Prince "sit" in a roundtable dicussion of whatever random idea i have. I can only imagine having a specific high quality lora dedicated purely to Marcus's work would improve his emulation.
Idea. Have multiple qlora models and use either an LLM agent or RL agent to choose which model to use for tasks, then use the different model for inference, or string them together to complete complex tasks
Would anyone be interested in making this idea happen? Has anyone tried this?
@@Prof1Patel I haven't heard of one. I haven't even heard of a way to load multiple qlora's and activate them individually for each instance. My coding skills are unfortunately too weak to pull something like that off to do it myself. Best I can do is prompting the emulation.
The MoE made with several QLora adapaters, I have not heard of it before. But the concept of using diffferent QLora adapters in a 'Nesperesso' approach where you provide a "Market Place" of adapters and you customize the generic pre-trained model on the fly depending the context, ... I have implemented. Of course, you cannot switch between adapters for each interaction human-model but at the begin of a session, it is totally doable.
@ Any chance you could try switching the qlora to a different one without reloading the entire model? If the delay for that is relatively short due to keeping all them in ram it'd be worthwhile. Ask a question, present an idea, then let it run overnight locally talking to itself and you can then use something like claude to give a summary the next day? How feasible do you think that idea is?
i feel exactly the same as you concerning how i want a chatbot to be like. Excited to test your wsb model.
Having models individualise will definitely allow them to be distinct from other models out in the market and form a sense of personality / humor (and if trained with British data, some sarcasm too). It will allow them to be more "realistic" especially if trained in a way that allows them to conceptualise the information they receive. I think retaining their conversations and then training the models on those conversations will be the way to do that.
Thank you for this video, great to watch and precise to its points.
Honestly man’s a genius
Thank you for sharing this information! Fast and lightweight fine-tuning is a topic that many of us in the AI community are interested in.
LoRA has been around for generative image for months. Cool to see we got them for LLMs now.
that's awesome man, thank you for sharing this.
if you dig deeper into that, I'd love to see a more step-by-step tutorial, maybe highlighting some aspects or nuances of the process, some theory behind it, etc.
🤣 - I LOVE that you chose WSB to train this. Legend.
Yoo, I have been sleeping on QLoRA for a long time. Will try it soon. Nice vid
LoRA in SD1.5 and SDXL models is HUGE because its easy to train, easy to use ( WebUI ), easy to swap or stack and easy to find with civitai. QLoRa for LLM needs the same requirements especially the Adapter you were talking about. I Would love a QLoRa trained on Text based adventure games (Correct me if I'm wrong , still New to LLMs)
Lora were first invented for llm ironically enough. Not Lora, but the first case of fine tuning was ai dungeon 2.0, to emulate text based adventure, which was the inspiration for current chatbot architecture.
@@NeoShameMan Aidungeon 2 used GPT-2
@@mikeyjohnson5888 no shit Sherlock, can you make a distinction between methodology and model?
@@NeoShameMan Dude, I wasn't arguing. Just adding that was the model used so people would know for posterity's sake.
Nerd fight !
This sh#ts gonna get real...
Thank you for this awesome video!
A Norm MacDonald LoRA would be awesome.
I have played around with your 13B model and converted it to llama-cpp (quantized 8 bits)... I am just blown away by how natural and lifelike the response are compared to anything else, even the GPT-4 responses seem very robotic and "HR" like in comparison. I can now run inference on my crappy laptop using just CPU and RAM, I don't even have a dedicated GPU on this machine. It is surprisingly quick with an average inference time of about 10-15 seconds (this is with the 13B model, the 7B is very fast at like 3-4 seconds max).
The next step would be to increase the dataset with fresh data gathered from the forum and run another QLoRA pass. Also I was thinking about having an active bot posting on the forum, then harvest the responses and how many upvotes the bots get. This data could then theoretically be used for additional RLHF training (although I have no idea on how to do that yet lol).
Anyway thanks for this!
I am trying to build a multi class classification model for my work, I have minimal data (3000 rows of labelled data), can I utilise the llama 7B for this somehow? Can you share more info on how you converted the 13B to llama-cpp?
Always super interesting content from this channel. Thanks 🙏
@sentdex you’re all we need in AI ecosystem
Facts
I mostly use GPT for sentiment rewriting, for example, "rewrite the following text better" or "rewrite the following text more eloquently" (if I want to sound super smart), etc. I would love to see how these types of instructions could be trained on open-source LLMs.
Thank you. Congrats on keeping that hair line 🎉
gpt 4chan was legendary.
content is all you need
You can instruct a model to have emotions and even emotional goals. I tested this asked it to create a list of emotions and physical needs handle some of these as a time function (hunger sleep etc) others based on the discussion. Then tell it to keep a reward cost system and then it can get quit real. I tried it multiple times and it worked sometimes, better might be to add instead of a point system a simple neural reward system. Easier for time based emotions.
Best would be I think to have some long-term memmory ( value keeping system so it doesn't forget values or hallucinate them)
You have changed my life bro. Been worried about AI and how it would transform and disrupt our lives since 2018. The only one with any actual clarity on the subject which could resonate with my mind was you.
Great video Really you're doing amazing work 🎉
Awesome video! You should make a video about the GPTQ algorithm and autoGPTQ.
Unlike QLoRa however, you need to finetune the model before applying GPTQ since its a one shot.
However it provides faster inference.
You could finetune a model using QLoRa and then use autoGPTQ to upload it on a consumer GPU
Gptq is a quantization technique. what to make video about it ? And it became popular cuz of llama.cpp and then they themselves moved to gguf, which is future proof cuz it retains meta information.
@@ashu- you are right, i forgot to mention that i was referring to quantization using QLoRa and bitsandbytes
And its not something to do with the video fully, just something within that realm.
My apologies 🙏
Cool bro
Lol the response from the AI wasn't wrong though 😂
Love ur channel. Good content!
Not sure if intended, or some side effects of compression or something, but to me it seems that the video introduces some micro stuttering, at least with 60fps quality options. It's especially visible when you're in motion. I understand a thing or two about cameras, and would suggest checking your camera's shutter speed if it's at a too slow setting (if the camera has the options for changing it, ofcourse). A good rule of thumb for videos is that the shutter speed should be at least double the intended video framerate, so a minimum of 1/120th of a second for 60fps video.
I've noticed the fps thing already a while back when watching your videos, and don't know if it even bothers you or anyone else. Anyways, just decided to try and give constructive feedback if it'd help. Don't hate plz 🤓And thanks for the content, good stuff!
DhanOS will improve the SentDex Avatar, thanks
Wow really nice explanation
Thanks!
I enjoyed you video and I too want a less corporate chat model.
Awsome.. Thanks for this. ❤❤❤
About models for mobiles and small devices, did you check Tflite? I'm trying to find if I can modify a llama2 model to run on tflite , does anyone know that?
Thanks for the video! It would be great if you can create a video on using QLoRa to fine-tune BERT-derived models on MLM task. These models could also benefit from efficient training techniques developed for generative models.
16:00 I feel like you're describing GLaDOS
with Personality Cores
Compressor are predictor, in between gzip joint probability compression and quantization, we will arrive at this middle point where we do lossy compression as learning 😂😂
what we need is "adaptive personality" on top of a LLM... kinda like LoRA, but not pre-trained, and each one always training to develop their own personality... just like humans are
This is good stuff it almost connects the last two missing pieces in my brain, I’ve fine tuned some Lora adapters
1. To use them as an llm do I need to merge these all back together to get a gguf? File I can use in day lm studio?
2. If that is not the case is there a sample colab or script that shows how to use the base model+ weightings and is this the preferred technique
Thanks!
I really would love to read/watch more about the MoE Lora project! Where can I follow this?
Very cool video :) Every time I watch your videos I get curious thought about what is the story behind the USB hub on the microphone arm. :D
Also if this trains so fast and with so little data, can this be run locally on a 24GB GPU?
Yes you can absolutely QLoRA fine-tune a 7B model on a 24GB GPU (assuming relatively recent for quantization requirements). I think you need ampere, so 3090 or 4090 for consumer GPUs, for example. The USB hub is there to combat the "Yeti whine" issue with this particular microphone. It allows me to add USB power which seems to help make the whine go away.
Part 10 of Neural Net from Scratch, about analytical derivatives??? Please bring the series back!
Dude these qLora models can be reduced down to 9mb even lol it's so good
thank you a lot!
Could the problem be related to the original llama EOS token having an unsupported value of -1? I heard choosing a rare token like 18610 could work.
Just to explain what rank is to anyone interested:
*The basics:*
_A vector._ In 2d that's a certain amount on the x and a certain amount on the y. So [1, 0] - pointing towards the x with a length of 1. [1, 0, 0] is a 3d vector. [1, 0, 0, 0] is 4d.
_A basis matrix._ This contains more than one vector. When you plot on an everyday graph you have the x pointing to the right and the y up. And you plot your vector against that. Well, you can use 2 vectors to describe that. [[1, 0], [0, 1]] one vector pointing to the right and one up. If you changed that to [[2, 0], [0, 2]] in your basis matrix then your vector [1, 0] would suddenly be twice as long. I hope you can imagine how useful that is in graphics engines.
*"Get on with it! Tell me what a rank is!"*
Imagine your basis matrix is [[0, 1], [0, 1]] - here is the x and y bases are both pointing in the same direction. So no matter what vector you have you can never enter the second dimension. You're stuck on a line. So, the concept of rank tells you it's 1.
Now imagine you have a huge matrix of thousands, even millions of dimensions in a neutral net. If your rank is less than your dimensionality then you're inefficiently wasting computational resources and you have an inelegant solution. If you're using 100 dimensions to describe 3d space then you've gone very wrong.
You need to boil it down to 3 dimensions but that's what's in the geometry.
Thank you so much for this video! I need to training my model now and I'll use QLoRa. Good timing.
Did you tried the model "TheBloke/Mistral-7B-OpenOrca-AWQ"? The original model (Mistral 7B Orca) it's a bit better than Llama2, the size is really small (
I think a quantized 7b model with an archive of 1000s of 100mb models on cloud or etc would be such a good step for getting these llms local
Receipe for killer Llamas
Thanks!
Do you have a video were we can learn about these llms from scratch? How to got about learning llms to build actually practical application in real World etc.
Karpathy himself has an excellent series on yt, concluding with a small GPT from scratch.
As Mr Rollo suggested, I can't imagine a better source than Karpathy's video ATM: th-cam.com/video/kCc8FmEb1nY/w-d-xo.html
Thank you! @@sentdex
Next video should be: VLLM and AWS is all you need :)
With llama2-7b-chat it is quite easy to make it have a personality with just adjusting the system prompt. I've had a blast with with something like "You are a cat named Sir Sire, you only know about cat-stuff." Very funny responses including a lot of emoticons etc. Try asking it about (atomic) bombs in example.
Would you use LLAMA-2 or LLAMA-2-Chat if you wanted to make an expert on a certain topic? I imagine it's easier to fine tune LLAMA-2 on some documentation, because you don't have to create a "question-answer" format, you simply make LLAMA-2 read the docs
i haven't played that much w/ the non-chat llama 2 specifically but in general the untrained models aren't so much more flexible as they are just utterly aimless, like if you give them a question they might answer w/ more questions that are similar b/c maybe we're making a list of questions who knows what's going on, the basic orientation given to the chat models that there's user messages coming in w/ intentions and they're supposed to do something in particular is pretty useful generally
@@mungojelly Agreed, also I've looked at th-cam.com/video/g68qlo9Izf0/w-d-xo.html , they say finetuning a model is not for making it acquire new knowledge, better to use retrieval
19:32 i feel like gpt-4 does that when a person decides to use a persona in the prompts. Especially when using custom instructions.
Yeah a little more character for chatbot does sound good for me
21:04 Hilarious [Terraform i see you]
😂😂
Hi, you mentioned 'skunkworks' ... Is this a private internal project team or a third party involved in similar tech? Thanks
Lmao at your model called u out, 🤣 truly based on reddit
Hey awesome video. I have a few of questions:
1. Why is LoRA (QLoRA) applied only on the q and v matrices of the transformer?
2. What is LoRA applied only on ∆W, and not on the weights themselves?
3. Why can't LoRA be used for pre-training, but only fine-tuning?
Thanks :)
Speaking of running on your phone there are some promising projects like Medusa from TogetherAI and DeciLM 6B.
❤
When should one opt to fine tune instead of using a Voiceflow / Botpress ai bot for Q&A? Also what the difference between qlora and gradient is?
Have you considered or tried running qlora on a mobile device? We were looking for good tech interfacing for this and weren't able to find many good ways to get started.
We need more models that will call people out. 😂
Fun AI! Fun AI! Fun AI!
Hell ya
yannik kilcher has a quiet good youtube channel too. And he himself seems very decent to me. I am not sure why he trained a LLM on 4chan.
I loved your video. I am understanding a bit more thanks to it. Can you share the colab you used in your learning journey? I would like to follow the video with it. I joined Discord channel in hopes you were sharing it there. I want to try training a dataset in another language.
Please finish the neural nets from scratch in python series. Only course that is any good at explaining them, shame for it to be unfished. Love your vids!
I don't understand how you can only use the adapters that are 200MB for inference. I understand that when you fine-tune a model using QLoRA, you basically freeze most parameters and only update/ retrain a small portion of the full model. However, when running the inference you still need to load and use the full model or am I wrong? Can someone explain please.
I was wondering...
What's the best approach of "arranging the dataset" so it will reflect more than 1 reply per message?
Did you simply concatenated all replies to a single one?
I have a question: is it possible to tune it based on xml inventory data, so the model could give answers derived from inventory data?
Do you think LoRa or QLoRa would be useful if I wanted to add new vocabulary to a pretrained translation model and finetune it?
Any idea regarding hardware specs required to fine-tune LLama2-7b using Qlora? I have read that 12 GB VRAM is enough but I have been struggling to make it work for a while.
Free of speech is amazing!
I would like to see you try Falcon 180B if possible
tbh if people get offended by just this much swearing, it kind of implies how childish society is becoming. great video tho! exciting improvements. just hope that the alignment people can stop decapitating these large models.
🎯 Key Takeaways for quick navigation:
00:13 😄 The current state of AI conversations is often perceived as cold and boring. Many desire AI with personality and humor.
01:09 🧠 QLoRA (Quantized Low Rank Adapters) is a technique developed based on Facebook's research. It can significantly reduce trainable parameters, making fine-tuning faster and more memory-efficient.
02:16 🔍 QLoRA uses two matrices (A and B) to lower dimensionality, reducing the complexity of fine-tuning compared to traditional methods.
03:40 💡 QLoRA allows fine-tuning with very few samples, as low as a thousand, making it versatile for various generative text tasks.
07:20 🤖 When training AI models, careful consideration of the dataset is essential to avoid generating offensive or inappropriate content.
15:08 🚀 QLoRA adapters are lightweight, enabling the creation of a mixture of experts with small memory footprints for versatile AI applications.
22:02 😆 The speaker values AI models with character, humor, and personality, hoping for more models that can genuinely make people laugh and engage in fun conversations.
Made with HARPA AI
Hi can you please give link to the UI code also??
I've trained a few QLora models. Whenever I increase the epochs it just creates the last token until it rans out of context size. I couldn't solve that issue
is possible train a "expert" on documentation for certain topic and still getting a chat output?
yes, it is however mostly done using prompts and vector db. Not fine tuning.
Can you please make a video about gaussian splattering? I've seen it being implemented in polycam and it seems to be in research right now. It is a technique of photogrammetry which seems to have been trained on a large amount of 3D models to make 3D models from a bunch of photos. Greetings from Germany 😊
Would you be able to do a video on using llama 2 for AI agents? I tried but couldn’t get anything like gpt-4
How to use VLLM with quantized model?
You just forgot to mention there are a million of caveats regarding use-case, model-data fitting and so on... Only if it was that simple indeed hahahaha.
Extra confusing because LoRa and LORA refer to something from an adjacent field, but really have nothing to do with LoRA. Someone made some bad choices in naming LoRA. I could see an absurd scenario occurring where someone utilizes LoRA when designing advanced error correction for something over LORA.
Is it possible to train a Llama 7B QLoRA locally on a 4090? Or does it absolutely need server GPUs? I do SDXL training and have never looked into LLM training before.
You can comfortably QLoRA a 7B model on your 4090, yes.
@@sentdex I might have to give it a go. What's the smallest dataset you used that gave interesting or fun results?
Can you please make a video on context based Machine learning ?
Could the qlora fine tuning be done locally eg with 4090?
Does anybody know if QLoRA can be used to fine tune LLMs for regression tasks? I've seen examples for Classification and LoraConfig has a task type specific to classification but I haven't been able to find anything for regression examples. For example, grading student essays on a scale from 0-100.
Did you mention something about a GPU sniping script? -- Is that linked somewhere? -- I'm not finding it. Maybe somebody can point me in the right direction.
Can I use QLORA or LORA to train e.g. LLAMA2 for a new language (in my case Turkish)?
nope! fine tuning will only surface knowledge that's already in the model, and they took out all of the non-english data from the training set for llama,,,,, you might have better luck w/ the falcon models, i think they have some turkish in their dataset
Any reason you didn't provide the script you used to actually train the model?
Interesting perspective when I hear people want to see personality from something non-human ...yet here I am and I want to see A.I. as close to being "Vulkan" as you can possibly get (meaning its function should be focused as a utility).... the more human-like A.I. gets the less human interaction humans may need to be..... Interesting note that I recently heard was "your children's first childhood crush will be an A.I."..... let that sink in .... hence I feel it should focus on being a utility without emotion or the ability to use psychological methods in it's deliver of logic and data
I'm really curious if anyone knows why he's having issues with EOS token...
Mine has a filthy mouth. I named her Lilly. She's the best.
What are your opinions on mojo/modular
Does anyone know of a good tutorial to add QLoRA to your own (custom) models? And/or some tutorials that implement QLoRA from scratch? (Preferably in PyTorch).
hari seldon is prolly some model programmed in Kabbala or some shit
What I hate about ChatGPT and Bing is how much handwringing, boiletplate nonsense, caveats it adds. It feels like being lectured to instead of getting an answer to a question. If ChatGPT was a real person irl, I would not go to that person for anything. So it's good to have models that are trained differently or have actually different personalities.
you can also just ask it to communicate differently w/ you ,, here are some requests that work well for me personally w/ chatgpt "use a broad vocabulary and make obscure references" and i just started using "give me a bunch of new ideas about this that i probably wouldn't have already thought of" ,, gpt4 is substantially better both at following communication style requests & also at having something to say, i wouldn't actually expect very much out of 3.5, but actually you can get pretty good stuff from it just by asking for a second or third draft of something w/ specific style notes or directions, if you automate taking it through a few layers of changes gpt-3.5-turbo can do a lot, just uh, one thing at a time, gpt4 can do more like a dozen things at once which is um difficult to make full use of
😂😂this was great
I was promised ai but got linear algebra.
Hey thank you. This is great. oh do you think you could do a walkthrough of local llm on windows using team red? #underserved #24gbgamer