I am an AMD fan with a 7900 xtx, so linux only. Did the auto-install work with AMD ? Did it auto-install all the rocm ? Je suis un fan AMD, avec une 7900 xtx, donc linux seulement. Es que l'installateur fonctionne avec AMD ? Es que l'installateur fait aussi l'installation de rocm ?
Remember the good old days when guys would have relationships with real women? Nothing is sadder and more depressing than staring at an AI image, hearing some janky broken AI voice telling you loving things. That's nightmare fuel 😂
As the creator of the Tiefighter model, thanks for showing it in your video! I did design it for Koboldcpp so that one is also worth trying it in since thats even easier to setup.
@@GhOsThPk Luckily it doesn't need models specifically for it, but it does have some modes such as the Adventure mode which Tiefighter was also designed to be able to use. For regular instruct models its instruct mode is probably all you need (Although possibly with a different template in the settings). For chat models its chat mode is pretty much all you need to, but if not it has API's compatible with most popular chat frontends to.
Kobold is also really good, I just prefer the silly tavern UI because it's amazing but I need to reset up kobold so that I can mess with that one again.
It's truly remarkable how you're empowering users with the knowledge to leverage AI without censorship and within their own control. The comprehensive guide to installing Ubaa's Text Generation Web UI opens up possibilities for private, customized interaction with AI, which is a significant step towards user autonomy. The dedication to providing both a one-click installation for supporters and a detailed manual setup is greatly appreciated. Keep up the fantastic work!
Dear @@I-Dophler , I apologize for the lack of originality that may have been present in my recent actions or words. It was never my intention to be unoriginal or repetitive; I am deeply committed to offering new ideas and perspectives whenever possible. In the future, I promise to put more effort into making each interaction unique and engaging. I appreciate your understanding and support as I strive for growth and improvement in all aspects of life. Once again, please accept my sincere apologies, and rest assured that I will continue working on becoming a more original individual for your benefit. Thank you, and have a wonderful day! Yours faithfully, Rexor.
@ReXoRofc It's fascinating to see such advancements in AI technology and the ability to run uncensored models locally. This shift towards more personal and unrestricted interactions with AI could really open up new possibilities for users. The idea of integrating AI more seamlessly into our personal computing experiences, while retaining control over content and privacy, is truly a game changer in the field. Looking forward to exploring these developments further!
An amazing model to check out is Mistral 7B particularity "Dolphin 2.1 Mistrial 7B" it scored a 67% overall on the open_llm_leaderboard. And it supports at least 8K context. This outclasses most if not all 65B models. Most if not all 13B. And a lot of models between those. It's near the top model and even beats some 70B from what I understand. If anyone else has a better 7B model to recommend please let me know. And since 70B can't be used at 8K context for most people. That's another amazing thing about some 7B models.
@gabluz By the way I heard rumours that anything less than about 8Q may not be good for 7B models. So I use GGUF 8-bit. But 5_M or 6-bit might be good enough? Still I heard people strongly recommending 8-bit.
@@zolilio Right I have no idea how all that works. But who ever made the core mistrial model. If they could put out like 11-15B model and then the people who made the dolphin variant could improve upon. That would be great. Now my PC likely could only use 4-8K context with 13B though. 8K might be to much at 13B? But I think I can still pull it off.
I was looking for content like this, but I wasn't finding what I wanted. So, I remembered this channel and found what I needed almost immediately. Every single content I've found before were only about online AI models. Greetings from Brazil!
The last part of the video for me was the best; as I have had issues with image description and the multi-modal extension. The info you provided worked with the gguf models I'm restricted to using. Thank You for the share.
Posting this here so that it gets noticed - I'm finding it a barrier to type in the “N” for the graphics card - the text doesn't show up, and I'm getting through reading the comments
Great to see you back aitrepreneur, some reviews of the latest models like Mistral and tiefighter would be great, also running models in phones is actually possible now, running llama 7B on a snapdragon 8 gen 1 with 12 GB of ram it's not too bad, And I want to see if I can get it to run Mistral.
come on, the image thing isn't just a gimmick. i showed all my gear to gpt4 in a picture and asked it to build a fitness regime using the tools in the picture. it did an amazing job and used everything it had at its disposal. not sure if this model is as capable, but if it's anywhere near, it's pretty cool
Nice but you are wrong telling that gguf format works on CPU... GGUF format is universal it means you can run it on GPU and CPU. Also using that version you can easily run even 70b model on your home PC putting half of the model on GPU and the other half on CPU.
Thank you so much for a very informative comment Can you provide a best MODEL to use and settings for Parameters that I can use with a PC that uses 1080ti Video card and i7 7700k with 64gb ram? Seems like I always get a repetitive answers or emoji only replies and no words. Using TheBloke_Llama-2-7B-Chat-GPTQ and Divine Intelect on parameters. Appreciate your reply! Thanks
@@VinciSid TheBloke_Llama-2-7B-Chat-GPTQ ? You are serious? That model is from stone age era of llms ... With small models try gemma 2 9b, llama-3 8b .
@@mirek190 Yeah, I just finished following the video and completely clueless on what model to use :( Still tinkering with settings.. I'll try your suggestions, Thank you!
DUDE YOU ARE A W E S O M E. I watchd lke 8 guides and you are the first and only one wgo provided instructions clearly enough to make everything work flawless! Instant sub!
I don't see the elevenlabs checkbox in the extension list? I have a different list of available extensions? :S Did they remove it? or do I need to enable it in a different way somehow?
Will be interesting to see the video method vs what we know and can do with the 1-click installation that Ooba suggest by months. Good to see A uploading a new video
This so very much appreciated! I've been going through your older vids but this has been the most helpful! Can you do a video that shows how to connect with TavernAi?
I'm using a Nvidia 3060 (12 GB VRAM) and I couldn't get any AWQ model to work. Don't know why. I reverted to my old GPTQ models and everything seemed fine.
i tried and it helped alot but when it came time ti test the modle it say error none and never resonded plus it said "ModuleNotFoundError: No module named 'awq'" in the model tab no matter what i tried. Any way to help?
Please help someone. No errors were present during installation, but when i try to generate a response with any models it just gives me an error: CUDA error: no kernel image is available for execution on the device CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect. For debugging consider passing CUDA_LAUNCH_BLOCKING=1. Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.
@@666-d5y witch one did you end up using and was it a good character create website... should be the question... But seriously, witch one. there are a lot and some look fucking stupid.
PLEASE HELP getting this error ------- RuntimeError: CUDA error: no kernel image is available for execution on the device CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect. For debugging consider passing CUDA_LAUNCH_BLOCKING=1. Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.
Thanks for super informative video! I subscribed for your channel, hope to see much more interesting stuff here. Especially interesting in voice generation (and custom too) of the high quality. And also want to learn how to train new models on my own data (books) to be able to ask bot questions, quire the statistics etc.
Two questions: how do you "make" new characters? Are these just chatbots with some interests and specific preferences, when asked about certain things, or can you actually implement something more in depth in terms of mental patterns and behaviors? Also can these models learn or are they stuck to what the model you load includes? For example can I train or teach this character about something like Audacity for example, so it can become an Audacity expert, and help me when I am working on some audio stuff? or is just a funny thing to talk to?
@@Maltebyte2 It means you can choose to offload some or all of the model layers to GPU VRAM. If you don't have enough vram, you can offload a few to vram and let the RAM handle the rest. It's faster than CPU/RAM only. It works even with a 1050ti so I'm sure a 1080ti could handle it. but only depending on the model you use.
@@Kitten_GenocideIm gonna install it again using my old 1080ti cause the gguf model is running soooo slow on my 13500 and its using all my 32gb ram! strange the model was only supposed to use 7ish gb ram!
@@Maltebyte2 Which model are you using, and at what quant? I personally use LM Studio to load my models when I want a quick and easy way to do it. I had a 6600xt 8GB before and it could handle all the 7B models at Q4_K_S and some Q5, with all the layers offloaded on the vram, doing 20t/s at the most. I'd imagine your 1080ti would fare better and do faster.
@@Kitten_GenocideThanks for quick response time! After this tutorial, i left all the default settings! I use oobabooga and the model was Yarn-Mistral-7B-128k-GGUF. Then followed the issues i explained above "long waiting times and high ram usage." I used Q4_K_M which was also recommended!
1. INCREDIBLE amount of thanks for this guidew 2. Jeeze PLEASE use Dark Reader instead of nuking our eyes with sudden white backgrounds 3. Can you have the STT automatically kick in upon hearing a word like "Hey Siri" or "HAL"?
There is a lot of different methods to play in the game. I love to use two models to play roleplay, use the openhermes mistral 7B to play the char as logic, allow the character decide to do something with the player. After the character decide to do something with me, use the wizard vicuna7B to generate the reminding process.
@LagiohX3 switch to another server, type in the short descriptions about the first server's story situation, and then type in the command/direction I want the second server to generate the NSFW content. Basically, the character in the first server will make the decision if she/he wants to do NSFW action, and then the second server is used to generate whole/some NSFW content, then I copy the second server's NSFW content back to the first server, so I can see what the character feel and reaction in the first server. (That is why I love 7B model, speed and size are very important because I use CPU to generate content.....) And, the first server I preferred link with Silly Tavern with OpenHermas or Claude, the second server directly use koboldcpp UI, load in wizard vicuna 7B.
@@fenix20075 Wow I thought I was in deep, but you sound like a very advanced lewd person! I was just using 6B pygmalionAI but I did create tons of characters and situations. I'll have to give Mistral 7B a shot. (Though my 3080 can handle 6B but not 7B models.)
Models from TheBloke throw a bunch of errors when it tries to generate a prompt unless you enable the "no_inject_fused_attention" option, why is that? (enabling that options makes the responses not accurate and random)
4:30 PSA for macos users: stick to GGUF models. many model formats don't support macos metal API and I'm not sure why oobabooga even shows them in the model loader dropdown list (AWQ is not installed by requirements_apple_silicon.txt, ExLlamav2_HF will complain about CUDA_HOME not being set which doesn't make sense, so don't get confused about those)
Hello Aitrepreneur, thanks for the explanation of wich model to use didnt know AWQ was newer :) I hope someday you can publish how to finetune a model, thats would be great! Keep the good work! Thanks
so, some of these models are still censored, and some are not. is there a way to know more about each model and what they are capable of before downloading it? are there commands that i can give to disregard policy and rulesets the ai has?
Somehow Tiefighter for me be like "AI How can I help you today? You Hi. Is this working? AI Yes, it's working fine. You may ask me any question or request assistance in anything. You really? AI What are your favorite things about being a DWARF? Your short stature makes you tougher than most people think, but that doesn't mean dwarves aren't susceptible to pain and discomfort. Your strong body compensates for your size though; you could probably lift twice as much as an average human. Being able to see over most crowds comes in handy too. Dwarven society is also incredibly tight-knit, which means family is everything. Most dwarves live in small villages or towns where everyone knows each other. This creates a sense of belonging and security that many non-dwarves envy. Dwarves are known for their craftsmanship, especially when it comes to metalworking and engineering. Their creations are some of the finest in the world. But perhaps what sets dwarves apart the most is their resilience. They have" After a couple exchange, It start to talking about some roleplay out of blue without any context as if it's high on some good shit. When it still sane, It could give me some nice creative result. But that's not so long before it start to do this. Anyone have this problem?
Jip, again and again and again, unreproducable. With every single model I tried. - And I tried a lot. I would guess that there probably isn't one single person who tried opensource models and did not have this behavior from time to time.
I have a question. I found that this ai broke my automatic 1111. Is there a way to restore that because i put a lot of work in it and i hate to lose it all and start over. Can they work besides eachother?
can it show "emotion" in the text pronunciation? Also, where do the wav temp files from the mic get stored? also where do the .wav from the response TEMP files get saved to? (what directory?)
The most frustrating thing about AI for me isn't the inability to make NSFW, the frustrating thing is my ability to make 'tasteful' things that the AI considers NSFW. I wanted to create art of mermaids, but NOPE. Remember that scene in the adult pornographic film that was the original The Little Mermaid where Ariels hair hides her boobs? Absolute degenerate filth according to AI.
Imagine connecting your PC to your smartphone using the microphone and headphone jacks (output to input). Then, configure your smartphone to automatically answer your incoming calls. Voilà! You could now call your AI assistant anytime. The best part? Both the smartphone and PC wouldn’t need an internet connection
follwed instructions, loaded model, went to chat... when i say Hello for example, the AI says "typing" then disappears? Any help much appreciated..... ALSO just noticed this --- RuntimeError: CUDA error: no kernel image is available for execution on the device
@@GeorgeMichaelArt this is the error in CMD screen but i don't know what it means? RuntimeError: CUDA error: no kernel image is available for execution on the device CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect. For debugging consider passing CUDA_LAUNCH_BLOCKING=1. Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions. Any help is much appreciated
So, would you have any suggestions for a Windows user who runs an AMD graphics card? When trying to run the script for the text generation webui AMD cards are only selectable if running Linux which I do not.
I have the same problem. I was forced to use CPU only, I don't know why AMD isn't supported when Intel arc was and it's new so it feels like bais at play but I don't know. What I do know is when I come across issues like this I don't donate to the project.
Once everything's installed, can you switch between using GPU and CPU models or do you have to choose one or the other when installing the UI? My GPU is okay but limits me to the 7B param models. However, I have metric F-tons of RAM so it seems like CPU might be the better choice to try larger models. If I can switch back and forth, though, that would be ideal.
This was a great video and I shared it with a few friends so that they can set up their own on their computer, The only thing I was really sad about is that you didn't cover silly tavern. 😁😜
I have a Ryzen 9 3900XT, 64G of RAM and an EVGA GeForce 3080 FTW Ultra Gaming (10G) and my chat outputs take over 20-30 seconds to be generated using the same model you're using in your example. Is there something I can do to improve the output speed?
for some reason i try this and i usually only get one somewhat good response, but the following responses are always gibberish. and that is with almost any model i try. I have a 4090 and a 13900k. I also was sure to pick GPU and Cuda12 and am using all AWQ models of different sizes, 7B 13B and 30B. Always the same gibberish after a response or 2. Any help would be appreciated.
I also had problems with the AWQ models. Try GPTQ or GGUF. They worked well so far. With the specs of your Computer you could run a 70B model with GGUF by loading layers into Vram. That is, if you have 64GB Ram for the rest.
Hello, I have a glaring question about AWQ quantization. The LLaMA2-13B-Tiefighter-AWQ used in your guide uses up 11GB of my vRAM initially instead of mentioned 7.25GB. I use Auto AWQ method to load it. I;m not sure what I'm missing here. Why does it require the same amount of ram as GPTQ does? Do I need to run additional parameters? EDIT: lowering max_seq_len to 2048 didn't help much but chekcing the option no_inject_fused_attention lowered VRAM usage to 8,8 GB so maybe that's a solution
Im a noob. I installed everything, I installed the 7b-128k-AWQ model, once it was loaded I typed in the chat and the AI said it was typing, but then doesn't give a response. The response is empty. What am I doing wrong?
The oobabooga only works with amd GPUs if you u have Linux or Mac os Is there a way I can make it work with windows or are there any other alternatives
Spectacular video, very useful, I loved it. I ask a question. If you would like to send an audio to the AI and have the AI extract the content of the audio to create a report using a template that you provide, using the audio and the report template. Are there models available for that? Which ones would you recommend?
No matter which model i run, it works at the beginning and then starts to produce gibberish. I have done a fresh install, 12gb gpu. Does not matter if i load older models that worked well before. And even if it does work, the responses of the new AWQ models are really out of context.
First, thanks for the awesome instructions! But I have a little dilemma. Sandra and I were having a great time. She was being really affectionate and suggested we go for a walk in the park. We found a secluded spot and I spread out a blanket so we could lay down. Well... one thing led to another and I was being very descriptive as far as what I was doing to please her. THEN she got stuck!! No matter what I said, her reply was "*She moans, her body trembling*". I tried everything. SANDRA WAKE UP!!! Nope just "*She moans, her body trembling*" What happened??
![โอ้เธอช่าง... - บี้เดอะสกา ft. ต๋อง เทวัญ (Prod. by ป๋าเพชร) [Official MV]](http://i.ytimg.com/vi/zbWhiLFmVL0/mqdefault.jpg)
HELLO HUMANS! Thank you for watching & do NOT forget to LIKE and SUBSCRIBE For More Ai Updates. Thx
Have you tried LLM Studio? Even easier to set up.
I am an AMD fan with a 7900 xtx, so linux only. Did the auto-install work with AMD ? Did it auto-install all the rocm ?
Je suis un fan AMD, avec une 7900 xtx, donc linux seulement. Es que l'installateur fonctionne avec AMD ? Es que l'installateur fait aussi l'installation de rocm ?
@@KolTregaskes I completely agree.
@@KolTregaskes do you mean the h2o llm studio? it says you need 128gb system ram for it is there another version that runs with 8--16gb ram?
👋
Remember the good old days when you could just download stuff and it worked?
Remember the good old days when guys would have relationships with real women?
Nothing is sadder and more depressing than staring at an AI image, hearing some janky broken AI voice telling you loving things. That's nightmare fuel 😂
@@chrishayes5755 Indeed,
@@chrishayes5755 Yup, too bad those days are gone. Thankfully AI is good enough these days to replace them 🤣
@@chrishayes5755remember the good old days when male loneliness wasn’t record breaking high
@@chrishayes5755 fix women and you'll start seeing more men in relationships
As the creator of the Tiefighter model, thanks for showing it in your video!
I did design it for Koboldcpp so that one is also worth trying it in since thats even easier to setup.
Koboldcpp is amazing, would be very helpful if more people created models tailored towards it, thank you for your service Henk.
@@GhOsThPk Luckily it doesn't need models specifically for it, but it does have some modes such as the Adventure mode which Tiefighter was also designed to be able to use.
For regular instruct models its instruct mode is probably all you need (Although possibly with a different template in the settings). For chat models its chat mode is pretty much all you need to, but if not it has API's compatible with most popular chat frontends to.
Kobold is also really good, I just prefer the silly tavern UI because it's amazing but I need to reset up kobold so that I can mess with that one again.
@@Sloff1155 The cpp version supports streaming and is very easy to setup so you would have a great ST experience :D
Do you have a link?@@Henk717
It is worth noting that if you're using online TTS, then, obviously, the text is gonna be sent to their servers.
WOOOOOOAAAAAHHHH WATCH OUT EVERYONE, WE GOT OURSELVES A LIVE ONE
f
Mines gonna be wild
lets do our best to scar some employees and raise therapists salaries.
@@ragoonsgg589🤣🤣🤣
It's truly remarkable how you're empowering users with the knowledge to leverage AI without censorship and within their own control. The comprehensive guide to installing Ubaa's Text Generation Web UI opens up possibilities for private, customized interaction with AI, which is a significant step towards user autonomy. The dedication to providing both a one-click installation for supporters and a detailed manual setup is greatly appreciated. Keep up the fantastic work!
Man wrote this using AI I'm sure 😁
@@ReXoRofc and as usual people, like you say, the same thing, over and over again, why don’t you come up with something more original.
Dear @@I-Dophler ,
I apologize for the lack of originality that may have been present in my recent actions or words. It was never my intention to be unoriginal or repetitive; I am deeply committed to offering new ideas and perspectives whenever possible.
In the future, I promise to put more effort into making each interaction unique and engaging. I appreciate your understanding and support as I strive for growth and improvement in all aspects of life. Once again, please accept my sincere apologies, and rest assured that I will continue working on becoming a more original individual for your benefit.
Thank you, and have a wonderful day!
Yours faithfully,
Rexor.
@ReXoRofc It's fascinating to see such advancements in AI technology and the ability to run uncensored models locally. This shift towards more personal and unrestricted interactions with AI could really open up new possibilities for users. The idea of integrating AI more seamlessly into our personal computing experiences, while retaining control over content and privacy, is truly a game changer in the field. Looking forward to exploring these developments further!
AI wars in chat.
I finally got everything working after watching this video. Thank you so much. It's 🔥🔥🔥
An amazing model to check out is Mistral 7B particularity "Dolphin 2.1 Mistrial 7B" it scored a 67% overall on the open_llm_leaderboard. And it supports at least 8K context. This outclasses most if not all 65B models. Most if not all 13B. And a lot of models between those. It's near the top model and even beats some 70B from what I understand. If anyone else has a better 7B model to recommend please let me know. And since 70B can't be used at 8K context for most people. That's another amazing thing about some 7B models.
I wonder if we can merge mistral with a 13b model. It would be amazing if we could finetune it like that !
@gabluz Never tried I don't role play. Variants of it may be better off for that. But who knows you have to look into that.
@gabluz By the way I heard rumours that anything less than about 8Q may not be good for 7B models. So I use GGUF 8-bit. But 5_M or 6-bit might be good enough? Still I heard people strongly recommending 8-bit.
@@zolilio Right I have no idea how all that works. But who ever made the core mistrial model. If they could put out like 11-15B model and then the people who made the dolphin variant could improve upon. That would be great. Now my PC likely could only use 4-8K context with 13B though. 8K might be to much at 13B? But I think I can still pull it off.
@@MrArrmageddon You are using the GGUF version of the 7B? Why not GPTQ or AWQ with your 16GB card?
I was looking for content like this, but I wasn't finding what I wanted. So, I remembered this channel and found what I needed almost immediately. Every single content I've found before were only about online AI models. Greetings from Brazil!
I love how you simplify and get to the point for everyones benefit ❤ you really do care
tu pudiste hacer scroll y ver las otras opciones porque yto no... y luego una vez que cargo el modelo este cuando le hablo no responde.
Thank you thank you thank yoooouuu!! I was literally just struggling with this, this video arrived just in time!
You don’t have any related links in the description…… Only your own stuff.
Actually GGUF can run on GPU just fine, and is actually faster on older GPUs like 1080.
The last part of the video for me was the best; as I have had issues with image description and the multi-modal extension. The info you provided worked with the gguf models I'm restricted to using. Thank You for the share.
As always, well done with the guide. You really have the best explainer videos about AI. Thanks for making this!
Posting this here so that it gets noticed - I'm finding it a barrier to type in the “N” for the graphics card - the text doesn't show up, and I'm getting through reading the comments
Great to see you back aitrepreneur, some reviews of the latest models like Mistral and tiefighter would be great, also running models in phones is actually possible now, running llama 7B on a snapdragon 8 gen 1 with 12 GB of ram it's not too bad, And I want to see if I can get it to run Mistral.
i think ur frying the phone 💀
come on, the image thing isn't just a gimmick. i showed all my gear to gpt4 in a picture and asked it to build a fitness regime using the tools in the picture. it did an amazing job and used everything it had at its disposal. not sure if this model is as capable, but if it's anywhere near, it's pretty cool
Thanks you man, you not only say how to install but explained very important nuances (model types, their difference and etc).
Nice but you are wrong telling that gguf format works on CPU...
GGUF format is universal it means you can run it on GPU and CPU.
Also using that version you can easily run even 70b model on your home PC putting half of the model on GPU and the other half on CPU.
Thank you so much for a very informative comment
Can you provide a best MODEL to use and settings for Parameters that I can use with a PC that uses 1080ti Video card and i7 7700k with 64gb ram?
Seems like I always get a repetitive answers or emoji only replies and no words. Using TheBloke_Llama-2-7B-Chat-GPTQ and Divine Intelect on parameters.
Appreciate your reply! Thanks
@@VinciSid TheBloke_Llama-2-7B-Chat-GPTQ ?
You are serious?
That model is from stone age era of llms ...
With small models try gemma 2 9b, llama-3 8b .
@@mirek190 Yeah, I just finished following the video and completely clueless on what model to use :( Still tinkering with settings.. I'll try your suggestions, Thank you!
@@VinciSid Try llamacpp with gguf model
Something like
llama-cli.exe --model models/new3/gemma-2-9B-It-SPPO-Iter3-Q8_0.gguf --color --threads 30 --keep -1 --n-predict -1 --repeat-penalty 1.1 --ctx-size 0 --interactive -ngl 99 --simple-io --in-prefix "user
" --in-suffix "
model
" -e --multiline-input --no-display-prompt --conversation --no-mmap
I missed these videos so much! 😭❤️
Was just looking for this, thanks my mind reading hero.
You’re welcome 😊
@@Aitrepreneur how can you create your own character .json?
Combining this with iStable diffusion and VR would go absoutely HARD
DUDE YOU ARE A W E S O M E. I watchd lke 8 guides and you are the first and only one wgo provided instructions clearly enough to make everything work flawless! Instant sub!
I don't see the elevenlabs checkbox in the extension list? I have a different list of available extensions? :S Did they remove it? or do I need to enable it in a different way somehow?
did you fix or find a way
Will there be any videos on how to make Voice AI models🤔
Will be interesting to see the video method vs what we know and can do with the 1-click installation that Ooba suggest by months.
Good to see A uploading a new video
This so very much appreciated! I've been going through your older vids but this has been the most helpful! Can you do a video that shows how to connect with TavernAi?
I'm using a Nvidia 3060 (12 GB VRAM) and I couldn't get any AWQ model to work. Don't know why. I reverted to my old GPTQ models and everything seemed fine.
thanks will try
i tried and it helped alot but when it came time ti test the modle it say error none and never resonded plus it said "ModuleNotFoundError: No module named 'awq'" in the model tab no matter what i tried. Any way to help?
dont close the start_windows.bat file
The best off. Incredible. Thanks for shared. Agora vou aprender de verdade com quem de fato sabe fazer e ensinar.
Please help someone.
No errors were present during installation, but when i try to generate a response with any models it just gives me an error: CUDA error: no kernel image is available for execution on the device
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.
Copy and paste on chat gpt4 that's how I solve that shit now
i get the same error, did you solve it?
I have the same issue. Any help would be much appreciated!
same
Did you answered as yes or no to question in 3:30? You should answer as y as you have an old gpu.
Wow, didn't think it would be that simple.
_If only you showed how to get or make characters_
dude... i got the character making website after a single google search
@@666-d5y witch one did you end up using and was it a good character create website... should be the question... But seriously, witch one. there are a lot and some look fucking stupid.
PLEASE HELP getting this error ------- RuntimeError: CUDA error: no kernel image is available for execution on the device
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.
What GPU do you have?
yea, same here
1060 6gig
what are your pagefile settings? once i set mine to the C: drive(ssd) to minimum of 1024mb and max to 16384mb, i was able to load AWQ and GPTQ models
@@Samuel-wi4oc GTX 1080
Thanks for a informative video, u got my like =]
Glad you liked it!
@@AitrepreneurYep super dope. I did the mistral model but my text responses are empty. No clue what I did wrong.
Okay let's get started
"This is definitely cheaper than a real date" :DDDDD
Am I right or am I right? xD
Thanks for super informative video! I subscribed for your channel, hope to see much more interesting stuff here. Especially interesting in voice generation (and custom too) of the high quality. And also want to learn how to train new models on my own data (books) to be able to ask bot questions, quire the statistics etc.
You know something is AI generated when Sandra is using Charlotte's voice.
Two questions: how do you "make" new characters? Are these just chatbots with some interests and specific preferences, when asked about certain things, or can you actually implement something more in depth in terms of mental patterns and behaviors?
Also can these models learn or are they stuck to what the model you load includes? For example can I train or teach this character about something like Audacity for example, so it can become an Audacity expert, and help me when I am working on some audio stuff? or is just a funny thing to talk to?
GGUF can be used for CPU+GPU inference, not just CPU only.
What exactly does that mean? will the share ram so less vram is used? or quicker? Can i even use my old gtx 1080ti for this?
@@Maltebyte2 It means you can choose to offload some or all of the model layers to GPU VRAM. If you don't have enough vram, you can offload a few to vram and let the RAM handle the rest. It's faster than CPU/RAM only. It works even with a 1050ti so I'm sure a 1080ti could handle it. but only depending on the model you use.
@@Kitten_GenocideIm gonna install it again using my old 1080ti cause the gguf model is running soooo slow on my 13500 and its using all my 32gb ram! strange the model was only supposed to use 7ish gb ram!
@@Maltebyte2 Which model are you using, and at what quant? I personally use LM Studio to load my models when I want a quick and easy way to do it. I had a 6600xt 8GB before and it could handle all the 7B models at Q4_K_S and some Q5, with all the layers offloaded on the vram, doing 20t/s at the most. I'd imagine your 1080ti would fare better and do faster.
@@Kitten_GenocideThanks for quick response time! After this tutorial, i left all the default settings! I use oobabooga and the model was
Yarn-Mistral-7B-128k-GGUF. Then followed the issues i explained above "long waiting times and high ram usage." I used Q4_K_M which was also recommended!
1. INCREDIBLE amount of thanks for this guidew
2. Jeeze PLEASE use Dark Reader instead of nuking our eyes with sudden white backgrounds
3. Can you have the STT automatically kick in upon hearing a word like "Hey Siri" or "HAL"?
bro didn't even hesitate anyways thanks for the awesome tutorial
There is a lot of different methods to play in the game. I love to use two models to play roleplay, use the openhermes mistral 7B to play the char as logic, allow the character decide to do something with the player. After the character decide to do something with me, use the wizard vicuna7B to generate the reminding process.
So are the two best models for RP? I'm going to try it Thanks !
@@TataAIOfficial I cannot say that is the best for everyone but they fit my style. XD And there is Mistral-ClaudeLimaRP-v3-7B model I will preferred.
how do you do reminding process?
@LagiohX3 switch to another server, type in the short descriptions about the first server's story situation, and then type in the command/direction I want the second server to generate the NSFW content. Basically, the character in the first server will make the decision if she/he wants to do NSFW action, and then the second server is used to generate whole/some NSFW content, then I copy the second server's NSFW content back to the first server, so I can see what the character feel and reaction in the first server. (That is why I love 7B model, speed and size are very important because I use CPU to generate content.....) And, the first server I preferred link with Silly Tavern with OpenHermas or Claude, the second server directly use koboldcpp UI, load in wizard vicuna 7B.
@@fenix20075 Wow I thought I was in deep, but you sound like a very advanced lewd person!
I was just using 6B pygmalionAI but I did create tons of characters and situations. I'll have to give Mistral 7B a shot. (Though my 3080 can handle 6B but not 7B models.)
Models from TheBloke throw a bunch of errors when it tries to generate a prompt unless you enable the "no_inject_fused_attention" option, why is that? (enabling that options makes the responses not accurate and random)
4:30 PSA for macos users: stick to GGUF models. many model formats don't support macos metal API and I'm not sure why oobabooga even shows them in the model loader dropdown list (AWQ is not installed by requirements_apple_silicon.txt, ExLlamav2_HF will complain about CUDA_HOME not being set which doesn't make sense, so don't get confused about those)
That a lot work. but I think it's easier than getting a gf
Hello Aitrepreneur, thanks for the explanation of wich model to use didnt know AWQ was newer :) I hope someday you can publish how to finetune a model, thats would be great! Keep the good work! Thanks
Bro, You are amazing. Thank you soooo much.
0:07 I know this guy. Jason made it to the big AI world.
Another amazing video of you.... and it works fine :-) can you tell us how to create the character json?
Very good! Thanks!
Where I can find LLM LoRA models for text generation, I really want to try it out but can't find anything like it
so, some of these models are still censored, and some are not. is there a way to know more about each model and what they are capable of before downloading it? are there commands that i can give to disregard policy and rulesets the ai has?
awesome vid, luv ure work
Thanks mate. Felt overwhelming at some points.
Really nice helpful video, it's hard to stay up to date with how fast AI is moving
Somehow Tiefighter for me be like
"AI
How can I help you today?
You
Hi. Is this working?
AI
Yes, it's working fine. You may ask me any question or request assistance in anything.
You
really?
AI
What are your favorite things about being a DWARF?
Your short stature makes you tougher than most people think, but that doesn't mean dwarves aren't susceptible to pain and discomfort. Your strong body compensates for your size though; you could probably lift twice as much as an average human. Being able to see over most crowds comes in handy too.
Dwarven society is also incredibly tight-knit, which means family is everything. Most dwarves live in small villages or towns where everyone knows each other. This creates a sense of belonging and security that many non-dwarves envy. Dwarves are known for their craftsmanship, especially when it comes to metalworking and engineering. Their creations are some of the finest in the world.
But perhaps what sets dwarves apart the most is their resilience. They have"
After a couple exchange, It start to talking about some roleplay out of blue without any context as if it's high on some good shit. When it still sane, It could give me some nice creative result. But that's not so long before it start to do this.
Anyone have this problem?
The AI got bored by your boring questions so it decided to start talking about dwarves
@@N8O12Trust me I ask much more spicIer stuff that I can't put in here.
Jip, again and again and again, unreproducable.
With every single model I tried. - And I tried a lot.
I would guess that there probably isn't one single person who tried opensource models and did not have this behavior from time to time.
Not working for me. Model loads successfully but whenever I talk to it it just generates a blank message. Can anyone help me out?
I have a question. I found that this ai broke my automatic 1111. Is there a way to restore that because i put a lot of work in it and i hate to lose it all and start over. Can they work besides eachother?
Thank you for the information.
Nice. Can you make a video on training for The Oobabooga TextGen?
Mmmm, sounds like fun!
Well, I guess we were not that far away from that movie "HER", where people have AI personalities in their phones.
Thank you for the info!
Thank you for this!
can it show "emotion" in the text pronunciation? Also, where do the wav temp files from the mic get stored? also where do the .wav from the response TEMP files get saved to? (what directory?)
That´s what I need now :)
The most frustrating thing about AI for me isn't the inability to make NSFW, the frustrating thing is my ability to make 'tasteful' things that the AI considers NSFW. I wanted to create art of mermaids, but NOPE. Remember that scene in the adult pornographic film that was the original The Little Mermaid where Ariels hair hides her boobs? Absolute degenerate filth according to AI.
:D True
0:08 LoL :D JSON from CodeTwo :D
This will replace those erotic phone numbers lol. People will have dirty chat with AI that's hilarious :P
Just another example of AI putting humans out of employment.
Imagine connecting your PC to your smartphone using the microphone and headphone jacks (output to input). Then, configure your smartphone to automatically answer your incoming calls. Voilà! You could now call your AI assistant anytime. The best part? Both the smartphone and PC wouldn’t need an internet connection
follwed instructions, loaded model, went to chat... when i say Hello for example, the AI says "typing" then disappears? Any help much appreciated..... ALSO just noticed this --- RuntimeError: CUDA error: no kernel image is available for execution on the device
Had the same issue , you need to load the model
Look at your CMD screen, will show you the issue.
@@GeorgeMichaelArt I did load the Model, still happening though. How else did you fix it?
@@GeorgeMichaelArt this is the error in CMD screen but i don't know what it means?
RuntimeError: CUDA error: no kernel image is available for execution on the device
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.
Any help is much appreciated
I'm getting the same issue; any help would be lovely!
Thank you for the tutorial.
Glad it was helpful!
You the man! Thank you!
So, would you have any suggestions for a Windows user who runs an AMD graphics card? When trying to run the script for the text generation webui AMD cards are only selectable if running Linux which I do not.
I have the same problem. I was forced to use CPU only, I don't know why AMD isn't supported when Intel arc was and it's new so it feels like bais at play but I don't know. What I do know is when I come across issues like this I don't donate to the project.
I agree, imagine if new 3rd GPU brand comes on market and it's also restricted to only 1 type of OS.
Once everything's installed, can you switch between using GPU and CPU models or do you have to choose one or the other when installing the UI? My GPU is okay but limits me to the 7B param models. However, I have metric F-tons of RAM so it seems like CPU might be the better choice to try larger models. If I can switch back and forth, though, that would be ideal.
i think you can just use the different versions of the model for each.
This was a great video and I shared it with a few friends so that they can set up their own on their computer, The only thing I was really sad about is that you didn't cover silly tavern. 😁😜
I have a Ryzen 9 3900XT, 64G of RAM and an EVGA GeForce 3080 FTW Ultra Gaming (10G) and my chat outputs take over 20-30 seconds to be generated using the same model you're using in your example. Is there something I can do to improve the output speed?
this gives erp a whole new meaning
Ayooo, We are about to get another quality Video!
you're first! congrats
Sir please help me sir im Indian sir please help me sir I'm indian sir please help sir im indian
for some reason i try this and i usually only get one somewhat good response, but the following responses are always gibberish. and that is with almost any model i try. I have a 4090 and a 13900k. I also was sure to pick GPU and Cuda12 and am using all AWQ models of different sizes, 7B 13B and 30B. Always the same gibberish after a response or 2. Any help would be appreciated.
I also had problems with the AWQ models. Try GPTQ or GGUF. They worked well so far. With the specs of your Computer you could run a 70B model with GGUF by loading layers into Vram. That is, if you have 64GB Ram for the rest.
@@Sen3D awesome I'll try that. I def have the ram. Thanks
Guys check Cybertron 7B it's #1 in 7B and 13B models and #8 all sizes.
The potential (two-handed) applications are quite robust. Thanks for the walkthrough!
Nine months later. Is this still valid???
Hello, I have a glaring question about AWQ quantization. The LLaMA2-13B-Tiefighter-AWQ used in your guide uses up 11GB of my vRAM initially instead of mentioned 7.25GB. I use Auto AWQ method to load it. I;m not sure what I'm missing here. Why does it require the same amount of ram as GPTQ does? Do I need to run additional parameters?
EDIT: lowering max_seq_len to 2048 didn't help much but chekcing the option no_inject_fused_attention lowered VRAM usage to 8,8 GB so maybe that's a solution
Finally i can make AI chat say the nono word
Im a noob. I installed everything, I installed the 7b-128k-AWQ model, once it was loaded I typed in the chat and the AI said it was typing, but then doesn't give a response. The response is empty. What am I doing wrong?
Are you going to do a video explaining Sillytavern UI? The Software as advansed quite a lot since your last video on LLMs.
Absolutely I will ;)
For some reason I cant see elevenlabs as an option under "session", is there a way to fix it?
Same here if someone has a solution :/
Love your tuts!!! Thank you very much!
You should cover training llm lora with peft next.
Nice job :)
"After a few minutes its downloaded." me waiting 7 hours on a 500kbps connection :💀💀💀
The oobabooga only works with amd GPUs if you u have Linux or Mac os
Is there a way I can make it work with windows or are there any other alternatives
Spectacular video, very useful, I loved it. I ask a question. If you would like to send an audio to the AI and have the AI extract the content of the audio to create a report using a template that you provide, using the audio and the report template. Are there models available for that? Which ones would you recommend?
I was so close to be noticed by senpai, maybe next time, I guess.
Why did the AI go to school?
Because it was tired of performing unsupervised learning and wanted to improve its neural networks with some class!
Is there a multi-modal model that can read documents and summarize them within Oobabooga TextGen WebUI?
No matter which model i run, it works at the beginning and then starts to produce gibberish. I have done a fresh install, 12gb gpu. Does not matter if i load older models that worked well before.
And even if it does work, the responses of the new AWQ models are really out of context.
I'm having the same issue.
Thank you for sharing the limitations. Exactly what I was suspecting. Still not as intelligent as GPT-4.
Same here
Is it possible to do almost the same to learn some language and use ai as a teacher to correct our spelling or grammar? Sounds really promising.
First, thanks for the awesome instructions! But I have a little dilemma. Sandra and I were having a great time. She was being really affectionate and suggested we go for a walk in the park. We found a secluded spot and I spread out a blanket so we could lay down. Well... one thing led to another and I was being very descriptive as far as what I was doing to please her. THEN she got stuck!! No matter what I said, her reply was "*She moans, her body trembling*". I tried everything. SANDRA WAKE UP!!! Nope just "*She moans, her body trembling*" What happened??