วีดีโอ

its not nsfw?

Download iRAG??!

can this model work for Arabic or it works for English only

Great informative video. Could you kindly suggest, if I should choose an AMD 9800X3D or AMD 9900X if I want to run LLMs locally?

Great video

Nobody knows how much precision they need

I run qwen2.5 72B at BF16 , which is what you can dl off HF, with intel cpu intel gpu arc a770 16GB and 64GB DDR4 RAM and 5TB 7400MB/sec NVME ssd and it runs good! Ok, the initial load takes a few mintues but inference is fine. I only swap to SSD disk about 8GB but if you have much less than 64GB RAM then you bettter have a fast SSD cuz it will be hammered alot. For my setup intel IPEX-LLM is crucial and it's what makes it all possible on the intel cpu intel gpu. YEs, you need to know how to code, basic python at least. I also first got running a qwen2.5 coder 7B so if you have less resources than my windows pc, so should stick to the 7B model. On the bright side, register with HF and you can call the qwen2.5 72B via their API where they host the model. You get 1000.....

These products are all garbage with several flaws; 1) impossible to accomplish for the average person running an average laptop. Unless you have a $2K cuda-enabled GPU and have significant knowledge of how to install and run custom add-ons, you won't get very far. 2) The quality is not imperfect, it's downright laughable. 3) You will have difficulty cloning your own custom voices because nobody really knows how to do this. In addition, many of these github based products are listed in other languages, making it even more impossible. Sorry, with all the advancement of AI, nobody has been able to make a useable, authentic voice changer that runs in real time with adjustable delay times.

❤thank you too much, subscribe and like

Didn't expect this topic from such ladies😂

Hello, thanks for that great video. how about that system : i5 13600f, 32 gb of ddr5 ram and rtx 3090. For the current published llms?

Free for trial?

Nice video!, does it work?, I've been running the code from the CMD for almost 1-2 days, this line has been repeated since then, is it normal? (RTX2060 14GBVRAM & 16GB RAM): "new version of transfomer, no need to use BetterTransformer, try setting attn impl to sdpa... attn imp: <class 'transformers.models.llama.modeling_llama.LlamaSdpaAttention'> running layers(cuda:0): 100%|██████████████████████████████████████████████████████████| 83/83 [07:22<00:00, 5.34s/it] total time for load_safe_tensor: 344.40438961982727 total time for compression_time: 86.546875 total time for create_layer_from_safe_tensor: 1.7751367092132568 total infer process time(including all above plus gpu compute): 105.0000 total infer wall time(including all above plus gpu compute): 444.1316 The class `optimum.bettertransformers.transformation.BetterTransformer` is deprecated and will be removed in a future release."

Is there a way to fine-tune this model via cloud

thank you, but t is very slow :-) why?

the sad thing is that they have cascading failure. all of them. windows 10, 11 and ubuntu.

you are loading a 4 bit bitsandbytes quantized LLM its already compromised on precision and then using AirLLM blockwise quantization the accuracy of the model will take a major hit

In infinity, even a calculator can run gpt-5

Note it wont run on an AMD RX 6700 xt, due to the absence of bitsandbytes module implementation for the AMD HIP on rocm. Bitsandbytes modules is key to the quantization of models. If u use any consumer AMD gpu < 7700 get an nvidia gpu or migrate to a higher module.

When you say 'full Adam training', is it full parameter fine-tuning or training an LLM from scratch?

According to Meta Blog, "We performed training runs on two custom-built 24K GPU clusters." How come only 13 H100 GPUs are required for a 70B model for full training on your webpage? Do you mean "at least"?

Why version 3.1 while 3.2 is available for months?

Sure, why put any links in the description, makes no sense. People should search, make so much more fun 🙄

Please share the script mentioned in the video

if this is true then it would be ollama and openwebui loooong ago. and it would mean ipex would be able to smash through it but alas i do not think it is as good as its said.

Considering this is Python we're talking about here, how long does it take to run.. 1000 years?

You did not run it on 8gb vram

Better wait for a BitNET version add: or TriLM

That's probably a software that processes layer after layer and yes in theory you can run a very large model that way. But nobody said you can expect a result before the next ice age....

And how long does the inference take? 40h/token? Just because you can run it does not mean its useful

...is this real? I've never heard of this o_O

Llama 3.1 is vastly superior for coding

i don't believe it. i'll wait till someone else beta-tests it. this sounds like a scam to me. well looking at the github, it apparently works, but at speeds like 40 seconds per token. not very useful to me, but cool nonetheless

Crazyyyyyyy😮

Well, If it works, I wonder if this should be incorporated into LM Studio and similar projects...

script ???????

WTF!!!!! 🫨

can you tell me the minimum CPU requirement for gemma2_2b model

Hey, I'm thinking of getting a new laptop with an RTX 4060 and 8GB of VRAM. But I'm also considering using Google Colab or Jupyter notebooks to learn about and play around with Large Language Models (LLMs) and their applications. The thing is, I need a new laptop anyway because my current one is ancient and barely hanging on. So, I'm wondering if it makes more sense to just buy my own machine or if I should go the Colab/Jupyter route. What do you think?

If you are running 2b model, you don’t need gps to begin with😂

Thank you for your video. Tell me please, what models can you advise for i9 9900, 32gb ram, rtx3090.

You never showed us how to clone any voice. He just showed us how to use presets why don’t you show us how to do what you said you were gonna do.

Will a 3060 ti with the i7 raptor k with integrated gpu and 64 gigs of ram on a asus pro art work?

is it possible to use AMD EPYC 9965 (192 cores, 576 GB/s memory bandwidth) for inference and training? maybe it is not as fast as GPUs but I can use much cheaper RAM modules and only one processor and it will be cheap enough

neat site, i like this kind of information. i have been using Exl2 models 8bpw Llama 3.1 70B and Llama 3.1 Nemotron. I can load them on 2 rtx a 6000's. i would love to see this information for different quant types. the hardest part for me is figuring out how much vram i need for the models i try and its more of a brute force just try it and see.

🙏❤️

Llama vision is not very good for transcribing text - it makes a lot of things up. I show that in the latest video on my channel. Claude is currently miles ahead of anything else. GPT-4o not far behind. Llama a bit of a joke.

What if you want to just run an LLM specifically for better speech recognition. It should be very small, a subset. Could that be done on integrated to keep the GPU free?

Have you tried LLAMA 3.2 vs GEMMA 2 ?

my left ear really liked this