Democratize AI: turn $95 AMD APU into a 16GB VRAM GPU AI workhorse. Demo of AI app. stable diffusion

แชร์
ฝัง
  • เผยแพร่เมื่อ 22 ส.ค. 2024

ความคิดเห็น • 125

  • @wolpumba4099
    @wolpumba4099 11 หลายเดือนก่อน +8

    *Summary*
    - 0:00 Introduction
    - 0:03 Artificial intelligence is transforming industries
    - 0:07 Access to AI technology has been limited by high costs and technical expertise
    - 0:16 GPU VRAM is an important factor for AI applications
    - 0:23 Consumer GPU price has been skyrocketing
    - 0:33 In 2022, a GPU with only 12 gigabytes of VRAM costs over $1500
    - 0:45 The situation hasn't improved much in 2023
    - 0:51 Demand for GPUs has been skyrocketing
    - 0:55 Nvidia has reached a 1 trillion dollar market cap and becomes the world's largest chip company
    - 1:01 Inflation and interest rates have been skyrocketing
    - 1:12 To democratize AI, we need both low cost consumer hardware and smart designed software
    - 1:30 Introducing AMD APU, a budget-friendly hardware solution
    - 1:45 AMD APU is a series of 64-bit macro processors combining CPU and GPU
    - 2:05 The least expensive AMD APU is 4600 G priced at 95 US dollars
    - 2:17 A slightly more expensive APU is 5600g which costs 127 US Dollars
    - 2:30 The GPU can be turned into a 16 gigabytes VRAM GPU
    - 2:53 System costs can be as low as 400 US dollars for all brand new components
    - 3:27 The power consumption is around 66 Watts for idle system and around 96 Watts at full usage
    - 3:51 The operation cost is about 2.4 US dollars for a month or 30 US dollars per year
    - 4:29 The system can run mainstream machine learning frameworks such as PyTorch and TensorFlow
    - 4:57 The system can run state-of-the-art AI applications
    - 6:00 The system has future expandability for more powerful discrete GPU
    - 6:23 A demo of running stable diffusion using command line
    - 7:55 A demo of automatic web UI running stable diffusion
    Positive Points:
    1. Affordable: AMD APU is a cost-effective solution. The total system cost can be as low as 400 US dollars for all brand-new components.
    2. Energy Efficient: The power consumption is minimal, around 66 Watts for an idle system and around 96 Watts at full usage.
    3. Versatile: The system can run mainstream machine learning frameworks such as PyTorch and TensorFlow, as well as state-of-the-art AI applications.
    4. Expandability: The system allows for future upgradeability to more powerful discrete GPUs without the need to replace any existing components.
    5. High VRAM: The GPU can be turned into a 16 gigabytes VRAM GPU, providing substantial memory for AI applications.
    Negative Points:
    1. Limited Availability: Access to AI technology has been limited by high costs and technical expertise. While the AMD APU is a more affordable option, availability could still be an issue for some.
    2. High Demand: The demand for GPUs has been skyrocketing, which could potentially lead to supply issues.
    3. Inflation and Rising Interest Rates: The rising inflation and interest rates could potentially increase the costs of these components.
    4. Limited Processing Power: While AMD APU is a cost-effective solution, its processing power may not match up to the more expensive, high-end GPUs in the market.
    5. Requires Technical Knowledge: While the system allows for future upgrades, this still requires a certain level of technical knowledge and expertise.

  • @piratediffusion
    @piratediffusion ปีที่แล้ว +8

    That's pretty awesome! Love to see it. We've been testing different hardware as well, I hope to post a video soon about it

    • @klaustrussel
      @klaustrussel ปีที่แล้ว +1

      Subbed, hope to see something soon!!

  • @JD-jdeener
    @JD-jdeener 3 หลายเดือนก่อน +1

    Excellent presentation. I had given up on using my two 4060 Tis until I saw your video on how to allocate VRAM per GPU via command. Keep forging ahead, you'll make a difference for sure.

  • @AmirZaimMohdZaini
    @AmirZaimMohdZaini ปีที่แล้ว +8

    Meanwhile, I'm just simply use DirectML version of Stable Diffusion. It just really works on AMD iGPUs. No hacker-level tricks, just configure the iGPU to the bigger VRAM and set up the Stable Diffusion DirectML version.
    Although it works, but slow, at least little bit faster than running it on the CPU. It needs at least 8GB of VRAM and of course, you will need at least 32GB of system RAM modules installed on your computer.

    • @tech-practice9805
      @tech-practice9805  ปีที่แล้ว +1

      yes, that's correct. DirectML is good for several tasks such as Stable diffusion! However it's limited compared to use Pytorch directly. To run most of Pytorch code, you would need ROCm on Linux.

    • @VioFax
      @VioFax 4 หลายเดือนก่อน

      I bought 2 P40's ($175 each used) and cannot get them to work in my workstation. Shame the workstation has 512 GB of ram. And each of these cards has 24GB vram. But dell workstations purposely doesn't support memory ReBar... so that you have to buy a quadro p6000 ($1200 used) instead to get the same thing. Dell is the worst. Nvidia hates us too.

  • @BenSima
    @BenSima ปีที่แล้ว +8

    actual demo is at 6:22

  • @FSK1138
    @FSK1138 ปีที่แล้ว +3

    Thanks A.I. will leap forward with inovations like this

  • @wargamingrefugee9065
    @wargamingrefugee9065 ปีที่แล้ว +7

    thanks for sharing with us

  • @yannkitson116
    @yannkitson116 ปีที่แล้ว +1

    This is really great! Thanks for sharing...

  • @cyberhard
    @cyberhard ปีที่แล้ว +3

    Good video. Would be better without the loud background music.

  • @onigirimen
    @onigirimen ปีที่แล้ว +3

    i'm looking at your terminal, it's still using torch 1.13 + ROCm 5.2 , i think if you updated with torch 2.0 + ROCm 5.6 it'll give 10-15% performance boost (maybe more, since 10-15% boost is for ROCm 5.6 when upgrade from 5.5)

    • @tech-practice9805
      @tech-practice9805  ปีที่แล้ว

      Thanks, that's great observation! I previously tried Pytorch 2.0, but at the time there may be bugs so it generated black images. It probably got fixed by now.

  • @onigirimen
    @onigirimen ปีที่แล้ว +7

    just wondering, if i have 128GB of RAM, and utilizing 50% to iGPU, so i'll have 64GB of "iGPU VRAM" right? With this i think train SDXL or LLM will be much much cheaper (although it's slower) than nVidia (which need at least A100 to achieve similar VRAM size)

    • @clarissamarsfiels7961
      @clarissamarsfiels7961 ปีที่แล้ว +4

      Good question! I would like to know that too. 128GB of RAM is available for $400+, so it's really "cheap" compared to a $3000+ GPU with 40GB of VRAM. I would upgrade my RAM to 128GB and get a Ryzen 5700G and I have 64GB VRAM? That would be awesome. 5700G is also slightly faster than 4600G shown in the Video. Maybe 10% I think.

    • @klaustrussel
      @klaustrussel ปีที่แล้ว

      Was thinking about the same thing!!@@clarissamarsfiels7961

    • @gertjanvandermeij4265
      @gertjanvandermeij4265 ปีที่แล้ว

      @@clarissamarsfiels7961 Sadly no , For now, some motherboards company's ( like ASRock ) tap out @ 16Gb allocation, to the GPU part of your APU. There are even motherboard brands, that give you, an max. of just 4Gb allocation, to the GPU ! So ..... UNLESS some motherboard company, or an good modder ( bios hacker ) can make an new bios, that allows more than 16Gb, to the GPU, It just wouldn't be possible ! Even an 16Gb allocation is quite rare, I only know one motherboard brand that does this, and that is ASRock !

    • @gertjanvandermeij4265
      @gertjanvandermeij4265 ปีที่แล้ว +4

      Sadly no , For now, some motherboards company's ( like ASRock ) tap out @ 16Gb allocation, to the GPU part of your APU. There are even motherboard brands, that give you, an max. of just 4Gb allocation, to the GPU ! So ..... UNLESS some motherboard company, or an good modder ( bios hacker ) can make an new bios, that allows more than 16Gb, to the GPU, It just wouldn't be possible ! Even an 16Gb allocation is quite rare, I only know one motherboard brand that does this, and that is ASRock !

    • @tech-practice9805
      @tech-practice9805  ปีที่แล้ว +1

      I hope they can increase it after seeing my video :D it can definitely help their sales!

  • @kkrolik2106
    @kkrolik2106 ปีที่แล้ว +2

    Now lets wait for Quad memory channel next gen BIG APUs, some leaks show AMD developing them but only for mobile market. Depends to configuration you will have 256GB RAM and 128GB for iGPU possible.

  • @raysplay2827
    @raysplay2827 ปีที่แล้ว +3

    Good to know that my work laptop m1 max is amazingly good with AI

  • @simpernchong
    @simpernchong ปีที่แล้ว

    Thanks for the smart method :)

  • @denvera1g1
    @denvera1g1 ปีที่แล้ว +1

    I'd love for ROCm to be integrated into TrueNAS Scale
    I'd love to use my 7950x file server's iGPU to not only transcode live TV for Plex(for playack only, long term storage CPU is way better than GPU when going from MPEG2 to H265 and AV1), but also for image recognition through something like PhotoPrism
    Dont get me wrong, the 7950x is an amazing CPU i can record and transcode in real time 6 channels while using only 75w at the socket using Plex DVR and TDARR with X265 CPU transcoding for ~1-2rd to 1/5th file size depending on efficiency of source.

  • @ickorling7328
    @ickorling7328 10 หลายเดือนก่อน

    Ultra helpful video, I've been waiting to confirm this 'list of tested applications!' You know about minu pc's right? Cheapest build, least hassle, and more power efficent!

    • @tech-practice9805
      @tech-practice9805  10 หลายเดือนก่อน

      I heard about mini PC, but are they customizable?

    • @ickorling7328
      @ickorling7328 10 หลายเดือนก่อน

      @@tech-practice9805 Yes. The SER6 is a great pick right now. Beelink SER6 1Tb is THE best bang for your buck on the market when it drops to $420 and below. It has two ram already holding 32gb, and has two m.2 slots: One filled with a 1tb m.2. AND it's thunderbolt port can take on a whole external GPU!!! I don't even think you can get more upgradeable at this rate. Eventually it's maxed out and you just leave it running for hopefully eternity. Keep buying the next most efficient installments of mini-PC's every 1-3 years. Very power effective, and easy to build and manage server rooms!

    • @ickorling7328
      @ickorling7328 10 หลายเดือนก่อน

      @@tech-practice9805 Fourtunately, my dad has done some research 👨 👏 he bought my sister a 6600H APU mini PC for her birthday, and yestwrdsy did a good comparing of PC's.
      We're eyeing the the SER6 MAX mini PC (32gb ram/1tb M.2) at prices about $410, untill like next year the SER7 MAX drops under $450, it becomes the most effective PC for the price.
      A quote from my dad: "... 6600H is 6c/12 thread and has inferior 660M rdna2 graphics with just 6CU enabled. The 6800H and 7735HS are 8c/16 thread with 680M graphics for 12CU rdna2. That's probably 50% more fps on average in gaming. So the 6600H is more like a 900p@30fps LOW gaming machine, while the 6800H/7735HS is a 1080p@30 LOW gaming machine in the SER6 Max... as showcased by TechDweeb."
      And, he also recomends another slightly cheaper mini PC, if you plan on buying based on availability. I'll paste the specs from his sheet (converted from CSV):
      If priced at ~$380: Name: GMKTech K1 ; passmark: 23,801 // Locked TDP and RAM speed; GMKTech cuts corners on cooling so 45W TDP is also the realistic max. // CPU: 6800H / iCPU: 680M / gen: 12CU RDNA 2 // DRAM 16GB / or Max 64GB DDR5 (speed) 4800 max. 1TB / 2TB 1x PCIe 3.0x4 No swcond Nvme slot, there is a 2.5" HDD slot. 3 displays / 2x HDMI 2.1 // 45W TDP. Idle/game/max = 16/58/79W // x3 USB A 3.2 & x1 USB C 4.0 % x1 USB 2.0

  • @bibr2393
    @bibr2393 ปีที่แล้ว +3

    More VRAM is not always better :)
    Your render from 11:00 took you 97 seconds. While my GTX 1080 with "only" 8GB of vram did the same render in 24 seconds.
    This video is pretty misleading as someone might assume that with 16 gb of vram he/she can do high res renders / batch renders / local lora training. Technically yes. But he/she would have to spend weeks waiting for this APU to do the job.

    • @xilix
      @xilix 11 หลายเดือนก่อน +1

      Yeah like, I think this is rad as hell because of the way the GPU market is right now, it's awesome that people can do this, but my 3090ti does that workload in literally 1 second. ONE. I'm usually doing 50-150 iterations, sometimes more, and at higher resolutions with high res fix on top. I suppose if I was coming from having nothing and wanting to gain admission at all, this would be cool, but if I had to go from this card to an APU like this, I'd likely just quit, tbh.

    • @tech-practice9805
      @tech-practice9805  11 หลายเดือนก่อน +3

      I am glad you can afford to high price and powerful cards!

  • @charimuvilla8693
    @charimuvilla8693 15 วันที่ผ่านมา

    Very skeptical about this. Stable diffusion 1.5 requires 4GB of VRAM. You can get a 4GB graphics card for very little money and it will be faster than this. For larger models this CPU will need so much time it would practically be unusable. And the budget 4GB graphics card will still completely outperform it anyway.

    • @user-vw3zx4rt9p
      @user-vw3zx4rt9p 5 วันที่ผ่านมา

      It's surprising the use cases now for a < 4GB small language model. Llama file also works well on AMD cpus

    • @songyang80
      @songyang80 2 วันที่ผ่านมา

      what about if you make 10+ PCs with 64GB VRAM to run large models at once 😂

    • @charimuvilla8693
      @charimuvilla8693 วันที่ผ่านมา

      @@songyang80 CPU-only is not the way to go. I mean you can get an A100 at this point lol. Pair it with a cutting edge SSD and you will probably get orders of magnitude better speed. I would argue that a 4080 alone can beat this. GPUs are just better at matrix operations. That's their purpose.

  • @pratyushbarikdev
    @pratyushbarikdev ปีที่แล้ว +1

    Thanks for putting in the effort.

  • @DamianTheFirst
    @DamianTheFirst 7 หลายเดือนก่อน

    Thanks for your insight! Found yout article on Medium and came here for details
    I'm still tryiyng to wrap my brain around running AI models locally. Do you think it will work also on Intel iGPUs or is there any technical limitation?

    • @tech-practice9805
      @tech-practice9805  7 หลายเดือนก่อน +1

      Hi! Intel iGPUs are weaker than AMD ones. Also they use different software stacks which I don't have experience with. But ideally I hope the iGPUs can all be used for local AI.

    • @DamianTheFirst
      @DamianTheFirst 7 หลายเดือนก่อน

      @@tech-practice9805 Thanks for reply! I'm aware that Intel's iGPUs perform worse that AMD's but I'm already on Intel platform, and I don't want to change for now. My GPU has only 6 GB of VRAM, so I cannot run most of AI models. So far I've been using online things but I'm really excited about running it locally.
      I'm saving for better GPU but maybe try experimenting with running something on Intel's iGPU, if I have some time to spare. Thanks for your work!

  • @MichaelMantion
    @MichaelMantion 7 หลายเดือนก่อน +2

    No clue what you are saying because I can't listen to that music. Maybe remove the music and re-upload.

    • @tech-practice9805
      @tech-practice9805  7 หลายเดือนก่อน

      Sorry about it. I have few posts about it also if that's easier: medium.com/@ttio2tech_28094/democratize-ai-turn-a-95-chip-into-a-16gb-vram-gpu-beats-most-of-the-discrete-gpus-51a8636a4719

  • @RicardoBanffy
    @RicardoBanffy ปีที่แล้ว +2

    From what I understand, it takes 16GB out of the box, right? Did you need to configure any kernel boot parameters for that?

    • @kegklaus5069
      @kegklaus5069 ปีที่แล้ว

      ++

    • @tech-practice9805
      @tech-practice9805  ปีที่แล้ว +3

      That's configurable in UEFI

    • @HikaruGCT
      @HikaruGCT ปีที่แล้ว +2

      It also requires specifik motherboard and bios to allow Uma 16

    • @mister-ace
      @mister-ace 3 หลายเดือนก่อน

      Is it possible to run with 16 gb ram?

  • @toddd.8496
    @toddd.8496 11 หลายเดือนก่อน

    Thank you for this insightful tip. Maybe help out and make a bit of money by providing a list of affiliate links to all the parts in this build? Everybody that watches this video is going to want to see the parts used and some will use your links to purchase.

  • @daves.software
    @daves.software 7 หลายเดือนก่อน +1

    How does this perform for running LLMs? I have an M1 Max Macbook that is actually faster, and can run larger LLM models than my 7900XTX because the integrated GPU uses system RAM as vram. If an AMD apu can do something similar, it would be a much cheaper way to getting large models running. I can run them in CPU mode on PC, but it's 1/10th to 1/100th the speed of my Macbook. It would be nice if I could just get an AMD APU, slap 128 or 256GB of RAM on it, and get 1/2 the performance of an M1 Max.

    • @jacquesdupontd
      @jacquesdupontd 16 วันที่ผ่านมา

      I am wondering the exact same thing (got a M1 Max too) but i can't get an answer and TH-cam videos about the topic seems non-existent

  • @driver7227
    @driver7227 หลายเดือนก่อน

    Buy a secondhand 3060 12GB, it's cheaper than this, much faster, and will work without issues with anything, since it's Nvidia

  • @Donkey-545
    @Donkey-545 8 หลายเดือนก่อน

    Did you follow the same installation procedure for this as your AMD GPU video with the dedicated card? I have been researching and attempting to get this going on a 5650GE to run whisper for home automation, but I had trouble installing on my system. I will be following your guide on Ubuntu 22.x to test if it works, then attempting to run that on top of Proxmox in the future.

    • @tech-practice9805
      @tech-practice9805  8 หลายเดือนก่อน

      Yes, installation is similar to dedicated cards. There is also a step by step guide for APU: th-cam.com/video/H9oaNZNJdrw/w-d-xo.html

  • @j_shelby_damnwird
    @j_shelby_damnwird 9 หลายเดือนก่อน

    Great, if only something similar was implemented for NVDIA gpus...

    • @tech-practice9805
      @tech-practice9805  9 หลายเดือนก่อน +1

      Hope so. It will be interesting. I heard Nvidia will sell CPU, maybe combine with igpu

  • @BravoSixGoingDark
    @BravoSixGoingDark ปีที่แล้ว +1

    So you are saying we can game on iGPUs now on CPUs from both Intel and AMD or just AMD alone? Like playing old console games on PS1 (Duckstation) /PS2 (PCSX2) /PS3 (RPCS3) emulators on Desktop PC?

    • @tech-practice9805
      @tech-practice9805  ปีที่แล้ว +1

      I would say that AMD iGPU is more powerful than Intel's. iGPUs can be used to play those.

    • @AmirZaimMohdZaini
      @AmirZaimMohdZaini ปีที่แล้ว

      If from Intel GPU, you might want to give a try with Stable Diffusion DirectML version.

    • @92kosta
      @92kosta ปีที่แล้ว

      You could always play games on every iGPU, but not very demanding ones. New(er) AMD iGPUs have been really powerful, much more than Intel's. You can easily play PS1 and PS2 emulators on new iGPUs. Don't know about the PS3 one because as far as I know it's a bit resource intensive.
      TH-cam search is your friend.

    • @BravoSixGoingDark
      @BravoSixGoingDark ปีที่แล้ว

      @@92kosta I think PS3 games can be played on AMD's APU G-series like the 5700G's Vega 8 iGPU with 8CUs (Core Units), almost equivalent to a GTX 770. correct me if i'm wrong

    • @bhedgepig9653
      @bhedgepig9653 ปีที่แล้ว +1

      I play DayZ on one and get a steady playable 50-60fps on a low/med mix of custom settings
      Here's RPCS3 emulator on 5600G. Looks pretty bad.
      th-cam.com/video/YmEGIyUrw-M/w-d-xo.html
      Wouldnt expect much at all from newer, flashier titles. I never bother with them so couldnt tell you about that but running UT2004 on max settings at 240fps is still very fun.
      edit: here's 40 games running on a 5600G (no GPU used) with only 16gb Ram
      th-cam.com/video/hCbybCYLJog/w-d-xo.html

  • @djmccullough9233
    @djmccullough9233 3 หลายเดือนก่อน

    Wince. That is. morbidly slow. would have been better to just slap an old 1070 in a workstation. would have been cheaper too.

  • @cmdr_talikarni
    @cmdr_talikarni หลายเดือนก่อน

    Thats a little slower than a $200 8GB RX7600 and a LOT slower than a similarly priced 8GB RTX3050...

    • @songyang80
      @songyang80 2 วันที่ผ่านมา

      good thing is you can make it up to 64GB VRam which you can run huge images, bad thing is it's too slow..

  • @bdd2ccca96
    @bdd2ccca96 ปีที่แล้ว

    888 subscribers. yay !

  • @Nightowl_IT
    @Nightowl_IT 15 วันที่ผ่านมา

    Which of the cheap mainboards can accommodate / allow the highest VRAM?

    • @tech-practice9805
      @tech-practice9805  8 วันที่ผ่านมา

      I listed some of them in this video th-cam.com/video/H9oaNZNJdrw/w-d-xo.html

  • @brencancer
    @brencancer ปีที่แล้ว

    deserve a subscribe

  • @007Marnie
    @007Marnie 7 หลายเดือนก่อน

    Nice idea, what is the name of motherboard you are using? thank you

    • @tech-practice9805
      @tech-practice9805  7 หลายเดือนก่อน +1

      thanks! I think most AM4 motherboard should work. I tested on MSI B450-A PRO MAX

  • @renebibibi9283
    @renebibibi9283 ปีที่แล้ว +2

    will that work with similar Mobile APU like ryzen 4500u or R7 5700u ?

    • @tech-practice9805
      @tech-practice9805  ปีที่แล้ว +1

      for laptops, the motherboard bios may be locked. It depends on the brand and can check with their manufacturers.

    • @VioFax
      @VioFax 4 หลายเดือนก่อน

      @@tech-practice9805 build your own. Don't buy dell or HP. You will NOT be accessing any of the UEFI stuff you need for ML research.

  • @ViteshSingh
    @ViteshSingh ปีที่แล้ว +1

    @tech-practice9805 Will this work with 4650G? You mentioned Asrock board, any chance it will work on any of the Asus motherboard?

    • @92kosta
      @92kosta ปีที่แล้ว

      It should work, but the question is how much RAM will that ASUS motherboard allow you to allocate to the iGPU. Some motherboards top out at 4GB of VRAM.

    • @tech-practice9805
      @tech-practice9805  ปีที่แล้ว +1

      I have an Asus MB and it works! The maximum is 16GB for iGPU.

    • @ViteshSingh
      @ViteshSingh ปีที่แล้ว

      That is so awesome! What's the total RAM you have on there? Did you have to flash a custom firmware on the motherboard?

  • @alx8439
    @alx8439 6 หลายเดือนก่อน

    Few observations from your video:
    1) you have a lot of "noise" on CPU activity side. Even before you actually started inference, there were like 30-40% CPU used, just visually from Gnome System Monitor. This is no good. You might be running some extra heavy shit on a background which is interfering with your test. OBS?
    2) before jumping to a conclusion like "hey just buy Ryzen APU and allocate more RAM to be used by iGPU", you should have made the same inference test on CPU only. I do own couple of Ryzen powered machines and the least powerful 3400g is having pretty much same figures (2 minutes for one image generation in SD 1.5 via a111) using CPU only. And my other beefy 7940HS is great at inferencing Mixtral 8x7B 4 bits quantized, giving me like 10-15 tokens per second in llama.cpp gguf.

    • @mister-ace
      @mister-ace 3 หลายเดือนก่อน

      I’m using stable diffusion web ui , but can only select my cpu (4600G), I can’t select igpu (vega7) , can you help me?

  • @kegklaus5069
    @kegklaus5069 ปีที่แล้ว +2

    is this work for LLaMa 2 - 13B?

    • @VioFax
      @VioFax 4 หลายเดือนก่อน

      No, 13b is too big for a machine like this. You would struggle to get a 7B to run faster than molasses.

  • @fritmore
    @fritmore 9 หลายเดือนก่อน

    great work , chief

  • @Techonsapevole
    @Techonsapevole ปีที่แล้ว +3

    Does SDXL work on CPU ?

    • @DerrangedGadgeteer
      @DerrangedGadgeteer ปีที่แล้ว +1

      All AI tasks will work on CPU. (In principal at least... Somebody still has to write a script for it.). You just have to be willing to wait. Even As fast as modern CPU cores are, there's still at most a dozen or two available. GPU'S have thousands. That's why on CPU an image generation takes three minutes, whereas on GPU it's closer to three seconds.

    • @Techonsapevole
      @Techonsapevole ปีที่แล้ว +1

      @@DerrangedGadgeteer with sdxl on cpu it works with 32gb ram it takes 60s with sd1.5 and 90s with sdxl for a 512px image

    • @DerrangedGadgeteer
      @DerrangedGadgeteer ปีที่แล้ว

      @@Techonsapevole That's pretty dang quick for CPU inference. What CPU did you use to get those times? I'm genuinely curious.
      I've been an advocate for repurposing old server GPU'S for hobbyist use. Nvidia Tesla M40'S came with 12 or 24GB of Vram, they can crank out Stable Diffusion images at about 12s per image, and they're getting CHEAP since data centers can't afford to feed whole racks of them at 250W each. That said, if power efficiency is your goal, then that's not gonna be the way.

  • @NoidoDev
    @NoidoDev 10 หลายเดือนก่อน

    Has this been tested with LLMs? Where is the best place to find more people trying this?

    • @tech-practice9805
      @tech-practice9805  10 หลายเดือนก่อน

      yes, it works but inferencing is slow

    • @NoidoDev
      @NoidoDev 10 หลายเดือนก่อน

      @@tech-practice9805
      Do have any examples for the speed? Benchmarks? Token per seconds? Did you try it with a 13B or bigger one? 16GB or 64GB? How does it compare to CPU inference? Maybe make another video about it.

  • @VioFax
    @VioFax 4 หลายเดือนก่อน +2

    16 GB isn't really enough unless you are okay with taking those HUGE accuracy losses and slow replies... Just isnt that good. I have 64GB and still need way more.. You want at least 128 for GOOD LLM fun. You REALLY want like a quadro or a pascal40 but those are too hard to get working on anything but very specific hardware. Artificial lockouts against workstations...Some people get them to work in certain gaming boards but then you are still limited by the maximum memory of gaming borads at usually 64GB... They have intentionally crippled ram on new systems also. soldered in, or capping new boards at 32 GB... they don't really want you messing with these LLM's in any REAL capacity... Not unless you got money. Or corpo/militech/government funding.

  • @PokettoMusic
    @PokettoMusic 11 หลายเดือนก่อน

    dude the info is good and all but you need to consolidate the info in a single video, like it is right now is a huge mess D:

  • @ZulfikarAjiKusworo
    @ZulfikarAjiKusworo 11 หลายเดือนก่อน

    Hi, i'm using Ryzen 4650G CPU (same Vega 7 iGP as 4600G), with Ubuntu 22.04 and ROCm 5.4.2, HSA_OVERRIDE_GFX_VERSION=9.0.0 . all installation is successfull, but all generated image are black. do you know what's wrong with it? i'm also trying ROCm 5.6, but it gives me error about HIP kernel and checkpoint model did'nt loaded successfully

    • @tech-practice9805
      @tech-practice9805  11 หลายเดือนก่อน

      which stable diffusion program were you using? it may because of half precision

    • @ZulfikarAjiKusworo
      @ZulfikarAjiKusworo 11 หลายเดือนก่อน

      @@tech-practice9805 i am using A1111 webui. as you said in other comment, that torch 2+ROCm5.4.2 is give black image result. it’s same as this problem. do you successfully running A1111 SD webui with torch 2 + ROCm 5.4.2 / 5.6?

    • @tech-practice9805
      @tech-practice9805  11 หลายเดือนก่อน +2

      @@ZulfikarAjiKusworo I will upload a new video soon which seems working with torch 2

    • @ZulfikarAjiKusworo
      @ZulfikarAjiKusworo 11 หลายเดือนก่อน

      @@tech-practice9805 nice. i’ll be wait

  • @selfs6111
    @selfs6111 ปีที่แล้ว +1

    can you game with this?

    • @tech-practice9805
      @tech-practice9805  ปีที่แล้ว

      Yes, I can run some benchmarking for it

    • @selfs6111
      @selfs6111 ปีที่แล้ว

      @@tech-practice9805 I would absolutely LOVE that

  • @mope4188
    @mope4188 ปีที่แล้ว

    What we do with this AI application ? earning money ?

    • @tech-practice9805
      @tech-practice9805  ปีที่แล้ว +1

      anything AI can do! including earning money!

    • @VioFax
      @VioFax 4 หลายเดือนก่อน

      You aren't really allowed to use AI to make money. Unless its your own trained model. I'm sure thats not stopping anybody though.

  • @HikaruGCT
    @HikaruGCT ปีที่แล้ว

    what is the motherboard and its bios version ?

  • @Violethood
    @Violethood ปีที่แล้ว

    How to install those AI models?

    • @tech-practice9805
      @tech-practice9805  ปีที่แล้ว

      I will upload a detailed tutorial. please subscribe to stay tuned!

  • @vin.k.k
    @vin.k.k ปีที่แล้ว

    3400G?

    • @tech-practice9805
      @tech-practice9805  ปีที่แล้ว

      It can work. But for old generations, the motherboard bios may not support it. So it depends on the brand of motherboard and need to test

    • @vin.k.k
      @vin.k.k ปีที่แล้ว

      @@tech-practice9805 awesome. I have one idling around

  • @lightningroood
    @lightningroood ปีที่แล้ว +3

    waste of money and time. you're way better off buying modded 3060/3070 with 12GB/16GB VRAM from taobao for around 200 bucks

    • @tech-practice9805
      @tech-practice9805  ปีที่แล้ว +12

      As mentioned in the title: "democratize AI". Assuming lots of people cannot afford discrete GPUs.

    • @jordanhofer7376
      @jordanhofer7376 ปีที่แล้ว +5

      Waste of time and money, just buy a $95 4600G.

    • @lightningroood
      @lightningroood ปีที่แล้ว

      ​@@jordanhofer7376nah, i use 3090ti

    • @simpernchong
      @simpernchong ปีที่แล้ว +1

      No, you can now run Stable Diffusion on Ryzen laptops.. Can now run SD on machines that are not supported for AI.

    • @AmirZaimMohdZaini
      @AmirZaimMohdZaini ปีที่แล้ว +1

      ​@@lightningrooodScrew those comsumer level gaming GPUs, use Nvidia RTX A4000/Teslas etc. for better Stable Diffusion memory handling.