In this video I demonstrate a Local LLM setup for software development dual a dual NVIDIA 3090 setup.
Documentation: github.com/letsRTFM/AI-Workstation
Motherboard: BD790i store.minisforum.com/products/minisforum-bd770i
CPU: AMD Ryzen 9 7945HX 5.4 GHz - 16 Cores 32 Threads browser.geekbench.com/v6/cpu/4875003
Affiliate Links:
Power Supply: Corsair 1000W SFF-L: amzn.to/459azXN
RAM: Crucial 5600 96GB Kit amzn.to/49RVpaD
Storage: Crucial T705 4TB Gen 5 NVME amzn.to/459Zh5p
Case: Geometric Future Model 8 amzn.to/457XwWw
PCIE Bifurcation Adapter: JMT PCI-E 4.0 x16 1 to 2 PCIe Bifurcation amzn.to/3VmQtWC
PCIE Vertical Mount: EZDIY-FAB Vertical GPU Mount with High-Speed PCIE 4.0 Riser Cable amzn.to/3X5NjI0
PCIE Riser: PCIE 4.0 Extension Cable Length 250mm amzn.to/3X5NjI0
Cooling:
5 x Noctua NF-A12x25 amzn.to/3KnKQRU
3 x Noctua NF-A14 amzn.to/3X1i2Gl
1 x Noctua NF-A12x15 amzn.to/3uNb0Ju
Music By Suno AI: drive.google.com/file/d/1LNic1nBz-yTRzE7kF5-LsrLTm4BdkEZz/view?usp=sharing
The 7x4090 build linked at 0:32 can be found here: www.mov-axbx.com/wopr/wopr_concept.html
00:00 - Intro
00:24 - Hardware Overview
00:48 - BD790i PCIE 8x8 Bifurcation
01:42 - Geometric Future Case and Rotated Dual GPU Mounting
02:10 - Rocky Linux NVIDIA CUDA Setup
03:20 - Postgres For Wordpress
03:28 - Running LLM Models Locally using Llama.cpp
04:41 - Coding LLM Model Selection
05:55 - Starting Llama.cpp
07:35 - Setting up Continue in VScode for with Llama.cpp
08:06 - AI Assisted Coding
10:45 - AI Assistant Caveats
11:07 - Outro