server parts list please, also thise vids analyzing the performance of various hardware and models are interesting, but what do you use this ai sever to actually accomplish????
I touch on the different node specs in several of the videos, but have been meaning to do short videos on each of them. As far as usage for these nodes, depends which one...but endless. Mostly api backends to different applications/automations I build and play with that mimic OpenAI standards. Research and Development Daily coding tasks Agent/Crew/Team based projects using several models all at once - my own agent based engineering team (or is the goal...) Proving me wrong Making up things that don't exist in code etc, etc
Wild.
Great info for a guy looking to create my own AI server for the larger models.
What about a 3090Ti 24GB plus a 4060 16GB (or two)?
I don't have a 3090 in the lab, but looking to sell/trade a few 4060's for one since everyone defaults to the 3090.
server parts list please, also thise vids analyzing the performance of various hardware and models are interesting, but what do you use this ai sever to actually accomplish????
I touch on the different node specs in several of the videos, but have been meaning to do short videos on each of them.
As far as usage for these nodes, depends which one...but endless.
Mostly api backends to different applications/automations I build and play with that mimic OpenAI standards.
Research and Development
Daily coding tasks
Agent/Crew/Team based projects using several models all at once - my own agent based engineering team (or is the goal...)
Proving me wrong
Making up things that don't exist in code
etc, etc
What about the P100? it has a bigger bit rate.
I don't have any pascal generation cards in the lab, well an old p2000 but that wouldn't stand up to a P100 for tests
@@RoboTFAI I would love to watch a video comparing tokens per second vs memory type, if HBM2 vs GDDR