ChatGPT Running Locally on Raspberry Pi (ft. Ryan Reynolds)
ฝัง
- เผยแพร่เมื่อ 1 ต.ค. 2024
- Is it just me or does Ryan look different?
Full Tutorial Instructions Here: bit.ly/gpt-on-rpi
Order the NEW Tinker Project Ultimate Dev Kit Here 👉 tinkerprojects...
In this tutorial, I guide you through setting up ChatGPT on a Raspberry Pi, allowing you to chat seamlessly with advanced AI models. Perfect for AI enthusiasts, Raspberry Pi users, and DIY tech lovers!
🔗 Links:
Step-by-Step Tutorial: bit.ly/gpt-on-rpi
GitHub Source Code for alpaca.cpp: github.com/ant...
Download Model Weights from HuggingFace: huggingface.co...
Instructions:
Choose your OS-specific zip from alpaca.cpp releases (including alpaca-win, alpaca-mac, alpaca-linux).
Pair downloaded model weights with the 'chat' executable.
Launch & initiate an interactive chat with AI on Mac/Linux using ./chat or on Windows via .\Release\chat.exe.
Build from Source: For hardcore DIY fans, we've got detailed build-from-source instructions.
🔥 Highlight: Meta's LLaMA - Pioneering the Future of NLP
Discover Meta's groundbreaking Large Language Model, LLaMA. It's a revolutionary step towards democratizing AI and making top-notch NLP accessible to all, even those with constrained resources.
LLaMA Features:
Ranges from 7B to an impressive 65B parameters.
Rigorous training with up to a staggering 1.4 trillion tokens.
Designed with versatility to combat biases and toxicities.
At the core of our mission? Combining cutting-edge research with responsibility. We're dedicated to minimizing biases in AI. By offering conditional access to LLaMA, we prioritize ethical AI use and champion significant AI breakthroughs.
Apply for LLaMA Access: Dive deep into our research paper for more insights.
Join us in shaping a collective AI future, pooling knowledge from academia, the tech industry, and policy-makers. Can't wait to see the innovations you'll spark with LLaMA!
![[Full Episode] The Restaurant War Thailand ศึกพ่อค้าซ่าแม่ค้าแซ่บ Episode 2 | 29 ก.ย. 67](http://i.ytimg.com/vi/r3lfLNH6Ke4/mqdefault.jpg)
![CAMPปลิ้น | EP.82[1/2] แก๊ง 3 สาวเพื่อนซี้บุก CAMPปลิ้นครั้งนี้ห้ามเสียอาการ](http://i.ytimg.com/vi/_6WUjXHe1TM/mqdefault.jpg)
Full Tutorial Instructions Here 👉 bit.ly/gpt-on-rpi
Use text to speech to transfer your question and a small speaker connected to get answer .plz you can do this
reported for misleading. Llama is NOT ChatGPT. and you show a Pico in the thumbnail, but actually install on a full size Pi.
@@peterbelanger4094lol reported? TH-cam don't care for ya false claims karen
@@peterbelanger4094 unfunny nerd
"Running ChatGPT on raspberry pi!" Proceeds to install llama....what's wrong with you?
😂
ChatGPT attracts more people, usual clickbait
Yes, and Click Bsit are NOT cool.
Chatgpt is quite literally a llm like most llama models, what is your point? Most people will not know what "llama running locally on a rapsberry Pi" means.
Basically instead of teaching ppl you used buzz words to get views. That's my point. Anyway, guess I can't judge. Don't know much about having a youtube channel.
Did he seriously just call like at most a 7B parameter AI model available for free on HuggingFace 'ChatGPT'?
well, AKSHUALLY...
professional research has been done and with recent distillation algs are producing LLMs that are consistently outperforming OpenAIs ChatGPT significantly.
for example, X's Grok LLM is quantized approx ⅓ of ChatGPT 4 and outperforms it on every single benchmark. also, to be fair, X even used OpenAIs own benchmarking tests, which should have given them the upper hand.
also, keep in mind that it only took the team behind Grok ~6 months to revolutionize LLM's. AI is moving at break neck speeds and the more advanced it gets will make it that much quicker to develop and implement much more powerful, efficient and extremely accurate models.
I was kinda piss like this is a win for meta give them their appreciation
@@SirToddmeme boy
Well, my guess is he used the llamaindex and embedded an OpenAI Model, which connects to the OpenAi API using the API key that one can get from his OpenAI account. This is probably also the reason he called it ChatGPT, because the OpenAI API used to use GPT 3.5 Turbo Up until recently. But I'm not sure either 🤷
Nevermind, rewatching the video, the console definitely wasn't outputting the correct GPT code, you were absolutely right
Bruh that ain’t Ryan Reynolds that’s Bryan Brainolds
Reynan Rynolds
Ha! A marvel jesus
Clickbait. Take the microcontroller model out of the thumbnail
Sick of this guy and his bait. Blocking.
Fr, the whole reason I clicked on this is because of the thumbnail.
you can't change thumbnails on shorts. youtube decides them for you.
@@timmmm5012 You definitely can, it's part of the upload process.
That’s not chatgpt, that’s facebook’s LLM
That had nothing to do with a Pi Pico. Why show it?
Because clickbait lol
I thought he just didn't know the difference
He didn't say pico
@@Jblow-u2m there’s a pico in the thumbnail
came here to say this too, wtf man lol
"i don't think raspberry pi os is up for the challenge"
*uses worse os*
@NirjharNaskar lol
@NirjharNaskar what.....
No.
There are nothing erong with Ubuntu, but Raspberry os is as much up for the challange. Might be better, because it doesn't use as much memory.
The problem is that the hardeare isn't up for the task of running large models, and the models that can run isn't good enough to challange ChatGTP.
And ollama with llama models are NOT ChatGTP.
@@AndersJackson snap packages + spyware ig on the ubuntu side
but yes, its fine for running a llm just like pi os
I hate this clickbait so much, pico is never used and uses LaMA instead of ChatGPT. What was this even for? to harvest dislikes I guess!
Most people dont know what llama is. So saying chatgpt is easier
@@ash-tn6flIt's okay to lie to people that won't know the difference?
"Im poor", meanwhile bros chilling in what looks like a penthouse 💀
For a second I was excited when I thought he was gonna run the language model on the pico
Clickbait...
Technically you can, llm on the edge is possible
Even with a 7B parameters model, best you could get is 0.2 tokens per second with that kind of hardware
yeah i made an ai voice assistant thats supposed to run on pi so i used 0.5b. it is crazy fast on my pc so im confident
Bro literally has no idea what he's doing lol. First he holds up a pico then proceeds to use a pi, then uses a worse OS because he doesn't understand how pis work, then uploads code he doesn't understand.
Good work buddy.
At least you're trying in fairness. If you do this enough hopefully you'll start to pick up on the differences. Really look into it though, try to not just blindly follow someone else's guide online
And he doesn't know that ChatGPT and Llama are completely different 😂
in my opinion he knows what he’s doing, at least I see that he baited you into comments
@@mrdiamond64its close enough. Dont try to be smart
Bro always talks about how poor he is:
Also his videos in big 1M dollars penthouse with a full sea view:
Tech "influencer" try to not to lie and be wildly incompetent challenge (impossible)
While flexing his high rise apartment.
The amount of hype and click bait in this video is incredible
You're probably the kind of person who call any SUV a JEEP.
Bro his voice is ai
It's not ChatGPT. It's GPT. ChatGPT is the webapp. GPT is the model. And yes, I know he's using Llama in the video.
im sorry what ??
“ I don’t think pi os is up for the challenge”
*proceeds to run cpp code*
How does that relate to pi os?
@@someidiot4311yeah I’m confused too lol, kid be talking gibberish
How many hours does it take for an answer to be generated in response to a prompt? Cheers!
😂. I tried it too. Its because they have no GPU and the CPU is too bad
chat gtp is not running on your computer it is a online service you can book. ollama is runing on your system aber a pi would be overwhelmed by this because the smallest models are 6gb and has only 8 billion parameters. even on my r630 every question takes 30sek up to 6 min. but running something bigger is more ridicules. there are bigger models who has 70 billion parameters and this models are 60gb big and some of them are up to 400 billion parameters. but this is nothing compared to gpt4 witch consists of 1.8 trillion parameters and is something about 700tb big. so jea that's why it is running online on specialized computers with hundreds of gpus in it. how ever you can run models at home. they need some time to answer but this is not that bad. the tool ollama is great and simple for that job. it is easy to use and models are 6gb each for the 8bilion models and 60gb for he 70billion models and you dont need a gpu if your gpu is not kompatible or slower as your cpu wold do the same task. but for real a gtx 730 is faster as your ryzen 9 5950x so yea your would probably have something bigger as this in your system and if not ollama is compatible with the google coral and this has the same performance like a 1660 but uses only 1.4w and is as usb device available for about 100 dollar.
how i said i use a ollama on my r630 in cpu mode and use the 70bilion models so yea they need some time but they work.
I have something in my mind that seems to had been never done about raspberry пайс. I'm thinking I'm gonna challenge you with it but it may be impossible by now. I think.
What was the token speed
Total clickbait. That thing is never going to run any capable LLM and you can't even run ChatGPT because it's private. Stop lying dude, any small kid can see you edited the responses.
If you're poor, why did you go for a $100 dollar board?
Exactly. The dude is a big nasty lie.
Why you damn promote raspberry Pi boards like seriously 😒
Wow, that is so fast! Looks like it might even reach... 10 seconds per token. I hope you'll trick many suckers today good sir.
Exactly, anyone who's used a local LLM on CPU only has incredibly slow inference speeds
And you cannot run a GPU on a Pi 4. @JeffGeerling is able to add a GPU to a Pi 5 - but not a 4 reliably.. Certainly NOT the Pi4.
You can
"erhm actually, there is a wii port of chatgpt that's only a few megabites 🤓☝️"
That's remote.
Does it have internet access capabilities?
I'm now wondering if I can put a talking A.I on my RC truck 😂
Bro this is goddam clickbait cuz you showed a pico which is a horrible computer
A Pico isn't a computer, its a microcontroller.
@@de-y look bro I’m calling him out on clickbait ok I’m not looking for grammar correction.
@@Calooonzy I'm just defining a Pico to you
@@de-y I own a pico dude I think I know that. People won’t understand if I said microcontroller.
@@Calooonzy I also own a Pico, but you should've stated it first instead of all this.
Anyways, what do you use your Pico for? I've been trying to do some micropython.
That’s a pi4 correct? If you’re going to copy him GET THE 8GB!! it will cut down on your massive latency. Better yet, get the pi5. The pi5 is the highest power to dollar computer you can buy. $100 for 25 TOPS. $4 a terra-operation.
What a sad soul.
He has no idea what he's doing, even though he mentions "installing an AI" on a Raspberry Pi and shows a Pico microcontroller. He mentions ChatGPT and proceeds to use Llama.
Get some decency, dude.
Why isn't raspberry pi OS up to the challenge?
Why is Ubuntu??
Just because he wants to.
Ignore that part
It's not running locally and having a pico for the thumbnail is pretty odd. Still cool but chatgpt is not running locally on your pi. You are running something locally that still depends on chatgpt which is outside.
It's 100% local, full tutorial here - th-cam.com/video/N0718RfpuWE/w-d-xo.html
@@DataSlayerMedia you charge for a tutorial, fuck that
Its not local you dofus, because OpenAI doesnt let you download the model weight. You are just making remote calls.
If it is running local this is not chatgpt lol
It's not local. 2nd, Llama and chatgpt aren't the same either, but hey, tech "influencer".
1. That aint ryan Reynolds
2. You showed a pi pico in thumbnail when its a pi 4. Big difference, one has 4gb of ram and the other 260k.
3. You didn't even run chatgpt, you ran LLAMA.
4. How do you even have that much storage to run the model?
"But I'm poor,
Look how poor my apartment is and how basic MacBook I've got"
But cool video, thanks, please create more!
_____
P.S. ruzzia is evil
You can right easy a 3 gb bash ai better than that ( more useful than you can use OpenAI), but you should use at least 12 GB or 30GB but everything else is kinda ridiculous.
Llama 2 isn't ChatGPT...
Exactly my thoughts
It's impossible on so many levels. You're using a SMALL, and OPEN SOURCE model, AS YOU KNOW. Liar liar pants on fire.
Either you are Mr. P. Solver or you be kinda copy pasting. Either way don't clickbait me with a pico thumbnail
Make a messaging app, have two instances communicate with random topics , take highlights and make a series .
Charge people to add topics , host results online.
Make money…
“If it’s so easy why don’t you do it? “ I never complete a project, too busy
Data Slayer is the worst clickbaiter on earth. There aint even any Ryan Reynolds in the vid (except for the start) and his "this will kill phone providers" vids are just clickbait asf. No way are they gonna kill phones.
Like raspberry pi isn't easy mode because drivers and chips are open source materials. If you programm the big thing yourself
You can do it but as you can see is so damn slow. It’s faster to search it manually, trust me.
Otherwise a good thing is to use an specific model for an specific task, so it’s lightweight and fast.
This kind of clickbait is the exact reason we have a “don’t recommend this channel” button (tap and hold on the screen while the video is playing)
I get calling it ChatGPT for the algorithm. But also this is for technical people who know the term llm and other models. So itd be better to not use a misleading title
This is not chatgpt though
I saw the title and wondered how did this guy run chatgpt which is a proprietary model on his local computer? I should have guessed. Disliked.
You advertised it with R Pico which is not in the video. I don't trust anything you say.
It's Llama2 by Meta
Not Got by OpenAI
(Gpt4/3.5 ain't open source)
Claickbait asf showing a pico, i knew you couldnt but had to see
I just liked cause of Ryan Raynolds
Didn't understand much else of the video
Actually, I was able to run Phi3 and Llama8b in the pi os. I got VS code running great too as well as Crew Ai
Dont. Vscode is laggy on pi
@@ash-tn6fl For you not me. I have zero issues working with VScode. In fact my pi is now where I do all my coding because it's a hell of a lot easier for me in Linux than Windows or osx. However, you are correct that an LLM will run slow for obvious reasons. However for a few hundred dollars you can get an adapter to connect an eGPU and increase the inference time.
Time to take over the world with my raspberry pi
LlamaGPT
How can you make history, by doing something that countless others have done?
you are not running the model out of the raspberry, you are lying to sound smart. You need a lof of those to run chatgpt...
He said Ryan Reynolds but I’m pretty sure that was Nick Swardson
There's no useful purpose for running an llm on such an instance ... as far as I have imagined
This guy is going places, where, I have no idea. Probably across the road
Ah, more lying for clicks, reported as spam and blocked
was it true or just a parody if gpt can run on pi then tell me how
Full Tutorial Instructions Here 👉 bit.ly/gpt-on-rpi
I bought a robot to take apart and make it talk, but i don't know nothing , but i do have a arduino, still learning.
You don’t have to be a SWE to know Llama is not ChatGPT….
Heyy! You shoulf stop uploading 😊
That´s not ChatGPT, just a less performant knock-off.
How many should I cluster to run Doom locally on gpt on piraspberry?
"Why did the chicken cross the road?" ... Ten years later ... "To get to the other side"
Dude its not that hard the way you do so you can do it on a 2 bit bash of an IBM first gen easily
You can run linux on a raspberry pi everything run on that.
I was deceived and now my morning is ruined
Where is the tutorial which outlines the processss step by step?
Bro is broke and he is flashing ubuntu from mac😂
Don't show pi Pico if you are not going to use it 😢, you made me sad.
My potato is way smarter. It just ignores the questions 😂
In turkey someone maked a thing like this for university exam😂
Bro the video thumbnail used a pico, what?
I've been scammed, i thought you would do it on the pico
A pico and a Pi 4 arent the same thing.... #stopclickbait
So no one's gonna talk about the internet history and interesting choices?🥶
Can it be used as a personal coding assistant with just 4 gb ram?
When you install llama, you should not say chatgpt.
Ctrl c ctrl c ctrl c terminate it before it terminates us
That is not ryan renolds that is ryan thick and olds
Llama model with a less parameter is lame, it still needs time
These devices will only get worse. Can we cease Ai?
OK is this about running gpt on raspberry pi 4 or pico?
That AI is made in China 😂
Ubuntu is literally a worse and much less optimized... Also it's a pico at the beginning
Blud couldve just used gpt 2
Is this the new Doom (runs on anything)?
that is not chatgpt or anything close to it cuh
'I totally get this code' lmao relatable
Becouse im poor *shows a 2k macbook*
So when are you going to do what you said instead of using llama?
Impossible. OpenAI's name is inaccurate so gpt is closed source
Ryan made a one sec appearance lol
Shows a Pico, doesn't use it.