ไม่สามารถเล่นวิดีโอนี้
ขออภัยในความไม่สะดวก
The hidden dangers of loading open-source AI models (ARBITRARY CODE EXPLOIT!)
ความคิดเห็น • 110
ต่อไป
เล่นอัตโนมัติ
How to make your CPU as fast as a GPU - Advances in Sparsity w/ Nir ShavitYannic Kilcher
มุมมอง 50K
the truth about ChatGPT generated codeLow Level Learning
มุมมอง 222K
NEW Mixtral 8x22b Tested - Mistral's New Flagship MoE Open-Source ModelMatthew Berman
มุมมอง 54K
[TH]FFWS SEA 2024 Fall - Knockout Stage Day 3Free Fire Esports TH
มุมมอง 100K
ทนายเดชา ล่าสุด (หมดที่ยืน)#บิ๊กป้อม#เพื่อไทย#เศรษฐา#เท้ง ณัฐพงษ์#พรรคประชาชนข่าวเด่นวันนี้
มุมมอง 475K
LISA - NEW WOMAN feat. Rosalía (Official Music Video)LLOUD Official
มุมมอง 40M
Try the #NewWomanChallenge now! Brought to you by YouTube Shorts #LISAxNewWomanLLOUD Official
มุมมอง 17M
Hugging Face SafeTensors LLMs in OllamaLearn Data with Mark
มุมมอง 3K
It’s Been a Good Run, Phone Providers.Data Slayer
มุมมอง 4.4M
AlphaGeometry: Solving olympiad geometry without human demonstrations (Paper Explained)Yannic Kilcher
มุมมอง 35K
Neural Networks are Decision Trees (w/ Alexander Mattick)Yannic Kilcher
มุมมอง 56K
All You Need To Know About Running LLMs Locallybycloud
มุมมอง 139K
Is Rust the New King of Data Science?Code to the Moon
มุมมอง 135K
Getting high paying tech job through open source contributionHarkirat Singh
มุมมอง 117K
AI, Machine Learning, Deep Learning and Generative AI ExplainedIBM Technology
มุมมอง 74K
Modern Python loggingmCoding
มุมมอง 176K
โผ ครม. "อุ๊งอิ๊ง 1" นัดส่งชื่อพรุ่งนี้ (20 ส.ค.) คาดเพื่อไทยขยับหลายตำแหน่งสํานักข่าวไทย TNAMCOT
มุมมอง 198K
🔴LIVE เชียร์สด : อิปสวิช ทาวน์ พบ ลิเวอร์พูล | หงส์แดงบุกรังม้าขาว นัดเปิดฤดูกาลGOTJISIRI CH
มุมมอง 307K
นี่คือ Texture Pack ที่โคตรจะสมจริงที่สุด!!! #เกมกับshorts #minecraftMrWattana
มุมมอง 906K
ติดบนแพ กลางทะเล 24 ชั่วโมง!! จะรอดหรือจม?!!My Mate Nate
มุมมอง 1.9M
เลือกยากจัง Ep.5 👥 #แป๋มวัดดวง #freefireแป๋มวัดดวง
มุมมอง 178K
ไฮไลท์ฟุตบอลพรีเมียร์ลีก 2024/25 สัปดาห์ที่ 1 : เชลซี พบ แมนเชสเตอร์ ซิตี้TrueVisionsOfficial
มุมมอง 1.3M
PASULOL กุ่ย นาย ฮอล และชีวิตติดหน่วงPASULOL
มุมมอง 7M
OUTLINE:
0:00 - Introduction
1:10 - Sponsor: Weights & Biases
3:20 - How Hugging Face models are loaded
5:30 - From PyTorch to pickle
7:10 - Understanding how pickle saves data
13:00 - Executing arbitrary code
15:05 - The final code
17:25 - How can you protect yourself?
Try the model: huggingface.co/ykilcher/totally-harmless-model
Get the code: github.com/yk/patch-torch-save
Sponsor: Weights & Biases
Go here: wandb.me/yannic
Lovely.... I already downloaded the Stable Diffusion open source v1.4 off Hugging Face a few days ago. Now I'm worried. Anything I can do after the fact?
TGO th-cam.com/video/2Hqc2QA5aU4/w-d-xo.html🌐📊📲🛰️✅🔚✔️🚀📉🆒🕶️🛰️📸☢️Tap on a clip to paste it in the text box.
I always new pickle was unsafe, but I never understood the programmatic underpinnings that of how pickle worked behind the scenes. Thanks for sharing.
This is awesome. in one video you have not only explained how you went about finding the exploit but also went full circle of how to give back to open source!
Best AI youtube channel, hands down
Thanks for the notice, almost watched the whole ad. Pretty good open for a vulnerability video.
Thanks for the video. I've used pickle endless times and was never aware of this pitfall. Also, thanks for the very understandable explanation of the technical details.
Thank you. I had been meaning to look at pickle properly for a while now. This has been a wonderful rundown, and explains some of the warnings I saw when people were discussing pickle, yet I hadn't realized precisely why it was so powerful/dangerous.
Edit: possibly we need a text extractor for pickled data that doesn't try and do anything other than reveal the contents in a human readable format.
Edit 2, your explanation allowed me to understand whats going on under the hood, and I was able to use this new knowledge to my advantage already, Thank you again, and 3 thumbs up.
Love your videos, keep up the good work!
This is awesome! I gave a talk today at my company to raise awareness of malicious code in AI models. Thanks for sharing! very useful examples.
Thanks. Instructions are very clear. I will be loading every model I can get my hands on.
I’ve been using wandb for months nonstop 🔥
Hey yannic, I think it would be cool if we get to see what you're working on from day to day (if possible), an example of your current projects and your day to day relationship with machine learning. Im curious because I'm at a stage where I dont know what I want to do with my current base knowledge and skills in machine learning, so this would be inspirational.
Thanks.
Wow really cool that you contacted them and made them aware before releasing this video too!
Nice video, thank you! I avoided this problem by using ONNX models. Does any1 know whether there are security holes for these models?
There is a PoC of ONNX malware on GitHub repo alkaet/LobotoMl but the attack path does not appear complete yet (requires custom-op registration).
Awareness is very much a double-edged sword, since it also gives people the idea to make the malware. It only takes one person to make malware, but it takes everyone to go out of their way to be careful to avoid these attacks.
No it's not, security by obscurity doesn't work. Cybersecurity 101
TLDR; Old man finds out about deserialization exploits.
It's surprising how few people understand how deserialization works and what sort of dangers lurks in deserializing untrusted data, great explanation!
Now the next step would be a rickrolling model.
Haha, OMG I can't believe they use pickle as it is in torch (and huggingface). Pickle is not save to use. Thanks Yannic.
Not as hopeless as you might think. Any medium to large sized tech company that's concerned with security can run models in a sandboxed environment. You could run something like strace to figure out what system calls the program is using and disallow most/all by default, whitelisting them one by one.
This sounds like a simple solution - strace, execute the reduce method, see what happens, then use that info to generate a file that shows user what syscalls will be made when you load that model. User can accept or deny. But even this is pretty easy to workaround, at the end of the day it is an insecure method of hosting content
Thanks for the video Yannic!
this is just one of those occasions where I need to find some time somewhere to actually dig into the code a bit more!
A more elegant alternative to patch_torch_save seems to be a function decorator as a function factory.
U made it simple man ur subscribers are well deserved
You’re doing the Lords Work, thanks for bringing this up to the community!!
according to the youtubes, I hit the first like! yeet
You should make a hugging face model that exploits this to open this video on the victim's machine, haha
I didnt realized that pytorch uses pickle!. Thanks.
This is very interesting. I was not aware of these problems! Thanks!!!
My recommendation is to run untrusted packages and models in Docker containers. This limits the blast radius if you do accidentally run malicious code.
Thanks for doing wonderful things to the society.
Thank you for making us aware, big security risk that should be fixed
The "where did it go" part is so ryt now
Great information. Thank you for sharing this video.
Wow an ad for development tools in your vid, thats saying something.
This is so cool, props
What up buddy Long x no se en Yannic God Bless You brother
This is generally useful python knowledge. Thanks for sharing!!!!
I'm now researching all the pages I looked at and I'm determining which models I downloaded are considered safe, which need to be purged, and which have a good amount of likes/downloads but are still pickles and still may need to be purged...
I don't think there is any model worth risking viruses for, but there are a few models that I'll be disappointed to not have still... Nonetheless, it must be done!
I get very paranoid over hackers and what not since it is so stressful having your stuff get hacked...
Hopefully everyone updates everything to safetensors (and hopefully safetensors are safe... I feel like if 'safe' is in the name it better be bloody safe...)
Thanks a lot...I learn so much from your videos
Is this really a danger of "open source", rather than just trusting unverified code? (in which case if its open source, at least its easier to verify?)
Wow...Bravo man!
Wow, never though about this issue! Seems like a good reason to implement __reduce__ method in a way it does some funny (but not harmful) thing like loading funny image on the screen or loud noise)
9:44 let me put my two cents on that, I was tinkering with raw CNN code and tensorflow uses hdf5 as a serializable. So, I did a mistake by not taking the input_size of the image in the ConV2D **kwargs. You know what it saved as usual. but when it comes to load_model it gave an error pointing the error specifically for me!! Yeah, hdf5 too evaluates the file while loading!!
Some things that can be done is not loading models as administrators and maybe doing it only in contained environments like docker or VMs…
"I feel like at some point, hugging face is going to be just full of features that they implemented because I did something stupid"
Hey, I'd take Yannic breaking my code over an actual black hat hacker any day :D
stay on tune with what you want to learn, just because it's hard now, doesn't an it's impossible. It's all about ntal mindset and
Very interesting, should have crossed my mind, but didn't!
It's no ";" SQL injection, but it's pretty cool.
Thank you
This looks like the anti-cheese model of security. Bubble up the warning until someone forgets about it. Instead they should ensure that models are scanned, there is a safe way to use pickle and torch mandates that method.
Pressing F12 in VSVode to jump to the definition of some class or function never ends.
Perhaps use VScode and read the docs to have docs in VScode like the website looks because it's all based on docstrings.
So it's impossible to rely on the super().__reduce__()?
How is there not a way of locking eval and exec down in these environments?!
Is this also an issue when loading .h5 files or only for models saved as .bin?
Hey Yannic, I am trying to work on a challenge I believe is based around this video. Is there a way I could share it with you and ask a few question but to see if I am implementing it correctly?
Sounds strange to me place in one sentence: hidden and open source! :-D
What is the music at the end?
When are you gonna do eye reveal :)
Thank you in advance
Is there a way to sandbox pickle initially so as to detect executable code or malware?
Would virtualization like running off a docker image protect you from this?
Any kind of sandboxing limits what arbitrary Python code can access in the local file system; but e.g. the malicious datafile's code could read whatever is in your local data directory and exfiltrate it with network calls.
love your videos
Thanks for the video. Still, I did not get the point "How the malicious code got onto my computer". So, you save with a "patched version of save", but I still cannot execute it locally or why should python automatically download it; do you point to it via url like in a javascript attack?
Usually, making things run on other people's machine is quite hard
Will there be a library to unpickle with reduced administrative rights?
Are PyPI, Tensorflow hub, Pytorch hub just as dangerous as HF?
Pickle trick!
Excellent
This is why we can't have nice things.
Do you work at meta? If so, then what is it like working with Yann LeCun?
thanks bro
very interesting
Great information that everybody should know.
But why would you make the github project? This would just make the exploit (which I agree is already easy) that much easier to be used. It just invites more people to exploit it and I see no use for the github project.
Could also just do eval('__import__("webbrowser")....')
Welp hacking people has never been this easy
Turns out, people found solutions for this. Your demonstration now errors out during the load on the United version of KoboldAI (Only takes GPT models, so had to test with a GPT version of your example). Curious if people will be able to bypass it.
Hey dude could you give me a bit more details about it? (Working on the Open Assistant project)
Huh?
"open source is dangerous"
*turns around*
"here look at the source code and see how dangerous it is"
😕
me getting feelings of a hacker
Ah yes my friend my fellow awareness giga chad. Once we make everyone aware, one of them will find the solution for us :)
pandas has a built in json that doesn't have the pickle security holes going on, cmon now.
How is your learning progression? I'm tNice tutorialnking about getting the Signature version soon, people like you are really persuading to give
Does anyone have experience with realESRGAN? It’s a popular image upscaler often used for AI art like Stable Diffusion. It involves a .pth which uses PyTorch and Pickle. I don’t know whether the source is trustworthy and how safe it is to run
Well there is a thing we can do... don't use pickle ! Any file format that is only used to store data and doesn't contain executable code is fit to solve this security issue.
dont use torch.load?
@@benjaming.8368 No, use a different saving protocol instead of pickle (torch.load calls the pickle module by default). Use one that doesn't execute arbitrary code on load.
Any low level function with such a non descriptive name such as pickle I'm always weary of.
pandas has a built in json loader that doesn't have the pickle exploits, cmon now.
Dynamic typing and its consequences have been a disaster for the human race.
You can't run these models on a container can you? Because of the GPU requirements... Well this sucks
Malwarebytes and nod 32? not recognize it?
Mbam is weaker than windows defender
@@xbon1
Mbam = malwarebytes?
what about nod32?
@@search620 mbam = malwarebytes yes. Windows Defender beats out everything except BitDefender right now, but BitDefender is only a bit better. AV research tests have shown that all you need nowadays is Windows Defender and common sense for the most part.
Maybe they should just not use pickle...?
"Does thinking", whilst this could be used for bad it could be used for good. Sigh. If only people leaned towards 'good' by default.
Java has a far superior serialization subsystem.
I never liked pickle..
So now that you've shown the whole world how to upload malware to hugging-face, do you think you can do enough good improving security as the harm you've done, releasing this code that helps anyone upload any malware they like with zero understanding or effort?
Wow I think you took away exactly the opposite of what he was trying to explain.
The only reason to prevent misuse is by spreading awareness. If you don't talk about vulnerabilities then they will fly under the hood for a longer period of time and more people will fall victim to truly malicious code.
I very much doubt that anyone with serious malicious intent would use his awareness-spreading repository. They would instead write something that's much better concealed and hidden but exploits the same vulnerabilities presented here.
@@PeterSeres nonsense. No one reads through the code when they download from hugging-face. Any exploit is as hidden as any other.
Security through obscurity is no security at all. If you think threat actors weren't already looking at this vector before he shared this, I don't know what to say to you.
Not reporting problems because you're scared of malicious people is how you get endemic vulnerabilities that are exploited for many years, sometimes even decades. It was responsible to spread awareness, and by construing it as "[doing] harm", the narrative presented is the same one people who don't want vulnerabilities patched use.
@@Kai-K no one forced him to release his code to the public, but he did!
And since hugging face would have to replace an entire module to fix this, they probably won't.
So now explain to me how we're safer.