Why I Switched from Python to Rust for AI Deployment
ฝัง
- เผยแพร่เมื่อ 12 ก.พ. 2024
- In this video, I share my journey of deploying the same AI project with both Rust and Python, highlighting the challenges, successes, and performance differences. I also compare The performance difference between Polars and Pandas.
🤔 Key Points:
* Polars VS Pandas
* Speed comparison between Rust VS Python
* Scalability comparison between Rust VS Python
So what's the conclusion? Is it ready for ML or datascience production systems
I wouldn't say so. Python still has more libraries for ML and is easier to develop. I would use Rust for only deployment and only in the condition that Python is too slow for your use case.
@@codeinajiffy gotcha. Have you tried the candle crate by huggingface. They say it is Rust alternative to Pytorch. You should give it a shot. May be make a video on that if you like it.
@@zendr0 That sounds super interesting. I will consider it in my next videos if I find it to be a useful.
It is. But very much "it depends".
Rust isn't just "faster to run" (less CPU and less memory, which the video dies not go into).
It's above all stricter and thus protects you against many mistakes and bugs. Not all, but many.
While that may even slow down a developer in the initial days or hours, it starts paying back within weeks. And seeing that we typically do some POC or exploration at first, Python (or even bash) is probably a much better fit there.
But the moment we put something into production, make it a team project and/or start maintenance and continued development, the strictness of rust pays off. Often withind days.
I typically cludge together a PoC in python. Once/if I'm happy with the direction and setup, I'll throw out that hodgepodge of python, bash/sed/awk, JavaScript and whatnot.
And write it in Rust.
@@berkes Yeah so design in python and deploy in Rust seems really good
Now you can switch from Rust to Mojo and make new video about why I switched
Agree❤
I’m curious if in the Rust implementation of polars, did you use lazy execution or eager?
I used Lazy mostly to optimize the execution as much as possible. But yes you will need to understand lazy and eager to make this work fast.
I have been trying rust and the learning curve it steep . Unfortunately python has a huge ecosystem and u cant just jump it. Thats why i am looking carefully at Mojo which has similar memory management features to rust , the borrower checker and how referencing is done in rust. This can be a game changer. Mojo can bring in C perforance, Rust security and python versatility in one language
From what I can see, Mojo is going to clean up in this area in the long run. Rust isn’t going anywhere but it will clean up outside of AI and scientific computing tasks.
Right now, F# is more mature for AI and can play nicely in the existing ecosystem.
Anyway polar can run also in 1 thread, maybe you were using only the default which is multi-thread and that's bad for non intensive request
👏👍👍
What about Mojo? The language that was literally designed for AI from the ground up? Plus Mojo's speed could be tested too.
Yup, I will look into that very soon. It's just a new language that might not yet be ready for production, but we can give it a try.
@@codeinajiffy
Their benchmark was recently revealed a lie after someone uncompleted the binary. Turns out something shady is happening in their compiler that doesn't match their claims. Big disappointment.
Mojo is low-key trash.
@@houstonbova3136 Its better than Rust in every way for AI though.
@@vectoralphaAI Having done Data Science in a mix of professional and hobby for the last 5 years I’m going to have to hard disagree with that.
I strongly prefer just using Python for exploration and then subbing out to raw Rust when I need gains. But, I also understand Rust. Mojo rarely matches its performance in practice and reduces my ability to create effective FFIs or directly manage the data when needed.
when trying to learn rust polars functionality , each time I run the code the compilation time takes around 10 seconds!
any body knows how to reduce the time spent in compilation to make it ideal to learn and try polars functionality as a newbie ?
Are you doing debug builds or release builds? Debug builds will be faster you can also set your debug builds to be incremental. You can also use sccache to cache compiler output. You could change the default linker or send -Z threads=8 to the rustic compiler. That is an unstable option but it will break up the front-end compilation of each package into 8 slices. But a 10 sec compile time isn't that bad considering the performance gains you get.
Oh there is also cargo-watch, it will recompile your code when the file is saved. It also depends on how you are running/testing this code. Are you reading in your data from a file or stdin or are you hardcoding data in the source code?
running multi workers for the api will solve the concrent request issue ,
Nice suggestion, but multi-workers here also increased resource consumption by quite a bit. It was quite close to increasing k8s pods.
look at virtually any published benchmarking of pandas vs polars and polars is many times faster. Without seeing your code it's impossible to prove, but this was almost certainly a skill issue in terms of writing the rust. I am also extremely skeptical about your Golang conclusions, were you using goroutines?
as for your last slide, you can spawn threads in Python too
but, for all the newbies out there, alas you're unlikely to see almost any of these options. Much more likely to see something like Java, Kafka, Flink, or even Postgres with plugins. For applications that need super quick inference speed, very likely to see C or CPP
is there any git code link to check your recommendation project?
no github link because this was company code. I am thinking of doing another project that I can share.
Why not mojo language
I am considering trying it out next actually. It seems very interesting.
@@codeinajiffy 🥰🥰😝
Biggest advantage of using rust is Zero-Cost-Abstraction, For example `sumation()` function could be write as:
(1..=n).sum()
and it would still out performed.
The only thing that was stopping me from moving from python to something like c/c++ before was ergonomics, not that c++ can't be concise, but a lot of libraries seem to require verbose interfacing. Rust, like python, seems to be very concise, so it offers a best of both, in brevity and performance, that I always thought were mutually exclusive. I was never under the illusion that python was efficient, it just had great syntax, which made prototyping a joy. It feels like I can prototype in rust same as I used to in python, and then get the proper implementation for free at the end.
All that said, wasn't the GIL getting removed and couldn't you use multiprocessing.pool?
I didn't try multiprocessing.pool but that is a good suggestion. It might not be consistent between requests, but I will try it out.
Now you can switch to Mojo
Well, I didn't learn anything from the video, but it did add yet another data point confirming what I already knew. Python makes it immensely difficult to write efficient code and requires external libraries written in better languages to achieve any degree of speed and general usefulness. Rust is basically a more obtuse C++ that nearly everyone in the future will use because the propaganda machine surrounding it is exceedingly effective. I know that a lot of people will claim that Rust is easier to learn, but since you can write your code in C++ exactly the same way as in Rust to achieve the same goal the same way and it will be equally as fast, the ease of use of C++ for simple projects shouldn't be understated. If you write idiomatic modern C++, it's as safe as Rust.
Sad but true reality.
"it's as safe as Rust", sorry but that's an absurd statement if you knew what you're talking about
@@Codotaku There were some qualifiers on that, but if you missed them and all the rest of what I said, then you must be a Rust user.
@@anon_y_mousse Idiomatic C++ won't guarantee memory safety because it's unreasonable to expect devs not to make mistakes. Not to mention the safety you get out of the box with Rust, thits a fact. 100% idiomatic C++ 100% of the time?
@@1____-____1 If you write idiomatic C++, then the compiler will catch most errors. The more insidious errors that aren't logic errors can be caught by external tools like the open source and often used `valgrind`, which catches everything the compiler misses and more than `rustc` will catch. If you had learned to program 20 years ago when they were still teaching people how to actually program instead of merely use a tool, then you would know these things. So I feel sorry for anyone learning how to use tools now instead of how to actually solve problems like an actual programmer, but we get what we allow to exist instead of what should because we've slacked off and not done our job for future generations. Sorry, my generation's intransigence in doing our duty for your generation has royally screwed you.
please provide free resources for rust
I learned Rust from this video: th-cam.com/video/ygL_xcavzQ4/w-d-xo.html .
Then just jump on project and figure things out along the way. This should enough if you have some background in programming.
Where's the AI?
Only c++, rust is a copy of that, dont be naive
I agree. But it's quite better imo
At the end it’s all assembly. Why complicate it?
That means you even don't know C++
@@slip-shape994
How old R U?
U Tell me what c++ is?
C--
do you have discord ?
Not yet, I am thinking of creating one.