I used Rust pyo3 to expose some validation tool logic our team wrote in Rust (well, me) to our Python scripts. A useful tip is you can specify a stub pyi file at the top level (alongside your toml files) to provide better type hint and editor support when you import them. Maturin picks these up when it builds the package as long as it's called [module name].pyi
Great video! A follow-up discussing/showing real life problems that can solved with combing Python and Rust together. Boosting performance, increasing safety etc. Personally I'm currently working a lot with async, websockets and JSON messages in python, any significant and potential performance rewards would be great to see.
th-cam.com/video/D88UAjyFpAY/w-d-xo.html it doesn't seem to need to be that much involved as you can experience the benefits already for seemingly simple uses cases like dict insertion.
Side note: In a way, it's actually Poetry which is non-compliant here. The standard format for pyproject.toml nowadays is defined by PEP 621, which seems to be used by pyo3, while Poetry still clings to their own legacy format.
That's right! It always bothered me with poetry. According to Maturin project page they comply with PEP 621. But TBF Arjan actually has incorrectly specified dependencies, they should be an array of strings. Maybe Maturin doesn't care and still reads it correctly but that's why Python VS Code extension is showing an error
Hey this is awesome. I actually picked up Rust recently and it’s been going great. Great timing to see this a month later when I’ve gotten a little more experienced. Great video!
Really great video and easy to follow guide! When i used pyo3 i decided to make a proc macro the help make wrappers for struct impls so you could basically just add it for python support with little to no hassle. Can definitely say that there are ALOT of limitations on pyo3 that aren't directly visible, maybe a video on these things would be awesome and help some newer people to pyo3!
Hey Arjan, I'm curious if you see yourself writing rust as a day-to-day? what areas would you apply it to? love the rise of rust tutorials on your channel :)
@@LieberLois I just used Rust for an embedded project using an E-Ink display on an ESP32, but I've also used Bevy to make games, Leptos for websites, and Clap to create command-line tools.
Is there some direct benefit of defining your python classes as Rust structs? I can understand that doing number crunching would be useful but not sure about the defining of objects (only). Would be interesting to hear your thoughts on what gives the most “bang for the buck” when combining the languages like this. Thanks for interesting videos.
Looks promising. But how about the interaction between Python and Rust regarding memory management? 🤔 Maybe show an example where you parse a numpy array and perform some transformation on it.
Ive been slowly incorporating rust through polars which uses pyo3. Im pretty convinced at this point replacing pandas for polars is just better. The api handles the same, its faster and infers column types (this is huge). The fact that these apis just copy sql operations and facilitate working on dataframes make the switch to polars all the more obvious.
Nice video thanks, I used maturin but didn't know about rustimport. Python and Rust are my favorites language, I also think that they fit well together.
I'm just using the standard `difflib.SequenceMatcher` and for sentences and snippets of code it was just fast enough. But atm I'm trying to throw some 1000s lines of JSON against it and want to display it in some Qt text fields. It's already MUCH faster in Python 3.11 and 12 but might still be a couple seconds! I can imagine that this would be a breeze to do with a little Rust plugin!
@ArjanCodes does the original module written in Rust can still be used as a rust library as well or now after importing py03 it can only be used by Python? Also does it give the same performance benefit of rust when we ran it as a module from Python?
Wow, great tutorial, I always wanted to rewrite the core routines in rust for my project "pyTermTk" (a TUI Lib) What about cross compilation? Does Maturing take care of it? One of the things that prevented me to use rust or c/c++ is that I don't know if I would be able to publish a package compatible with different architectures.
I can't speak for C/C++ but in Rust you publish your package as a "crate" which can then be freely used by other devs. It will be compiled for their architecture when they download and use it (usually with Cargo).
Love it .. going back to c after learning python from you.. today i learned how to cheat c compiler. I was coding for luhn check for three days. Today i made a dumb change in the code by suppying an argument which i assigned to something but not used. The compiler wasnt happy and error parameter not in use. So what i did was i just make a variable int error=0; and then error += s; The error was gone programe compiked 🎉🎉🎉🎉🎉. Tomorroelw i will work on how to get rid of the error variable 😂😂😂😂3
Hmm, I been wanting to make a bulk texture converter and possibly downscaler settings too and this might be useful to expose rusts image and image_dds crates to python to do the conversion in rust itself and all the front end code in PyQt. Maybe create the entire backend code in rust and just front end gui be python.
I want to experiment a bit with rust together with python, the reason is I've been using python for a long while, but I don't have any compiled language fully under control. I toyed a bit with c++ and pybind11, but maybe rust is more modern? I guess I'm here because of curiosity :P
This adds a significant amount of complication to a python project. What I did not see is any measurement at all of how much speed up inserting less modules could achieve! How do I know if I should bother with all this, if I have no idea how much speed up it might get me?
Yes, but with certain caveats. Moving objects back and fourth into python can be "slow"-ish. The more/bigger chunks of work you can shift over the boundary into rust, the bigger the benefits :)
It looks great! But as an amateur python programmer I don't think I am going to use it. I don't know anything about rust so I have to learn a new language from scratch. When I have done that, I have to understand when it is usefull to to use rust. So I stick by python, if you don't mind....🙂😇
We want the two decade experiance of " How would I do it if i start to learn software engineering today " video, Am sure the internet will expolde if you make such video because of the experiance you have.
I agree that in general this would be the better approach, problem is mojo is missing a lot of features and is atm just a proof of concept, but not a full language.
Fancy words like memory safety, superman performance blah blah are delusional. The lucky enough people who work at netflix level software - you know, the 0.0 something percent - need to worry about memory safety and are normally not watching introduction videos on TH-cam. The rest of us - mental model shift is a necessary consideration for our day to day jobs.
@@squishy-tomatoIf you write something in rust vs python, you need less resources, if you are in a cloud, could save you some money and the less power we use, the less carbon we put into the air. ;-)
I am just not sold on rust... Its likely that AI will just be able to read your C code and find menory vulnerabilities, use after free, buffer overflows... When that happens, i think rust will just fade a lot...
If you haven't already, give Rust a try! It's a lot easier (and more fun) writing code that just doesn't have those memory vulnerabilities in the first place.
👷 Join the FREE Code Diagnosis Workshop to help you review code more effectively using my 3-Factor Diagnosis Framework: www.arjancodes.com/diagnosis
I used Rust pyo3 to expose some validation tool logic our team wrote in Rust (well, me) to our Python scripts. A useful tip is you can specify a stub pyi file at the top level (alongside your toml files) to provide better type hint and editor support when you import them. Maturin picks these up when it builds the package as long as it's called [module name].pyi
I was going to ask about stub files, glad to know this is supported!
Nice! Gonna push my team to write some Rust around our python projects I guess
Babe, wake up. Arjan uploaded a new tutorial
Fair enough
This is mind boggling I'm enthralled. The publishing seems so incredibly easy wow
Great video! A follow-up discussing/showing real life problems that can solved with combing Python and Rust together. Boosting performance, increasing safety etc. Personally I'm currently working a lot with async, websockets and JSON messages in python, any significant and potential performance rewards would be great to see.
th-cam.com/video/D88UAjyFpAY/w-d-xo.html it doesn't seem to need to be that much involved as you can experience the benefits already for seemingly simple uses cases like dict insertion.
This is awesome! I’ve been waiting for a video like this. Thanks for the great overview 🎉
Im glad you liked the content!
Side note: In a way, it's actually Poetry which is non-compliant here. The standard format for pyproject.toml nowadays is defined by PEP 621, which seems to be used by pyo3, while Poetry still clings to their own legacy format.
poproject.yaml is the new generic format, so Arjan could've simply disabled Poetry plugin and look for a generic one for pyproject files.
That's right! It always bothered me with poetry. According to Maturin project page they comply with PEP 621. But TBF Arjan actually has incorrectly specified dependencies, they should be an array of strings. Maybe Maturin doesn't care and still reads it correctly but that's why Python VS Code extension is showing an error
Hey this is awesome. I actually picked up Rust recently and it’s been going great. Great timing to see this a month later when I’ve gotten a little more experienced. Great video!
Glad you enjoyed the video!
Really great video and easy to follow guide! When i used pyo3 i decided to make a proc macro the help make wrappers for struct impls so you could basically just add it for python support with little to no hassle. Can definitely say that there are ALOT of limitations on pyo3 that aren't directly visible, maybe a video on these things would be awesome and help some newer people to pyo3!
Thanks so much! Very clear! Will definitely give it a try!!
Glad the video helped you!
Hey Arjan, I'm curious if you see yourself writing rust as a day-to-day? what areas would you apply it to? love the rise of rust tutorials on your channel :)
I feel the same way. With Go or similar languages it's easy to find use cases, but Rust ... not so much
@@LieberLois I just used Rust for an embedded project using an E-Ink display on an ESP32, but I've also used Bevy to make games, Leptos for websites, and Clap to create command-line tools.
Is there some direct benefit of defining your python classes as Rust structs? I can understand that doing number crunching would be useful but not sure about the defining of objects (only). Would be interesting to hear your thoughts on what gives the most “bang for the buck” when combining the languages like this.
Thanks for interesting videos.
This is GREAT! That's why I love Python. It's so easy to integrate with other stuff to overcome limitations of Python. Thank you for this video
Glad you enjoyed t!
If rust is more efficient, I might put some bigger/slow projects into rust and see if I can trim the time down. Thanks for the info!
You’re welcome!
Looks promising. But how about the interaction between Python and Rust regarding memory management? 🤔
Maybe show an example where you parse a numpy array and perform some transformation on it.
Ive been slowly incorporating rust through polars which uses pyo3. Im pretty convinced at this point replacing pandas for polars is just better. The api handles the same, its faster and infers column types (this is huge). The fact that these apis just copy sql operations and facilitate working on dataframes make the switch to polars all the more obvious.
this is very nicely explained! Question how do we add python typehints to rust functions?
Nice video thanks, I used maturin but didn't know about rustimport. Python and Rust are my favorites language, I also think that they fit well together.
Can you make one for Go and Python too?
How do cross-platform builds work when you publish the project to pypi?
I'm just using the standard `difflib.SequenceMatcher` and for sentences and snippets of code it was just fast enough. But atm I'm trying to throw some 1000s lines of JSON against it and want to display it in some Qt text fields. It's already MUCH faster in Python 3.11 and 12 but might still be a couple seconds! I can imagine that this would be a breeze to do with a little Rust plugin!
@ArjanCodes does the original module written in Rust can still be used as a rust library as well or now after importing py03 it can only be used by Python?
Also does it give the same performance benefit of rust when we ran it as a module from Python?
Wondering if you have looked at Robyn not just a fast async python web framework but with a Rust core.
A way forward for Python/Rust integration?
Have you looked into Julia Programming Language? What do you think about the multiple dispatch?
Amazing video as usual! :)
What do you use to have auto-completion on your terminal? :D
this rustimport tools looks perfect for simple scientific calculations that need a speedup in the middle of a simple script or notebook.
Yaaay! :) Rustimport is pretty interesting for cross platform support... publishing compiled binaries for many platforms is a bit hectic
Wow, great tutorial, I always wanted to rewrite the core routines in rust for my project "pyTermTk" (a TUI Lib)
What about cross compilation?
Does Maturing take care of it?
One of the things that prevented me to use rust or c/c++ is that I don't know if I would be able to publish a package compatible with different architectures.
I can't speak for C/C++ but in Rust you publish your package as a "crate" which can then be freely used by other devs. It will be compiled for their architecture when they download and use it (usually with Cargo).
Love it .. going back to c after learning python from you.. today i learned how to cheat c compiler.
I was coding for luhn check for three days. Today i made a dumb change in the code by suppying an argument which i assigned to something but not used. The compiler wasnt happy and error parameter not in use. So what i did was i just make a variable int error=0; and then error += s;
The error was gone programe compiked 🎉🎉🎉🎉🎉. Tomorroelw i will work on how to get rid of the error variable 😂😂😂😂3
Hmm, I been wanting to make a bulk texture converter and possibly downscaler settings too and this might be useful to expose rusts image and image_dds crates to python to do the conversion in rust itself and all the front end code in PyQt. Maybe create the entire backend code in rust and just front end gui be python.
what about Cython? i think is less complex and better in performance terms
what is the advantage over c++ modules
Please do more rust tutorials 🙏
I want to experiment a bit with rust together with python, the reason is I've been using python for a long while, but I don't have any compiled language fully under control. I toyed a bit with c++ and pybind11, but maybe rust is more modern? I guess I'm here because of curiosity :P
This adds a significant amount of complication to a python project. What I did not see is any measurement at all of how much speed up inserting less modules could achieve! How do I know if I should bother with all this, if I have no idea how much speed up it might get me?
Are you going to create a rust programming course?
Is it possible to run python code from rust as well?
Thank you....
does this give python the performance of rust?
Yes, but with certain caveats. Moving objects back and fourth into python can be "slow"-ish. The more/bigger chunks of work you can shift over the boundary into rust, the bigger the benefits :)
no
It looks great! But as an amateur python programmer I don't think I am going to use it. I don't know anything about rust so I have to learn a new language from scratch. When I have done that, I have to understand when it is usefull to to use rust. So I stick by python, if you don't mind....🙂😇
Julia combines both in one language
I'd be curious to see how Cython compares to this
I will use this for sure. Do you need to think about accidentally spawning new processes?
Whatever they do, name the result "Rython". Silly as that name may sound, it sure as hell beats programming in "Pust".
I would call it: Ruthon
a litlle more credit for Rust 🤣😂😁
this would create a dependency on an extra toolkit i.e. rust
Time to have chat gpt write me rust modules and scripts to import into my projects, perfect
If C++ can be pronounced as Sepples
C++-ians can also be called Sepplesians
I was thinking how you can reverse the rusty python pun. Like pythony rust or pythoned rust but it won`t make any sense
Make armorers great again
I haven’t tried rust yet. But I might give it a go. But,I haven’t tried go yet but I will wait and see. I haven’t tried c yet.
Nah, I'll wait for Mojo.
the thing i hate about rust is that you can only get blockchain job with it
We want the two decade experiance of " How would I do it if i start to learn software engineering today " video, Am sure the internet will expolde if you make such video because of the experiance you have.
You should start when you’re 10
Nice but not convinced. It you really want Python + Performance - you are better off giving Mojo language a go..
I agree that in general this would be the better approach, problem is mojo is missing a lot of features and is atm just a proof of concept, but not a full language.
Also you should stop thinking in just performance, rust offers memory safety.
Fancy words like memory safety, superman performance blah blah are delusional. The lucky enough people who work at netflix level software - you know, the 0.0 something percent - need to worry about memory safety and are normally not watching introduction videos on TH-cam. The rest of us - mental model shift is a necessary consideration for our day to day jobs.
@@squishy-tomatoIf you write something in rust vs python, you need less resources, if you are in a cloud, could save you some money and the less power we use, the less carbon we put into the air. ;-)
I’d rather just call Rust libraries (like polars) from Python 😂😂
Read "maturin init" in a Bri'ish accent for extra lols.
mojo is an existing superset of python which is already 50% faster than rust
More like the worst of both languages
Why would we use rust when we can do all of this in Python?
Arjan simply does not explain things clearly or coherently. People just like his voice.
I am just not sold on rust... Its likely that AI will just be able to read your C code and find menory vulnerabilities, use after free, buffer overflows... When that happens, i think rust will just fade a lot...
If you haven't already, give Rust a try! It's a lot easier (and more fun) writing code that just doesn't have those memory vulnerabilities in the first place.
It smells bad to me if it uses a non compliant pyproject.toml
Why not use another name?
Babe, wake up. Arjan uploaded a new tutorial