Make an AI using Rust! Matrices and Neural Networks Tutorial
ฝัง
- เผยแพร่เมื่อ 21 พ.ย. 2022
- * GitHub Repo *
github.com/mathletedev/rust-ml
* Contact *
Website: mathletedev.github.io/
GitHub: github.com/mathletedev/
Discord: / discord - วิทยาศาสตร์และเทคโนโลยี
I haven't found the same type and quality of content anywhere else on youtube! Keep it up!
I am so glad you are making a tutorial using Rust! All the other tutorials are using Python, with stuff like Numpy, so it becomes way too hard to translate them to Rust!
You just need the mathematical concepts to be able to implement it in any programming language
@@AbuAl7sn1 I mean... yeah... technically it is simple to translate any Turing Complete language into another Turing Complete language, but Python and Rust have completely different paradigms, so a Rust tutorial is nice.
Great video! Would love to see a series that builds on top of this. to the point where you have your own framework :)
Well explained and a good introduction into machine learning, good job!
Awesome example, specially explaining theory first and then the actual coding to implement the same. Great wrork
thanks for your work, i really like it
Awesome, very well explain. Please do more Rust content.
Thank you for providing this content
Love this, thank you
This is top 0.1% of content! Please keep up the good work.
But just noticed: did you confused dot multi with multiplication or did i :D (i'm not a native english speaker)
very good video, I thought I would watch it for 20 min now one hour later I am still here
now I am inspired to make something like this in rust.
Awesome, dude!
I really think you should implement the traits Add, Sub, and Mul on your Matrix struct rather than just implementing add, subtract and multiply methods in a standard impl block. The amount of work is almost identical (two extra lines of boilerplate for each trait), and what you get from it is that you can use +, -, and * with your matrices. You could also steal some other unused operator, like ^ or | or & or /, implement the relevant trait, and use that for scalar multiplication.
Yooo that's cool, I didn't realise you could do that in Rust. I'll keep that in mind for the future. Thanks for the suggestion!
@@mathletedev It's been a while since I used these traits myself, but I seem to recall there were some issues with the fact that the trait methods consume / take ownership of their arguments. So I believe you need to add references in strategic places. If you look at the documentation of one of the traits and the list of implementors, you will see & everywhere.
I do love that you are doing this, but it makes me chuckle a little bit because after a few minutes of explaining neural networks and matrices, your drawings look unhinged and are completely unreadable. It gave me serious flashbacks to my math professors at uni. ;)
LMAO
Amazing video.
Am I right in saying thought that the 3x2 matrix in the top right hand corner at 8:42 is actually a 2x3 matrix by mistake?
Yup 😂 Nice catch!
Is there a way to replace for loops with a vectorized calculation
I'm not quite sure what vectorised calculations are, could you explain? I'm not an expert on this, and for loops were just easier to implement 😅
Wow really enjoyed watching that!
Very underrated channel, keep it up + 1 for the algorithm XD
great content thou
any advice on how to save the network? my nodejs-developer brain says json, but I'm wondering if there are better ways (bc I have very little experience in Rust)
JSON is perfect! Serde is a pretty popular choice for serializing and deserializing data to write to files; here's an example on Stack Overflow: stackoverflow.com/questions/62771576/how-do-i-save-structured-data-to-file#62771895
@@mathletedev this is the fastest I've ever seen a youtuber reply to a comment yet ADHD brain managed to get it working with JSON already 😅 ty for confirming I didn't make a fool of myself by using it!
@@raphaelmorgan2307 Awesome!
Thank you for this video you beautiful beautiful man
Cool
up!
Nice video bro! Btw, are you Japanese? The accent is pretty similar :p
No haha, I'm Chinese. I've gotten that from people though 😄 Thanks for watching! And happy (late) New Years!
Tnaks, it works well!!! Can you update code with optimizers?
There's definitely a lot of ways to optimise it if I was smarter 😅. You can definitely try! I haven't touched this project in a while.
The easiest to do would probably to use intrinsics instead of normal operations. This would still be safe as we won't encounter infinity as a value. Make sure to build with "-Ctarget-cpu=native" and "--release". Using #[inline] on the important functions could also help. The aim is to make the compiler vectorize the code. So you would have to check the assembly for it. The matrix multiplication algorithm should also be optimized to minimize cache misses, the same with the other operations. And most importantly the program is currently single threaded. For performance it would have to be multi-threaded. This is something I have no idea how to do in an optimal way.
@@ferdynandkiepski5026 Thanks for the reply! This video was just proof-of-concept, as I am a complete newbie in terms of AI (I haven't even taken calculus in high school yet 😕). Compiling it to native CPU definitely sounds like a great way to boost performance, and multithreading was something I was looking into. I'm not too familiar with intrinsics; if you want to, you could fork the GitHub repo or make a pull request, I'd be happy to look it over!
Have a wonderful day!
52:12 x * (1 - x) isn't the derivative. It's f(x) * (1 - f(x)) where f(x) is the sigmoid function
Yeah, sorry if I was unclear; "x" in this case represents f(x), since it was the output of the forward passthrough. Hope that makes sense!
You should change the variable names to make it clear that's what you're doing. That also doesn't generalize well when you assume x is f(x) when writing the derivative function. There are plenty of other activation functions which don't have a nice derivative in terms of it's output. Softmax is an example of a function that would be hard to write an activation for using your method.@@mathletedev
How do I use the save function?
The save() function just takes a single argument in the form of a String, the save file's name. It'll save the network data into the file name at your current working directory (or you can specify an absolute path). To load a network back, simply call the load() function using the same file name.
From your experience, do you recommend learning rust for AI ML purposes?
Unless performance is an issue, I would generally stick to Python because of the huge AI community behind it. This video was intended more towards an exploration of Rust with an introduction to ML. Hope that helps!
@@mathletedevthanks for a great video, can you give a specific performance example that makes sense to use Rust? 😊
@@thanhquachable Sure! Most of the popular libraries for ML right now (such as TensorFlow) are written in C/C++ due to the low-level performance boost. Rust has the benefits of C/C++ performance, but I prefer Rust's memory management and general syntax.
For instance, if you're writing AI for a computationally intensive game where speed is crucial, Rust may have better performance than a dynamic language like Python.
Thanks for the question!
which os do u use?
NixOS, which is based on Linux
what’s your ide/configuration
Check out github.com/mathletedev/nix-home
Everything should be there! I'm using Neovim as my text editor, and its config is under src/nvim
Which zsh theme you use? By the way
I'm using the fish shell, which is similar to zsh: fishshell.com/
And I'm using Catppuccin for the theme: github.com/catppuccin/fish
If you're using zsh, you could take a look at this: github.com/catppuccin/zsh-fsh
Happy late Thanksgiving!
Thanks man :), you are awesome
Hey man, what's the installation command for it I have pasted fsh file in . config but the last command doesn't work
Fsh folder*
@@am_editx278 I'm not so sure... I've never used zsh before.
github.com/catppuccin/zsh-syntax-highlighting
You could try this link, sorry I can't be of much help!
You discord invite link has expired
discord.gg/UN2nfu3T2b
(The server's kind of dead lmao)
@@mathletedev can u hop on disc
WHAT IS UR COMPUTER SETUP?
I'm using NixOS and xmonad with a lot of configuration. Here's my dotfiles: github.com/mathletedev/nix-home
Feel free to steal code from there!
thx a lot for reply, and I really want to know are u using PC or laptop and what is its setup
@@ruiuniver4 Hope you're having a great day! I mostly use a PC (a random Dell Optiplex 7010 that I found on eBay), but I have a small laptop for school. By setup do you mean the programs I use, or just the equipment I use to record videos?
@@mathletedev oh I just simply mean your computer accessories like ur cpu, gpu and sth
@@ruiuniver4 Oh haha, I haven't really looked into that... GPUs are way too expensive, so I just have a 4 core CPU, 8GB RAM, and a 500GB hard drive (though I should probably get an SSD)
It works for the stuff I do 😂
The video is corrupted!
What do you mean?
@@mathletedev I don't think I have ever seen this but the video was super glitched out before, I can see the video fine now but it was the coloured bars on old televisions before.
Might be an issue with TH-cam itself, but glad to hear it's working now!