PSA Since this video was made, the Rustformers crate, which is used for interacting with language models, has been archived. You can still use Rustformers if you'd like to follow along directly with the video, but I'd recommend using an alternative if you can. Here are some alternative crates to consider, in roughly in ascending order by level of ML knowledge required by the consuming developer: Kalosm - floneum.com/kalosm/ Mistral rs - github.com/EricLBuehler/mistral.rs Candle - github.com/huggingface/candle/blob/main/candle-examples/examples/llama/main.rs
It's so cool you use Rust on backend and frontend the same time! Would be great to see more videos on using Rust to build full stack! And the bot itself is super fun!
Super interesting video! I love watching someone go trough a project small-ish but complete project. Would love to see more videos like this in future!
I’m just starting to code again. Last time I wrote code was in 1997 and was promptly discouraged as I am quite dyslexic. Not anymore with ChatGPT, holy heck has it been helpful for me! Watching projects from start to finish, and the methods for doing so is what I need to see and learn. =]
Your videos are some of the most timely, spot-on code tutorials available. It is as if I get an idea to do a project, and out of nowhere, you post a video showing me just exactly how to approach it. Really amazing stuff you produce.
At 37:49, you should probably be using Leptos' `For` component, not such a simple setup with iterators - otherwise you'll have reactivity issues, i.e. overrendering. EDIT: also at 39:30, yes - there is indeed a better way: not misusing `create_effect`. Just set the property and whenever the underlying signal updates, so will the property. ```rust let chat_div = create_node_ref::(); ... ``` From the docs: Effects are intended to run side-effects of the system, not to synchronize state within the system. In other words: don’t write to signals within effects, unless you’re coordinating with some other non-reactive side effect.
good point - I discovered the `for` component after doing this video. not sure if that's because it wasn't in the framework at the time I made this video, or if I just overlooked it.
Just too nice, two things I wanted to have a look at united: a Leptos example and how to access G. Gerganov's language models in Rust. So, thanks a lot! 🦀👍
The pattern you mentioned at 15:10 or so is very similar to the way things are done in PHP land via a package for the Laravel framework called Livewire. Essentially, two parts: A Livewire PHP component class, and a Blade (html templating engine) file that references it. So you might have class MyComponent extends Livewire { public string $someValue = "Test"; public function save() { // Do saving stuff here } } and then your HTML can just do stuff like and and you never have to write any of the transport logic.
this is the most disgustingly insane thing I've seen in a while. I will be trying trying both the llm integration and the all rust NoJS stack immediately
That was the rustiest video I've seen in a while! Thank you! Keep up the good work. Really like the content and presentation. Also a bit of offtopic. I've noticed that you've switched from neovim to doom emacs several months ago. Maybe you can share your experience? I'd be very interested.
thanks for the kind words! my primary reason for switching to doom emacs is really org mode and org-roam, so I can take notes and manage knowledge in the same place I'm writing code. To me that's very valuable. I did made a video primarily about org mode ("Why I Prefer Emacs over vim") but I would like to do a more in depth comparison between doom emacs and a Neovim configuration framework like NvChad (which I've also been very impressed with)
Would be cool, if the LLM itself is streaming, to use a streaming HTTP call, updating the bot's message every time the client receives a chunk of the message. This is also what ChatGPT does-it's not just fancy effects! That way you could even halt the response while it's still running mid sentence as well.
Funny you mention this - after the video I actually implemented what you’re describing using web sockets. It’s in the repository if you’d like to have a look 😎
@@codetothemoon Yeah, that's cool! Though I specifially meant with Server-Sent Events since it's only a one-way streaming connection that is needed, and so there would not be a need for change of protocol nor the overhead of the WebSocket connection on the client. Though, WebSockets works too!
yeah I can't remember why I went with WebSockets instead of Server-sent events. I think maybe to keep the conversation context handy for the next prompt from the user? which you could do without WebSockets but there wouldn't be a guarantee the next request will go to the same host. not sure.@@dealloc
Great stuff! But `unwrap_or_else(|err| panic!(...))` looks a bit silly to me, considering the behavior of unwrap is to panic with the error message (and expect(...) to panic with a custom error message).
thanks! Everything I've learned about Rust has come from making videos about it and doing little projects like the one in this video. But most of my knowledge of software development comes from my ~13 years in the industry prior to learning Rust 😎 As for projects that can elevate your skills, the best choice is always the project that you're most excited about. I don't think one specific type of project lends itself more to learning than another - I think for any given type of project there are are ways of approaching it that are more suited for a beginner and other approaches that an expert might take. Examples: make a game, make a webapp, create generative art, create a command line utility, etc. All of these are suitable for beginners and experts alike, though the approach might differ depending on the level of experience.
yeah I took a look quick look at TRPC - at first glance it appeared to be similar but not quite as concise as clean as what Leptos has. could be completely wrong about that - maybe I just need to do a deeper dive.
Can confirm I was able to get this working on my desktop. It's got a ryzen 5 so responses do take a bit longer, but just wanna say awesome guide and ty. Still trying to get my head around rust, and I don't use js/ts at all, so I just copied the tailwind stuff from the repo and it was fine. The rust error handler is such a joy. I just went step by step and fixed everything (I had lots of typos from typing by hand to get used to the syntax), and leptos built at the end just fine.
Hey there! This's awesome! I have been wanting to do try full-stack Rust but for some reason I cannot get Leptos to deploy properly online. Locally everything works but when deployed to any host (e.g Github pages or AWS), it always shows an empty page LOL I do have my backend running in Rust though via Actix Web :D
While there are robust web frameworks like React or Angular available for frontend development, it's worth considering Rust only for backend tasks to ensure high performance.
holy, spent like 3 hours trying to figure out why mine didnt compile. turns out future_utils is not futures_util. *eye roll* there should be a law about naming your libraries too close to another. lol
15:00 honestly sounds a lot like some of the benefits u get from graphql but for a little less verbosity (depending on which gql libraries/frameworks u use)
Would the wizard-vicuna-13B-GGML model work if I switched it out with the current model you are using? Amazing video by the way taught me all the stuff needed to bootstrap a fullstack Rust project, also thanks a ton for the bonus tailwindcss integration part haha
Exceptional example showing a lot of interesting and useful things/techs! Instant subscriber! :D The `[profile.dev.package.ggml-sys]` specifies to compile the listed dependency package (or all if you write `[profile.dev.package."*"]`) with optimization level 3, the same that's used when compiling with --release: that way the dependencies (that don't change often unlike our code under active development) can __run__ fast while our code will still __compile__ faster and have debug data
I'd always been curious about this approach, and maybe it would be a better fit. it seems like it's well supported in the Elixir world - what do you think the best approach in Rust land would be? This crate? github.com/lunatic-solutions/submillisecond-live-view
It really depends on the nature of the website that you're looking to integrate it into. You could use this project as a starting point and built the rest of the application around it, or you could use this project as an example for building something similar into whatever tech stack your website currently uses.
Interesting tutorial! I will definitely give it a try. I have a question. Is it also possible to define your own applications, similar to OpenAi? Example: Can I link a specifications document (.pdf/.text file) and then make requests for it? And then the bot gives me an answer in the context of the document.
thanks! what you're describing is absolutely possible, and there are a couple approaches you can take 1. If you're using a language model with a large context window, you might be able to simply prefix your prompt with the entire contents of the relevant documents 2. Implement RAG (Retrieval augmented generaton) yourself - where semantic search is done on the documents in question and the pieces deemed most relevant to the prompt are extracted and included in the prompt to the language model. There are databases that provide semantic search capability like Pinecone and Nuclia that can help with this
you're asking what the advantages of using Rust on the frontend are? Type safety guarantees (because TypeScript is a superset of JavaScript, it can't really do such a thing), no garbage collection which yields better performance for certain use cases. Language isomorphism for projects where the backend is also in Rust (such as this one). Basically any reason you'd prefer using Rust over some compile-to-js language on the backend would likely apply to the frontend as well.
Noderefs allow you to pass a reference to a specific dom node around to say a function or closure, that way the closure can make changes and do things to the dom node and you don’t have to try to inline the statements as part of the dom
F#/websharper has this function call inside of api call. i tought this is coolest thing I ever seen but, in time this abstraction is to much for mind. I prefer standard http api call with http client(fetch,axios etc). Also last few month using htmx with maud(server side html generator :P) make things easier and more hypermedia style.
Idk if im doing something massively wrong but inference doesn't work for me. I think cpu bottleneck as the request doesn't complete at all. i have an older ryzen 5 but can i not use gpu for inference? i have a 3060ti.
I think there is a way to enable CUDA for inference (or Metal) but I hadn't looked into that yet prior to creating this video. Maybe hop on the Rustformers/llm Discord, I'm pretty sure CUDA is supported, and I'd think the 3060ti should be plenty to run Vicuna 7B
thanks - I took a look at tRPC and it looked close but not quite as clean as what Leptos gives you. I didn't spend too much time investigating though - maybe I just need to dig a little deeper
by "single file component" do you mean SPA? if so then the answer is yes. If you really mean single file component, it's completely up to you as the developer whether you want files to be 1:1 with components
this is achievable but not without a bit of effort. the biggest obstacle there is the limited input size of the models, so in most cases you can't just prefix your prompts with the entirety of the documents. Many work around this by doing a semantic search on the document base to prune out the parts most relevant to the prompt, then prepend those excerpts to the prompt that is given to the language model. The problem with this approach is that there isn't a guarantee that the semantic search will surface all of the context necessary to accurately respond to the prompt.
just took a quick look and it actually looks like I used nightly when I was making this video - that was a mistake, I had originally intended to use stable. Up until very recently, I believe Leptos required you to use nightly. However, the code in the GitHub repository does use stable, so take a look at that if you want to avoid nightly.
@@codetothemoon If you read the leptos readme, it is explained that nightly permit to use a simpler syntax. But nightly should not be used in a long term project. Thank you for your very instructive video. By curiosity, I have tried to generate it on Windows, but Visual studio 17 compiler fail with the C code of Rust former llm. I will try on my Linux computer or with WSL.
you seem to be a lot into rust and so am i, however i've never used lisp much and i presume you did since you are a emacs user. what comparison of the two would you make in term of workflow ?
Quick question, if I wanted to enable CUDA support where would I do that (just kind of a general direction thing). I went to the docs for Rustformers and *attempted* to try their implementation, however that didn't seem to do anything. This may just be a matter of "git-gud" for doc reading.
I actually had the same question, but in my case pertaining to metal support. I found instructions on how to enable it for the Rustformers command line tool, so my next step was going to be looking at the code for that to see what it's doing under the hood. Wasn't able to get there prior to making this video unfortunately :/
Good timing! I've just added CUDA/OpenCL support yesterday. Yeah, you're right, our docs on how to use GPU acceleration could use some work. I'll admit, documentation isn't really my strong suit.
I've got most of the leptos components working great but for some reason I can't seem to get Rustformers LLM tools to work with the LLM; I've snagged the same Vicuna model you linked. I've git cloned the entire repository, and I've tried targeting a single node from it, and I've tried targeting the directory that all the nodes live in Should I be zipping up the repo into a single file or what is the best way to connect the Rustformers/LLM tool to my actual LLM?
what do you mean by "targeting" in this context? Make sure to specify the path to the language model in the `.env` file in the root of the repository (by default it's set to the path on my system). More broadly, make sure you've followed all of the steps in the readme (I just updated it a bit yesterday) github.com/Me163/rusty_llama/blob/main/README.md
this is kind of something I'm actively researching. Fine tuning using your own documents is an option, but I don't believe Rustformers supports this so you'd have to use a different library and I'm not sure how effective it is. The other option is the "few-shot" approach, where you provide your documents inline with your prompts - this is an easier and more effective approach as long as your documents fit in the context length of the model, but that can very quickly become limiting. To overcome the context length limitation, another option is to put your documents in a vector database and use semantic search to find the parts most relevant for a particular prompt, then feed just those pieces into the prompt that goes to the LLM. I think this is currently the most common pattern for this use case - check out something like NucliaDB (open source), Pinecone (not open source), or Weaviate (open source)
I'm a huge fan of Svelte as well! Incredible framework. I took a cursory glance at krausest.github.io/js-framework-benchmark/2023/table_chrome_114.0.5735.90.html and I'd personally describe the performance difference between it, Leptos and Vue as negligible at the moment. I definitely wouldn't switch from Svelte to Leptos for the performance - it'd be for the ability to use Rust on the frontend.
check out this example if you haven't already: github.com/leptos-rs/leptos/tree/main/examples/tailwind re: axum, I'm actually not sure as I've never actually done this. maybe check the Leptos docs and if it's not there, ask on their Discord server - Greg is very active on there answering questions!
@@codetothemoon Update. I never got it to work. I'm having a can't find cc error and it's a bit of a pain to figure out on NixOS to the point I needed to stop. I was getting the model loaded and everything on the build I did. Unfortunately I couldn't get my version or your version. Working. Yours less than mine. Thanks for the video anyway. I'll have to come back later and try again sometime.
yeah that was my impression as well - it seems like it aims to automatically manage the contract between the client and server, but just a quick glance at the examples led me to believe that it wasn't quite as clean as what Leptos offers. I may need to do a deeper dive to see whether that is actually true...
I cannot compile the code on Windows 11 - both repository codes from the "video" branch and my hand-written variant gives the following error: error: linking with `link.exe` failed: exit code: 1120
The bottleneck is the language model inference, which uses a non trivial amount of compute power. By “relatively fast” on the MBP, I’m talking about 10 full seconds or so
Yeah this is a great video, but I also don't see the relevance of showing that FE performance chart here, That said, the video does go on to explain the other advantages to using a rust-based FE framework, including sharing type definitions across FE/BE, and that cool leptos feature where the FE can automatically (via macros, I think?) invoke a BE function to trigger an HTTP request.
@@mikejamesbelanger the main reason is that until recently, Rust frontend frameworks had performance profiles that were inferior to most of the popular JavaScript frameworks, so it was often cited as a good reason not to use Rust on the frontend. With these numbers it seems like that can't really be used as a reason anymore.
PSA Since this video was made, the Rustformers crate, which is used for interacting with language models, has been archived. You can still use Rustformers if you'd like to follow along directly with the video, but I'd recommend using an alternative if you can. Here are some alternative crates to consider, in roughly in ascending order by level of ML knowledge required by the consuming developer:
Kalosm - floneum.com/kalosm/
Mistral rs - github.com/EricLBuehler/mistral.rs
Candle - github.com/huggingface/candle/blob/main/candle-examples/examples/llama/main.rs
It's so cool you use Rust on backend and frontend the same time! Would be great to see more videos on using Rust to build full stack! And the bot itself is super fun!
nice, glad to hear this one worked for many folks - looks like I'll be making more!
@@codetothemoon Please do!
Super interesting video! I love watching someone go trough a project small-ish but complete project. Would love to see more videos like this in future!
I’m just starting to code again. Last time I wrote code was in 1997 and was promptly discouraged as I am quite dyslexic. Not anymore with ChatGPT, holy heck has it been helpful for me! Watching projects from start to finish, and the methods for doing so is what I need to see and learn. =]
Really happy you enjoyed it! Will definitely focus on making more like it!
Your videos are some of the most timely, spot-on code tutorials available. It is as if I get an idea to do a project, and out of nowhere, you post a video showing me just exactly how to approach it. Really amazing stuff you produce.
someone's got a crush... but is it maybe likely that folks are about building chatgpt clones now, perhaps? :)
@@adicide9070 bahaaha :D
nice! thank you for the kind words, and glad it aligned with something you already had in mind to build!
Absolutely love Leptos! Completely game changing for web development, and along with Tailwind makes webdev fun again! ^^
agree 💯
Rusty Bros... Get ready for another great ride
🚀
K
At 37:49, you should probably be using Leptos' `For` component, not such a simple setup with iterators - otherwise you'll have reactivity issues, i.e. overrendering.
EDIT: also at 39:30, yes - there is indeed a better way: not misusing `create_effect`. Just set the property and whenever the underlying signal updates, so will the property.
```rust
let chat_div = create_node_ref::();
...
```
From the docs:
Effects are intended to run side-effects of the system, not to synchronize state within the system. In other words: don’t write to signals within effects, unless you’re coordinating with some other non-reactive side effect.
good point - I discovered the `for` component after doing this video. not sure if that's because it wasn't in the framework at the time I made this video, or if I just overlooked it.
Great video! I typically use these videos as reference when I'm building in Rust. Adding timestamps would be a 10x improvement for me!
thanks, really happy you got value out of it! and thanks for the feedback!
15:10 that pattern reminds me of how tRPC works while using Typescript
Thanks, yeah I hadn't tried tRPC but a few others mentioned this as well! Good to know the pattern is available in the JavaScript ecosystem as well!
Just too nice, two things I wanted to have a look at united:
a Leptos example and
how to access G. Gerganov's language models in Rust.
So, thanks a lot! 🦀👍
thanks! glad you got something out of it!
Trpc i think gives you the same functionality in JavaScript land for not having to write fetches manually and just calling a function name.
The pattern you mentioned at 15:10 or so is very similar to the way things are done in PHP land via a package for the Laravel framework called Livewire. Essentially, two parts: A Livewire PHP component class, and a Blade (html templating engine) file that references it. So you might have
class MyComponent extends Livewire {
public string $someValue = "Test";
public function save() {
// Do saving stuff here
}
}
and then your HTML can just do stuff like
and
and you never have to write any of the transport logic.
this is the most disgustingly insane thing I've seen in a while. I will be trying trying both the llm integration and the all rust NoJS stack immediately
Great let us know how it goes! hoping to be able to build more stuff like this...
That was the rustiest video I've seen in a while! Thank you! Keep up the good work. Really like the content and presentation. Also a bit of offtopic. I've noticed that you've switched from neovim to doom emacs several months ago. Maybe you can share your experience? I'd be very interested.
thanks for the kind words! my primary reason for switching to doom emacs is really org mode and org-roam, so I can take notes and manage knowledge in the same place I'm writing code. To me that's very valuable. I did made a video primarily about org mode ("Why I Prefer Emacs over vim") but I would like to do a more in depth comparison between doom emacs and a Neovim configuration framework like NvChad (which I've also been very impressed with)
Would be cool, if the LLM itself is streaming, to use a streaming HTTP call, updating the bot's message every time the client receives a chunk of the message. This is also what ChatGPT does-it's not just fancy effects! That way you could even halt the response while it's still running mid sentence as well.
Funny you mention this - after the video I actually implemented what you’re describing using web sockets. It’s in the repository if you’d like to have a look 😎
@@codetothemoon Yeah, that's cool! Though I specifially meant with Server-Sent Events since it's only a one-way streaming connection that is needed, and so there would not be a need for change of protocol nor the overhead of the WebSocket connection on the client. Though, WebSockets works too!
yeah I can't remember why I went with WebSockets instead of Server-sent events. I think maybe to keep the conversation context handy for the next prompt from the user? which you could do without WebSockets but there wouldn't be a guarantee the next request will go to the same host. not sure.@@dealloc
Great stuff!
But `unwrap_or_else(|err| panic!(...))` looks a bit silly to me, considering the behavior of unwrap is to panic with the error message (and expect(...) to panic with a custom error message).
Cool to see Doom Emacs! The Monokai themes are nice but I prefer the default Doom One theme.
I agree the default one is pretty good. it's fun to switch things up now and then, maybe I'll give it another shot
This is awesome. Can you release some content on how you got to the level of rust you are today? What projects do you think could elevate a novice??
thanks! Everything I've learned about Rust has come from making videos about it and doing little projects like the one in this video. But most of my knowledge of software development comes from my ~13 years in the industry prior to learning Rust 😎
As for projects that can elevate your skills, the best choice is always the project that you're most excited about. I don't think one specific type of project lends itself more to learning than another - I think for any given type of project there are are ways of approaching it that are more suited for a beginner and other approaches that an expert might take.
Examples: make a game, make a webapp, create generative art, create a command line utility, etc. All of these are suitable for beginners and experts alike, though the approach might differ depending on the level of experience.
Could you please also teach how to host/deploy this project on a live server? Thanks
Thanks for this! You always bring out quality content.
glad you got something out of it! 😎
Great work as always! Love your teaching style. Any plans on paid courses?
thank you! I do have something along those lines in the works...
Interesting! Would you be coming with another tutorial with the latest leptos?
I'm definitely going to cover the latest leptos in some form, not sure if it will be with this particular project
server actions are what you're looking for at 14:00
The Leptos Server Functions pattern is effectively TRPC
yeah I took a look quick look at TRPC - at first glance it appeared to be similar but not quite as concise as clean as what Leptos has. could be completely wrong about that - maybe I just need to do a deeper dive.
Can confirm I was able to get this working on my desktop. It's got a ryzen 5 so responses do take a bit longer, but just wanna say awesome guide and ty.
Still trying to get my head around rust, and I don't use js/ts at all, so I just copied the tailwind stuff from the repo and it was fine.
The rust error handler is such a joy. I just went step by step and fixed everything (I had lots of typos from typing by hand to get used to the syntax), and leptos built at the end just fine.
Hey there! This's awesome! I have been wanting to do try full-stack Rust but for some reason I cannot get Leptos to deploy properly online. Locally everything works but when deployed to any host (e.g Github pages or AWS), it always shows an empty page LOL
I do have my backend running in Rust though via Actix Web :D
hey, man thanks for this awesome video, I wanted to ask, which hardware specifications does your laptop has
thanks for watching! It's an M1 MacbookPro with an M1 Max and 32GB RAM
While there are robust web frameworks like React or Angular available for frontend development, it's worth considering Rust only for backend tasks to ensure high performance.
supabase and featherjs offer similar front/back integration for ease of dev…
thanks for making this vid!!
ohh nice thanks for letting me know, I haven't used either of those - maybe I should check them out. Thanks for watching, glad you liked it!
holy, spent like 3 hours trying to figure out why mine didnt compile. turns out future_utils is not futures_util. *eye roll* there should be a law about naming your libraries too close to another. lol
15:00 honestly sounds a lot like some of the benefits u get from graphql but for a little less verbosity (depending on which gql libraries/frameworks u use)
Great job man! What font are you using in your editor?
What is your neovim plugin for the file manager? I've looked over to your github but I didn't find your dotfiles. Great video btw!
Ah nvm you are using emacs
yep doom emacs! of which I'm a big fan
Loved the video, please more Leptos!
correct me if im wrong, but i think you can call the backend functions in a similar way in the T3 stack can you not? Also really great video.
Trpc which is what t3 uses for that l, is about as close to serverfn as your gonna get as far as I know when doing client and server in typescript
never tried T3 myself, been hearing great things about it though. I may have to give it a spin!
Would the wizard-vicuna-13B-GGML model work if I switched it out with the current model you are using? Amazing video by the way taught me all the stuff needed to bootstrap a fullstack Rust project, also thanks a ton for the bonus tailwindcss integration part haha
i think the closest thing i can think of to this in the JS world is NextJS server actions which is still beta
nice, I may have to check those out!
I also had to install "npm install -g sass" to make "cargo leptos build" work
oh interesting thanks for pointing this out!
Great video! What window manager do you use?
thank you! I use Amethyst - I think it's fantastic, but it's also the only window manager I've really used for more than a trivial amount of time 😎
@@codetothemoon what is the code editor? thanks
@@guisoliven8022 doom emacs 😎
Exceptional example showing a lot of interesting and useful things/techs! Instant subscriber! :D
The `[profile.dev.package.ggml-sys]` specifies to compile the listed dependency package (or all if you write `[profile.dev.package."*"]`) with optimization level 3, the same that's used when compiling with --release: that way the dependencies (that don't change often unlike our code under active development) can __run__ fast while our code will still __compile__ faster and have debug data
thank you, very happy to have you onboard!
and thanks for clarifying the meaning of `[profile.dev.package.ggml-sys]`!
Remix uses loaders to simplify (abstract away) endpoints that are used by the client
Thank you for this, what is the plugin you're using to find your imports?
Super useful, thank you!
nice, really happy you got something out of it!
Why not liveview? And yes rust and elixir have liveview that can do SPA without JS(only for websocket) render everything at the the backend
I'd always been curious about this approach, and maybe it would be a better fit. it seems like it's well supported in the Elixir world - what do you think the best approach in Rust land would be? This crate? github.com/lunatic-solutions/submillisecond-live-view
N00b question: If I wanted to integrate such a chatbot into a website, what would be the best approach?😮
It really depends on the nature of the website that you're looking to integrate it into. You could use this project as a starting point and built the rest of the application around it, or you could use this project as an example for building something similar into whatever tech stack your website currently uses.
Nice! what theme you using in Emacs?
thanks! doom-monokai-pro
A link to your dotfiles please :) specifically vim, that's some awesome status line!
it's actually doom emacs! theme is doom-monokai-pro. other than that, everything else is pretty much what you get out of the box.
@@codetothemoon I haven't noticed the window title :facepalm: Thanks :) Great content (regardless of the inferior editor preference).
this is awesome!! excited to learn from this vid
nice! hope you get some value out of it!
Qwik has a pattern like Leptos's server APIs!
I would like a Tailwind video! :D
nice thanks for the feedback! will try to make one...
@@codetothemoon nice!! It would be cool to see it integrated with Leptos. Thanks for all your work and for taking the time to answer me :))))
Interesting tutorial! I will definitely give it a try. I have a question. Is it also possible to define your own applications, similar to OpenAi? Example: Can I link a specifications document (.pdf/.text file) and then make requests for it? And then the bot gives me an answer in the context of the document.
thanks! what you're describing is absolutely possible, and there are a couple approaches you can take
1. If you're using a language model with a large context window, you might be able to simply prefix your prompt with the entire contents of the relevant documents
2. Implement RAG (Retrieval augmented generaton) yourself - where semantic search is done on the documents in question and the pieces deemed most relevant to the prompt are extracted and included in the prompt to the language model. There are databases that provide semantic search capability like Pinecone and Nuclia that can help with this
I just do a insta like when I see a video about rust with more than 10min.
can you do a video that, which areas rust is helpfull in using in react app like that.
you're asking what the advantages of using Rust on the frontend are? Type safety guarantees (because TypeScript is a superset of JavaScript, it can't really do such a thing), no garbage collection which yields better performance for certain use cases. Language isomorphism for projects where the backend is also in Rust (such as this one). Basically any reason you'd prefer using Rust over some compile-to-js language on the backend would likely apply to the frontend as well.
Very easy to understand! I still didn't understand the whole node_ref thing. Why is that necessary?
Noderefs allow you to pass a reference to a specific dom node around to say a function or closure, that way the closure can make changes and do things to the dom node and you don’t have to try to inline the statements as part of the dom
F#/websharper has this function call inside of api call. i tought this is coolest thing I ever seen but, in time this abstraction is to much for mind. I prefer standard http api call with http client(fetch,axios etc). Also last few month using htmx with maud(server side html generator :P) make things easier and more hypermedia style.
Wow, great job!
thanks, glad you liked it!
15:13 you might be looking for trpc? It kind of accomplishes the same thing.
Some checkpoints in the video would help.
Idk if im doing something massively wrong but inference doesn't work for me. I think cpu bottleneck as the request doesn't complete at all. i have an older ryzen 5 but can i not use gpu for inference? i have a 3060ti.
I think there is a way to enable CUDA for inference (or Metal) but I hadn't looked into that yet prior to creating this video. Maybe hop on the Rustformers/llm Discord, I'm pretty sure CUDA is supported, and I'd think the 3060ti should be plenty to run Vicuna 7B
tRPC does the typesafe server side api thing you want.
thanks - I took a look at tRPC and it looked close but not quite as clean as what Leptos gives you. I didn't spend too much time investigating though - maybe I just need to dig a little deeper
This is awesome, thanks
nice, glad you got something out of it!
Great Thanks for Sharing.
thanks for watching!
Is leptos or dioxus a single file component? I come from svelte and blazor and single file components are a lot cleaner from my experience
by "single file component" do you mean SPA? if so then the answer is yes. If you really mean single file component, it's completely up to you as the developer whether you want files to be 1:1 with components
It's possible to use this lib to load personal documents and query them?
this is achievable but not without a bit of effort. the biggest obstacle there is the limited input size of the models, so in most cases you can't just prefix your prompts with the entirety of the documents. Many work around this by doing a semantic search on the document base to prune out the parts most relevant to the prompt, then prepend those excerpts to the prompt that is given to the language model. The problem with this approach is that there isn't a guarantee that the semantic search will surface all of the context necessary to accurately respond to the prompt.
Are you using leptos with stable or nightly rust ?
just took a quick look and it actually looks like I used nightly when I was making this video - that was a mistake, I had originally intended to use stable. Up until very recently, I believe Leptos required you to use nightly.
However, the code in the GitHub repository does use stable, so take a look at that if you want to avoid nightly.
@@codetothemoon If you read the leptos readme, it is explained that nightly permit to use a simpler syntax. But nightly should not be used in a long term project.
Thank you for your very instructive video. By curiosity, I have tried to generate it on Windows, but Visual studio 17 compiler fail with the C code of Rust former llm. I will try on my Linux computer or with WSL.
you seem to be a lot into rust and so am i, however i've never used lisp much and i presume you did since you are a emacs user.
what comparison of the two would you make in term of workflow ?
I think the project is a bit high level for me for now, will need to learn th rust syntax and come back.
Super awesome! ❤
thanks, glad you liked it!
Yes, actix!
😎
Quick question, if I wanted to enable CUDA support where would I do that (just kind of a general direction thing). I went to the docs for Rustformers and *attempted* to try their implementation, however that didn't seem to do anything. This may just be a matter of "git-gud" for doc reading.
I actually had the same question, but in my case pertaining to metal support. I found instructions on how to enable it for the Rustformers command line tool, so my next step was going to be looking at the code for that to see what it's doing under the hood. Wasn't able to get there prior to making this video unfortunately :/
Good timing! I've just added CUDA/OpenCL support yesterday. Yeah, you're right, our docs on how to use GPU acceleration could use some work. I'll admit, documentation isn't really my strong suit.
@@lukaskreussel669 nice, thanks Lukas! Really appreciate your efforts!
How about dioxus?
it's extremely high on my list of stuff to check out!
I've got most of the leptos components working great
but for some reason I can't seem to get Rustformers LLM tools to work with the LLM; I've snagged the same Vicuna model you linked.
I've git cloned the entire repository, and I've tried targeting a single node from it, and I've tried targeting the directory that all the nodes live in
Should I be zipping up the repo into a single file or what is the best way to connect the Rustformers/LLM tool to my actual LLM?
what do you mean by "targeting" in this context? Make sure to specify the path to the language model in the `.env` file in the root of the repository (by default it's set to the path on my system). More broadly, make sure you've followed all of the steps in the readme (I just updated it a bit yesterday) github.com/Me163/rusty_llama/blob/main/README.md
Thanks!
thank you thank you so much for the support!
Is it possible to train this LLM to use some own database/documents?
Nice content! thks for share
this is kind of something I'm actively researching. Fine tuning using your own documents is an option, but I don't believe Rustformers supports this so you'd have to use a different library and I'm not sure how effective it is. The other option is the "few-shot" approach, where you provide your documents inline with your prompts - this is an easier and more effective approach as long as your documents fit in the context length of the model, but that can very quickly become limiting. To overcome the context length limitation, another option is to put your documents in a vector database and use semantic search to find the parts most relevant for a particular prompt, then feed just those pieces into the prompt that goes to the LLM. I think this is currently the most common pattern for this use case - check out something like NucliaDB (open source), Pinecone (not open source), or Weaviate (open source)
i'm a big fan of svelte, so poor to hear it slower than vue and leptos???
I'm a huge fan of Svelte as well! Incredible framework. I took a cursory glance at krausest.github.io/js-framework-benchmark/2023/table_chrome_114.0.5735.90.html and I'd personally describe the performance difference between it, Leptos and Vue as negligible at the moment. I definitely wouldn't switch from Svelte to Leptos for the performance - it'd be for the ability to use Rust on the frontend.
Wow. Thanks.
glad you got something out of it!
Awesome tutorial!
How to get a Tailwind CSS, for some reason it doesn't work on my end. And how to swap over Axum from Actix? Thank you
check out this example if you haven't already: github.com/leptos-rs/leptos/tree/main/examples/tailwind
re: axum, I'm actually not sure as I've never actually done this. maybe check the Leptos docs and if it's not there, ask on their Discord server - Greg is very active on there answering questions!
why did you use the loop inj 37:11?
trpc implment a similar pattern for api requests
Thanks!!
thanks for watching!
Danke!
thank you so much for the support!!! 🙏
Great video!
thank you!
What font is that?
I think it was likely Fira Code, if not it would be Monaco. I made the switch from the latter to the former around the time of this video
you got a sub
nice! very happy to have you onboard!
Now create_action needs to take a function which returns a future
I hate my NixOS right now. I can't figure out how to get cargo-leptos to install. I'd probably already have it done with ubuntu or similar
I posted this. Right when I figured out the build. Hallelujah!
nice glad you got it sorted out!
@@codetothemoon gotta love a 45 minutes video turns into a 4 hour setup 😅. It's all good though. At least I learned how to get past it for next time.
@@codetothemoon Update. I never got it to work. I'm having a can't find cc error and it's a bit of a pain to figure out on NixOS to the point I needed to stop.
I was getting the model loaded and everything on the build I did. Unfortunately I couldn't get my version or your version. Working.
Yours less than mine.
Thanks for the video anyway. I'll have to come back later and try again sometime.
@@Script_Alchemist can you paste the exact error you're getting? or hop on the Discord and I can try to help there! discord.gg/2G35Sp6kpQ
what code editor do you use. not like neovim, thanks
doom emacs!
@@codetothemoon thanks
Doom Emacs
much
Is rust good to learn vs go
Really depends heavily on both what you enjoy working on and your personal taste in programming languages
i think trpc is similar but not completely
yeah that was my impression as well - it seems like it aims to automatically manage the contract between the client and server, but just a quick glance at the examples led me to believe that it wasn't quite as clean as what Leptos offers. I may need to do a deeper dive to see whether that is actually true...
Guys, why am I getting error: linking with cc failed exit status 1 when I run cargo leptos watch...
use theme in video ?
doom-monokai-pro!
I cannot compile the code on Windows 11 - both repository codes from the "video" branch and my hand-written variant gives the following error: error: linking with `link.exe` failed: exit code: 1120
error occurred: Failed to find tool. Is `gcc.exe` installed?
yikes - I don't have any experience building this on Windows, is that what you're on?
@@codetothemoon Yes
Great
thank you, glad you got something out of it!
It was really a great learning@@codetothemoon
oo which font do you use!?
Monaco
@@780Chris thanks :D
I don't think you're going to have any FE perf issues for what your building haha "
An input field, button and some message box's
The bottleneck is the language model inference, which uses a non trivial amount of compute power. By “relatively fast” on the MBP, I’m talking about 10 full seconds or so
@@codetothemoon The LLM for sure! I just meant when you where talking about different FE framework pros / cons 🚀
Yeah this is a great video, but I also don't see the relevance of showing that FE performance chart here, That said, the video does go on to explain the other advantages to using a rust-based FE framework, including sharing type definitions across FE/BE, and that cool leptos feature where the FE can automatically (via macros, I think?) invoke a BE function to trigger an HTTP request.
@@mikejamesbelanger the main reason is that until recently, Rust frontend frameworks had performance profiles that were inferior to most of the popular JavaScript frameworks, so it was often cited as a good reason not to use Rust on the frontend. With these numbers it seems like that can't really be used as a reason anymore.
@@codetothemoon Oh I see. Fair enough.
7:24 I think naming things is nobody’s forte 😅
hah thanks, glad I'm not the only one! 😎
смотрю через перевод нейросетью, классно