Decouple your NestJS code with this technique!
ฝัง
- เผยแพร่เมื่อ 14 ต.ค. 2024
- In this video we talk about how we can use the EventEmitter module to easily decouple our NestJS code. We can also take this approach and apply it to any NodeJS / ExpressJS code.
Been using this pattern for a while in my Nest projects, nice to see someone else promoting it. It makes the application so much more modular and maintainable. A couple of things - I always recommend wrapping all your OnEvent decorated method in a try/catch if doing anything async. When errors are thrown, they bubble up to the EE2 try/catch block so you may lose immediate context as to where it occurred, and for synchronous emittng, errors will bubble up to where you emitted from, resulting in 500 errors in your API. Also, just so people know, you can use property promotion in TypeScript, where you just add readonly, private or public to the contructor args, then you don't need to declare them or have constructor logic.
Got served this video and this seems like the perfect solution for a pain point I was having running a lot of services that use the same core data and triggers.
Simply amazing way to decouple things in a safe way
I really like this technique - I personally use it for handling Stripe webhook events :)
Just curious - So I guess you have a NestJS app that listens to Stripe webhook events. Do you then publish events exactly the same events as the once you receive from Stripe? And then all of your different services in Stripe would act accordingly? E.g. usersService would update user object to be paid etc.
@@Sebastian71732 I have dedicated controller for webhook with standard validation of the event - if everything is valid then I just emitAsync the event. Previously I had a lot of injections and big swtich/case and it was painful.
In most cases I consider my event handler as something similar to route handler in controller - it sometimes transforms data from event and then invokes specific service methods to reduce amount of Stripe specific abstractions inside my business logic - in the same way I want to avoid http specific abstraction in services when using controller route handlers.
Thank you for this!
For the same effect, I'm actually doing a Facade pattern where multiple services are being called whenever "something" happens.
Subbed.
Thank you Marius you are always on top 👍. Please post videos on all those event-based mechanisms you mentioned; I really would love to see them. Thank you.
Fantastic guide! Explained the "why" very well here. Immediate sub.
Thank you!
yeah I also liked it. You didnt just go with the extra line of code, but I can already see the benefit for extra effort within the first 5 Minutes.
Thanks!
Super nice! I mostly think about this event driven architecture to be used between completely different services with e.g. Redis as you mentioned. But nice to get a reminder to also use it when necessary between modules/services with Nest :)
it makes the code more declarative, which i really like coming from Angular 😀
Excellent way to decouple things. Thanks for the great tool
That is very cool and another way to decouple the code it's using ports and adapters, it will continue using the imports of all the services but it will became modular like you video
Are you talking about hexagonal architecture?
@@mariusespejo yeah ports and adapters are inside the hex arch but you van use without all the concepts of hex arch
nice video! Anyway, how to to test it? previously I had bunch of code in Service, so I just had to mock the service and ensure all the business logic are handled
Interesting, I'm going to use that for cache invalidation 👍. Unfortunately, it looks like we're loosing type safety, which is definitely a dealbreaker in more professional setups.
Very good and clean way of explain. Very good tutorial.
useful and concise as always helpful and interesting .
counting the seconds for your next event-based videos.
Holy fuck! This is what I was looking for! I have been using rxjs to emit "events" so far, but with the problem of not being able to get a return. This is what I needed!
Also: looks like a perfect way to start disconnecting code before migrating to microservices. After you want microservices, you will simply swap from this to rabbitMq. Can't wait to implement it in my side-project!
Local decoupling as a setup for potential future microservices, totally! That’s a great thought 👍🏼
Beautiful video, this pattern is very cool, I just have one question: how would you handle the case in which the services are interdependent? Like service 2 needs as input the output of service one. In general: can you only use this pattern when the listener functions are strictly independent one another or you can control the execution flow? (I have to say I really appreciate your nestjs videos, i'm looking forward to level up my nest code)
I would say this pattern is best for things that are not directly dependent. You can still use it of course even with the dependencies but it’s not going to offer much value
What a good way to teach things. Thannks alot.
Very nice video! I've just subscribed! Nest is an amazing framework.
Super cool video! What do you use to record and edit?
Great video! Right now we are handling some big mailing events where we have to get information about tasks, submissions, permissions, email service and also log into the db every single email sent or error. We considered using event emitters but what differentiate it to execute in a async function just before the response being sent? The event emitter is just calling a function, why not calling it directly? Thanks, love your videos, hope you do more!
The point is decoupling. Calling directly means being directly dependent on the module/service that contains those methods. If that’s fine for your use case then go for it, just call directly.
With the eventemitter you now would just be dispatching events, what happens after the caller no longer “knows”
Thank you for sharing, it will help differentiate software engineers from the rest of the AI world.
need more videos like this, do one video with kafka nestjs, it should be fully module based like this.
The video is great, but the pattern is insane. The amount of work that people go through to avoid calling a function and using its return value is impressive. It's all functions, pals. Keep it simple.
Obviously if you used this on everything it would be overkill. The point is to understand its purpose and use it in a scenario where decoupling is important. It’s not about avoiding calling a function, that’s an oversimplification. Although perhaps I didn’t explain the why behind using it well enough
Thank you, this was beneficial!
The technique is really cool when you have to sort out a big monolite. What would you do with the contracts like VideoCreatedEvent, when there will be many, put them in a separate folder/module?
I’d just put them in their own events folder, within each module folder. Similar to what you might do with DTOs, but it’s really up to you
This seems a bit of a rare use case in the real world that's really better than just using a localized list of Promises where you have strong typing and directly understandable context rather than scattered subscribers causing potential spaghetti code.
Having said that, I know of at least one real world use case for this - Logging to AWS CloudWatch where you really want to buffer and batch. It's entirely fire and forget - and you want to latch and batch log messages so you're not invoking AWS CloudWatch APIs a gazillion times. You want to batch up a list of log messages and log them once per time-division - maybe every 100ms so your application isn't spamming the CloudWatch APIs.
Yup agreed, like with anything, use it when it makes sense. It’s not meant to be the only or primary pattern to be used
I also feel the same, in the erp system, I got a bunch of logic resides in service. Yes it coupled with several injectable-classes, but the upsides is I can unit test and verify all the huge business-flow.
Instead, how do we ensure all the subscriber are not missing a thing in such cases by using above CQRS/ Event emitter?
anyway nice video though, the pace is perfect.
Nice video, Also what's the best practise to manage transactions and cascading in such cases, Especially when you wish to transition from a monolithic to microservices in future ?
Learned something new!
Really cool implementation, and thank you for sharing it! However, you demonstrated the decoupling in a Service class. As far as I understand, we should avoid having dependencies from other modules inside Service classes. If we were to move this code sample (without EventEmitter) to a Controller class, would it still be fine design-wise?
Not entirely sure what you mean. There’s no “rule” that you should avoid having dependencies from other modules inside service classes. Nest revolves around the idea of dependency injection. Services are where your business logic should live, not in controller classes, so although yes you could have your controller call multiple different services I personally would not do it that way.
Cool Technique! Thanks!
thank you so much haha perfect video
Does this approach get rid of potential memory leak?
Great video. Actually watched it several times. I'm trying to work out how I can type the response from emitAsync? Any ideas?
I don’t know that you can, you might just need to type cast so it’s not an any
Ye you're right. I got thrown slightly by it returning an array even though I only had one listener. Thanks 😁
Very good content thank you Marius
Thanks for video! How do you track the exact order of events across a whole application not only in a service borders?
Might be a stupid answer but it should be in the same order of code/module registration. You could also name your event keys specifically to include/symbolize desired order. Ideally, personally I would only use these things in a situation where the order of side effects isn’t really important.
Yes. please, make a video on cache with redis.
Thanks
Thanks man! Great content
Your editor looks awesome
Any chance you can share
Icon Theme and Font ❤
Cascadia code! I’m not sure I’m using anything for icons, it might have come with my theme?
Thank you so much ❤
Really helpful
Thank you for this , really helpful (y)
Hi, i really love your videos. But each time i use Nestjs (3 years of exp using it), i just noticed that is not fully typesafe and make you write unsafe code a lot of the time (usually decorators fault). But you seems not having a problem with that. Can you explain why you don't be upset about that lack of typesafety? Do the Nodejs ecosystem need a new backend framework?
If you’re coming from express, Nestjs is already a massive improvement in terms of architecture and type safety. I personally have not had too many problems. Why not just add types where you need it? There are already a ton of frameworks, but not many have successfully integrated into the express ecosystem like Nest has
@@mariusespejo Yeah, i know. When i started Nestjs was amazing because taught me about architecture my code in a better way. But after years i noticed too many problems with the architecture and typesafety. And yeah, i can add types when i need it, but some times it turn into Typescript is lie to me. I noticed this when i started using tRPC.
Example 1: Decorators are liars
@Get()
someExample(@Body() body:string){}
// This should fail, because you can't recieve a body in a Get endpoint, but Nestjs Arquitecture allow it.
Example 2: Class Validator and ClassTransform is bad and also a liar because decorators
class ExampleDto(){
@Transform((value) => Array(value))
@IsInt()
name: string
}
//Why it allow me to validate a string value as a int? or the other way around. Worst things happen with Class Transform is added
Example 3: Middlewares/Guards/Interceptor/Pipes are unsafe
This is simple, they don't give you typesafety at all. Also i don't know why in some places the Nestjs team decided to return any instead of unkown, because any is a very bad practice that will ended in many bugs.
Example 4: Clases are just bad for building statless APIs.
This is controversial, but i noticed that Clases are just bad for building statless APIs, because they usually force you to use decorators per example, and force you to separate your DTO from the controller function. In my opinión, this only work well when the APIs are simple and small, but in big applications they turn a mess very quickly. I suggest function based APIs and divide endpoints in folders when they grow too much, having DTO, Controller and Service function in the same file for that endpoint.
These are some of the things i noticed, they are so many. But this let me thinking that maybe we need a new backend only framework for Nodejs, one with better typesafety. And better intergration with new technologies and libraries like Zod. I'm currently using Nestjs in my work, but 10 times a day i think about this 😅 . What's your opinion about this. Sorry about my English
OscarL Jimenez
1. You CAN have a body on a GET http request. There is nothing stopping you from doing that. However for REST a body on a GET just doesn’t have any semantic meaning. So that’s not Nest’s problem
2. That’s class-validator and class-transformer …. Not NestJS. It’s the suggested schema validator but you can use whatever you want in pipes.
3. At the middleware layer I don’t know how any framework could possibly predict ahead of time what the contents of a request is… for example until the body is parsed, how do you know what’s in there? You can add a schema check, nothing is stopping you from doing that, but I don’t know how any solution could get full type safety here, did you have ideas?
4. Classes in Nest are not used for state, they’re just used to allow decorators. If you know javascript well, you would know that classes are just syntactic sugar. Underneath they are just functions… classes in Nest don’t inherently determine if your API is stateless or not, that’s an engineering problem not a framework problem.
what you’re describing of having dto/controller/service in the same file is exactly the spaghetti mess that Nest pushes you away from. Dividing them only when it grows means your code will lack conventions and consistency. Sometimes business logic is in the endpoint file sometimes it’s not. You also need to document your conventions well otherwise, it definitely will turn into spaghetti. That’s not going to work well for a large team/project. Nest actually solves that problem by having conventions built in to how you use it
But yes I do generally agree that it’s not perfect. But I have personally found it to be the most productive choice with the best feature set for standalone APIs. Trpc is great if you’re using Next.
@@mariusespejo But there is too many not Nestjs problem but they encourge them in the docs, why is that? Class Validator and Class Transform is first class support in Nestjs but they are a roo to many bugs. How can a framework be entrerpise ready when you encourge so many bugs all of the time, i don't understand. About the spagueti code, the same thing every person said about React breaking separation of concerns in frontend world and they prove that was a mistake. I bet this is a mistake again in backend code, separate DTO, Controller and Services always it turn to having to open 3 o 4 files always when you want to debug something leading to spagueti code and 1000 lines or more lines of code files. And Yeah, Middlwares are hard to make typesafe. But 1, Nestjs return any instead of unknow, 2. there is default values that you at least can give some type hint (all optional maybe to avoid bugs), 3, you can always give a better way to share the context of the app. In tRPC you can't use a middleware with the wrong context. 4, yeah, typesafe middleware for multiple endpoints is imposible or at least very hard, i'm usually more concern about using a middleware in wrong context. Support for Zod? or any other new library? They just frozend and they can't add anything new to fix their buggs. I know is hard, but is time to move on and build the future of backend framework in Nodejs
Most of the problems you’re calling out are non-major. If you don’t think it’s enterprise ready why use it? To expect that open source projects are perfect with zero bugs is simply unrealistic.
Class-validator for example is already good for 90% of the typical use cases. If you’re mad about it giving you wrong types, then you can: 1. cast it to the correct types, 2. Contribute to the project and submit a PR 3. Write your own solution from scratch. Or 4. Use something else. Be the change that you want to see.
great content again.
Thank you!
Goat
Great! Easy pease! 🙂👌
hi, what font were you using for your IDE in this video? it looks so good
Im away from the computer right now but I believe it’s called Cascadia Code, it’s free!
Instead of the array from the promise I get an object:
[
Immediate {
_idleNext: null,
_idlePrev: null,
_onImmediate: [Function (anonymous)],
_argv: undefined,
_destroyed: false,
[Symbol(refed)]: true,
[Symbol(asyncId)]: 159,
[Symbol(triggerId)]: 0
}
]
I can't seem to resolve the promise whatever I do 🥲, didn't find a similar issue on google, I wonder what dumb mistake I'm making
Edit: I removed {async: true} options in OnEvent and I could retrieve the values, but I want to know why I wouldn't retrieve them with async option :/
probably you forgot the await syntax
Hi and thanks again for the amazing tutorial. Would you use CQRS without DDD or would you stick to 3 layers architecture? I like CQRS even without DDD, but today I was playing with handling events in CQRS and without DDD it was a little bit odd (commands and queries are fine though). Just the requirement to use AggregateRoot limits what you can return, and I had to put AggregateRoot on the EntitySchema itself, which is probably not the best thing to do
That topic is a little bit out of context for this video. I hope it’s clear that this isn’t necessarily the solution that you would use to do CQRS. Nest has a specific module for that as I’m sure you probably know. In general the answer to would I use “X” is always: it depends… context matters. CQRS I imagine is only useful in a scenario that you have very strict scalability requirements and you’re willing to make the tradeoff of having a more complex architecture. So I can’t give a simple answer for “would you”. Hope that makes sense. If you need input on your chosen architecture I suggest asking in other forums where you can provide details of your requirements and be able to get multiple responses from others
It's really nice! But I think it might be difficult to mock up the subscribers, are there any better approaches for testing in this situation? Besides that, can I have the name of font used in your vscode? haha
Cascadia Mono, it’s free!
I have a question. The service layer should always follow the SRP from SOLID, the observer pattern it's really good but what about using the services in the controller (which is the interface adapter layer and has more scope) and calling each method o requirement in that place, what do you think about that?
In general your business logic should be in the service layer. Yes you can move these to the controller if you wish. However in my opinion controller methods should be as lean as possible, at most they should care about routing, http status codes, etc. The rest are meant to be handled by decorators (e.g. guards, pipes, etc.) and services.
Also the point behind SRP is to make it so that your method only has one reason to change. The whole point of this pattern is to decouple external concerns so that it no longer is the responsibility of the service method. I don’t think simply emitting events counts as “more responsibility” or “more reason to change”. It actually allows you to add more observers without changing the method. I can totally see others having the opposite opinion, but it’s important to understand that the point behind SRP isn’t about literally just doing one thing. It’s about having one purpose/concern. If you are dispatching events relating to the same concern and not something completely unrelated then it should be fine
Hi, thanks for the video. What is the difference between using EventEmitter from Nest and events from CQRS? Where would you use each? Also, if your Emit events are on different files/folders and services how do you know which event will be emitted first? thanks
EventEmitter is just a simple observer pattern basically for when you want a basic internal pub/sub mechanism. CQRS is like an entire architecture pattern that can be event-based. I suggest reading more about it.
As for how you know the order, you won’t inherently know. It depends on the order of registration, so that means take into account the order of your module registration, then providers, then the methods of those providers which are listeners. I’d probably suggest only use it for things where the order really doesn’t matter too much.
yap, how to also handle transaction rollback?
I would love to see how you could solve this by using AWS queues
Thank you
You’re welcome!
Thanks for your video, can you share your vscode theme
what if you have to call different functions from different services? How can you know the order in which they will be executed?
You technically can predict the order based on the order in which the handlers are registered, but that is kind of missing the point. If you really need specific ordering of calls, this might not be the right pattern for you. This is best in situations where you can broadcast an event, and the broadcaster can sort of be “dumb” to what happens next
I was using this EventEmitter functionality to perform operations based on some events, but what happens if there's an error on any of those operations? how can this error be handled?
If the event is emitted asynchronously, the emit won't throw an error, the error will simply be logged on the terminal.
yeah, I also curious about how we should rollback some transactions
@@MuhammadFahreza all this events functionality of NestJS led me to dive deeper into the fundamentals of CQRS and DDD architecture and understanding that the event emitter is not meant for transactional operations.
@@jrs_devs I see. I agree with that. If the event emitter throws error, do they have failover / retry mechanism ?
@@MuhammadFahreza to make the emitter throw an error, you have to set up that option in the handler, by default won't throw any errors.
And I suppose the retry functionality will need to be implemented according to your needs
I think a point that is being missed here is that you probably should only use this in scenarios where you “broadcast and move on”. e.g. fully decouple from whatever the subscribers might do (including failing). if you’re trying to handle the error of your subscribers this might not be the correct mechanism for you, because you have an obvious dependency that you can’t remove
2:50 i Just noticed when you say subscribe in a video it highlights the subscribe button of the channel, guess I have to subscribe now
Hi Marius, I was recently trying to setup AdminJS with a Nestjs project, but since AdminJS uses ESM and nest uses CommonJS, I was having issues importing packages into the app module. Do you've any hints on how I can resolve this issues. Thanks
It’s in the docs docs.adminjs.co/installation/plugins/nest
I’m not familiar with adminis though honestly, but it sounds like you might be better off doing it with express
the events subscribers are based on order within the same file, but what about across different services? does it comply with the order that the services are registered in the main module?
From my understanding yes
What's the difference with mediator pattern. Seems a lot like it
I think this would be closer to the observer pattern. But the mediator pattern can be implemented by using the observer pattern I think so I can see why they would seem similar
which vs code theme are you using?
I didnt get exactly when should use eventEmitters in nestjs? , I read so much documentation in google but nothing give me the idea, why should ı use eventEmitter except sending a request to service, so replacing all service injection with the eventEmitter is the best practise?
“Replace all” … no absolutely not. Don’t perceive it as black and white. Like I mention in the video if you have multiple things that need to subscribe to something and can be safely decoupled/decomposed, that’s a good use case for it. Like with everything else the answer is “it depends”
Could this be a problem if the listener class is not working well?
what do you mean by not working well? If your code is the problem then it has nothing to do with the approach
WHat's the font that you're using in your IDE?
Thanks
Cascadia Mono, it’s free!
@@mariusespejo Thanks man!
What about error handling and in theory rollbacks if one of subscribers failed?
Catch the error in the handlers themselves, from there it’s up to you how you handle it. Or if you’re using emitAsync, you can wrap that in a try/catch block and handle it in the publisher. I think also if an error is thrown in the listener the library will also emit an “error” event. Nest also has its own concept of exception filters which can be setup to catch specific errors
What if I want another app to listen for an event being published on a different app, would be possible to do that?
I think eventemitters is really mostly meant for delegation from within your one app. If you want to do pub/sub across different apps you need to introduce infrastructure that can do that like Redis. Then you can have both apps connected to it, one publishes events the other subscribes/consumes
Badly needing help... I wanna know why my server doesn't continue serving a user while another user is making a request. it's like queued rather than being asynchronous. How do I fix this? I am already using async functions in the service, which are mostly CRUD operations with a database working with json data objects.
Are you talking about when using eventemitters?
@@mariusespejo In my case it's just a route from my controller receiving a request from a user, which then saves (multiple objects) to a database. When this is happening, any thing else doesn't get processed by the server (even requests from another user). I tried looking it up and thought that maybe it's the event loop getting blocked? What are some good solutions for this? Thanks for your reply!
So it’s not related to this video? Sorry but it’s not simple to debug in a comment, I suggest asking in stackoverflow
wow
I think it's great to use it to architect your app but I wonder why it's not that common.
Maybe it's because It's not transactional if your app crashed you would lose all the published events.
It’s just a basic observer pattern, it’s not really meant for more sophisticated event-driven architectures. Most people probably would just call their functions directly, decoupling like this does have the tradeoff of making parts of your code harder to follow, because now you have to track down listeners
Hi Marius, thanks for that technique, nice.
Since you're familiar with NestJS, typescript and TypeORM, do you mind sharing your thought about FoalTS? Thank you!
Not as familiar with that but I don’t see a good reason to even consider it over nest. Less features, smaller community. Not seeing anything special about it, just my take
@@mariusespejo thanks for such quick response, love it!
I see your points, nice angle.
why not using rxjs instead of event emitter ?
Because EventEmitter is built into node, it’s a pattern/api that most people will easily pickup vs having to learn how to do the equivalent in a reactive programming library.
Rxjs would fit better in a scenario where you’re actually working with streams. Here we’re simply doing a really basic synchronous event pub/sub to decouple the code, rxjs would be overkill
@@mariusespejo ohh I rushed to ask, of course rxjs would be overkill when EventEmitter is built into node.
btw once you learn rxjs, suddenly everything becomes "ooohh yeah this could be better using this operator" hhh
Definitely on my todo list to learn more!
@@mariusespejo Agreed ! I have on a side project and it make it easier to maintain in the long term ! Thanks for your content
whats the name of your vs code font?
Cascadia mono!
What are your theme and fonta
he mentioned Cascadia mono in previous comment
A little harder to read and added overhead of mentally mapping logic. You can also get into some nasty circular event if you (or juniors) are not careful. The core module behavior has circular dependencies built in for you to make sure your abstractions are layered efficiently.
Nice tool in the belt though.
i feel like Appwrite was created with NestJS 🤣
This has nothing to do with Nest. This is Event Driven Architecture.
Honestly I don’t understand comments like yours... It’s like if I said “Cars have nothing to do with Driving”
Can you share your recording setup? Like what Mic and camera you use? 🫶
sm7b and some random sony camera. If you’re looking to start a channel like mine, you can get a pretty good mic for less than 200, and even a basic webcam will do if you’re mostly showing code
What is the difference between this pattern and using microservices?