Get the source code for this video for FREE → the-dotnet-weekly.ck.page/quartz Want to master Clean Architecture? Go here: bit.ly/3PupkOJ Want to unlock Modular Monoliths? Go here: bit.ly/3SXlzSt
I really liked this video, but I had hoped to see an example of a request that starts a long running background process It would go something like this: 1. http request is received 2. long running task is initiated based on given request parameters 3. if initiated correctly, http response is sent 4. request sender will get long running task update by webhook All of this is probably too long for a single video, so maybe you can split it up between points 1,2,3 and 4 Hope you like this idea!
I imagine an API that post the construction of new routine in quartz with the given params about how often it will work and the destination or subjects involved. Every time the post method in the controller is called, another instance is launched doing a different thing. This will be for configuring multiple data refreshing jobs for several powerbi dashboards, using their rest api
I see... You can configure a job instance, and trigger the schedule manually. Also take a look at Hangfire, because I think it's more suited for what you're looking for.
what's the difference between using Quartz vs setting up a simple worker service and scheduling with a scheduler something like `Dollar Universe` or `Active Batch Job`? Apart from the fact that using quartz we won't need scheduling tools.
@@yardeZ93 Hangfire is built upon the concept of persistence layer (we use Redis) so the scheduling of jobs and the executing of jobs can be in completely different processes. Their only point of contact is the job details stored in the persistence layer. It also comes with a Web Console so you can monitor the state of jobs as they are processed, which is especially important when you are running scheduled and recurring jobs. The Web Console can also run in a different process. It also has the concept of batches and continuations which we have found very useful. For example, a parent job that fires up once a day and locates all services without outstanding amounts due, then launches a batch of children jobs that take a payment and generate an invoice. And once the batch is finished it is followed up by a continuation job that sends out a summary via email of the payments taken that day.
You can also scale out horizontally and vertically with Hangfire. We have a number of worker processes per EC2 Instance (vertical) and then a number of EC2 Instances (horizontal) to increase the level of parallelism we can get with our batches. The Elasticache (Redis) persistence layer can handle these levels of concurrency with ease.
I'm wondering what do you think about background jobs from a DDD perspective in a microservices environment. Usually you have some service that deals with some business features of a specific bounded context. And this service usually manages its own database (or at least its own tables). However, it's a common scenario that inside a bounded context there's some need of background jobs that run aside the main microservice. So I'm struggling with this concept: can these background jobs access the same database/tables of the main microservice, cause they are working virtually in the same bounded context? Or they should call some API exposed by the main microservice, so it is the only one that access the persistence layers with aggregates and entities?
thank you for the video on Quartz. Short and precise info provided. I want access the same quartz db from 2 apps: 1 desktop app (to manage jobs and triggers) and 1 windows service to run the jobs. Is it possible to do this using Quartz?
Great tutorial Milan. So there will list of customer Name and DOB where I job will check at start of day and suppose there is dob found then send email to particular customer. So do I put this logic and job will not work whole day.
Hi everyone, I have a question about this. I can't use usemicrosoftdependencyinjection.I keep getting this error message "There must be an empty constructor method". How can I use di in my business class?
I would have made the backgroud job class inherit from the IConfiguration interface directly, so you will have both the job action and it's schedule in the same class, and that it's pretty neat for small operations (In case multiple job have the same schedule i would make a parent class)
On the production environment, we would typically have multiple containers/pods. How can we make sure always one single pod/container runs all jobs. We dont want all pods/containers to execute/run the jobs parallely. How can we implement this. Please help
Hi Milan, I was able to follow your tutorials successfully, and my job works perfectly in dev, however, after deploying the app in IIS the task does not get executed I am using a job in an API service core net what considerations do I have to take?
@@MilanJovanovicTech Interestingly, seems like the IIS put the application in sleep mode and shut the application down, and the schedule, when a new request comes to the API the scheduled start.. now I need to figure out how to keep the application always running, or decouple my API from the Quartz service.. any advice is welcome.. thanks -> Application is shutting down... ->Scheduler "QuartzScheduler_$_NON_CLUSTERED" shutting down. ->Scheduler QuartzScheduler_$_NON_CLUSTERED paused. ->Scheduler QuartzScheduler_$_NON_CLUSTERED Shutdown complete.
@@victors16811 On IIS configure you set the always running in the IIS .Default IIS deployed service running when the request come in , but if you configure it always running.
Nice one! Can you make a video how to make an email email reminder/ like your reservation will start before 48 hours / using Quartz and SendGrid! I would really appreciate it 😊
@@MilanJovanovicTech that would be amazing or you can make a guideline how should to do! / for testing might you can reduce a time for 1 minute or make it dynamic like between 1 hour to 60 hours not fixed value ?!
Is there a way to configure all instances of IConfigureOptions at once? It would be nice if there was a way that I could pass in the assembly and let the magic happen 😄
I did this in production but encountered some errors where it stopped running after x amount of time on server. I think it was application pool timeout or something that caused it. Worth noting though, you would have to set timout to 0 (on windows server)
Nice as always Milan! I have a question about this. How to create a project that launches different instances of the same routine in quartz? Off course I haven't finished watching this. But I want to ask you asap
@@MilanJovanovicTech Thanks for the answer. So I just get an ISender from MediatR in the IJob in the Infrastructure and send a new instance of the command every day for example? Is that an allowed approach when we are using Domain Driven Design?
@@nikolaymarchin9296 Absolutely. Just be mindful that you're sending the command outside an API request, in case you have any dependencies on the HTTP context.
I've more than 1 process inside 1 method. Let say 5 process. I want next process run in background so after finished 1st process i want to direct return without waiting other process completed. How?
@@MilanJovanovicTech Public Task Method(){ await Process1(); //after this process finished Process2();// in background Process3();// in background Process4();// in background Process5();// in background Return "success"; }
After investigating, I found the problem and two possible solutions: 1. Use a HostBuilder within the console, or 2. Create your own QuartzHostedService with your own factory to resolve jobs, etc. Something that doesn't help much is that when Quartz is unable to build the dependencies for a job's constructor, it simply does nothing and doesn't throw any error.
I got problem when i querying data 2024-03-01 at midnight. Its convert it in utc supposed 2024-02-29.. If query only datey only data. What happened explain this situation??🤔🤔🤔please
Quartz is just overall inferior to Hangfire. It's extremely verbose and doesn't have built-in, default support for concurrency or distributed environments.
You did not dig many holes have you? Better to dig a small hole in minutes with no worries versus doing it by hand. There are actually arguments for using Quartz, but the video nor your comment mention them.
@@GreyTaubeI'm actually really interested, when would you need to use Quartz, over hangfire. I can see why something like Serilog (maybe with seq) is soooo much better than built in ILogger or even Elmah. But I am still stuck between what background task scheduler to use.. Hnagfire seems a Lil simpler, comes with dashboard, idk.. everyone recommends Quartz though but I don't yet understand why.
Get the source code for this video for FREE → the-dotnet-weekly.ck.page/quartz
Want to master Clean Architecture? Go here: bit.ly/3PupkOJ
Want to unlock Modular Monoliths? Go here: bit.ly/3SXlzSt
I was really hoping to see how you move the configuration to an external xml or json file. Maybe something for a “short” video? 😊
Dang Milan bro, you've got an answer for every .NET question I can possibly ask. Please don't ever stop making these videos.
Really glad to hear that! 😁 And I'm glad this video was valuable
No matter how varied what I am looking for may be, there always be a Milan's video which explain it in the best way
Hundreds of topics left to cover, but I'm glad some questions are being answered 😁
Exactly, and when i am in need to implement something in my project, milan video just pop up at my notifications
Totally serious!!! You need to do a clean architecture bootcamp and spread the news, Mr. Clean Architecture Evangelist!!!
My Clean Architecture course is coming out in the next 7-10 days, if all goes well. So that should be a nice start 😁😁
Waiting
@@MilanJovanovicTechso we are waiting for TH-cam notification 😊
Can’t wait for the course 🎉
I really liked this video, but I had hoped to see an example of a request that starts a long running background process
It would go something like this:
1. http request is received
2. long running task is initiated based on given request parameters
3. if initiated correctly, http response is sent
4. request sender will get long running task update by webhook
All of this is probably too long for a single video, so maybe you can split it up between points 1,2,3 and 4
Hope you like this idea!
That's a very nice idea for a future video!
You've made me a .NET beast. Thank you.
Hah, that's awesome! Glad to hear :)
Thanks so much! this was very helpful!! Just about to write my first background job application with quartz
Glad it was helpful!
Excellent explanation, we need more videos like this one for understanding background tasks. Keep up the great work!
Glad it was helpful!
Thanks brother for sharing scheduler knowledge. it help me a lot.
My pleasure
Exciting content! Many thanks to you
Glad you enjoy it!
Nice share! I was looking into a solution for a problem lately and this was one of my options. I prefer a simple interface surface to work with.
Glad it helped!
I imagine an API that post the construction of new routine in quartz with the given params about how often it will work and the destination or subjects involved. Every time the post method in the controller is called, another instance is launched doing a different thing. This will be for configuring multiple data refreshing jobs for several powerbi dashboards, using their rest api
I see... You can configure a job instance, and trigger the schedule manually. Also take a look at Hangfire, because I think it's more suited for what you're looking for.
@@MilanJovanovicTechHi, why is hangfire more suitable for that?
Thanks for this video!
You're welcome! :)
Thank you for the video, it was interesting and informative.
Glad it was helpful!
Another great video, glory to Slavic programmers 😁
💪
Great video!
Thanks!
Thank you for this helpful video. Glad I found it. I'm a new subscriber!
Glad you enjoyed this video 😁
Thank you for the video, love your blog also
Thanks a million!
nice contribution! thank you!
Glad you liked it!
I use to create custom windows service, after some time I moved to "Hangfire"
I will try this by creating sample project soon
Hangfire is very good also
what's the difference between using Quartz vs setting up a simple worker service and scheduling with a scheduler something like `Dollar Universe` or `Active Batch Job`? Apart from the fact that using quartz we won't need scheduling tools.
I'm not familiar with either of the schedulers you mentioned, so can't give you a proper answer
do they provide any dashboard like hangfire, and how is this better than hangfire?
No (I'll make a separate video for Hangfire to compare them)
@@MilanJovanovicTech ok, thanks
So nice approch. Thanks.
Glad you liked it!
Great!! men!! thanks a lot!!👏🏽👏🏽👏🏽👏🏽🎉🎉🎉🎉
No problem
Which is better Quartz or HangFire ?
After watching this video I am glad we are using Hangfire Pro for our purposes.
@@cliffwakefield Can you elaborate about the reason?
@@yardeZ93 Hangfire is built upon the concept of persistence layer (we use Redis) so the scheduling of jobs and the executing of jobs can be in completely different processes. Their only point of contact is the job details stored in the persistence layer. It also comes with a Web Console so you can monitor the state of jobs as they are processed, which is especially important when you are running scheduled and recurring jobs. The Web Console can also run in a different process. It also has the concept of batches and continuations which we have found very useful. For example, a parent job that fires up once a day and locates all services without outstanding amounts due, then launches a batch of children jobs that take a payment and generate an invoice. And once the batch is finished it is followed up by a continuation job that sends out a summary via email of the payments taken that day.
You can also scale out horizontally and vertically with Hangfire. We have a number of worker processes per EC2 Instance (vertical) and then a number of EC2 Instances (horizontal) to increase the level of parallelism we can get with our batches. The Elasticache (Redis) persistence layer can handle these levels of concurrency with ease.
But to get the more advanced features and official Redis support you need to pay for a license.
Nice one!
Thanks a lot :)
I'm wondering what do you think about background jobs from a DDD perspective in a microservices environment. Usually you have some service that deals with some business features of a specific bounded context. And this service usually manages its own database (or at least its own tables). However, it's a common scenario that inside a bounded context there's some need of background jobs that run aside the main microservice. So I'm struggling with this concept: can these background jobs access the same database/tables of the main microservice, cause they are working virtually in the same bounded context? Or they should call some API exposed by the main microservice, so it is the only one that access the persistence layers with aggregates and entities?
Since they are highly coupled anyway, using the same database wouldn't make things much worse.
What is the difference between this and .NET 6 introduced PeriodicTimer?
www.milanjovanovic.tech/blog/running-background-tasks-in-asp-net-core
@@MilanJovanovicTech Thanks!
thank you for the video on Quartz. Short and precise info provided.
I want access the same quartz db from 2 apps: 1 desktop app (to manage jobs and triggers) and 1 windows service to run the jobs.
Is it possible to do this using Quartz?
It should be doable. Did you try it?
Great tutorial Milan. So there will list of customer Name and DOB where I job will check at start of day and suppose there is dob found then send email to particular customer. So do I put this logic and job will not work whole day.
Yes, you can make the job run once a day
Is a Hangfire vidro coming soon?
Yes! 😁
What is the best way to prevent concurrency on the scaled instance?
Move the job into its own process
Hi everyone, I have a question about this. I can't use usemicrosoftdependencyinjection.I keep getting this error message "There must be an empty constructor method". How can I use di in my business class?
Give me a repo I can take a look at?
I can't do this because of the privacy policy. The version I use is 3.7.0 and it is said that this feature is ignored.
Is there any video for implementing the same with embedding Silkier Quartz UI in it
I haven't used it
Does Quartz work with Maui or only web?
Does MAUI support IHostedService?
@@MilanJovanovicTech after some research, looks like it does not. Thanks
wow! hope i could be an awesome software engineer as you are!!
You can do it
I would have made the backgroud job class inherit from the IConfiguration interface directly, so you will have both the job action and it's schedule in the same class, and that it's pretty neat for small operations
(In case multiple job have the same schedule i would make a parent class)
Inherit from IConfiguration?
Shorthand for the IConfigurationOptions
Of course it's a preference of mine, a lot of people would feel uncofortable having such a multiporpuose class
On the production environment, we would typically have multiple containers/pods. How can we make sure always one single pod/container runs all jobs. We dont want all pods/containers to execute/run the jobs parallely. How can we implement this. Please help
Environment variable? Or running jobs in a separate container
I apologize in advance if you mentioned this in the video (my English is very poor so far), what happens if an Exception occurs?
The task fails 😁 Next one starts as usual... You'd have to handle exceptions, of course
Hi Milan, I was able to follow your tutorials successfully, and my job works perfectly in dev, however, after deploying the app in IIS the task does not get executed I am using a job in an API service core net what considerations do I have to take?
No idea. It should work the same.
@@MilanJovanovicTech Interestingly, seems like the IIS put the application in sleep mode and shut the application down, and the schedule, when a new request comes to the API the scheduled start.. now I need to figure out how to keep the application always running, or decouple my API from the Quartz service.. any advice is welcome.. thanks
-> Application is shutting down...
->Scheduler "QuartzScheduler_$_NON_CLUSTERED" shutting down.
->Scheduler QuartzScheduler_$_NON_CLUSTERED paused.
->Scheduler QuartzScheduler_$_NON_CLUSTERED Shutdown complete.
@@victors16811 On IIS configure you set the always running in the IIS .Default IIS deployed service running when the request come in , but if you configure it always running.
Nice one! Can you make a video how to make an email email reminder/ like your reservation will start before 48 hours / using Quartz and SendGrid! I would really appreciate it 😊
Might be an interesting video to make
@@MilanJovanovicTech that would be amazing or you can make a guideline how should to do! / for testing might you can reduce a time for 1 minute or make it dynamic like between 1 hour to 60 hours not fixed value ?!
Is there a way to configure all instances of IConfigureOptions at once? It would be nice if there was a way that I could pass in the assembly and let the magic happen 😄
With an extension method?
I did this in production but encountered some errors where it stopped running after x amount of time on server. I think it was application pool timeout or something that caused it. Worth noting though, you would have to set timout to 0 (on windows server)
That's the problem with your hosting model, not Quartz
Of course, just thought it was worth mentioning in case someone else ran into it or could help someone before they deploy it :)
What's that mean, hosting model
If you own the server and can control IIS, you need to remove idle timeout.
If you don't own the server you'll need to trigger one network call within the timeout so that the app pool doesn't sleep.
Nice as always Milan! I have a question about this. How to create a project that launches different instances of the same routine in quartz? Off course I haven't finished watching this. But I want to ask you asap
What do you mean by different instances?
Is it possible to trigger a command from the Application layer from a quartz job?
It's possible
@@MilanJovanovicTech Thanks for the answer. So I just get an ISender from MediatR in the IJob in the Infrastructure and send a new instance of the command every day for example? Is that an allowed approach when we are using Domain Driven Design?
@@nikolaymarchin9296 Absolutely. Just be mindful that you're sending the command outside an API request, in case you have any dependencies on the HTTP context.
it says that UseMicrosoftDependencyInjectionJobFactory is obsolete. how do we change it?
You can freely remove that call, it's no longer needed
Is it an open source library
Yes
I've more than 1 process inside 1 method. Let say 5 process. I want next process run in background so after finished 1st process i want to direct return without waiting other process completed. How?
I didn't understand this
@@MilanJovanovicTech
Public Task Method(){
await Process1(); //after this process finished
Process2();// in background
Process3();// in background
Process4();// in background
Process5();// in background
Return "success";
}
hi Milan, I am trying to run the code on a console app, but job never start. can you explain how to get it running on console app
No idea why
@@MilanJovanovicTech
After investigating, I found the problem and two possible solutions:
1. Use a HostBuilder within the console,
or
2. Create your own QuartzHostedService with your own factory to resolve jobs, etc.
Something that doesn't help much is that when Quartz is unable to build the dependencies for a job's constructor, it simply does nothing and doesn't throw any error.
What do you think will Quartz be a good option or using built-in background jobs provided by aspnetcore ?
Quartz is more robust
got it. @@MilanJovanovicTech
Please make video about best practice with postgres datetime type. Querying different scene.. With different time zone issue
Just use UTC 😅
@@MilanJovanovicTech i already used.. Actually problem is timespan grouping with dateonly,timeonly..
I got problem when i querying data 2024-03-01 at midnight. Its convert it in utc supposed 2024-02-29.. If query only datey only data. What happened explain this situation??🤔🤔🤔please
[Obsolete("MicrosoftDependencyInjectionJobFactory is the default for DI configuration, this method will be removed later on")] ？
The one used here isn't obsolete
我知道，但是最新的包过期了，RabbitMq那个视频使用的包也过期了，能更新一下视频吗@@MilanJovanovicTech
thanks for share videos
You're welcome
using this but only 1 of my job gets registered
Use a different name for the job?
muy bueno crack
Ty
+1 like, +1 sub. Thanks man
Awesome! Glad to have you on board!
mašina čovjek
Hvala, živ bio! 😁
Quartz is just overall inferior to Hangfire. It's extremely verbose and doesn't have built-in, default support for concurrency or distributed environments.
Which is why it's great for simple recurring tasks. You don't need a bulldozer to drill a small hole.
You did not dig many holes have you? Better to dig a small hole in minutes with no worries versus doing it by hand.
There are actually arguments for using Quartz, but the video nor your comment mention them.
@@GreyTaubeI'm actually really interested, when would you need to use Quartz, over hangfire.
I can see why something like Serilog (maybe with seq) is soooo much better than built in ILogger or even Elmah.
But I am still stuck between what background task scheduler to use.. Hnagfire seems a Lil simpler, comes with dashboard, idk.. everyone recommends Quartz though but I don't yet understand why.
Which country was quartz developed in?
No idea
@@MilanJovanovicTech Tampere, Pirkanmaa, Finland