I use Horizon, it's great! I process a lot of data with it, currently sitting at 1800 jobs/minute. I made a big NOOB error when I started by passing the data to be processed into the job itself, which was occasionally multi-megabytes. This worked fine most of the time but if anything went wrong, the multiple job attempts plus storage of errors meant it quickly filled up the memory and made the problem catastrophic! It also made the horizon dashboard sluggish and fail, as it tried to show me the multi-megabyte data in each job attempt. The solution: Store the data separately, either on disk, DB or in Redis, and pass only an ID or reference to the data into the job. Then Horizon will fly.
Great comment, working at a similar scale and found this myself. In my case nested eager-loaded relationships were the usual explanation. Just pass a model ID to the job, then load the model from the DB in handle(). This also ensures you’re using the most up-to-date model instance at runtime too!
@@vasiovasio it depends what the jobs are doing. If it’s something simple like sending an email, a basic VPS in the region of $50/mo. could handle this with 10 Horizon workers. 1800 jobs/min. across 10 workers = 3 jobs/sec./process. More complex tasks will vary significantly.
@@rcnhsuailsnyfiue2 Thank you! In these days of Unreal and I mean Unbelievable Inflation ( after Covid and the start of the war in Ukraine, the food pricing in my country, Bulgaria are basically Three times higher than 2019 ) the prices of hosting, servers, cloud, storage and everything in the sphere is relatively normal as a price and I'm thankful about it, because tell me for other opportunity to test some business idea for $50-$100 per month and everything else is your work, time, energy and dedication!
These are awesome video's to help people get to see some of the cool sides of modern PHP development options. I have been using Laravel since version 3 and the whole ecosystem has come so far. Thank you for making this series.
bro... I'm an elixir dev and you are getting my hooked on Laravel. Such a cool tool for building projects. Next startup I'm at, I might give it a shot. Also, looked at your website and saw you are a twin dad too! ✊ solidarity!
Why there is no comprehensive and up to date laravel course that shows how to build an API? We don't need intertia, fullstack, htmlx, blade, just API, like in 99% of companies.
Gonna be honest, don't really care for Laravel. But anything that Aaron Francis makes deserve consideration. Probably one of the best "dev evangelist"-type channels.
Great intro video! I recently discovered Horizon and now using it at a reasonably big scale. Just one question. If I don't specify the queue worker name while dispatching a job, how do I make sure that the processes get distributed to different queue workers automatically? Currently, all of them are going to the default worker only.
Spin up a copy of your app on another machine in the same data center/hosting region. No need to handle frontend requests, this copy only needs access to your central Redis database, and therefore can process queued jobs like any other copy of your application. Horizon will automatically assign jobs to workers, both multiple workers on one machine, or across workers on multiple machines. Horizontal scaling is simply a case of spinning up more instances of your app.
I have used Laravel Horizon for years and it's even been a deciding factor for bringing Laravel into a project. That said, I still don't get why the environment setup is so overy complicated and hard-coded into to config/horizon.php file. If I have 3 or 4 environments and need to scale up or down the worker limits on one of them then I would rather just update the .env file for that server than push a commit with per-server/environment logic...
@@rcnhsuailsnyfiue2 yes that’s my preference but that’s not the documented /recommended way to do it. Docs show a big array of repeated hardcoded values per environment, rather than simply setting your own values per queue in the .env file which seems more logical to me
@@JonnyJKFthe hard coded configs are just fallbacks/defaults and an example of what can be configured. The env() calls should be telling enough to hint at the fact, that they are meant to be overridden by the actual environment variables in your systems.
Laravel Horizon. The point at which you cross over to coding PHP with no hope of returning. You become increasingly redshifted until you simply vanish from sight.
You are going to drive SO MANY new devs to Laravel with this content ✨
Thanks, I hope so!
True. I choose laravel for my startup half a year ago because Aaron briefly talked about it. Loving every second working with it!
I use Horizon, it's great! I process a lot of data with it, currently sitting at 1800 jobs/minute. I made a big NOOB error when I started by passing the data to be processed into the job itself, which was occasionally multi-megabytes. This worked fine most of the time but if anything went wrong, the multiple job attempts plus storage of errors meant it quickly filled up the memory and made the problem catastrophic! It also made the horizon dashboard sluggish and fail, as it tried to show me the multi-megabyte data in each job attempt. The solution: Store the data separately, either on disk, DB or in Redis, and pass only an ID or reference to the data into the job. Then Horizon will fly.
Great comment, working at a similar scale and found this myself. In my case nested eager-loaded relationships were the usual explanation. Just pass a model ID to the job, then load the model from the DB in handle(). This also ensures you’re using the most up-to-date model instance at runtime too!
What server specs do you use for 1800 jobs per minute?
@@vasiovasio it depends what the jobs are doing. If it’s something simple like sending an email, a basic VPS in the region of $50/mo. could handle this with 10 Horizon workers. 1800 jobs/min. across 10 workers = 3 jobs/sec./process. More complex tasks will vary significantly.
@@rcnhsuailsnyfiue2 Thank you!
In these days of Unreal and I mean Unbelievable Inflation ( after Covid and the start of the war in Ukraine, the food pricing in my country, Bulgaria are basically Three times higher than 2019 ) the prices of hosting, servers, cloud, storage and everything in the sphere is relatively normal as a price and I'm thankful about it, because tell me for other opportunity to test some business idea for $50-$100 per month and everything else is your work, time, energy and dedication!
Not a Laravel dev, but still enjoy learning about the ecosystem from Aaron. Nice work!
These are awesome video's to help people get to see some of the cool sides of modern PHP development options. I have been using Laravel since version 3 and the whole ecosystem has come so far. Thank you for making this series.
Keep up the excellent work, Aaron! I don't care what topic it is, I just know that I need MORE!
bro... I'm an elixir dev and you are getting my hooked on Laravel. Such a cool tool for building projects. Next startup I'm at, I might give it a shot.
Also, looked at your website and saw you are a twin dad too! ✊ solidarity!
Hey nice! How old are yours?
@@aarondfrancis just turned 4 months! It's a challenge but a total blessing at the same time. We are also in the Dallas area so ➕➕ on common things
@@zacbarnes2187 oh no way! ours are 10 months and 3 years. DM me on Twitter, we should grab coffee sometime
Thanks Aaron! Great video, i really like this format. Can't wait for the next one!
Sweet intro! Never checked out Horizon before, thanks for the overview.
You explanations make the topics more interesting.
Cool how Laravel has 1st class support for everything you need. Unlike some other popular frameworks where you need dozen 3rd party SaaS providers.
This series keep interesting more and more ❤
Interesting!,
Now we need a video explaining Laravel Jobs😁
Thanks Aaron! C-Ya
Why there is no comprehensive and up to date laravel course that shows how to build an API? We don't need intertia, fullstack, htmlx, blade, just API, like in 99% of companies.
thats basically 1 page of the official documentation
I love Laravel, except the new trend of make it “Blade first”… We do not need a full stack PHP, we need a good API
Laracasts have just done a Laravel API Masterclass
Laracasts probably has that! Let me know what's missing and I might be able to fill it in
“We” do need fullstack Laravel. “You” may not.
You should be doing the official Laravel channel videos
Love every single one of your video ❤, please keep it up
this is very good, thank you!
Amazing video as always! Great details and examples. By the way Aaron, which editor do you use? It looks nice and clean.
A pretty customized PHPStorm!
@@aarondfrancis Awesome! I work with PHPStorm every day and did not recognize it at all.
Gonna be honest, don't really care for Laravel. But anything that Aaron Francis makes deserve consideration. Probably one of the best "dev evangelist"-type channels.
Gah that rules. Thanks for telling me
I'm not sure if you covered this already but would love to have a short video of your streaming setup :)
I was anxiously awaiting for you to click the Live update button in the top right hand corner :)
Thank you Aaron! Great video.
btw what font do you use for the terminal?
JetBrains Mono!
Great intro video! I recently discovered Horizon and now using it at a reasonably big scale. Just one question. If I don't specify the queue worker name while dispatching a job, how do I make sure that the processes get distributed to different queue workers automatically? Currently, all of them are going to the default worker only.
Pulse or Reverb next?
I think it is good to mention that Redis must be installed and running
1:31 Both Sidekiq and Oban are free and open source. They do have paid addon features, but these are strictly addons to the core functionality.
A better comparison for Horizon would be Sidekiq Pro IMO!
Awsome Intro
Thanks quite interesting
What terminal font do you use? It looks breathtaking.
I think it's just jetbrains mono
Is there actually a proper concept for horizontally scaling?
You can just add more machines working the queues
Spin up a copy of your app on another machine in the same data center/hosting region. No need to handle frontend requests, this copy only needs access to your central Redis database, and therefore can process queued jobs like any other copy of your application. Horizon will automatically assign jobs to workers, both multiple workers on one machine, or across workers on multiple machines. Horizontal scaling is simply a case of spinning up more instances of your app.
@@rcnhsuailsnyfiue2 exactly correct! Thanks for the better explanation!
Real lenses in your glasses yayyyyyh
A real OG here
How are you creating the horizon.test url? Silly take away out of this entire video but I am really curious
That's a feature of Laravel Herd
Hi! How's it going? Is it possible to allow special characters on Laravel ( password creation ).
Out of topic. Is there a plan to work with jwt tokens for authentication with api?
I have used Laravel Horizon for years and it's even been a deciding factor for bringing Laravel into a project. That said, I still don't get why the environment setup is so overy complicated and hard-coded into to config/horizon.php file. If I have 3 or 4 environments and need to scale up or down the worker limits on one of them then I would rather just update the .env file for that server than push a commit with per-server/environment logic...
You can do this easily! Just call env() within your Horizon config file. Then you can vary your configuration per-instance in the way you’d expect.
@@rcnhsuailsnyfiue2 yes that’s my preference but that’s not the documented /recommended way to do it. Docs show a big array of repeated hardcoded values per environment, rather than simply setting your own values per queue in the .env file which seems more logical to me
@@JonnyJKFthe hard coded configs are just fallbacks/defaults and an example of what can be configured. The env() calls should be telling enough to hint at the fact, that they are meant to be overridden by the actual environment variables in your systems.
How would you configure it for an app that behind load balancer (3 app servers. 1 DB, and 1 Redis server)?
Did this broaden your horizon?
I see what you did there!
@@aarondfrancis dad jokes 101
Laravel Horizon, Larave Telescope 😂, Laravel continues to keep me in it's ecosystem. Everything is just available for you.
I think there's a typo in the e-mail address you give in the video. Mails keep bouncing!
Hmmm that doesn't seem right
Laravel Horizon. The point at which you cross over to coding PHP with no hope of returning. You become increasingly redshifted until you simply vanish from sight.
Waiting for the what is the laravel php artisan optimize:clear command 😂
Great video but Redis became source-available now, open-source no more :(
That doesn't affect us though right? Just companies trying to resell Redis?
@@aarondfrancis Exactly that Aaron.
if only it worked on windows...
¯\_(ツ)_/¯
Bummer!
we hope that PHP team add the pcntl extension to windows :D
@@IbrahimIsmail98 yeah, yeah, i know... I use Horizon on my current job and i have not to really complain, this was just more of a wish... 🥲
WSL FTW 🎉