I think rate limiting the dispatching of jobs onto the queue might be a better option. That will give opportunity to insert other tenant jobs in between any huge batch of jobs. Even releasing a job back onto the queue might be good enough if that puts the job at the end of the FIFO queue.
Very interesting! But since you have a queue per tenant, and you can have like 300+ tenants, how would this be compatible with Laravel Horizon? Also, I'm not a big fan of that pick random tenant part, maybe you can check the redis keys to see which tenants have pending jobs? 🤔
Agree, you can use something like: $tenants = static::tenants ?: Tenant::pluck('id'); shuffle($tenants); As for Horizon. It'll work if your balancing strategy is "none".
@@themsaid For the picking up tenants I meant something like `Redis::keys('queue:tenant-*')->get()` to check which tenants need to run queues, then the shuffle. Could you elaborate more about the horizon stuff? As far as I know, I need to tell horizon an array of queues to listen, but with this approach my queues will be dynamic and more than 300 at the same time
@@themsaid good approach thanks. will be very helpful if you can record a video when u have some time to show us how we can use this approach with horizon. I'm also not sure how we can define queues dynamically with current horizon implementation.
In the scenario where the queue is empty, won’t this approach basically get into an infinite loop? Since there is no delay between rounds, and say one round takes 25ms to run, it will now do 40 runs/second.
@@GeorgeBoot That's a good point yes. This is just an example of what you can do, in production I think you'd need to use a Redis list for example and pick from that instead of a DB. Or you can actually have counters that you increment on each dispatch to show you which tenants actually have pending jobs.
I don't like this solution. It doesn't seem very elegant to brute-force the database at 0 interval. I would propose randomizing only between the tenants that have jobs queued. This way a job is always picked by a worker if the queue is not empty. You have the jobs table, why not `DB::table('jobs')->distinct()->pluck('tenant_id')->random()` to randomly get a tenant? I don't even think you'd need multiple queues for this.
ماتعملنا حاجه عربي يا هندسة أو تعمل نسخه من الفيديو بتعليق عربي .. المحتوى العربي بيفتقد النوع ده من التقنيات المتقدمة والإنجليزى بتاعنا حكومى زى مانت عارف 🤣
![[LIVE] : ONE ลุมพินี 89 | คู่เอก "ยอดไอคิว vs คิริลล์"](http://i.ytimg.com/vi/Z3vYkHJ_5Ig/mqdefault.jpg)
I think rate limiting the dispatching of jobs onto the queue might be a better option. That will give opportunity to insert other tenant jobs in between any huge batch of jobs. Even releasing a job back onto the queue might be good enough if that puts the job at the end of the FIFO queue.
Ia gree
Very intresting ❤️
hi friend you are now 3,4k congrats and thanks for your super chatter
Not a fan of the final solution, but I like the idea - Its a good stepping stone into something more! Good thoughts
I'm also not a fan. I posted my idea of a solution here: th-cam.com/video/F16Qw4zAo28/w-d-xo.html&lc=Ugw0RbZ37s0MF4CgArF4AaABAg
Very interesting! But since you have a queue per tenant, and you can have like 300+ tenants, how would this be compatible with Laravel Horizon?
Also, I'm not a big fan of that pick random tenant part, maybe you can check the redis keys to see which tenants have pending jobs? 🤔
Agree, you can use something like:
$tenants = static::tenants ?: Tenant::pluck('id');
shuffle($tenants);
As for Horizon. It'll work if your balancing strategy is "none".
@@themsaid For the picking up tenants I meant something like `Redis::keys('queue:tenant-*')->get()` to check which tenants need to run queues, then the shuffle.
Could you elaborate more about the horizon stuff? As far as I know, I need to tell horizon an array of queues to listen, but with this approach my queues will be dynamic and more than 300 at the same time
@@themsaid good approach thanks. will be very helpful if you can record a video when u have some time to show us how we can use this approach with horizon. I'm also not sure how we can define queues dynamically with current horizon implementation.
thank you for your super chatter and support for premier of #mommybethlogan
In the scenario where the queue is empty, won’t this approach basically get into an infinite loop? Since there is no delay between rounds, and say one round takes 25ms to run, it will now do 40 runs/second.
I don't see how this would lead to an infinite loop. Each worker loop we pick 5 different queues to look for jobs.
Mohamed Said yeah sorry, infinite is not correct word here. Meant more in the sense of it will check lots of times/second, butting your db quite hard.
@@GeorgeBoot That's a good point yes. This is just an example of what you can do, in production I think you'd need to use a Redis list for example and pick from that instead of a DB. Or you can actually have counters that you increment on each dispatch to show you which tenants actually have pending jobs.
god bless you ans thanks for your support and super chatter to #mommybethlogan
I don't like this solution. It doesn't seem very elegant to brute-force the database at 0 interval.
I would propose randomizing only between the tenants that have jobs queued. This way a job is always picked by a worker if the queue is not empty.
You have the jobs table, why not `DB::table('jobs')->distinct()->pluck('tenant_id')->random()` to randomly get a tenant?
I don't even think you'd need multiple queues for this.
thanks for your support and super chatter of #mommybethlogan
ماتعملنا حاجه عربي يا هندسة أو تعمل نسخه من الفيديو بتعليق عربي .. المحتوى العربي بيفتقد النوع ده من التقنيات المتقدمة والإنجليزى بتاعنا حكومى زى مانت عارف 🤣
thanks for your super chatter to 3mommybethlogan to premeir