How to Create a Celery Task Progress Bar in Django
ฝัง
- เผยแพร่เมื่อ 23 มิ.ย. 2020
- In this video I will show you how to use the library Django Celery Results to display a progress bar for any Celery tasks you have running.
Need one-on-one help with your project? I can help through my coaching program. Learn more here: prettyprinted.com/coaching
docs.celeryproject.org/en/sta...
github.com/czue/celery-progress
github.com/celery/django-cele...
Get the code here: prettyprinted.com/l/V9v
Twitter: / pretty_printed
Github: github.com/prettyprinted
Thanks for highlighting how to manually request the task progress, this makes it useful for people using Django as just an API using DRF. Much appreciated!
most tutorials edit the messed up parts and show them as smooth flow. but you admitin and showing the error and fixing is very helpful. Thanks a lot.
Only channel on youtube covering this, been helpful. Thanks a ton :)
Thanks again for another great video Anthony... please make more videos like this!!
Hey Anthony, Thanks for another great video. Your videos deserve more views and likes.
This is a criminally underviewed video; helped me a lot! Thank you!
Glad it helped you!
Awesome! You made my day! Thank you for the awesome video.
Thanks for clear explanation.
Man you are just awesome. I always follow your videos 👏👏
Great video and worked really well when assigning a workr as --pool=solo for windows users
Wow! Loved it. Unique content
I'm glad you liked it!
work as a charm
Thank you very much i just need this and you uploaded the video. What a coincidence?😄
That's awesome! I'm glad I could help.
Awesome as ever.
Thanks for watching!
Thanks a lot!
Thank you.
Thank you ! thanks kinda cool
Glad you liked it!
It's a great video, thanks! What would really help is to have a block diagram that shows what elements are used and how they talk to each other. It is hard to grasp like this (with as little background as I have), and because of that, it looks unnecessarily complex. In other words, the necessity for this complexity is not explained.
Which part is too complex specifically? if you mean the general reason why you would use this, in that case it's more for running things you know are going to take a lot of time, have many steps, some of which could fail, and more along these lines. Celery is something I would say most don't use. I am for example only using it for the first time in my 7 year Development Career because I need to create an Application for working with an AI, one which has a lot of steps, which I want to communicate to the user.
if want to send this progress over to a React project which is currently using REST API from django backend
how I can achieve that
?
Cool video.Does celery support windows now?
Please make an app based on react and django
how to get redis url?
Hii, how to redirect to another page after progress bar shows success...
My code is working properly and the celery worker is also working properly but I don't see anything in django admin . What might be the problem?
Is it Necessary to use redis broker ?
How to get this live response in react js?
Hi Anthony Can you please do a flask one?
I'll look into making a Flask version.
Please upload for flask as well.
I'll look into making one for Flask.
@@prettyprinted Thank you very much. :) You are awesome 👏
its important to add -------- 'DIRS': [os.path.join(BASE_DIR, 'templates')] in your settings.py
Hi, great video. I followed it and my async tasks are working when I run django locally (e.g. runserver), but it doesn’t work when deployed on Heroku. Redis receives the task requests, but they aren’t added to the Heroku postgres database (thus not displaying in the admin app). Any advice?
Same problem here
I figured out how to fix it. Compile Celery with "--pool=solo" argument.
Example: celery -A progress worker -l info --pool=solo
@@andrehenriquemendes7270 Thanks man this helped!
Please, please help me, Why I receive this error? I installed ubuntu 20.04, I also connect to redis locally. May I have to change any my laptop configuration settings?
[2021-04-11 19:58:44,253: ERROR/MainProcess] consumer: Cannot connect to redis://:**@ec2-54-217-183-159.eu-west-1.compute.amazonaws.com:22600//: Error while reading from socket: (104, 'Connection reset by peer').
Trying again in 26.00 seconds... (13/100)
Why does the index page say 'done' while the progress is still building up? i.e. If it is already done, then what is the need for the progress bar.
Not getting Task Results in Django Admin..! CAn some one help?
task is not getting reflected in database
Warninig. Redis url in app/nameapp/settings, Config Vars.
I'm trying to conect to redis locally using :
"CELERY_BROKER_URL = 'redis://127.0.0.1:6379/1'
CELERY_ACCEPT_CONTENT = ['json']
CELERY_TASK_SERIALIZER = 'json'
CELERY_RESULT_BACKEND = 'redis'"
When I run celery i get:
I'm getting an error "ModuleNotFoundError: No module named 'django_celery_results'" I have the module installed.
Pip freeze reveals:
amqp==5.0.2
asgiref==3.3.1
billiard==3.6.3.0
celery==5.0.4
certifi==2020.12.5
click==7.1.2
click-didyoumean==0.0.3
click-plugins==1.1.1
click-repl==0.1.6
Django==3.1.4
django-celery-results==2.0.0
kombu==5.0.2
prompt-toolkit==3.0.8
pytz==2020.4
redis==3.5.3
six==1.15.0
sqlparse==0.4.1
vine==5.0.0
wcwidth==0.2.5
Can you guys help?
If like me your task get send but never succed mind
'pip install eventlet'
And launch celry with this following command line :
celery -A progress worker --loglevel=info -P eventlet
Hey, Its not adding tasks to my database table in admin panel, hence rest of the things also not working, can I get the solution. I have followed all steps as it is
Celery results is always empty. I don't know whats lacking. can you please help
Same problema here. Did you fix it?
I figured out how to fix it. Compile Celery with "--pool=solo" argument.
Example: celery -A progress worker -l info --pool=solo
@@andrehenriquemendes7270 Thanks! This worked for me.
@@andrehenriquemendes7270 You are the GOAT man! Thank you!!
I still don’t get how you can implement this in a real project, it’s as if you need to know exactly how long your task is gonna run ( which in still manageable by putting a timeout) but the hardest part is knowing where to pinpoint the progress of the task, because nobody’s using sleep() in real applications, so how do you know where and when to update the progress?
This would be just the same as showing a filling progress bar to the user with JavaScript while the page loads (in other words faking it) I genuinely want to know how to implement this for a real long running task because it seems unpractical.
I very much agree with you
You can set a timer from the beginning of your function and a timer at the end, then make the substraction and set your progress bar right after
for example i have an app where the user can input words, and it will run a function with each of those words. I update the progressbar once a word is done and the user can see 145/500words done
Now back to this after I have had more experience in software engineering 😅, this is an easy fix: just use a different thread. To get the runtime of the executed function I’d probably warp it in a decorator
@@BohdanDuCiel celery is useful because you can queue multiple tasks. This addon for progress is purely for informational purpose, since in my case the user can input an unlimited number of words, and it's designed for batches of 2-3k words, and each one can take up to 1-2 minutes, it's useful that they can close the browser and check once in a while to see of their session is complete. It's purely a UX thing:)
Unfortunately, the vast majority of youtube tutorials come down to "we do this, we do that, let's paste it here, let's install that". It's just sad. I want to know why we're doing this or that. Otherwise, it's totaly useless. The equevalent of this tutorial is a link to source code. No need for 20 minutes video.
celery looks very over engineered.
maybe in this example. But in a prod setting with any amount of traffic it is capable of handling large task queues.