Using task queues and background workers can greatly speed up your web applications by offloading work to a different process. This prevents your web app from being blocked from responding to user requests while it does the work.
I've been looking at a couple of options for setting up a task queue and background worker, and it's something that a lot of readers and students have asked for.
The two options I'm looking at are
celery, and having tried both I can recommend
rq for most applications. It's a bit simpler, scales well, and it does most of what you might need for a task queue.
On the other hand,
celery is more feature-rich but also more complicated (both to learn and use). I found that I could use
rq after just a short learning curve, whereas
celery took substantially longer. Since developer time is one of our most valuable assets, that's definitely something to have in mind!