Using task queues and background workers can greatly speed up your web applications by offloading work to a different process. This prevents your web app from being blocked from responding to user requests while it does the work.
I've been looking at a couple of options for setting up a task queue and background worker, and it's something that a lot of readers and students have asked for.
The two options I'm looking at are rq and celery, and having tried both I can recommend rq for most applications. It's a bit simpler, scales well, and it does most of what you might need for a task queue.
On the other hand, celery is more feature-rich but also more complicated (both to learn and use). I found that I could use rq after just a short learning curve, whereas celery took substantially longer. Since developer time is one of our most valuable assets, that's definitely something to have in mind!
