Yet Another Redis Queue

https://img.shields.io/pypi/v/yarqueue.svg https://img.shields.io/travis/clbarnes/yarqueue.svg Documentation Status

Yet Another python queue backed by redis; but modern and compliant

Heavily inspired by hotqueue.

Features

  • Compatible with the API of multiprocessing.Queue

  • LIFO, FIFO and Double-Ended variants

    • Each has a non-joinable and a joinable form (compatible with multiprocessing.JoinableQueue)

  • Additional convenience methods:

    • get_many(), put_many(), clear() for all queues

    • n_tasks() and n_in_progress() for joinable queues

  • Can be used as a context manager

    • Deletes the queue on exit

    • For Joinable variants, first waits for all tasks to be done

  • Can use custom serializers, or none at all

    • By default, uses the highest pickle protocol available, using the pickle5 backport if possible.

  • As thread-safe as the underlying Redis client instance

  • Simple queue watchers

    • yarqwatch: command-line progress bars for given queues

    • yarqserve: simple REST API and browser page for given queues

Credits

This package was created with Cookiecutter and the audreyr/cookiecutter-pypackage project template.