django-gearman

A convenience wrapper for Gearman clients and workers in Django/Python. Designed and tested against Gearman 1.14.

7 years after

django-gearman

django-gearman is a convenience wrapper for the Gearman Python Bindings.

With django-gearman, you can code jobs as well as clients in a Django project with minimal overhead in your application. Server connections etc. all take place in django-gearman and don't unnecessarily clog your application code.

Installation

It's the same for both the client and worker instances of your django project:

pip install -e git://github.com/fwenzel/django-gearman.git#egg=django-gearman

Add django_gearman to the INSTALLED_APPS section of settings.py.

Specify the following settings in your local settings.py file:

# One or more gearman servers
GEARMAN_SERVERS = ['127.0.0.1:4730']

Workers

Registering jobs

Create a file gearman_jobs.py in any of your django apps, and define as many jobs as functions as you like. The jobs must accept a single argument as passed by the caller and must return the result of the operation (if there is a result).

This module implements transparent pickling, so workers both send and receive native python datatypes.

Mark each of these functions as gearman jobs by decorating them with django_gearman.decorators.gearman_job.

For an example, look at the gearman_example app's gearman_jobs.py file.

Starting a worker

To start a worker, run python manage.py gearman_worker. It will start serving all registered jobs.

To spawn more than one worker (if, e.g., most of your jobs are I/O bound), use the -w option:

python manage.py gearman_worker -w 5

will start five workers.

Since the process will keep running while waiting for and executing jobs, you probably want to run this in a screen session or similar.

Clients

To make your workers work, you need a client app passing data to them. Create and instance of the django_gearman.GearmanClient class and execute it:

from django_gearman import DjangoGearmanClient
client = DjangoGearmanClient()
sentence = "Simple Example"
completed_job_request = client.submit_job("gearman_example.reverse", sentence))
print "Result: '%s'" % completed_job_request.result

The notation for the task name is appname.jobname.

Dispatching a background event without waiting for the result is easy as well:

completed_job_request = client.submit_job("gearman_example.reverse", wait_until_complete=False))

For a live example look at the gearman_example app, in the management/commands/gearman_example_client.py file.

For more information, see the gearman documentation: http://packages.python.org/gearman/

Example App

For a full, working, example application, add gearman_example to your INSTALLED_APPS, then run a worker in one shell:

python manage.py gearman_worker -w 4

and execute the example app in another:

python manage.py gearman_example_client

You can see the client sending data and the worker(s) working on it.

Licensing

This software is licensed under the Mozilla Tri-License:

***** BEGIN LICENSE BLOCK *****
Version: MPL 1.1/GPL 2.0/LGPL 2.1

The contents of this file are subject to the Mozilla Public License Version
1.1 (the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.mozilla.org/MPL/

Software distributed under the License is distributed on an "AS IS" basis,
WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
for the specific language governing rights and limitations under the
License.

The Original Code is django-gearman.

The Initial Developer of the Original Code is Mozilla.
Portions created by the Initial Developer are Copyright (C) 2010
the Initial Developer. All Rights Reserved.

Contributor(s):
  Frederic Wenzel <[email protected]>
  Aidan Lister <[email protected]

Alternatively, the contents of this file may be used under the terms of
either the GNU General Public License Version 2 or later (the "GPL"), or
the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
in which case the provisions of the GPL or the LGPL are applicable instead
of those above. If you wish to allow use of your version of this file only
under the terms of either the GPL or the LGPL, and not to allow others to
use your version of this file under the terms of the MPL, indicate your
decision by deleting the provisions above and replace them with the notice
and other provisions required by the GPL or the LGPL. If you do not delete
the provisions above, a recipient may use your version of this file under
the terms of any one of the MPL, the GPL or the LGPL.

***** END LICENSE BLOCK *****

Related Repositories

Stream-Framework

Stream-Framework

Stream Framework is a Python library, which allows you to build news feed, activ ...

awesome-php

awesome-php

收集整理一些常用的PHP类库, 资源以及技巧. 以便在工作中迅速的查找所需... ...

django-beanstalkd

django-beanstalkd

A convenience wrapper for beanstalkd clients and workers in Django using the bea ...

django-taskforce

django-taskforce

django-taskforce implements a job server for Django apps. It lets you execute lo ...

ospi

ospi

Open Source Presence Infographic of Indian Startups ...


Top Contributors

fwenzel aidanlister jbalogh jonasvp