elasticluster python

Create clusters of VMs on the cloud and configure them with Ansible

======================================================================== ElastiCluster |travis-ci-status|

.. This file follows reStructuredText markup syntax; see http://docutils.sf.net/rst.html for more information

.. |travis-ci-status| image:: https://travis-ci.org/gc3-uzh-ch/elasticluster.svg?branch=master

ElastiCluster aims to provide a user-friendly command line tool to create, manage and setup computing clusters hosted on cloud infrastructures (like Amazon's Elastic Compute Cloud EC2, Google Compute Engine, or a private OpenStack cloud). Its main goal is to get your own private cluster up and running with just a few commands; this video_ demoes ElastiCluster setting up a computational batch-queueing cluster.

Complete documentation for ElastiCluster is available on the Read The Docs <http://elasticluster.readthedocs.org/> website. General discussion over ElastiCluster's usage, features, and bugs takes place on the [email protected] <https://groups.google.com/forum/#!forum/elasticluster> mailing-list (only subscribers can post).

The ElastiCluster project is an effort of the Services and Support for Science IT (S3IT) unit at the University of Zurich, licensed under the GNU General Public License version 3.

Features

ElastiCluster_ is in active development, and offers the following features at the moment:

  • INI-style configuration file to define cluster templates
  • Can start and manage multiple independent clusters at the same time
  • Automated setup of:
    • HPC clusters using SLURM or GridEngine;
    • Spark / Hadoop clusters with HDFS and Hive/SQL;
    • distributed storage clusters using GlusterFS, OrangeFS, or Ceph_;
    • ...or anything that you can install with an Ansible_ playbook!
  • Growing and shrinking a running cluster.

ElastiCluster is currently in active development: please use the GitHub issue tracker to file enhancement requests and ideas, or the mailing-list_ for discussion.

We appreciate pull requests for new features and enhancements. Please use the master branch as starting point.

Quickstart

ElastiCluster is a Python program; Python version 2.6 or 2.7 is required to run it. Python 3 is not (yet) supported.

It's quite easy to install ElastiCluster using pip_; the command below is all you need to install ElastiCluster on your system::

pip install elasticluster

If you want to try out the development version instead, please read the Installing development code from GitHub section in the manual.

.. : http://elasticluster.readthedocs.io/en/master/install.html#installing-development-code-from-github .. : http://elasticluster.readthedocs.io/en/latest/

Note: The 1.3 release is nearing, which has seen quite some changes from the 1.2 code that's on PyPI. For the moment, you are therefore encouraged to run the development code from GitHub__ and report on any bugs you find!

.. __: http://elasticluster.readthedocs.io/en/master/install.html#installing-development-code-from-github

.. References

References should be sorted by link name (case-insensitively), to make it easy to spot a missing or duplicate reference.

.. Amazon's Elastic Compute Cloud EC2: http://aws.amazon.com/ec2/ .. Ansible: https://ansible.com/ .. CentOS: http://www.centos.org/ .. Ceph: http://ceph.com/ .. Debian GNU/Linux: http://www.debian.org .. elasticluster: http://gc3-uzh-ch.github.io/elasticluster/ .. example configuration file: https://github.com/gc3-uzh-ch/elasticluster/raw/develop/elasticluster/share/etc/config.template .. enhancement requests and ideas: https://github.com/gc3-uzh-ch/elasticluster/issues .. Ganglia: http://ganglia.info .. GC3 Hobbes cloud: http://www.gc3.uzh.ch/infrastructure/hobbes .. github elasticluster repository: https://github.com/gc3-uzh-ch/elasticluster .. github: https://github.com/ .. GlusterFS: http://www.gluster.org/ .. GNU General Public License version 3: http://www.gnu.org/licenses/gpl.html .. Google Compute Engine: https://cloud.google.com/products/compute-engine .. Grid Computing Competence Center: http://www.gc3.uzh.ch/ .. GridEngine: http://gridengine.info .. Hadoop: http://hadoop.apache.org/ .. IPython cluster: http://ipython.org/ipython-doc/dev/parallel/ .. mailing-list: https://groups.google.com/forum/#!forum/elasticluster .. OpenStack: http://www.openstack.org/ .. OrangeFS: http://orangefs.org/ .. pip: https://pypi.python.org/pypi/pip .. python virtualenv: https://pypi.python.org/pypi/virtualenv .. Python: http://www.python.org .. Services and Support for Science IT: http://www.s3it.uzh.ch/ .. Spark: http://spark.apache.org/ .. SLURM: https://slurm.schedmd.com/ .. TORQUE+MAUI: http://www.adaptivecomputing.com/products/open-source/torque/ .. Ubuntu: http://www.ubuntu.com .. University of Zurich: http://www.uzh.ch .. video: http://youtu.be/cR3C7XCSMmA

.. (for Emacs only) .. Local variables: mode: rst End:

Related Repositories

bcbio-nextgen-vm

bcbio-nextgen-vm

Run bcbio-nextgen genomic sequencing analyses using isolated containers and virt ...

elasticluster-web

elasticluster-web

Web interface for elasticluster ...

elasticluster-config-tools

elasticluster-config-tools

Tools used to configure nodes in a cloud cluster created with elasticluster. ...

bcbio-nextgen-vm

bcbio-nextgen-vm

Run bcbio-nextgen genomic sequencing analyses using isolated containers and virt ...


Top Contributors

arcimboldo riccardomurri nicolasbaer mbookman pinguinkiste verdurin kern3020 corburn jsharpe nicoulaj jwm machbio cjdrake hmartiniano hammer jlothian tzachar Moisan yarikoptic chapmanb cocoy dsteinkraus

Releases

-   v1.0-rc1 zip tar
-   v0.2 zip tar
-   1.2.0 zip tar
-   1.1.1 zip tar
-   1.1.0 zip tar
-   1.0.4 zip tar
-   1.0.3 zip tar
-   1.0.2 zip tar
-   1.0.1 zip tar
-   1.0.0 zip tar