scrapy travis-ci python

Scrapy, a fast high-level screen scraping and web crawling framework for Python.

3 years after

====== Scrapy

.. image:: https://secure.travis-ci.org/scrapy/scrapy.png?branch=master :target: http://travis-ci.org/scrapy/scrapy

Overview

Scrapy is a fast high-level screen scraping and web crawling framework, used to crawl websites and extract structured data from their pages. It can be used for a wide range of purposes, from data mining to monitoring and automated testing.

For more information including a list of features check the Scrapy homepage at: http://scrapy.org

Requirements

  • Python 2.6 or up
  • Works on Linux, Windows, Mac OSX, BSD

Install

The quick way::

pip install scrapy

For more details see the install section in the documentation: http://doc.scrapy.org/en/latest/intro/install.html

Releases

You can download the latest stable and development releases from: http://scrapy.org/download/

Documentation

Documentation is available online at http://doc.scrapy.org/ and in the docs directory.

Community (blog, twitter, mail list, IRC)

See http://scrapy.org/community/

Contributing

See http://doc.scrapy.org/en/latest/contributing.html

Companies using Scrapy

See http://scrapy.org/companies/

Commercial Support

See http://scrapy.org/support/

Related Repositories

scrapy-redis

scrapy-redis

Redis-based components for Scrapy. ...

scrapy-splash

scrapy-splash

Scrapy+Splash for JavaScript integration ...

scrapy-cluster

scrapy-cluster

This Scrapy project uses Redis and Kafka to create a distributed on demand scrap ...

scrapy-mongodb

scrapy-mongodb

MongoDB pipeline for Scrapy. This module supports both MongoDB in standalone set ...

scrapy-djangoitem

scrapy-djangoitem

Scrapy extension to write scraped items using Django models ...


Top Contributors

pablohoffman dangra invalid-email-address void omab rolando andrix pkufranky alexcepoi shaneaevans stav pedrofaustino kalessin calarval nramirezuy LucianU enparticular Mimino666 llonchj msabramo djm nuklea coagulant mhlakhani emschorsch euphoris jsyeo sejo brunsgaard opyate

Releases

-   0.17.0 zip tar
-   0.16.4 zip tar
-   0.16.3 zip tar
-   0.16.2 zip tar
-   0.16.1 zip tar
-   0.16.0 zip tar
-   0.15.1 zip tar
-   0.15.0 zip tar
-   0.14.4 zip tar
-   0.14.3 zip tar
-   0.14.2 zip tar
-   0.14.1 zip tar
-   0.14.0 zip tar
-   0.10.3 zip tar
-   0.10.2 zip tar
-   0.10.1 zip tar
-   0.10 zip tar
-   0.10-rc1 zip tar
-   0.9 zip tar
-   0.9-rc1 zip tar
-   0.8 zip tar
-   0.7 zip tar
-   0.7-rc1 zip tar