ANK - Python Microservices

Overview:

Python Microservices for Queue services (rabbitMQ, Kafka, zeroMQ), Streaming, REST-API and Schedule task.

Requirements:

  • Python 2.x
  • Python 3.x

How to use:

  • Install:

    • From github:
      • clone this project: git clone [email protected]:sunary/ank.git
      • install: python setup.py install
    • From pypi:
      • pip install ank
  • Create new Service:

    $ ank -c TestService`
    $ cd TestService
    

    then continue using below commands

  • Edit app (processor.py):

    • Example: “`python from apps.app import BaseApp

    class ExampleApp(BaseApp):

    def run(self, process=None):
        '''
        Implement this if your App start by this chain
        Arguments:
            process: process method, was assign into self._process
    
    
        '''
        super(TestApp, self).run(process)
    
    
        for i in range(100):
            self._process(i)
    
    
    def process(self, message=None):
        # after processed, message will be return for next chain
        message += 1
        return message
    

    ”`

  • Edit services and chains (services.yml):

    • Syntax: “`yaml services: Object1:
      • class: module.name.ClassName
      • arguments: [$Object, %variable%]

    AnkChain2: - class: chains.module_name.Chain - arguments: ~

    chains: - Object1 - AnkChain2

    * *Example:*
    ```yaml
    services:
      WorkerClass:
        class: processor.DemoApp
        arguments: [$Mongodb, $Redis, '%batch_size%']
    
    
      Mongodb:
        class: utilities.my_mongo.Mongodb
        arguments: ['%mongo_db%', '%mongo_col%', '%mongo_host%', '%mongo_port%']
    
    
      Redis:
        class: redis.client.StrictRedis
        arguments: ['%redis_host%', '%redis_port%']
    
    
      OtherWorker:
        class: processor.OtherApp
        arguments: ~
    
    
      LogHandle:
        class: chains.log_handle.LogHandle
        arguments: ~
    
    
    chains:
      - WorkerClass
      - LogHandle
      - OtherWorker
    

    ANK will read top-down chains, find correspond services and get parameters from settings.yml.

  • Generate and edit setting (settings.yml):

     $ ank -s
    
    • Example: “`yaml parameters: mongo_host: localhost mongo_port: 27017 mongo_db: crawl_db mongo_col: twitter

    redis_host: localhost redis_port: 6379

    queue_uri: ‘amqp://admin:[email protected]:5672/’ exchange_name: InputExchange routing_key: ExchangeToQueue

    batch_size: 100 “ Help you createsettingstemplate file. Just rename from_settings.ymltosettings.yml` and fill in values.

  • Build Service (create docker image):

    $ ank -b
    docker run --entrypoint /bin/sh docker_image_id
    
  • Generate processor (_processor.py):

    $ ank -p
    
  • Edit and test Service (test_service.py):

    $ ank -t -f test-settings.yml
    
  • Run Service:

    $ ank -r
    

Apps:

  • App: Normal App.
  • API App: REST-API using flask.
  • Schedule App: Using crontab-time format to set schedule.

Chains:

  • LogHandle: Log every messages.
  • JoinProcessor: Join messages into one.
  • SplitProcessor: Split message.
  • —Consumer: Get message from queue.
  • —Producer: Push message to queue.

Related Repositories

Ank-Pixiv-Tool

Ank-Pixiv-Tool

An extension which supports one-click downloads of illustrations from Pixiv. It's able to choose automatically appropriate file names. It also supports full screen view of large illustrations on medium-sized screens. ...

ank

ank

ANK - Python Microservices ...

ANK-NG

ANK-NG

Initial fork to create AutoNetkit v3 ...

Ank-Pixiv-Tool

Ank-Pixiv-Tool

ここに置いてある自家branchは、fork元本家にマージ依頼を出すまでの、評価中の中間成果物です。最終成果物ではないので随時削除されます。 ...

ank-pr

ank-pr

...