transporter 0,1,1,1,0,1,0 vagrant circleci

Sync data between persistence engines, like ETL only not stodgy

Circle CI Go Report Card

Compose helps with database transformations from one store to another. It can also sync from one to another or several stores.

Transporter

Build

make sure godep is installed, go get github.com/tools/godep and then build with godep restore godep go build -a ./cmd/...

Configure

There is a sample config in test/config.yaml. The config defines the endpoints, (either sources or sinks) that are available to the application.

api:
  interval: 60s # time interval between metrics posts to the api endpoint
  uri: "http://requestb.in/1a0zlf11"
nodes:
  localmongo:
    type: mongo
    uri: mongodb://localhost/boom
  supernick:
    type: elasticsearch
    uri: http://10.0.0.1,10.0.0.2:9200/indexname
  debug:
    type: file
    uri: stdout://
  foofile:
    type: file
    uri: file:///tmp/foo
  stdout:
    type: file
    uri: stdout://

There is also a sample ‘application.js’ in test/application.js. The application is responsible for building transporter pipelines. Given the above config, this Transporter application.js will copy from a file (in /tmp/foo) to stdout.

Source({name:"foofile"}).save({name:"stdout"})

This application.js will copy from the local mongo to a file on the local disk

Source({name:"localmongo", namespace: "boom.foo"}).save({name:"tofile"})

Transformers can also be configured in the application.js as follows

var pipeline = Source({name:"mongodb-production", namespace: "compose.milestones2"})
pipeline = pipeline.transform("transformers/transform1.js").transform("transformers/transform2.js")
pipeline.save({name:"supernick", namespace: "something/posts2"});

Run

  • list transporter list --config ./test/config.yaml
  • run transporter run --config ./test/config.yaml ./test/application.js
  • eval transporter eval --config ./test/config.yaml 'Source({name:"localmongo", namespace: "boom.foo"}).save({name:"tofile"})'
  • test transporter test --config ./test/config.yaml test/application.js

Complete beginners guide

OS X

  • ensure you have mercurial installed as it is required for a dependency
  • install the Mac OS X binary build from https://golang.org/dl/
  • follow instructions on http://golang.org/doc/install
  • VERY IMPORTANT: Go has a required directory structure which the GOPATH needs to point to. Instructions can be found on http://golang.org/doc/code.html or by typing go help gopath in terminal.
  • setup the directory structure in $GOPATH
    • cd $GOPATH; mkdir src pkg bin
    • create the github.com path and compose mkdir -p src/github.com/compose; cd src/github.com/compose
    • clone transporter git clone https://github.com/compose/transporter; cd transporter
    • make sure godep is installed, go get github.com/tools/godep
    • run godep restore to get all the dependencies as specified in Godeps.json
    • now you can build with godep go build -a ./cmd/...

At this point you should be able to run transporter via $GOPATH/bin/transporter, you may need to add $GOPATH to your PATH environment variable. Something along the lines of export PATH="$GOPATH/bin:$PATH" should work.

Vagrant

> cd transporter
> vagrant up
...
> vagrant ssh
...
vagrant> ./run-test

Windows

See READMEWINDOWS.md

Transporter in the Media

Contributing to Transporter

Want to help out with Transporter? Great! There are instructions to get you started here.

IRC

Want to talk about Transporter? Join us in #compose on irc.freenode.net!

Licensing

Transporter is licensed under the New BSD License. See LICENSE for full license text.

Related Repositories

transporter

transporter

Sync data between persistence engines, like ETL only not stodgy ...

Transporter

Transporter

A tiny library makes uploading and downloading easier ...

RMConnecter

RMConnecter

Connecter is an OS X app to help you interact with iTunes Connect’s Transporter binary, which allows you to interact with app metadata in iTunes Connect. ...

Transporter

Transporter

Modern finite-state machine implemented in pure Swift ...

itunes_store_transporter

itunes_store_transporter

Upload and manage your assets in the iTunes Store using the iTunes Store’s Transporter (iTMSTransporter). ...


Top Contributors

nstott jipperinbham alindeman trinchan codepope franklinwise Snorge sberryman Winslett mbyczkowski tomByrer chrishepner Alino letsgolesco ExNexu SukhikhN mm-

Releases

-   v0.1.2-beta+93cb201 zip tar
-   v0.1.2-beta+4c8bb7d zip tar
-   v0.1.2-beta+fc2073a zip tar
-   v0.1.1 zip tar
-   v0.1.0 zip tar
-   v0.0.4 zip tar
-   v0.0.3 zip tar
-   v0.0.2 zip tar
-   0.0.1 zip tar