Fluentd: Open-Source Log Collector

Code Climate

Fluentd collects events from various data sources and writes them to files, RDBMS, NoSQL, IaaS, SaaS, Hadoop and so on. Fluentd helps you unify your logging infrastructure (Learn more about the Unified Logging Layer).

An event consists of tag, time and record. Tag is a string separated with '.' (e.g. myapp.access). It is used to categorize events. Time is a UNIX time recorded at occurrence of an event. Record is a JSON object.

Example Use Cases

Use Case Description Diagram
Centralizing Apache/Nginx Server Logs Fluentd can be used to tail access/error logs and transport them reliably to remote systems.
Syslog Alerting Fluentd can "grep" for events and send out alerts.
Mobile/Web Application Logging Fluentd can function as middleware to enable asynchronous, scalable logging for user action events.

Quick Start

$ gem install fluentd
$ fluentd -s conf
$ fluentd -c conf/fluent.conf &
$ echo '{"json":"message"}' | fluent-cat debug.test

Development

Prerequisites

  • Ruby 2.1 or later
  • git

git should be in PATH. On Windows, you can use Github for Windows and GitShell for easy setup.

Install dependent gems

Use bundler:

$ gem install bundler
$ bundle install --path vendor/bundle

Run test

$ bundle exec rake test

You can run specified test via TEST environment variable:

$ bundle exec rake test TEST=test/test_specified_path.rb
$ bundle exec rake test TEST=test/test_*.rb

Fluentd UI: Admin GUI

Fluentd UI is a graphical user interface to start/stop/configure Fluentd.

More Information

Contributors:

Patches contributed by great developers.

Related Repositories

fluentd-ui

fluentd-ui

Web UI for Fluentd ...

fluentd-docker-image

fluentd-docker-image

Docker image for Fluentd ...

fluentd-server

fluentd-server

Fluentd config distribution server ...

ansible-fluentd

ansible-fluentd

An Ansible role that installs Fluentd, in the format for Ansible Galaxy ...

django-fluentd

django-fluentd

django-fluentd allows you to use django's logging framework to log directly to a ...


Top Contributors

tagomoris repeatedly frsyuki okkez cosmo0920 sonots ganmacs okahashi117 mururu nurse kzk kiyoto kazegusuri itiut MasayukiKiyota naritta kentaro hotchpotch llby toyama0919 y-ken komamitsu Telmo edsiper unak mr-salty miyakawataku mzp a-robinson ajohnstone

Releases

-   v0.14.6 zip tar
-   v0.14.5 zip tar
-   v0.14.4 zip tar
-   v0.14.2 zip tar
-   v0.14.1 zip tar
-   v0.14.0 zip tar
-   v0.14.0.rc.3 zip tar
-   v0.14.0.rc.2 zip tar
-   v0.14.0.rc.1 zip tar
-   v0.14.0.pre.1 zip tar
-   v0.12.29 zip tar
-   v0.12.28 zip tar
-   v0.12.27 zip tar
-   v0.12.26 zip tar
-   v0.12.25 zip tar
-   v0.12.24 zip tar
-   v0.12.23 zip tar
-   v0.12.22 zip tar
-   v0.12.21 zip tar
-   v0.12.20 zip tar
-   v0.12.19 zip tar
-   v0.12.18 zip tar
-   v0.12.17 zip tar
-   v0.12.16 zip tar
-   v0.12.15 zip tar
-   v0.12.14 zip tar
-   v0.12.13 zip tar
-   v0.12.12 zip tar
-   v0.12.11 zip tar
-   v0.12.10 zip tar