A quick project to load some realtime filtered Twitter API data into Elasticsearch, and then visualize the results in Kibana.
Uses an ELK (Elasticsearch/Logstash/Kibana) Docker container for easy reproducibility. Tested with Python 2.7 (Anaconda Distro) on Windows 10 for this project.
To get Elasticsearch and Kibana up and running quickly, run these docker commands:
docker pull sebp/elk:es235_l234_k454 docker run -p 5601:5601 -p 9200:9200 -p 5044:5044 -p 5000:5000 -it --name elk sebp/elk:es235_l234_k454
After the Docker container is up and running, you should be able to hit these 2 URLs if everything is working properly:
There is 1 step you’ll want to do in the Kibana Web GUI. Put in add “twitteranalysis” in the Add Index prompt.
After cloning this repo, run pip install to get all of the Python libraries:
pip install -r requirements.txt
For reference: Python libraries used:
After filling in your Twitter API credentials in the configlocal.py, you can let the script run for a few minutes and load up the Index in Elasticsearch. After some data is loaded up, try a few URL’s like this: