Inicio Explorar Axuda
Iniciar sesión
Febbweiss
/
docker-log-centralizer
réplica de https://github.com/Febbweiss/docker-log-centralizer
1
0
Fork 0
Ficheiros Incidencias 0 Wiki

Sen descrición

Árbore: 63a33b97e9
Ramas Etiquetas
docker-log-cent...
ZIP TAR.GZ
Febbweiss 63a33b97e9 Feature: add README %!s(int64=8) %!d(string=hai) anos
config e0b11ef1a2 Initial commit - Fully working architecture %!s(int64=8) %!d(string=hai) anos
elasticsearch e0b11ef1a2 Initial commit - Fully working architecture %!s(int64=8) %!d(string=hai) anos
filebeat e0b11ef1a2 Initial commit - Fully working architecture %!s(int64=8) %!d(string=hai) anos
kibana e0b11ef1a2 Initial commit - Fully working architecture %!s(int64=8) %!d(string=hai) anos
logstash 4cc77930d9 Feature: add Logstash-Forwarder as agent %!s(int64=8) %!d(string=hai) anos
logstash-forwarder 4cc77930d9 Feature: add Logstash-Forwarder as agent %!s(int64=8) %!d(string=hai) anos
.env e0b11ef1a2 Initial commit - Fully working architecture %!s(int64=8) %!d(string=hai) anos
.gitignore e0b11ef1a2 Initial commit - Fully working architecture %!s(int64=8) %!d(string=hai) anos
README.md 63a33b97e9 Feature: add README %!s(int64=8) %!d(string=hai) anos
docker-compose.yml 4cc77930d9 Feature: add Logstash-Forwarder as agent %!s(int64=8) %!d(string=hai) anos

README.md

This a end-to-end log centralizer powered by the ELK stask.

Embedded containers

  1. Filebeat - An agent to poll logs
  2. Logstash-Forwarder - An other agent to poll logs
  3. Logstash - The collector / analyzer / parser solution
  4. Kafka - The queueing solution for logs
  5. ZooKeeper - The cluster on which Kafka is running
  6. ElasticSearch - The indexing engine
  7. Kibana - The visualization / dashboard tool for ElasticSearch
  8. Kafka Manager - The Kafka cluster web manager

How it works

There are 2 agent types :

  • Filebeat
  • Logstash-Forward

These agents push logs to a Logstasth shipper filling a Kafka queue (one type of log for one topic). A Logstash indexer polls the Kafka topics indexing logs into a ElasticSearch.

A short schema :

Agent -> Logstach shipper -> Kafka <- Logstash indexer -> ElasticSearch

Tools access

Kibana is available at http://localhost:5601. Kafka Manager is available at http://localhost:9000