Febbweiss
ab310288ef
Feature: update README.md
|
8 anni fa | |
|---|---|---|
| elasticsearch | 8 anni fa | |
| filebeat | 8 anni fa | |
| kibana | 8 anni fa | |
| logstash | 8 anni fa | |
| logstash-forwarder | 8 anni fa | |
| rsyslog | 8 anni fa | |
| .env | 8 anni fa | |
| .gitignore | 8 anni fa | |
| README.md | 8 anni fa | |
| docker-compose.yml | 8 anni fa |
README.md
This a end-to-end log centralizer powered by the ELK stask.
Embedded containers
- Filebeat - An agent to poll logs
- Logstash-Forwarder - An other agent to poll logs
- rsyslog - A standard Linux log manager
- Logstash - The collector / analyzer / parser solution
- Kafka - The queueing solution for logs
- ZooKeeper - The cluster on which Kafka is running
- ElasticSearch - The indexing engine
- Kibana - The visualization / dashboard tool for ElasticSearch
- Kafka Manager - The Kafka cluster web manager
- Apache log generator - A container generating fake apache logs
- Random log generator - A container generating text logs (Star Wars quotes)
- Java log generator - A container generating Java logs (with exception stack trace)
How it works
There are 3 agent types :
- Filebeat
- Logstash-Forward
- rsyslog
These agents push logs (from the generators) to a Logstasth shipper filling a Kafka queue (one type of log for one topic). A Logstash indexer polls the Kafka topics indexing logs into a ElasticSearch.
A short schema :
Agent -> Logstach shipper -> Kafka <- Logstash indexer -> ElasticSearch
Tools access
Kibana is available at http://localhost:5601. Kafka Manager is available at http://localhost:9000