# Filebeat

*Created by Seokje Park at 2020.02.21*

keyword

* `centralized logging`
* `log shipper`

## What is Beats ?

* Beats are collection of lightweight and open source log shipper
* Beats are built on top of a Go framework called libbeat.
* It acts as agents installed on the different servers for collecting lgos or metrics
* Filebeat : Collects log files
* Packetbeat : Collects network data
* Metricbeat : Collects server metrics
* Once collected, the data is sent to either directly into Elasticsearch or to Logstash.

## Quick start

Described on AWS Linux(AMI) basis.

* Download and install the public signing key:

  ```bash
  $ sudo rpm --import https://packages.elastic.co/GPG-KEY-elasticsearch
  ```
* Create `elastic.repo` and add the following lines in your /etc/yum.repos.d/ directory:

  ```bash
  [elastic-7.x]
  name=Elastic repository for 7.x packages
  baseurl=https://artifacts.elastic.co/packages/7.x/yum
  gpgcheck=1
  gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
  enabled=1
  autorefresh=1
  type=rpm-md
  ```
* Modify `filebeat.yml` in /etc/filebeat/filebeat.yml:

  ```
  filebeat.inputs:
    - type: log
      enabled: true
      paths:
        - /home/ec2-user/.pm2/[APPNAME]*.log
  output.elasticsearch:
    hosts: ["<es_url>"]
  username: "elastic"
  password: "<password>"
  setup.kibana:
    host: "<kibana_url>"
  ```
* Add the following line `/etc/filebeat/modules.d/system.yml.disabled`: (Optional)

  This way you send log events in Elasticsearch with a UTC timestamp. Kibana can simply convert from UTC to whatever timezone you browser is in at request time.

  ```bash
  var.convert_timezone: true
  ```
* Enable logstash module

  ```bash
  $ sudo filebeat modules enable logstash
  ```
* Setup and start Filebeat

  ```bash
  $ sudo filebeat setup # A new index will be created in Elasticsearch which you can define in Kibana
  $ sudo service filebeat start
  $ sudo chkconfig --add filebeat # Start filebeat automatically during boot
  ```
* Kibana Management > Index Patterns create index pattern `filebeat-*`
* In Kibana Discover, change current index pattern to `filebeat-*` and set date filters.
* Click Refresh. if you see data, you made it. Congratulation!

## 참고

* [A Beats Tutorial](https://logz.io/blog/beats-tutorial/)
* [Filebeat and AWS Elasticsearch](https://www.partiallydisassembled.net/posts/filebeat-kibana-aws.html)
* [Repositories for APT and YUM](https://www.elastic.co/guide/en/beats/filebeat/current/setup-repositories.html)
* [System Log Aggregation with the Elastic Stack](https://linuxacademy.com/blog/certifications/system-log-aggregation-with-the-elastic-stack/)