# Filebeat

*Created by Seokje Park at 2020.02.21*

keyword

* `centralized logging`
* `log shipper`

## What is Beats ?

* Beats are collection of lightweight and open source log shipper
* Beats are built on top of a Go framework called libbeat.
* It acts as agents installed on the different servers for collecting lgos or metrics
* Filebeat : Collects log files
* Packetbeat : Collects network data
* Metricbeat : Collects server metrics
* Once collected, the data is sent to either directly into Elasticsearch or to Logstash.

## Quick start

Described on AWS Linux(AMI) basis.

* Download and install the public signing key:

  ```bash
  $ sudo rpm --import https://packages.elastic.co/GPG-KEY-elasticsearch
  ```
* Create `elastic.repo` and add the following lines in your /etc/yum.repos.d/ directory:

  ```bash
  [elastic-7.x]
  name=Elastic repository for 7.x packages
  baseurl=https://artifacts.elastic.co/packages/7.x/yum
  gpgcheck=1
  gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
  enabled=1
  autorefresh=1
  type=rpm-md
  ```
* Modify `filebeat.yml` in /etc/filebeat/filebeat.yml:

  ```
  filebeat.inputs:
    - type: log
      enabled: true
      paths:
        - /home/ec2-user/.pm2/[APPNAME]*.log
  output.elasticsearch:
    hosts: ["<es_url>"]
  username: "elastic"
  password: "<password>"
  setup.kibana:
    host: "<kibana_url>"
  ```
* Add the following line `/etc/filebeat/modules.d/system.yml.disabled`: (Optional)

  This way you send log events in Elasticsearch with a UTC timestamp. Kibana can simply convert from UTC to whatever timezone you browser is in at request time.

  ```bash
  var.convert_timezone: true
  ```
* Enable logstash module

  ```bash
  $ sudo filebeat modules enable logstash
  ```
* Setup and start Filebeat

  ```bash
  $ sudo filebeat setup # A new index will be created in Elasticsearch which you can define in Kibana
  $ sudo service filebeat start
  $ sudo chkconfig --add filebeat # Start filebeat automatically during boot
  ```
* Kibana Management > Index Patterns create index pattern `filebeat-*`
* In Kibana Discover, change current index pattern to `filebeat-*` and set date filters.
* Click Refresh. if you see data, you made it. Congratulation!

## 참고

* [A Beats Tutorial](https://logz.io/blog/beats-tutorial/)
* [Filebeat and AWS Elasticsearch](https://www.partiallydisassembled.net/posts/filebeat-kibana-aws.html)
* [Repositories for APT and YUM](https://www.elastic.co/guide/en/beats/filebeat/current/setup-repositories.html)
* [System Log Aggregation with the Elastic Stack](https://linuxacademy.com/blog/certifications/system-log-aggregation-with-the-elastic-stack/)


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://molla4455.gitbook.io/dev-log/elastic/filebeat.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.