The role of Filebeat, in the context of PAS for OpenEdge, is to send log messages to Elasticsearch. As part of setting up Filebeat, you must minimally configure two properties--the filepaths of your log files and the connection details of Elasticsearch.
Filebeat has two key components: inputs and harvesters. The inputs component uses the filepaths that you configure to find files that need to be read. For each file, it starts a harvester. Each harvester opens its assigned file, reads it line by line, and sends the contents to Elasticsearch. It continues to do this until configuration settings in Filebeat instruct it to close.
If a log file is deleted or moved, as may happen if you use TCMAN clean or if your PAS for OpenEdge instance's log files roll over, the harvester may continue to try to read the file, reserving space on your disk until it closes. To address situations like this, you need to correctly configure Filebeat.
You must install Filebeat on every host machine that is running a PAS for OpenEdge instance that you want to monitor.
Download Filebeat for your operating system from the Download Filebeat page.
Then, install it by following the instructions for your operating system in the Install Filebeat documentation.
Configure Filebeat
To configure Filebeat, edit the Filebeat.yml configuration file that is located in the Filebeat installation directory. Minimally, you need to configure two properties:
The filepaths of the directories that contain the PAS for OpenEdge logs that you want to centralize. You may typically want to include the Tomcat server logs (catalina, localhost, localhost_access), ABL application logs, and PAS for OpenEdge agent logs.
Connection details of the Elasticsearch server. This includes the host, port, username, and password.
The following is an example that shows how to configure the filepaths and the connection details:
The filebeat.inputs: section enables you to configure the input component of Filebeat. You can configure different input types (using the - type key as in the example), including log, Stdin, etc.
You can specify multiple inputs, and you can specify the same input type more than once. You can specify individual log filenames or specify *.log, which will instruct Filebeat to read all log files in the directory.
Other input configuration options depend on the input type. The following are some key configuration options for the log input type.
Filtering logs
The following configuration options enable you to filter log messages before they are sent to Elasticsearch. These options must be specified at the same indentation level as the paths: option.
exclude_lines
A list of regular expressions that matches the lines that you want to exclude. For example, exclude_lines: ['^INFO'].
include_lines
A list of regular expressions that matches the lines that you want to include. Only lines that pass this filter will be sent to Elasticsearch. For example, include_lines: ['^ERR', '^WARN'].
exclude_files
A list of regular expressions that matches the filenames that you want Filebeat to ignore.
File handling
The following configuration options determine how the Filebeat harvesters handle files.
scan_frequency
A time period that determines how often Filebeat checks for new files in the log directory. This is applicable only if you have specified *.log for a path in the paths: configuration. Use the syntax <Time>s (for example, 10s for 10 seconds).
close_inactive
A time period that determines a period of inactivity after which Filebeat closes the file. The time period begins when the last line in the log file is read by the harvester. For example, if you specify 3s (3 seconds), and no new log messages are written to the file in the 3 seconds since the last line was read, Filebeat closes the file. Use the syntax <Time>s (to specify time in seconds), <Time>m for minutes, or <Time>h for hours.
close_removed
A Boolean value of true or false. If set to true, which is the default, Filebeat closes the file if it has been moved or deleted.
To set up Filebeat as a Windows service, run the PowerShell script install-service-filebeat located in the Filebeat installation directory. Then start Filebeat either from services.msc or by entering Start-Service filebeat in a command prompt that points to the Filebeat installation directory. To start Filebeat in the foreground in a Windows operating system, open a command prompt, change the directory to the Filebeat installation folder, and then enter filebeat.exe -e.
The filepaths of the directories that contain the PAS for OpenEdge logs that you want to centralize. You may typically want to include the Tomcat server logs (catalina, localhost, localhost_access), ABL application logs, and PAS for OpenEdge agent logs.