HTTP

Sends messages over HTTP. For details on the available options of the output, see the documentation of the AxoSyslog syslog-ng distribution.

Example

A simple example sending logs over HTTP to a fluentbit HTTP endpoint:

kind: SyslogNGOutput
apiVersion: logging.banzaicloud.io/v1beta1
metadata:
  name: http
spec:
  http:
    #URL of the ingest endpoint
    url: http://fluentbit-endpoint:8080/tag
    method: POST
    headers:
      - "Content-type: application/json"

A more complex example to demonstrate sending logs to OpenObserve

kind: SyslogNGOutput
apiVersion: logging.banzaicloud.io/v1beta1
metadata:
  name: openobserve
spec:
  http:
    #URL of the ingest endpoint
    url: https://openobserve-endpoint/api/default/log-generator/_json
    user: "username"
    password:
      valueFrom:
        secretKeyRef:
          name: openobserve
          key: password
    method: POST
    # Parameters for sending logs in batches
    batch-lines: 5000
    batch-bytes: 4096
    batch-timeout: 300
    headers:
      - "Connection: keep-alive"
    # Disable TLS peer verification for demo
    tls:
      peer_verify: "no"
    body-prefix: "["
    body-suffix: "]"
    delimiter: ","
    body: "${MESSAGE}"

For details, see the documentation of the AxoSyslog syslog-ng distribution.

Configuration

headers ([]string, optional)

Custom HTTP headers to include in the request, for example, headers(“HEADER1: header1”, “HEADER2: header2”).

Default: empty

time_reopen (int, optional)

The time to wait in seconds before a dead connection is reestablished.

Default: 60

tls (*TLS, optional)

This option sets various options related to TLS encryption, for example, key/certificate files and trusted CA locations. TLS can be used only with tcp-based transport protocols. For details, see TLS for syslog-ng outputs and the documentation of the AxoSyslog syslog-ng distribution.

disk_buffer (*DiskBuffer, optional)

This option enables putting outgoing messages into the disk buffer of the destination to avoid message loss in case of a system failure on the destination side. For details, see the Syslog-ng DiskBuffer options.

Default: false

(Batch, required)

Batching parameters

body (string, optional)

The body of the HTTP request, for example, body("${ISODATE} ${MESSAGE}"). You can use strings, macros, and template functions in the body. If not set, it will contain the message received from the source by default.

body-prefix (string, optional)

The string syslog-ng OSE puts at the beginning of the body of the HTTP request, before the log message.

body-suffix (string, optional)

The string syslog-ng OSE puts to the end of the body of the HTTP request, after the log message.

delimiter (string, optional)

By default, syslog-ng OSE separates the log messages of the batch with a newline character.

log-fifo-size (int, optional)

The number of messages that the output queue can store.

method (string, optional)

Specifies the HTTP method to use when sending the message to the server. POST | PUT

password (secret.Secret, optional)

The password that syslog-ng OSE uses to authenticate on the server where it sends the messages.

persist_name (string, optional)

If you receive the following error message during syslog-ng startup, set the persist-name() option of the duplicate drivers: Error checking the uniqueness of the persist names, please override it with persist-name option. Shutting down. See the documentation of the AxoSyslog syslog-ng distribution for more information.

response-action (filter.RawArrowMap, optional)

Specifies what syslog-ng does with the log message, based on the response code received from the HTTP server. See the documentation of the AxoSyslog syslog-ng distribution for more information.

retries (int, optional)

The number of times syslog-ng OSE attempts to send a message to this destination. If syslog-ng OSE could not send a message, it will try again until the number of attempts reaches retries, then drops the message.

time_reopen (int, optional)

The time to wait in seconds before a dead connection is reestablished.

Default: 60

timeout (int, optional)

Sets the maximum number of messages sent to the destination per second. Use this output-rate-limiting functionality only when using disk-buffer as well to avoid the risk of losing messages. Specifying 0 or a lower value sets the output limit to unlimited.

tls (*TLS, optional)

This option sets various options related to TLS encryption, for example, key/certificate files and trusted CA locations. TLS can be used only with tcp-based transport protocols. For details, see TLS for syslog-ng outputs and the documentation of the AxoSyslog syslog-ng distribution.

url (string, optional)

Specifies the hostname or IP address and optionally the port number of the web service that can receive log data via HTTP. Use a colon (:) after the address to specify the port number of the server. For example: http://127.0.0.1:8000

user (string, optional)

The username that syslog-ng OSE uses to authenticate on the server where it sends the messages.

user-agent (string, optional)

The value of the USER-AGENT header in the messages sent to the server.

workers (int, optional)

Specifies the number of worker threads (at least 1) that syslog-ng OSE uses to send messages to the server. Increasing the number of worker threads can drastically improve the performance of the destination.

Batch

batch-bytes (int, optional)

Description: Sets the maximum size of payload in a batch. If the size of the messages reaches this value, syslog-ng OSE sends the batch to the destination even if the number of messages is less than the value of the batch-lines() option. Note that if the batch-timeout() option is enabled and the queue becomes empty, syslog-ng OSE flushes the messages only if batch-timeout() expires, or the batch reaches the limit set in batch-bytes().

batch-lines (int, optional)

Description: Specifies how many lines are flushed to a destination in one batch. The syslog-ng OSE application waits for this number of lines to accumulate and sends them off in a single batch. Increasing this number increases throughput as more messages are sent in a single batch, but also increases message latency. For example, if you set batch-lines() to 100, syslog-ng OSE waits for 100 messages.

batch-timeout (int, optional)

Description: Specifies the time syslog-ng OSE waits for lines to accumulate in the output buffer. The syslog-ng OSE application sends batches to the destinations evenly. The timer starts when the first message arrives to the buffer, so if only few messages arrive, syslog-ng OSE sends messages to the destination at most once every batch-timeout() milliseconds.