For developers

This documentation helps to set-up a developer environment and writing plugins for the Logging operator.

Setting up Kind

1. Install Kind on your computer

   go get sigs.k8s.io/kind@v0.5.1
   

2. Create cluster

   kind create cluster --name logging
   

3. Install prerequisites (this is a Kubebuilder makefile that will generate and install crds)

   make install
   

4. Run the Operator

   go run main.go
   

Writing a plugin

To add a plugin to the logging operator you need to define the plugin struct.

Note: Place your plugin in the corresponding directory pkg/sdk/model/filter or pkg/sdk/model/output

type MyExampleOutput struct {
	// Path that is required for the plugin
	Path string `json:"path,omitempty"`
}

The plugin uses the JSON tags to parse and validate configuration. Without tags the configuration is not valid. The fluent parameter name must match with the JSON tag. Don’t forget to use omitempty for non required parameters.

Implement ToDirective

To render the configuration you have to implement the ToDirective function.

func (c *S3OutputConfig) ToDirective(secretLoader secret.SecretLoader) (types.Directive, error) {
	...
}

For simple Plugins you can use the NewFlatDirective function.

func (c *ExampleOutput) ToDirective(secretLoader secret.SecretLoader) (types.Directive, error) {
	return types.NewFlatDirective(types.PluginMeta{
		Type:      "example",
		Directive: "output",
		Tags: "**",
	}, c, secretLoader)
}

For more example please check the available plugins.

Reuse existing Plugin sections

You can embed existing configuration for your plugins. For example modern Output plugins have Buffer section.

// +docLink:"Buffer,./buffer.md"
Buffer *Buffer `json:"buffer,omitempty"`

If you are using embedded sections you must call its ToDirective method manually and append it as a SubDirective

if c.Buffer != nil {
	if buffer, err := c.Buffer.ToDirective(secretLoader); err != nil {
		return nil, err
	} else {
		s3.SubDirectives = append(s3.SubDirectives, buffer)
	}
}

Special plugin tags

To document the plugins logging-operator uses the Go tags (like JSON tags). Logging operator uses plugin named tags for special instructions.

Special tag default

The default tag helps to give default values for parameters. These parameters are explicitly set in the generated fluentd configuration.

RetryForever bool `json:"retry_forever" plugin:"default:true"`

Special tag required

The required tag ensures that the attribute cannot be empty

RetryForever bool `json:"retry_forever" plugin:"required"`

Generate documentation for Plugin

The operator parse the docstrings for the documentation.

...
// AWS access key id
AwsAccessKey *secret.Secret `json:"aws_key_id,omitempty"`
...

Will generate the following Markdown

You can hint default values in docstring via (default: value). This is useful if you don’t want to set default explicitly with tag. However during rendering defaults in tags have priority over docstring.

...
// The format of S3 object keys (default: %{path}%{time_slice}_%{index}.%{file_extension})
S3ObjectKeyFormat string `json:"s3_object_key_format,omitempty"`
...

Special docstrings

• +docName:"Title for the plugin section"
• +docLink:"Buffer,./buffer.md"

You can declare document title and description above the type _doc* interface{} variable declaration.

Example Document headings:

// +docName:"Amazon S3 plugin for Fluentd"
// **s3** output plugin buffers event logs in local file and upload it to S3 periodically. This plugin splits files exactly by using the time of event logs (not the time when the logs are received). For example, a log '2011-01-02 message B' is reached, and then another log '2011-01-03 message B' is reached in this order, the former one is stored in "20110102.gz" file, and latter one in "20110103.gz" file.
type _docS3 interface{}

Example Plugin headings:

// +kubebuilder:object:generate=true
// +docName:"Shared Credentials"
type S3SharedCredentials struct {
...

Example linking embedded sections

// +docLink:"Buffer,./buffer.md"
Buffer *Buffer `json:"buffer,omitempty"`

Generate docs for your Plugin

make docs