Sample Processor
Description
The Sample processor filters incoming events at a specified rate. This reduces the total number of events that are used in a subsequent processor, or sent to a destination.
Use
The Sample processor is useful when the number of events is in excess of the number required to understand the data. For instance, when determining performance metrics over time it's often sufficient to have one measurement per second rather than per millisecond.
Configuration
The number of events for which a sample is set is computed based on an included rate number. The number of events forwarded is determined by the ratio 1/N, where N is the rate. The minimum sampling rate is 2, meaning every 1 of 2 events would be forwarded.
Data can be excluded from the sampling algorithm and always forwarded through the pipeline based on a conditional match of JSON key-value pairs. Data that matches the condition will always be retained.
Options
| Option | Description | Example |
|---|---|---|
| Rate | The rate at which events will be forwarded, expressed as 1/n. | rate = 10 means 1 out of every 10 events will be forwarded and the rest will be dropped |
| Always Include | Define a Field and conditions for log data to always be forwarded regardless of the Rate. |