> ## Documentation Index > Fetch the complete documentation index at: https://docs.mezmo.com/llms.txt > Use this file to discover all available pages before exploring further. # Initialize a pipeline with provided components. All sources, transforms, and sinks are linked. > Populates and publishes a new pipeline as specified. ## OpenAPI ````yaml /pipeline.yaml post /pipeline/with-components openapi: 3.1.0 info: title: Pipeline API description: API for the Pipeline product version: 3.345.5 servers: - url: https://api.mezmo.com/v3 security: - PipelineServiceKey: [] tags: - name: Internal description: Routes not accessible by outside users - name: Experimental description: Experimental routes, not intended for public consumption - name: Account description: Perform account level actions - name: Alerting description: API to control alerts on pipelines - name: Data Classification & Profiling description: API for classifying and profiling data - name: Edge - Local Deploy description: User routes that are available to use with local Edge instances - name: Incident Mode description: Change or read state of pipeline (incident mode on/off) - name: Log Volume Reduction description: CRUD operations for log volume reduction - name: Metrics description: Usage metrics for pipelines - name: Pipeline Export description: Export pipeline to terraform - name: Pipeline Management description: CRUD Operations for pipeline - name: Pipeline Nodes description: CRUD operations for sources, processors, and destinations - name: Processor Groups description: CRUD operations for Processor Groups - name: Shared Sources description: Create and manage sources for use across multiple pipelines - name: Sampling and Simulation description: >- Create samples of data and run them through a simulated pipeline without affecting your published pipeline externalDocs: url: https://docs.mezmo.com/pipeline-api paths: /pipeline/with-components: post: tags: - Pipeline Management summary: >- Initialize a pipeline with provided components. All sources, transforms, and sinks are linked. description: Populates and publishes a new pipeline as specified. requestBody: content: application/json: schema: type: object additionalProperties: false properties: origin: type: string enum: - ui - system default: system pipeline: oneOf: - type: 'null' - type: object properties: id: type: string format: uuid title: oneOf: - type: string minLength: 1 maxLength: 512 pattern: ^[^<>"]+$ - type: 'null' errorMessage: invalid title name sources: type: array minItems: 1 items: type: object allOf: - if: properties: type: enum: - http deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: HTTP description: Receive data from incoming HTTP requests additionalProperties: false properties: capture_metadata: type: boolean title: Capture Metadata description: >- Enable the inclusion of all http headers and query string parameters that were sent from the source. ui: feature_flag: pipeline-capture-metadata display_type: hidden default: true auto_parse: type: object title: Configure automated parsing additionalProperties: false ui: display_type: hidden properties: enabled: type: boolean default: false enabled_event_types: type: array items: type: string enum: - HTTPD_COMBINEDLOG - HTTPD_COMMONLOG - HTTPD_ERRORLOG - SYSLOG5424LINE - SYSLOGLINE - SYSLOGPAMSESSION - CRONLOG - MONGO3_LOG - NAGIOSLOGLINE - POSTGRESQL - RAILS3 - RUBY_LOGGER - REDISLOG - REDISMONLOG - S3_ACCESS_LOG - ELB_ACCESS_LOG - CLOUDFRONT_ACCESS_LOG - CATALINALOG - TOMCATLOG - SQUID3 - BIND9 - HAPROXYTCP - HAPROXYHTTP - BACULA_LOGLINE - BRO_HTTP - BRO_DNS - BRO_CONN - BRO_FILES - NETSCREENSESSIONLOG - CISCO_TAGGED_SYSLOG - CISCOFW104001 - CISCOFW104002 - CISCOFW104003 - CISCOFW104004 - CISCOFW105003 - CISCOFW105004 - CISCOFW105005 - CISCOFW105008 - CISCOFW105009 - CISCOFW106001 - CISCOFW106006_106007_106010 - CISCOFW106014 - CISCOFW106015 - CISCOFW106021 - CISCOFW106023 - CISCOFW106100_2_3 - CISCOFW106100 - CISCOFW304001 - CISCOFW110002 - CISCOFW302010 - CISCOFW302013_302014_302015_302016 - CISCOFW302020_302021 - CISCOFW305011 - CISCOFW313001_313004_313008 - CISCOFW313005 - CISCOFW321001 - CISCOFW402117 - CISCOFW402119 - CISCOFW419001 - CISCOFW419002 - CISCOFW500004 - CISCOFW602303_602304 - >- CISCOFW710001_710002_710003_710005_710006 - CISCOFW713172 - CISCOFW733100 - SHOREWALL - SFW2 decoding: oneOf: - enum: - bytes - enum: - json - enum: - ndjson - enum: - auto ui: display_type: list errorMessage: must be equal to one of the allowed values description: >- The decoding method for converting frames into data events. title: Decoding Method default: auto required: - decoding - if: properties: type: enum: - splunk-hec deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Splunk HEC description: Receive Splunk logs additionalProperties: false properties: capture_metadata: type: boolean title: Capture Metadata description: >- Enable the inclusion of all http headers and query string parameters that were sent from the source. ui: feature_flag: pipeline-capture-metadata display_type: hidden default: true auto_parse: type: object title: Configure automated parsing additionalProperties: false ui: display_type: hidden properties: enabled: type: boolean default: false enabled_event_types: type: array items: type: string enum: - HTTPD_COMBINEDLOG - HTTPD_COMMONLOG - HTTPD_ERRORLOG - SYSLOG5424LINE - SYSLOGLINE - SYSLOGPAMSESSION - CRONLOG - MONGO3_LOG - NAGIOSLOGLINE - POSTGRESQL - RAILS3 - RUBY_LOGGER - REDISLOG - REDISMONLOG - S3_ACCESS_LOG - ELB_ACCESS_LOG - CLOUDFRONT_ACCESS_LOG - CATALINALOG - TOMCATLOG - SQUID3 - BIND9 - HAPROXYTCP - HAPROXYHTTP - BACULA_LOGLINE - BRO_HTTP - BRO_DNS - BRO_CONN - BRO_FILES - NETSCREENSESSIONLOG - CISCO_TAGGED_SYSLOG - CISCOFW104001 - CISCOFW104002 - CISCOFW104003 - CISCOFW104004 - CISCOFW105003 - CISCOFW105004 - CISCOFW105005 - CISCOFW105008 - CISCOFW105009 - CISCOFW106001 - CISCOFW106006_106007_106010 - CISCOFW106014 - CISCOFW106015 - CISCOFW106021 - CISCOFW106023 - CISCOFW106100_2_3 - CISCOFW106100 - CISCOFW304001 - CISCOFW110002 - CISCOFW302010 - CISCOFW302013_302014_302015_302016 - CISCOFW302020_302021 - CISCOFW305011 - CISCOFW313001_313004_313008 - CISCOFW313005 - CISCOFW321001 - CISCOFW402117 - CISCOFW402119 - CISCOFW419001 - CISCOFW419002 - CISCOFW500004 - CISCOFW602303_602304 - >- CISCOFW710001_710002_710003_710005_710006 - CISCOFW713172 - CISCOFW733100 - SHOREWALL - SFW2 - if: properties: type: enum: - s3 deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: AWS S3 description: Collect data from object contents in AWS S3 additionalProperties: false properties: sqs_queue_url: type: string title: SQS Queue URL description: >- The URL of a AWS SQS queue configured to receive S3 bucket notifications for the desired S3 buckets format: uri errorMessage: format: Must be a valid URI auth: type: object title: AWS Authentication description: Configures AWS authentication additionalProperties: false properties: access_key_id: type: string title: Access Key ID minLength: 1 description: The AWS access key id secret_access_key: type: string title: Secret Access Key minLength: 1 description: The AWS secret access key ui: display_type: secret required: - access_key_id - secret_access_key compression: type: string title: Compression description: The compression format of the S3 objects enum: - auto - gzip - none - zstd default: auto region: type: string title: Region minLength: 1 description: The name of the source AWS region. required: - sqs_queue_url - auth - compression - region - if: properties: type: enum: - sqs deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: AWS SQS description: Collect messages from AWS SQS additionalProperties: false properties: queue_url: type: string title: Queue URL description: The URL of an AWS SQS queue format: uri errorMessage: format: Must be a valid URI maxLength: 128 auth: type: object title: AWS Authentication description: Configures AWS authentication additionalProperties: false properties: access_key_id: type: string title: Access Key ID minLength: 1 description: The AWS access key id maxLength: 128 secret_access_key: type: string title: Secret Access Key minLength: 1 description: The AWS secret access key ui: display_type: secret maxLength: 128 required: - access_key_id - secret_access_key region: type: string title: Region minLength: 1 description: The name of the source AWS region. maxLength: 128 required: - queue_url - auth - region - if: properties: type: enum: - demo-logs deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: title: Demo Logs description: >- Generate log messages for testing and demonstration. additionalProperties: type: array title: Log Lines description: >- A list of user-provided log lines that will be sent through the pipeline infinitely. A maximum of 25 lines is allowed. The lines can be either plain text or ndjson. If ndjson is detected, the encoding for the source will be set to JSON, and lines that are not valid will show up as a visible component error. Otherwise, text encoding will be used. minItems: 1 maxItems: 25 items: type: string minLength: 1 maxLength: 4096 title: Line description: >- A single line of log data, either stringified JSON or plain text. ui: display_type: user-provided-lines allOf: - if: properties: format: enum: - user_provided required: - format then: properties: lines: type: array title: Log Lines description: >- A list of user-provided log lines that will be sent through the pipeline infinitely. A maximum of 25 lines is allowed. The lines can be either plain text or ndjson. If ndjson is detected, the encoding for the source will be set to JSON, and lines that are not valid will show up as a visible component error. Otherwise, text encoding will be used. minItems: 1 maxItems: 25 items: type: string minLength: 1 maxLength: 4096 title: Line description: >- A single line of log data, either stringified JSON or plain text. ui: display_type: user-provided-lines required: - lines - allOf: - errorMessage: >- `lines` is only valid when `format` is `user_provided` if: not: type: object properties: format: enum: - user_provided required: - format then: not: type: object properties: lines: type: array title: Log Lines description: >- A list of user-provided log lines that will be sent through the pipeline infinitely. A maximum of 25 lines is allowed. The lines can be either plain text or ndjson. If ndjson is detected, the encoding for the source will be set to JSON, and lines that are not valid will show up as a visible component error. Otherwise, text encoding will be used. minItems: 1 maxItems: 25 items: type: string minLength: 1 maxLength: 4096 title: Line description: >- A single line of log data, either stringified JSON or plain text. ui: display_type: user-provided-lines required: - lines properties: interval_ms: type: number title: Interval description: >- The amount of time to pause between each demo log. Fractions of a second are accepted to increase throughput. e.g. 500 ms == 120 lines per minute default: 1000 minimum: 500 maximum: 3600000 ui: display_type: milliseconds-to-human-readable format: title: Format description: >- The format of the log messages. All of the data is generated. ui: display_type: list errorMessage: >- must be one of the available types of demo data oneOf: - label: HTTP JSON description: HTTP request data in JSON format. enum: - json - label: Sensor JSON description: >- Data from environment sensors in JSON format. enum: - env_sensor - label: Financial JSON description: Financial data in JSON format. enum: - financial - label: Nginx Requests description: Nginx logs in text format. enum: - nginx - label: Apache Requests description: Apache request logs in text format. enum: - apache_common - label: Apache Errors description: Apache error logs in text format. enum: - apache_error - label: BSD Syslog description: >- Mock BSD syslog entries in rfc3164 text format. enum: - bsd_syslog - label: Syslog description: >- Mock syslog entries in rfc5424 text format. enum: - syslog - label: HTTP Metrics description: >- HTTP-based metrics in the Mezmo metrics JSON format. enum: - http_metrics - label: Generic Metrics description: >- Generic metrics in the Mezmo metrics JSON format enum: - generic_metrics - label: User-Provided Logs description: >- Custom logs provided by the user. These can either be plain text or serialized JSON. enum: - user_provided - label: Infrastructure Logs description: >- Infrastructure logs consisting of sysdig, nginx, HDFS and spark logs enum: - infrastructure - label: Kubernetes Logs description: Kubernetes logs in JSON format enum: - kubernetes default: json required: - format - if: properties: type: enum: - kinesis-firehose deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Kinesis Firehose description: Receive Kinesis Firehose data additionalProperties: false properties: capture_metadata: type: boolean title: Capture Metadata description: >- Enable the inclusion of all http headers and query string parameters that were sent from the source. ui: feature_flag: pipeline-capture-metadata display_type: hidden default: true auto_parse: type: object title: Configure automated parsing additionalProperties: false ui: display_type: hidden properties: enabled: type: boolean default: false enabled_event_types: type: array items: type: string enum: - HTTPD_COMBINEDLOG - HTTPD_COMMONLOG - HTTPD_ERRORLOG - SYSLOG5424LINE - SYSLOGLINE - SYSLOGPAMSESSION - CRONLOG - MONGO3_LOG - NAGIOSLOGLINE - POSTGRESQL - RAILS3 - RUBY_LOGGER - REDISLOG - REDISMONLOG - S3_ACCESS_LOG - ELB_ACCESS_LOG - CLOUDFRONT_ACCESS_LOG - CATALINALOG - TOMCATLOG - SQUID3 - BIND9 - HAPROXYTCP - HAPROXYHTTP - BACULA_LOGLINE - BRO_HTTP - BRO_DNS - BRO_CONN - BRO_FILES - NETSCREENSESSIONLOG - CISCO_TAGGED_SYSLOG - CISCOFW104001 - CISCOFW104002 - CISCOFW104003 - CISCOFW104004 - CISCOFW105003 - CISCOFW105004 - CISCOFW105005 - CISCOFW105008 - CISCOFW105009 - CISCOFW106001 - CISCOFW106006_106007_106010 - CISCOFW106014 - CISCOFW106015 - CISCOFW106021 - CISCOFW106023 - CISCOFW106100_2_3 - CISCOFW106100 - CISCOFW304001 - CISCOFW110002 - CISCOFW302010 - CISCOFW302013_302014_302015_302016 - CISCOFW302020_302021 - CISCOFW305011 - CISCOFW313001_313004_313008 - CISCOFW313005 - CISCOFW321001 - CISCOFW402117 - CISCOFW402119 - CISCOFW419001 - CISCOFW419002 - CISCOFW500004 - CISCOFW602303_602304 - >- CISCOFW710001_710002_710003_710005_710006 - CISCOFW713172 - CISCOFW733100 - SHOREWALL - SFW2 format: type: string enum: - json - text default: json title: Data Format description: >- This specifies what the data format will be after it is base64 decoded. If it is JSON, it will be automatically parsed. required: - format - if: properties: type: enum: - fluent deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Fluent (Fluentd, Fluent Bit) description: Receive data from Fluentd or Fluent Bit additionalProperties: false properties: capture_metadata: type: boolean title: Capture Metadata description: >- Enable the inclusion of all http headers and query string parameters that were sent from the source. ui: feature_flag: pipeline-capture-metadata display_type: hidden default: true auto_parse: type: object title: Configure automated parsing additionalProperties: false ui: display_type: hidden properties: enabled: type: boolean default: false enabled_event_types: type: array items: type: string enum: - HTTPD_COMBINEDLOG - HTTPD_COMMONLOG - HTTPD_ERRORLOG - SYSLOG5424LINE - SYSLOGLINE - SYSLOGPAMSESSION - CRONLOG - MONGO3_LOG - NAGIOSLOGLINE - POSTGRESQL - RAILS3 - RUBY_LOGGER - REDISLOG - REDISMONLOG - S3_ACCESS_LOG - ELB_ACCESS_LOG - CLOUDFRONT_ACCESS_LOG - CATALINALOG - TOMCATLOG - SQUID3 - BIND9 - HAPROXYTCP - HAPROXYHTTP - BACULA_LOGLINE - BRO_HTTP - BRO_DNS - BRO_CONN - BRO_FILES - NETSCREENSESSIONLOG - CISCO_TAGGED_SYSLOG - CISCOFW104001 - CISCOFW104002 - CISCOFW104003 - CISCOFW104004 - CISCOFW105003 - CISCOFW105004 - CISCOFW105005 - CISCOFW105008 - CISCOFW105009 - CISCOFW106001 - CISCOFW106006_106007_106010 - CISCOFW106014 - CISCOFW106015 - CISCOFW106021 - CISCOFW106023 - CISCOFW106100_2_3 - CISCOFW106100 - CISCOFW304001 - CISCOFW110002 - CISCOFW302010 - CISCOFW302013_302014_302015_302016 - CISCOFW302020_302021 - CISCOFW305011 - CISCOFW313001_313004_313008 - CISCOFW313005 - CISCOFW321001 - CISCOFW402117 - CISCOFW402119 - CISCOFW419001 - CISCOFW419002 - CISCOFW500004 - CISCOFW602303_602304 - >- CISCOFW710001_710002_710003_710005_710006 - CISCOFW713172 - CISCOFW733100 - SHOREWALL - SFW2 decoding: type: string description: >- The decoding method for converting frames into data events. enum: - bytes - json - ndjson title: Decoding Method default: json required: - decoding - if: properties: type: enum: - log-analysis deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Mezmo Log Analysis description: >- Receive data directly from your Mezmo Log Analysis account additionalProperties: false - if: properties: type: enum: - log-analysis-ingestion deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Mezmo Log Analysis Ingestion description: >- Receive unparsed source data directly from your Mezmo Log Analysis account. additionalProperties: type: string - if: properties: type: enum: - logstash deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Logstash description: Receive Logstash data additionalProperties: false properties: capture_metadata: type: boolean title: Capture Metadata description: >- Enable the inclusion of all http headers and query string parameters that were sent from the source. ui: feature_flag: pipeline-capture-metadata display_type: hidden default: true auto_parse: type: object title: Configure automated parsing additionalProperties: false ui: display_type: hidden properties: enabled: type: boolean default: false enabled_event_types: type: array items: type: string enum: - HTTPD_COMBINEDLOG - HTTPD_COMMONLOG - HTTPD_ERRORLOG - SYSLOG5424LINE - SYSLOGLINE - SYSLOGPAMSESSION - CRONLOG - MONGO3_LOG - NAGIOSLOGLINE - POSTGRESQL - RAILS3 - RUBY_LOGGER - REDISLOG - REDISMONLOG - S3_ACCESS_LOG - ELB_ACCESS_LOG - CLOUDFRONT_ACCESS_LOG - CATALINALOG - TOMCATLOG - SQUID3 - BIND9 - HAPROXYTCP - HAPROXYHTTP - BACULA_LOGLINE - BRO_HTTP - BRO_DNS - BRO_CONN - BRO_FILES - NETSCREENSESSIONLOG - CISCO_TAGGED_SYSLOG - CISCOFW104001 - CISCOFW104002 - CISCOFW104003 - CISCOFW104004 - CISCOFW105003 - CISCOFW105004 - CISCOFW105005 - CISCOFW105008 - CISCOFW105009 - CISCOFW106001 - CISCOFW106006_106007_106010 - CISCOFW106014 - CISCOFW106015 - CISCOFW106021 - CISCOFW106023 - CISCOFW106100_2_3 - CISCOFW106100 - CISCOFW304001 - CISCOFW110002 - CISCOFW302010 - CISCOFW302013_302014_302015_302016 - CISCOFW302020_302021 - CISCOFW305011 - CISCOFW313001_313004_313008 - CISCOFW313005 - CISCOFW321001 - CISCOFW402117 - CISCOFW402119 - CISCOFW419001 - CISCOFW419002 - CISCOFW500004 - CISCOFW602303_602304 - >- CISCOFW710001_710002_710003_710005_710006 - CISCOFW713172 - CISCOFW733100 - SHOREWALL - SFW2 format: type: string enum: - json - text default: json title: Data Format description: The format of the logstash data required: - format - if: properties: type: enum: - mezmo-agent deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Mezmo Agent description: Receive logs from the Mezmo Agent additionalProperties: false properties: capture_metadata: type: boolean title: Capture Metadata description: >- Enable the inclusion of all http headers and query string parameters that were sent from the source. ui: feature_flag: pipeline-capture-metadata display_type: hidden default: true auto_parse: type: object title: Configure automated parsing additionalProperties: false ui: display_type: hidden properties: enabled: type: boolean default: false enabled_event_types: type: array items: type: string enum: - HTTPD_COMBINEDLOG - HTTPD_COMMONLOG - HTTPD_ERRORLOG - SYSLOG5424LINE - SYSLOGLINE - SYSLOGPAMSESSION - CRONLOG - MONGO3_LOG - NAGIOSLOGLINE - POSTGRESQL - RAILS3 - RUBY_LOGGER - REDISLOG - REDISMONLOG - S3_ACCESS_LOG - ELB_ACCESS_LOG - CLOUDFRONT_ACCESS_LOG - CATALINALOG - TOMCATLOG - SQUID3 - BIND9 - HAPROXYTCP - HAPROXYHTTP - BACULA_LOGLINE - BRO_HTTP - BRO_DNS - BRO_CONN - BRO_FILES - NETSCREENSESSIONLOG - CISCO_TAGGED_SYSLOG - CISCOFW104001 - CISCOFW104002 - CISCOFW104003 - CISCOFW104004 - CISCOFW105003 - CISCOFW105004 - CISCOFW105005 - CISCOFW105008 - CISCOFW105009 - CISCOFW106001 - CISCOFW106006_106007_106010 - CISCOFW106014 - CISCOFW106015 - CISCOFW106021 - CISCOFW106023 - CISCOFW106100_2_3 - CISCOFW106100 - CISCOFW304001 - CISCOFW110002 - CISCOFW302010 - CISCOFW302013_302014_302015_302016 - CISCOFW302020_302021 - CISCOFW305011 - CISCOFW313001_313004_313008 - CISCOFW313005 - CISCOFW321001 - CISCOFW402117 - CISCOFW402119 - CISCOFW419001 - CISCOFW419002 - CISCOFW500004 - CISCOFW602303_602304 - >- CISCOFW710001_710002_710003_710005_710006 - CISCOFW713172 - CISCOFW733100 - SHOREWALL - SFW2 - if: properties: type: enum: - mezmo-datadog-source deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Datadog Agent description: >- Send logs and metrics data directly from an installed datadog agent additionalProperties: false properties: capture_metadata: type: boolean title: Capture Metadata description: >- Enable the inclusion of all http headers and query string parameters that were sent from the source. ui: feature_flag: pipeline-capture-metadata display_type: hidden default: true auto_parse: type: object title: Configure automated parsing additionalProperties: false ui: display_type: hidden properties: enabled: type: boolean default: false enabled_event_types: type: array items: type: string enum: - HTTPD_COMBINEDLOG - HTTPD_COMMONLOG - HTTPD_ERRORLOG - SYSLOG5424LINE - SYSLOGLINE - SYSLOGPAMSESSION - CRONLOG - MONGO3_LOG - NAGIOSLOGLINE - POSTGRESQL - RAILS3 - RUBY_LOGGER - REDISLOG - REDISMONLOG - S3_ACCESS_LOG - ELB_ACCESS_LOG - CLOUDFRONT_ACCESS_LOG - CATALINALOG - TOMCATLOG - SQUID3 - BIND9 - HAPROXYTCP - HAPROXYHTTP - BACULA_LOGLINE - BRO_HTTP - BRO_DNS - BRO_CONN - BRO_FILES - NETSCREENSESSIONLOG - CISCO_TAGGED_SYSLOG - CISCOFW104001 - CISCOFW104002 - CISCOFW104003 - CISCOFW104004 - CISCOFW105003 - CISCOFW105004 - CISCOFW105005 - CISCOFW105008 - CISCOFW105009 - CISCOFW106001 - CISCOFW106006_106007_106010 - CISCOFW106014 - CISCOFW106015 - CISCOFW106021 - CISCOFW106023 - CISCOFW106100_2_3 - CISCOFW106100 - CISCOFW304001 - CISCOFW110002 - CISCOFW302010 - CISCOFW302013_302014_302015_302016 - CISCOFW302020_302021 - CISCOFW305011 - CISCOFW313001_313004_313008 - CISCOFW313005 - CISCOFW321001 - CISCOFW402117 - CISCOFW402119 - CISCOFW419001 - CISCOFW419002 - CISCOFW500004 - CISCOFW602303_602304 - >- CISCOFW710001_710002_710003_710005_710006 - CISCOFW713172 - CISCOFW733100 - SHOREWALL - SFW2 - if: properties: type: enum: - pipeline-data-restoration deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Mezmo Pipeline Data Restoration description: >- Receive data directly from your Mezmo Pipline Data Restoration additionalProperties: false - if: properties: type: enum: - webhook deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Webhook (WebSub) description: >- Receive data from incoming webhooks using the WebSub protocol. additionalProperties: false properties: capture_metadata: type: boolean title: Capture Metadata description: >- Enable the inclusion of all http headers and query string parameters that were sent from the source. ui: feature_flag: pipeline-capture-metadata display_type: hidden default: true auto_parse: type: object title: Configure automated parsing additionalProperties: false ui: display_type: hidden properties: enabled: type: boolean default: false enabled_event_types: type: array items: type: string enum: - HTTPD_COMBINEDLOG - HTTPD_COMMONLOG - HTTPD_ERRORLOG - SYSLOG5424LINE - SYSLOGLINE - SYSLOGPAMSESSION - CRONLOG - MONGO3_LOG - NAGIOSLOGLINE - POSTGRESQL - RAILS3 - RUBY_LOGGER - REDISLOG - REDISMONLOG - S3_ACCESS_LOG - ELB_ACCESS_LOG - CLOUDFRONT_ACCESS_LOG - CATALINALOG - TOMCATLOG - SQUID3 - BIND9 - HAPROXYTCP - HAPROXYHTTP - BACULA_LOGLINE - BRO_HTTP - BRO_DNS - BRO_CONN - BRO_FILES - NETSCREENSESSIONLOG - CISCO_TAGGED_SYSLOG - CISCOFW104001 - CISCOFW104002 - CISCOFW104003 - CISCOFW104004 - CISCOFW105003 - CISCOFW105004 - CISCOFW105005 - CISCOFW105008 - CISCOFW105009 - CISCOFW106001 - CISCOFW106006_106007_106010 - CISCOFW106014 - CISCOFW106015 - CISCOFW106021 - CISCOFW106023 - CISCOFW106100_2_3 - CISCOFW106100 - CISCOFW304001 - CISCOFW110002 - CISCOFW302010 - CISCOFW302013_302014_302015_302016 - CISCOFW302020_302021 - CISCOFW305011 - CISCOFW313001_313004_313008 - CISCOFW313005 - CISCOFW321001 - CISCOFW402117 - CISCOFW402119 - CISCOFW419001 - CISCOFW419002 - CISCOFW500004 - CISCOFW602303_602304 - >- CISCOFW710001_710002_710003_710005_710006 - CISCOFW713172 - CISCOFW733100 - SHOREWALL - SFW2 - if: properties: type: enum: - kafka deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Kafka Source description: Collect events from Kafka compatible brokers additionalProperties: type: string properties: brokers: type: array title: Brokers uniqueItems: true minItems: 1 description: >- An array of key/val objects for each host/port of a Kafka broker. items: type: object title: Host/Port description: The host and port for the Kafka broker. additionalProperties: false properties: host: title: Host description: >- The hostname or IP address of the broker. anyOf: - type: string format: ipv4 errorMessage: format: Must be a valid IPv4 address - type: string format: hostname maxLength: 512 errorMessage: format: Must be a valid hostname port: type: number minimum: 1 maximum: 65535 title: Port description: The port that the broker listens on. required: - host - port topics: type: array title: Topics items: type: string title: Topic minLength: 1 maxLength: 256 description: The Kafka topic name. minItems: 1 description: >- List of Kafka topics names to read events from. group_id: type: string title: Consumer Group minLength: 1 description: >- The consumer group name to be used to consume events from Kafka. tls_enabled: type: boolean title: TLS enabled description: >- Whether or not to require TLS for outgoing connections. default: true sasl_enabled: type: boolean title: SASL/SCRAM Enabled description: Enable SASL/SCRAM authentication with Kafka default: true decoding_codec: type: string enum: - bytes - json title: Decoding description: >- Configures how events are decoded from raw bytes default: bytes if: properties: sasl_enabled: enum: - true then: properties: sasl_mechanism: type: string enum: - SCRAM-SHA-512 - SCRAM-SHA-256 - PLAIN title: SASL description: The Kafka SASL/SCRAM mechanism default: PLAIN sasl_username: type: string title: Username description: The SASL/SCRAM username minLength: 1 sasl_password: type: string title: Password description: The SASL/SCRAM username ui: display_type: secret minLength: 1 required: - sasl_mechanism - sasl_username - sasl_password required: - brokers - topics - group_id - decoding_codec - if: properties: type: enum: - azure-event-hub deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Azure Event Hub Source description: >- Collect events from Azure Event Hub using Azure Kafka Endpoint additionalProperties: false properties: connection_string: type: string title: Connection String minLength: 1 maxLength: 512 description: >- The Connection String as it appears in hub consumer SAS Policy ui: display_type: secret namespace: type: string title: Namespace minLength: 1 maxLength: 256 description: The Event Hub Namespace topics: type: array title: Event Hub Names items: type: string title: Name minLength: 1 maxLength: 256 description: Azure Event Hub instance name. minItems: 1 description: >- The list of Azure Event Hub name(s) to read events from. group_id: type: string title: Consumer Group minLength: 1 maxLength: 256 description: >- The consumer group name that this consumer belongs to. decoding_codec: type: string enum: - bytes - json title: Decoding description: >- Configures how events are decoded from raw bytes default: bytes required: - connection_string - namespace - topics - group_id - decoding_codec - if: properties: type: enum: - prometheus-remote-write deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Prometheus Remote Write description: >- Receive prometheus remote write protobuf data from HTTP requests additionalProperties: false properties: capture_metadata: type: boolean title: Capture Metadata description: >- Enable the inclusion of all http headers and query string parameters that were sent from the source. ui: feature_flag: pipeline-capture-metadata display_type: hidden default: true auto_parse: type: object title: Configure automated parsing additionalProperties: false ui: display_type: hidden properties: enabled: type: boolean default: false enabled_event_types: type: array items: type: string enum: - HTTPD_COMBINEDLOG - HTTPD_COMMONLOG - HTTPD_ERRORLOG - SYSLOG5424LINE - SYSLOGLINE - SYSLOGPAMSESSION - CRONLOG - MONGO3_LOG - NAGIOSLOGLINE - POSTGRESQL - RAILS3 - RUBY_LOGGER - REDISLOG - REDISMONLOG - S3_ACCESS_LOG - ELB_ACCESS_LOG - CLOUDFRONT_ACCESS_LOG - CATALINALOG - TOMCATLOG - SQUID3 - BIND9 - HAPROXYTCP - HAPROXYHTTP - BACULA_LOGLINE - BRO_HTTP - BRO_DNS - BRO_CONN - BRO_FILES - NETSCREENSESSIONLOG - CISCO_TAGGED_SYSLOG - CISCOFW104001 - CISCOFW104002 - CISCOFW104003 - CISCOFW104004 - CISCOFW105003 - CISCOFW105004 - CISCOFW105005 - CISCOFW105008 - CISCOFW105009 - CISCOFW106001 - CISCOFW106006_106007_106010 - CISCOFW106014 - CISCOFW106015 - CISCOFW106021 - CISCOFW106023 - CISCOFW106100_2_3 - CISCOFW106100 - CISCOFW304001 - CISCOFW110002 - CISCOFW302010 - CISCOFW302013_302014_302015_302016 - CISCOFW302020_302021 - CISCOFW305011 - CISCOFW313001_313004_313008 - CISCOFW313005 - CISCOFW321001 - CISCOFW402117 - CISCOFW402119 - CISCOFW419001 - CISCOFW419002 - CISCOFW500004 - CISCOFW602303_602304 - >- CISCOFW710001_710002_710003_710005_710006 - CISCOFW713172 - CISCOFW733100 - SHOREWALL - SFW2 - if: properties: type: enum: - open-telemetry-metrics deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: OpenTelemetry Metrics description: Receive metrics from OpenTelemetry over HTTP additionalProperties: false properties: capture_metadata: type: boolean title: Capture Metadata description: >- Enable the inclusion of all http headers and query string parameters that were sent from the source. ui: feature_flag: pipeline-capture-metadata display_type: hidden default: true auto_parse: type: object title: Configure automated parsing additionalProperties: false ui: display_type: hidden properties: enabled: type: boolean default: false enabled_event_types: type: array items: type: string enum: - HTTPD_COMBINEDLOG - HTTPD_COMMONLOG - HTTPD_ERRORLOG - SYSLOG5424LINE - SYSLOGLINE - SYSLOGPAMSESSION - CRONLOG - MONGO3_LOG - NAGIOSLOGLINE - POSTGRESQL - RAILS3 - RUBY_LOGGER - REDISLOG - REDISMONLOG - S3_ACCESS_LOG - ELB_ACCESS_LOG - CLOUDFRONT_ACCESS_LOG - CATALINALOG - TOMCATLOG - SQUID3 - BIND9 - HAPROXYTCP - HAPROXYHTTP - BACULA_LOGLINE - BRO_HTTP - BRO_DNS - BRO_CONN - BRO_FILES - NETSCREENSESSIONLOG - CISCO_TAGGED_SYSLOG - CISCOFW104001 - CISCOFW104002 - CISCOFW104003 - CISCOFW104004 - CISCOFW105003 - CISCOFW105004 - CISCOFW105005 - CISCOFW105008 - CISCOFW105009 - CISCOFW106001 - CISCOFW106006_106007_106010 - CISCOFW106014 - CISCOFW106015 - CISCOFW106021 - CISCOFW106023 - CISCOFW106100_2_3 - CISCOFW106100 - CISCOFW304001 - CISCOFW110002 - CISCOFW302010 - CISCOFW302013_302014_302015_302016 - CISCOFW302020_302021 - CISCOFW305011 - CISCOFW313001_313004_313008 - CISCOFW313005 - CISCOFW321001 - CISCOFW402117 - CISCOFW402119 - CISCOFW419001 - CISCOFW419002 - CISCOFW500004 - CISCOFW602303_602304 - >- CISCOFW710001_710002_710003_710005_710006 - CISCOFW713172 - CISCOFW733100 - SHOREWALL - SFW2 - if: properties: type: enum: - open-telemetry-logs deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: OpenTelemetry Logs description: Receive logs from OpenTelemetry over HTTP additionalProperties: false properties: capture_metadata: type: boolean title: Capture Metadata description: >- Enable the inclusion of all http headers and query string parameters that were sent from the source. ui: feature_flag: pipeline-capture-metadata display_type: hidden default: true auto_parse: type: object title: Configure automated parsing additionalProperties: false ui: display_type: hidden properties: enabled: type: boolean default: false enabled_event_types: type: array items: type: string enum: - HTTPD_COMBINEDLOG - HTTPD_COMMONLOG - HTTPD_ERRORLOG - SYSLOG5424LINE - SYSLOGLINE - SYSLOGPAMSESSION - CRONLOG - MONGO3_LOG - NAGIOSLOGLINE - POSTGRESQL - RAILS3 - RUBY_LOGGER - REDISLOG - REDISMONLOG - S3_ACCESS_LOG - ELB_ACCESS_LOG - CLOUDFRONT_ACCESS_LOG - CATALINALOG - TOMCATLOG - SQUID3 - BIND9 - HAPROXYTCP - HAPROXYHTTP - BACULA_LOGLINE - BRO_HTTP - BRO_DNS - BRO_CONN - BRO_FILES - NETSCREENSESSIONLOG - CISCO_TAGGED_SYSLOG - CISCOFW104001 - CISCOFW104002 - CISCOFW104003 - CISCOFW104004 - CISCOFW105003 - CISCOFW105004 - CISCOFW105005 - CISCOFW105008 - CISCOFW105009 - CISCOFW106001 - CISCOFW106006_106007_106010 - CISCOFW106014 - CISCOFW106015 - CISCOFW106021 - CISCOFW106023 - CISCOFW106100_2_3 - CISCOFW106100 - CISCOFW304001 - CISCOFW110002 - CISCOFW302010 - CISCOFW302013_302014_302015_302016 - CISCOFW302020_302021 - CISCOFW305011 - CISCOFW313001_313004_313008 - CISCOFW313005 - CISCOFW321001 - CISCOFW402117 - CISCOFW402119 - CISCOFW419001 - CISCOFW419002 - CISCOFW500004 - CISCOFW602303_602304 - >- CISCOFW710001_710002_710003_710005_710006 - CISCOFW713172 - CISCOFW733100 - SHOREWALL - SFW2 - if: properties: type: enum: - open-telemetry-traces deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: OpenTelemetry Traces description: Receive traces from OpenTelemetry over HTTP additionalProperties: false properties: capture_metadata: type: boolean title: Capture Metadata description: >- Enable the inclusion of all http headers and query string parameters that were sent from the source. ui: feature_flag: pipeline-capture-metadata display_type: hidden default: true auto_parse: type: object title: Configure automated parsing additionalProperties: false ui: display_type: hidden properties: enabled: type: boolean default: false enabled_event_types: type: array items: type: string enum: - HTTPD_COMBINEDLOG - HTTPD_COMMONLOG - HTTPD_ERRORLOG - SYSLOG5424LINE - SYSLOGLINE - SYSLOGPAMSESSION - CRONLOG - MONGO3_LOG - NAGIOSLOGLINE - POSTGRESQL - RAILS3 - RUBY_LOGGER - REDISLOG - REDISMONLOG - S3_ACCESS_LOG - ELB_ACCESS_LOG - CLOUDFRONT_ACCESS_LOG - CATALINALOG - TOMCATLOG - SQUID3 - BIND9 - HAPROXYTCP - HAPROXYHTTP - BACULA_LOGLINE - BRO_HTTP - BRO_DNS - BRO_CONN - BRO_FILES - NETSCREENSESSIONLOG - CISCO_TAGGED_SYSLOG - CISCOFW104001 - CISCOFW104002 - CISCOFW104003 - CISCOFW104004 - CISCOFW105003 - CISCOFW105004 - CISCOFW105005 - CISCOFW105008 - CISCOFW105009 - CISCOFW106001 - CISCOFW106006_106007_106010 - CISCOFW106014 - CISCOFW106015 - CISCOFW106021 - CISCOFW106023 - CISCOFW106100_2_3 - CISCOFW106100 - CISCOFW304001 - CISCOFW110002 - CISCOFW302010 - CISCOFW302013_302014_302015_302016 - CISCOFW302020_302021 - CISCOFW305011 - CISCOFW313001_313004_313008 - CISCOFW313005 - CISCOFW321001 - CISCOFW402117 - CISCOFW402119 - CISCOFW419001 - CISCOFW419002 - CISCOFW500004 - CISCOFW602303_602304 - >- CISCOFW710001_710002_710003_710005_710006 - CISCOFW713172 - CISCOFW733100 - SHOREWALL - SFW2 properties: title: oneOf: - type: string minLength: 1 maxLength: 512 pattern: ^[^<>"]+$ - type: 'null' errorMessage: invalid title name description: oneOf: - type: string minLength: 1 pattern: ^[^<>"]+$ - type: 'null' errorMessage: invalid description type: type: string enum: - http - splunk-hec - s3 - sqs - demo-logs - kinesis-firehose - fluent - log-analysis - log-analysis-ingestion - logstash - mezmo-agent - mezmo-datadog-source - pipeline-data-restoration - webhook - kafka - azure-event-hub - prometheus-remote-write - open-telemetry-metrics - open-telemetry-logs - open-telemetry-traces user_config: type: object required: - user_config transforms: type: array minItems: 1 items: type: object allOf: - if: properties: type: enum: - filter deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Filter description: >- Define condition(s) to include or exclude events from the pipeline additionalProperties: false properties: action: title: Action description: How to handle events matching this criteria oneOf: - label: Allow events matching this criteria enum: - allow - label: Drop events matching this criteria enum: - drop default: allow ui: display_type: radio-group conditional: allOf: - $ref: '#condition-group' title: Conditional description: >- A group of expressions joined together by an and/or operator. ui: display_type: rules-engine required: - action - conditional - if: properties: type: enum: - sample deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Sample description: >- Sample data at a given rate, retaining only a subset of data events for further processing. additionalProperties: false properties: rate: type: integer title: Rate description: >- The rate at which events will be forwarded, expressed as 1/N. For example, `rate = 10` means 1 out of every 10 events will be forwarded and the rest will be dropped minimum: 2 maximum: 10000 default: 10 always_include: title: Always Include description: >- Certain events may always want to be included (not sampled), regardless of the sample rate. Events matching this criteria will always show up in the results. additionalProperties: type: - string - number dependencies: field: - str_operator str_operator: - field allOf: - if: title: Operator properties: str_operator: type: string enum: - greater - greater_or_equal - less - less_or_equal ui: display_type: list-operators title: Operator description: The comparison operator required: - str_operator then: title: Value properties: value: errorMessage: pattern: Value must be a number type: - string - number title: Value description: >- The value used in the expression, ie the "operand" minLength: 1 pattern: ^\s*-?(\d*\.)?\d+\s*$ required: - value - if: title: Operator properties: str_operator: type: string enum: - greater - greater_or_equal - less - less_or_equal - equal - contains - is_ip_in_cidr_range - ends_with - starts_with - drain_matches - contains_token_term - contains_token_prefix - percent_change_greater - percent_change_greater_or_equal - percent_change_less - percent_change_less_or_equal - value_change_greater - value_change_greater_or_equal - value_change_less - value_change_less_or_equal ui: display_type: list-operators title: Operator description: The comparison operator required: - str_operator then: title: Value properties: value: type: - string - number title: Value description: >- The value used in the expression, ie the "operand" minLength: 1 required: - value - if: title: Operator properties: str_operator: type: string enum: - contains - equal - ends_with - starts_with - drain_matches - contains_token_term - contains_token_prefix ui: display_type: list-operators title: Operator required: - str_operator then: title: Case sensitive properties: case_sensitive: type: boolean title: Case sensitive description: Perform case sensitive comparison? default: true properties: field: type: string title: Field description: >- The JSON field to use in a condition to always include in sampling pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference str_operator: type: string enum: - greater - greater_or_equal - less - less_or_equal - equal - contains - is_ip_in_cidr_range - is_metric - is_array - is_boolean - is_empty - is_null - is_number - is_object - is_string - exists - ends_with - starts_with - contains_token_term - contains_token_prefix - regex_match ui: display_type: list-operators title: Operator description: The comparison operator negate: type: boolean default: false title: Negate description: Negate the comparison expression required: - rate - if: properties: type: enum: - drop-fields deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Remove Fields description: Remove fields additionalProperties: false properties: fields: type: array title: Fields description: A list of fields to be removed items: type: string title: Field pattern: >- ^(?:metadata|\.|(?:metadata|message)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ minLength: 1 errorMessage: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields). Note that dropping an event's message or timestamp is not allowed minItems: 1 required: - fields - if: properties: type: enum: - parse deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: title: Parse description: Parse a specified field using the chosen parser additionalProperties: type: object errorMessage: properties: target_field: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) allOf: - if: properties: parser: enum: - parse_key_value then: properties: options: $ref: '#key-value-parser-options' - if: properties: parser: enum: - parse_timestamp then: properties: options: $ref: '#timestamp-parser-options' required: - options - if: properties: parser: enum: - parse_grok then: properties: options: $ref: '#grok-pattern-parser-options' required: - options - if: properties: parser: enum: - parse_regex then: properties: options: allOf: - $ref: '#regex-parser-options' ui: display_type: regex-pattern required: - options - if: properties: parser: enum: - parse_apache_log then: properties: options: $ref: '#apache-parser-options' required: - options - if: properties: parser: enum: - parse_nginx_log then: properties: options: $ref: '#nginx-parser-options' required: - options - if: properties: parser: enum: - parse_cef then: properties: options: $ref: '#cef-parser-options' - if: properties: parser: enum: - parse_csv then: properties: options: $ref: '#csv-parser-options' properties: field: type: string title: Field description: The JSON field whose value should be parsed. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ default: . errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference target_field: oneOf: - type: 'null' - type: string pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ maxLength: 512 errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference title: Target field description: >- The field into which the parsed value should be inserted. Leave blank to insert the parsed data into the original field. default: null parser: oneOf: - label: AWS Cloudfront Access Log group: Line Parsers ui: {} enum: - aws_cloudfront_access_log - label: AWS Cloudwatch Log group: Line Parsers ui: {} enum: - >- parse_aws_cloudwatch_log_subscription_message - label: AWS Load Balancer Log group: Line Parsers ui: {} enum: - parse_aws_alb_log - label: AWS S3 Access Log group: Line Parsers ui: {} enum: - aws_s3_access_log - label: Apache Log group: Line Parsers ui: {} enum: - parse_apache_log - label: CSV group: Line Parsers ui: {} enum: - parse_csv - label: Common Event Format (CEF) Log group: Line Parsers ui: {} enum: - parse_cef - label: Common Log group: Line Parsers ui: {} enum: - parse_common_log - label: Elasticsearch Log group: Line Parsers ui: {} enum: - elasticsearch_log - label: Golang Log group: Line Parsers ui: {} enum: - golang_log - label: Google Log group: Line Parsers ui: {} enum: - parse_glog - label: Grok Pattern group: Line Parsers ui: {} enum: - parse_grok - label: Integer group: Field Parsers ui: {} enum: - parse_int - label: JSON group: Line Parsers ui: {} enum: - parse_json - label: Key/Value group: Line Parsers ui: {} enum: - parse_key_value - label: Kubernetes Log group: Line Parsers ui: {} enum: - parse_klog - label: Linux Authorization group: Line Parsers ui: {} enum: - parse_linux_authorization - label: Mongo Log group: Line Parsers ui: {} enum: - mongo_log - label: Mongo3 Log group: Line Parsers ui: {} enum: - mongo3_log - label: Nginx Log group: Line Parsers ui: {} enum: - parse_nginx_log - label: PSQL Log group: Line Parsers ui: {} enum: - psql_log - label: Query String group: Field Parsers ui: {} enum: - parse_query_string - label: Regular Expression group: Line Parsers ui: feature_flag: pipeline-parser-regex enum: - parse_regex - label: Syslog group: Line Parsers ui: {} enum: - parse_syslog - label: Timestamp group: Field Parsers ui: {} enum: - parse_timestamp - label: Tokens group: Field Parsers ui: {} enum: - parse_tokens - label: URL group: Field Parsers ui: {} enum: - parse_url - label: User Agent group: Field Parsers ui: {} enum: - parse_user_agent - label: VPC Flow Log group: Line Parsers ui: {} description: >- Parse VPC logs using default format for fields from v2 to v4 enum: - parse_aws_vpc_flow_log ui: display_type: list errorMessage: must be one of allowed parsers title: Parser description: >- The kind of parser to use against the input value from "field". required: - field - parser - if: properties: type: enum: - parse-sequentially deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Parse Sequentially description: >- Parse fields using one or more parsers in order, sending out the event on the first successful match. additionalProperties: false errorMessage: properties: target_field: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) properties: field: type: string title: Field description: The field whose value should be parsed. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ default: . maxLength: 512 errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference target_field: oneOf: - type: 'null' - type: string pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ maxLength: 512 errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference title: Target field description: >- The field into which the parsed value should be inserted. Leave blank to insert the parsed data into the original field. default: null parsers: type: array title: List of parsers description: >- The list of parsers to use in order against the input valuefrom "field", short-circuiting on the first successful match. items: type: object allOf: - if: properties: parser: enum: - parse_key_value then: properties: options: $ref: '#key-value-parser-options' - if: properties: parser: enum: - parse_timestamp then: properties: options: $ref: '#timestamp-parser-options' required: - options - if: properties: parser: enum: - parse_grok then: properties: options: $ref: '#grok-pattern-parser-options' required: - options - if: properties: parser: enum: - parse_regex then: properties: options: allOf: - $ref: '#regex-parser-options' ui: display_type: regex-pattern required: - options - if: properties: parser: enum: - parse_apache_log then: properties: options: $ref: '#apache-parser-options' required: - options - if: properties: parser: enum: - parse_nginx_log then: properties: options: $ref: '#nginx-parser-options' required: - options - if: properties: parser: enum: - parse_cef then: properties: options: $ref: '#cef-parser-options' - if: properties: parser: enum: - parse_csv then: properties: options: $ref: '#csv-parser-options' properties: label: oneOf: - type: 'null' - type: string maxLength: 20 title: Title description: >- An arbitrary name you choose to identify the results of this processor for use when connecting them to other components. parser: oneOf: - label: AWS Cloudfront Access Log group: Line Parsers ui: {} enum: - aws_cloudfront_access_log - label: AWS Cloudwatch Log group: Line Parsers ui: {} enum: - >- parse_aws_cloudwatch_log_subscription_message - label: AWS Load Balancer Log group: Line Parsers ui: {} enum: - parse_aws_alb_log - label: AWS S3 Access Log group: Line Parsers ui: {} enum: - aws_s3_access_log - label: Apache Log group: Line Parsers ui: {} enum: - parse_apache_log - label: CSV group: Line Parsers ui: {} enum: - parse_csv - label: Common Event Format (CEF) Log group: Line Parsers ui: {} enum: - parse_cef - label: Common Log group: Line Parsers ui: {} enum: - parse_common_log - label: Elasticsearch Log group: Line Parsers ui: {} enum: - elasticsearch_log - label: Golang Log group: Line Parsers ui: {} enum: - golang_log - label: Google Log group: Line Parsers ui: {} enum: - parse_glog - label: Grok Pattern group: Line Parsers ui: {} enum: - parse_grok - label: Integer group: Field Parsers ui: {} enum: - parse_int - label: JSON group: Line Parsers ui: {} enum: - parse_json - label: Key/Value group: Line Parsers ui: {} enum: - parse_key_value - label: Kubernetes Log group: Line Parsers ui: {} enum: - parse_klog - label: Linux Authorization group: Line Parsers ui: {} enum: - parse_linux_authorization - label: Mongo Log group: Line Parsers ui: {} enum: - mongo_log - label: Mongo3 Log group: Line Parsers ui: {} enum: - mongo3_log - label: Nginx Log group: Line Parsers ui: {} enum: - parse_nginx_log - label: PSQL Log group: Line Parsers ui: {} enum: - psql_log - label: Query String group: Field Parsers ui: {} enum: - parse_query_string - label: Regular Expression group: Line Parsers ui: feature_flag: pipeline-parser-regex enum: - parse_regex - label: Syslog group: Line Parsers ui: {} enum: - parse_syslog - label: Timestamp group: Field Parsers ui: {} enum: - parse_timestamp - label: Tokens group: Field Parsers ui: {} enum: - parse_tokens - label: URL group: Field Parsers ui: {} enum: - parse_url - label: User Agent group: Field Parsers ui: {} enum: - parse_user_agent - label: VPC Flow Log group: Line Parsers ui: {} description: >- Parse VPC logs using default format for fields from v2 to v4 enum: - parse_aws_vpc_flow_log ui: display_type: list errorMessage: must be one of allowed parsers title: Parser description: >- The kind of parser to use against the input value from "field". _output_name: type: string title: '[INTERNAL]' description: >- Internal-only field that holds a static id for the output_name. This way, labels for outputs can be updated independent of the id. default: '' required: - parser - _output_name additionalProperties: type: object minItems: 1 ui: display_type: multi-output-map unique_parser_and_options: true errorMessage: uniqueItemProperties: parser can not be defined more than once required: - field - parsers - if: properties: type: enum: - route deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Route description: >- Route data based on whether it matches (or doesn't match) logical comparisons additionalProperties: false properties: conditionals: type: array minItems: 1 title: Conditionals description: >- A list of complete conditionals, including expressions and label items: type: object additionalProperties: false title: Conditional description: A list of conditions and a label properties: conditional: $ref: '#condition-group' label: type: string minLength: 1 maxLength: 255 title: Label description: >- An arbitrary name you choose to identify the results of this processor for use when connecting them to other components. _output_name: type: string title: '[INTERNAL]' description: >- Internal-only field that holds a static id for the output_name. This way, labels for outputs can be updated independent of the id. default: '' required: - conditional - label - _output_name required: - conditionals - if: properties: type: enum: - stringify deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Stringify description: Stringify messages as JSON additionalProperties: false - if: properties: type: enum: - compact-fields deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Compact Fields description: Remove empty values from a list of fields additionalProperties: false properties: fields: type: array title: Fields description: A list of fields to remove empty values from items: type: string title: Field description: Field to remove empty value from pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference minItems: 1 options: type: object additionalProperties: false title: Compact Options description: >- Additional options for removing empty values from fields properties: compact_array: type: boolean title: Compact Array description: Remove empty arrays from a field default: true compact_object: type: boolean title: Compact Object description: Remove empty objects from a field default: true required: - fields - if: properties: type: enum: - flatten-fields deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Flatten Fields description: >- Flattens the object or array value of a field into a single level representation additionalProperties: false properties: fields: type: array title: Fields description: >- Define specific nested objects you wish to flatten. This will not flatten the entire object, just the specified fields. Leave blank to flatten the entire object items: type: string title: Field description: Field to flatten pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference options: type: object additionalProperties: false title: Flatten Options description: Additional options for flattening fields properties: delimiter: type: string default: _ title: Delimiter description: >- When flattening, the nested field labels will be appended to the higher level field names along with a delimiter to allow understanding of the original structure. The default is an underscore (_) to ensure no collisions with the . notation used for field identifiers minLength: 1 maxLength: 1 - if: properties: type: enum: - dedupe deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Dedupe description: Dedupe (remove duplicates) from the data stream additionalProperties: false properties: number_of_events: type: number description: Number of events to compare across title: Number of Events minimum: 2 maximum: 5000 default: 5000 comparison_type: type: string enum: - Ignore - Match description: >- Match: Only compares across the fields which are specified by the user.Ignore: Compares everything but the fields specified by the user. title: Comparison Type default: Match fields: type: array title: Fields description: A list of fields on which to base deduping items: type: string title: Field pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference minItems: 1 required: - number_of_events - comparison_type - fields - if: properties: type: enum: - encrypt-fields deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: title: Encrypt Field description: Encrypts the value of the provided field additionalProperties: false allOf: - if: properties: algorithm: enum: - AES-256-CFB required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CFB required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CFB required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-OFB required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-OFB required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-OFB required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-CTR required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CTR required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CTR required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-CBC-PKCS7 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CBC-PKCS7 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CBC-PKCS7 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-CBC-ANSIX923 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CBC-ANSIX923 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CBC-ANSIX923 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-CBC-ISO7816 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CBC-ISO7816 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CBC-ISO7816 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-CBC-ISO10126 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CBC-ISO10126 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CBC-ISO10126 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 16 maxLength: 16 required: - key properties: field: type: string title: Field description: >- Field to encrypt. The value of the field must be a primitive (string, number, boolean) pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference algorithm: oneOf: - label: >- AES-256-CFB (key = 32 characters, iv = 16 characters) description: >- AES-256-CFB (key = 32 characters, iv = 16 characters) enum: - AES-256-CFB - label: >- AES-192-CFB (key = 24 characters, iv = 16 characters) description: >- AES-192-CFB (key = 24 characters, iv = 16 characters) enum: - AES-192-CFB - label: >- AES-128-CFB (key = 16 characters, iv = 16 characters) description: >- AES-128-CFB (key = 16 characters, iv = 16 characters) enum: - AES-128-CFB - label: >- AES-256-OFB (key = 32 characters, iv = 16 characters) description: >- AES-256-OFB (key = 32 characters, iv = 16 characters) enum: - AES-256-OFB - label: >- AES-192-OFB (key = 24 characters, iv = 16 characters) description: >- AES-192-OFB (key = 24 characters, iv = 16 characters) enum: - AES-192-OFB - label: >- AES-128-OFB (key = 16 characters, iv = 16 characters) description: >- AES-128-OFB (key = 16 characters, iv = 16 characters) enum: - AES-128-OFB - label: >- AES-256-CTR (key = 32 characters, iv = 16 characters) description: >- AES-256-CTR (key = 32 characters, iv = 16 characters) enum: - AES-256-CTR - label: >- AES-192-CTR (key = 24 characters, iv = 16 characters) description: >- AES-192-CTR (key = 24 characters, iv = 16 characters) enum: - AES-192-CTR - label: >- AES-128-CTR (key = 16 characters, iv = 16 characters) description: >- AES-128-CTR (key = 16 characters, iv = 16 characters) enum: - AES-128-CTR - label: >- AES-256-CBC-PKCS7 (key = 32 characters, iv = 16 characters) description: >- AES-256-CBC-PKCS7 (key = 32 characters, iv = 16 characters) enum: - AES-256-CBC-PKCS7 - label: >- AES-192-CBC-PKCS7 (key = 24 characters, iv = 16 characters) description: >- AES-192-CBC-PKCS7 (key = 24 characters, iv = 16 characters) enum: - AES-192-CBC-PKCS7 - label: >- AES-128-CBC-PKCS7 (key = 16 characters, iv = 16 characters) description: >- AES-128-CBC-PKCS7 (key = 16 characters, iv = 16 characters) enum: - AES-128-CBC-PKCS7 - label: >- AES-256-CBC-ANSIX923 (key = 32 characters, iv = 16 characters) description: >- AES-256-CBC-ANSIX923 (key = 32 characters, iv = 16 characters) enum: - AES-256-CBC-ANSIX923 - label: >- AES-192-CBC-ANSIX923 (key = 24 characters, iv = 16 characters) description: >- AES-192-CBC-ANSIX923 (key = 24 characters, iv = 16 characters) enum: - AES-192-CBC-ANSIX923 - label: >- AES-128-CBC-ANSIX923 (key = 16 characters, iv = 16 characters) description: >- AES-128-CBC-ANSIX923 (key = 16 characters, iv = 16 characters) enum: - AES-128-CBC-ANSIX923 - label: >- AES-256-CBC-ISO7816 (key = 32 characters, iv = 16 characters) description: >- AES-256-CBC-ISO7816 (key = 32 characters, iv = 16 characters) enum: - AES-256-CBC-ISO7816 - label: >- AES-192-CBC-ISO7816 (key = 24 characters, iv = 16 characters) description: >- AES-192-CBC-ISO7816 (key = 24 characters, iv = 16 characters) enum: - AES-192-CBC-ISO7816 - label: >- AES-128-CBC-ISO7816 (key = 16 characters, iv = 16 characters) description: >- AES-128-CBC-ISO7816 (key = 16 characters, iv = 16 characters) enum: - AES-128-CBC-ISO7816 - label: >- AES-256-CBC-ISO10126 (key = 32 characters, iv = 16 characters) description: >- AES-256-CBC-ISO10126 (key = 32 characters, iv = 16 characters) enum: - AES-256-CBC-ISO10126 - label: >- AES-192-CBC-ISO10126 (key = 24 characters, iv = 16 characters) description: >- AES-192-CBC-ISO10126 (key = 24 characters, iv = 16 characters) enum: - AES-192-CBC-ISO10126 - label: >- AES-128-CBC-ISO10126 (key = 16 characters, iv = 16 characters) description: >- AES-128-CBC-ISO10126 (key = 16 characters, iv = 16 characters) enum: - AES-128-CBC-ISO10126 ui: display_type: list title: Encryption algorithm description: The encryption algorithm to use key: type: string title: Encryption key description: The encryption key minLength: 16 maxLength: 32 iv_field: type: string title: Initialization vector (IV) field description: >- The field in which to store the generated initialization vector, IV. Each encrypted value will have a unique IV pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference encode_raw_bytes: type: boolean title: >- Encode encrypted field and IV as Base 64 text description: >- Encode the encrypted value and generated initialization vector as Base64 text default: true required: - field - algorithm - key - iv_field - if: properties: type: enum: - decrypt-fields deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: title: Decrypt Field additionalProperties: false description: Decrypts the value of the provided field allOf: - if: properties: algorithm: enum: - AES-256-CFB required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CFB required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CFB required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-OFB required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-OFB required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-OFB required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-CTR required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CTR required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CTR required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-CBC-PKCS7 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CBC-PKCS7 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CBC-PKCS7 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-CBC-ANSIX923 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CBC-ANSIX923 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CBC-ANSIX923 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-CBC-ISO7816 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CBC-ISO7816 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CBC-ISO7816 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-CBC-ISO10126 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CBC-ISO10126 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CBC-ISO10126 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 16 maxLength: 16 required: - key properties: field: type: string title: Field description: >- Field to decrypt. The value of the field must be a string pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference algorithm: oneOf: - label: >- AES-256-CFB (key = 32 characters, iv = 16 characters) description: >- AES-256-CFB (key = 32 characters, iv = 16 characters) enum: - AES-256-CFB - label: >- AES-192-CFB (key = 24 characters, iv = 16 characters) description: >- AES-192-CFB (key = 24 characters, iv = 16 characters) enum: - AES-192-CFB - label: >- AES-128-CFB (key = 16 characters, iv = 16 characters) description: >- AES-128-CFB (key = 16 characters, iv = 16 characters) enum: - AES-128-CFB - label: >- AES-256-OFB (key = 32 characters, iv = 16 characters) description: >- AES-256-OFB (key = 32 characters, iv = 16 characters) enum: - AES-256-OFB - label: >- AES-192-OFB (key = 24 characters, iv = 16 characters) description: >- AES-192-OFB (key = 24 characters, iv = 16 characters) enum: - AES-192-OFB - label: >- AES-128-OFB (key = 16 characters, iv = 16 characters) description: >- AES-128-OFB (key = 16 characters, iv = 16 characters) enum: - AES-128-OFB - label: >- AES-256-CTR (key = 32 characters, iv = 16 characters) description: >- AES-256-CTR (key = 32 characters, iv = 16 characters) enum: - AES-256-CTR - label: >- AES-192-CTR (key = 24 characters, iv = 16 characters) description: >- AES-192-CTR (key = 24 characters, iv = 16 characters) enum: - AES-192-CTR - label: >- AES-128-CTR (key = 16 characters, iv = 16 characters) description: >- AES-128-CTR (key = 16 characters, iv = 16 characters) enum: - AES-128-CTR - label: >- AES-256-CBC-PKCS7 (key = 32 characters, iv = 16 characters) description: >- AES-256-CBC-PKCS7 (key = 32 characters, iv = 16 characters) enum: - AES-256-CBC-PKCS7 - label: >- AES-192-CBC-PKCS7 (key = 24 characters, iv = 16 characters) description: >- AES-192-CBC-PKCS7 (key = 24 characters, iv = 16 characters) enum: - AES-192-CBC-PKCS7 - label: >- AES-128-CBC-PKCS7 (key = 16 characters, iv = 16 characters) description: >- AES-128-CBC-PKCS7 (key = 16 characters, iv = 16 characters) enum: - AES-128-CBC-PKCS7 - label: >- AES-256-CBC-ANSIX923 (key = 32 characters, iv = 16 characters) description: >- AES-256-CBC-ANSIX923 (key = 32 characters, iv = 16 characters) enum: - AES-256-CBC-ANSIX923 - label: >- AES-192-CBC-ANSIX923 (key = 24 characters, iv = 16 characters) description: >- AES-192-CBC-ANSIX923 (key = 24 characters, iv = 16 characters) enum: - AES-192-CBC-ANSIX923 - label: >- AES-128-CBC-ANSIX923 (key = 16 characters, iv = 16 characters) description: >- AES-128-CBC-ANSIX923 (key = 16 characters, iv = 16 characters) enum: - AES-128-CBC-ANSIX923 - label: >- AES-256-CBC-ISO7816 (key = 32 characters, iv = 16 characters) description: >- AES-256-CBC-ISO7816 (key = 32 characters, iv = 16 characters) enum: - AES-256-CBC-ISO7816 - label: >- AES-192-CBC-ISO7816 (key = 24 characters, iv = 16 characters) description: >- AES-192-CBC-ISO7816 (key = 24 characters, iv = 16 characters) enum: - AES-192-CBC-ISO7816 - label: >- AES-128-CBC-ISO7816 (key = 16 characters, iv = 16 characters) description: >- AES-128-CBC-ISO7816 (key = 16 characters, iv = 16 characters) enum: - AES-128-CBC-ISO7816 - label: >- AES-256-CBC-ISO10126 (key = 32 characters, iv = 16 characters) description: >- AES-256-CBC-ISO10126 (key = 32 characters, iv = 16 characters) enum: - AES-256-CBC-ISO10126 - label: >- AES-192-CBC-ISO10126 (key = 24 characters, iv = 16 characters) description: >- AES-192-CBC-ISO10126 (key = 24 characters, iv = 16 characters) enum: - AES-192-CBC-ISO10126 - label: >- AES-128-CBC-ISO10126 (key = 16 characters, iv = 16 characters) description: >- AES-128-CBC-ISO10126 (key = 16 characters, iv = 16 characters) enum: - AES-128-CBC-ISO10126 ui: display_type: list title: Decryption algorithm description: >- The algorithm with which the data was encrypted key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 16 maxLength: 32 iv_field: type: string title: Initialization vector (IV) field description: >- The field from which to read the initialization vector, IV pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference decode_raw_bytes: type: boolean title: >- Decode the encrypted value and IV as Base 64 text description: >- Decode the encrypted value and initialization vector as Base64 text default: true required: - field - algorithm - key - iv_field - if: properties: type: enum: - js-script deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Script Execution description: >- Use JavaScript to reshape and transform your data. You can combine multiple actions like filtering, dropping, mapping, and casting inside of a single js script. additionalProperties: true properties: script: type: string title: Script ui: display_type: code-editor options: language: js description: >- The JavaScript representation of the event transformation. minLength: 1 maxLength: 10000 default: > // Modify the event using a subset of the JavaScript language. // The function must return the modified event function processEvent(message, metadata, timestamp, annotations) { return message } - if: properties: type: enum: - unroll deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Unroll description: >- Takes an array of events and emits them all as individual events. additionalProperties: false properties: field: type: string title: Field description: >- The JSON field name that contains an array of events pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference values_only: type: boolean title: Values only? description: >- When enabled, the values from the specified array field will be emitted as new events. Otherwise, the original event will be duplicated for each value in the array field, with the unrolled value present in the field specified. default: true required: - field - if: properties: type: enum: - vrl deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: VRL description: >- Modify your observability data using Vector Remap Language (VRL) additionalProperties: false properties: source: type: string title: VRL Source description: VRL code to execute for each event ui: display_type: code-editor options: language: vrl minLength: 1 default: .message = .message required: - source - if: properties: type: enum: - reduce deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Reduce description: >- Combine multiple events over time into one based on a set of criteria additionalProperties: false properties: duration_ms: type: number maximum: 7200000 title: Duration description: >- The amount of time (in milliseconds) to allow streaming events to accumulate into a single "reduced" event. The process repeats indefinitely, or until an "ends when" condition is satisfied. ui: display_type: milliseconds-to-human-readable default: 30000 max_events: type: number minimum: 1 maximum: 100000 title: Max Events description: >- The maximum number of events that can be included in a time window (specified by Duration above). The reduce operation will stop once it has reached this number of events, regardless of whether the Duration seconds have elapsed. group_by: type: array minItems: 1 title: Group By Field Path items: type: string title: Field description: The dot-notated property path errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ description: >- Before reducing, group events based on matching data from each of these field paths. Supports nesting via dot-notation. date_formats: type: array title: Date Formats description: >- Describes which root-level properties are dates, and their expected format. Dot-notation is supported, but nested field lookup paths will be an error. minItems: 1 items: type: object additionalProperties: false properties: field: type: string title: Date Field Path description: >- Specifies a root-level path property that contains a date value. maxLength: 200 errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ format: type: string title: Date Format description: The template describing the date format format: strftime_datetime errorMessage: must be a valid strftime datetime format maxLength: 200 required: - field - format merge_strategies: type: array title: Merge Strategy per Field minItems: 1 description: >- Specify merge strategies for individual root-level properties. Dot-notation is supported, but nested field lookup paths will be an error. items: type: object additionalProperties: false properties: field: type: string title: Field Path description: >- This is a root-level path property to apply a merge strategy to its value maxLength: 200 errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ strategy: oneOf: - label: Array description: Append each value onto an array enum: - array - label: Smallest array description: >- Keep the smallest array within each reduce time window enum: - shortest_array - label: Largest array description: >- Keep the largest array within each reduce time window enum: - longest_array - label: Unique Array description: >- All unique values go into a flattened array enum: - flat_unique - label: Join with space description: >- Concatenate into a single space-delimited string enum: - concat - label: Join with newline description: >- Concatenate into a single newline-delimited string enum: - concat_newline - label: Join description: >- Concatenate into a single string without using a delimiter enum: - concat_raw - label: Keep first description: >- Keep only the first value and discard the rest enum: - discard - label: Keep last description: Keep only the last non-null value enum: - retain - label: Min value description: >- Keep the minimum numeric value within each reduce time window enum: - min - label: Max value description: >- Keep the maximum numeric value within each reduce time window enum: - max - label: Sum description: Add numeric values together enum: - sum ui: display_type: list errorMessage: >- must be one of the allowed merge strategies title: Merge Strategy description: >- The merge strategy to be used for the specified property required: - field - strategy flush_condition: title: Event Flush Condition description: >- Force accumulated event reduction to flush the result when a conditional expression evaluates to true on an inbound event. allOf: - type: object properties: when: type: string enum: - none - starts_when - ends_when default: none ui: display_type: list errorMessage: '' title: Flush When required: - when - if: properties: when: not: enum: - none then: properties: conditional: allOf: - $ref: '#condition-group' ui: display_type: rules-engine required: - conditional - if: properties: type: enum: - map-fields deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Map Fields description: >- Maps data from one field to another, either by moving or copying. additionalProperties: false properties: mappings: type: array title: Mappings description: >- A list of field mappings. Mappings are applied in the order they are defined. minItems: 1 ui: display_type: sortable items: type: object properties: source_field: type: string title: Source field description: The field to copy data from pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference target_field: type: string title: Target field description: The field to copy data into pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference drop_source: type: boolean title: Drop source? description: >- When enabled, the source field is dropped after the data is copied to the target field. Otherwise, it is preserved. default: false overwrite_target: type: boolean title: Overwrite target? description: >- When enabled, any existing data in the target field is overwritten. Otherwise, the target field will be preserved and this mapping will have no effect. default: false required: - source_field - target_field - drop_source - overwrite_target if: properties: drop_source: enum: - true required: - drop_source then: properties: source_field: type: string title: Source field description: The field to copy data from pattern: >- ^(?:metadata|\.|(?:metadata|message)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ minLength: 1 errorMessage: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields). Note that dropping an event's message or timestamp is not allowed required: - source_field required: - mappings - if: properties: type: enum: - metrics-tag-cardinality-limit deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: title: Tag Cardinality Limit (Metrics) description: >- Limits the cardinality of metric events by either dropping events or tags that exceed a specified value limit additionalProperties: false allOf: - if: title: Mode properties: mode: type: string enum: - exact then: title: Value Limit properties: value_limit: type: integer title: Value Limit description: Maximum number of unique values for tags minimum: 1 maximum: 500 required: - value_limit - if: title: Mode properties: mode: type: string enum: - probabilistic then: title: Value Limit properties: value_limit: type: integer title: Value Limit description: Maximum number of unique values for tags minimum: 1 maximum: 5000 required: - value_limit properties: tags: type: array title: Tags description: >- A list of tags to apply cardinality limits. If none are provided, all tags will be considered. items: type: string title: Tag minLength: 1 maxLength: 100 minItems: 0 maxItems: 10 exclude_tags: type: array title: Exclude Tags description: >- A list of tags to explicitly exclude from cardinality limits. items: type: string title: Tag minLength: 1 maxLength: 100 minItems: 0 maxItems: 10 action: type: string title: Action description: >- The action to take when a tag's cardinality exceeds the value limit enum: - drop_tag - drop_event default: drop_event value_limit: type: integer title: Value Limit description: Maximum number of unique values for tags mode: type: string title: Mode description: >- The method to used to reduce tag value cardinality enum: - exact - probabilistic default: exact - if: properties: type: enum: - aggregate deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Aggregate description: >- Aggregates multiple events into a single event using either a tumbling interval window or a sliding interval window. additionalProperties: false properties: group_by: type: array minItems: 1 maxItems: 100 title: Group By Field Path description: >- When aggregating, group events based on matching values from each of these field paths. Supports nesting via dot-notation. items: type: string title: Field description: The dot-notated property path errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ ui: default_hint: - .name - .namespace - .tags evaluate: additionalProperties: type: string title: Evaluate description: >- How events should be aggregated within an interval. allOf: - if: properties: operation: enum: - CUSTOM then: properties: script: type: string title: Script ui: display_type: code-editor options: language: js description: >- The JavaScript representation of an event aggregation. minLength: 1 maxLength: 2048 default: > // Perform custom aggregation of the current event into the accumulated event. // The accumulated event is persisted across the entire aggregation window. // The function must return the accumulated event, which will be emitted when // the aggregation window elapses. function aggregateEvent(accum, event, metadata, timestamp, annotations) { return accum } required: - script properties: operation: title: Operation oneOf: - label: add description: >- Adds new values together if they are of the same type. For counters and gauges the values are added together. For sets, this is a union of the elements. For distribution, the samples are concatenated. For histograms, values in the same bucket will be added together. enum: - DEFAULT - label: sum description: >- Values are added together. Works for the GAUGE and COUNT metric types. enum: - SUM - label: minimum description: >- Get the minimum value for a series of events. Works for the GAUGE and COUNT metric types. enum: - MIN - label: maximum description: >- Get the maximum value for a series of events. Works for the GAUGE and COUNT metric types. enum: - MAX - label: average description: >- Values are averaged together. Works for the GAUGE and COUNT metric types. enum: - AVG - label: set intersection description: >- The aggregate produces a set of unique values. enum: - SET_INTERSECTION - label: distribution concatenation description: >- The aggregate produces a list of values combined together. enum: - DIST_CONCAT - label: custom description: >- Use a custom script to perform aggregation. enum: - CUSTOM ui: display_type: list required: - operation window: default: {} additionalProperties: type: integer title: Window Type description: >- Sliding windows can overlap, whereas tumbling windows are disjoint. For example, a tumbling window has a fixed time span and any events that fall within the "interval" will be used in the aggregate. Whereas in a sliding window, the aggregation occurs every "window duration" seconds after an event is encountered. allOf: - if: properties: type: enum: - sliding required: - type then: properties: window_min: type: integer title: Slide Duration (seconds) description: >- The amount of time to wait before creating a new sliding window for future events. minimum: 1 maximum: 200 default: 5 required: - window_min properties: type: type: string title: Type enum: - tumbling - sliding default: tumbling interval: type: integer title: Interval (seconds) description: >- The interval over which events are aggregated in seconds. default: 5 minimum: 1 maximum: 90000 required: - type - interval event_timestamp: type: string title: Event Timestamp description: >- The path to a field on the event that contains a epoch timestamp value. If an event does not have a timestamp field, events will be associated to the wall clock value when the event is processed. default: timestamp maxLength: 200 errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ conditional: ui: display_type: rules-engine title: Event Flush Condition description: >- Force accumulated event reduction to flush the result when a conditional expression evaluates to true on an inbound event. allOf: - $ref: '#condition-group' required: - evaluate - window - if: properties: type: enum: - event-to-metric deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Event to Metric description: >- Allows conversion between arbitrary events and a Metric. additionalProperties: false properties: metric_name: type: string title: Metric Name description: The machine name of the metric to emit. minLength: 1 maxLength: 128 pattern: ^[a-zA-Z][a-zA-Z0-9_:]*$ errorMessage: must be a valid metric name metric_kind: title: Kind description: >- The kind of metric to emit, Absolute or Incremental. Absolute metrics represent a complete value, and will generally replace an existing value for the metric in the target destination. Incremental metrics represent an additive value which is aggregated in the target destination to produce a new value. oneOf: - label: Incremental enum: - incremental - label: Absolute enum: - absolute default: incremental ui: display_type: radio-group metric_type: title: Type oneOf: - label: Counter enum: - counter - label: Sum enum: - sum - label: Gauge enum: - gauge default: counter ui: display_type: list value: title: Value description: The value of this metric. additionalProperties: false allOf: - if: properties: value_type: enum: - field then: title: Value properties: value: type: string title: Field value description: The field to use as the value. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference required: - value - if: properties: value_type: enum: - value then: title: Value properties: value: type: number title: Value description: The value. required: - value properties: value_type: title: Value type description: >- Either a value from an event field, or a new value input. oneOf: - label: Value from event field enum: - field - label: New value enum: - value default: field ui: display_type: radio-group value: type: - number - string required: - value_type namespace: title: Namespace description: The namespace for this metric. additionalProperties: false allOf: - if: properties: value_type: enum: - field then: title: Value properties: value: type: string title: Field value description: The field to use as the value. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference required: - value - if: properties: value_type: enum: - value required: - value_type then: title: Value properties: value: type: string title: Value description: The value. minLength: 1 maxLength: 128 required: - value properties: value_type: title: Value type description: >- Either a value from an event field, a new value input, or none to omit the namespace. oneOf: - label: Value from event field enum: - field - label: New value enum: - value - label: None enum: - none default: none ui: display_type: radio-group value: type: - string - 'null' required: - value_type tags: type: array maxItems: 10 title: Tags description: >- A set of tags (also called labels) to apply to the metric event. items: type: object title: Tag description: A tag to apply to the metric event. allOf: - if: properties: value_type: enum: - field then: title: Value properties: value: type: string title: Field value description: >- The field to use as the value of this tag. Note that fields with highly-variable values will result in high-cardinality metrics, which may impact storage or cost in downstream destinations. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference fallback: type: string title: Fallback value description: >- The value to use for this tag when the field value is missing. minLength: 1 maxLength: 512 required: - value - if: properties: value_type: enum: - value then: title: Value properties: value: type: string title: Value description: The value for this tag. minLength: 1 maxLength: 128 required: - value properties: name: type: string title: Name description: The tag name pattern: ^[a-zA-Z][a-zA-Z0-9_]*$ minLength: 1 maxLength: 128 errorMessage: must be a valid metric tag name value_type: title: Value type description: >- Either a value from an event field, or a new value input. oneOf: - label: Value from event field enum: - field - label: New value enum: - value default: field ui: display_type: radio-group value: type: string required: - name - value_type - value additionalProperties: type: string required: - metric_name - metric_kind - metric_type - value - namespace - if: properties: type: enum: - clustering deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Clustering description: Finds clusters of similar values additionalProperties: false properties: cluster_field: type: string title: Field description: Field to cluster values on pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference default: .message similarity_threshold: type: number title: Similarity threshold description: >- A threshold from 0.0 (not similar) to 1.0 (exactly similar) that determines how close two values need to be to be consider as being part of the same cluster default: 0.4 minimum: 0 maximum: 1 max_clusters: type: integer title: Maximum cluster description: >- Maximum number of clusters kept cached by the transform default: 1000 minimum: 1 maximum: 2000 max_node_depth: type: integer title: Maximum tree depth description: Maximum depth of the prefix tree default: 4 minimum: 2 maximum: 8 max_children: type: integer title: Maximum children default: 100 minimum: 2 maximum: 200 description: >- Maximum number of children per node of the prefix tree required: - cluster_field - if: properties: type: enum: - redact deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Redact description: >- Detect and anonymize sensitive data in logs and metrics additionalProperties: false properties: field: type: string title: Field description: >- Detect patterns in the specified field. If the field is empty, the processor will look for patterns across all the fields. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ default: . maxLength: 256 errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference mask_patterns: type: array title: Redact Patterns description: List of PII patterns to detect and mask minItems: 1 maxItems: 20 items: type: object additionalProperties: type: string title: Redact Pattern description: A PII pattern to detect and/or anonymize allOf: - if: properties: action: enum: - Replace required: - action then: properties: replacement: type: string title: Replacement description: >- The text to replace the detected pattern. Example: 000-00-0000 minLength: 1 maxLength: 512 required: - replacement - if: properties: action: enum: - Hash required: - action then: properties: algorithm: title: Algorithm description: >- Hash and encode (base16 or hex) matched PII data using the specified algorithm oneOf: - label: sha2 (sha-512_256) enum: - sha2 - label: sha3 (sha3-512) enum: - sha3 ui: display_type: list required: - algorithm - if: properties: name: enum: - custom required: - name then: properties: custom_options: allOf: - type: object additionalProperties: false properties: pattern: type: string title: Pattern description: The custom pattern minLength: 1 maxLength: 4096 case_sensitive: type: boolean title: Case sensitive description: Perform case sensitive matching default: true multiline: type: boolean title: Multiline mode description: >- ^ and $ match beginning and end of line respectively default: false match_newline: type: boolean title: Dot all mode description: Dot matches newline character default: false crlf_newline: type: boolean title: Carriage return (CRLF) mode description: "When multiline mode is enabled, newline character is interpreted as \r\n" default: false ignore_whitespace: type: boolean title: Verbose mode description: >- Ignore whitespace and allow comment lines default: false required: - pattern title: Replace Expression description: >- The regular expression used to find replacements. Named capture groups are supported, e.g. `(?P<key>\w+):(?P<value>.+)`. ui: display_type: regex-pattern required: - custom_options properties: name: title: Pattern description: The PII pattern to match oneOf: - label: Credit card number enum: - credit_card - label: Email address enum: - email_address - label: IPv4 address enum: - ipv4_address - label: US or Canada phone number enum: - us_canada_phone_number - label: US social security number enum: - us_social_security_number - label: Custom pattern ui: feature_flag: pipeline-parser-regex enum: - custom ui: display_type: list action: type: string title: Action description: Action to take on the detected PII data enum: - Hash - Replace - Detect Only default: Replace required: - name - action required: - mask_patterns - if: properties: type: enum: - throttle deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Throttle description: >- Throttle (rate-limit) events passing through this component additionalProperties: false properties: threshold: type: number minimum: 1 maximum: 10000000 title: Limit description: >- The number of events to allow over the time window. If a Key Field is specified, the limit will be applied to events for each unique value of that field. default: 1 window_ms: type: number minimum: 1000 maximum: 2592000000 title: Window description: >- The time window over which the configured limit is applied ui: display_type: milliseconds-to-human-readable default: 5000 key_field: type: string title: Key Field description: >- When configured, the value from this field will be used to group events into separate buckets for throttling. If left blank, or if the event does not contain a value in this field, the event will be throttled as part of the default bucket. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference exclude: allOf: - $ref: '#condition-group' title: Exclude description: >- When configured, events matching this condition will not be throttled. ui: display_type: rules-engine required: - threshold - window_ms - if: properties: type: enum: - set-timestamp deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Set Timestamp description: >- Parse a list of fields using a chosen time format. First match will override the default timestamp of the event. additionalProperties: false properties: parsers: type: array title: List of Fields and Parsers description: >- The list of fields and parsers to use in order of priority, short-circuiting on the first successful match. ui: display_type: multi-output-map options: sortable: false items: type: object properties: field: type: string title: Field description: >- The field whose value will be parsed by the chosen date pattern pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ default: . maxLength: 512 errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference options: $ref: '#timestamp-parser-options' required: - field - options minItems: 1 required: - parsers - if: properties: type: enum: - data-profiler deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Data Profiler description: >- Profile the data sent through your pipeline. The system will generate annotations such as event type and total bytes to be used downstream. It will also normalize the app/host/level/line fields before sending the event downstream. A profile of the data including total, events, event size, log composition, and message templates will be generated as well. additionalProperties: false properties: app_fields: type: array title: App Path(s) description: A list of paths to look for the app value minItems: 1 maxItems: 10 uniqueItems: true items: type: string title: App description: The path to the app maxLength: 512 pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference ui: default_hint: - .app - .application - .container host_fields: type: array title: Host Path(s) description: A list of paths to look for the host value minItems: 1 maxItems: 10 uniqueItems: true items: type: string title: Host description: The path to the host maxLength: 512 pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference ui: default_hint: - .host - .hostname level_fields: type: array title: Level Path(s) description: A list of paths to look for the log level minItems: 1 maxItems: 10 uniqueItems: true items: type: string title: Level description: The path to the level maxLength: 512 pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference ui: default_hint: - .level - .log_level line_fields: type: array title: Line Path(s) description: A list of paths to look for the event line minItems: 1 maxItems: 10 uniqueItems: true items: type: string title: Line description: The path to the line maxLength: 512 pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference ui: default_hint: - .line - .message label_fields: type: array title: Label Path(s) description: A list of paths to look for the event labels maxItems: 10 uniqueItems: true items: type: string title: Label description: The path to the labels maxLength: 512 pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference ui: default_hint: - .labels field_summary: type: boolean default: false title: Field Summary description: >- If enabled the profiler will attempt to summarize the values in fields of events passing through the node. ui: display_type: hidden ff_default_value: pipeline-node-transform-field-summary required: - app_fields - host_fields - level_fields - line_fields - if: properties: type: enum: - replace deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: title: Replace description: >- Replace parts of a string using a regular expression or static string additionalProperties: type: object allOf: - if: properties: replacement_type: enum: - regex required: - replacement_type then: properties: options: allOf: - type: object additionalProperties: false properties: pattern: type: string minLength: 1 maxLength: 4096 case_sensitive: type: boolean title: Case sensitive description: Perform case sensitive matching default: true multiline: type: boolean title: Multiline mode description: >- ^ and $ match beginning and end of line respectively default: false match_newline: type: boolean title: Dot all mode description: Dot matches newline character default: false crlf_newline: type: boolean title: Carriage return (CRLF) mode description: "When multiline mode is enabled, newline character is interpreted as \r\n" default: false ignore_whitespace: type: boolean title: Verbose mode description: >- Ignore whitespace and allow comment lines default: false required: - pattern title: Replace Expression description: >- The regular expression used to find replacements. Named capture groups are supported, e.g. `(?P<key>\w+):(?P<value>.+)`. ui: display_type: regex-pattern replace_with: type: string minLength: 1 maxLength: 4096 title: Replacement String description: >- The replacement string can be static or optionally contain capture group references. Capture groups can be positional or named. For example, to switch the position of two strings in the source string `onetwo`, the pattern could be `(?Pone)(?P.+)`, and the replacement would be `$two$one`. To do the same thing with positional capture groups, a pattern of `(\w{3})(\w{3})`, and a replacement of `$2$1` would have the same result.

To render a literal dollar sign `$` in the replacement, escape it with `$$`. So, if you had a capture group named `price`, to prepend a dollar sign to the replacement, you would use `$$$price`. required: - options - replace_with - if: properties: replacement_type: enum: - string required: - replacement_type then: properties: string_pattern: type: string title: String Pattern minLength: 1 maxLength: 4096 description: >- A static, case-sensitive string, which will be searched and replaced. replace_with: type: string minLength: 1 maxLength: 4096 title: Replacement String description: >- The static string used in each replacement. required: - string_pattern - replace_with properties: source_field: type: string title: Source Field description: >- The field that contains a string for replacement to operate on. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ default: . errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference max_times: type: number title: Maximum Replacements description: >- The maximum number of replacements to be made. If not specified, all occurrences will be replaced. minimum: 1 replacement_type: title: Replacement Type description: >- Specifies how the source string will be searched for replacement. ui: display_type: list errorMessage: must be one of the available pattern types oneOf: - label: String description: >- Replace a specified string one or more times. enum: - string - label: Regular Expression description: Replace using a regular expression. enum: - regex required: - source_field - replacement_type - if: properties: type: enum: - trace-sampling deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: title: Trace Sampling description: >- Performs sampling of traces per a given rate using the Trace ID as the unique identifier. If a trace is rate selected, all spans of the trace are forwarded. If the Trace ID cannot be found, the event is dropped. allOf: - if: properties: sample_type: enum: - head then: properties: rate: type: integer title: Rate description: >- The rate at which events will be forwarded, expressed as 1/N. For example, `rate = 10` means 1 out of every 10 unique traces will be forwarded and all others will be dropped default: 10 minimum: 2 maximum: 100000 parent_span_id_field: type: 'null' conditionals: type: 'null' required: - rate - if: properties: sample_type: enum: - tail then: properties: parent_span_id_field: type: string default: .parent_span_id title: Parent Span ID description: >- The field to use as the parent span identifier pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference conditionals: type: array minItems: 1 title: Conditionals description: >- A list of complete conditionals, including expressions and a sample rate to maintain items: type: object additionalProperties: false title: Conditional description: >- A list of conditions and a sample rate to maintain properties: rate: type: integer title: Rate description: >- The rate at which events will be forwarded, expressed as 1/N. For example, `rate = 10` means 1 out of every 10 unique traces will be forwarded and all others will be dropped. A `rate = 1` means keep all traces that match the condition. minimum: 1 maximum: 100000 default: 10 conditional: allOf: - $ref: '#condition-group' title: Conditional description: >- A group of expressions joined together by an and/or operator. ui: display_type: rules-engine _output_name: type: string title: '[INTERNAL]' description: >- Internal-only field that holds a static id for the output_name. This way, labels for outputs can be updated independent of the id. required: - rate - conditional required: - parent_span_id_field - conditionals properties: sample_type: type: string description: The type of trace sampling to apply. enum: - head title: Sample Type default: head trace_id_field: type: string default: .trace_id title: Trace ID description: >- The field to use as the identifier of a trace. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference required: - sample_type - trace_id_field properties: title: oneOf: - type: string minLength: 1 maxLength: 512 pattern: ^[^<>"]+$ - type: 'null' errorMessage: invalid title name description: oneOf: - type: string minLength: 1 pattern: ^[^<>"]+$ - type: 'null' errorMessage: invalid description type: type: string enum: - filter - sample - drop-fields - parse - parse-sequentially - route - stringify - compact-fields - flatten-fields - dedupe - encrypt-fields - decrypt-fields - js-script - unroll - vrl - reduce - map-fields - metrics-tag-cardinality-limit - aggregate - event-to-metric - clustering - redact - throttle - set-timestamp - data-profiler - replace - trace-sampling user_config: type: object required: - user_config sinks: type: array minItems: 1 items: type: object additionalProperties: false allOf: - if: properties: type: enum: - mezmo deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Mezmo Log Analysis description: Send log data to Mezmo Log Analysis additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: >- Sets end-to-end acknowledgement on the destination default: true mezmo_host: type: string title: Mezmo Host description: The host for your Log Analysis environment default: null minLength: 1 ingestion_key: type: string title: Ingestion Key description: Ingestion key ui: display_type: mezmo-ingestion-key minLength: 1 query: type: object title: Query Parameters default: hostname: mezmo properties: hostname: type: string title: Hostname description: >- Hostname string or template to extract event values to form a hostname. Depending on the type of ingestion, this value is usually set from other places so the default value is often sufficient. If the value in the event is null, empty, or doesn't exist, when using a template then it will be set to "mezmo" default: mezmo minLength: 1 ui: display_type: data-template tags: type: array title: Tags description: >- List of tag strings or templates to attach to logs. Template must resolve to either an array of strings, or a string. Commas are treated as tag delimiters. items: type: string title: Tag description: Tag string or template to attach to logs minLength: 1 maxLength: 512 ui: display_type: data-template ip: type: string title: IP description: IP address template to attach to logs minLength: 1 maxLength: 512 ui: display_type: data-template mac: type: string title: MAC description: MAC address template to attach to logs minLength: 1 maxLength: 512 ui: display_type: data-template message: title: Log Construction Scheme default: scheme: Explicit field selection allOf: - if: properties: scheme: enum: - Explicit field selection then: properties: line: type: string title: Line description: >- Template or field reference to use as the log line. Field reference can point to an object. minLength: 1 maxLength: 1024 meta_field: type: string title: Meta Field description: >- Field containing the meta object for the log pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference maxLength: 512 app: type: string title: App description: App name template to attach to logs minLength: 1 maxLength: 512 ui: display_type: data-template file: type: string title: File description: File name template to attach to logs minLength: 1 maxLength: 512 ui: display_type: data-template timestamp_field: type: string title: Timestamp description: >- Field containing the timestamp for the log pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference maxLength: 512 env: type: string title: Env description: >- Environment name template to attach to logs minLength: 1 maxLength: 512 ui: display_type: data-template properties: scheme: type: string title: Scheme description: How to construct the log message. enum: - Explicit field selection - Message pass-through default: Explicit field selection required: - scheme required: - mezmo_host - ingestion_key - if: properties: type: enum: - mezmo-archive deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Mezmo Archive Destination description: >- Store logs for future imports into mezmo pipeline or log analysis. additionalProperties: false properties: batch_timeout_secs: type: integer title: Batch Timeout (seconds) description: >- The maximum amount of time, in seconds, events will be buffered before being flushed to the destination enum: - 30 - 60 - 90 - 120 - 300 default: 300 ack_enabled: type: boolean title: End-to-end Acknowledgement description: >- Sets end-to-end acknowledgement on the destination default: true destination: title: Destination description: '' allOf: - if: properties: archive_provider: enum: - Azure then: properties: container_name: type: string title: Container Name minLength: 1 description: >- The name of the container for blob storage. connection_string: type: string title: Connection String minLength: 1 description: >- A connection string for the account that contains an access key required: - container_name - connection_string - if: properties: archive_provider: enum: - S3 then: properties: access_key_id: type: string title: Access Key ID minLength: 1 description: The AWS access key id secret_access_key: type: string title: Secret Access Key minLength: 1 description: The AWS secret access key ui: display_type: secret bucket: type: string title: Bucket pattern: >- (?!(^xn--|.+-s3alias$))^[a-z0-9][a-z0-9-]{1,61}[a-z0-9]$ description: >- The S3 bucket name. Do not include a leading s3:// or a trailing / region: type: string title: Region minLength: 1 description: >- The name of the AWS region that is targeted. required: - access_key_id - secret_access_key - bucket - region properties: archive_provider: type: string title: Archive Provider description: The storage destination for your data. enum: - Azure - S3 default: Azure required: - archive_provider required: - destination - if: properties: type: enum: - http deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: HTTP description: Send data to a HTTP endpoint additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: >- Sets end-to-end acknowledgement on the destination default: true uri: type: string title: URI format: uri errorMessage: format: Must be a valid URI description: >- The full URI to make HTTP requests to. This should include the protocol and host, but can also include the port, path, and any other valid part of a URI. minLength: 1 encoding: type: string description: The encoding to apply to the data. enum: - json - ndjson - text title: Encoding default: text compression: type: string description: >- The compression strategy used on the encoded data prior to sending. enum: - gzip - none title: Compression default: none auth: title: Authentication description: Configures HTTP authentication allOf: - if: properties: strategy: enum: - basic then: properties: user: type: string title: User minLength: 1 description: The basic authentication user password: type: string title: Password minLength: 1 description: The basic authentication password ui: display_type: secret required: - user - password - if: properties: strategy: enum: - bearer then: properties: token: type: string title: Token minLength: 1 description: The bearer token ui: display_type: secret required: - token properties: strategy: type: string enum: - basic - bearer - none title: Strategy description: The authentication strategy to use default: none headers: type: array title: HTTP Headers description: >- An array of key/value objects for the headers to be sent with the request. items: type: object title: Headers description: >- A key/value object describing a header name and its value. additionalProperties: false properties: header_name: type: string title: Header Name minLength: 1 description: The name of the header. header_value: type: string title: Header Value minLength: 1 description: The value of the header. required: - header_name - header_value advanced_options: type: object ui: display_type: collapsable feature_flag: pipeline-node-sink-http-advanced-options title: Advanced Options description: >- Settings available for fine-tuning the destination behavior. additionalProperties: false dependencies: payload_prefix: - payload_suffix payload_suffix: - payload_prefix properties: max_bytes: type: integer title: Payload Size description: >- The maximum number of uncompressed bytes when batching data to the destination. minimum: 1 maximum: 2097152 timeout_secs: type: integer title: Timeout Seconds description: >- The number of seconds before a destination write timeout. minimum: 5 method: type: string enum: - post - put - patch - delete - get - head - options - trace title: HTTP Method description: >- The HTTP method to use for the destination. payload_prefix: type: string title: Payload Prefix description: >- Add a prefix to the payload. Only used for serialized JSON chunks. This option also requires "Payload Suffix" to form a valid JSON string. examples: - '{"event":"' - '{"extra_prop": true, "our_thing":"' minLength: 1 payload_suffix: type: string title: Payload Suffix description: >- Used in combination with "Payload Prefix" to form valid JSON from the payload. examples: - '}' - '"extra_prop": true }' minLength: 1 proxy: type: object additionalProperties: false title: Proxy Settings description: >- Configure proxy settings for the destination. properties: enabled: type: boolean title: Enabled description: Turns proxying on/off default: false endpoint: type: string minLength: 1 title: HTTP(S) Endpoint description: >- HTTP or HTTPS Endpoint to use for traffic. format: uri errorMessage: format: Must be a valid URI hosts_bypass_proxy: type: array minItems: 1 title: Hosts to Bypass description: >- A list of hosts to bypass proxying. Can be specified as a domain name, IP address, or CIDR block. Wildcards are supported as a dot (.) in domain names, or as a star (*) to match all hosts. examples: - 93.184.216.34 - ignorehost.example.com - .ignoreall.example.com - 93.184.216.0/24 items: type: string title: Host description: Domain name, IP address, or CIDR block. minLength: 1 tls_protocols: type: array title: TLS Protocols description: >- A list of ALPN protocols to use during TLS negotiation. They are attempted in the order they appear. items: type: string title: Protocol description: An ALPN protocol name minLength: 1 minItems: 1 rate_limiting: type: object additionalProperties: false title: Rate Limiting description: >- Settings for controlling rate limiting to the destination. properties: request_limit: type: integer minimum: 1 title: Request Limit description: >- The max number of requests allowed within the specified "Duration Seconds" setting. duration_secs: type: integer default: 1 title: Duration Seconds description: >- The window of time used to apply "Request Limit." minimum: 1 required: - uri - encoding - compression - if: properties: type: enum: - cloudwatch-logs deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: AWS Cloudwatch Logs description: Publish log events to AWS Cloudwatch Logs additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: >- Sets end-to-end acknowledgement on the destination default: true auth: type: object title: AWS Authentication description: Configures AWS authentication additionalProperties: false properties: access_key_id: type: string title: Access Key ID minLength: 1 description: The AWS access key id secret_access_key: type: string title: Secret Access Key minLength: 1 description: The AWS secret access key ui: display_type: secret required: - access_key_id - secret_access_key encoding: type: string description: The encoding to apply to the data. enum: - json - text title: Encoding default: text compression: type: string description: >- The compression strategy used on the encoded data prior to sending. enum: - gzip - none title: Compression default: none group_name: type: string title: Group Name minLength: 1 description: >- The name of the log group for the targeted log stream. region: type: string title: Region minLength: 1 description: The name of the AWS region that is targeted. stream_name: type: string title: Stream Name minLength: 1 description: The name of the targeted log stream. required: - auth - encoding - compression - group_name - region - stream_name - if: properties: type: enum: - cloudwatch-metrics deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: AWS Cloudwatch Metrics description: Send metric events to AWS Cloudwatch Metrics additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: >- Sets end-to-end acknowledgement on the destination default: true auth: type: object title: AWS Authentication description: Configures AWS authentication additionalProperties: false properties: access_key_id: type: string title: Access Key ID minLength: 1 description: The AWS access key id secret_access_key: type: string title: Secret Access Key minLength: 1 description: The AWS secret access key ui: display_type: secret required: - access_key_id - secret_access_key compression: type: string description: >- The compression strategy used on the encoded data prior to sending. enum: - gzip - none title: Compression default: none namespace: type: string title: Namespace minLength: 1 description: >- Name for the container that will isolate metrics from one another. region: type: string title: Region minLength: 1 description: The name of the AWS region that is targeted. required: - auth - compression - namespace - region - if: properties: type: enum: - sqs deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: AWS Simple Queue Service (SQS) description: Publish log events to AWS Simple Queue Service additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: >- Sets end-to-end acknowledgement on the destination default: true auth: type: object title: AWS Authentication description: Configures AWS authentication additionalProperties: false properties: access_key_id: type: string title: Access Key ID minLength: 1 description: The AWS access key id secret_access_key: type: string title: Secret Access Key minLength: 1 description: The AWS secret access key ui: display_type: secret required: - access_key_id - secret_access_key encoding: type: string description: The encoding to apply to the data. enum: - json - text title: Encoding default: text sqs_queue_url: type: string title: SQS Queue URL format: url errorMessage: format: Must be a valid URL description: >- The URL of the SQS queue that the data is publishing to. region: type: string title: Region minLength: 1 description: The name of the AWS region that is targeted. required: - auth - encoding - sqs_queue_url - region - if: properties: type: enum: - notification-channel deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Notification Channel description: Send a Message to a 3rd Party Service. additionalProperties: false properties: service: title: Service description: >- Configure a service to receive the payload. Currently supports Slack, PagerDuty, Webhook or Mezmo (Log Analysis). Where allowed, messagetemplating can be used. To use a template, surround field names with double curly braces, e.g., {{.my_field}} or {{."my-object".thing}}. See [documentation](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) for more examples. additionalProperties: type: string title: URI format: uri errorMessage: format: Must be a valid URI description: >- The full URI where the payload is sent. This should include the protocol and host, but can also include the port, path, and any other valid part of a URI. minLength: 1 allOf: - if: properties: name: enum: - slack required: - name then: properties: uri: type: string title: URI format: uri errorMessage: format: Must be a valid URI description: >- The full URI where the payload is sent. This should include the protocol and host, but can also include the port, path, and any other valid part of a URI. minLength: 1 message_text: type: string title: Message minLength: 1 ui: display_type: data-template options: multiline: true description: The text to be sent as a Slack message. required: - uri - message_text - if: properties: name: enum: - pager_duty required: - name then: properties: uri: type: string title: URI format: uri errorMessage: format: Must be a valid URI description: >- The full URI where the payload is sent. This should include the protocol and host, but can also include the port, path, and any other valid part of a URI. minLength: 1 summary: type: string title: Summary description: >- Summarize the alert details for PagerDuty. minLength: 1 ui: display_type: data-template severity: type: string enum: - INFO - WARNING - ERROR - CRITICAL title: Severity description: The severity level of the alert. source: type: string title: Source description: The source of the alert. minLength: 1 ui: display_type: data-template routing_key: type: string title: Routing Key description: The service integration routing key. minLength: 1 ui: display_type: secret event_action: type: string enum: - trigger - acknowledge - resolve title: Event Action description: >- The event action associated with the alert. required: - uri - summary - severity - source - routing_key - event_action - if: properties: name: enum: - webhook required: - name then: properties: uri: type: string title: URI format: uri errorMessage: format: Must be a valid URI description: >- The full URI where the payload is sent. This should include the protocol and host, but can also include the port, path, and any other valid part of a URI. minLength: 1 method: type: string title: HTTP Method description: The HTTP method to use for the request. enum: - post - put - patch - delete - get - head - options - trace ui: default_hint: post message_text: type: string title: Message minLength: 1 ui: display_type: data-template options: multiline: true description: >- The Webhook payload to send. This can be a text string or stringified JSON. If the message can be parsed as JSON, it will be sent as such. headers: type: array title: HTTP Headers description: >- A list of key/value pairs to be included in the HTTP request. items: type: object title: Headers description: >- A key/value object describing a header name and its value. additionalProperties: false properties: header_name: type: string title: Header Name minLength: 1 description: The name of the header. header_value: type: string title: Header Value minLength: 1 maxLength: 1024 description: The value of the header. required: - header_name - header_value maxItems: 20 auth: title: Authentication Options description: Configures HTTP authentication allOf: - if: properties: strategy: enum: - basic then: properties: user: type: string title: User minLength: 1 description: The basic authentication user password: type: string title: Password minLength: 1 description: The basic authentication password ui: display_type: secret required: - user - password - if: properties: strategy: enum: - bearer then: properties: token: type: string title: Token minLength: 1 description: The bearer token ui: display_type: secret required: - token properties: strategy: type: string enum: - basic - bearer - none title: Strategy description: >- Choose basic or token-based authentication. default: none required: - uri - message_text - if: properties: name: enum: - log_analysis required: - name then: properties: severity: type: string enum: - INFO - WARNING - ERROR - CRITICAL title: Severity description: The severity level of the alert. default: INFO subject: type: string title: Subject description: >- A string to be used as the subject of the alert. minLength: 1 maxLength: 200 ui: display_type: data-template body: type: string title: Body description: >- A string to be used as the alert message body. minLength: 1 maxLength: 1024 ui: display_type: data-template ingestion_key: type: string title: Ingestion Key description: >- The key required to communicate to Log Analysis for sending the alert. ui: display_type: mezmo-ingestion-key minLength: 1 required: - severity - subject - body - ingestion_key properties: name: title: Name description: >- The name of the service that will receive the notification. ui: display_type: list errorMessage: must be a valid service name oneOf: - label: Slack enum: - slack - label: PagerDuty enum: - pager_duty - label: Webhook enum: - webhook - label: Mezmo Log Analysis enum: - log_analysis required: - name throttling: type: object title: Throttling description: >- Only allow a certain number of notifications to be sent within a given time window. properties: window_secs: type: integer title: Window Seconds description: >- The time frame during which the number of notifications (set by the Threshold) is permitted. default: 60 minimum: 1 threshold: type: integer title: Threshold description: >- The maximum number of notifications allowed over the given time window set by "Window Seconds" (default: 1). default: 1 minimum: 1 required: - service - throttling - if: properties: type: enum: - azure-blob-storage deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Azure Blob Storage description: Publish log events to Azure Blob Storage additionalProperties: false properties: batch_timeout_secs: type: integer title: Batch Timeout (seconds) description: >- The maximum amount of time, in seconds, events will be buffered before being flushed to the destination enum: - 30 - 60 - 90 - 120 - 300 default: 300 ack_enabled: type: boolean title: End-to-end Acknowledgement description: >- Sets end-to-end acknowledgement on the destination default: true encoding: type: string description: The encoding to apply to the data. enum: - json - ndjson - text title: Encoding default: text compression: type: string description: >- The compression strategy used on the encoded data prior to sending. enum: - gzip - none title: Compression default: none container_name: type: string title: Container Name minLength: 1 description: The name of the container for blob storage. connection_string: type: string title: Connection String minLength: 1 description: >- A connection string for the account that contains an access key prefix: type: string title: Prefix minLength: 1 description: A prefix to be applied to all object keys ui: display_type: data-template file_consolidation: type: object ui: display_type: collapsable feature_flag: >- pipeline-node-sink-azure-blob-file-consolidation title: File Consolidation description: >- This sink writes many small files out to azure blob storage. Enabling this process will allow the automatic consolidation of these small files into larger files of your choosing. This process will enable upon deployment and run on the chosen interval from `Processing Interval` creating files named `merged_[timestamp].log` where `timestamp` is the time since epoch when the actual file was created. The process will recursively access all files under the `Base Path` to handle merging sub-directory logging structures. additionalProperties: false properties: enabled: type: boolean default: false title: Enabled description: Toggles whether the process is enabled. process_every_seconds: type: integer title: Processing Interval (seconds) description: >- How often to run the consolidation process. default: 600 minimum: 300 maximum: 3600 requested_size_bytes: type: integer title: Requested File Size (bytes) description: >- The requested size of the consolidated files. default: 500000000 minimum: 50000000 maximum: 10000000000 base_path: type: string title: Base Path default: '' maxLength: 512 description: >- The path from the container to begin recursivelylooking for files to merge. A blank value indicates root.Merged files will only contain data from the respectivefolder. required: - encoding - compression - container_name - connection_string - if: properties: type: enum: - kinesis-firehose deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: AWS Kinesis Data Firehose description: Publish log events to AWS Kinesis Data Firehose additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: >- Sets end-to-end acknowledgement on the destination default: true auth: type: object title: AWS Authentication description: Configures AWS authentication additionalProperties: false properties: access_key_id: type: string title: Access Key ID minLength: 1 description: The AWS access key id secret_access_key: type: string title: Secret Access Key minLength: 1 description: The AWS secret access key ui: display_type: secret required: - access_key_id - secret_access_key encoding: type: string description: The encoding to apply to the data. enum: - json - text title: Encoding default: text compression: type: string description: >- The compression strategy used on the encoded data prior to sending. enum: - gzip - none title: Compression default: none region: type: string title: Region minLength: 1 description: The name of the AWS region that is targeted. stream_name: type: string title: Stream Name minLength: 1 description: The name of the Kinesis Firehose stream required: - auth - encoding - compression - region - stream_name - if: properties: type: enum: - kinesis-streams deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: AWS Kinesis Streams description: Publish log events to AWS Kinesis Streams additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: >- Sets end-to-end acknowledgement on the destination default: true auth: type: object title: AWS Authentication description: Configures AWS authentication additionalProperties: false properties: access_key_id: type: string title: Access Key ID minLength: 1 description: The AWS access key id secret_access_key: type: string title: Secret Access Key minLength: 1 description: The AWS secret access key ui: display_type: secret required: - access_key_id - secret_access_key encoding: type: string description: The encoding to apply to the data. enum: - json - text title: Encoding default: text compression: type: string description: >- The compression strategy used on the encoded data prior to sending. enum: - gzip - none title: Compression default: none partition_key_field: type: string title: Partition Key Field pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ description: >- The field in the log whose value is used as Kinesis' partition key. errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference region: type: string title: Region minLength: 1 description: The name of the AWS region that is targeted. stream_name: type: string title: Stream Name minLength: 1 description: The name of the targeted log stream. required: - auth - encoding - compression - region - stream_name - if: properties: type: enum: - elasticsearch deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Elasticsearch description: Publish log events to Elasticsearch additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: >- Sets end-to-end acknowledgement on the destination default: true compression: type: string description: >- The compression strategy used on the encoded data prior to sending. enum: - gzip - none title: Compression default: none auth: title: ElasticSearch Auth Strategy description: ElasticSearch authentication strategy allOf: - if: properties: strategy: enum: - basic then: properties: user: type: string title: User minLength: 1 maxLength: 512 description: The basic authentication user password: type: string title: Password minLength: 1 maxLength: 512 description: The basic authentication password ui: display_type: secret required: - user - password - if: properties: strategy: enum: - aws then: properties: access_key_id: type: string title: Access Key ID minLength: 1 maxLength: 1024 description: AWS access key ID ui: display_type: secret secret_access_key: type: string title: Secrete Access Key minLength: 1 maxLength: 1024 description: AWS secrete access key ui: display_type: secret region: type: string title: Region minLength: 1 maxLength: 128 description: >- The name of the AWS region that is targeted. required: - access_key_id - secret_access_key - region properties: strategy: type: string title: Strategy description: The ES authentication strategy to use enum: - basic - aws required: - strategy endpoints: type: array title: Endpoints uniqueItems: true minItems: 1 description: An array of ElasticSearch endpoints items: type: string title: URL format: url errorMessage: format: Must be a valid URL description: >- The full Elasticsearch URL to send data to. minLength: 1 maxLength: 1024 pipeline: type: string title: Pipeline minLength: 1 description: Name of an ES ingest pipeline to include. index: type: string minLength: 1 maxLength: 512 title: Custom Index description: >- Index to use when writing ES events (default = mezmo-%Y.%m.%d) default: mezmo-%Y.%m.%d required: - compression - auth - endpoints - if: properties: type: enum: - gcp-cloud-monitoring deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: GCP Cloud Monitoring description: Publish metrics events to GCP Cloud Monitoring additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: >- Sets end-to-end acknowledgement on the destination default: true credentials_json: type: string title: JSON Credentials minLength: 1 description: JSON Credentials ui: display_type: secret-file project_id: type: string title: Project ID minLength: 1 description: The Project ID as defined in Google Cloud. resource_type: type: string title: Resource Type minLength: 1 description: >- The `monitored-resource` type as defined in Monitoring. resource_labels: type: array title: Resource Labels description: >- An array of key/val objects describing labels of the Monitoring resource. items: type: object title: Label/Value description: >- A key/val object describing a label name and its value. additionalProperties: false properties: label_name: type: string title: Label Name minLength: 1 description: The name of the label. label_value: type: string title: Label Value minLength: 1 description: The value of the label. required: - label_name - label_value required: - credentials_json - project_id - resource_type - if: properties: type: enum: - gcp-cloud-storage deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: GCP Cloud Storage description: Publish log events to GCP Cloud Storage additionalProperties: false properties: batch_timeout_secs: type: integer title: Batch Timeout (seconds) description: >- The maximum amount of time, in seconds, events will be buffered before being flushed to the destination enum: - 30 - 60 - 90 - 120 - 300 default: 300 ack_enabled: type: boolean title: End-to-end Acknowledgement description: >- Sets end-to-end acknowledgement on the destination default: true credentials_json: type: string title: JSON Credentials minLength: 1 description: JSON Credentials ui: display_type: secret-file encoding: type: string description: >- Dictates how the data will be serialized before storing. enum: - json - ndjson - text title: Encoding default: text bucket: type: string title: Bucket minLength: 1 description: >- The name of the bucket in GCP where the data will be stored. compression: type: string description: >- The compression strategy used on the encoded data prior to sending. enum: - gzip - none title: Compression default: none bucket_prefix: type: string title: Bucket Prefix minLength: 1 description: >- The prefix applied to the bucket name, giving the appearance of having directories. required: - credentials_json - encoding - bucket - compression - if: properties: type: enum: - gcp-cloud-operations deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: GCP Cloud Operations description: Publish log events to GCP Cloud Operations additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: >- Sets end-to-end acknowledgement on the destination default: true credentials_json: type: string title: JSON Credentials minLength: 1 description: JSON Credentials ui: display_type: secret-file log_id: type: string title: Log ID minLength: 1 description: Concise reference for the log stream name. project_id: type: string title: Project ID minLength: 1 description: The Project ID as defined in Google Cloud. resource_type: type: string title: Resource Type minLength: 1 description: >- The `monitored-resource` type as defined in Monitoring. resource_labels: type: array title: Resource Labels description: >- An array of key/val objects describing labels of the Monitoring resource. items: type: object title: Label/Value description: >- A key/val object describing a label name and its value. additionalProperties: false properties: label_name: type: string title: Label Name minLength: 1 description: The name of the label. label_value: type: string title: Label Value minLength: 1 description: The value of the label. required: - label_name - label_value required: - credentials_json - log_id - project_id - resource_type - if: properties: type: enum: - gcp-cloud-pubsub deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: GCP Cloud PubSub description: Publish log events to GCP Cloud PubSub additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: >- Sets end-to-end acknowledgement on the destination default: true credentials_json: type: string title: JSON Credentials minLength: 1 description: JSON Credentials ui: display_type: secret-file encoding: type: string description: >- Dictates how the data will be serialized before storing. enum: - json - text title: Encoding default: text project_id: type: string title: Project ID minLength: 1 description: The Project ID as defined in Google Cloud. topic: type: string title: Topic minLength: 1 description: >- The name of the topic to publish messages to. required: - credentials_json - encoding - project_id - topic - if: properties: type: enum: - honeycomb-logs deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Honeycomb Logs description: Send log data to Honeycomb additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: >- Sets end-to-end acknowledgement on the destination default: true api_key: type: string title: Honeycomb API key description: Your Honeycomb API key minLength: 1 maxLength: 512 ui: display_type: secret dataset: type: string title: Dataset description: The name of the targeted dataset. minLength: 1 maxLength: 512 required: - api_key - dataset - if: properties: type: enum: - indexed-search deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Mezmo Indexed Search description: >- Publish log events to Mezmo's data store for Indexed Search capabilities additionalProperties: true properties: search_field: type: string title: Full Text Search Field pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ default: .line maxLength: 256 description: >- If your data reaches this stage as a plain-text string, the entire payload will be indexed for full-text searching. If your data reaches this stage as an object, the field you enter here will be used as the full-text search field. Nested fields are allowed. errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference timestamp_field: type: string title: Timestamp Field pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ default: timestamp maxLength: 256 description: >- If your data reaches this stage as an object, this field represents the timestamp value to be cataloged for searching. If the field does not exists the current time will be used. We accept UNIX timestamps (in milliseconds), ISO8601/RFC3389, and RFC2822 formats. errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference required: - search_field - timestamp_field - if: properties: type: enum: - kafka deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: title: Kafka description: Publish log events to Kafka additionalProperties: type: string allOf: - if: properties: sasl_enabled: enum: - true then: properties: sasl_mechanism: type: string enum: - SCRAM-SHA-512 - SCRAM-SHA-256 - PLAIN title: SASL description: The Kafka SASL/SCRAM mechanism default: SCRAM-SHA-512 sasl_username: type: string title: Username description: The SASL/SCRAM username minLength: 1 sasl_password: type: string title: Password description: The SASL/SCRAM username ui: display_type: secret minLength: 1 required: - sasl_mechanism - sasl_username - sasl_password - if: properties: tls_enabled: enum: - true then: properties: tls_ca_certificate: type: string title: TLS CA Certificate Chain minLength: 1 description: >- The CA certificate chain in the PEM format ui: display_type: secret-file format: x509_certificate errorMessage: format: Must be a valid PEM or DER certificate tls_verify_certificate: type: boolean title: TLS Verify Certificate description: Verify the TLS certificates default: true required: - tls_verify_certificate properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: >- Sets end-to-end acknowledgement on the destination default: true encoding: type: string description: The encoding to apply to the data. enum: - json - text title: Encoding default: text compression: type: string description: >- The compression strategy used on the encoded data prior to sending. enum: - gzip - lz4 - snappy - zstd - none title: Compression default: none event_key_field: type: string title: Event Key Field minLength: 1 description: >- The field in the log whose value is used as Kafka's event key. brokers: type: array title: Brokers uniqueItems: true minItems: 1 description: >- An array of key/val objects for each host/port of a Kafka broker. items: type: object title: Host/Port description: The host and port for the Kafka broker. additionalProperties: false properties: host: title: Host description: >- The hostname or IP address of the broker. anyOf: - type: string format: ipv4 errorMessage: format: Must be a valid IPv4 address - type: string format: hostname errorMessage: format: Must be a valid hostname port: type: number title: Port description: The port that the broker listens on. required: - host - port topic: type: string title: Topic minLength: 1 description: The name of the topic to publish to. group_id: type: string title: Group ID minLength: 1 description: The consumer group identifier sasl_enabled: type: boolean title: SASL/SCRAM Enabled description: Enable SASL/SCRAM authentication with Kafka default: false tls_enabled: type: boolean title: Use TLS description: Enable TLS during connections to the remote default: false required: - encoding - compression - brokers - topic - if: properties: type: enum: - loki deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Loki description: Publish log events to Loki additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: >- Sets end-to-end acknowledgement on the destination default: true auth: title: Loki Auth Strategy description: Loki authentication strategy allOf: - if: properties: strategy: enum: - basic then: properties: user: type: string title: User minLength: 1 maxLength: 512 description: The basic authentication user password: type: string title: Password minLength: 1 maxLength: 512 description: The basic authentication password ui: display_type: secret required: - user - password - if: properties: strategy: enum: - bearer then: properties: token: type: string title: Token minLength: 1 maxLength: 1024 description: The bearer token (passed as-is) ui: display_type: secret required: - token properties: strategy: type: string title: Strategy description: >- The authentication strategy to use (only basic supported) enum: - basic - bearer required: - strategy encoding: type: object title: Encoding description: Configures how event are encoded properties: codec: type: string title: Codec enum: - json - text required: - codec endpoint: type: string title: Endpoint format: url errorMessage: format: Must be a valid URL minLength: 1 maxLength: 512 description: The Loki base URL path: type: string title: Path minLength: 1 maxLength: 512 description: >- The path appended to the Loki base URL(defaults to /loki/api/v1/push) default: /loki/api/v1/push labels: type: array title: Loki Labels description: Key/Value pair used to describe Loki data items: type: object title: Labels description: Label name and values additionalProperties: false properties: label_name: type: string title: Label Name description: The name of the label minLength: 1 maxLength: 512 label_value: type: string title: Label Value description: The value of the label minLength: 1 maxLength: 512 required: - label_name - label_value required: - auth - encoding - endpoint - labels - if: properties: type: enum: - new-relic deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: New Relic description: Publish to New Relic additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: >- Sets end-to-end acknowledgement on the destination default: true account_id: type: string title: New Relic Account ID description: Account ID for your New Relic account minLength: 1 maxLength: 1024 ui: display_type: secret api: type: string title: API description: >- New Relic API endpoint (events not supported) enum: - logs - metrics default: logs license_key: type: string title: New Relic License Key description: License key for your New Relic account minLength: 1 maxLength: 1024 ui: display_type: secret required: - account_id - api - license_key - if: properties: type: enum: - prometheus-remote-write deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Prometheus Remote Write description: >- Publish metrics to Prometheus Remote Write endpoint additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: >- Sets end-to-end acknowledgement on the destination default: true endpoint: type: string title: Endpoint format: uri errorMessage: format: Must be a valid URI description: >- The full URI to make HTTP requests to. This should include the protocol and host, but can also include the port, path, and any other valid part of a URI. Example: http://example.org:8080/api/v1/push minLength: 1 maxLength: 2000 auth: title: Authentication description: Configures HTTP authentication allOf: - if: properties: strategy: enum: - basic then: properties: user: type: string title: User minLength: 1 maxLength: 512 description: The basic authentication user password: type: string title: Password minLength: 1 maxLength: 512 description: The basic authentication password ui: display_type: secret required: - user - password - if: properties: strategy: enum: - bearer then: properties: token: type: string title: Token minLength: 1 maxLength: 1024 description: The bearer token ui: display_type: secret required: - token properties: strategy: type: string enum: - basic - bearer - none title: Strategy description: The authentication strategy to use default: none required: - strategy required: - endpoint - if: properties: type: enum: - pulsar deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Pulsar description: Publish log events to Apache Pulsar additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: >- Sets end-to-end acknowledgement on the destination default: true encoding: type: string description: The encoding to apply to the data. enum: - json - text title: Encoding default: text url: type: string title: URL format: url errorMessage: format: Must be a valid URL description: The full Pulsar URL to send data to. topic: type: string title: Topic minLength: 1 description: The name of the Pulsar topic to publish to. required: - encoding - url - topic - if: properties: type: enum: - redis deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Redis description: Publish log events to Redis additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: >- Sets end-to-end acknowledgement on the destination default: true encoding: type: string description: The encoding to apply to the data. enum: - json - text title: Encoding default: text url: type: string title: URL format: uri errorMessage: format: Must be a valid URL description: The full Redis URL to send data to. key: type: string title: Key minLength: 1 description: The name of the Redis key to write data to. required: - encoding - url - key - if: properties: type: enum: - splunk-hec-logs deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Splunk HTTP Event Collector description: >- Publish log events to Splunk HTTP Event Collector additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: >- Sets end-to-end acknowledgement on the destination default: true compression: type: string description: >- The compression strategy used on the encoded data prior to sending. enum: - gzip - none title: Compression default: none url: type: string title: Endpoint format: url errorMessage: format: Must be a valid URL description: >- The base URL for the Splunk instance. The collector path, such as `/services/collector/events`, will be automatically inferred from the destination's configuration. token: type: string title: Token minLength: 1 ui: display_type: secret description: >- The default token to authenticate to Splunk HEC. tls_verify_certificate: type: boolean title: Verify TLS Certificate description: >- If unchecked, the TLS certificate of the remote host will not be verified. **This is insecure**, but may be useful for testing or when evaluating Splunk in a trial instance. default: true host_field: type: string title: Host Field description: >- The field that contains the hostname to include in the event. default: metadata.host timestamp_field: type: string title: Timestamp Field description: >- The field that contains the timestamp to include in the event. default: metadata.time pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ minLength: 1 maxLength: 512 errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: feature_flag: pipeline-splunk-sink-v2 display_type: field-reference source: title: Source description: >- The source of events sent to this sink. This is typically the filename the logs originated from. Use the field path ".metadata.source" to use the upstream source value from a HEC log source additionalProperties: false allOf: - if: properties: value_type: enum: - field then: title: Value properties: value: type: string title: Field value description: The field to use as the value. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference required: - value - if: properties: value_type: enum: - value required: - value_type then: title: Value properties: value: type: string title: Value description: The value. minLength: 1 maxLength: 512 required: - value properties: value_type: title: Value type description: >- Either a value from an event field, a new value input, or none to omit value. oneOf: - label: Value from event field enum: - field - label: New value enum: - value - label: None enum: - none default: none ui: display_type: radio-group value: ui: feature_flag: pipeline-splunk-sink-v2 type: - string - 'null' required: - value_type sourcetype: title: Source Type description: >- The sourcetype of events sent to this sink. Use the field path ".metadata.sourcetype" to use the upstream sourcetype value from a HEC log source additionalProperties: false allOf: - if: properties: value_type: enum: - field then: title: Value properties: value: type: string title: Field value description: The field to use as the value. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference required: - value - if: properties: value_type: enum: - value required: - value_type then: title: Value properties: value: type: string title: Value description: The value. minLength: 1 maxLength: 512 required: - value properties: value_type: title: Value type description: >- Either a value from an event field, a new value input, or none to omit value. oneOf: - label: Value from event field enum: - field - label: New value enum: - value - label: None enum: - none default: none ui: display_type: radio-group value: ui: feature_flag: pipeline-splunk-sink-v2 type: - string - 'null' required: - value_type index: title: Index description: >- The name of the index to send events to. Use the field path ".metadata.index" to use the upstream index value from a HEC log source additionalProperties: false allOf: - if: properties: value_type: enum: - field then: title: Value properties: value: type: string title: Field value description: The field to use as the value. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference required: - value - if: properties: value_type: enum: - value required: - value_type then: title: Value properties: value: type: string title: Value description: The value. minLength: 1 maxLength: 512 required: - value properties: value_type: title: Value type description: >- Either a value from an event field, a new value input, or none to omit value. oneOf: - label: Value from event field enum: - field - label: New value enum: - value - label: None enum: - none default: none ui: display_type: radio-group value: ui: feature_flag: pipeline-splunk-sink-v2 type: - string - 'null' required: - value_type required: - compression - url - token - source - sourcetype - index - if: properties: type: enum: - sumo-logic-logs deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Sumo Logic Logs description: Publish log data to Sumo Logic additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: >- Sets end-to-end acknowledgement on the destination default: true endpoint: type: string title: Collector Endpoint format: url errorMessage: format: Must be a valid URL minLength: 1 maxLength: 512 description: >- The Sumo Logic Collector URL that receives the data ui: display_type: secret compression: type: string title: Compression description: The compression strategy (defaults to none). enum: - gzip - none default: none category: type: string minLength: 1 maxLength: 512 title: Category description: >- Category to use when publishing data to Sumo Logic (default = mezmo_default_category) default: mezmo_default_category required: - endpoint - compression - if: properties: type: enum: - sumo-logic-metrics deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Sumo Logic Metrics description: Publish metrics data to Sumo Logic additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: >- Sets end-to-end acknowledgement on the destination default: true endpoint: type: string title: Collector Endpoint format: url errorMessage: format: Must be a valid URL minLength: 1 maxLength: 512 description: >- The Sumo Logic Collector URL that receives the data ui: display_type: secret compression: type: string title: Compression description: The compression strategy (defaults to none). enum: - gzip - none default: none category: type: string minLength: 1 maxLength: 512 title: Category description: >- Category to use when publishing data to Sumo Logic (default = mezmo_default_category) default: mezmo_default_category required: - endpoint - compression - if: properties: type: enum: - vector deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Vector description: Publish log events to Vector additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: >- Sets end-to-end acknowledgement on the destination default: true compression: type: string description: >- The compression strategy used on the encoded data prior to sending. enum: - gzip - none title: Compression default: none vector_ip: type: string title: Vector IP format: ipv4 description: The IP portion of the Vector host vector_port: type: number title: Vector Port description: The port that the Vector host listens on. version: type: string enum: - '1' - '2' title: Version default: '2' description: >- The version of the Vector sink which must match the source version. required: - compression - vector_ip - vector_port - version - if: properties: type: enum: - s3 deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: AWS S3 description: Publish events as objects in AWS S3 additionalProperties: false errorMessage: properties: bucket: must be a valid bucket name tags: must be a valid tag identifier properties: batch_timeout_secs: type: integer title: Batch Timeout (seconds) description: >- The maximum amount of time, in seconds, events will be buffered before being flushed to the destination enum: - 30 - 60 - 90 - 120 - 300 default: 300 ack_enabled: type: boolean title: End-to-end Acknowledgement description: >- Sets end-to-end acknowledgement on the destination default: true auth: type: object title: AWS Authentication description: Configures AWS authentication additionalProperties: false properties: access_key_id: type: string title: Access Key ID minLength: 1 description: The AWS access key id secret_access_key: type: string title: Secret Access Key minLength: 1 description: The AWS secret access key ui: display_type: secret required: - access_key_id - secret_access_key bucket: type: string title: Bucket pattern: >- (?!(^xn--|.+-s3alias$))^[a-z0-9][a-z0-9-]{1,61}[a-z0-9]$ description: >- The S3 bucket name. Do not include a leading s3:// or a trailing / prefix: type: string title: Prefix description: A prefix to apply to all object key names. default: / ui: display_type: data-template tags: type: array maxItems: 10 title: Tags description: >- A set of tags (key/value pairs) to apply to the object in S3. items: type: object properties: key: type: string title: Name description: The tag name pattern: ^[A-Za-z0-9_.:=+@/\-\s]*$ minLength: 1 maxLength: 128 value: type: string title: Value description: The tag value default: '' pattern: ^[A-Za-z0-9_.:=+@/\-\s]*$ minLength: 0 maxLength: 256 required: - key - value encoding: type: string description: The encoding to apply to the data. enum: - json - ndjson - text title: Encoding default: text compression: type: string description: >- The compression strategy used on the encoded data prior to sending. enum: - gzip - none title: Compression default: none region: type: string title: Region minLength: 1 description: The name of the AWS region that is targeted. file_consolidation: type: object ui: display_type: collapsable feature_flag: pipeline-node-sink-s3-file-consolidation title: File Consolidation description: >- This sink writes many small files out to S3. Enabling this process will allow the automatic consolidation of these small files into larger files of your choosing. This process will enable upon deployment and run on the chosen interval from `Processing Interval` creating files named `merged_[timestamp].log` where `timestamp` is the time since epoch. *NOTE*: only `text` and `ndjson` files written by the sink are consolidated. additionalProperties: false properties: enabled: type: boolean default: false title: Enabled description: Toggles whether the process is enabled. process_every_seconds: type: integer title: Processing Interval (seconds) description: >- How often to run the consolidation process. default: 600 minimum: 300 maximum: 3600 requested_size_bytes: type: integer title: Requested File Size (bytes) description: The size of the requested files. default: 500000000 minimum: 50000000 maximum: 10000000000 base_path: type: string title: Base Path default: '' maxLength: 512 description: >- The path from the container to begin recursivelylooking for files to merge. A blank value indicates root.Merged files will only contain data from the respectivefolder. required: - auth - bucket - prefix - encoding - compression - region - if: properties: type: enum: - blackhole deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Blackhole description: Drop data arriving at this destination additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: >- Sets end-to-end acknowledgement on the destination default: true - if: properties: type: enum: - datadog-logs deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Datadog Logs description: Publish log events to Datadog additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: >- Sets end-to-end acknowledgement on the destination default: true api_key: type: string title: Datadog API Key description: Datadog log application API key minLength: 1 ui: display_type: secret compression: type: string description: >- The compression strategy used on the encoded data prior to sending. enum: - gzip - none title: Compression default: gzip site: type: string description: The Datadog site (region) to send logs to. enum: - us1 - us3 - us5 - eu1 title: Datadog Site default: us1 required: - api_key - compression - site - if: properties: type: enum: - datadog-metrics deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Datadog Metrics description: >- Publish metrics events to Datadog. Sending custom metrics to Datadog may result in extra charges on your Datadog bill. Please go to https://docs.datadoghq.com/metrics/custom_metrics/ for more information on Datadog custom metrics pricing. additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: >- Sets end-to-end acknowledgement on the destination default: true api_key: type: string title: Datadog API Key description: Datadog metrics application API key minLength: 1 maxLength: 512 ui: display_type: secret site: type: string description: >- The Datadog site (region) to send metrics to. enum: - us1 - us3 - us5 - eu1 title: Datadog Site default: us1 required: - api_key - site - if: properties: type: enum: - clickhouse deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Clickhouse description: Deliver log data to the ClickHouse database additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: >- Sets end-to-end acknowledgement on the destination default: true endpoint: type: string minLength: 1 maxLength: 2048 title: Endpoint description: Endpoint of the Clickhouse server format: uri errorMessage: format: Must be a valid URI auth: title: Auth Strategy description: Authentication strategy allOf: - if: properties: strategy: enum: - basic then: properties: user: type: string title: User minLength: 1 maxLength: 512 description: The basic authentication user password: type: string title: Password minLength: 1 maxLength: 512 description: The basic authentication password ui: display_type: secret required: - user - password - if: properties: strategy: enum: - bearer then: properties: token: type: string title: Token minLength: 1 maxLength: 2048 description: >- The bearer token to use for authentication ui: display_type: secret required: - token properties: strategy: type: string title: Strategy description: The ES authentication strategy to use enum: - basic - bearer required: - strategy table: type: string title: Table description: The table that data is inserted into pattern: ^[a-zA-Z0-9_]+$ errorMessage: Invalid table name ui: display_type: data-template database: type: string title: Database description: >- The database that contains the table that data is inserted into pattern: ^[a-zA-Z0-9_]+$ errorMessage: Invalid database name ui: display_type: data-template compression: type: string enum: - gzip - none - snappy - zlib - zstd title: Compression description: The compression algorithm to use default: gzip skip_unknown_fields: type: boolean title: Skip Unknown Fields description: >- Allows ClickHouse to discard fields not present in the table schema default: false required: - endpoint - auth - table - if: properties: type: enum: - opentelemetry deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: OpenTelemetry description: Publish data to OpenTelemetry additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: >- Sets end-to-end acknowledgement on the destination default: true endpoint: type: string title: Collector Endpoint format: url errorMessage: format: Must be a valid URL minLength: 1 maxLength: 512 description: >- The OpenTelemetry Collector URL that receives the data auth: title: Authentication description: Configures HTTP authentication additionalProperties: type: string allOf: - if: properties: strategy: enum: - basic then: properties: user: type: string title: User minLength: 1 maxLength: 512 description: The basic authentication user password: type: string title: Password minLength: 1 maxLength: 512 description: The basic authentication password ui: display_type: secret required: - user - password - if: properties: strategy: enum: - bearer then: properties: token: type: string title: Token minLength: 1 maxLength: 1024 description: The bearer token ui: display_type: secret required: - token properties: strategy: type: string enum: - basic - bearer - none title: Strategy description: The authentication strategy to use default: none headers: type: array title: HTTP Headers description: >- An array of key/value objects for the headers to be sent with the request. items: type: object title: Headers description: >- A key/value object describing a header name and its value. additionalProperties: false properties: header_name: type: string title: Header Name minLength: 1 description: The name of the header. header_value: type: string title: Header Value minLength: 1 description: The value of the header. required: - header_name - header_value compression: type: string title: Compression description: The compression strategy (defaults to none). enum: - gzip - none default: none required: - endpoint - compression properties: title: oneOf: - type: string minLength: 1 maxLength: 512 pattern: ^[^<>"]+$ - type: 'null' errorMessage: invalid title name description: oneOf: - type: string minLength: 1 pattern: ^[^<>"]+$ - type: 'null' errorMessage: invalid description type: type: string enum: - mezmo - mezmo-archive - http - cloudwatch-logs - cloudwatch-metrics - sqs - notification-channel - azure-blob-storage - kinesis-firehose - kinesis-streams - elasticsearch - gcp-cloud-monitoring - gcp-cloud-storage - gcp-cloud-operations - gcp-cloud-pubsub - honeycomb-logs - indexed-search - kafka - loki - new-relic - prometheus-remote-write - pulsar - redis - splunk-hec-logs - sumo-logic-logs - sumo-logic-metrics - vector - s3 - blackhole - datadog-logs - datadog-metrics - clickhouse - opentelemetry user_config: type: object required: - user_config required: - sources - transforms - sinks required: true responses: '200': description: Default Response components: securitySchemes: PipelineServiceKey: type: apiKey name: Authorization in: header description: 'Enter your token in the format: Token mytokengoeshere' ````