openapi: 3.1.0 x-generated-by: merge-api info: title: Mezmo API description: |- *REST API for the Mezmo platform: send and export log data, build and manage telemetry pipelines, and administer enterprise accounts.* To use the inline testing functionality here, go into your Mezmo account and add the following address to your CORS Origins page: `https://docs.mezmo.com`. version: 3.0.0 contact: name: Mezmo email: support@mezmo.com url: https://mezmo.com/support servers: - url: https://api.mezmo.com externalDocs: url: https://docs.mezmo.com security: - AccessToken: [] tags: - name: Account description: Perform account level actions - name: Alerting description: API to control alerts on pipelines - name: Archiving description: > >*Authentication*: You can find instructions on authentication [here](/log-analysis-api/ref#authentication). The LogDNA Archiving API enables you to programmatically configure your archiving instances via code. Specifically you can set up cold storage buckets to send your logs for long term storage. - name: Board description: > >*Authentication*: You can find instructions on authentication [here](/log-analysis-api/ref#authentication). The LogDNA's API's Board endpoints are used to create, get, list, and delete boards. - name: Configuration description: > >*Authentication*: You can find instructions on authentication [here](/log-analysis-api/ref#authentication). The LogDNA API's Configuration endpoints are used to create, update, and delete views, alerts, and categories. - name: Data Classification & Profiling description: API for classifying and profiling data - name: Edge - Local Deploy description: User routes that are available to use with local Edge instances - name: Enterprise description: Routes for management enterprise level resources - name: Enterprise v1 description: Legacy enterprise management endpoints, superseded by the Enterprise API. - name: Exclusions description: > >*Authentication*: You can find instructions on authentication [here](/log-analysis-api/ref#authentication). Control what you store by creating exclusion rules. New lines that match an exclusion rule will not be stored and will not count toward your storage quota. - name: Export description: > LogDNA provides two endpoints for exporting your log lines. Read the sections below to learn about each, and determine which is appropriate for your use case. ## Export API v1 The Export API v1 is used to export log lines in [JSONL](https://jsonlines.org/) format. Based on your specific plan, the maximum number of logs returned is limited to 10,000 / 20,000. There are two distinct functionalities: 1. Streaming desired log lines directly from the response as raw text data 2. Specifying an email to receive a link that can be used to the download the desired log lines > Note > When an email address is specified, the desired log lines will not be streamed to the response body. > The response will instead send a notification indicating that the request was successful and that the results > will be sent to the email address provided in the request. This entails archiving and uploading the desired logs > to a storage provider. The email includes a URI to download the logs. Be aware that access to this URI will expire > after 24-48 hours. ## Export API v2 The Export API v2 provides an enhancement over v1, allowing you to export any number of log lines using pagination and by-passing the plan-specific limit imposed in v1. By making (multiple) subsequent requests using the `pagination_id` parameter, you can retrieve logs in batches of JSON until all matching log lines have been returned. Each Export API v2 request is limited to 10,000 lines. ### Pagination Functionality Pagination is only necessary when the desired set of log lines is larger than 10,000, since this is the maximum number of results that can be returned in a given page. If an export does not exceed this size, the behavior is similar to v1 -- all of the logs will be returned in a single "page" and no further requests are needed. When an export does exceed this size, refer to the steps outlined below. This is the expected workflow to export a set (cardinality > 10,000) of log lines that requires pagination: 1. Send an initial request with valid query parameters to retrieve logs. Pass a `null` value for the `pagination_id` in the first request. 2. The JSON response contains 2 fields: `lines` and `pagination_id`. The former is an array containing the corresponding batch (the first batch in this case) of logs and the latter is a token used to retrieve the next page of results. 3. Send a subsequent request using the same initial query parameters and also passing in the token received in the response to the `pagination_id` parameter. This will retrieve the next page of results. 4. Repeat this request-response cycle, updating the `pagination_id` parameter with the token returned in every new response, until the response eventually returns a `null` value for the token. This indicates that the corresponding batch of logs is the last and that you have reached the final page. > Important Considerations > In subsequent requests to paginate through results, make sure none of the query parameters & values are modified from the initial request > (excluding `pagination_id`) > The tokens used in the `pagination_id` parameter are tied to a specific export and page so they can be used repeatedly in requests to return > deterministic results > "0" can be passed in to the `from` and `to` parameters which will be resolved to timestamps for a plan's retention boundary and the current > time respectively. This behavior is independent so the value can be passed interchangeably to the parameters -- resulting in 4 possible scenarios: > - [`from`: \, `to`: \] > - [`from`: 0 (retention boundary), `to`: \] > - [`from`: \, `to`: 0 (current time)] > - [`from`: 0 (retention boundary), `to`: 0 (current time)] - name: Groups description: Manage log groups that scope which data is accessible to members in that group. - name: Incident Mode description: Change or read state of pipeline (incident mode on/off) - name: Index Rate Alert description: Configure alerts that fire when the log index rate crosses a threshold. - name: Ingestion description: Ingestion API - name: Keys description: Create and manage ingestion and service keys for an account. - name: Log Volume Reduction description: CRUD operations for log volume reduction - name: Members description: Manage organization members, their roles, and RBAC group assignments. - name: Metrics description: Usage metrics for pipelines - name: Parsing Template description: Manage custom log parsing templates. - name: Pipeline Export description: Export pipeline to terraform - name: Pipeline Management description: CRUD Operations for pipeline - name: Pipeline Nodes description: CRUD operations for sources, processors, and destinations - name: Processor Groups description: CRUD operations for Processor Groups - name: Sampling and Simulation description: >- Create samples of data and run them through a simulated pipeline without affecting your published pipeline - name: Screens description: Create and manage custom UI layouts for display on large format screens. - name: Shared Sources description: Create and manage sources for use across multiple pipelines - name: Suspensions description: > >*Authentication*: You can find instructions on authentication [here](/log-analysis-api/ref#authentication). The LogDNA Start/Stop Ingestion API allows users to programmatically manage their data ingestion status. Users can get the current status of their ingestion, suspend their ingestion, and resume their ingestion. To prevent an accidental API call, the suspension of ingestion requires two subsequent API calls. > You can find instructions on authentication [here](/log-analysis-api/ref#authentication). - name: Usage v1 description: > >*Authentication*: You can find instructions on authentication [here](/log-analysis-api/ref#authentication). The Usage API provides endpoints for retrieving aggregated usage information for applications, hosts and tags during a time period. While the `to` and `from` times are unix timestamps, the report is granular by day. Therefore only the day portion is used to establish a date range. ### How is **Usage** defined? This API defines usage as a percentage of the total number of lines ingested by the account within the defined period of time. For example, an app called `myapp` that returns a percentage of `15` in the response object has 15 percent of their logs coming from `myapp`. - name: Usage v2 description: > >*Authentication*: You can find instructions on authentication [here](/log-analysis-api/ref#authentication). The Usage API provides endpoints for retrieving aggregated usage information for applications, hosts and tags during a time period. While the `to` and `from` times are unix timestamps, the report is granular by day. Therefore only the day portion is used to establish a date range. ### How is **Usage** defined? This API defines usage as the number of bytes used by the log lines stored on disk by the account within the defined period of time. For example, an app called `myapp` that returns `150,000` total bytes in the response object indicates that `myapp` is consuming `150,000` bytes of disk storage over the time range. paths: /v3/pipeline/account: get: summary: Get account details tags: - Account description: Get list of settings applied to an account responses: '200': description: Default Response patch: summary: Update account details tags: - Account description: Update account settings requestBody: content: application/json: schema: type: object properties: local_deploy_config: type: object additionalProperties: false ui: feature_flag: pipeline-pulse properties: start_port: type: integer title: Start port minimum: 1 maximum: 65535 description: The lower bound of the port range end_port: type: integer exclusiveMinimum: $data: 1/start_port maximum: 65535 title: End port description: The upper bound of the port range required: - start_port - end_port responses: '200': description: Default Response /v3/pipeline/{pipeline_id}/processing-status: put: summary: Update pipeline processing status tags: - Account description: Change a pipeline to enabled or disabled requestBody: content: application/json: schema: type: object properties: processing_status: type: string enum: - enabled - disabled required: - processing_status required: true parameters: - schema: type: string format: uuid in: path name: pipeline_id required: true responses: '200': description: Default Response /v3/pipeline/{pipeline_id}/{component_kind}/{component_id}/alert/{alert_id}: delete: summary: Delete component alert tags: - Alerting description: Removes an alert attached to a pipeline component parameters: - schema: type: string enum: - node_alert - surge_detection_alert default: node_alert in: query name: component_alert_type required: false - schema: type: string format: uuid in: path name: pipeline_id required: true - schema: type: string enum: - source - transform in: path name: component_kind required: true - schema: type: string format: uuid in: path name: component_id required: true - schema: type: string format: uuid in: path name: alert_id required: true responses: '200': description: Default Response put: summary: Update component alert tags: - Alerting description: Update an alert attached to a pipeline component requestBody: content: application/json: schema: type: object additionalProperties: false properties: inputs: type: array items: type: string minItems: 1 alert_config: type: object properties: general: type: object evaluation: type: object alert_payload: type: object required: - general - evaluation - alert_payload active: type: boolean component_alert_type: type: string enum: - node_alert - surge_detection_alert default: node_alert required: - inputs required: true parameters: - schema: type: string format: uuid in: path name: pipeline_id required: true - schema: type: string enum: - source - transform in: path name: component_kind required: true - schema: type: string format: uuid in: path name: component_id required: true - schema: type: string format: uuid in: path name: alert_id required: true responses: '200': description: Default Response /v3/pipeline/{pipeline_id}/alert/{alert_id}: get: summary: Fetch component alert tags: - Alerting description: Retrieves an alert attached to a pipeline component parameters: - schema: type: string enum: - node_alert - surge_detection_alert default: node_alert in: query name: component_alert_type required: false - schema: type: string format: uuid in: path name: pipeline_id required: true - schema: type: string format: uuid in: path name: alert_id required: true responses: '200': description: Default Response /v3/pipeline/{pipeline_id}/alert: get: summary: List component alerts tags: - Alerting description: List of alerts attached to a pipeline's components parameters: - schema: type: string format: uuid in: path name: pipeline_id required: true responses: '200': description: Default Response /v3/pipeline/{pipeline_id}/{component_kind}/{component_id}/alert: post: summary: Create component alert tags: - Alerting description: Create an alert attached to a pipeline component requestBody: content: application/json: schema: type: object additionalProperties: false properties: inputs: type: array items: type: string minItems: 1 alert_config: type: object properties: general: type: object evaluation: type: object alert_payload: type: object required: - general - evaluation - alert_payload component_alert_type: type: string enum: - node_alert - surge_detection_alert default: node_alert required: - inputs - component_alert_type required: true parameters: - schema: type: string format: uuid in: path name: pipeline_id required: true - schema: type: string enum: - source - transform in: path name: component_kind required: true - schema: type: string format: uuid in: path name: component_id required: true responses: '200': description: Default Response /v3/pipeline/{pipeline_id}/usage-threshold-alert/{usage_threshold_alert_id}: delete: summary: Remove usage threshold alert tags: - Alerting description: Removes an existing threshold alert parameters: - schema: type: string format: uuid in: path name: pipeline_id required: true - schema: type: string format: uuid in: path name: usage_threshold_alert_id required: true responses: '200': description: Default Response get: summary: Retrieves an existing usage threshold alert tags: - Alerting description: Returns usage threshold configuration parameters: - schema: type: string format: uuid in: path name: pipeline_id required: true - schema: type: string format: uuid in: path name: usage_threshold_alert_id required: true responses: '200': description: Default Response patch: summary: Partially update an existing threshold alert tags: - Alerting description: Update a part of usage threshold alert requestBody: content: application/json: schema: type: object properties: enabled: type: boolean required: - enabled required: true parameters: - schema: type: string format: uuid in: path name: pipeline_id required: true - schema: type: string format: uuid in: path name: usage_threshold_alert_id required: true responses: '200': description: Default Response put: summary: Update threshold alert tags: - Alerting description: Replace an existing usage alert with this one requestBody: content: application/json: schema: type: object additionalProperties: false allOf: - if: allOf: - type: object properties: no_data: not: enum: - true - type: object properties: percent_decrease: anyOf: - enum: - 0 - type: 'null' - type: object properties: percent_increase: anyOf: - enum: - 0 - type: 'null' then: properties: threshold_bytes: type: number minimum: 1 errorMessage: minimum: must be greater than or equal to 1 required: - threshold_bytes - if: allOf: - type: object properties: no_data: not: enum: - true - type: object properties: percent_increase: anyOf: - enum: - 0 - type: 'null' - type: object properties: threshold_bytes: anyOf: - enum: - 0 - type: 'null' then: properties: percent_decrease: type: number minimum: 1 errorMessage: minimum: must be greater than or equal to 1 required: - percent_decrease - if: allOf: - type: object properties: no_data: not: enum: - true - type: object properties: percent_decrease: anyOf: - enum: - 0 - type: 'null' - type: object properties: threshold_bytes: anyOf: - enum: - 0 - type: 'null' then: properties: percent_increase: type: number minimum: 1 errorMessage: minimum: must be greater than or equal to 1 required: - percent_increase properties: data_window: type: object additionalProperties: false title: Data period options properties: value: $ref: '#alert_period_value' unit: $ref: '#alert_period_unit' if: properties: unit: title: Data period unit enum: - minute required: - unit then: properties: value: type: number title: Data period value minimum: 5 required: - value required: - value - unit max_daily_notifications: $ref: '#alert_max_notifications' no_data: type: boolean percent_decrease: $ref: '#alert_percent_decrease' percent_increase: $ref: '#alert_percent_increase' threshold_bytes: $ref: '#alert_threshold_bytes' title: $ref: '#alert_title' traffic_type: $ref: '#alert_traffic_type' alert_channels: $ref: '#alert_channels' required: - traffic_type - alert_channels required: true parameters: - schema: type: string format: uuid in: path name: pipeline_id required: true responses: '200': description: Default Response /v3/pipeline/{pipeline_id}/usage-threshold-alert: get: summary: List all threshold alerts tags: - Alerting description: Returns configuration of existing threshold alerts parameters: - schema: type: string format: uuid in: path name: pipeline_id required: true responses: '200': description: Default Response post: summary: Add usage alert tags: - Alerting description: Creates a new usage threshold alert configuration requestBody: content: application/json: schema: type: object additionalProperties: false allOf: - if: allOf: - type: object properties: no_data: not: enum: - true - type: object properties: percent_decrease: anyOf: - enum: - 0 - type: 'null' - type: object properties: percent_increase: anyOf: - enum: - 0 - type: 'null' then: properties: threshold_bytes: type: number minimum: 1 errorMessage: minimum: must be greater than or equal to 1 required: - threshold_bytes - if: allOf: - type: object properties: no_data: not: enum: - true - type: object properties: percent_increase: anyOf: - enum: - 0 - type: 'null' - type: object properties: threshold_bytes: anyOf: - enum: - 0 - type: 'null' then: properties: percent_decrease: type: number minimum: 1 errorMessage: minimum: must be greater than or equal to 1 required: - percent_decrease - if: allOf: - type: object properties: no_data: not: enum: - true - type: object properties: percent_decrease: anyOf: - enum: - 0 - type: 'null' - type: object properties: threshold_bytes: anyOf: - enum: - 0 - type: 'null' then: properties: percent_increase: type: number minimum: 1 errorMessage: minimum: must be greater than or equal to 1 required: - percent_increase properties: data_window: type: object additionalProperties: false title: Data period options properties: value: $ref: '#alert_period_value' unit: $ref: '#alert_period_unit' if: properties: unit: title: Data period unit enum: - minute required: - unit then: properties: value: type: number title: Data period value minimum: 5 required: - value required: - value - unit max_daily_notifications: $ref: '#alert_max_notifications' no_data: type: boolean percent_decrease: $ref: '#alert_percent_decrease' percent_increase: $ref: '#alert_percent_increase' threshold_bytes: $ref: '#alert_threshold_bytes' title: $ref: '#alert_title' traffic_type: $ref: '#alert_traffic_type' alert_channels: $ref: '#alert_channels' required: - traffic_type - alert_channels required: true parameters: - schema: type: string format: uuid in: path name: pipeline_id required: true responses: '200': description: Default Response /v1/config/archiving: get: tags: - Archiving summary: Get Archive Configuration description: Use this method to get an existing archiving configuration. operationId: get-v1-config-archiving responses: '200': description: OK content: application/json: schema: $ref: '#/components/schemas/archivingConfigRequest' '404': description: Archiving configuration not found content: '*/*': schema: $ref: '#/components/schemas/errPropsDefault' post: tags: - Archiving summary: Create Archive Configuration description: >- Use this method to configure archiving for an instance. Only one archiving configuration may exist at any time. operationId: post-v1-config-archiving requestBody: description: Request parameters content: application/json: schema: $ref: '#/components/schemas/archivingConfigRequest' required: true responses: '200': description: OK content: application/json: schema: $ref: '#/components/schemas/archivingConfigRequest' '400': description: Bad request content: '*/*': schema: $ref: '#/components/schemas/errPropsDefault' '409': description: An archiving configuration already exists. Cannot create a new one content: '*/*': schema: $ref: '#/components/schemas/errPropsDefault' deprecated: false x-codegen-request-body-name: configRequest put: tags: - Archiving summary: Update Archive Configuration description: Use this method to update an existing archiving configuration. operationId: put-v1-config-archiving requestBody: description: Request parameters content: application/json: schema: $ref: '#/components/schemas/archivingConfigRequest' required: true responses: '200': description: OK content: application/json: schema: $ref: '#/components/schemas/archivingConfigRequest' '400': description: Bad request content: '*/*': schema: $ref: '#/components/schemas/errPropsDefault' '404': description: Existing archiving integration not found content: '*/*': schema: $ref: '#/components/schemas/errPropsDefault' deprecated: false x-codegen-request-body-name: configRequest delete: tags: - Archiving summary: Delete Archive Configuration description: Use this method to delete the archiving configuration. operationId: delete-v1-config-archiving responses: '200': description: OK content: {} '404': description: Archiving configuration not found content: '*/*': schema: $ref: '#/components/schemas/errPropsDefault' /v1/config/board: get: tags: - Board summary: List Boards description: Get a list of boards. operationId: list-board responses: '200': description: OK content: application/json: schema: type: array items: type: object properties: boardid: type: string example: '0000000000' account: $ref: '#/components/schemas/accountId' title: type: string description: Name of the board '400': description: Bad request content: {} post: tags: - Board summary: Create Board description: Use this method to create a board. operationId: create-board requestBody: description: Request parameters content: application/json: schema: $ref: '#/components/schemas/boardRequest' required: true responses: '200': description: OK content: application/json: schema: type: object additionalProperties: type: object properties: {} '400': description: Bad request content: {} deprecated: false x-codegen-request-body-name: configRequest /v1/config/board/{boardId}: get: tags: - Board summary: Get Board description: Gets the configuration for a specific board. operationId: get-board parameters: - name: boardId in: path description: ID of a board. required: true schema: type: string responses: '200': description: OK content: application/json: schema: type: object properties: boardid: type: string example: '0000000000' account: $ref: '#/components/schemas/accountId' title: type: string description: Name of the board graphs: type: array items: type: object order: type: array items: type: string category: type: array items: type: string '400': description: Bad request content: {} delete: tags: - Board summary: Delete Board description: Use this method to delete a board. operationId: delete-board parameters: - name: boardId in: path description: ID of a board. required: true schema: type: string responses: '200': description: OK content: application/json: schema: type: object additionalProperties: type: object properties: {} '400': description: Bad request content: {} deprecated: false x-codegen-request-body-name: boardRequest /v1/config/view: get: tags: - Configuration summary: List Views description: Get a list of views. operationId: list-view responses: '200': description: OK content: application/json: schema: type: object additionalProperties: type: object properties: {} '400': description: Bad request content: {} post: tags: - Configuration summary: Create View description: Use this method to create a view and attach alerts to the view. operationId: create-view requestBody: description: Request parameters content: application/json: schema: $ref: '#/components/schemas/configRequest' required: true responses: '200': description: OK content: application/json: schema: type: object additionalProperties: type: object properties: {} '400': description: Bad request content: {} deprecated: false x-codegen-request-body-name: configRequest /v1/config/view/{viewId}: get: tags: - Configuration summary: Get View description: Gets the configuration for a specific view. operationId: get-view parameters: - name: viewId in: path description: ID of a view. required: true schema: type: string responses: '200': description: OK content: application/json: schema: type: object additionalProperties: type: object properties: {} '400': description: Bad request content: {} put: tags: - Configuration summary: Update View description: >- Use this method to update a view. You can change the view configuration details; add or remove view specific alerts; or attach and detach preset alerts. operationId: update-view parameters: - name: viewId in: path description: ID of a view. required: true schema: type: string requestBody: description: Request parameters content: application/json: schema: $ref: '#/components/schemas/configRequest' required: true responses: '200': description: OK content: application/json: schema: type: object additionalProperties: type: object properties: {} '400': description: Bad request content: {} deprecated: false x-codegen-request-body-name: configRequest delete: tags: - Configuration summary: Delete View description: Use this method to delete a view and any attached alerts. operationId: delete-view parameters: - name: viewId in: path description: ID of a view. required: true schema: type: string responses: '200': description: OK content: application/json: schema: type: object additionalProperties: type: object properties: {} '400': description: Bad request content: {} deprecated: false x-codegen-request-body-name: configRequest /v1/config/presetalert: get: tags: - Configuration summary: List Preset Alerts description: Returns all Preset Alerts in the account. operationId: list-alerts responses: '200': description: OK content: application/json: schema: type: object additionalProperties: type: object properties: {} '400': description: Bad request content: {} post: tags: - Configuration summary: Create Preset Alert description: Use this method to create a preset alert. operationId: create-alert requestBody: description: Request parameters content: application/json: schema: $ref: '#/components/schemas/presetRequest' required: true responses: '200': description: OK content: application/json: schema: type: object additionalProperties: type: object properties: {} '400': description: Bad request content: {} /v1/config/presetalert/{presetId}: get: tags: - Configuration summary: Get Preset Alert description: Get a specific preset alert operationId: get-alert parameters: - name: presetId in: path description: ID of a preset alert. required: true schema: type: string responses: '200': description: OK content: application/json: schema: type: object additionalProperties: type: object properties: {} '400': description: Bad request content: {} put: tags: - Configuration summary: Update Alert description: Update a specific preset alert. operationId: update-preset parameters: - name: presetId in: path description: ID of a preset alert. required: true schema: type: string requestBody: description: Request parameters content: application/json: schema: $ref: '#/components/schemas/presetRequest' required: true responses: '200': description: OK content: application/json: schema: type: object additionalProperties: type: object properties: {} '400': description: Bad request content: {} delete: tags: - Configuration summary: Delete Alert description: Use this method to delete a preset alert. operationId: delete-alert parameters: - name: presetId in: path description: ID of a preset alert. required: true schema: type: string responses: '200': description: OK content: application/json: schema: type: object additionalProperties: type: object properties: {} '400': description: Bad request content: {} /v1/config/categories/{type}: get: tags: - Configuration summary: List Categories description: Returns all Categories of a type. operationId: list-categories parameters: - name: type in: path description: Category type. Must be [views, boards, screens] required: true schema: type: string responses: '200': description: OK content: application/json: schema: type: object additionalProperties: type: object properties: {} '400': description: Bad request content: {} post: tags: - Configuration summary: Create New Category description: Creates a new category of this type. operationId: create-category parameters: - name: type in: path description: Category type. Must be [views, boards, screens] required: true schema: type: string requestBody: description: Request parameters content: application/json: schema: $ref: '#/components/schemas/categoryName' required: true responses: '200': description: OK content: application/json: schema: type: object additionalProperties: type: object properties: {} '400': description: Bad request content: {} /v1/config/categories/{type}/{id}: get: tags: - Configuration summary: Get Category description: Get a specific category operationId: get-category parameters: - name: type in: path description: Category type. Must be [views, boards, screens] required: true schema: type: string - name: id in: path description: ID of category. required: true schema: type: string responses: '200': description: OK content: application/json: schema: type: object additionalProperties: type: object properties: {} '400': description: Bad request content: {} put: tags: - Configuration summary: Update Category description: Update a specific category. operationId: update-category parameters: - name: type in: path description: Category type. Must be [views, boards, screens] required: true schema: type: string - name: id in: path description: ID of category. required: true schema: type: string requestBody: description: Request parameters content: application/json: schema: $ref: '#/components/schemas/categoryName' required: true responses: '200': description: OK content: application/json: schema: type: object additionalProperties: type: object properties: {} '400': description: Bad request content: {} delete: tags: - Configuration summary: Delete Category description: Use this method to delete a category. operationId: delete-category parameters: - name: type in: path description: Category type. Must be [views, boards, screens] required: true schema: type: string - name: id in: path description: ID of category. required: true schema: type: string responses: '200': description: OK content: application/json: schema: type: object additionalProperties: type: object properties: {} '400': description: Bad request content: {} /v3/pipeline/classifier/analysis: get: summary: Fetch classifier information tags: - Data Classification & Profiling description: Retrieve information about a classification of a shared source parameters: - schema: type: string enum: - in_progress - complete in: query name: status required: false description: Fetch the classifier analyses having this status responses: '200': description: Default Response /v3/pipeline/classifier/analysis/{analysis_id}: patch: summary: Update analysis options tags: - Data Classification & Profiling description: Reset classification and profiling results requestBody: content: application/json: schema: type: object additionalProperties: false properties: status: type: string enum: - in_progress - complete description: The new status of the classifier analysis required: - status required: true parameters: - schema: type: boolean default: false in: query name: restart_analysis required: false description: >- Reset the current event count so that classification can begin when 'status' is set to "in_progress". - schema: type: boolean default: false in: query name: reset_profile required: false description: Reset the profiling data associated with this analysis. - schema: type: boolean default: false in: query name: create_new_classifier_analysis required: false description: >- Create a new classifier analysis record to replace the current record. Must be used in conjunction with a "complete" status - schema: type: string format: uuid in: path name: analysis_id required: true description: The id of the classifier analysis to update responses: '200': description: Default Response /v3/pipeline/classifier/{analysis_id}/fields: get: summary: Get Field Summary for an analysis tags: - Data Classification & Profiling description: Get Field Summary for an analysis parameters: - schema: type: string format: uuid in: path name: analysis_id required: true description: The classifier analysis to get fields of responses: '200': description: Default Response /v3/pipeline/classifier/{analysis_id}/fields/{field_name}: get: summary: Get Field Summary values for a field in an analysis tags: - Data Classification & Profiling description: Get Field Summary values for a field in an analysis parameters: - schema: type: string format: uuid in: path name: analysis_id required: true description: The classifier analysis to get values of - schema: type: string in: path name: field_name required: true description: The name of the field to get values of responses: '200': description: Default Response /v3/pipeline/metric/usage/profile: get: summary: Fetch usage metrics tags: - Data Classification & Profiling description: Query and fetch usage metrics from data profiling. parameters: - schema: type: string format: date-time in: query name: from required: true description: A date/time string that defines the beginning of the window - schema: type: string format: date-time in: query name: to required: true description: A date/time string that defines the end of the window - schema: type: string in: query name: id required: true description: The identifier of the classifier analysis - schema: type: string enum: - app - host - level - log_type in: query name: annotation required: true - schema: type: integer default: 20 minimum: 1 maximum: 500 in: query name: limit required: false responses: '200': description: Default Response /v3/pipeline/{pipeline_id}/transform/{pipeline_transform_id}/classifier/analysis: get: tags: - Data Classification & Profiling parameters: - schema: enum: - in_progress - complete in: query name: status required: false description: Fetch the classifier analyses having this status - schema: type: string format: uuid in: path name: pipeline_id required: true - schema: type: string format: uuid in: path name: pipeline_transform_id required: true responses: '200': description: Default Response /v3/pipeline/transform/classifier/analysis: get: tags: - Data Classification & Profiling parameters: - schema: enum: - in_progress - complete in: query name: status required: false description: Fetch the transform analysis having this status - schema: type: string format: uuid in: query name: pipeline_id required: false description: Fetch the processor analyses for the pipeline with the specified ID responses: '200': description: Default Response /v3/pipeline/account/local-deploy/config: get: summary: Get client config tags: - Edge - Local Deploy description: Gets client configuration for Edge pipelines parameters: - schema: type: string in: query name: deployment_group required: false description: The deployment group for which to construct configuration responses: '200': description: Default Response post: summary: Post client config tags: - Edge - Local Deploy description: Posts deployment configuration+liveness for Edge pipelines, returning pipeline config requestBody: content: application/json: schema: type: object properties: edge_id: type: string pattern: ^[a-zA-Z0-9-.]{1,253}$ replica: type: string pattern: ^[a-zA-Z0-9-.]{1,253}$ namespace: type: string pattern: ^[a-zA-Z0-9-.]{1,253}$ name: type: string pattern: ^[a-zA-Z0-9-.]{1,253}$ deployment_group: type: string pattern: ^[a-zA-Z0-9-_]{1,256}$ ports: type: array items: type: integer version: type: string chart_version: type: string vector_version: type: string uptime_sec: type: integer responses: '200': description: Default Response /v3/pipeline/account/local-deploy/log: post: summary: Fetch logs from Edge tags: - Edge - Local Deploy description: Receive logs from an Edge pipeline requestBody: content: application/json: schema: type: array items: type: object additionalProperties: false properties: pipeline_id: type: string format: uuid component_id: type: string format: uuid message: type: string minLength: 1 maxLength: 16384 level: type: string enum: - DEBUG - INFO - WARN - ERROR last_seen_at: type: string format: date-time required: - pipeline_id - component_id - message - level - last_seen_at minItems: 1 parameters: - schema: type: string minLength: 1 in: query name: edge_id required: false description: Logs for specific edge_id responses: '200': description: Default Response /v3/pipeline/account/local-deploy/tasks: get: summary: Poll tasks tags: - Edge - Local Deploy description: Poll and claim available local deployment tasks for a pipeline parameters: - schema: type: integer default: 10 minimum: 1 maximum: 100 in: query name: limit required: false description: The maximum number of tasks to return - schema: type: string minLength: 1 in: query name: edge_id required: false description: Specific edge_id instance tap responses: '200': description: Default Response /v3/pipeline/account/local-deploy/heartbeat/{edge_id}: delete: summary: Remove heartbeat tags: - Edge - Local Deploy description: Removes existing Edge heartbeats parameters: - schema: type: string in: path name: edge_id required: true responses: '200': description: Default Response /v3/pipeline/account/local-deploy/heartbeat: get: summary: List all heartbeats tags: - Edge - Local Deploy description: List all Edge check-ins for the account responses: '200': description: Default Response /v3/pipeline/account/local-deploy/metric/usage: post: summary: Get usage metrics tags: - Edge - Local Deploy description: Receive usage metrics from an Edge pipeline requestBody: content: application/json: schema: type: array items: type: object additionalProperties: false properties: event_ts: type: integer minimum: 0 pipeline_id: type: string format: uuid component_id: type: string minLength: 1 maxLength: 1024 processor: type: string minLength: 1 maxLength: 256 total_count: type: integer minimum: 0 total_size: type: integer minimum: 0 required: - event_ts - pipeline_id - component_id - processor - total_count - total_size minItems: 1 parameters: - schema: type: string minLength: 1 in: query name: edge_id required: false description: Metrics for specific edge_id responses: '200': description: Default Response /v3/pipeline/account/local-deploy/tasks/{task_id}/results: post: summary: Add task result tags: - Edge - Local Deploy description: Add results for a given task requestBody: content: application/json: schema: type: object parameters: - schema: type: string format: uuid in: path name: task_id required: true responses: '200': description: Default Response /v3/enterprise: get: operationId: getEnterpriseDetail summary: Get an Enterprise and its child Accounts tags: - Enterprise description: Get an Enterprise and its child Accounts security: - AccessToken: [] responses: '200': description: Response format containing the Enterprise and all child Accounts content: application/json: schema: additionalProperties: false type: object title: GetEnterpriseDetailResponse description: Response format containing the Enterprise and all child Accounts properties: meta: type: object description: Contains additional contextual information about the api response. properties: type: type: string description: A human readable identifier denoting the resource type minLength: 1 maxLength: 50 page: type: object properties: next: type: string nullable: true maxLength: 256 previous: type: string nullable: true maxLength: 256 pk: type: string description: Indicates the data property which holds the resource identifier nullable: true maxLength: 50 links: type: object description: An object containing uri templates to this, and related resources additionalProperties: true properties: self: type: object additionalProperties: true properties: list: nullable: true additionalProperties: false type: object properties: method: enum: - acl - bind - checkout - connect - copy - delete - get - head - link - lock - m-search - merge - mkactivity - mkcalendar - mkcol - move - notify - options - patch - post - propfind - proppatch - purge - put - rebind - report - search - source - subscribe - trace - unbind - unlink - unlock - unsubscribe type: string example: get uri: type: string format: uri-template maxLength: 250 description: A uri template for retrieving a list of the resource type example: https://api.mezmo.com/v3/enterprise/resource detail: nullable: true additionalProperties: false type: object properties: method: enum: - acl - bind - checkout - connect - copy - delete - get - head - link - lock - m-search - merge - mkactivity - mkcalendar - mkcol - move - notify - options - patch - post - propfind - proppatch - purge - put - rebind - report - search - source - subscribe - trace - unbind - unlink - unlock - unsubscribe type: string example: get uri: type: string format: uri-template maxLength: 250 description: |- A uri template for retrieving a single detail representation of the resource type example: https://api.mezmo.com/v3/enterprise/resource/{resource_id} update: nullable: true additionalProperties: false type: object properties: method: enum: - acl - bind - checkout - connect - copy - delete - get - head - link - lock - m-search - merge - mkactivity - mkcalendar - mkcol - move - notify - options - patch - post - propfind - proppatch - purge - put - rebind - report - search - source - subscribe - trace - unbind - unlink - unlock - unsubscribe type: string example: patch uri: type: string format: uri-template maxLength: 250 description: A uri template for partially updating an individual resource example: https://api.mezmo.com/v3/enterprise/resource/{resource_id} replace: nullable: true additionalProperties: false type: object properties: method: enum: - acl - bind - checkout - connect - copy - delete - get - head - link - lock - m-search - merge - mkactivity - mkcalendar - mkcol - move - notify - options - patch - post - propfind - proppatch - purge - put - rebind - report - search - source - subscribe - trace - unbind - unlink - unlock - unsubscribe type: string example: put uri: type: string format: uri-template maxLength: 250 description: >- A uri template for fully replacing or updating an individual resource example: https://api.mezmo.com/v3/enterprise/resource/{resource_id} create: nullable: true additionalProperties: false type: object properties: method: enum: - acl - bind - checkout - connect - copy - delete - get - head - link - lock - m-search - merge - mkactivity - mkcalendar - mkcol - move - notify - options - patch - post - propfind - proppatch - purge - put - rebind - report - search - source - subscribe - trace - unbind - unlink - unlock - unsubscribe type: string example: post uri: type: string format: uri-template maxLength: 250 description: >- A uri template for creating a new instance of a specific resource example: https://api.mezmo.com/v3/enterprise/resource related: type: object additionalProperties: true description: |- Additional URI templates for interacting with resources which are tightly related required: - self required: - type - page - pk - links data: $ref: '#/components/schemas/enterprise' required: - meta - data '400': description: |- A normalized representation of an error that is returned from an http request in response to failed input validation content: application/json: schema: $ref: '#/components/schemas/validation-error' '401': description: A normalized representation of an error that is returned from an http request content: application/json: schema: $ref: '#/components/schemas/http-error' '429': description: A normalized representation of an error that is returned from an http request content: application/json: schema: $ref: '#/components/schemas/http-error' '500': description: A normalized representation of an error that is returned from an http request content: application/json: schema: $ref: '#/components/schemas/http-error' 5XX: description: A normalized representation of an error that is returned from an http request content: application/json: schema: $ref: '#/components/schemas/http-error' 4XX: description: A normalized representation of an error that is returned from an http request content: application/json: schema: $ref: '#/components/schemas/http-error' /v3/enterprise/account: patch: operationId: patchEnterpriseAccountList summary: Perform bulk operations on Enterprise Accounts tags: - Enterprise description: Perform bulk operations (CREATE, UPDATE, DELETE) on Enterprise Accounts requestBody: content: application/x-bulk+json: schema: type: object title: Account Bulk Operations Request description: Request body for bulk operations on accounts properties: operations: type: array minItems: 1 maxItems: 100 description: List of bulk operations items: type: object title: Bulk Operation description: Represents a single bulk operation on an entity allOf: - if: properties: action: enum: - UPDATE - DELETE type: string then: required: - entity_id - if: properties: action: enum: - CREATE - UPDATE type: string then: required: - entity properties: operation_id: type: string title: Client generated id to associate operation with result description: |- Used for navigation only, not persisted in database. If not provided, the array index (as string) will be used in the response example: 4b1a1504fa47866dafe8b5d4 action: type: string enum: - CREATE - UPDATE - DELETE description: Operation type (enum, not extensible) entity_id: type: string title: Entity unique identifier description: |- The identifier of the entity for UPDATE and DELETE operations. MUST be null or omitted for CREATE operations example: 471ef1c4154b987a94509213 entity: $ref: '#/components/schemas/account-create-request' required: - action required: - operations required: true description: Request body for bulk operations on accounts security: - AccessToken: [] responses: '400': description: |- A normalized representation of an error that is returned from an http request in response to failed input validation content: application/json: schema: $ref: '#/components/schemas/validation-error' '401': description: A normalized representation of an error that is returned from an http request content: application/json: schema: $ref: '#/components/schemas/http-error' '422': description: |- Response format containing the results of bulk operations, in addition to the standard metadata object. content: application/json: schema: additionalProperties: false type: object title: PatchEnterpriseAccountListResponse description: |- Response format containing the results of bulk operations, in addition to the standard metadata object. properties: meta: type: object description: Contains additional contextual information about the api response. properties: type: type: string description: A human readable identifier denoting the resource type minLength: 1 maxLength: 50 page: type: object properties: next: type: string nullable: true maxLength: 256 previous: type: string nullable: true maxLength: 256 pk: type: string description: Indicates the data property which holds the resource identifier nullable: true maxLength: 50 links: type: object description: An object containing uri templates to this, and related resources additionalProperties: true properties: self: type: object additionalProperties: true properties: list: nullable: true additionalProperties: false type: object properties: method: enum: - acl - bind - checkout - connect - copy - delete - get - head - link - lock - m-search - merge - mkactivity - mkcalendar - mkcol - move - notify - options - patch - post - propfind - proppatch - purge - put - rebind - report - search - source - subscribe - trace - unbind - unlink - unlock - unsubscribe type: string example: get uri: type: string format: uri-template maxLength: 250 description: A uri template for retrieving a list of the resource type example: https://api.mezmo.com/v3/enterprise/resource detail: nullable: true additionalProperties: false type: object properties: method: enum: - acl - bind - checkout - connect - copy - delete - get - head - link - lock - m-search - merge - mkactivity - mkcalendar - mkcol - move - notify - options - patch - post - propfind - proppatch - purge - put - rebind - report - search - source - subscribe - trace - unbind - unlink - unlock - unsubscribe type: string example: get uri: type: string format: uri-template maxLength: 250 description: |- A uri template for retrieving a single detail representation of the resource type example: https://api.mezmo.com/v3/enterprise/resource/{resource_id} update: nullable: true additionalProperties: false type: object properties: method: enum: - acl - bind - checkout - connect - copy - delete - get - head - link - lock - m-search - merge - mkactivity - mkcalendar - mkcol - move - notify - options - patch - post - propfind - proppatch - purge - put - rebind - report - search - source - subscribe - trace - unbind - unlink - unlock - unsubscribe type: string example: patch uri: type: string format: uri-template maxLength: 250 description: A uri template for partially updating an individual resource example: https://api.mezmo.com/v3/enterprise/resource/{resource_id} replace: nullable: true additionalProperties: false type: object properties: method: enum: - acl - bind - checkout - connect - copy - delete - get - head - link - lock - m-search - merge - mkactivity - mkcalendar - mkcol - move - notify - options - patch - post - propfind - proppatch - purge - put - rebind - report - search - source - subscribe - trace - unbind - unlink - unlock - unsubscribe type: string example: put uri: type: string format: uri-template maxLength: 250 description: >- A uri template for fully replacing or updating an individual resource example: https://api.mezmo.com/v3/enterprise/resource/{resource_id} create: nullable: true additionalProperties: false type: object properties: method: enum: - acl - bind - checkout - connect - copy - delete - get - head - link - lock - m-search - merge - mkactivity - mkcalendar - mkcol - move - notify - options - patch - post - propfind - proppatch - purge - put - rebind - report - search - source - subscribe - trace - unbind - unlink - unlock - unsubscribe type: string example: post uri: type: string format: uri-template maxLength: 250 description: >- A uri template for creating a new instance of a specific resource example: https://api.mezmo.com/v3/enterprise/resource related: type: object additionalProperties: true description: |- Additional URI templates for interacting with resources which are tightly related required: - self required: - type - page - pk - links data: type: object title: Account Bulk Operations Response description: Response body for bulk operations on accounts properties: status: type: string enum: - SUCCEEDED - FAILED - PARTIAL title: Overall status of the bulk operation operations: type: array title: List of bulk operations description: >- Results for each operation from the request. Should be in the same order as the request operations items: type: object title: Bulk Operation Result description: Represents the result of a single bulk operation on an entity properties: operation_id: type: string title: Matching operation_id or index from request description: |- Matches the operation_id from the request if provided, otherwise the index of the operation as a string example: 408efce8aa57bd0d8a861298 action: type: string enum: - CREATE - UPDATE - DELETE description: Operation type (enum, not extensible) entity_id: type: string title: Entity identifier description: Entity identifier for UPDATE / DELETE operations example: cb7b7c1f5100d2d5d31ad4f1 entity: $ref: '#/components/schemas/account' result: type: object title: Operation result description: Result details for a single operation properties: status: type: string enum: - SUCCEEDED - FAILED title: Operation result status detail: type: string title: Human readable message description: Detailed message about the operation result example: Account created successfully context: type: array title: Detailed error information description: |- List of objects providing additional context for errors or failures items: type: object title: Detailed error information properties: message: type: string title: Specific error details example: Account not found code: type: string title: Machine readable error code example: ECONTRACT required: - message - code required: - status - detail - context required: - operation_id - action required: - status - operations required: - meta - data '429': description: A normalized representation of an error that is returned from an http request content: application/json: schema: $ref: '#/components/schemas/http-error' '500': description: A normalized representation of an error that is returned from an http request content: application/json: schema: $ref: '#/components/schemas/http-error' 5XX: description: A normalized representation of an error that is returned from an http request content: application/json: schema: $ref: '#/components/schemas/http-error' 4XX: description: A normalized representation of an error that is returned from an http request content: application/json: schema: $ref: '#/components/schemas/http-error' 2XX: description: |- Response format containing the results of bulk operations, in addition to the standard metadata object. content: application/json: schema: additionalProperties: false type: object title: PatchEnterpriseAccountListResponse description: |- Response format containing the results of bulk operations, in addition to the standard metadata object. properties: meta: type: object description: Contains additional contextual information about the api response. properties: type: type: string description: A human readable identifier denoting the resource type minLength: 1 maxLength: 50 page: type: object properties: next: type: string nullable: true maxLength: 256 previous: type: string nullable: true maxLength: 256 pk: type: string description: Indicates the data property which holds the resource identifier nullable: true maxLength: 50 links: type: object description: An object containing uri templates to this, and related resources additionalProperties: true properties: self: type: object additionalProperties: true properties: list: nullable: true additionalProperties: false type: object properties: method: enum: - acl - bind - checkout - connect - copy - delete - get - head - link - lock - m-search - merge - mkactivity - mkcalendar - mkcol - move - notify - options - patch - post - propfind - proppatch - purge - put - rebind - report - search - source - subscribe - trace - unbind - unlink - unlock - unsubscribe type: string example: get uri: type: string format: uri-template maxLength: 250 description: A uri template for retrieving a list of the resource type example: https://api.mezmo.com/v3/enterprise/resource detail: nullable: true additionalProperties: false type: object properties: method: enum: - acl - bind - checkout - connect - copy - delete - get - head - link - lock - m-search - merge - mkactivity - mkcalendar - mkcol - move - notify - options - patch - post - propfind - proppatch - purge - put - rebind - report - search - source - subscribe - trace - unbind - unlink - unlock - unsubscribe type: string example: get uri: type: string format: uri-template maxLength: 250 description: |- A uri template for retrieving a single detail representation of the resource type example: https://api.mezmo.com/v3/enterprise/resource/{resource_id} update: nullable: true additionalProperties: false type: object properties: method: enum: - acl - bind - checkout - connect - copy - delete - get - head - link - lock - m-search - merge - mkactivity - mkcalendar - mkcol - move - notify - options - patch - post - propfind - proppatch - purge - put - rebind - report - search - source - subscribe - trace - unbind - unlink - unlock - unsubscribe type: string example: patch uri: type: string format: uri-template maxLength: 250 description: A uri template for partially updating an individual resource example: https://api.mezmo.com/v3/enterprise/resource/{resource_id} replace: nullable: true additionalProperties: false type: object properties: method: enum: - acl - bind - checkout - connect - copy - delete - get - head - link - lock - m-search - merge - mkactivity - mkcalendar - mkcol - move - notify - options - patch - post - propfind - proppatch - purge - put - rebind - report - search - source - subscribe - trace - unbind - unlink - unlock - unsubscribe type: string example: put uri: type: string format: uri-template maxLength: 250 description: >- A uri template for fully replacing or updating an individual resource example: https://api.mezmo.com/v3/enterprise/resource/{resource_id} create: nullable: true additionalProperties: false type: object properties: method: enum: - acl - bind - checkout - connect - copy - delete - get - head - link - lock - m-search - merge - mkactivity - mkcalendar - mkcol - move - notify - options - patch - post - propfind - proppatch - purge - put - rebind - report - search - source - subscribe - trace - unbind - unlink - unlock - unsubscribe type: string example: post uri: type: string format: uri-template maxLength: 250 description: >- A uri template for creating a new instance of a specific resource example: https://api.mezmo.com/v3/enterprise/resource related: type: object additionalProperties: true description: |- Additional URI templates for interacting with resources which are tightly related required: - self required: - type - page - pk - links data: type: object title: Account Bulk Operations Response description: Response body for bulk operations on accounts properties: status: type: string enum: - SUCCEEDED - FAILED - PARTIAL title: Overall status of the bulk operation operations: type: array title: List of bulk operations description: >- Results for each operation from the request. Should be in the same order as the request operations items: type: object title: Bulk Operation Result description: Represents the result of a single bulk operation on an entity properties: operation_id: type: string title: Matching operation_id or index from request description: |- Matches the operation_id from the request if provided, otherwise the index of the operation as a string example: 408efce8aa57bd0d8a861298 action: type: string enum: - CREATE - UPDATE - DELETE description: Operation type (enum, not extensible) entity_id: type: string title: Entity identifier description: Entity identifier for UPDATE / DELETE operations example: cb7b7c1f5100d2d5d31ad4f1 entity: $ref: '#/components/schemas/account' result: type: object title: Operation result description: Result details for a single operation properties: status: type: string enum: - SUCCEEDED - FAILED title: Operation result status detail: type: string title: Human readable message description: Detailed message about the operation result example: Account created successfully context: type: array title: Detailed error information description: |- List of objects providing additional context for errors or failures items: type: object title: Detailed error information properties: message: type: string title: Specific error details example: Account not found code: type: string title: Machine readable error code example: ECONTRACT required: - message - code required: - status - detail - context required: - operation_id - action required: - status - operations required: - meta - data post: operationId: postEnterpriseAccountList summary: Create a new Enterprise Child Organization (Account) tags: - Enterprise description: Create a new Enterprise Child Organization (Account) requestBody: content: application/json: schema: type: object title: Account description: |- Represents a platform level account registration, authentication and authorization. properties: company: type: string maxLength: 250 description: Name of the company which owns the account example: Snippet Factory owneremail: type: string maxLength: 320 format: email description: Email address of account owner example: admin@snippetfactory.com external_id: type: string maxLength: 255 default: null nullable: true description: And identifier that can be used to reference and id from a different system example: d615396c40 plan: type: object additionalProperties: true description: Billing plan and associated feature information pertaining properties: type: type: string minLength: 4 maxLength: 50 description: The new billing plan type to assign the target account once attached example: enterprise retention: type: integer minimum: 0 maximum: 90 format: int32 description: Data retention policy information pertaining to the account enable_syslog: type: boolean default: false title: Enable Syslog Ingestion description: Enables syslog ingestion for the new account via custom ports. include_admins: type: boolean default: false title: Auto join all enterprise admins description: If true, all enterprise admins will be added as members of the child account description: |- Represents a platform level account registration, authentication and authorization. security: - AccessToken: [] responses: '201': description: |- Response format containing the newly created account, in addition to the standard metadata object. content: application/json: schema: additionalProperties: false type: object title: PostEnterpriseAccountListResponse description: |- Response format containing the newly created account, in addition to the standard metadata object. properties: meta: type: object description: Contains additional contextual information about the api response. properties: type: type: string description: A human readable identifier denoting the resource type minLength: 1 maxLength: 50 page: type: object properties: next: type: string nullable: true maxLength: 256 previous: type: string nullable: true maxLength: 256 pk: type: string description: Indicates the data property which holds the resource identifier nullable: true maxLength: 50 links: type: object description: An object containing uri templates to this, and related resources additionalProperties: true properties: self: type: object additionalProperties: true properties: list: nullable: true additionalProperties: false type: object properties: method: enum: - acl - bind - checkout - connect - copy - delete - get - head - link - lock - m-search - merge - mkactivity - mkcalendar - mkcol - move - notify - options - patch - post - propfind - proppatch - purge - put - rebind - report - search - source - subscribe - trace - unbind - unlink - unlock - unsubscribe type: string example: get uri: type: string format: uri-template maxLength: 250 description: A uri template for retrieving a list of the resource type example: https://api.mezmo.com/v3/enterprise/resource detail: nullable: true additionalProperties: false type: object properties: method: enum: - acl - bind - checkout - connect - copy - delete - get - head - link - lock - m-search - merge - mkactivity - mkcalendar - mkcol - move - notify - options - patch - post - propfind - proppatch - purge - put - rebind - report - search - source - subscribe - trace - unbind - unlink - unlock - unsubscribe type: string example: get uri: type: string format: uri-template maxLength: 250 description: |- A uri template for retrieving a single detail representation of the resource type example: https://api.mezmo.com/v3/enterprise/resource/{resource_id} update: nullable: true additionalProperties: false type: object properties: method: enum: - acl - bind - checkout - connect - copy - delete - get - head - link - lock - m-search - merge - mkactivity - mkcalendar - mkcol - move - notify - options - patch - post - propfind - proppatch - purge - put - rebind - report - search - source - subscribe - trace - unbind - unlink - unlock - unsubscribe type: string example: patch uri: type: string format: uri-template maxLength: 250 description: A uri template for partially updating an individual resource example: https://api.mezmo.com/v3/enterprise/resource/{resource_id} replace: nullable: true additionalProperties: false type: object properties: method: enum: - acl - bind - checkout - connect - copy - delete - get - head - link - lock - m-search - merge - mkactivity - mkcalendar - mkcol - move - notify - options - patch - post - propfind - proppatch - purge - put - rebind - report - search - source - subscribe - trace - unbind - unlink - unlock - unsubscribe type: string example: put uri: type: string format: uri-template maxLength: 250 description: >- A uri template for fully replacing or updating an individual resource example: https://api.mezmo.com/v3/enterprise/resource/{resource_id} create: nullable: true additionalProperties: false type: object properties: method: enum: - acl - bind - checkout - connect - copy - delete - get - head - link - lock - m-search - merge - mkactivity - mkcalendar - mkcol - move - notify - options - patch - post - propfind - proppatch - purge - put - rebind - report - search - source - subscribe - trace - unbind - unlink - unlock - unsubscribe type: string example: post uri: type: string format: uri-template maxLength: 250 description: >- A uri template for creating a new instance of a specific resource example: https://api.mezmo.com/v3/enterprise/resource related: type: object additionalProperties: true description: |- Additional URI templates for interacting with resources which are tightly related required: - self required: - type - page - pk - links data: $ref: '#/components/schemas/account' required: - meta - data '400': description: |- A normalized representation of an error that is returned from an http request in response to failed input validation content: application/json: schema: $ref: '#/components/schemas/validation-error' '401': description: A normalized representation of an error that is returned from an http request content: application/json: schema: $ref: '#/components/schemas/http-error' '429': description: A normalized representation of an error that is returned from an http request content: application/json: schema: $ref: '#/components/schemas/http-error' '500': description: A normalized representation of an error that is returned from an http request content: application/json: schema: $ref: '#/components/schemas/http-error' 5XX: description: A normalized representation of an error that is returned from an http request content: application/json: schema: $ref: '#/components/schemas/http-error' 4XX: description: A normalized representation of an error that is returned from an http request content: application/json: schema: $ref: '#/components/schemas/http-error' /v3/enterprise/account/{account_id}: get: operationId: getEnterpriseAccountDetail summary: Retrieve a specific child account under an enterprise organization tags: - Enterprise description: Retrieve a specific child account under an enterprise organization parameters: - schema: type: string minLength: 1 maxLength: 24 in: path name: account_id required: true description: The unique identifier of the account to retrieve security: - AccessToken: [] responses: '200': description: Response format containing the Enterprise and all child Accounts content: application/json: schema: additionalProperties: false type: object title: GetEnterpriseAccountDetailResponse description: Response format containing the Enterprise and all child Accounts properties: meta: type: object description: Contains additional contextual information about the api response. properties: type: type: string description: A human readable identifier denoting the resource type minLength: 1 maxLength: 50 page: type: object properties: next: type: string nullable: true maxLength: 256 previous: type: string nullable: true maxLength: 256 pk: type: string description: Indicates the data property which holds the resource identifier nullable: true maxLength: 50 links: type: object description: An object containing uri templates to this, and related resources additionalProperties: true properties: self: type: object additionalProperties: true properties: list: nullable: true additionalProperties: false type: object properties: method: enum: - acl - bind - checkout - connect - copy - delete - get - head - link - lock - m-search - merge - mkactivity - mkcalendar - mkcol - move - notify - options - patch - post - propfind - proppatch - purge - put - rebind - report - search - source - subscribe - trace - unbind - unlink - unlock - unsubscribe type: string example: get uri: type: string format: uri-template maxLength: 250 description: A uri template for retrieving a list of the resource type example: https://api.mezmo.com/v3/enterprise/resource detail: nullable: true additionalProperties: false type: object properties: method: enum: - acl - bind - checkout - connect - copy - delete - get - head - link - lock - m-search - merge - mkactivity - mkcalendar - mkcol - move - notify - options - patch - post - propfind - proppatch - purge - put - rebind - report - search - source - subscribe - trace - unbind - unlink - unlock - unsubscribe type: string example: get uri: type: string format: uri-template maxLength: 250 description: |- A uri template for retrieving a single detail representation of the resource type example: https://api.mezmo.com/v3/enterprise/resource/{resource_id} update: nullable: true additionalProperties: false type: object properties: method: enum: - acl - bind - checkout - connect - copy - delete - get - head - link - lock - m-search - merge - mkactivity - mkcalendar - mkcol - move - notify - options - patch - post - propfind - proppatch - purge - put - rebind - report - search - source - subscribe - trace - unbind - unlink - unlock - unsubscribe type: string example: patch uri: type: string format: uri-template maxLength: 250 description: A uri template for partially updating an individual resource example: https://api.mezmo.com/v3/enterprise/resource/{resource_id} replace: nullable: true additionalProperties: false type: object properties: method: enum: - acl - bind - checkout - connect - copy - delete - get - head - link - lock - m-search - merge - mkactivity - mkcalendar - mkcol - move - notify - options - patch - post - propfind - proppatch - purge - put - rebind - report - search - source - subscribe - trace - unbind - unlink - unlock - unsubscribe type: string example: put uri: type: string format: uri-template maxLength: 250 description: >- A uri template for fully replacing or updating an individual resource example: https://api.mezmo.com/v3/enterprise/resource/{resource_id} create: nullable: true additionalProperties: false type: object properties: method: enum: - acl - bind - checkout - connect - copy - delete - get - head - link - lock - m-search - merge - mkactivity - mkcalendar - mkcol - move - notify - options - patch - post - propfind - proppatch - purge - put - rebind - report - search - source - subscribe - trace - unbind - unlink - unlock - unsubscribe type: string example: post uri: type: string format: uri-template maxLength: 250 description: >- A uri template for creating a new instance of a specific resource example: https://api.mezmo.com/v3/enterprise/resource related: type: object additionalProperties: true description: |- Additional URI templates for interacting with resources which are tightly related required: - self required: - type - page - pk - links data: type: object title: Account description: |- Represents a platform level account registration, authentication and authorization. properties: account_id: type: string maxLength: 24 description: A value that uniquely identifies the account. example: 8aca3b99b4ee99db2c0fb6b7 enterprise_id: type: string maxLength: 24 description: >- A value that uniquely identifies the enterprise to which the account belongs. example: c0c41da6c8 account: type: string maxLength: 20 description: A shorthand identifier for the associated account example: b35eda6673 company: type: string maxLength: 250 description: Name of the company which owns the account example: Snippet Factory deactivated: type: string maxLength: 35 format: date-time nullable: true description: Indicates if the account has been deactivated or not owneremail: type: string maxLength: 320 format: email description: Email address of account owner example: admin@snippetfactory.com external_id: type: string maxLength: 255 default: null nullable: true description: >- And identifier that can be used to reference and id from a different system example: d615396c40 plan: type: object additionalProperties: true description: Billing plan and associated feature information pertaining properties: type: type: string minLength: 4 maxLength: 50 description: The new billing plan type to assign the target account once attached example: enterprise quota: type: object additionalProperties: true description: Quota information pertaining to the account properties: retention: type: integer minimum: 0 maximum: 90 format: int32 description: Data retention policy information pertaining to the account example: 14 retention: type: integer minimum: 0 maximum: 90 format: int32 description: Data retention policy information pertaining to the account status: type: string enum: - active - deactivated - pending description: Indication the account's current activation state created: type: string maxLength: 35 format: date-time description: Indicates when the account was created meta: type: object description: Metadata about the account additionalProperties: true owner: $ref: '#/components/schemas/account-owner' required: - account_id - enterprise_id required: - meta - data '400': description: |- A normalized representation of an error that is returned from an http request in response to failed input validation content: application/json: schema: $ref: '#/components/schemas/validation-error' '401': description: A normalized representation of an error that is returned from an http request content: application/json: schema: $ref: '#/components/schemas/http-error' '429': description: A normalized representation of an error that is returned from an http request content: application/json: schema: $ref: '#/components/schemas/http-error' '500': description: A normalized representation of an error that is returned from an http request content: application/json: schema: $ref: '#/components/schemas/http-error' 5XX: description: A normalized representation of an error that is returned from an http request content: application/json: schema: $ref: '#/components/schemas/http-error' 4XX: description: A normalized representation of an error that is returned from an http request content: application/json: schema: $ref: '#/components/schemas/http-error' /v3/enterprise/account/{account_id}/attach: patch: operationId: patchAttachEnterpriseAccount summary: Link Child Account tags: - Enterprise description: |- Moves an existing individual customer account under the owner ship of an existing enterprise with the option to change its direct owner requestBody: content: application/json: schema: type: object description: Object to serialize as the body of the http request properties: plan: type: string minLength: 4 maxLength: 50 description: The new billing plan type to assign the target account once attached default: enterprise example: enterprise owner: type: string description: |- Email address of the user acount who will be the new owner of the target account once it is successfully attached format: email example: admin@snippetfactory.com description: Object to serialize as the body of the http request parameters: - schema: type: string minLength: 1 maxLength: 24 examples: 5fbca354be: value: 5fbca354be 6c486cb57b5d2bf2776d999d: value: 6c486cb57b5d2bf2776d999d in: path name: account_id required: true description: The unique identifier of the account to attach security: - AccessToken: [] responses: '201': description: Response format containing the information of the account that was attached. content: application/json: schema: additionalProperties: false type: object title: PatchEnterpriseAccountDetailResponse description: Response format containing the information of the account that was attached. properties: meta: type: object description: Contains additional contextual information about the api response. properties: type: type: string description: A human readable identifier denoting the resource type minLength: 1 maxLength: 50 page: type: object properties: next: type: string nullable: true maxLength: 256 previous: type: string nullable: true maxLength: 256 pk: type: string description: Indicates the data property which holds the resource identifier nullable: true maxLength: 50 links: type: object description: An object containing uri templates to this, and related resources additionalProperties: true properties: self: type: object additionalProperties: true properties: list: nullable: true additionalProperties: false type: object properties: method: enum: - acl - bind - checkout - connect - copy - delete - get - head - link - lock - m-search - merge - mkactivity - mkcalendar - mkcol - move - notify - options - patch - post - propfind - proppatch - purge - put - rebind - report - search - source - subscribe - trace - unbind - unlink - unlock - unsubscribe type: string example: get uri: type: string format: uri-template maxLength: 250 description: A uri template for retrieving a list of the resource type example: https://api.mezmo.com/v3/enterprise/resource detail: nullable: true additionalProperties: false type: object properties: method: enum: - acl - bind - checkout - connect - copy - delete - get - head - link - lock - m-search - merge - mkactivity - mkcalendar - mkcol - move - notify - options - patch - post - propfind - proppatch - purge - put - rebind - report - search - source - subscribe - trace - unbind - unlink - unlock - unsubscribe type: string example: get uri: type: string format: uri-template maxLength: 250 description: |- A uri template for retrieving a single detail representation of the resource type example: https://api.mezmo.com/v3/enterprise/resource/{resource_id} update: nullable: true additionalProperties: false type: object properties: method: enum: - acl - bind - checkout - connect - copy - delete - get - head - link - lock - m-search - merge - mkactivity - mkcalendar - mkcol - move - notify - options - patch - post - propfind - proppatch - purge - put - rebind - report - search - source - subscribe - trace - unbind - unlink - unlock - unsubscribe type: string example: patch uri: type: string format: uri-template maxLength: 250 description: A uri template for partially updating an individual resource example: https://api.mezmo.com/v3/enterprise/resource/{resource_id} replace: nullable: true additionalProperties: false type: object properties: method: enum: - acl - bind - checkout - connect - copy - delete - get - head - link - lock - m-search - merge - mkactivity - mkcalendar - mkcol - move - notify - options - patch - post - propfind - proppatch - purge - put - rebind - report - search - source - subscribe - trace - unbind - unlink - unlock - unsubscribe type: string example: put uri: type: string format: uri-template maxLength: 250 description: >- A uri template for fully replacing or updating an individual resource example: https://api.mezmo.com/v3/enterprise/resource/{resource_id} create: nullable: true additionalProperties: false type: object properties: method: enum: - acl - bind - checkout - connect - copy - delete - get - head - link - lock - m-search - merge - mkactivity - mkcalendar - mkcol - move - notify - options - patch - post - propfind - proppatch - purge - put - rebind - report - search - source - subscribe - trace - unbind - unlink - unlock - unsubscribe type: string example: post uri: type: string format: uri-template maxLength: 250 description: >- A uri template for creating a new instance of a specific resource example: https://api.mezmo.com/v3/enterprise/resource related: type: object additionalProperties: true description: |- Additional URI templates for interacting with resources which are tightly related required: - self required: - type - page - pk - links data: type: object title: Account description: |- Represents a platform level account registration, authentication and authorization. properties: account_id: type: string maxLength: 24 description: A value that uniquely identifies the account. example: 8aca3b99b4ee99db2c0fb6b7 enterprise_id: type: string maxLength: 24 description: >- A value that uniquely identifies the enterprise to which the account belongs. example: c0c41da6c8 account: type: string maxLength: 20 description: A shorthand identifier for the associated account example: b35eda6673 company: type: string maxLength: 250 description: Name of the company which owns the account example: Snippet Factory deactivated: type: string maxLength: 35 format: date-time nullable: true description: Indicates if the account has been deactivated or not owneremail: type: string maxLength: 320 format: email description: Email address of account owner example: admin@snippetfactory.com external_id: type: string maxLength: 255 default: null nullable: true description: >- And identifier that can be used to reference and id from a different system example: d615396c40 plan: type: object additionalProperties: true description: Billing plan and associated feature information pertaining properties: type: type: string minLength: 4 maxLength: 50 description: The new billing plan type to assign the target account once attached example: enterprise quota: type: object additionalProperties: true description: Quota information pertaining to the account properties: retention: type: integer minimum: 0 maximum: 90 format: int32 description: Data retention policy information pertaining to the account example: 14 retention: type: integer minimum: 0 maximum: 90 format: int32 description: Data retention policy information pertaining to the account status: type: string enum: - active - deactivated - pending description: Indication the account's current activation state created: type: string maxLength: 35 format: date-time description: Indicates when the account was created meta: type: object description: Metadata about the account additionalProperties: true owner: $ref: '#/components/schemas/account-owner' required: - account_id - enterprise_id required: - meta - data '400': description: |- A normalized representation of an error that is returned from an http request in response to failed input validation content: application/json: schema: $ref: '#/components/schemas/validation-error' '401': description: A normalized representation of an error that is returned from an http request content: application/json: schema: $ref: '#/components/schemas/http-error' '429': description: A normalized representation of an error that is returned from an http request content: application/json: schema: $ref: '#/components/schemas/http-error' '500': description: A normalized representation of an error that is returned from an http request content: application/json: schema: $ref: '#/components/schemas/http-error' 5XX: description: A normalized representation of an error that is returned from an http request content: application/json: schema: $ref: '#/components/schemas/http-error' 4XX: description: A normalized representation of an error that is returned from an http request content: application/json: schema: $ref: '#/components/schemas/http-error' /v3/enterprise/account/{account_id}/detach: patch: operationId: patchDetachEnterpriseAccount summary: Unlink Child Account tags: - Enterprise description: |- Removes a specific account from under an enterprise making it an independant account. The account will be downgraded to a trial account once the process is complete. parameters: - schema: type: boolean default: false in: query name: deactivate required: false description: If true, the child org will be deactivated after detachment. - schema: type: string minLength: 1 maxLength: 24 examples: 74de7c06e4: value: 74de7c06e4 396e69fec77bfb7f02a91bbe: value: 396e69fec77bfb7f02a91bbe in: path name: account_id required: true description: The unique identifier of the account to detach security: - AccessToken: [] responses: '200': description: Response format containing the information of the account that was detached. content: application/json: schema: additionalProperties: false type: object title: PatchEnterpriseAccountDetailResponse description: Response format containing the information of the account that was detached. properties: meta: type: object description: Contains additional contextual information about the api response. properties: type: type: string description: A human readable identifier denoting the resource type minLength: 1 maxLength: 50 page: type: object properties: next: type: string nullable: true maxLength: 256 previous: type: string nullable: true maxLength: 256 pk: type: string description: Indicates the data property which holds the resource identifier nullable: true maxLength: 50 links: type: object description: An object containing uri templates to this, and related resources additionalProperties: true properties: self: type: object additionalProperties: true properties: list: nullable: true additionalProperties: false type: object properties: method: enum: - acl - bind - checkout - connect - copy - delete - get - head - link - lock - m-search - merge - mkactivity - mkcalendar - mkcol - move - notify - options - patch - post - propfind - proppatch - purge - put - rebind - report - search - source - subscribe - trace - unbind - unlink - unlock - unsubscribe type: string example: get uri: type: string format: uri-template maxLength: 250 description: A uri template for retrieving a list of the resource type example: https://api.mezmo.com/v3/enterprise/resource detail: nullable: true additionalProperties: false type: object properties: method: enum: - acl - bind - checkout - connect - copy - delete - get - head - link - lock - m-search - merge - mkactivity - mkcalendar - mkcol - move - notify - options - patch - post - propfind - proppatch - purge - put - rebind - report - search - source - subscribe - trace - unbind - unlink - unlock - unsubscribe type: string example: get uri: type: string format: uri-template maxLength: 250 description: |- A uri template for retrieving a single detail representation of the resource type example: https://api.mezmo.com/v3/enterprise/resource/{resource_id} update: nullable: true additionalProperties: false type: object properties: method: enum: - acl - bind - checkout - connect - copy - delete - get - head - link - lock - m-search - merge - mkactivity - mkcalendar - mkcol - move - notify - options - patch - post - propfind - proppatch - purge - put - rebind - report - search - source - subscribe - trace - unbind - unlink - unlock - unsubscribe type: string example: patch uri: type: string format: uri-template maxLength: 250 description: A uri template for partially updating an individual resource example: https://api.mezmo.com/v3/enterprise/resource/{resource_id} replace: nullable: true additionalProperties: false type: object properties: method: enum: - acl - bind - checkout - connect - copy - delete - get - head - link - lock - m-search - merge - mkactivity - mkcalendar - mkcol - move - notify - options - patch - post - propfind - proppatch - purge - put - rebind - report - search - source - subscribe - trace - unbind - unlink - unlock - unsubscribe type: string example: put uri: type: string format: uri-template maxLength: 250 description: >- A uri template for fully replacing or updating an individual resource example: https://api.mezmo.com/v3/enterprise/resource/{resource_id} create: nullable: true additionalProperties: false type: object properties: method: enum: - acl - bind - checkout - connect - copy - delete - get - head - link - lock - m-search - merge - mkactivity - mkcalendar - mkcol - move - notify - options - patch - post - propfind - proppatch - purge - put - rebind - report - search - source - subscribe - trace - unbind - unlink - unlock - unsubscribe type: string example: post uri: type: string format: uri-template maxLength: 250 description: >- A uri template for creating a new instance of a specific resource example: https://api.mezmo.com/v3/enterprise/resource related: type: object additionalProperties: true description: |- Additional URI templates for interacting with resources which are tightly related required: - self required: - type - page - pk - links data: type: object title: Account description: |- Represents a platform level account registration, authentication and authorization. properties: account_id: type: string maxLength: 24 description: A value that uniquely identifies the account. example: 8aca3b99b4ee99db2c0fb6b7 enterprise_id: type: string maxLength: 24 description: >- A value that uniquely identifies the enterprise to which the account belongs. example: c0c41da6c8 account: type: string maxLength: 20 description: A shorthand identifier for the associated account example: b35eda6673 company: type: string maxLength: 250 description: Name of the company which owns the account example: Snippet Factory deactivated: type: string maxLength: 35 format: date-time nullable: true description: Indicates if the account has been deactivated or not owneremail: type: string maxLength: 320 format: email description: Email address of account owner example: admin@snippetfactory.com external_id: type: string maxLength: 255 default: null nullable: true description: >- And identifier that can be used to reference and id from a different system example: d615396c40 plan: type: object additionalProperties: true description: Billing plan and associated feature information pertaining properties: type: type: string minLength: 4 maxLength: 50 description: The new billing plan type to assign the target account once attached example: enterprise quota: type: object additionalProperties: true description: Quota information pertaining to the account properties: retention: type: integer minimum: 0 maximum: 90 format: int32 description: Data retention policy information pertaining to the account example: 14 retention: type: integer minimum: 0 maximum: 90 format: int32 description: Data retention policy information pertaining to the account status: type: string enum: - active - deactivated - pending description: Indication the account's current activation state created: type: string maxLength: 35 format: date-time description: Indicates when the account was created meta: type: object description: Metadata about the account additionalProperties: true owner: $ref: '#/components/schemas/account-owner' required: - account_id - enterprise_id required: - meta - data '400': description: |- A normalized representation of an error that is returned from an http request in response to failed input validation content: application/json: schema: $ref: '#/components/schemas/validation-error' '401': description: A normalized representation of an error that is returned from an http request content: application/json: schema: $ref: '#/components/schemas/http-error' '429': description: A normalized representation of an error that is returned from an http request content: application/json: schema: $ref: '#/components/schemas/http-error' '500': description: A normalized representation of an error that is returned from an http request content: application/json: schema: $ref: '#/components/schemas/http-error' 5XX: description: A normalized representation of an error that is returned from an http request content: application/json: schema: $ref: '#/components/schemas/http-error' 4XX: description: A normalized representation of an error that is returned from an http request content: application/json: schema: $ref: '#/components/schemas/http-error' /v3/enterprise/pipeline: get: operationId: getEnterprisePipelineList summary: Get pipelines for all child organizations of an enterprise tags: - Enterprise description: Get pipelines for all child organizations of an enterprise security: - AccessToken: [] responses: '200': description: Response containing pipelines for all child organizations of the enterprise content: application/json: schema: additionalProperties: false type: object title: GetEnterprisePipelineListResponse description: Response containing pipelines for all child organizations of the enterprise properties: meta: type: object description: Contains additional contextual information about the api response. properties: type: type: string description: A human readable identifier denoting the resource type minLength: 1 maxLength: 50 page: type: object properties: next: type: string nullable: true maxLength: 256 previous: type: string nullable: true maxLength: 256 pk: type: string description: Indicates the data property which holds the resource identifier nullable: true maxLength: 50 links: type: object description: An object containing uri templates to this, and related resources additionalProperties: true properties: self: type: object additionalProperties: true properties: list: nullable: true additionalProperties: false type: object properties: method: enum: - acl - bind - checkout - connect - copy - delete - get - head - link - lock - m-search - merge - mkactivity - mkcalendar - mkcol - move - notify - options - patch - post - propfind - proppatch - purge - put - rebind - report - search - source - subscribe - trace - unbind - unlink - unlock - unsubscribe type: string example: get uri: type: string format: uri-template maxLength: 250 description: A uri template for retrieving a list of the resource type example: https://api.mezmo.com/v3/enterprise/resource detail: nullable: true additionalProperties: false type: object properties: method: enum: - acl - bind - checkout - connect - copy - delete - get - head - link - lock - m-search - merge - mkactivity - mkcalendar - mkcol - move - notify - options - patch - post - propfind - proppatch - purge - put - rebind - report - search - source - subscribe - trace - unbind - unlink - unlock - unsubscribe type: string example: get uri: type: string format: uri-template maxLength: 250 description: |- A uri template for retrieving a single detail representation of the resource type example: https://api.mezmo.com/v3/enterprise/resource/{resource_id} update: nullable: true additionalProperties: false type: object properties: method: enum: - acl - bind - checkout - connect - copy - delete - get - head - link - lock - m-search - merge - mkactivity - mkcalendar - mkcol - move - notify - options - patch - post - propfind - proppatch - purge - put - rebind - report - search - source - subscribe - trace - unbind - unlink - unlock - unsubscribe type: string example: patch uri: type: string format: uri-template maxLength: 250 description: A uri template for partially updating an individual resource example: https://api.mezmo.com/v3/enterprise/resource/{resource_id} replace: nullable: true additionalProperties: false type: object properties: method: enum: - acl - bind - checkout - connect - copy - delete - get - head - link - lock - m-search - merge - mkactivity - mkcalendar - mkcol - move - notify - options - patch - post - propfind - proppatch - purge - put - rebind - report - search - source - subscribe - trace - unbind - unlink - unlock - unsubscribe type: string example: put uri: type: string format: uri-template maxLength: 250 description: >- A uri template for fully replacing or updating an individual resource example: https://api.mezmo.com/v3/enterprise/resource/{resource_id} create: nullable: true additionalProperties: false type: object properties: method: enum: - acl - bind - checkout - connect - copy - delete - get - head - link - lock - m-search - merge - mkactivity - mkcalendar - mkcol - move - notify - options - patch - post - propfind - proppatch - purge - put - rebind - report - search - source - subscribe - trace - unbind - unlink - unlock - unsubscribe type: string example: post uri: type: string format: uri-template maxLength: 250 description: >- A uri template for creating a new instance of a specific resource example: https://api.mezmo.com/v3/enterprise/resource related: type: object additionalProperties: true description: |- Additional URI templates for interacting with resources which are tightly related required: - self required: - type - page - pk - links data: type: array items: $ref: '#/components/schemas/pipeline' required: - meta - data '400': description: |- A normalized representation of an error that is returned from an http request in response to failed input validation content: application/json: schema: $ref: '#/components/schemas/validation-error' '401': description: A normalized representation of an error that is returned from an http request content: application/json: schema: $ref: '#/components/schemas/http-error' '429': description: A normalized representation of an error that is returned from an http request content: application/json: schema: $ref: '#/components/schemas/http-error' '500': description: A normalized representation of an error that is returned from an http request content: application/json: schema: $ref: '#/components/schemas/http-error' 5XX: description: A normalized representation of an error that is returned from an http request content: application/json: schema: $ref: '#/components/schemas/http-error' 4XX: description: A normalized representation of an error that is returned from an http request content: application/json: schema: $ref: '#/components/schemas/http-error' /v3/enterprise/pipeline/metric/health: get: operationId: getEnterprisePipelineMetricHealth summary: Get pipeline health metrics for all child organizations of an enterprise tags: - Enterprise description: Get pipeline health metrics for all child organizations of an enterprise parameters: - schema: type: boolean default: true in: query name: include_logs required: false description: Fetch previous log messages for transforms. security: - AccessToken: [] responses: '200': description: Response containing pipeline health metrics for all child orgs of the enterprise content: application/json: schema: additionalProperties: false type: object title: GetEnterprisePipelineMetricHealthResponse description: Response containing pipeline health metrics for all child orgs of the enterprise properties: meta: type: object description: Contains additional contextual information about the api response. properties: type: type: string description: A human readable identifier denoting the resource type minLength: 1 maxLength: 50 page: type: object properties: next: type: string nullable: true maxLength: 256 previous: type: string nullable: true maxLength: 256 pk: type: string description: Indicates the data property which holds the resource identifier nullable: true maxLength: 50 links: type: object description: An object containing uri templates to this, and related resources additionalProperties: true properties: self: type: object additionalProperties: true properties: list: nullable: true additionalProperties: false type: object properties: method: enum: - acl - bind - checkout - connect - copy - delete - get - head - link - lock - m-search - merge - mkactivity - mkcalendar - mkcol - move - notify - options - patch - post - propfind - proppatch - purge - put - rebind - report - search - source - subscribe - trace - unbind - unlink - unlock - unsubscribe type: string example: get uri: type: string format: uri-template maxLength: 250 description: A uri template for retrieving a list of the resource type example: https://api.mezmo.com/v3/enterprise/resource detail: nullable: true additionalProperties: false type: object properties: method: enum: - acl - bind - checkout - connect - copy - delete - get - head - link - lock - m-search - merge - mkactivity - mkcalendar - mkcol - move - notify - options - patch - post - propfind - proppatch - purge - put - rebind - report - search - source - subscribe - trace - unbind - unlink - unlock - unsubscribe type: string example: get uri: type: string format: uri-template maxLength: 250 description: |- A uri template for retrieving a single detail representation of the resource type example: https://api.mezmo.com/v3/enterprise/resource/{resource_id} update: nullable: true additionalProperties: false type: object properties: method: enum: - acl - bind - checkout - connect - copy - delete - get - head - link - lock - m-search - merge - mkactivity - mkcalendar - mkcol - move - notify - options - patch - post - propfind - proppatch - purge - put - rebind - report - search - source - subscribe - trace - unbind - unlink - unlock - unsubscribe type: string example: patch uri: type: string format: uri-template maxLength: 250 description: A uri template for partially updating an individual resource example: https://api.mezmo.com/v3/enterprise/resource/{resource_id} replace: nullable: true additionalProperties: false type: object properties: method: enum: - acl - bind - checkout - connect - copy - delete - get - head - link - lock - m-search - merge - mkactivity - mkcalendar - mkcol - move - notify - options - patch - post - propfind - proppatch - purge - put - rebind - report - search - source - subscribe - trace - unbind - unlink - unlock - unsubscribe type: string example: put uri: type: string format: uri-template maxLength: 250 description: >- A uri template for fully replacing or updating an individual resource example: https://api.mezmo.com/v3/enterprise/resource/{resource_id} create: nullable: true additionalProperties: false type: object properties: method: enum: - acl - bind - checkout - connect - copy - delete - get - head - link - lock - m-search - merge - mkactivity - mkcalendar - mkcol - move - notify - options - patch - post - propfind - proppatch - purge - put - rebind - report - search - source - subscribe - trace - unbind - unlink - unlock - unsubscribe type: string example: post uri: type: string format: uri-template maxLength: 250 description: >- A uri template for creating a new instance of a specific resource example: https://api.mezmo.com/v3/enterprise/resource related: type: object additionalProperties: true description: |- Additional URI templates for interacting with resources which are tightly related required: - self required: - type - page - pk - links data: type: array items: $ref: '#/components/schemas/pipeline-metric-health' required: - meta - data '400': description: |- A normalized representation of an error that is returned from an http request in response to failed input validation content: application/json: schema: $ref: '#/components/schemas/validation-error' '401': description: A normalized representation of an error that is returned from an http request content: application/json: schema: $ref: '#/components/schemas/http-error' '429': description: A normalized representation of an error that is returned from an http request content: application/json: schema: $ref: '#/components/schemas/http-error' '500': description: A normalized representation of an error that is returned from an http request content: application/json: schema: $ref: '#/components/schemas/http-error' 5XX: description: A normalized representation of an error that is returned from an http request content: application/json: schema: $ref: '#/components/schemas/http-error' 4XX: description: A normalized representation of an error that is returned from an http request content: application/json: schema: $ref: '#/components/schemas/http-error' /v3/enterprise/pipeline/metric/usage: get: operationId: getEnterprisePipelineMetricUsage summary: Get pipeline usage metrics for all child organizations of an enterprise tags: - Enterprise description: Get pipeline usage metrics for all child organizations of an enterprise parameters: - schema: type: string format: date-time in: query name: from required: true description: Define the beginning of the window - schema: type: string format: date-time in: query name: to required: true description: Define the end of the window - schema: type: integer in: query name: bucket required: true description: Size of the interval represented by each returned metric result - schema: type: boolean default: true in: query name: include_deleted required: false description: Include pipelines and nodes that have been deleted - schema: type: string format: uuid in: query name: pipeline_id required: false description: >- Limit usage metrics to the given pipeline id. If not defined, the metrics will include all pipelines for the account. - schema: type: string enum: - ingress - egress in: query name: traffic required: false description: >- Limit usage metrics to a single traffic direction. If not defined, the metrics for both directions will be included. security: - AccessToken: [] responses: '200': description: Response containing pipeline usage metrics for all child orgs of the enterprise content: application/json: schema: additionalProperties: false type: object title: GetEnterprisePipelineMetricUsageResponse description: Response containing pipeline usage metrics for all child orgs of the enterprise properties: meta: type: object description: Contains additional contextual information about the api response. properties: type: type: string description: A human readable identifier denoting the resource type minLength: 1 maxLength: 50 page: type: object properties: next: type: string nullable: true maxLength: 256 previous: type: string nullable: true maxLength: 256 pk: type: string description: Indicates the data property which holds the resource identifier nullable: true maxLength: 50 links: type: object description: An object containing uri templates to this, and related resources additionalProperties: true properties: self: type: object additionalProperties: true properties: list: nullable: true additionalProperties: false type: object properties: method: enum: - acl - bind - checkout - connect - copy - delete - get - head - link - lock - m-search - merge - mkactivity - mkcalendar - mkcol - move - notify - options - patch - post - propfind - proppatch - purge - put - rebind - report - search - source - subscribe - trace - unbind - unlink - unlock - unsubscribe type: string example: get uri: type: string format: uri-template maxLength: 250 description: A uri template for retrieving a list of the resource type example: https://api.mezmo.com/v3/enterprise/resource detail: nullable: true additionalProperties: false type: object properties: method: enum: - acl - bind - checkout - connect - copy - delete - get - head - link - lock - m-search - merge - mkactivity - mkcalendar - mkcol - move - notify - options - patch - post - propfind - proppatch - purge - put - rebind - report - search - source - subscribe - trace - unbind - unlink - unlock - unsubscribe type: string example: get uri: type: string format: uri-template maxLength: 250 description: |- A uri template for retrieving a single detail representation of the resource type example: https://api.mezmo.com/v3/enterprise/resource/{resource_id} update: nullable: true additionalProperties: false type: object properties: method: enum: - acl - bind - checkout - connect - copy - delete - get - head - link - lock - m-search - merge - mkactivity - mkcalendar - mkcol - move - notify - options - patch - post - propfind - proppatch - purge - put - rebind - report - search - source - subscribe - trace - unbind - unlink - unlock - unsubscribe type: string example: patch uri: type: string format: uri-template maxLength: 250 description: A uri template for partially updating an individual resource example: https://api.mezmo.com/v3/enterprise/resource/{resource_id} replace: nullable: true additionalProperties: false type: object properties: method: enum: - acl - bind - checkout - connect - copy - delete - get - head - link - lock - m-search - merge - mkactivity - mkcalendar - mkcol - move - notify - options - patch - post - propfind - proppatch - purge - put - rebind - report - search - source - subscribe - trace - unbind - unlink - unlock - unsubscribe type: string example: put uri: type: string format: uri-template maxLength: 250 description: >- A uri template for fully replacing or updating an individual resource example: https://api.mezmo.com/v3/enterprise/resource/{resource_id} create: nullable: true additionalProperties: false type: object properties: method: enum: - acl - bind - checkout - connect - copy - delete - get - head - link - lock - m-search - merge - mkactivity - mkcalendar - mkcol - move - notify - options - patch - post - propfind - proppatch - purge - put - rebind - report - search - source - subscribe - trace - unbind - unlink - unlock - unsubscribe type: string example: post uri: type: string format: uri-template maxLength: 250 description: >- A uri template for creating a new instance of a specific resource example: https://api.mezmo.com/v3/enterprise/resource related: type: object additionalProperties: true description: |- Additional URI templates for interacting with resources which are tightly related required: - self required: - type - page - pk - links data: type: array items: $ref: '#/components/schemas/pipeline-metric-usage' required: - meta - data '400': description: |- A normalized representation of an error that is returned from an http request in response to failed input validation content: application/json: schema: $ref: '#/components/schemas/validation-error' '401': description: A normalized representation of an error that is returned from an http request content: application/json: schema: $ref: '#/components/schemas/http-error' '429': description: A normalized representation of an error that is returned from an http request content: application/json: schema: $ref: '#/components/schemas/http-error' '500': description: A normalized representation of an error that is returned from an http request content: application/json: schema: $ref: '#/components/schemas/http-error' 5XX: description: A normalized representation of an error that is returned from an http request content: application/json: schema: $ref: '#/components/schemas/http-error' 4XX: description: A normalized representation of an error that is returned from an http request content: application/json: schema: $ref: '#/components/schemas/http-error' /v3/enterprise/pipeline/node: get: operationId: getEnterprisePipelineNode summary: Get pipeline node types and schemas tags: - Enterprise description: Get pipeline node types, operators and alert schemas security: - AccessToken: [] responses: '200': description: Response containing pipeline node types, operators and alert schemas content: application/json: schema: additionalProperties: false type: object title: GetEnterprisePipelineNodeResponse description: Response containing pipeline node types, operators and alert schemas properties: meta: type: object description: Contains additional contextual information about the api response. properties: type: type: string description: A human readable identifier denoting the resource type minLength: 1 maxLength: 50 page: type: object properties: next: type: string nullable: true maxLength: 256 previous: type: string nullable: true maxLength: 256 pk: type: string description: Indicates the data property which holds the resource identifier nullable: true maxLength: 50 links: type: object description: An object containing uri templates to this, and related resources additionalProperties: true properties: self: type: object additionalProperties: true properties: list: nullable: true additionalProperties: false type: object properties: method: enum: - acl - bind - checkout - connect - copy - delete - get - head - link - lock - m-search - merge - mkactivity - mkcalendar - mkcol - move - notify - options - patch - post - propfind - proppatch - purge - put - rebind - report - search - source - subscribe - trace - unbind - unlink - unlock - unsubscribe type: string example: get uri: type: string format: uri-template maxLength: 250 description: A uri template for retrieving a list of the resource type example: https://api.mezmo.com/v3/enterprise/resource detail: nullable: true additionalProperties: false type: object properties: method: enum: - acl - bind - checkout - connect - copy - delete - get - head - link - lock - m-search - merge - mkactivity - mkcalendar - mkcol - move - notify - options - patch - post - propfind - proppatch - purge - put - rebind - report - search - source - subscribe - trace - unbind - unlink - unlock - unsubscribe type: string example: get uri: type: string format: uri-template maxLength: 250 description: |- A uri template for retrieving a single detail representation of the resource type example: https://api.mezmo.com/v3/enterprise/resource/{resource_id} update: nullable: true additionalProperties: false type: object properties: method: enum: - acl - bind - checkout - connect - copy - delete - get - head - link - lock - m-search - merge - mkactivity - mkcalendar - mkcol - move - notify - options - patch - post - propfind - proppatch - purge - put - rebind - report - search - source - subscribe - trace - unbind - unlink - unlock - unsubscribe type: string example: patch uri: type: string format: uri-template maxLength: 250 description: A uri template for partially updating an individual resource example: https://api.mezmo.com/v3/enterprise/resource/{resource_id} replace: nullable: true additionalProperties: false type: object properties: method: enum: - acl - bind - checkout - connect - copy - delete - get - head - link - lock - m-search - merge - mkactivity - mkcalendar - mkcol - move - notify - options - patch - post - propfind - proppatch - purge - put - rebind - report - search - source - subscribe - trace - unbind - unlink - unlock - unsubscribe type: string example: put uri: type: string format: uri-template maxLength: 250 description: >- A uri template for fully replacing or updating an individual resource example: https://api.mezmo.com/v3/enterprise/resource/{resource_id} create: nullable: true additionalProperties: false type: object properties: method: enum: - acl - bind - checkout - connect - copy - delete - get - head - link - lock - m-search - merge - mkactivity - mkcalendar - mkcol - move - notify - options - patch - post - propfind - proppatch - purge - put - rebind - report - search - source - subscribe - trace - unbind - unlink - unlock - unsubscribe type: string example: post uri: type: string format: uri-template maxLength: 250 description: >- A uri template for creating a new instance of a specific resource example: https://api.mezmo.com/v3/enterprise/resource related: type: object additionalProperties: true description: |- Additional URI templates for interacting with resources which are tightly related required: - self required: - type - page - pk - links data: $ref: '#/components/schemas/pipeline-node' required: - meta - data '400': description: |- A normalized representation of an error that is returned from an http request in response to failed input validation content: application/json: schema: $ref: '#/components/schemas/validation-error' '401': description: A normalized representation of an error that is returned from an http request content: application/json: schema: $ref: '#/components/schemas/http-error' '429': description: A normalized representation of an error that is returned from an http request content: application/json: schema: $ref: '#/components/schemas/http-error' '500': description: A normalized representation of an error that is returned from an http request content: application/json: schema: $ref: '#/components/schemas/http-error' 5XX: description: A normalized representation of an error that is returned from an http request content: application/json: schema: $ref: '#/components/schemas/http-error' 4XX: description: A normalized representation of an error that is returned from an http request content: application/json: schema: $ref: '#/components/schemas/http-error' /v1/enterprise/account: get: security: - enterpriseServiceKeyAuth: [] tags: - Enterprise v1 summary: List all accounts description: |- > **Deprecated.** Superseded by the `/v3/enterprise` endpoints under the **Enterprise** tag. Retrieves a list of all child organizations that belong to the enterprise. operationId: list-orgs responses: '200': description: A list of child organizations is returned on success. content: application/json: schema: type: array items: $ref: '#/components/schemas/enterpriseResource' '400': description: A poorly formatted request returns with a `BadRequest` code. content: application/json: schema: $ref: '#/components/schemas/errResponse' example: error: '"test" is not allowed' code: BadRequest status: error deprecated: true post: security: - enterpriseServiceKeyAuth: [] basicAuth: [] - enterpriseServiceKeyAuth: [] servicekeyAuth: [] tags: - Enterprise v1 summary: Attach a child organization to an enterprise description: >- > **Deprecated.** Superseded by the `/v3/enterprise` endpoints under the **Enterprise** tag. Attaches an existing organization to an enterprise. The `retention` and `owner` field are optional, and will be defaulted if not specified. To specify which child organization to attach, one of its service keys must be supplied through our typical service key authorization schema; either as basic authorization or through the `servicekey` header. operationId: create-org requestBody: description: Request parameters. content: application/json: schema: $ref: '#/components/schemas/enterprisePut' required: true responses: '200': description: Child organization successfully attached. content: application/json: schema: $ref: '#/components/schemas/enterpriseResource' '400': description: A poorly formatted request returns with a `BadRequest` code. content: application/json: schema: $ref: '#/components/schemas/errResponse' example: error: 'Service Key Validation Error: Invalid or deactivated servicekey' code: BadRequest status: error '409': description: >- The supplied request leads to an invalid state. The returned error will contain more helpful information. content: application/json: schema: $ref: '#/components/schemas/errPropsDefault' example: code: Conflict error: No contract found for this enterprise. status: error deprecated: true /v1/enterprise/account/{accountId}: get: security: - enterpriseServiceKeyAuth: [] tags: - Enterprise v1 summary: Get details of a child organization description: >- > **Deprecated.** Superseded by the `/v3/enterprise` endpoints under the **Enterprise** tag. Retrieves a specific child organization by its id. If the organization does not belong to the enterprise, a 404 error is returned. operationId: get-org parameters: - name: accountId in: path required: true schema: $ref: '#/components/schemas/accountId' responses: '200': description: The child organization was found and returned. content: application/json: schema: $ref: '#/components/schemas/enterpriseResource' deprecated: true put: security: - enterpriseServiceKeyAuth: [] tags: - Enterprise v1 summary: Update Child Organization Contract description: >- > **Deprecated.** Superseded by the `/v3/enterprise` endpoints under the **Enterprise** tag. Makes an update to the specified child organization. Used to modify the current retention with the `retention` field or the current owner with the `owner` field. operationId: put-org parameters: - name: accountId in: path required: true schema: $ref: '#/components/schemas/accountId' requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/enterprisePut' responses: '200': description: The account was found, the enterpriseId and retention details are updated. content: application/json: schema: $ref: '#/components/schemas/enterpriseResource' '400': description: Invalid body content: application/json: schema: $ref: '#/components/schemas/errResponse' example: error: '"plan" is not allowed' code: BadRequest status: error '404': description: Unable to update account which not belong to enterprise content: application/json: schema: $ref: '#/components/schemas/errResponse' example: error: Invalid Account. code: NotFound status: error '409': description: Return error when account does not have contract tier content: application/json: schema: $ref: '#/components/schemas/errResponse' example: error: No contract found for this account. code: Conflict status: error deprecated: true delete: security: - enterpriseServiceKeyAuth: [] tags: - Enterprise v1 summary: Detach a child org from Enterprise description: >- > **Deprecated.** Superseded by the `/v3/enterprise` endpoints under the **Enterprise** tag. Detaches a child organization from the enterprise. The child organization is not deleted and can continue to work afterwards. operationId: delete-org parameters: - name: accountId in: path required: true schema: $ref: '#/components/schemas/accountId' responses: '200': description: The child org was removed successfully, a simple status is returned. content: application/json: schema: type: object properties: account: $ref: '#/components/schemas/accountId' '400': description: Invalid body content: application/json: schema: $ref: '#/components/schemas/errResponse' example: error: '"account_id" is not allowed' code: BadRequest status: error '409': description: Account enterprise id mismatch content: application/json: schema: $ref: '#/components/schemas/errResponse' example: error: Invalid Account code: Conflict status: error deprecated: true /v1/enterprise/keys: get: security: - enterpriseServiceKeyAuth: [] tags: - Enterprise v1 summary: List all enterprise keys description: |- > **Deprecated.** Superseded by the `/v3/enterprise` endpoints under the **Enterprise** tag. Retrieves all keys of the enterprise operationId: list-enterpriseKeys responses: '200': description: A list of enterprise service keys is returned on success. content: application/json: schema: type: array items: $ref: '#/components/schemas/keyResource' '400': description: Invalid query params content: application/json: schema: $ref: '#/components/schemas/errResponse' example: error: '"value" must be of type object' code: BadRequest status: error deprecated: true post: security: - enterpriseServiceKeyAuth: [] tags: - Enterprise v1 summary: Generate a new Key for the Enterprise description: |- > **Deprecated.** Superseded by the `/v3/enterprise` endpoints under the **Enterprise** tag. Generate a new ServiceKey for the enterprise operationId: create-enterpriseKey responses: '200': description: 'A new Service Key is generated and saved to the Key collection ' content: application/json: schema: $ref: '#/components/schemas/keyResource' '400': description: The supplied request is invalid or malformed. content: application/json: schema: $ref: '#/components/schemas/errResponse' example: error: '"param" is not allowed' code: BadRequest status: error '409': description: >- The supplied request leads to an invalid state. The returned error will contain more helpful information. content: application/json: schema: $ref: '#/components/schemas/errPropsDefault' example: code: Conflict error: This account is not allowed more than 50 enterprise-service keys. status: error deprecated: true /v1/enterprise/keys/{id}: get: security: - enterpriseServiceKeyAuth: [] tags: - Enterprise v1 summary: Retrieve a specific entperise key description: |- > **Deprecated.** Superseded by the `/v3/enterprise` endpoints under the **Enterprise** tag. Retrieves a specific enterprise key by its `id`. operationId: get-entperiseKey parameters: - name: id in: path required: true schema: $ref: '#/components/schemas/keyId' responses: '200': description: The key with the supplied `id` was found, the entire resource is returned. content: application/json: schema: $ref: '#/components/schemas/keyResource' '400': description: Invalid query params content: application/json: schema: $ref: '#/components/schemas/errResponse' example: error: '"value" must be of type object' code: BadRequest status: error deprecated: true delete: security: - enterpriseServiceKeyAuth: [] tags: - Enterprise v1 summary: Delete an Enterprise ServiceKey description: |- > **Deprecated.** Superseded by the `/v3/enterprise` endpoints under the **Enterprise** tag. Delete an Enterprise ServiceKey from list of keys for the Enterprise operationId: delete-entperiseKey parameters: - name: id in: path required: true schema: $ref: '#/components/schemas/keyId' responses: '200': description: The Enterprise ServiceKey was deleted successfully, a simple status is returned. content: application/json: schema: type: object properties: message: type: string description: The status of the request in a human-readable form example: Key deleted successfully. deleted: type: boolean description: Indicates whether the member was deleted example: true deprecated: true /v1/config/ingestion/exclusions: get: tags: - Exclusions summary: List Exclusion Rules description: 'Returns a list of exclusion rules. Note: This does not return usage quota rules.' operationId: get-v1-config-exclusions responses: '200': description: OK content: application/json: schema: type: array items: $ref: '#/components/schemas/streamExclusionRequest' '500': description: Internal Server error occured when trying to fetch exclusion rules content: '*/*': schema: $ref: '#/components/schemas/errPropsDefault' post: tags: - Exclusions summary: Create Exclusion Rule description: Create a new ingestion exclusion rule to help reduce log volume operationId: post-v1-config-exclusions requestBody: description: Request parameters content: application/json: schema: $ref: '#/components/schemas/streamExclusionRequest' required: true responses: '200': description: OK content: application/json: schema: $ref: '#/components/schemas/streamExclusionRequest' '400': description: Bad request. Query inside of exclusion rule failed to parse. content: '*/*': schema: $ref: '#/components/schemas/errPropsDefault' '500': description: An internal error occured when trying to create exclusion rule. content: '*/*': schema: $ref: '#/components/schemas/errPropsDefault' /v1/config/ingestion/exclusions/{id}: get: tags: - Exclusions summary: Get Exclusion Rule description: Returns an exclusion rule with the id specified. operationId: get-v1-config-exclusions-single parameters: - name: id in: path description: ID of an exclusion rule. required: true schema: type: string responses: '200': description: OK content: application/json: schema: type: array items: $ref: '#/components/schemas/streamExclusionRequest' '400': description: >- Malformed ingestion exclusion rule response payload. The exclusion rule may be stored incorrectly content: '*/*': schema: $ref: '#/components/schemas/errPropsDefault' '404': description: Exclusion rule not found content: '*/*': schema: $ref: '#/components/schemas/errPropsDefault' '500': description: Internal Server Error. Failed to get exclusion rule content: '*/*': schema: $ref: '#/components/schemas/errPropsDefault' patch: tags: - Exclusions summary: Update Exclusion Rule description: Updates an existing exclusion rule operationId: patch-v1-config-exclusions-single parameters: - name: id in: path description: ID of an exclusion rule. required: true schema: type: string requestBody: description: Request parameters content: application/json: schema: $ref: '#/components/schemas/streamExclusionRequest' required: true responses: '200': description: OK content: application/json: schema: $ref: '#/components/schemas/streamExclusionRequest' '400': description: Bad request. The rule is invalid. Double check rule and query syntax and try again. content: '*/*': schema: $ref: '#/components/schemas/errPropsDefault' '500': description: Internal server error. The rule could not be updated at this time. content: '*/*': schema: $ref: '#/components/schemas/errPropsDefault' delete: tags: - Exclusions summary: Delete Exclusion Rule description: Use this method to delete an exclusion rule. operationId: delete-v1-config-exclusions-single parameters: - name: id in: path description: ID of an exclusion rule. required: true schema: type: string responses: '200': description: OK content: {} '404': description: Exclusion rule not found content: '*/*': schema: $ref: '#/components/schemas/errPropsDefault' '500': description: Internal Server Error. Failed to delete exclusion rule. content: '*/*': schema: $ref: '#/components/schemas/errPropsDefault' /v1/export: get: tags: - Export summary: Export log lines description: Use this method to export logs in JSON format from a logging instance. operationId: export parameters: - name: from in: query description: Start time. Set as UNIX timestamp in seconds or milliseconds. required: true schema: type: string - name: to in: query description: End time. Set as UNIX timestamp in seconds or milliseconds. required: true schema: type: string - name: size in: query description: Number of log lines to include in the export. schema: type: string - name: hosts in: query description: Comma-separated list of hosts. schema: type: string - name: apps in: query description: Comma-separated list of applications. schema: type: string - name: levels in: query description: Comma-separated list of log levels. schema: type: string - name: query in: query description: Search query. schema: type: string - name: prefer in: query description: >- Defines the log lines that you want to export. Valid values are head, first log lines, and tail, last log lines. If not specified, defaults to tail. schema: type: string - name: email in: query description: >- Specifies the email with the downloadable link of your export. By default, the log lines are streamed. schema: type: string - name: emailSubject in: query description: >- Use to set the subject of the email. Use to represent a space. For example, a sample value is Export logs. schema: type: string responses: '200': description: OK content: application/json: schema: type: object additionalProperties: type: object properties: {} '400': description: Bad request content: {} deprecated: false /v2/export: get: tags: - Export summary: Export log lines description: Use this method to export logs in JSON format from a logging instance. operationId: exportv2 parameters: - name: from in: query description: Start time (inclusive). Set as UNIX timestamp in seconds or milliseconds. required: true schema: type: string - name: to in: query description: End time (inclusive). Set as UNIX timestamp in seconds or milliseconds. required: true schema: type: string - name: size in: query description: Number of log lines to include in the export. schema: type: string - name: hosts in: query description: Comma-separated list of hosts. schema: type: string - name: apps in: query description: Comma-separated list of applications. schema: type: string - name: levels in: query description: Comma-separated list of log levels. schema: type: string - name: query in: query description: Search query. schema: type: string - name: prefer in: query description: >- Defines the log lines that you want to export. Valid values are head, first log lines, and tail, last log lines. If not specified, defaults to tail. schema: type: string - name: pagination_id in: query description: >- ID that indicates which page of results to be retrieved. Leave empty for the initial export request. schema: type: string responses: '200': description: OK content: application/json: schema: type: object additionalProperties: type: object properties: {} '400': description: Bad request content: {} deprecated: false /v1/config/groups: get: tags: - Groups summary: List Groups description: Use this method to list the log groups. operationId: list-group responses: '200': description: OK content: application/json: schema: type: object additionalProperties: type: object properties: {} '400': description: Bad request content: {} deprecated: false post: tags: - Groups summary: Create Group description: >- Use this method to create a log group where you can scope the data that is accessible by members in that group. operationId: create-group parameters: - name: servicekey in: header description: A provisioned service key for your account. required: true schema: type: string requestBody: description: Request parameters content: application/json: schema: $ref: '#/components/schemas/groupsRequest' required: true responses: '200': description: OK content: application/json: schema: type: object additionalProperties: type: object properties: {} '400': description: Bad request content: {} deprecated: false x-codegen-request-body-name: groupsRequest /v1/config/groups/{groupId}: get: tags: - Groups summary: Get Group description: Use this method to get information on a group. operationId: read-group parameters: - name: groupId in: path description: ID of a group. required: true schema: type: string responses: '200': description: OK content: application/json: schema: type: object additionalProperties: type: object properties: {} '400': description: Bad request content: {} deprecated: false delete: tags: - Groups summary: Delete Group description: Use this method to list the log groups. operationId: delete-group parameters: - name: servicekey in: header description: A provisioned service key for your account. required: true schema: type: string - name: groupId in: path description: ID of a group. required: true schema: type: string responses: '200': description: OK content: application/json: schema: type: object additionalProperties: type: object properties: {} '400': description: Bad request content: {} deprecated: false patch: tags: - Groups summary: Update Group description: >- Use this method to modify a log group. You can change the name of the group and the access scope that defines the data that is accessible by members in that group. operationId: update-group parameters: - name: servicekey in: header description: A provisioned service key for your account. required: true schema: type: string - name: groupId in: path description: ID of a group. required: true schema: type: string requestBody: description: Request parameters content: application/json: schema: $ref: '#/components/schemas/groupsRequest' required: true responses: '200': description: OK content: application/json: schema: type: object additionalProperties: type: object properties: {} '400': description: Bad request content: {} deprecated: false x-codegen-request-body-name: groupsRequest /v3/pipeline/state-variable/{state_variable_id}: get: summary: Fetch state variable tags: - Incident Mode description: Fetches the value of a state variable parameters: - schema: type: string format: uuid in: path name: state_variable_id required: true responses: '200': description: Default Response put: summary: Update state variable tags: - Incident Mode description: Edits an existing state variable requestBody: content: application/json: schema: type: object properties: pipeline_id: type: string format: uuid state: type: object required: - pipeline_id - state required: true parameters: - schema: type: string format: uuid in: path name: state_variable_id required: true responses: '200': description: Default Response /v3/pipeline/state-variable: get: summary: List all state variables tags: - Incident Mode description: List all state variables defined in the account parameters: - schema: type: string format: uuid in: query name: pipeline_id required: false responses: '200': description: Default Response patch: summary: Partially update state variable tags: - Incident Mode description: Update one or many state variables requestBody: content: application/json: schema: type: object properties: pipeline_id: type: string format: uuid state: type: object required: - pipeline_id - state required: true responses: '200': description: Default Response post: summary: Create state variable tags: - Incident Mode description: Add a pipeline state variable to the account requestBody: content: application/json: schema: type: object properties: pipeline_id: type: string format: uuid state: type: object required: - pipeline_id - state required: true responses: '200': description: Default Response /v1/config/index-rate: get: tags: - Index Rate Alert summary: Get Index Rate Alert description: Gets the configuration for an Index Rate Alert operationId: get-index-rate-alert parameters: [] responses: '200': description: OK content: application/json: schema: $ref: '#/components/schemas/indexRateAlertConfigResponse' '400': description: Bad request content: {} '500': description: Internal Server error occurred when trying to fetch index rate alerts content: '*/*': schema: $ref: '#/components/schemas/errPropsDefault' put: tags: - Index Rate Alert summary: Update Index Rate Alert description: >- Use this method to update an index rate alert. You can change the alert's configuration details. operationId: update-index-rate-alert requestBody: description: Request parameters content: application/json: schema: $ref: '#/components/schemas/indexRateAlertConfigResponse' required: true responses: '200': description: OK content: application/json: schema: $ref: '#/components/schemas/indexRateAlertConfigResponse' '400': description: Bad request content: {} '500': description: Internal Server error occurred when trying to fetch index rate alerts content: '*/*': schema: $ref: '#/components/schemas/errPropsDefault' deprecated: false x-codegen-request-body-name: configRequest /logs/ingest: servers: - url: https://logs.mezmo.com post: security: - apikeyAuth: [] - basicAuth: [] tags: - Ingestion summary: Send Log Lines description: > Use this method to send logs to a logging instance. ### Authentication You can find instructions on authentication [here](/log-analysis-api/ref#authentication). Ingestion is similar to the other APIs, but instead of `servicekey` you will use `apikey` if you are using the header style authentication. ### Metadata Meta is a field reserved for custom information associated with a log line. To add metadata to an API call, specify the `meta` field under the lines object. Metadata can be viewed inside that line's context WARNING: If inconsistent value types are used, that line's metadata, will not be parsed. For example, if a line is passed with a meta object, such as `meta.myfield` of type String, any subsequent lines with `meta.myfield` must have a String as the value type for `meta.myfield`. > Please be aware of [service limits](https://docs.mezmo.com/docs/Mezmo-ingestion-service-limits) > on this endpoint operationId: ingest parameters: - name: hostname in: query description: Host name of the source. required: true schema: type: string - name: mac in: query description: The network mac address of the host computer. schema: $ref: '#/components/schemas/mac' - name: ip in: query description: The local IP address of the host computer. schema: $ref: '#/components/schemas/ip' - name: now in: query description: >- The source UNIX timestamp in milliseconds at the time of the request. Used to calculate time drift. schema: type: string - name: tags in: query description: Tags that are used to dynamically group hosts. schema: type: string requestBody: description: You can send multiple log lines in a request. content: application/json: schema: $ref: '#/components/schemas/ingestlines' required: true responses: '200': description: OK content: application/json: schema: type: object additionalProperties: type: object properties: {} '401': description: Unauthorized content: {} '403': description: Forbidden content: {} '404': description: Not Found content: {} deprecated: false x-codegen-request-body-name: ingestlines /v1/config/keys: get: tags: - Keys summary: Retrieve a list of all the keys description: >- Retrieve a list of all the keys for an account. Supported key types include `ingestion` and `service`. Use the `type` parameter to filter which type of keys to retrieve. operationId: list-key parameters: - name: type in: query description: The type of key to retrieve. Defaults to "all" schema: type: string enum: - ingestion - service - all responses: '200': description: A list of keys is returned on success. content: application/json: schema: type: array items: $ref: '#/components/schemas/keyResource' '400': description: The supplied request is invalid or malformed. content: application/json: schema: $ref: '#/components/schemas/errResponse' example: error: '"type" must be one of [ingestion, service, all]' code: BadRequest status: error post: tags: - Keys summary: Create a new key description: >- Create a new key of the type specified by the `type` parameter. If a name is not supplied, one will be auto generated. operationId: create-key parameters: - name: type in: query description: The type of key to be created required: true schema: type: string enum: - ingestion - service - all requestBody: description: Modifiable fields can be set in the request body. content: application/json: schema: $ref: '#/components/schemas/keysPut' required: false responses: '200': description: A new key was created successfully! The entire key resource is returned. content: application/json: schema: $ref: '#/components/schemas/keyResource' '400': description: The supplied request is invalid or malformed. content: application/json: schema: $ref: '#/components/schemas/errResponse' example: error: '"type" is required' code: BadRequest status: error /v1/config/keys/{id}: get: tags: - Keys summary: Retrieve a specific key description: Retrieves a specific key by its `id`. operationId: get-key parameters: - name: id in: path required: true description: The id of the key to retrieve schema: $ref: '#/components/schemas/keyId' responses: '200': description: The key with the supplied `id` was found, the entire resource is returned. content: application/json: schema: $ref: '#/components/schemas/keyResource' '404': description: The specified key was not found. content: application/json: schema: $ref: '#/components/schemas/errResponse' example: error: Key with id 629eb5369e5b231f51d3830f was not found code: NotFound status: error delete: tags: - Keys summary: Delete a specific key description: Delete a specific key by its `id`. operationId: delete-key parameters: - name: id in: path required: true description: The id of the key to delete schema: $ref: '#/components/schemas/keyId' responses: '200': description: The key was deleted successfully, a simple status is returned. content: application/json: schema: type: object properties: message: type: string description: The status of the request in a human-readable form example: Key deleted successfully deleted: type: boolean description: Indicates whether the key was deleted example: true '404': description: The specified key was not found. content: application/json: schema: $ref: '#/components/schemas/errResponse' example: error: Key with id 629eb5369e5b231f51d3830f was not found code: NotFound status: error put: tags: - Keys summary: Update one or more fields of a key resource description: >- Updates all of the fields described in the request body of the specified key. Fields left out of the body will remain unaffected. operationId: put-key parameters: - name: id in: path required: true description: The id of the key to update schema: $ref: '#/components/schemas/keyId' requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/keysPut' responses: '200': description: The update was successful; the key resource is returned with updated fields. content: application/json: schema: $ref: '#/components/schemas/keyResource' '400': description: The supplied request is invalid or malformed. content: application/json: schema: $ref: '#/components/schemas/errResponse' example: error: '"name" length must be less than or equal to 30 characters long' code: BadRequest status: error '404': description: The specified key was not found. content: application/json: schema: $ref: '#/components/schemas/errResponse' example: error: Key with id 629eb5369e5b231f51d3830f was not found code: NotFound status: error /v3/pipeline/{pipeline_id}/log-volume-reduction: post: summary: Log Volume Reduction tags: - Log Volume Reduction description: Post log volume reduction actions to a pipeline requestBody: content: application/json: schema: type: object properties: place_before_transform_id: type: string format: uuid processor_title: type: string actions: type: array minItems: 1 items: type: object oneOf: - type: object required: - drain - type: object required: - field_summary properties: type: type: string enum: - dedupe - filter - sample - throttle - drop-field title: oneOf: - type: string minLength: 1 maxLength: 512 pattern: ^[^<>"]+$ - type: 'null' errorMessage: invalid title name drain: type: string field_summary: type: object properties: field: type: string minLength: 1 value: oneOf: - type: string minLength: 1 - type: 'null' required: - field description: oneOf: - type: string minLength: 1 pattern: ^[^<>"]+$ - type: 'null' errorMessage: invalid description user_config: type: object if: properties: type: enum: - drop-field then: allOf: - type: object required: - field_summary - type: object properties: field_summary: type: object properties: value: type: 'null' else: properties: type: not: enum: - drop-field required: - processor_title - actions required: true parameters: - schema: type: boolean default: false in: query name: create_new_classifier_analysis required: false - schema: type: string format: uuid in: path name: pipeline_id required: true responses: '200': description: Default Response /v1/config/members: get: tags: - Members summary: Lists all members description: >- Retrieves all of the current members in an organization and returns their email, role, and groups. operationId: list-members responses: '200': description: A list of members is returned on success. content: application/json: schema: type: array items: $ref: '#/components/schemas/memberResource' post: tags: - Members summary: Creates a new member description: >- Create a new user and add to the team by providing 'email', 'role' and an optional list of RBAC groups. operationId: create-member requestBody: description: Request parameters. content: application/json: schema: $ref: '#/components/schemas/memberResource' required: true responses: '200': description: A new member was created successfully! Entire memberResource is returned. content: application/json: schema: $ref: '#/components/schemas/memberResource' '400': description: The supplied request is invalid or malformed. content: application/json: schema: $ref: '#/components/schemas/errResponse' example: error: '"role" is required' code: BadRequest status: error '409': description: >- The supplied request leads to an invalid state. The returned error will contain more helpful information. content: application/json: schema: $ref: '#/components/schemas/errPropsDefault' example: code: Conflict error: Admins can\'t be assigned to any groups /v1/config/members/{email}: get: tags: - Members summary: Retrieve a member description: Returns the role and groups of the specified member. operationId: get-member parameters: - name: email in: path required: true schema: $ref: '#/components/schemas/memberEmail' responses: '200': description: The member with the supplied `email` was found, the entire resource is returned. content: application/json: schema: $ref: '#/components/schemas/memberResource' '404': description: The specified member was not found. content: application/json: schema: $ref: '#/components/schemas/errResponse' example: error: Member "fakeaccount@mezmo.com" not found code: NotFound status: error delete: tags: - Members summary: Delete a member description: Removes a member from the organization. This does not delete the user. operationId: delete-member parameters: - name: email in: path required: true schema: $ref: '#/components/schemas/memberEmail' responses: '200': description: The member was removed successfully, a simple status is returned. content: application/json: schema: type: object properties: message: type: string description: The status of the request in a human-readable form example: Member removed successfully. deleted: type: boolean description: Indicates whether the member was deleted example: true '404': description: The specified member was not found. content: application/json: schema: $ref: '#/components/schemas/errResponse' example: error: The provided email is not an active member of this account. code: NotFound status: error '409': description: The specified member (owner) cannot be removed content: application/json: schema: $ref: '#/components/schemas/errResponse' example: error: The owner of the account can\'t be removed. code: Conflict status: error put: tags: - Members summary: Update a member description: >- Updates all the fields described in the request body of the specified member. Creates a member if the member doesn't exist already. Fields left out of the body will remain unaffected. operationId: put-member parameters: - name: email in: path required: true schema: $ref: '#/components/schemas/memberEmail' requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/membersPut' responses: '200': description: The update was successful; the member resource is returned with updated fields. content: application/json: schema: $ref: '#/components/schemas/memberResource' '400': description: The supplied request is invalid or malformed. content: application/json: schema: $ref: '#/components/schemas/errResponse' example: error: '"value" must contain at least one of [role, groups]' code: BadRequest status: error '409': description: >- The supplied request leads to an invalid state. The returned error will contain more helpful information. content: application/json: schema: $ref: '#/components/schemas/errResponse' example: error: Admins can\'t be assigned to any groups code: Conflict status: error /v3/pipeline/{pipeline_id}/event_metrics: get: summary: Get event metrics for a pipeline tags: - Metrics description: Query and fetch event metrics for a pipeline parameters: - schema: type: number minimum: 300 maximum: 1296000 title: Bucket size bucket_within_range: unit: seconds from_field: from to_field: to in: query name: bucket required: true description: The interval, in seconds, over which events will be grouped - schema: enum: - app - host - label - level default: app title: The field to group by in: query name: field required: false description: The field in the event to group events by - schema: type: string format: date-time title: Events from in: query name: from required: true description: The event metrics start date - schema: type: string format: date-time title: Events to date_time_greater_than: field: from in: query name: to required: true description: The event metrics end date - schema: type: boolean title: With metric changes default: false in: query name: with_changes required: false description: Include metric changes compared to the previous duration - schema: type: string format: uuid in: path name: pipeline_id required: true responses: '200': description: Default Response /v3/pipeline/metric/health: get: summary: Fetch pipeline health metrics tags: - Metrics description: Query and fetch health information for the current account. parameters: - schema: type: string format: uuid in: query name: pipeline_id required: false description: >- Limit health metrics to the given pipeline id. If not defined, the metrics will include all pipelines for the account. - schema: type: boolean default: true in: query name: include_logs required: false description: Fetch previous log messages for transforms. responses: '200': description: Default Response /v3/pipeline/metric/usage: get: summary: Fetch account usage metics tags: - Metrics description: Query and fetch usage metrics for the current account. parameters: - schema: type: string format: date-time in: query name: from required: true description: Define the beginning of the window - schema: type: string format: date-time in: query name: to required: true description: Define the end of the window - schema: type: integer minimum: 300 in: query name: bucket required: true description: Size of the interval represented by each returned metric result - schema: type: boolean default: true in: query name: include_deleted required: false description: Include Pipelines and nodes that have been deleted - schema: type: string format: uuid in: query name: pipeline_id required: false description: >- Limit usage metrics to the given pipeline id. If not defined, the metrics will include all pipelines for the account. - schema: type: string enum: - ingress - egress in: query name: traffic required: false description: >- Limit usage metrics to a single traffic direction. If not defined, the metrics for both directions will be included. responses: '200': description: Default Response /v3/pipeline/metric/usage/profile/log-clustering: get: summary: Fetch log usage metrics tags: - Metrics description: Query and fetch log clustering usage metrics from data profiling. parameters: - schema: type: string format: date-time in: query name: from required: true description: A date/time string that defines the beginning of the window - schema: type: string format: date-time in: query name: to required: false description: A date/time string that defines the end of the window - schema: type: string in: query name: id required: true description: The identifier of the classifier analysis - schema: type: integer default: 20 minimum: 1 maximum: 200 in: query name: limit required: false responses: '200': description: Default Response /v3/pipeline/metric/usage/profile/log-clustering/samples: get: summary: Fetch log cluster samples tags: - Metrics description: Query and fetch log cluster samples from data profiling. parameters: - schema: type: string in: query name: id required: true description: The identifier of the classifier analysis - schema: type: string in: query name: log_cluster_id required: true description: The cluster identifier - schema: type: string format: date-time in: query name: from required: true description: A date/time string that defines the beginning of the window - schema: type: string format: date-time in: query name: to required: false description: A date/time string that defines the end of the window - schema: type: integer default: 20 minimum: 1 maximum: 200 in: query name: limit required: false responses: '200': description: Default Response /v1/config/parsing-template: get: tags: - Parsing Template summary: List Parsing Templates description: Get a list of custom parsing templates. operationId: list-parsing-template responses: '200': description: OK content: application/json: schema: type: array items: type: object properties: templateid: type: string example: '0000000000' account: $ref: '#/components/schemas/accountId' name: type: string description: Name of the template '400': description: Bad request content: {} /v1/config/parsing-template/{templateId}: get: tags: - Parsing Template summary: Get Parsing Template description: Gets the configuration for a specific parsing template. operationId: get-parsing-template parameters: - name: templateId in: path description: ID of a template. required: true schema: type: string responses: '200': description: OK content: application/json: schema: type: object properties: templateid: type: string example: '0000000000' account: $ref: '#/components/schemas/accountId' name: type: string description: Name of the template priority: type: number draft: type: boolean active: type: boolean legacy: type: boolean query: type: string line: type: string draftof: type: string operators: $ref: '#/components/schemas/templateOperators' '400': description: Bad request content: {} delete: tags: - Parsing Template summary: Delete Parsing Template description: Use this method to delete a parsing template. operationId: delete-parsing-template parameters: - name: templateId in: path description: ID of the template. required: true schema: type: string responses: '200': description: OK content: application/json: schema: type: object additionalProperties: type: object properties: {} '400': description: Bad request content: {} deprecated: false /v3/pipeline/{pipeline_id}/export: post: summary: Trigger pipeline export tags: - Pipeline Export description: Prepare a pipeline for export into a terraform workspace. parameters: - schema: type: string format: uuid in: path name: pipeline_id required: true responses: '200': description: Default Response /v3/pipeline/{pipeline_id}: delete: summary: Deletes a pipeline tags: - Pipeline Management description: Removes a pipeline from the account parameters: - schema: type: string format: uuid in: path name: pipeline_id required: true responses: '200': description: Default Response get: summary: Fetch pipeline confg tags: - Pipeline Management description: Retrieve a pipeline configuration parameters: - schema: type: string format: uuid in: path name: pipeline_id required: true responses: '200': description: Default Response put: summary: Update pipeline tags: - Pipeline Management description: Update pipeline properties, like title requestBody: content: application/json: schema: type: object properties: title: type: string pattern: ^[^<>"]+$ errorMessage: invalid title name maxLength: 512 deploy_type: type: string enum: - saas - local default: saas deployment_groups: type: array maxItems: 10 items: type: string pattern: ^[a-zA-Z0-9-_]{1,256}$ required: - title required: true parameters: - schema: type: string format: uuid in: path name: pipeline_id required: true responses: '200': description: Default Response /v3/pipeline: get: summary: List all pipelines tags: - Pipeline Management description: List all pipelines for the account parameters: - schema: type: string enum: - ui - system in: query name: origin required: false description: Include Pipelines with a specific `origin` responses: '200': description: Default Response post: summary: Create pipeline tags: - Pipeline Management description: Creates a new pipeline (non published) requestBody: content: application/json: schema: type: object properties: title: type: string pattern: ^[^<>"]+$ errorMessage: invalid title name maxLength: 512 deploy_type: type: string enum: - saas - local default: saas deployment_groups: type: array maxItems: 10 items: type: string pattern: ^[a-zA-Z0-9-_]{1,256}$ required: - title required: true responses: '200': description: Default Response /v3/pipeline/audit-event: get: summary: Fetch audit events tags: - Pipeline Management description: Retrieve a paginated list of pipeline audit events for an account parameters: - schema: type: string format: date-time in: query name: from required: true - schema: type: string format: date-time default: 2022-01-01T00:00:00.000Z in: query name: to required: true - schema: type: integer minimum: 1 maximum: 200 default: 20 in: query name: page_size required: true - schema: type: integer minimum: 0 default: 0 in: query name: page_index required: true responses: '200': description: Default Response /v3/pipeline/health: get: summary: Summarize pipeline health for one or more pipelines tags: - Pipeline Management description: >- Returns overall ingestion health as well as pipeline-specific health, such as lag in a pipeline's sources. parameters: - schema: type: integer minimum: 1 maximum: 60 default: 15 in: query name: interval_minutes required: false description: The window interval for measuring ingestion volume - schema: type: array items: type: string format: uuid description: A pipeline id in: query name: pipeline_ids required: false description: Return only pipelines with the given IDs responses: '200': description: Default Response content: application/json: schema: type: object properties: meta: type: object additionalProperties: true data: type: object description: Overall health of data ingestion and pipeline sources properties: interval_minutes: type: integer description: The window interval for measuring ingestion volume ingestion_health: type: object properties: ingestion_metrics_status: type: string description: >- A success or failure message for the ability to fetch metrics related to ingestion routes routes: type: array description: Ingestion status code health metrics per route items: type: object properties: route: type: string description: The route path health: type: array description: >- The health of a specific route measured by changes in status code items: type: object properties: status_code: type: integer description: The HTTP status code change_percent: type: number description: >- The percentage change for the status code over time. The result is a comparison of the current interval and the same interval offset by `interval_minutes` ago. For example, when `{interval_minutes: 5, offset_minutes: 5}`, that calculates the percentage change in a 5-minute window from 5 minutes ago to a 5-minute window 10 minutes ago. pipeline_health: type: object properties: lag_metrics_status: type: string description: A success or failure message for the ability to fetch lag metrics pipelines: type: array description: >- All active and published pipelines. Inactive or unpublished pipelines are not included. items: type: object properties: id: type: string format: uuid description: The pipeline id title: type: string description: The pipeline title published_at: type: string format: date-time description: The date/time of the pipeline's last publish (if at all) sources: type: array description: Sources associated with this pipeline items: type: object properties: id: type: string description: The source id, which can also be the consumer group id type: type: string description: The type of source component title: type: string description: The title of the source component is_consumer_group: type: boolean description: >- Whether this source is a consumer group that could have lag lag_event_count: anyOf: - type: number description: >- A relative measure of how far this source is falling behind in processing. A higher value indicates the source is further behind, and a value of `0` indicates it is caught up. Use this as a directional signal, not an exact count of individual events. If this value is `null`, then it could mean the source has never had consumers, or that the lag metrics were unavailable (check `lag_metrics_status`). - type: 'null' /v3/pipeline/node: get: summary: List available pipeline nodes tags: - Pipeline Management description: A list of all available nodes for use when creating pipelines responses: '200': description: Default Response /v3/pipeline/pre-built: post: summary: Add prebuilt pipeline to account tags: - Pipeline Management description: Creates a new pipeline in the account that is prebuilt requestBody: content: application/json: schema: type: object properties: origin: type: string enum: - ui - system default: system requested_types: type: array items: type: string enum: - DEMO - OTEL - OTELLOGS minItems: 1 metadata: type: object properties: userFullName: type: string minLength: 1 maxLength: 200 baseUrl: type: string minLength: 1 maxLength: 200 required: - userFullName - baseUrl required: - requested_types - metadata required: true responses: '200': description: Default Response /v3/pipeline/{pipeline_id}/revision/{pipeline_revision_lookup}: get: summary: Fetch pipeline revision tags: - Pipeline Management description: Returns a pipeline configuration from a particular version parameters: - schema: type: string format: uuid in: path name: pipeline_id required: true - schema: oneOf: - type: string format: uuid - enum: - latest in: path name: pipeline_revision_lookup required: true responses: '200': description: Default Response /v3/pipeline/{pipeline_id}/revision: get: summary: List all pipeline revisions tags: - Pipeline Management description: Returns a list of revisions that exist in the pipeline parameters: - schema: type: integer default: 200 minimum: 1 maximum: 1000 in: query name: limit required: false description: The maximum number of revisions to return - schema: type: string format: uuid in: path name: pipeline_id required: true responses: '200': description: Default Response /v3/pipeline/{pipeline_id}/publish: post: summary: Publish pipeline tags: - Pipeline Management description: Creates a new pipeline in the account parameters: - schema: type: boolean default: false in: query name: allow_unconnected_edges required: false - schema: type: boolean default: false in: query name: allow_empty_access_keys required: false - schema: type: boolean default: false in: query name: allow_disabled required: false - schema: type: string format: uuid in: path name: pipeline_id required: true responses: '200': description: Default Response /v3/pipeline/{pipeline_id}/revert: post: summary: Revert pipeline to version tags: - Pipeline Management description: Updates a pipeline configuration to a previous version requestBody: content: application/json: schema: type: object properties: pipeline_revision_id: type: string format: uuid required: - pipeline_revision_id required: true parameters: - schema: type: boolean default: false in: query name: allow_invalid_pipeline_state required: false - schema: type: string format: uuid in: path name: pipeline_id required: true responses: '200': description: Default Response /v3/pipeline/with-components: post: summary: >- Initialize a pipeline with provided components. All sources, transforms, and sinks are linked. tags: - Pipeline Management description: Populates and publishes a new pipeline as specified. requestBody: content: application/json: schema: type: object additionalProperties: false properties: origin: type: string enum: - ui - system default: system pipeline: oneOf: - type: 'null' - type: object properties: id: type: string format: uuid title: oneOf: - type: string minLength: 1 maxLength: 512 pattern: ^[^<>"]+$ - type: 'null' errorMessage: invalid title name sources: type: array minItems: 1 items: type: object allOf: - if: properties: type: enum: - http deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: HTTP description: Receive data from incoming HTTP requests additionalProperties: false properties: capture_metadata: type: boolean title: Capture Metadata description: >- Enable the inclusion of all http headers and query string parameters that were sent from the source. ui: feature_flag: pipeline-capture-metadata display_type: hidden default: true auto_parse: type: object title: Configure automated parsing additionalProperties: false ui: display_type: hidden properties: enabled: type: boolean default: false enabled_event_types: type: array items: type: string enum: - HTTPD_COMBINEDLOG - HTTPD_COMMONLOG - HTTPD_ERRORLOG - SYSLOG5424LINE - SYSLOGLINE - SYSLOGPAMSESSION - CRONLOG - MONGO3_LOG - NAGIOSLOGLINE - POSTGRESQL - RAILS3 - RUBY_LOGGER - REDISLOG - REDISMONLOG - S3_ACCESS_LOG - ELB_ACCESS_LOG - CLOUDFRONT_ACCESS_LOG - CATALINALOG - TOMCATLOG - SQUID3 - BIND9 - HAPROXYTCP - HAPROXYHTTP - BACULA_LOGLINE - BRO_HTTP - BRO_DNS - BRO_CONN - BRO_FILES - NETSCREENSESSIONLOG - CISCO_TAGGED_SYSLOG - CISCOFW104001 - CISCOFW104002 - CISCOFW104003 - CISCOFW104004 - CISCOFW105003 - CISCOFW105004 - CISCOFW105005 - CISCOFW105008 - CISCOFW105009 - CISCOFW106001 - CISCOFW106006_106007_106010 - CISCOFW106014 - CISCOFW106015 - CISCOFW106021 - CISCOFW106023 - CISCOFW106100_2_3 - CISCOFW106100 - CISCOFW304001 - CISCOFW110002 - CISCOFW302010 - CISCOFW302013_302014_302015_302016 - CISCOFW302020_302021 - CISCOFW305011 - CISCOFW313001_313004_313008 - CISCOFW313005 - CISCOFW321001 - CISCOFW402117 - CISCOFW402119 - CISCOFW419001 - CISCOFW419002 - CISCOFW500004 - CISCOFW602303_602304 - CISCOFW710001_710002_710003_710005_710006 - CISCOFW713172 - CISCOFW733100 - SHOREWALL - SFW2 decoding: oneOf: - enum: - bytes - enum: - json - enum: - ndjson - enum: - auto ui: display_type: list errorMessage: must be equal to one of the allowed values description: The decoding method for converting frames into data events. title: Decoding Method default: auto required: - decoding - if: properties: type: enum: - splunk-hec deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Splunk HEC description: Receive Splunk logs additionalProperties: false properties: capture_metadata: type: boolean title: Capture Metadata description: >- Enable the inclusion of all http headers and query string parameters that were sent from the source. ui: feature_flag: pipeline-capture-metadata display_type: hidden default: true auto_parse: type: object title: Configure automated parsing additionalProperties: false ui: display_type: hidden properties: enabled: type: boolean default: false enabled_event_types: type: array items: type: string enum: - HTTPD_COMBINEDLOG - HTTPD_COMMONLOG - HTTPD_ERRORLOG - SYSLOG5424LINE - SYSLOGLINE - SYSLOGPAMSESSION - CRONLOG - MONGO3_LOG - NAGIOSLOGLINE - POSTGRESQL - RAILS3 - RUBY_LOGGER - REDISLOG - REDISMONLOG - S3_ACCESS_LOG - ELB_ACCESS_LOG - CLOUDFRONT_ACCESS_LOG - CATALINALOG - TOMCATLOG - SQUID3 - BIND9 - HAPROXYTCP - HAPROXYHTTP - BACULA_LOGLINE - BRO_HTTP - BRO_DNS - BRO_CONN - BRO_FILES - NETSCREENSESSIONLOG - CISCO_TAGGED_SYSLOG - CISCOFW104001 - CISCOFW104002 - CISCOFW104003 - CISCOFW104004 - CISCOFW105003 - CISCOFW105004 - CISCOFW105005 - CISCOFW105008 - CISCOFW105009 - CISCOFW106001 - CISCOFW106006_106007_106010 - CISCOFW106014 - CISCOFW106015 - CISCOFW106021 - CISCOFW106023 - CISCOFW106100_2_3 - CISCOFW106100 - CISCOFW304001 - CISCOFW110002 - CISCOFW302010 - CISCOFW302013_302014_302015_302016 - CISCOFW302020_302021 - CISCOFW305011 - CISCOFW313001_313004_313008 - CISCOFW313005 - CISCOFW321001 - CISCOFW402117 - CISCOFW402119 - CISCOFW419001 - CISCOFW419002 - CISCOFW500004 - CISCOFW602303_602304 - CISCOFW710001_710002_710003_710005_710006 - CISCOFW713172 - CISCOFW733100 - SHOREWALL - SFW2 - if: properties: type: enum: - s3 deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: AWS S3 description: Collect data from object contents in AWS S3 additionalProperties: false properties: sqs_queue_url: type: string title: SQS Queue URL description: >- The URL of a AWS SQS queue configured to receive S3 bucket notifications for the desired S3 buckets format: uri errorMessage: format: Must be a valid URI auth: type: object title: AWS Authentication description: Configures AWS authentication additionalProperties: false properties: access_key_id: type: string title: Access Key ID minLength: 1 description: The AWS access key id secret_access_key: type: string title: Secret Access Key minLength: 1 description: The AWS secret access key ui: display_type: secret required: - access_key_id - secret_access_key compression: type: string title: Compression description: The compression format of the S3 objects enum: - auto - gzip - none - zstd default: auto region: type: string title: Region minLength: 1 description: The name of the source AWS region. required: - sqs_queue_url - auth - compression - region - if: properties: type: enum: - sqs deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: AWS SQS description: Collect messages from AWS SQS additionalProperties: false properties: queue_url: type: string title: Queue URL description: The URL of an AWS SQS queue format: uri errorMessage: format: Must be a valid URI maxLength: 128 auth: type: object title: AWS Authentication description: Configures AWS authentication additionalProperties: false properties: access_key_id: type: string title: Access Key ID minLength: 1 description: The AWS access key id maxLength: 128 secret_access_key: type: string title: Secret Access Key minLength: 1 description: The AWS secret access key ui: display_type: secret maxLength: 128 required: - access_key_id - secret_access_key region: type: string title: Region minLength: 1 description: The name of the source AWS region. maxLength: 128 required: - queue_url - auth - region - if: properties: type: enum: - demo-logs deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: title: Demo Logs description: Generate log messages for testing and demonstration. additionalProperties: type: array title: Log Lines description: >- A list of user-provided log lines that will be sent through the pipeline infinitely. A maximum of 25 lines is allowed. The lines can be either plain text or ndjson. If ndjson is detected, the encoding for the source will be set to JSON, and lines that are not valid will show up as a visible component error. Otherwise, text encoding will be used. minItems: 1 maxItems: 25 items: type: string minLength: 1 maxLength: 4096 title: Line description: >- A single line of log data, either stringified JSON or plain text. ui: display_type: user-provided-lines allOf: - if: properties: format: enum: - user_provided required: - format then: properties: lines: type: array title: Log Lines description: >- A list of user-provided log lines that will be sent through the pipeline infinitely. A maximum of 25 lines is allowed. The lines can be either plain text or ndjson. If ndjson is detected, the encoding for the source will be set to JSON, and lines that are not valid will show up as a visible component error. Otherwise, text encoding will be used. minItems: 1 maxItems: 25 items: type: string minLength: 1 maxLength: 4096 title: Line description: >- A single line of log data, either stringified JSON or plain text. ui: display_type: user-provided-lines required: - lines - allOf: - errorMessage: '`lines` is only valid when `format` is `user_provided`' if: not: type: object properties: format: enum: - user_provided required: - format then: not: type: object properties: lines: type: array title: Log Lines description: >- A list of user-provided log lines that will be sent through the pipeline infinitely. A maximum of 25 lines is allowed. The lines can be either plain text or ndjson. If ndjson is detected, the encoding for the source will be set to JSON, and lines that are not valid will show up as a visible component error. Otherwise, text encoding will be used. minItems: 1 maxItems: 25 items: type: string minLength: 1 maxLength: 4096 title: Line description: >- A single line of log data, either stringified JSON or plain text. ui: display_type: user-provided-lines required: - lines properties: interval_ms: type: number title: Interval description: >- The amount of time to pause between each demo log. Fractions of a second are accepted to increase throughput. e.g. 500 ms == 120 lines per minute default: 1000 minimum: 500 maximum: 3600000 ui: display_type: milliseconds-to-human-readable format: title: Format description: The format of the log messages. All of the data is generated. ui: display_type: list errorMessage: must be one of the available types of demo data oneOf: - label: HTTP JSON description: HTTP request data in JSON format. enum: - json - label: Sensor JSON description: Data from environment sensors in JSON format. enum: - env_sensor - label: Financial JSON description: Financial data in JSON format. enum: - financial - label: Nginx Requests description: Nginx logs in text format. enum: - nginx - label: Apache Requests description: Apache request logs in text format. enum: - apache_common - label: Apache Errors description: Apache error logs in text format. enum: - apache_error - label: BSD Syslog description: Mock BSD syslog entries in rfc3164 text format. enum: - bsd_syslog - label: Syslog description: Mock syslog entries in rfc5424 text format. enum: - syslog - label: HTTP Metrics description: HTTP-based metrics in the Mezmo metrics JSON format. enum: - http_metrics - label: Generic Metrics description: Generic metrics in the Mezmo metrics JSON format enum: - generic_metrics - label: User-Provided Logs description: >- Custom logs provided by the user. These can either be plain text or serialized JSON. enum: - user_provided - label: Infrastructure Logs description: >- Infrastructure logs consisting of sysdig, nginx, HDFS and spark logs enum: - infrastructure - label: Kubernetes Logs description: Kubernetes logs in JSON format enum: - kubernetes default: json required: - format - if: properties: type: enum: - kinesis-firehose deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Kinesis Firehose description: Receive Kinesis Firehose data additionalProperties: false properties: capture_metadata: type: boolean title: Capture Metadata description: >- Enable the inclusion of all http headers and query string parameters that were sent from the source. ui: feature_flag: pipeline-capture-metadata display_type: hidden default: true auto_parse: type: object title: Configure automated parsing additionalProperties: false ui: display_type: hidden properties: enabled: type: boolean default: false enabled_event_types: type: array items: type: string enum: - HTTPD_COMBINEDLOG - HTTPD_COMMONLOG - HTTPD_ERRORLOG - SYSLOG5424LINE - SYSLOGLINE - SYSLOGPAMSESSION - CRONLOG - MONGO3_LOG - NAGIOSLOGLINE - POSTGRESQL - RAILS3 - RUBY_LOGGER - REDISLOG - REDISMONLOG - S3_ACCESS_LOG - ELB_ACCESS_LOG - CLOUDFRONT_ACCESS_LOG - CATALINALOG - TOMCATLOG - SQUID3 - BIND9 - HAPROXYTCP - HAPROXYHTTP - BACULA_LOGLINE - BRO_HTTP - BRO_DNS - BRO_CONN - BRO_FILES - NETSCREENSESSIONLOG - CISCO_TAGGED_SYSLOG - CISCOFW104001 - CISCOFW104002 - CISCOFW104003 - CISCOFW104004 - CISCOFW105003 - CISCOFW105004 - CISCOFW105005 - CISCOFW105008 - CISCOFW105009 - CISCOFW106001 - CISCOFW106006_106007_106010 - CISCOFW106014 - CISCOFW106015 - CISCOFW106021 - CISCOFW106023 - CISCOFW106100_2_3 - CISCOFW106100 - CISCOFW304001 - CISCOFW110002 - CISCOFW302010 - CISCOFW302013_302014_302015_302016 - CISCOFW302020_302021 - CISCOFW305011 - CISCOFW313001_313004_313008 - CISCOFW313005 - CISCOFW321001 - CISCOFW402117 - CISCOFW402119 - CISCOFW419001 - CISCOFW419002 - CISCOFW500004 - CISCOFW602303_602304 - CISCOFW710001_710002_710003_710005_710006 - CISCOFW713172 - CISCOFW733100 - SHOREWALL - SFW2 format: type: string enum: - json - text default: json title: Data Format description: >- This specifies what the data format will be after it is base64 decoded. If it is JSON, it will be automatically parsed. required: - format - if: properties: type: enum: - fluent deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Fluent (Fluentd, Fluent Bit) description: Receive data from Fluentd or Fluent Bit additionalProperties: false properties: capture_metadata: type: boolean title: Capture Metadata description: >- Enable the inclusion of all http headers and query string parameters that were sent from the source. ui: feature_flag: pipeline-capture-metadata display_type: hidden default: true auto_parse: type: object title: Configure automated parsing additionalProperties: false ui: display_type: hidden properties: enabled: type: boolean default: false enabled_event_types: type: array items: type: string enum: - HTTPD_COMBINEDLOG - HTTPD_COMMONLOG - HTTPD_ERRORLOG - SYSLOG5424LINE - SYSLOGLINE - SYSLOGPAMSESSION - CRONLOG - MONGO3_LOG - NAGIOSLOGLINE - POSTGRESQL - RAILS3 - RUBY_LOGGER - REDISLOG - REDISMONLOG - S3_ACCESS_LOG - ELB_ACCESS_LOG - CLOUDFRONT_ACCESS_LOG - CATALINALOG - TOMCATLOG - SQUID3 - BIND9 - HAPROXYTCP - HAPROXYHTTP - BACULA_LOGLINE - BRO_HTTP - BRO_DNS - BRO_CONN - BRO_FILES - NETSCREENSESSIONLOG - CISCO_TAGGED_SYSLOG - CISCOFW104001 - CISCOFW104002 - CISCOFW104003 - CISCOFW104004 - CISCOFW105003 - CISCOFW105004 - CISCOFW105005 - CISCOFW105008 - CISCOFW105009 - CISCOFW106001 - CISCOFW106006_106007_106010 - CISCOFW106014 - CISCOFW106015 - CISCOFW106021 - CISCOFW106023 - CISCOFW106100_2_3 - CISCOFW106100 - CISCOFW304001 - CISCOFW110002 - CISCOFW302010 - CISCOFW302013_302014_302015_302016 - CISCOFW302020_302021 - CISCOFW305011 - CISCOFW313001_313004_313008 - CISCOFW313005 - CISCOFW321001 - CISCOFW402117 - CISCOFW402119 - CISCOFW419001 - CISCOFW419002 - CISCOFW500004 - CISCOFW602303_602304 - CISCOFW710001_710002_710003_710005_710006 - CISCOFW713172 - CISCOFW733100 - SHOREWALL - SFW2 decoding: type: string description: The decoding method for converting frames into data events. enum: - bytes - json - ndjson title: Decoding Method default: json required: - decoding - if: properties: type: enum: - log-analysis deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Mezmo Log Analysis description: Receive data directly from your Mezmo Log Analysis account additionalProperties: false - if: properties: type: enum: - log-analysis-ingestion deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Mezmo Log Analysis Ingestion description: >- Receive unparsed source data directly from your Mezmo Log Analysis account. additionalProperties: type: string - if: properties: type: enum: - logstash deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Logstash description: Receive Logstash data additionalProperties: false properties: capture_metadata: type: boolean title: Capture Metadata description: >- Enable the inclusion of all http headers and query string parameters that were sent from the source. ui: feature_flag: pipeline-capture-metadata display_type: hidden default: true auto_parse: type: object title: Configure automated parsing additionalProperties: false ui: display_type: hidden properties: enabled: type: boolean default: false enabled_event_types: type: array items: type: string enum: - HTTPD_COMBINEDLOG - HTTPD_COMMONLOG - HTTPD_ERRORLOG - SYSLOG5424LINE - SYSLOGLINE - SYSLOGPAMSESSION - CRONLOG - MONGO3_LOG - NAGIOSLOGLINE - POSTGRESQL - RAILS3 - RUBY_LOGGER - REDISLOG - REDISMONLOG - S3_ACCESS_LOG - ELB_ACCESS_LOG - CLOUDFRONT_ACCESS_LOG - CATALINALOG - TOMCATLOG - SQUID3 - BIND9 - HAPROXYTCP - HAPROXYHTTP - BACULA_LOGLINE - BRO_HTTP - BRO_DNS - BRO_CONN - BRO_FILES - NETSCREENSESSIONLOG - CISCO_TAGGED_SYSLOG - CISCOFW104001 - CISCOFW104002 - CISCOFW104003 - CISCOFW104004 - CISCOFW105003 - CISCOFW105004 - CISCOFW105005 - CISCOFW105008 - CISCOFW105009 - CISCOFW106001 - CISCOFW106006_106007_106010 - CISCOFW106014 - CISCOFW106015 - CISCOFW106021 - CISCOFW106023 - CISCOFW106100_2_3 - CISCOFW106100 - CISCOFW304001 - CISCOFW110002 - CISCOFW302010 - CISCOFW302013_302014_302015_302016 - CISCOFW302020_302021 - CISCOFW305011 - CISCOFW313001_313004_313008 - CISCOFW313005 - CISCOFW321001 - CISCOFW402117 - CISCOFW402119 - CISCOFW419001 - CISCOFW419002 - CISCOFW500004 - CISCOFW602303_602304 - CISCOFW710001_710002_710003_710005_710006 - CISCOFW713172 - CISCOFW733100 - SHOREWALL - SFW2 format: type: string enum: - json - text default: json title: Data Format description: The format of the logstash data required: - format - if: properties: type: enum: - mezmo-agent deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Mezmo Agent description: Receive logs from the Mezmo Agent additionalProperties: false properties: capture_metadata: type: boolean title: Capture Metadata description: >- Enable the inclusion of all http headers and query string parameters that were sent from the source. ui: feature_flag: pipeline-capture-metadata display_type: hidden default: true auto_parse: type: object title: Configure automated parsing additionalProperties: false ui: display_type: hidden properties: enabled: type: boolean default: false enabled_event_types: type: array items: type: string enum: - HTTPD_COMBINEDLOG - HTTPD_COMMONLOG - HTTPD_ERRORLOG - SYSLOG5424LINE - SYSLOGLINE - SYSLOGPAMSESSION - CRONLOG - MONGO3_LOG - NAGIOSLOGLINE - POSTGRESQL - RAILS3 - RUBY_LOGGER - REDISLOG - REDISMONLOG - S3_ACCESS_LOG - ELB_ACCESS_LOG - CLOUDFRONT_ACCESS_LOG - CATALINALOG - TOMCATLOG - SQUID3 - BIND9 - HAPROXYTCP - HAPROXYHTTP - BACULA_LOGLINE - BRO_HTTP - BRO_DNS - BRO_CONN - BRO_FILES - NETSCREENSESSIONLOG - CISCO_TAGGED_SYSLOG - CISCOFW104001 - CISCOFW104002 - CISCOFW104003 - CISCOFW104004 - CISCOFW105003 - CISCOFW105004 - CISCOFW105005 - CISCOFW105008 - CISCOFW105009 - CISCOFW106001 - CISCOFW106006_106007_106010 - CISCOFW106014 - CISCOFW106015 - CISCOFW106021 - CISCOFW106023 - CISCOFW106100_2_3 - CISCOFW106100 - CISCOFW304001 - CISCOFW110002 - CISCOFW302010 - CISCOFW302013_302014_302015_302016 - CISCOFW302020_302021 - CISCOFW305011 - CISCOFW313001_313004_313008 - CISCOFW313005 - CISCOFW321001 - CISCOFW402117 - CISCOFW402119 - CISCOFW419001 - CISCOFW419002 - CISCOFW500004 - CISCOFW602303_602304 - CISCOFW710001_710002_710003_710005_710006 - CISCOFW713172 - CISCOFW733100 - SHOREWALL - SFW2 - if: properties: type: enum: - mezmo-datadog-source deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Datadog Agent description: Send logs and metrics data directly from an installed datadog agent additionalProperties: false properties: capture_metadata: type: boolean title: Capture Metadata description: >- Enable the inclusion of all http headers and query string parameters that were sent from the source. ui: feature_flag: pipeline-capture-metadata display_type: hidden default: true auto_parse: type: object title: Configure automated parsing additionalProperties: false ui: display_type: hidden properties: enabled: type: boolean default: false enabled_event_types: type: array items: type: string enum: - HTTPD_COMBINEDLOG - HTTPD_COMMONLOG - HTTPD_ERRORLOG - SYSLOG5424LINE - SYSLOGLINE - SYSLOGPAMSESSION - CRONLOG - MONGO3_LOG - NAGIOSLOGLINE - POSTGRESQL - RAILS3 - RUBY_LOGGER - REDISLOG - REDISMONLOG - S3_ACCESS_LOG - ELB_ACCESS_LOG - CLOUDFRONT_ACCESS_LOG - CATALINALOG - TOMCATLOG - SQUID3 - BIND9 - HAPROXYTCP - HAPROXYHTTP - BACULA_LOGLINE - BRO_HTTP - BRO_DNS - BRO_CONN - BRO_FILES - NETSCREENSESSIONLOG - CISCO_TAGGED_SYSLOG - CISCOFW104001 - CISCOFW104002 - CISCOFW104003 - CISCOFW104004 - CISCOFW105003 - CISCOFW105004 - CISCOFW105005 - CISCOFW105008 - CISCOFW105009 - CISCOFW106001 - CISCOFW106006_106007_106010 - CISCOFW106014 - CISCOFW106015 - CISCOFW106021 - CISCOFW106023 - CISCOFW106100_2_3 - CISCOFW106100 - CISCOFW304001 - CISCOFW110002 - CISCOFW302010 - CISCOFW302013_302014_302015_302016 - CISCOFW302020_302021 - CISCOFW305011 - CISCOFW313001_313004_313008 - CISCOFW313005 - CISCOFW321001 - CISCOFW402117 - CISCOFW402119 - CISCOFW419001 - CISCOFW419002 - CISCOFW500004 - CISCOFW602303_602304 - CISCOFW710001_710002_710003_710005_710006 - CISCOFW713172 - CISCOFW733100 - SHOREWALL - SFW2 - if: properties: type: enum: - pipeline-data-restoration deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Mezmo Pipeline Data Restoration description: Receive data directly from your Mezmo Pipline Data Restoration additionalProperties: false - if: properties: type: enum: - webhook deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Webhook (WebSub) description: Receive data from incoming webhooks using the WebSub protocol. additionalProperties: false properties: capture_metadata: type: boolean title: Capture Metadata description: >- Enable the inclusion of all http headers and query string parameters that were sent from the source. ui: feature_flag: pipeline-capture-metadata display_type: hidden default: true auto_parse: type: object title: Configure automated parsing additionalProperties: false ui: display_type: hidden properties: enabled: type: boolean default: false enabled_event_types: type: array items: type: string enum: - HTTPD_COMBINEDLOG - HTTPD_COMMONLOG - HTTPD_ERRORLOG - SYSLOG5424LINE - SYSLOGLINE - SYSLOGPAMSESSION - CRONLOG - MONGO3_LOG - NAGIOSLOGLINE - POSTGRESQL - RAILS3 - RUBY_LOGGER - REDISLOG - REDISMONLOG - S3_ACCESS_LOG - ELB_ACCESS_LOG - CLOUDFRONT_ACCESS_LOG - CATALINALOG - TOMCATLOG - SQUID3 - BIND9 - HAPROXYTCP - HAPROXYHTTP - BACULA_LOGLINE - BRO_HTTP - BRO_DNS - BRO_CONN - BRO_FILES - NETSCREENSESSIONLOG - CISCO_TAGGED_SYSLOG - CISCOFW104001 - CISCOFW104002 - CISCOFW104003 - CISCOFW104004 - CISCOFW105003 - CISCOFW105004 - CISCOFW105005 - CISCOFW105008 - CISCOFW105009 - CISCOFW106001 - CISCOFW106006_106007_106010 - CISCOFW106014 - CISCOFW106015 - CISCOFW106021 - CISCOFW106023 - CISCOFW106100_2_3 - CISCOFW106100 - CISCOFW304001 - CISCOFW110002 - CISCOFW302010 - CISCOFW302013_302014_302015_302016 - CISCOFW302020_302021 - CISCOFW305011 - CISCOFW313001_313004_313008 - CISCOFW313005 - CISCOFW321001 - CISCOFW402117 - CISCOFW402119 - CISCOFW419001 - CISCOFW419002 - CISCOFW500004 - CISCOFW602303_602304 - CISCOFW710001_710002_710003_710005_710006 - CISCOFW713172 - CISCOFW733100 - SHOREWALL - SFW2 - if: properties: type: enum: - kafka deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Kafka Source description: Collect events from Kafka compatible brokers additionalProperties: type: string properties: brokers: type: array title: Brokers uniqueItems: true minItems: 1 description: >- An array of key/val objects for each host/port of a Kafka broker. items: type: object title: Host/Port description: The host and port for the Kafka broker. additionalProperties: false properties: host: title: Host description: The hostname or IP address of the broker. anyOf: - type: string format: ipv4 errorMessage: format: Must be a valid IPv4 address - type: string format: hostname maxLength: 512 errorMessage: format: Must be a valid hostname port: type: number minimum: 1 maximum: 65535 title: Port description: The port that the broker listens on. required: - host - port topics: type: array title: Topics items: type: string title: Topic minLength: 1 maxLength: 256 description: The Kafka topic name. minItems: 1 description: List of Kafka topics names to read events from. group_id: type: string title: Consumer Group minLength: 1 description: The consumer group name to be used to consume events from Kafka. tls_enabled: type: boolean title: TLS enabled description: Whether or not to require TLS for outgoing connections. default: true sasl_enabled: type: boolean title: SASL/SCRAM Enabled description: Enable SASL/SCRAM authentication with Kafka default: true decoding_codec: type: string enum: - bytes - json title: Decoding description: Configures how events are decoded from raw bytes default: bytes if: properties: sasl_enabled: enum: - true then: properties: sasl_mechanism: type: string enum: - SCRAM-SHA-512 - SCRAM-SHA-256 - PLAIN title: SASL description: The Kafka SASL/SCRAM mechanism default: PLAIN sasl_username: type: string title: Username description: The SASL/SCRAM username minLength: 1 sasl_password: type: string title: Password description: The SASL/SCRAM username ui: display_type: secret minLength: 1 required: - sasl_mechanism - sasl_username - sasl_password required: - brokers - topics - group_id - decoding_codec - if: properties: type: enum: - azure-event-hub deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Azure Event Hub Source description: Collect events from Azure Event Hub using Azure Kafka Endpoint additionalProperties: false properties: connection_string: type: string title: Connection String minLength: 1 maxLength: 512 description: The Connection String as it appears in hub consumer SAS Policy ui: display_type: secret namespace: type: string title: Namespace minLength: 1 maxLength: 256 description: The Event Hub Namespace topics: type: array title: Event Hub Names items: type: string title: Name minLength: 1 maxLength: 256 description: Azure Event Hub instance name. minItems: 1 description: The list of Azure Event Hub name(s) to read events from. group_id: type: string title: Consumer Group minLength: 1 maxLength: 256 description: The consumer group name that this consumer belongs to. decoding_codec: type: string enum: - bytes - json title: Decoding description: Configures how events are decoded from raw bytes default: bytes required: - connection_string - namespace - topics - group_id - decoding_codec - if: properties: type: enum: - prometheus-remote-write deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Prometheus Remote Write description: Receive prometheus remote write protobuf data from HTTP requests additionalProperties: false properties: capture_metadata: type: boolean title: Capture Metadata description: >- Enable the inclusion of all http headers and query string parameters that were sent from the source. ui: feature_flag: pipeline-capture-metadata display_type: hidden default: true auto_parse: type: object title: Configure automated parsing additionalProperties: false ui: display_type: hidden properties: enabled: type: boolean default: false enabled_event_types: type: array items: type: string enum: - HTTPD_COMBINEDLOG - HTTPD_COMMONLOG - HTTPD_ERRORLOG - SYSLOG5424LINE - SYSLOGLINE - SYSLOGPAMSESSION - CRONLOG - MONGO3_LOG - NAGIOSLOGLINE - POSTGRESQL - RAILS3 - RUBY_LOGGER - REDISLOG - REDISMONLOG - S3_ACCESS_LOG - ELB_ACCESS_LOG - CLOUDFRONT_ACCESS_LOG - CATALINALOG - TOMCATLOG - SQUID3 - BIND9 - HAPROXYTCP - HAPROXYHTTP - BACULA_LOGLINE - BRO_HTTP - BRO_DNS - BRO_CONN - BRO_FILES - NETSCREENSESSIONLOG - CISCO_TAGGED_SYSLOG - CISCOFW104001 - CISCOFW104002 - CISCOFW104003 - CISCOFW104004 - CISCOFW105003 - CISCOFW105004 - CISCOFW105005 - CISCOFW105008 - CISCOFW105009 - CISCOFW106001 - CISCOFW106006_106007_106010 - CISCOFW106014 - CISCOFW106015 - CISCOFW106021 - CISCOFW106023 - CISCOFW106100_2_3 - CISCOFW106100 - CISCOFW304001 - CISCOFW110002 - CISCOFW302010 - CISCOFW302013_302014_302015_302016 - CISCOFW302020_302021 - CISCOFW305011 - CISCOFW313001_313004_313008 - CISCOFW313005 - CISCOFW321001 - CISCOFW402117 - CISCOFW402119 - CISCOFW419001 - CISCOFW419002 - CISCOFW500004 - CISCOFW602303_602304 - CISCOFW710001_710002_710003_710005_710006 - CISCOFW713172 - CISCOFW733100 - SHOREWALL - SFW2 - if: properties: type: enum: - open-telemetry-metrics deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: OpenTelemetry Metrics description: Receive metrics from OpenTelemetry over HTTP additionalProperties: false properties: capture_metadata: type: boolean title: Capture Metadata description: >- Enable the inclusion of all http headers and query string parameters that were sent from the source. ui: feature_flag: pipeline-capture-metadata display_type: hidden default: true auto_parse: type: object title: Configure automated parsing additionalProperties: false ui: display_type: hidden properties: enabled: type: boolean default: false enabled_event_types: type: array items: type: string enum: - HTTPD_COMBINEDLOG - HTTPD_COMMONLOG - HTTPD_ERRORLOG - SYSLOG5424LINE - SYSLOGLINE - SYSLOGPAMSESSION - CRONLOG - MONGO3_LOG - NAGIOSLOGLINE - POSTGRESQL - RAILS3 - RUBY_LOGGER - REDISLOG - REDISMONLOG - S3_ACCESS_LOG - ELB_ACCESS_LOG - CLOUDFRONT_ACCESS_LOG - CATALINALOG - TOMCATLOG - SQUID3 - BIND9 - HAPROXYTCP - HAPROXYHTTP - BACULA_LOGLINE - BRO_HTTP - BRO_DNS - BRO_CONN - BRO_FILES - NETSCREENSESSIONLOG - CISCO_TAGGED_SYSLOG - CISCOFW104001 - CISCOFW104002 - CISCOFW104003 - CISCOFW104004 - CISCOFW105003 - CISCOFW105004 - CISCOFW105005 - CISCOFW105008 - CISCOFW105009 - CISCOFW106001 - CISCOFW106006_106007_106010 - CISCOFW106014 - CISCOFW106015 - CISCOFW106021 - CISCOFW106023 - CISCOFW106100_2_3 - CISCOFW106100 - CISCOFW304001 - CISCOFW110002 - CISCOFW302010 - CISCOFW302013_302014_302015_302016 - CISCOFW302020_302021 - CISCOFW305011 - CISCOFW313001_313004_313008 - CISCOFW313005 - CISCOFW321001 - CISCOFW402117 - CISCOFW402119 - CISCOFW419001 - CISCOFW419002 - CISCOFW500004 - CISCOFW602303_602304 - CISCOFW710001_710002_710003_710005_710006 - CISCOFW713172 - CISCOFW733100 - SHOREWALL - SFW2 - if: properties: type: enum: - open-telemetry-logs deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: OpenTelemetry Logs description: Receive logs from OpenTelemetry over HTTP additionalProperties: false properties: capture_metadata: type: boolean title: Capture Metadata description: >- Enable the inclusion of all http headers and query string parameters that were sent from the source. ui: feature_flag: pipeline-capture-metadata display_type: hidden default: true auto_parse: type: object title: Configure automated parsing additionalProperties: false ui: display_type: hidden properties: enabled: type: boolean default: false enabled_event_types: type: array items: type: string enum: - HTTPD_COMBINEDLOG - HTTPD_COMMONLOG - HTTPD_ERRORLOG - SYSLOG5424LINE - SYSLOGLINE - SYSLOGPAMSESSION - CRONLOG - MONGO3_LOG - NAGIOSLOGLINE - POSTGRESQL - RAILS3 - RUBY_LOGGER - REDISLOG - REDISMONLOG - S3_ACCESS_LOG - ELB_ACCESS_LOG - CLOUDFRONT_ACCESS_LOG - CATALINALOG - TOMCATLOG - SQUID3 - BIND9 - HAPROXYTCP - HAPROXYHTTP - BACULA_LOGLINE - BRO_HTTP - BRO_DNS - BRO_CONN - BRO_FILES - NETSCREENSESSIONLOG - CISCO_TAGGED_SYSLOG - CISCOFW104001 - CISCOFW104002 - CISCOFW104003 - CISCOFW104004 - CISCOFW105003 - CISCOFW105004 - CISCOFW105005 - CISCOFW105008 - CISCOFW105009 - CISCOFW106001 - CISCOFW106006_106007_106010 - CISCOFW106014 - CISCOFW106015 - CISCOFW106021 - CISCOFW106023 - CISCOFW106100_2_3 - CISCOFW106100 - CISCOFW304001 - CISCOFW110002 - CISCOFW302010 - CISCOFW302013_302014_302015_302016 - CISCOFW302020_302021 - CISCOFW305011 - CISCOFW313001_313004_313008 - CISCOFW313005 - CISCOFW321001 - CISCOFW402117 - CISCOFW402119 - CISCOFW419001 - CISCOFW419002 - CISCOFW500004 - CISCOFW602303_602304 - CISCOFW710001_710002_710003_710005_710006 - CISCOFW713172 - CISCOFW733100 - SHOREWALL - SFW2 - if: properties: type: enum: - open-telemetry-traces deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: OpenTelemetry Traces description: Receive traces from OpenTelemetry over HTTP additionalProperties: false properties: capture_metadata: type: boolean title: Capture Metadata description: >- Enable the inclusion of all http headers and query string parameters that were sent from the source. ui: feature_flag: pipeline-capture-metadata display_type: hidden default: true auto_parse: type: object title: Configure automated parsing additionalProperties: false ui: display_type: hidden properties: enabled: type: boolean default: false enabled_event_types: type: array items: type: string enum: - HTTPD_COMBINEDLOG - HTTPD_COMMONLOG - HTTPD_ERRORLOG - SYSLOG5424LINE - SYSLOGLINE - SYSLOGPAMSESSION - CRONLOG - MONGO3_LOG - NAGIOSLOGLINE - POSTGRESQL - RAILS3 - RUBY_LOGGER - REDISLOG - REDISMONLOG - S3_ACCESS_LOG - ELB_ACCESS_LOG - CLOUDFRONT_ACCESS_LOG - CATALINALOG - TOMCATLOG - SQUID3 - BIND9 - HAPROXYTCP - HAPROXYHTTP - BACULA_LOGLINE - BRO_HTTP - BRO_DNS - BRO_CONN - BRO_FILES - NETSCREENSESSIONLOG - CISCO_TAGGED_SYSLOG - CISCOFW104001 - CISCOFW104002 - CISCOFW104003 - CISCOFW104004 - CISCOFW105003 - CISCOFW105004 - CISCOFW105005 - CISCOFW105008 - CISCOFW105009 - CISCOFW106001 - CISCOFW106006_106007_106010 - CISCOFW106014 - CISCOFW106015 - CISCOFW106021 - CISCOFW106023 - CISCOFW106100_2_3 - CISCOFW106100 - CISCOFW304001 - CISCOFW110002 - CISCOFW302010 - CISCOFW302013_302014_302015_302016 - CISCOFW302020_302021 - CISCOFW305011 - CISCOFW313001_313004_313008 - CISCOFW313005 - CISCOFW321001 - CISCOFW402117 - CISCOFW402119 - CISCOFW419001 - CISCOFW419002 - CISCOFW500004 - CISCOFW602303_602304 - CISCOFW710001_710002_710003_710005_710006 - CISCOFW713172 - CISCOFW733100 - SHOREWALL - SFW2 properties: title: oneOf: - type: string minLength: 1 maxLength: 512 pattern: ^[^<>"]+$ - type: 'null' errorMessage: invalid title name description: oneOf: - type: string minLength: 1 pattern: ^[^<>"]+$ - type: 'null' errorMessage: invalid description type: type: string enum: - http - splunk-hec - s3 - sqs - demo-logs - kinesis-firehose - fluent - log-analysis - log-analysis-ingestion - logstash - mezmo-agent - mezmo-datadog-source - pipeline-data-restoration - webhook - kafka - azure-event-hub - prometheus-remote-write - open-telemetry-metrics - open-telemetry-logs - open-telemetry-traces user_config: type: object required: - user_config transforms: type: array minItems: 1 items: type: object allOf: - if: properties: type: enum: - filter deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Filter description: Define condition(s) to include or exclude events from the pipeline additionalProperties: false properties: action: title: Action description: How to handle events matching this criteria oneOf: - label: Allow events matching this criteria enum: - allow - label: Drop events matching this criteria enum: - drop default: allow ui: display_type: radio-group conditional: allOf: - $ref: '#condition-group' title: Conditional description: A group of expressions joined together by an and/or operator. ui: display_type: rules-engine required: - action - conditional - if: properties: type: enum: - sample deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Sample description: >- Sample data at a given rate, retaining only a subset of data events for further processing. additionalProperties: false properties: rate: type: integer title: Rate description: >- The rate at which events will be forwarded, expressed as 1/N. For example, `rate = 10` means 1 out of every 10 events will be forwarded and the rest will be dropped minimum: 2 maximum: 10000 default: 10 always_include: title: Always Include description: >- Certain events may always want to be included (not sampled), regardless of the sample rate. Events matching this criteria will always show up in the results. additionalProperties: type: - string - number dependencies: field: - str_operator str_operator: - field allOf: - if: title: Operator properties: str_operator: type: string enum: - greater - greater_or_equal - less - less_or_equal ui: display_type: list-operators title: Operator description: The comparison operator required: - str_operator then: title: Value properties: value: errorMessage: pattern: Value must be a number type: - string - number title: Value description: The value used in the expression, ie the "operand" minLength: 1 pattern: ^\s*-?(\d*\.)?\d+\s*$ required: - value - if: title: Operator properties: str_operator: type: string enum: - greater - greater_or_equal - less - less_or_equal - equal - contains - is_ip_in_cidr_range - ends_with - starts_with - drain_matches - contains_token_term - contains_token_prefix - percent_change_greater - percent_change_greater_or_equal - percent_change_less - percent_change_less_or_equal - value_change_greater - value_change_greater_or_equal - value_change_less - value_change_less_or_equal ui: display_type: list-operators title: Operator description: The comparison operator required: - str_operator then: title: Value properties: value: type: - string - number title: Value description: The value used in the expression, ie the "operand" minLength: 1 required: - value - if: title: Operator properties: str_operator: type: string enum: - contains - equal - ends_with - starts_with - drain_matches - contains_token_term - contains_token_prefix ui: display_type: list-operators title: Operator required: - str_operator then: title: Case sensitive properties: case_sensitive: type: boolean title: Case sensitive description: Perform case sensitive comparison? default: true properties: field: type: string title: Field description: >- The JSON field to use in a condition to always include in sampling pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference str_operator: type: string enum: - greater - greater_or_equal - less - less_or_equal - equal - contains - is_ip_in_cidr_range - is_metric - is_array - is_boolean - is_empty - is_null - is_number - is_object - is_string - exists - ends_with - starts_with - contains_token_term - contains_token_prefix - regex_match ui: display_type: list-operators title: Operator description: The comparison operator negate: type: boolean default: false title: Negate description: Negate the comparison expression required: - rate - if: properties: type: enum: - drop-fields deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Remove Fields description: Remove fields additionalProperties: false properties: fields: type: array title: Fields description: A list of fields to be removed items: type: string title: Field pattern: >- ^(?:metadata|\.|(?:metadata|message)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ minLength: 1 errorMessage: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields). Note that dropping an event's message or timestamp is not allowed minItems: 1 required: - fields - if: properties: type: enum: - parse deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: title: Parse description: Parse a specified field using the chosen parser additionalProperties: type: object errorMessage: properties: target_field: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) allOf: - if: properties: parser: enum: - parse_key_value then: properties: options: $ref: '#key-value-parser-options' - if: properties: parser: enum: - parse_timestamp then: properties: options: $ref: '#timestamp-parser-options' required: - options - if: properties: parser: enum: - parse_grok then: properties: options: $ref: '#grok-pattern-parser-options' required: - options - if: properties: parser: enum: - parse_regex then: properties: options: allOf: - $ref: '#regex-parser-options' ui: display_type: regex-pattern required: - options - if: properties: parser: enum: - parse_apache_log then: properties: options: $ref: '#apache-parser-options' required: - options - if: properties: parser: enum: - parse_nginx_log then: properties: options: $ref: '#nginx-parser-options' required: - options - if: properties: parser: enum: - parse_cef then: properties: options: $ref: '#cef-parser-options' - if: properties: parser: enum: - parse_csv then: properties: options: $ref: '#csv-parser-options' properties: field: type: string title: Field description: The JSON field whose value should be parsed. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ default: . errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference target_field: oneOf: - type: 'null' - type: string pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ maxLength: 512 errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference title: Target field description: >- The field into which the parsed value should be inserted. Leave blank to insert the parsed data into the original field. default: null parser: oneOf: - label: AWS Cloudfront Access Log group: Line Parsers ui: {} enum: - aws_cloudfront_access_log - label: AWS Cloudwatch Log group: Line Parsers ui: {} enum: - parse_aws_cloudwatch_log_subscription_message - label: AWS Load Balancer Log group: Line Parsers ui: {} enum: - parse_aws_alb_log - label: AWS S3 Access Log group: Line Parsers ui: {} enum: - aws_s3_access_log - label: Apache Log group: Line Parsers ui: {} enum: - parse_apache_log - label: CSV group: Line Parsers ui: {} enum: - parse_csv - label: Common Event Format (CEF) Log group: Line Parsers ui: {} enum: - parse_cef - label: Common Log group: Line Parsers ui: {} enum: - parse_common_log - label: Elasticsearch Log group: Line Parsers ui: {} enum: - elasticsearch_log - label: Golang Log group: Line Parsers ui: {} enum: - golang_log - label: Google Log group: Line Parsers ui: {} enum: - parse_glog - label: Grok Pattern group: Line Parsers ui: {} enum: - parse_grok - label: Integer group: Field Parsers ui: {} enum: - parse_int - label: JSON group: Line Parsers ui: {} enum: - parse_json - label: Key/Value group: Line Parsers ui: {} enum: - parse_key_value - label: Kubernetes Log group: Line Parsers ui: {} enum: - parse_klog - label: Linux Authorization group: Line Parsers ui: {} enum: - parse_linux_authorization - label: Mongo Log group: Line Parsers ui: {} enum: - mongo_log - label: Mongo3 Log group: Line Parsers ui: {} enum: - mongo3_log - label: Nginx Log group: Line Parsers ui: {} enum: - parse_nginx_log - label: PSQL Log group: Line Parsers ui: {} enum: - psql_log - label: Query String group: Field Parsers ui: {} enum: - parse_query_string - label: Regular Expression group: Line Parsers ui: feature_flag: pipeline-parser-regex enum: - parse_regex - label: Syslog group: Line Parsers ui: {} enum: - parse_syslog - label: Timestamp group: Field Parsers ui: {} enum: - parse_timestamp - label: Tokens group: Field Parsers ui: {} enum: - parse_tokens - label: URL group: Field Parsers ui: {} enum: - parse_url - label: User Agent group: Field Parsers ui: {} enum: - parse_user_agent - label: VPC Flow Log group: Line Parsers ui: {} description: Parse VPC logs using default format for fields from v2 to v4 enum: - parse_aws_vpc_flow_log ui: display_type: list errorMessage: must be one of allowed parsers title: Parser description: The kind of parser to use against the input value from "field". required: - field - parser - if: properties: type: enum: - parse-sequentially deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Parse Sequentially description: >- Parse fields using one or more parsers in order, sending out the event on the first successful match. additionalProperties: false errorMessage: properties: target_field: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) properties: field: type: string title: Field description: The field whose value should be parsed. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ default: . maxLength: 512 errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference target_field: oneOf: - type: 'null' - type: string pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ maxLength: 512 errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference title: Target field description: >- The field into which the parsed value should be inserted. Leave blank to insert the parsed data into the original field. default: null parsers: type: array title: List of parsers description: >- The list of parsers to use in order against the input valuefrom "field", short-circuiting on the first successful match. items: type: object allOf: - if: properties: parser: enum: - parse_key_value then: properties: options: $ref: '#key-value-parser-options' - if: properties: parser: enum: - parse_timestamp then: properties: options: $ref: '#timestamp-parser-options' required: - options - if: properties: parser: enum: - parse_grok then: properties: options: $ref: '#grok-pattern-parser-options' required: - options - if: properties: parser: enum: - parse_regex then: properties: options: allOf: - $ref: '#regex-parser-options' ui: display_type: regex-pattern required: - options - if: properties: parser: enum: - parse_apache_log then: properties: options: $ref: '#apache-parser-options' required: - options - if: properties: parser: enum: - parse_nginx_log then: properties: options: $ref: '#nginx-parser-options' required: - options - if: properties: parser: enum: - parse_cef then: properties: options: $ref: '#cef-parser-options' - if: properties: parser: enum: - parse_csv then: properties: options: $ref: '#csv-parser-options' properties: label: oneOf: - type: 'null' - type: string maxLength: 20 title: Title description: >- An arbitrary name you choose to identify the results of this processor for use when connecting them to other components. parser: oneOf: - label: AWS Cloudfront Access Log group: Line Parsers ui: {} enum: - aws_cloudfront_access_log - label: AWS Cloudwatch Log group: Line Parsers ui: {} enum: - parse_aws_cloudwatch_log_subscription_message - label: AWS Load Balancer Log group: Line Parsers ui: {} enum: - parse_aws_alb_log - label: AWS S3 Access Log group: Line Parsers ui: {} enum: - aws_s3_access_log - label: Apache Log group: Line Parsers ui: {} enum: - parse_apache_log - label: CSV group: Line Parsers ui: {} enum: - parse_csv - label: Common Event Format (CEF) Log group: Line Parsers ui: {} enum: - parse_cef - label: Common Log group: Line Parsers ui: {} enum: - parse_common_log - label: Elasticsearch Log group: Line Parsers ui: {} enum: - elasticsearch_log - label: Golang Log group: Line Parsers ui: {} enum: - golang_log - label: Google Log group: Line Parsers ui: {} enum: - parse_glog - label: Grok Pattern group: Line Parsers ui: {} enum: - parse_grok - label: Integer group: Field Parsers ui: {} enum: - parse_int - label: JSON group: Line Parsers ui: {} enum: - parse_json - label: Key/Value group: Line Parsers ui: {} enum: - parse_key_value - label: Kubernetes Log group: Line Parsers ui: {} enum: - parse_klog - label: Linux Authorization group: Line Parsers ui: {} enum: - parse_linux_authorization - label: Mongo Log group: Line Parsers ui: {} enum: - mongo_log - label: Mongo3 Log group: Line Parsers ui: {} enum: - mongo3_log - label: Nginx Log group: Line Parsers ui: {} enum: - parse_nginx_log - label: PSQL Log group: Line Parsers ui: {} enum: - psql_log - label: Query String group: Field Parsers ui: {} enum: - parse_query_string - label: Regular Expression group: Line Parsers ui: feature_flag: pipeline-parser-regex enum: - parse_regex - label: Syslog group: Line Parsers ui: {} enum: - parse_syslog - label: Timestamp group: Field Parsers ui: {} enum: - parse_timestamp - label: Tokens group: Field Parsers ui: {} enum: - parse_tokens - label: URL group: Field Parsers ui: {} enum: - parse_url - label: User Agent group: Field Parsers ui: {} enum: - parse_user_agent - label: VPC Flow Log group: Line Parsers ui: {} description: >- Parse VPC logs using default format for fields from v2 to v4 enum: - parse_aws_vpc_flow_log ui: display_type: list errorMessage: must be one of allowed parsers title: Parser description: >- The kind of parser to use against the input value from "field". _output_name: type: string title: '[INTERNAL]' description: >- Internal-only field that holds a static id for the output_name. This way, labels for outputs can be updated independent of the id. default: '' required: - parser - _output_name additionalProperties: type: object minItems: 1 ui: display_type: multi-output-map unique_parser_and_options: true errorMessage: uniqueItemProperties: parser can not be defined more than once required: - field - parsers - if: properties: type: enum: - route deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Route description: >- Route data based on whether it matches (or doesn't match) logical comparisons additionalProperties: false properties: conditionals: type: array minItems: 1 title: Conditionals description: A list of complete conditionals, including expressions and label items: type: object additionalProperties: false title: Conditional description: A list of conditions and a label properties: conditional: $ref: '#condition-group' label: type: string minLength: 1 maxLength: 255 title: Label description: >- An arbitrary name you choose to identify the results of this processor for use when connecting them to other components. _output_name: type: string title: '[INTERNAL]' description: >- Internal-only field that holds a static id for the output_name. This way, labels for outputs can be updated independent of the id. default: '' required: - conditional - label - _output_name required: - conditionals - if: properties: type: enum: - stringify deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Stringify description: Stringify messages as JSON additionalProperties: false - if: properties: type: enum: - compact-fields deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Compact Fields description: Remove empty values from a list of fields additionalProperties: false properties: fields: type: array title: Fields description: A list of fields to remove empty values from items: type: string title: Field description: Field to remove empty value from pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference minItems: 1 options: type: object additionalProperties: false title: Compact Options description: Additional options for removing empty values from fields properties: compact_array: type: boolean title: Compact Array description: Remove empty arrays from a field default: true compact_object: type: boolean title: Compact Object description: Remove empty objects from a field default: true required: - fields - if: properties: type: enum: - flatten-fields deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Flatten Fields description: >- Flattens the object or array value of a field into a single level representation additionalProperties: false properties: fields: type: array title: Fields description: >- Define specific nested objects you wish to flatten. This will not flatten the entire object, just the specified fields. Leave blank to flatten the entire object items: type: string title: Field description: Field to flatten pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference options: type: object additionalProperties: false title: Flatten Options description: Additional options for flattening fields properties: delimiter: type: string default: _ title: Delimiter description: >- When flattening, the nested field labels will be appended to the higher level field names along with a delimiter to allow understanding of the original structure. The default is an underscore (_) to ensure no collisions with the . notation used for field identifiers minLength: 1 maxLength: 1 - if: properties: type: enum: - dedupe deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Dedupe description: Dedupe (remove duplicates) from the data stream additionalProperties: false properties: number_of_events: type: number description: Number of events to compare across title: Number of Events minimum: 2 maximum: 5000 default: 5000 comparison_type: type: string enum: - Ignore - Match description: >- Match: Only compares across the fields which are specified by the user.Ignore: Compares everything but the fields specified by the user. title: Comparison Type default: Match fields: type: array title: Fields description: A list of fields on which to base deduping items: type: string title: Field pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference minItems: 1 required: - number_of_events - comparison_type - fields - if: properties: type: enum: - encrypt-fields deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: title: Encrypt Field description: Encrypts the value of the provided field additionalProperties: false allOf: - if: properties: algorithm: enum: - AES-256-CFB required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CFB required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CFB required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-OFB required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-OFB required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-OFB required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-CTR required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CTR required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CTR required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-CBC-PKCS7 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CBC-PKCS7 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CBC-PKCS7 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-CBC-ANSIX923 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CBC-ANSIX923 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CBC-ANSIX923 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-CBC-ISO7816 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CBC-ISO7816 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CBC-ISO7816 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-CBC-ISO10126 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CBC-ISO10126 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CBC-ISO10126 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 16 maxLength: 16 required: - key properties: field: type: string title: Field description: >- Field to encrypt. The value of the field must be a primitive (string, number, boolean) pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference algorithm: oneOf: - label: AES-256-CFB (key = 32 characters, iv = 16 characters) description: AES-256-CFB (key = 32 characters, iv = 16 characters) enum: - AES-256-CFB - label: AES-192-CFB (key = 24 characters, iv = 16 characters) description: AES-192-CFB (key = 24 characters, iv = 16 characters) enum: - AES-192-CFB - label: AES-128-CFB (key = 16 characters, iv = 16 characters) description: AES-128-CFB (key = 16 characters, iv = 16 characters) enum: - AES-128-CFB - label: AES-256-OFB (key = 32 characters, iv = 16 characters) description: AES-256-OFB (key = 32 characters, iv = 16 characters) enum: - AES-256-OFB - label: AES-192-OFB (key = 24 characters, iv = 16 characters) description: AES-192-OFB (key = 24 characters, iv = 16 characters) enum: - AES-192-OFB - label: AES-128-OFB (key = 16 characters, iv = 16 characters) description: AES-128-OFB (key = 16 characters, iv = 16 characters) enum: - AES-128-OFB - label: AES-256-CTR (key = 32 characters, iv = 16 characters) description: AES-256-CTR (key = 32 characters, iv = 16 characters) enum: - AES-256-CTR - label: AES-192-CTR (key = 24 characters, iv = 16 characters) description: AES-192-CTR (key = 24 characters, iv = 16 characters) enum: - AES-192-CTR - label: AES-128-CTR (key = 16 characters, iv = 16 characters) description: AES-128-CTR (key = 16 characters, iv = 16 characters) enum: - AES-128-CTR - label: AES-256-CBC-PKCS7 (key = 32 characters, iv = 16 characters) description: AES-256-CBC-PKCS7 (key = 32 characters, iv = 16 characters) enum: - AES-256-CBC-PKCS7 - label: AES-192-CBC-PKCS7 (key = 24 characters, iv = 16 characters) description: AES-192-CBC-PKCS7 (key = 24 characters, iv = 16 characters) enum: - AES-192-CBC-PKCS7 - label: AES-128-CBC-PKCS7 (key = 16 characters, iv = 16 characters) description: AES-128-CBC-PKCS7 (key = 16 characters, iv = 16 characters) enum: - AES-128-CBC-PKCS7 - label: >- AES-256-CBC-ANSIX923 (key = 32 characters, iv = 16 characters) description: >- AES-256-CBC-ANSIX923 (key = 32 characters, iv = 16 characters) enum: - AES-256-CBC-ANSIX923 - label: >- AES-192-CBC-ANSIX923 (key = 24 characters, iv = 16 characters) description: >- AES-192-CBC-ANSIX923 (key = 24 characters, iv = 16 characters) enum: - AES-192-CBC-ANSIX923 - label: >- AES-128-CBC-ANSIX923 (key = 16 characters, iv = 16 characters) description: >- AES-128-CBC-ANSIX923 (key = 16 characters, iv = 16 characters) enum: - AES-128-CBC-ANSIX923 - label: >- AES-256-CBC-ISO7816 (key = 32 characters, iv = 16 characters) description: >- AES-256-CBC-ISO7816 (key = 32 characters, iv = 16 characters) enum: - AES-256-CBC-ISO7816 - label: >- AES-192-CBC-ISO7816 (key = 24 characters, iv = 16 characters) description: >- AES-192-CBC-ISO7816 (key = 24 characters, iv = 16 characters) enum: - AES-192-CBC-ISO7816 - label: >- AES-128-CBC-ISO7816 (key = 16 characters, iv = 16 characters) description: >- AES-128-CBC-ISO7816 (key = 16 characters, iv = 16 characters) enum: - AES-128-CBC-ISO7816 - label: >- AES-256-CBC-ISO10126 (key = 32 characters, iv = 16 characters) description: >- AES-256-CBC-ISO10126 (key = 32 characters, iv = 16 characters) enum: - AES-256-CBC-ISO10126 - label: >- AES-192-CBC-ISO10126 (key = 24 characters, iv = 16 characters) description: >- AES-192-CBC-ISO10126 (key = 24 characters, iv = 16 characters) enum: - AES-192-CBC-ISO10126 - label: >- AES-128-CBC-ISO10126 (key = 16 characters, iv = 16 characters) description: >- AES-128-CBC-ISO10126 (key = 16 characters, iv = 16 characters) enum: - AES-128-CBC-ISO10126 ui: display_type: list title: Encryption algorithm description: The encryption algorithm to use key: type: string title: Encryption key description: The encryption key minLength: 16 maxLength: 32 iv_field: type: string title: Initialization vector (IV) field description: >- The field in which to store the generated initialization vector, IV. Each encrypted value will have a unique IV pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference encode_raw_bytes: type: boolean title: Encode encrypted field and IV as Base 64 text description: >- Encode the encrypted value and generated initialization vector as Base64 text default: true required: - field - algorithm - key - iv_field - if: properties: type: enum: - decrypt-fields deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: title: Decrypt Field additionalProperties: false description: Decrypts the value of the provided field allOf: - if: properties: algorithm: enum: - AES-256-CFB required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CFB required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CFB required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-OFB required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-OFB required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-OFB required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-CTR required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CTR required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CTR required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-CBC-PKCS7 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CBC-PKCS7 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CBC-PKCS7 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-CBC-ANSIX923 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CBC-ANSIX923 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CBC-ANSIX923 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-CBC-ISO7816 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CBC-ISO7816 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CBC-ISO7816 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-CBC-ISO10126 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CBC-ISO10126 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CBC-ISO10126 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 16 maxLength: 16 required: - key properties: field: type: string title: Field description: Field to decrypt. The value of the field must be a string pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference algorithm: oneOf: - label: AES-256-CFB (key = 32 characters, iv = 16 characters) description: AES-256-CFB (key = 32 characters, iv = 16 characters) enum: - AES-256-CFB - label: AES-192-CFB (key = 24 characters, iv = 16 characters) description: AES-192-CFB (key = 24 characters, iv = 16 characters) enum: - AES-192-CFB - label: AES-128-CFB (key = 16 characters, iv = 16 characters) description: AES-128-CFB (key = 16 characters, iv = 16 characters) enum: - AES-128-CFB - label: AES-256-OFB (key = 32 characters, iv = 16 characters) description: AES-256-OFB (key = 32 characters, iv = 16 characters) enum: - AES-256-OFB - label: AES-192-OFB (key = 24 characters, iv = 16 characters) description: AES-192-OFB (key = 24 characters, iv = 16 characters) enum: - AES-192-OFB - label: AES-128-OFB (key = 16 characters, iv = 16 characters) description: AES-128-OFB (key = 16 characters, iv = 16 characters) enum: - AES-128-OFB - label: AES-256-CTR (key = 32 characters, iv = 16 characters) description: AES-256-CTR (key = 32 characters, iv = 16 characters) enum: - AES-256-CTR - label: AES-192-CTR (key = 24 characters, iv = 16 characters) description: AES-192-CTR (key = 24 characters, iv = 16 characters) enum: - AES-192-CTR - label: AES-128-CTR (key = 16 characters, iv = 16 characters) description: AES-128-CTR (key = 16 characters, iv = 16 characters) enum: - AES-128-CTR - label: AES-256-CBC-PKCS7 (key = 32 characters, iv = 16 characters) description: AES-256-CBC-PKCS7 (key = 32 characters, iv = 16 characters) enum: - AES-256-CBC-PKCS7 - label: AES-192-CBC-PKCS7 (key = 24 characters, iv = 16 characters) description: AES-192-CBC-PKCS7 (key = 24 characters, iv = 16 characters) enum: - AES-192-CBC-PKCS7 - label: AES-128-CBC-PKCS7 (key = 16 characters, iv = 16 characters) description: AES-128-CBC-PKCS7 (key = 16 characters, iv = 16 characters) enum: - AES-128-CBC-PKCS7 - label: >- AES-256-CBC-ANSIX923 (key = 32 characters, iv = 16 characters) description: >- AES-256-CBC-ANSIX923 (key = 32 characters, iv = 16 characters) enum: - AES-256-CBC-ANSIX923 - label: >- AES-192-CBC-ANSIX923 (key = 24 characters, iv = 16 characters) description: >- AES-192-CBC-ANSIX923 (key = 24 characters, iv = 16 characters) enum: - AES-192-CBC-ANSIX923 - label: >- AES-128-CBC-ANSIX923 (key = 16 characters, iv = 16 characters) description: >- AES-128-CBC-ANSIX923 (key = 16 characters, iv = 16 characters) enum: - AES-128-CBC-ANSIX923 - label: >- AES-256-CBC-ISO7816 (key = 32 characters, iv = 16 characters) description: >- AES-256-CBC-ISO7816 (key = 32 characters, iv = 16 characters) enum: - AES-256-CBC-ISO7816 - label: >- AES-192-CBC-ISO7816 (key = 24 characters, iv = 16 characters) description: >- AES-192-CBC-ISO7816 (key = 24 characters, iv = 16 characters) enum: - AES-192-CBC-ISO7816 - label: >- AES-128-CBC-ISO7816 (key = 16 characters, iv = 16 characters) description: >- AES-128-CBC-ISO7816 (key = 16 characters, iv = 16 characters) enum: - AES-128-CBC-ISO7816 - label: >- AES-256-CBC-ISO10126 (key = 32 characters, iv = 16 characters) description: >- AES-256-CBC-ISO10126 (key = 32 characters, iv = 16 characters) enum: - AES-256-CBC-ISO10126 - label: >- AES-192-CBC-ISO10126 (key = 24 characters, iv = 16 characters) description: >- AES-192-CBC-ISO10126 (key = 24 characters, iv = 16 characters) enum: - AES-192-CBC-ISO10126 - label: >- AES-128-CBC-ISO10126 (key = 16 characters, iv = 16 characters) description: >- AES-128-CBC-ISO10126 (key = 16 characters, iv = 16 characters) enum: - AES-128-CBC-ISO10126 ui: display_type: list title: Decryption algorithm description: The algorithm with which the data was encrypted key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 16 maxLength: 32 iv_field: type: string title: Initialization vector (IV) field description: The field from which to read the initialization vector, IV pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference decode_raw_bytes: type: boolean title: Decode the encrypted value and IV as Base 64 text description: >- Decode the encrypted value and initialization vector as Base64 text default: true required: - field - algorithm - key - iv_field - if: properties: type: enum: - js-script deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Script Execution description: >- Use JavaScript to reshape and transform your data. You can combine multiple actions like filtering, dropping, mapping, and casting inside of a single js script. additionalProperties: true properties: script: type: string title: Script ui: display_type: code-editor options: language: js description: The JavaScript representation of the event transformation. minLength: 1 maxLength: 10000 default: > // Modify the event using a subset of the JavaScript language. // The function must return the modified event function processEvent(message, metadata, timestamp, annotations) { return message } - if: properties: type: enum: - unroll deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Unroll description: Takes an array of events and emits them all as individual events. additionalProperties: false properties: field: type: string title: Field description: The JSON field name that contains an array of events pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference values_only: type: boolean title: Values only? description: >- When enabled, the values from the specified array field will be emitted as new events. Otherwise, the original event will be duplicated for each value in the array field, with the unrolled value present in the field specified. default: true required: - field - if: properties: type: enum: - vrl deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: VRL description: Modify your observability data using Vector Remap Language (VRL) additionalProperties: false properties: source: type: string title: VRL Source description: VRL code to execute for each event ui: display_type: code-editor options: language: vrl minLength: 1 default: .message = .message required: - source - if: properties: type: enum: - reduce deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Reduce description: >- Combine multiple events over time into one based on a set of criteria additionalProperties: false properties: duration_ms: type: number maximum: 7200000 title: Duration description: >- The amount of time (in milliseconds) to allow streaming events to accumulate into a single "reduced" event. The process repeats indefinitely, or until an "ends when" condition is satisfied. ui: display_type: milliseconds-to-human-readable default: 30000 max_events: type: number minimum: 1 maximum: 100000 title: Max Events description: >- The maximum number of events that can be included in a time window (specified by Duration above). The reduce operation will stop once it has reached this number of events, regardless of whether the Duration seconds have elapsed. group_by: type: array minItems: 1 title: Group By Field Path items: type: string title: Field description: The dot-notated property path errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ description: >- Before reducing, group events based on matching data from each of these field paths. Supports nesting via dot-notation. date_formats: type: array title: Date Formats description: >- Describes which root-level properties are dates, and their expected format. Dot-notation is supported, but nested field lookup paths will be an error. minItems: 1 items: type: object additionalProperties: false properties: field: type: string title: Date Field Path description: >- Specifies a root-level path property that contains a date value. maxLength: 200 errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ format: type: string title: Date Format description: The template describing the date format format: strftime_datetime errorMessage: must be a valid strftime datetime format maxLength: 200 required: - field - format merge_strategies: type: array title: Merge Strategy per Field minItems: 1 description: >- Specify merge strategies for individual root-level properties. Dot-notation is supported, but nested field lookup paths will be an error. items: type: object additionalProperties: false properties: field: type: string title: Field Path description: >- This is a root-level path property to apply a merge strategy to its value maxLength: 200 errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ strategy: oneOf: - label: Array description: Append each value onto an array enum: - array - label: Smallest array description: Keep the smallest array within each reduce time window enum: - shortest_array - label: Largest array description: Keep the largest array within each reduce time window enum: - longest_array - label: Unique Array description: All unique values go into a flattened array enum: - flat_unique - label: Join with space description: Concatenate into a single space-delimited string enum: - concat - label: Join with newline description: Concatenate into a single newline-delimited string enum: - concat_newline - label: Join description: >- Concatenate into a single string without using a delimiter enum: - concat_raw - label: Keep first description: Keep only the first value and discard the rest enum: - discard - label: Keep last description: Keep only the last non-null value enum: - retain - label: Min value description: >- Keep the minimum numeric value within each reduce time window enum: - min - label: Max value description: >- Keep the maximum numeric value within each reduce time window enum: - max - label: Sum description: Add numeric values together enum: - sum ui: display_type: list errorMessage: must be one of the allowed merge strategies title: Merge Strategy description: The merge strategy to be used for the specified property required: - field - strategy flush_condition: title: Event Flush Condition description: >- Force accumulated event reduction to flush the result when a conditional expression evaluates to true on an inbound event. allOf: - type: object properties: when: type: string enum: - none - starts_when - ends_when default: none ui: display_type: list errorMessage: '' title: Flush When required: - when - if: properties: when: not: enum: - none then: properties: conditional: allOf: - $ref: '#condition-group' ui: display_type: rules-engine required: - conditional - if: properties: type: enum: - map-fields deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Map Fields description: Maps data from one field to another, either by moving or copying. additionalProperties: false properties: mappings: type: array title: Mappings description: >- A list of field mappings. Mappings are applied in the order they are defined. minItems: 1 ui: display_type: sortable items: type: object properties: source_field: type: string title: Source field description: The field to copy data from pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference target_field: type: string title: Target field description: The field to copy data into pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference drop_source: type: boolean title: Drop source? description: >- When enabled, the source field is dropped after the data is copied to the target field. Otherwise, it is preserved. default: false overwrite_target: type: boolean title: Overwrite target? description: >- When enabled, any existing data in the target field is overwritten. Otherwise, the target field will be preserved and this mapping will have no effect. default: false required: - source_field - target_field - drop_source - overwrite_target if: properties: drop_source: enum: - true required: - drop_source then: properties: source_field: type: string title: Source field description: The field to copy data from pattern: >- ^(?:metadata|\.|(?:metadata|message)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ minLength: 1 errorMessage: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields). Note that dropping an event's message or timestamp is not allowed required: - source_field required: - mappings - if: properties: type: enum: - metrics-tag-cardinality-limit deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: title: Tag Cardinality Limit (Metrics) description: >- Limits the cardinality of metric events by either dropping events or tags that exceed a specified value limit additionalProperties: false allOf: - if: title: Mode properties: mode: type: string enum: - exact then: title: Value Limit properties: value_limit: type: integer title: Value Limit description: Maximum number of unique values for tags minimum: 1 maximum: 500 required: - value_limit - if: title: Mode properties: mode: type: string enum: - probabilistic then: title: Value Limit properties: value_limit: type: integer title: Value Limit description: Maximum number of unique values for tags minimum: 1 maximum: 5000 required: - value_limit properties: tags: type: array title: Tags description: >- A list of tags to apply cardinality limits. If none are provided, all tags will be considered. items: type: string title: Tag minLength: 1 maxLength: 100 minItems: 0 maxItems: 10 exclude_tags: type: array title: Exclude Tags description: A list of tags to explicitly exclude from cardinality limits. items: type: string title: Tag minLength: 1 maxLength: 100 minItems: 0 maxItems: 10 action: type: string title: Action description: >- The action to take when a tag's cardinality exceeds the value limit enum: - drop_tag - drop_event default: drop_event value_limit: type: integer title: Value Limit description: Maximum number of unique values for tags mode: type: string title: Mode description: The method to used to reduce tag value cardinality enum: - exact - probabilistic default: exact - if: properties: type: enum: - aggregate deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Aggregate description: >- Aggregates multiple events into a single event using either a tumbling interval window or a sliding interval window. additionalProperties: false properties: group_by: type: array minItems: 1 maxItems: 100 title: Group By Field Path description: >- When aggregating, group events based on matching values from each of these field paths. Supports nesting via dot-notation. items: type: string title: Field description: The dot-notated property path errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ ui: default_hint: - .name - .namespace - .tags evaluate: additionalProperties: type: string title: Evaluate description: How events should be aggregated within an interval. allOf: - if: properties: operation: enum: - CUSTOM then: properties: script: type: string title: Script ui: display_type: code-editor options: language: js description: The JavaScript representation of an event aggregation. minLength: 1 maxLength: 2048 default: > // Perform custom aggregation of the current event into the accumulated event. // The accumulated event is persisted across the entire aggregation window. // The function must return the accumulated event, which will be emitted when // the aggregation window elapses. function aggregateEvent(accum, event, metadata, timestamp, annotations) { return accum } required: - script properties: operation: title: Operation oneOf: - label: add description: >- Adds new values together if they are of the same type. For counters and gauges the values are added together. For sets, this is a union of the elements. For distribution, the samples are concatenated. For histograms, values in the same bucket will be added together. enum: - DEFAULT - label: sum description: >- Values are added together. Works for the GAUGE and COUNT metric types. enum: - SUM - label: minimum description: >- Get the minimum value for a series of events. Works for the GAUGE and COUNT metric types. enum: - MIN - label: maximum description: >- Get the maximum value for a series of events. Works for the GAUGE and COUNT metric types. enum: - MAX - label: average description: >- Values are averaged together. Works for the GAUGE and COUNT metric types. enum: - AVG - label: set intersection description: The aggregate produces a set of unique values. enum: - SET_INTERSECTION - label: distribution concatenation description: >- The aggregate produces a list of values combined together. enum: - DIST_CONCAT - label: custom description: Use a custom script to perform aggregation. enum: - CUSTOM ui: display_type: list required: - operation window: default: {} additionalProperties: type: integer title: Window Type description: >- Sliding windows can overlap, whereas tumbling windows are disjoint. For example, a tumbling window has a fixed time span and any events that fall within the "interval" will be used in the aggregate. Whereas in a sliding window, the aggregation occurs every "window duration" seconds after an event is encountered. allOf: - if: properties: type: enum: - sliding required: - type then: properties: window_min: type: integer title: Slide Duration (seconds) description: >- The amount of time to wait before creating a new sliding window for future events. minimum: 1 maximum: 200 default: 5 required: - window_min properties: type: type: string title: Type enum: - tumbling - sliding default: tumbling interval: type: integer title: Interval (seconds) description: The interval over which events are aggregated in seconds. default: 5 minimum: 1 maximum: 90000 required: - type - interval event_timestamp: type: string title: Event Timestamp description: >- The path to a field on the event that contains a epoch timestamp value. If an event does not have a timestamp field, events will be associated to the wall clock value when the event is processed. default: timestamp maxLength: 200 errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ conditional: ui: display_type: rules-engine title: Event Flush Condition description: >- Force accumulated event reduction to flush the result when a conditional expression evaluates to true on an inbound event. allOf: - $ref: '#condition-group' required: - evaluate - window - if: properties: type: enum: - event-to-metric deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Event to Metric description: Allows conversion between arbitrary events and a Metric. additionalProperties: false properties: metric_name: type: string title: Metric Name description: The machine name of the metric to emit. minLength: 1 maxLength: 128 pattern: ^[a-zA-Z][a-zA-Z0-9_:]*$ errorMessage: must be a valid metric name metric_kind: title: Kind description: >- The kind of metric to emit, Absolute or Incremental. Absolute metrics represent a complete value, and will generally replace an existing value for the metric in the target destination. Incremental metrics represent an additive value which is aggregated in the target destination to produce a new value. oneOf: - label: Incremental enum: - incremental - label: Absolute enum: - absolute default: incremental ui: display_type: radio-group metric_type: title: Type oneOf: - label: Counter enum: - counter - label: Sum enum: - sum - label: Gauge enum: - gauge default: counter ui: display_type: list value: title: Value description: The value of this metric. additionalProperties: false allOf: - if: properties: value_type: enum: - field then: title: Value properties: value: type: string title: Field value description: The field to use as the value. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference required: - value - if: properties: value_type: enum: - value then: title: Value properties: value: type: number title: Value description: The value. required: - value properties: value_type: title: Value type description: Either a value from an event field, or a new value input. oneOf: - label: Value from event field enum: - field - label: New value enum: - value default: field ui: display_type: radio-group value: type: - number - string required: - value_type namespace: title: Namespace description: The namespace for this metric. additionalProperties: false allOf: - if: properties: value_type: enum: - field then: title: Value properties: value: type: string title: Field value description: The field to use as the value. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference required: - value - if: properties: value_type: enum: - value required: - value_type then: title: Value properties: value: type: string title: Value description: The value. minLength: 1 maxLength: 128 required: - value properties: value_type: title: Value type description: >- Either a value from an event field, a new value input, or none to omit the namespace. oneOf: - label: Value from event field enum: - field - label: New value enum: - value - label: None enum: - none default: none ui: display_type: radio-group value: type: - string - 'null' required: - value_type tags: type: array maxItems: 10 title: Tags description: A set of tags (also called labels) to apply to the metric event. items: type: object title: Tag description: A tag to apply to the metric event. allOf: - if: properties: value_type: enum: - field then: title: Value properties: value: type: string title: Field value description: >- The field to use as the value of this tag. Note that fields with highly-variable values will result in high-cardinality metrics, which may impact storage or cost in downstream destinations. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference fallback: type: string title: Fallback value description: >- The value to use for this tag when the field value is missing. minLength: 1 maxLength: 512 required: - value - if: properties: value_type: enum: - value then: title: Value properties: value: type: string title: Value description: The value for this tag. minLength: 1 maxLength: 128 required: - value properties: name: type: string title: Name description: The tag name pattern: ^[a-zA-Z][a-zA-Z0-9_]*$ minLength: 1 maxLength: 128 errorMessage: must be a valid metric tag name value_type: title: Value type description: Either a value from an event field, or a new value input. oneOf: - label: Value from event field enum: - field - label: New value enum: - value default: field ui: display_type: radio-group value: type: string required: - name - value_type - value additionalProperties: type: string required: - metric_name - metric_kind - metric_type - value - namespace - if: properties: type: enum: - clustering deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Clustering description: Finds clusters of similar values additionalProperties: false properties: cluster_field: type: string title: Field description: Field to cluster values on pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference default: .message similarity_threshold: type: number title: Similarity threshold description: >- A threshold from 0.0 (not similar) to 1.0 (exactly similar) that determines how close two values need to be to be consider as being part of the same cluster default: 0.4 minimum: 0 maximum: 1 max_clusters: type: integer title: Maximum cluster description: Maximum number of clusters kept cached by the transform default: 1000 minimum: 1 maximum: 2000 max_node_depth: type: integer title: Maximum tree depth description: Maximum depth of the prefix tree default: 4 minimum: 2 maximum: 8 max_children: type: integer title: Maximum children default: 100 minimum: 2 maximum: 200 description: Maximum number of children per node of the prefix tree required: - cluster_field - if: properties: type: enum: - redact deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Redact description: Detect and anonymize sensitive data in logs and metrics additionalProperties: false properties: field: type: string title: Field description: >- Detect patterns in the specified field. If the field is empty, the processor will look for patterns across all the fields. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ default: . maxLength: 256 errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference mask_patterns: type: array title: Redact Patterns description: List of PII patterns to detect and mask minItems: 1 maxItems: 20 items: type: object additionalProperties: type: string title: Redact Pattern description: A PII pattern to detect and/or anonymize allOf: - if: properties: action: enum: - Replace required: - action then: properties: replacement: type: string title: Replacement description: >- The text to replace the detected pattern. Example: 000-00-0000 minLength: 1 maxLength: 512 required: - replacement - if: properties: action: enum: - Hash required: - action then: properties: algorithm: title: Algorithm description: >- Hash and encode (base16 or hex) matched PII data using the specified algorithm oneOf: - label: sha2 (sha-512_256) enum: - sha2 - label: sha3 (sha3-512) enum: - sha3 ui: display_type: list required: - algorithm - if: properties: name: enum: - custom required: - name then: properties: custom_options: allOf: - type: object additionalProperties: false properties: pattern: type: string title: Pattern description: The custom pattern minLength: 1 maxLength: 4096 case_sensitive: type: boolean title: Case sensitive description: Perform case sensitive matching default: true multiline: type: boolean title: Multiline mode description: >- ^ and $ match beginning and end of line respectively default: false match_newline: type: boolean title: Dot all mode description: Dot matches newline character default: false crlf_newline: type: boolean title: Carriage return (CRLF) mode description: "When multiline mode is enabled, newline character is interpreted as \r\n" default: false ignore_whitespace: type: boolean title: Verbose mode description: Ignore whitespace and allow comment lines default: false required: - pattern title: Replace Expression description: >- The regular expression used to find replacements. Named capture groups are supported, e.g. `(?P<key>\w+):(?P<value>.+)`. ui: display_type: regex-pattern required: - custom_options properties: name: title: Pattern description: The PII pattern to match oneOf: - label: Credit card number enum: - credit_card - label: Email address enum: - email_address - label: IPv4 address enum: - ipv4_address - label: US or Canada phone number enum: - us_canada_phone_number - label: US social security number enum: - us_social_security_number - label: Custom pattern ui: feature_flag: pipeline-parser-regex enum: - custom ui: display_type: list action: type: string title: Action description: Action to take on the detected PII data enum: - Hash - Replace - Detect Only default: Replace required: - name - action required: - mask_patterns - if: properties: type: enum: - throttle deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Throttle description: Throttle (rate-limit) events passing through this component additionalProperties: false properties: threshold: type: number minimum: 1 maximum: 10000000 title: Limit description: >- The number of events to allow over the time window. If a Key Field is specified, the limit will be applied to events for each unique value of that field. default: 1 window_ms: type: number minimum: 1000 maximum: 2592000000 title: Window description: The time window over which the configured limit is applied ui: display_type: milliseconds-to-human-readable default: 5000 key_field: type: string title: Key Field description: >- When configured, the value from this field will be used to group events into separate buckets for throttling. If left blank, or if the event does not contain a value in this field, the event will be throttled as part of the default bucket. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference exclude: allOf: - $ref: '#condition-group' title: Exclude description: >- When configured, events matching this condition will not be throttled. ui: display_type: rules-engine required: - threshold - window_ms - if: properties: type: enum: - set-timestamp deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Set Timestamp description: >- Parse a list of fields using a chosen time format. First match will override the default timestamp of the event. additionalProperties: false properties: parsers: type: array title: List of Fields and Parsers description: >- The list of fields and parsers to use in order of priority, short-circuiting on the first successful match. ui: display_type: multi-output-map options: sortable: false items: type: object properties: field: type: string title: Field description: >- The field whose value will be parsed by the chosen date pattern pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ default: . maxLength: 512 errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference options: $ref: '#timestamp-parser-options' required: - field - options minItems: 1 required: - parsers - if: properties: type: enum: - data-profiler deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Data Profiler description: >- Profile the data sent through your pipeline. The system will generate annotations such as event type and total bytes to be used downstream. It will also normalize the app/host/level/line fields before sending the event downstream. A profile of the data including total, events, event size, log composition, and message templates will be generated as well. additionalProperties: false properties: app_fields: type: array title: App Path(s) description: A list of paths to look for the app value minItems: 1 maxItems: 10 uniqueItems: true items: type: string title: App description: The path to the app maxLength: 512 pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference ui: default_hint: - .app - .application - .container host_fields: type: array title: Host Path(s) description: A list of paths to look for the host value minItems: 1 maxItems: 10 uniqueItems: true items: type: string title: Host description: The path to the host maxLength: 512 pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference ui: default_hint: - .host - .hostname level_fields: type: array title: Level Path(s) description: A list of paths to look for the log level minItems: 1 maxItems: 10 uniqueItems: true items: type: string title: Level description: The path to the level maxLength: 512 pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference ui: default_hint: - .level - .log_level line_fields: type: array title: Line Path(s) description: A list of paths to look for the event line minItems: 1 maxItems: 10 uniqueItems: true items: type: string title: Line description: The path to the line maxLength: 512 pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference ui: default_hint: - .line - .message label_fields: type: array title: Label Path(s) description: A list of paths to look for the event labels maxItems: 10 uniqueItems: true items: type: string title: Label description: The path to the labels maxLength: 512 pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference ui: default_hint: - .labels field_summary: type: boolean default: false title: Field Summary description: >- If enabled the profiler will attempt to summarize the values in fields of events passing through the node. ui: display_type: hidden ff_default_value: pipeline-node-transform-field-summary required: - app_fields - host_fields - level_fields - line_fields - if: properties: type: enum: - replace deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: title: Replace description: >- Replace parts of a string using a regular expression or static string additionalProperties: type: object allOf: - if: properties: replacement_type: enum: - regex required: - replacement_type then: properties: options: allOf: - type: object additionalProperties: false properties: pattern: type: string minLength: 1 maxLength: 4096 case_sensitive: type: boolean title: Case sensitive description: Perform case sensitive matching default: true multiline: type: boolean title: Multiline mode description: >- ^ and $ match beginning and end of line respectively default: false match_newline: type: boolean title: Dot all mode description: Dot matches newline character default: false crlf_newline: type: boolean title: Carriage return (CRLF) mode description: "When multiline mode is enabled, newline character is interpreted as \r\n" default: false ignore_whitespace: type: boolean title: Verbose mode description: Ignore whitespace and allow comment lines default: false required: - pattern title: Replace Expression description: >- The regular expression used to find replacements. Named capture groups are supported, e.g. `(?P<key>\w+):(?P<value>.+)`. ui: display_type: regex-pattern replace_with: type: string minLength: 1 maxLength: 4096 title: Replacement String description: >- The replacement string can be static or optionally contain capture group references. Capture groups can be positional or named. For example, to switch the position of two strings in the source string `onetwo`, the pattern could be `(?Pone)(?P.+)`, and the replacement would be `$two$one`. To do the same thing with positional capture groups, a pattern of `(\w{3})(\w{3})`, and a replacement of `$2$1` would have the same result.

To render a literal dollar sign `$` in the replacement, escape it with `$$`. So, if you had a capture group named `price`, to prepend a dollar sign to the replacement, you would use `$$$price`. required: - options - replace_with - if: properties: replacement_type: enum: - string required: - replacement_type then: properties: string_pattern: type: string title: String Pattern minLength: 1 maxLength: 4096 description: >- A static, case-sensitive string, which will be searched and replaced. replace_with: type: string minLength: 1 maxLength: 4096 title: Replacement String description: The static string used in each replacement. required: - string_pattern - replace_with properties: source_field: type: string title: Source Field description: The field that contains a string for replacement to operate on. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ default: . errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference max_times: type: number title: Maximum Replacements description: >- The maximum number of replacements to be made. If not specified, all occurrences will be replaced. minimum: 1 replacement_type: title: Replacement Type description: >- Specifies how the source string will be searched for replacement. ui: display_type: list errorMessage: must be one of the available pattern types oneOf: - label: String description: Replace a specified string one or more times. enum: - string - label: Regular Expression description: Replace using a regular expression. enum: - regex required: - source_field - replacement_type - if: properties: type: enum: - trace-sampling deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: title: Trace Sampling description: >- Performs sampling of traces per a given rate using the Trace ID as the unique identifier. If a trace is rate selected, all spans of the trace are forwarded. If the Trace ID cannot be found, the event is dropped. allOf: - if: properties: sample_type: enum: - head then: properties: rate: type: integer title: Rate description: >- The rate at which events will be forwarded, expressed as 1/N. For example, `rate = 10` means 1 out of every 10 unique traces will be forwarded and all others will be dropped default: 10 minimum: 2 maximum: 100000 parent_span_id_field: type: 'null' conditionals: type: 'null' required: - rate - if: properties: sample_type: enum: - tail then: properties: parent_span_id_field: type: string default: .parent_span_id title: Parent Span ID description: The field to use as the parent span identifier pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference conditionals: type: array minItems: 1 title: Conditionals description: >- A list of complete conditionals, including expressions and a sample rate to maintain items: type: object additionalProperties: false title: Conditional description: A list of conditions and a sample rate to maintain properties: rate: type: integer title: Rate description: >- The rate at which events will be forwarded, expressed as 1/N. For example, `rate = 10` means 1 out of every 10 unique traces will be forwarded and all others will be dropped. A `rate = 1` means keep all traces that match the condition. minimum: 1 maximum: 100000 default: 10 conditional: allOf: - $ref: '#condition-group' title: Conditional description: >- A group of expressions joined together by an and/or operator. ui: display_type: rules-engine _output_name: type: string title: '[INTERNAL]' description: >- Internal-only field that holds a static id for the output_name. This way, labels for outputs can be updated independent of the id. required: - rate - conditional required: - parent_span_id_field - conditionals properties: sample_type: type: string description: The type of trace sampling to apply. enum: - head title: Sample Type default: head trace_id_field: type: string default: .trace_id title: Trace ID description: The field to use as the identifier of a trace. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference required: - sample_type - trace_id_field properties: title: oneOf: - type: string minLength: 1 maxLength: 512 pattern: ^[^<>"]+$ - type: 'null' errorMessage: invalid title name description: oneOf: - type: string minLength: 1 pattern: ^[^<>"]+$ - type: 'null' errorMessage: invalid description type: type: string enum: - filter - sample - drop-fields - parse - parse-sequentially - route - stringify - compact-fields - flatten-fields - dedupe - encrypt-fields - decrypt-fields - js-script - unroll - vrl - reduce - map-fields - metrics-tag-cardinality-limit - aggregate - event-to-metric - clustering - redact - throttle - set-timestamp - data-profiler - replace - trace-sampling user_config: type: object required: - user_config sinks: type: array minItems: 1 items: type: object additionalProperties: false allOf: - if: properties: type: enum: - mezmo deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Mezmo Log Analysis description: Send log data to Mezmo Log Analysis additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true mezmo_host: type: string title: Mezmo Host description: The host for your Log Analysis environment default: null minLength: 1 ingestion_key: type: string title: Ingestion Key description: Ingestion key ui: display_type: mezmo-ingestion-key minLength: 1 query: type: object title: Query Parameters default: hostname: mezmo properties: hostname: type: string title: Hostname description: >- Hostname string or template to extract event values to form a hostname. Depending on the type of ingestion, this value is usually set from other places so the default value is often sufficient. If the value in the event is null, empty, or doesn't exist, when using a template then it will be set to "mezmo" default: mezmo minLength: 1 ui: display_type: data-template tags: type: array title: Tags description: >- List of tag strings or templates to attach to logs. Template must resolve to either an array of strings, or a string. Commas are treated as tag delimiters. items: type: string title: Tag description: Tag string or template to attach to logs minLength: 1 maxLength: 512 ui: display_type: data-template ip: type: string title: IP description: IP address template to attach to logs minLength: 1 maxLength: 512 ui: display_type: data-template mac: type: string title: MAC description: MAC address template to attach to logs minLength: 1 maxLength: 512 ui: display_type: data-template message: title: Log Construction Scheme default: scheme: Explicit field selection allOf: - if: properties: scheme: enum: - Explicit field selection then: properties: line: type: string title: Line description: >- Template or field reference to use as the log line. Field reference can point to an object. minLength: 1 maxLength: 1024 meta_field: type: string title: Meta Field description: Field containing the meta object for the log pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference maxLength: 512 app: type: string title: App description: App name template to attach to logs minLength: 1 maxLength: 512 ui: display_type: data-template file: type: string title: File description: File name template to attach to logs minLength: 1 maxLength: 512 ui: display_type: data-template timestamp_field: type: string title: Timestamp description: Field containing the timestamp for the log pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference maxLength: 512 env: type: string title: Env description: Environment name template to attach to logs minLength: 1 maxLength: 512 ui: display_type: data-template properties: scheme: type: string title: Scheme description: How to construct the log message. enum: - Explicit field selection - Message pass-through default: Explicit field selection required: - scheme required: - mezmo_host - ingestion_key - if: properties: type: enum: - mezmo-archive deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Mezmo Archive Destination description: Store logs for future imports into mezmo pipeline or log analysis. additionalProperties: false properties: batch_timeout_secs: type: integer title: Batch Timeout (seconds) description: >- The maximum amount of time, in seconds, events will be buffered before being flushed to the destination enum: - 30 - 60 - 90 - 120 - 300 default: 300 ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true destination: title: Destination description: '' allOf: - if: properties: archive_provider: enum: - Azure then: properties: container_name: type: string title: Container Name minLength: 1 description: The name of the container for blob storage. connection_string: type: string title: Connection String minLength: 1 description: >- A connection string for the account that contains an access key required: - container_name - connection_string - if: properties: archive_provider: enum: - S3 then: properties: access_key_id: type: string title: Access Key ID minLength: 1 description: The AWS access key id secret_access_key: type: string title: Secret Access Key minLength: 1 description: The AWS secret access key ui: display_type: secret bucket: type: string title: Bucket pattern: >- (?!(^xn--|.+-s3alias$))^[a-z0-9][a-z0-9-]{1,61}[a-z0-9]$ description: >- The S3 bucket name. Do not include a leading s3:// or a trailing / region: type: string title: Region minLength: 1 description: The name of the AWS region that is targeted. required: - access_key_id - secret_access_key - bucket - region properties: archive_provider: type: string title: Archive Provider description: The storage destination for your data. enum: - Azure - S3 default: Azure required: - archive_provider required: - destination - if: properties: type: enum: - http deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: HTTP description: Send data to a HTTP endpoint additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true uri: type: string title: URI format: uri errorMessage: format: Must be a valid URI description: >- The full URI to make HTTP requests to. This should include the protocol and host, but can also include the port, path, and any other valid part of a URI. minLength: 1 encoding: type: string description: The encoding to apply to the data. enum: - json - ndjson - text title: Encoding default: text compression: type: string description: >- The compression strategy used on the encoded data prior to sending. enum: - gzip - none title: Compression default: none auth: title: Authentication description: Configures HTTP authentication allOf: - if: properties: strategy: enum: - basic then: properties: user: type: string title: User minLength: 1 description: The basic authentication user password: type: string title: Password minLength: 1 description: The basic authentication password ui: display_type: secret required: - user - password - if: properties: strategy: enum: - bearer then: properties: token: type: string title: Token minLength: 1 description: The bearer token ui: display_type: secret required: - token properties: strategy: type: string enum: - basic - bearer - none title: Strategy description: The authentication strategy to use default: none headers: type: array title: HTTP Headers description: >- An array of key/value objects for the headers to be sent with the request. items: type: object title: Headers description: A key/value object describing a header name and its value. additionalProperties: false properties: header_name: type: string title: Header Name minLength: 1 description: The name of the header. header_value: type: string title: Header Value minLength: 1 description: The value of the header. required: - header_name - header_value advanced_options: type: object ui: display_type: collapsable feature_flag: pipeline-node-sink-http-advanced-options title: Advanced Options description: Settings available for fine-tuning the destination behavior. additionalProperties: false dependencies: payload_prefix: - payload_suffix payload_suffix: - payload_prefix properties: max_bytes: type: integer title: Payload Size description: >- The maximum number of uncompressed bytes when batching data to the destination. minimum: 1 maximum: 2097152 timeout_secs: type: integer title: Timeout Seconds description: The number of seconds before a destination write timeout. minimum: 5 method: type: string enum: - post - put - patch - delete - get - head - options - trace title: HTTP Method description: The HTTP method to use for the destination. payload_prefix: type: string title: Payload Prefix description: >- Add a prefix to the payload. Only used for serialized JSON chunks. This option also requires "Payload Suffix" to form a valid JSON string. examples: - '{"event":"' - '{"extra_prop": true, "our_thing":"' minLength: 1 payload_suffix: type: string title: Payload Suffix description: >- Used in combination with "Payload Prefix" to form valid JSON from the payload. examples: - '}' - '"extra_prop": true }' minLength: 1 proxy: type: object additionalProperties: false title: Proxy Settings description: Configure proxy settings for the destination. properties: enabled: type: boolean title: Enabled description: Turns proxying on/off default: false endpoint: type: string minLength: 1 title: HTTP(S) Endpoint description: HTTP or HTTPS Endpoint to use for traffic. format: uri errorMessage: format: Must be a valid URI hosts_bypass_proxy: type: array minItems: 1 title: Hosts to Bypass description: >- A list of hosts to bypass proxying. Can be specified as a domain name, IP address, or CIDR block. Wildcards are supported as a dot (.) in domain names, or as a star (*) to match all hosts. examples: - 93.184.216.34 - ignorehost.example.com - .ignoreall.example.com - 93.184.216.0/24 items: type: string title: Host description: Domain name, IP address, or CIDR block. minLength: 1 tls_protocols: type: array title: TLS Protocols description: >- A list of ALPN protocols to use during TLS negotiation. They are attempted in the order they appear. items: type: string title: Protocol description: An ALPN protocol name minLength: 1 minItems: 1 rate_limiting: type: object additionalProperties: false title: Rate Limiting description: Settings for controlling rate limiting to the destination. properties: request_limit: type: integer minimum: 1 title: Request Limit description: >- The max number of requests allowed within the specified "Duration Seconds" setting. duration_secs: type: integer default: 1 title: Duration Seconds description: The window of time used to apply "Request Limit." minimum: 1 required: - uri - encoding - compression - if: properties: type: enum: - cloudwatch-logs deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: AWS Cloudwatch Logs description: Publish log events to AWS Cloudwatch Logs additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true auth: type: object title: AWS Authentication description: Configures AWS authentication additionalProperties: false properties: access_key_id: type: string title: Access Key ID minLength: 1 description: The AWS access key id secret_access_key: type: string title: Secret Access Key minLength: 1 description: The AWS secret access key ui: display_type: secret required: - access_key_id - secret_access_key encoding: type: string description: The encoding to apply to the data. enum: - json - text title: Encoding default: text compression: type: string description: >- The compression strategy used on the encoded data prior to sending. enum: - gzip - none title: Compression default: none group_name: type: string title: Group Name minLength: 1 description: The name of the log group for the targeted log stream. region: type: string title: Region minLength: 1 description: The name of the AWS region that is targeted. stream_name: type: string title: Stream Name minLength: 1 description: The name of the targeted log stream. required: - auth - encoding - compression - group_name - region - stream_name - if: properties: type: enum: - cloudwatch-metrics deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: AWS Cloudwatch Metrics description: Send metric events to AWS Cloudwatch Metrics additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true auth: type: object title: AWS Authentication description: Configures AWS authentication additionalProperties: false properties: access_key_id: type: string title: Access Key ID minLength: 1 description: The AWS access key id secret_access_key: type: string title: Secret Access Key minLength: 1 description: The AWS secret access key ui: display_type: secret required: - access_key_id - secret_access_key compression: type: string description: >- The compression strategy used on the encoded data prior to sending. enum: - gzip - none title: Compression default: none namespace: type: string title: Namespace minLength: 1 description: >- Name for the container that will isolate metrics from one another. region: type: string title: Region minLength: 1 description: The name of the AWS region that is targeted. required: - auth - compression - namespace - region - if: properties: type: enum: - sqs deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: AWS Simple Queue Service (SQS) description: Publish log events to AWS Simple Queue Service additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true auth: type: object title: AWS Authentication description: Configures AWS authentication additionalProperties: false properties: access_key_id: type: string title: Access Key ID minLength: 1 description: The AWS access key id secret_access_key: type: string title: Secret Access Key minLength: 1 description: The AWS secret access key ui: display_type: secret required: - access_key_id - secret_access_key encoding: type: string description: The encoding to apply to the data. enum: - json - text title: Encoding default: text sqs_queue_url: type: string title: SQS Queue URL format: url errorMessage: format: Must be a valid URL description: The URL of the SQS queue that the data is publishing to. region: type: string title: Region minLength: 1 description: The name of the AWS region that is targeted. required: - auth - encoding - sqs_queue_url - region - if: properties: type: enum: - notification-channel deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Notification Channel description: Send a Message to a 3rd Party Service. additionalProperties: false properties: service: title: Service description: >- Configure a service to receive the payload. Currently supports Slack, PagerDuty, Webhook or Mezmo (Log Analysis). Where allowed, messagetemplating can be used. To use a template, surround field names with double curly braces, e.g., {{.my_field}} or {{."my-object".thing}}. See [documentation](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) for more examples. additionalProperties: type: string title: URI format: uri errorMessage: format: Must be a valid URI description: >- The full URI where the payload is sent. This should include the protocol and host, but can also include the port, path, and any other valid part of a URI. minLength: 1 allOf: - if: properties: name: enum: - slack required: - name then: properties: uri: type: string title: URI format: uri errorMessage: format: Must be a valid URI description: >- The full URI where the payload is sent. This should include the protocol and host, but can also include the port, path, and any other valid part of a URI. minLength: 1 message_text: type: string title: Message minLength: 1 ui: display_type: data-template options: multiline: true description: The text to be sent as a Slack message. required: - uri - message_text - if: properties: name: enum: - pager_duty required: - name then: properties: uri: type: string title: URI format: uri errorMessage: format: Must be a valid URI description: >- The full URI where the payload is sent. This should include the protocol and host, but can also include the port, path, and any other valid part of a URI. minLength: 1 summary: type: string title: Summary description: Summarize the alert details for PagerDuty. minLength: 1 ui: display_type: data-template severity: type: string enum: - INFO - WARNING - ERROR - CRITICAL title: Severity description: The severity level of the alert. source: type: string title: Source description: The source of the alert. minLength: 1 ui: display_type: data-template routing_key: type: string title: Routing Key description: The service integration routing key. minLength: 1 ui: display_type: secret event_action: type: string enum: - trigger - acknowledge - resolve title: Event Action description: The event action associated with the alert. required: - uri - summary - severity - source - routing_key - event_action - if: properties: name: enum: - webhook required: - name then: properties: uri: type: string title: URI format: uri errorMessage: format: Must be a valid URI description: >- The full URI where the payload is sent. This should include the protocol and host, but can also include the port, path, and any other valid part of a URI. minLength: 1 method: type: string title: HTTP Method description: The HTTP method to use for the request. enum: - post - put - patch - delete - get - head - options - trace ui: default_hint: post message_text: type: string title: Message minLength: 1 ui: display_type: data-template options: multiline: true description: >- The Webhook payload to send. This can be a text string or stringified JSON. If the message can be parsed as JSON, it will be sent as such. headers: type: array title: HTTP Headers description: >- A list of key/value pairs to be included in the HTTP request. items: type: object title: Headers description: >- A key/value object describing a header name and its value. additionalProperties: false properties: header_name: type: string title: Header Name minLength: 1 description: The name of the header. header_value: type: string title: Header Value minLength: 1 maxLength: 1024 description: The value of the header. required: - header_name - header_value maxItems: 20 auth: title: Authentication Options description: Configures HTTP authentication allOf: - if: properties: strategy: enum: - basic then: properties: user: type: string title: User minLength: 1 description: The basic authentication user password: type: string title: Password minLength: 1 description: The basic authentication password ui: display_type: secret required: - user - password - if: properties: strategy: enum: - bearer then: properties: token: type: string title: Token minLength: 1 description: The bearer token ui: display_type: secret required: - token properties: strategy: type: string enum: - basic - bearer - none title: Strategy description: Choose basic or token-based authentication. default: none required: - uri - message_text - if: properties: name: enum: - log_analysis required: - name then: properties: severity: type: string enum: - INFO - WARNING - ERROR - CRITICAL title: Severity description: The severity level of the alert. default: INFO subject: type: string title: Subject description: A string to be used as the subject of the alert. minLength: 1 maxLength: 200 ui: display_type: data-template body: type: string title: Body description: A string to be used as the alert message body. minLength: 1 maxLength: 1024 ui: display_type: data-template ingestion_key: type: string title: Ingestion Key description: >- The key required to communicate to Log Analysis for sending the alert. ui: display_type: mezmo-ingestion-key minLength: 1 required: - severity - subject - body - ingestion_key properties: name: title: Name description: The name of the service that will receive the notification. ui: display_type: list errorMessage: must be a valid service name oneOf: - label: Slack enum: - slack - label: PagerDuty enum: - pager_duty - label: Webhook enum: - webhook - label: Mezmo Log Analysis enum: - log_analysis required: - name throttling: type: object title: Throttling description: >- Only allow a certain number of notifications to be sent within a given time window. properties: window_secs: type: integer title: Window Seconds description: >- The time frame during which the number of notifications (set by the Threshold) is permitted. default: 60 minimum: 1 threshold: type: integer title: Threshold description: >- The maximum number of notifications allowed over the given time window set by "Window Seconds" (default: 1). default: 1 minimum: 1 required: - service - throttling - if: properties: type: enum: - azure-blob-storage deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Azure Blob Storage description: Publish log events to Azure Blob Storage additionalProperties: false properties: batch_timeout_secs: type: integer title: Batch Timeout (seconds) description: >- The maximum amount of time, in seconds, events will be buffered before being flushed to the destination enum: - 30 - 60 - 90 - 120 - 300 default: 300 ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true encoding: type: string description: The encoding to apply to the data. enum: - json - ndjson - text title: Encoding default: text compression: type: string description: >- The compression strategy used on the encoded data prior to sending. enum: - gzip - none title: Compression default: none container_name: type: string title: Container Name minLength: 1 description: The name of the container for blob storage. connection_string: type: string title: Connection String minLength: 1 description: A connection string for the account that contains an access key prefix: type: string title: Prefix minLength: 1 description: A prefix to be applied to all object keys ui: display_type: data-template file_consolidation: type: object ui: display_type: collapsable feature_flag: pipeline-node-sink-azure-blob-file-consolidation title: File Consolidation description: >- This sink writes many small files out to azure blob storage. Enabling this process will allow the automatic consolidation of these small files into larger files of your choosing. This process will enable upon deployment and run on the chosen interval from `Processing Interval` creating files named `merged_[timestamp].log` where `timestamp` is the time since epoch when the actual file was created. The process will recursively access all files under the `Base Path` to handle merging sub-directory logging structures. additionalProperties: false properties: enabled: type: boolean default: false title: Enabled description: Toggles whether the process is enabled. process_every_seconds: type: integer title: Processing Interval (seconds) description: How often to run the consolidation process. default: 600 minimum: 300 maximum: 3600 requested_size_bytes: type: integer title: Requested File Size (bytes) description: The requested size of the consolidated files. default: 500000000 minimum: 50000000 maximum: 10000000000 base_path: type: string title: Base Path default: '' maxLength: 512 description: >- The path from the container to begin recursivelylooking for files to merge. A blank value indicates root.Merged files will only contain data from the respectivefolder. required: - encoding - compression - container_name - connection_string - if: properties: type: enum: - kinesis-firehose deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: AWS Kinesis Data Firehose description: Publish log events to AWS Kinesis Data Firehose additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true auth: type: object title: AWS Authentication description: Configures AWS authentication additionalProperties: false properties: access_key_id: type: string title: Access Key ID minLength: 1 description: The AWS access key id secret_access_key: type: string title: Secret Access Key minLength: 1 description: The AWS secret access key ui: display_type: secret required: - access_key_id - secret_access_key encoding: type: string description: The encoding to apply to the data. enum: - json - text title: Encoding default: text compression: type: string description: >- The compression strategy used on the encoded data prior to sending. enum: - gzip - none title: Compression default: none region: type: string title: Region minLength: 1 description: The name of the AWS region that is targeted. stream_name: type: string title: Stream Name minLength: 1 description: The name of the Kinesis Firehose stream required: - auth - encoding - compression - region - stream_name - if: properties: type: enum: - kinesis-streams deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: AWS Kinesis Streams description: Publish log events to AWS Kinesis Streams additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true auth: type: object title: AWS Authentication description: Configures AWS authentication additionalProperties: false properties: access_key_id: type: string title: Access Key ID minLength: 1 description: The AWS access key id secret_access_key: type: string title: Secret Access Key minLength: 1 description: The AWS secret access key ui: display_type: secret required: - access_key_id - secret_access_key encoding: type: string description: The encoding to apply to the data. enum: - json - text title: Encoding default: text compression: type: string description: >- The compression strategy used on the encoded data prior to sending. enum: - gzip - none title: Compression default: none partition_key_field: type: string title: Partition Key Field pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ description: >- The field in the log whose value is used as Kinesis' partition key. errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference region: type: string title: Region minLength: 1 description: The name of the AWS region that is targeted. stream_name: type: string title: Stream Name minLength: 1 description: The name of the targeted log stream. required: - auth - encoding - compression - region - stream_name - if: properties: type: enum: - elasticsearch deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Elasticsearch description: Publish log events to Elasticsearch additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true compression: type: string description: >- The compression strategy used on the encoded data prior to sending. enum: - gzip - none title: Compression default: none auth: title: ElasticSearch Auth Strategy description: ElasticSearch authentication strategy allOf: - if: properties: strategy: enum: - basic then: properties: user: type: string title: User minLength: 1 maxLength: 512 description: The basic authentication user password: type: string title: Password minLength: 1 maxLength: 512 description: The basic authentication password ui: display_type: secret required: - user - password - if: properties: strategy: enum: - aws then: properties: access_key_id: type: string title: Access Key ID minLength: 1 maxLength: 1024 description: AWS access key ID ui: display_type: secret secret_access_key: type: string title: Secrete Access Key minLength: 1 maxLength: 1024 description: AWS secrete access key ui: display_type: secret region: type: string title: Region minLength: 1 maxLength: 128 description: The name of the AWS region that is targeted. required: - access_key_id - secret_access_key - region properties: strategy: type: string title: Strategy description: The ES authentication strategy to use enum: - basic - aws required: - strategy endpoints: type: array title: Endpoints uniqueItems: true minItems: 1 description: An array of ElasticSearch endpoints items: type: string title: URL format: url errorMessage: format: Must be a valid URL description: The full Elasticsearch URL to send data to. minLength: 1 maxLength: 1024 pipeline: type: string title: Pipeline minLength: 1 description: Name of an ES ingest pipeline to include. index: type: string minLength: 1 maxLength: 512 title: Custom Index description: Index to use when writing ES events (default = mezmo-%Y.%m.%d) default: mezmo-%Y.%m.%d required: - compression - auth - endpoints - if: properties: type: enum: - gcp-cloud-monitoring deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: GCP Cloud Monitoring description: Publish metrics events to GCP Cloud Monitoring additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true credentials_json: type: string title: JSON Credentials minLength: 1 description: JSON Credentials ui: display_type: secret-file project_id: type: string title: Project ID minLength: 1 description: The Project ID as defined in Google Cloud. resource_type: type: string title: Resource Type minLength: 1 description: The `monitored-resource` type as defined in Monitoring. resource_labels: type: array title: Resource Labels description: >- An array of key/val objects describing labels of the Monitoring resource. items: type: object title: Label/Value description: A key/val object describing a label name and its value. additionalProperties: false properties: label_name: type: string title: Label Name minLength: 1 description: The name of the label. label_value: type: string title: Label Value minLength: 1 description: The value of the label. required: - label_name - label_value required: - credentials_json - project_id - resource_type - if: properties: type: enum: - gcp-cloud-storage deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: GCP Cloud Storage description: Publish log events to GCP Cloud Storage additionalProperties: false properties: batch_timeout_secs: type: integer title: Batch Timeout (seconds) description: >- The maximum amount of time, in seconds, events will be buffered before being flushed to the destination enum: - 30 - 60 - 90 - 120 - 300 default: 300 ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true credentials_json: type: string title: JSON Credentials minLength: 1 description: JSON Credentials ui: display_type: secret-file encoding: type: string description: Dictates how the data will be serialized before storing. enum: - json - ndjson - text title: Encoding default: text bucket: type: string title: Bucket minLength: 1 description: The name of the bucket in GCP where the data will be stored. compression: type: string description: >- The compression strategy used on the encoded data prior to sending. enum: - gzip - none title: Compression default: none bucket_prefix: type: string title: Bucket Prefix minLength: 1 description: >- The prefix applied to the bucket name, giving the appearance of having directories. required: - credentials_json - encoding - bucket - compression - if: properties: type: enum: - gcp-cloud-operations deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: GCP Cloud Operations description: Publish log events to GCP Cloud Operations additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true credentials_json: type: string title: JSON Credentials minLength: 1 description: JSON Credentials ui: display_type: secret-file log_id: type: string title: Log ID minLength: 1 description: Concise reference for the log stream name. project_id: type: string title: Project ID minLength: 1 description: The Project ID as defined in Google Cloud. resource_type: type: string title: Resource Type minLength: 1 description: The `monitored-resource` type as defined in Monitoring. resource_labels: type: array title: Resource Labels description: >- An array of key/val objects describing labels of the Monitoring resource. items: type: object title: Label/Value description: A key/val object describing a label name and its value. additionalProperties: false properties: label_name: type: string title: Label Name minLength: 1 description: The name of the label. label_value: type: string title: Label Value minLength: 1 description: The value of the label. required: - label_name - label_value required: - credentials_json - log_id - project_id - resource_type - if: properties: type: enum: - gcp-cloud-pubsub deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: GCP Cloud PubSub description: Publish log events to GCP Cloud PubSub additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true credentials_json: type: string title: JSON Credentials minLength: 1 description: JSON Credentials ui: display_type: secret-file encoding: type: string description: Dictates how the data will be serialized before storing. enum: - json - text title: Encoding default: text project_id: type: string title: Project ID minLength: 1 description: The Project ID as defined in Google Cloud. topic: type: string title: Topic minLength: 1 description: The name of the topic to publish messages to. required: - credentials_json - encoding - project_id - topic - if: properties: type: enum: - honeycomb-logs deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Honeycomb Logs description: Send log data to Honeycomb additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true api_key: type: string title: Honeycomb API key description: Your Honeycomb API key minLength: 1 maxLength: 512 ui: display_type: secret dataset: type: string title: Dataset description: The name of the targeted dataset. minLength: 1 maxLength: 512 required: - api_key - dataset - if: properties: type: enum: - indexed-search deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Mezmo Indexed Search description: >- Publish log events to Mezmo's data store for Indexed Search capabilities additionalProperties: true properties: search_field: type: string title: Full Text Search Field pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ default: .line maxLength: 256 description: >- If your data reaches this stage as a plain-text string, the entire payload will be indexed for full-text searching. If your data reaches this stage as an object, the field you enter here will be used as the full-text search field. Nested fields are allowed. errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference timestamp_field: type: string title: Timestamp Field pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ default: timestamp maxLength: 256 description: >- If your data reaches this stage as an object, this field represents the timestamp value to be cataloged for searching. If the field does not exists the current time will be used. We accept UNIX timestamps (in milliseconds), ISO8601/RFC3389, and RFC2822 formats. errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference required: - search_field - timestamp_field - if: properties: type: enum: - kafka deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: title: Kafka description: Publish log events to Kafka additionalProperties: type: string allOf: - if: properties: sasl_enabled: enum: - true then: properties: sasl_mechanism: type: string enum: - SCRAM-SHA-512 - SCRAM-SHA-256 - PLAIN title: SASL description: The Kafka SASL/SCRAM mechanism default: SCRAM-SHA-512 sasl_username: type: string title: Username description: The SASL/SCRAM username minLength: 1 sasl_password: type: string title: Password description: The SASL/SCRAM username ui: display_type: secret minLength: 1 required: - sasl_mechanism - sasl_username - sasl_password - if: properties: tls_enabled: enum: - true then: properties: tls_ca_certificate: type: string title: TLS CA Certificate Chain minLength: 1 description: The CA certificate chain in the PEM format ui: display_type: secret-file format: x509_certificate errorMessage: format: Must be a valid PEM or DER certificate tls_verify_certificate: type: boolean title: TLS Verify Certificate description: Verify the TLS certificates default: true required: - tls_verify_certificate properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true encoding: type: string description: The encoding to apply to the data. enum: - json - text title: Encoding default: text compression: type: string description: >- The compression strategy used on the encoded data prior to sending. enum: - gzip - lz4 - snappy - zstd - none title: Compression default: none event_key_field: type: string title: Event Key Field minLength: 1 description: The field in the log whose value is used as Kafka's event key. brokers: type: array title: Brokers uniqueItems: true minItems: 1 description: >- An array of key/val objects for each host/port of a Kafka broker. items: type: object title: Host/Port description: The host and port for the Kafka broker. additionalProperties: false properties: host: title: Host description: The hostname or IP address of the broker. anyOf: - type: string format: ipv4 errorMessage: format: Must be a valid IPv4 address - type: string format: hostname errorMessage: format: Must be a valid hostname port: type: number title: Port description: The port that the broker listens on. required: - host - port topic: type: string title: Topic minLength: 1 description: The name of the topic to publish to. group_id: type: string title: Group ID minLength: 1 description: The consumer group identifier sasl_enabled: type: boolean title: SASL/SCRAM Enabled description: Enable SASL/SCRAM authentication with Kafka default: false tls_enabled: type: boolean title: Use TLS description: Enable TLS during connections to the remote default: false required: - encoding - compression - brokers - topic - if: properties: type: enum: - loki deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Loki description: Publish log events to Loki additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true auth: title: Loki Auth Strategy description: Loki authentication strategy allOf: - if: properties: strategy: enum: - basic then: properties: user: type: string title: User minLength: 1 maxLength: 512 description: The basic authentication user password: type: string title: Password minLength: 1 maxLength: 512 description: The basic authentication password ui: display_type: secret required: - user - password - if: properties: strategy: enum: - bearer then: properties: token: type: string title: Token minLength: 1 maxLength: 1024 description: The bearer token (passed as-is) ui: display_type: secret required: - token properties: strategy: type: string title: Strategy description: The authentication strategy to use (only basic supported) enum: - basic - bearer required: - strategy encoding: type: object title: Encoding description: Configures how event are encoded properties: codec: type: string title: Codec enum: - json - text required: - codec endpoint: type: string title: Endpoint format: url errorMessage: format: Must be a valid URL minLength: 1 maxLength: 512 description: The Loki base URL path: type: string title: Path minLength: 1 maxLength: 512 description: >- The path appended to the Loki base URL(defaults to /loki/api/v1/push) default: /loki/api/v1/push labels: type: array title: Loki Labels description: Key/Value pair used to describe Loki data items: type: object title: Labels description: Label name and values additionalProperties: false properties: label_name: type: string title: Label Name description: The name of the label minLength: 1 maxLength: 512 label_value: type: string title: Label Value description: The value of the label minLength: 1 maxLength: 512 required: - label_name - label_value required: - auth - encoding - endpoint - labels - if: properties: type: enum: - new-relic deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: New Relic description: Publish to New Relic additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true account_id: type: string title: New Relic Account ID description: Account ID for your New Relic account minLength: 1 maxLength: 1024 ui: display_type: secret api: type: string title: API description: New Relic API endpoint (events not supported) enum: - logs - metrics default: logs license_key: type: string title: New Relic License Key description: License key for your New Relic account minLength: 1 maxLength: 1024 ui: display_type: secret required: - account_id - api - license_key - if: properties: type: enum: - prometheus-remote-write deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Prometheus Remote Write description: Publish metrics to Prometheus Remote Write endpoint additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true endpoint: type: string title: Endpoint format: uri errorMessage: format: Must be a valid URI description: >- The full URI to make HTTP requests to. This should include the protocol and host, but can also include the port, path, and any other valid part of a URI. Example: http://example.org:8080/api/v1/push minLength: 1 maxLength: 2000 auth: title: Authentication description: Configures HTTP authentication allOf: - if: properties: strategy: enum: - basic then: properties: user: type: string title: User minLength: 1 maxLength: 512 description: The basic authentication user password: type: string title: Password minLength: 1 maxLength: 512 description: The basic authentication password ui: display_type: secret required: - user - password - if: properties: strategy: enum: - bearer then: properties: token: type: string title: Token minLength: 1 maxLength: 1024 description: The bearer token ui: display_type: secret required: - token properties: strategy: type: string enum: - basic - bearer - none title: Strategy description: The authentication strategy to use default: none required: - strategy required: - endpoint - if: properties: type: enum: - pulsar deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Pulsar description: Publish log events to Apache Pulsar additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true encoding: type: string description: The encoding to apply to the data. enum: - json - text title: Encoding default: text url: type: string title: URL format: url errorMessage: format: Must be a valid URL description: The full Pulsar URL to send data to. topic: type: string title: Topic minLength: 1 description: The name of the Pulsar topic to publish to. required: - encoding - url - topic - if: properties: type: enum: - redis deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Redis description: Publish log events to Redis additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true encoding: type: string description: The encoding to apply to the data. enum: - json - text title: Encoding default: text url: type: string title: URL format: uri errorMessage: format: Must be a valid URL description: The full Redis URL to send data to. key: type: string title: Key minLength: 1 description: The name of the Redis key to write data to. required: - encoding - url - key - if: properties: type: enum: - splunk-hec-logs deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Splunk HTTP Event Collector description: Publish log events to Splunk HTTP Event Collector additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true compression: type: string description: >- The compression strategy used on the encoded data prior to sending. enum: - gzip - none title: Compression default: none url: type: string title: Endpoint format: url errorMessage: format: Must be a valid URL description: >- The base URL for the Splunk instance. The collector path, such as `/services/collector/events`, will be automatically inferred from the destination's configuration. token: type: string title: Token minLength: 1 ui: display_type: secret description: The default token to authenticate to Splunk HEC. tls_verify_certificate: type: boolean title: Verify TLS Certificate description: >- If unchecked, the TLS certificate of the remote host will not be verified. **This is insecure**, but may be useful for testing or when evaluating Splunk in a trial instance. default: true host_field: type: string title: Host Field description: The field that contains the hostname to include in the event. default: metadata.host timestamp_field: type: string title: Timestamp Field description: The field that contains the timestamp to include in the event. default: metadata.time pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ minLength: 1 maxLength: 512 errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: feature_flag: pipeline-splunk-sink-v2 display_type: field-reference source: title: Source description: >- The source of events sent to this sink. This is typically the filename the logs originated from. Use the field path ".metadata.source" to use the upstream source value from a HEC log source additionalProperties: false allOf: - if: properties: value_type: enum: - field then: title: Value properties: value: type: string title: Field value description: The field to use as the value. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference required: - value - if: properties: value_type: enum: - value required: - value_type then: title: Value properties: value: type: string title: Value description: The value. minLength: 1 maxLength: 512 required: - value properties: value_type: title: Value type description: >- Either a value from an event field, a new value input, or none to omit value. oneOf: - label: Value from event field enum: - field - label: New value enum: - value - label: None enum: - none default: none ui: display_type: radio-group value: ui: feature_flag: pipeline-splunk-sink-v2 type: - string - 'null' required: - value_type sourcetype: title: Source Type description: >- The sourcetype of events sent to this sink. Use the field path ".metadata.sourcetype" to use the upstream sourcetype value from a HEC log source additionalProperties: false allOf: - if: properties: value_type: enum: - field then: title: Value properties: value: type: string title: Field value description: The field to use as the value. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference required: - value - if: properties: value_type: enum: - value required: - value_type then: title: Value properties: value: type: string title: Value description: The value. minLength: 1 maxLength: 512 required: - value properties: value_type: title: Value type description: >- Either a value from an event field, a new value input, or none to omit value. oneOf: - label: Value from event field enum: - field - label: New value enum: - value - label: None enum: - none default: none ui: display_type: radio-group value: ui: feature_flag: pipeline-splunk-sink-v2 type: - string - 'null' required: - value_type index: title: Index description: >- The name of the index to send events to. Use the field path ".metadata.index" to use the upstream index value from a HEC log source additionalProperties: false allOf: - if: properties: value_type: enum: - field then: title: Value properties: value: type: string title: Field value description: The field to use as the value. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference required: - value - if: properties: value_type: enum: - value required: - value_type then: title: Value properties: value: type: string title: Value description: The value. minLength: 1 maxLength: 512 required: - value properties: value_type: title: Value type description: >- Either a value from an event field, a new value input, or none to omit value. oneOf: - label: Value from event field enum: - field - label: New value enum: - value - label: None enum: - none default: none ui: display_type: radio-group value: ui: feature_flag: pipeline-splunk-sink-v2 type: - string - 'null' required: - value_type required: - compression - url - token - source - sourcetype - index - if: properties: type: enum: - sumo-logic-logs deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Sumo Logic Logs description: Publish log data to Sumo Logic additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true endpoint: type: string title: Collector Endpoint format: url errorMessage: format: Must be a valid URL minLength: 1 maxLength: 512 description: The Sumo Logic Collector URL that receives the data ui: display_type: secret compression: type: string title: Compression description: The compression strategy (defaults to none). enum: - gzip - none default: none category: type: string minLength: 1 maxLength: 512 title: Category description: >- Category to use when publishing data to Sumo Logic (default = mezmo_default_category) default: mezmo_default_category required: - endpoint - compression - if: properties: type: enum: - sumo-logic-metrics deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Sumo Logic Metrics description: Publish metrics data to Sumo Logic additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true endpoint: type: string title: Collector Endpoint format: url errorMessage: format: Must be a valid URL minLength: 1 maxLength: 512 description: The Sumo Logic Collector URL that receives the data ui: display_type: secret compression: type: string title: Compression description: The compression strategy (defaults to none). enum: - gzip - none default: none category: type: string minLength: 1 maxLength: 512 title: Category description: >- Category to use when publishing data to Sumo Logic (default = mezmo_default_category) default: mezmo_default_category required: - endpoint - compression - if: properties: type: enum: - vector deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Vector description: Publish log events to Vector additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true compression: type: string description: >- The compression strategy used on the encoded data prior to sending. enum: - gzip - none title: Compression default: none vector_ip: type: string title: Vector IP format: ipv4 description: The IP portion of the Vector host vector_port: type: number title: Vector Port description: The port that the Vector host listens on. version: type: string enum: - '1' - '2' title: Version default: '2' description: >- The version of the Vector sink which must match the source version. required: - compression - vector_ip - vector_port - version - if: properties: type: enum: - s3 deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: AWS S3 description: Publish events as objects in AWS S3 additionalProperties: false errorMessage: properties: bucket: must be a valid bucket name tags: must be a valid tag identifier properties: batch_timeout_secs: type: integer title: Batch Timeout (seconds) description: >- The maximum amount of time, in seconds, events will be buffered before being flushed to the destination enum: - 30 - 60 - 90 - 120 - 300 default: 300 ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true auth: type: object title: AWS Authentication description: Configures AWS authentication additionalProperties: false properties: access_key_id: type: string title: Access Key ID minLength: 1 description: The AWS access key id secret_access_key: type: string title: Secret Access Key minLength: 1 description: The AWS secret access key ui: display_type: secret required: - access_key_id - secret_access_key bucket: type: string title: Bucket pattern: (?!(^xn--|.+-s3alias$))^[a-z0-9][a-z0-9-]{1,61}[a-z0-9]$ description: >- The S3 bucket name. Do not include a leading s3:// or a trailing / prefix: type: string title: Prefix description: A prefix to apply to all object key names. default: / ui: display_type: data-template tags: type: array maxItems: 10 title: Tags description: A set of tags (key/value pairs) to apply to the object in S3. items: type: object properties: key: type: string title: Name description: The tag name pattern: ^[A-Za-z0-9_.:=+@/\-\s]*$ minLength: 1 maxLength: 128 value: type: string title: Value description: The tag value default: '' pattern: ^[A-Za-z0-9_.:=+@/\-\s]*$ minLength: 0 maxLength: 256 required: - key - value encoding: type: string description: The encoding to apply to the data. enum: - json - ndjson - text title: Encoding default: text compression: type: string description: >- The compression strategy used on the encoded data prior to sending. enum: - gzip - none title: Compression default: none region: type: string title: Region minLength: 1 description: The name of the AWS region that is targeted. file_consolidation: type: object ui: display_type: collapsable feature_flag: pipeline-node-sink-s3-file-consolidation title: File Consolidation description: >- This sink writes many small files out to S3. Enabling this process will allow the automatic consolidation of these small files into larger files of your choosing. This process will enable upon deployment and run on the chosen interval from `Processing Interval` creating files named `merged_[timestamp].log` where `timestamp` is the time since epoch. *NOTE*: only `text` and `ndjson` files written by the sink are consolidated. additionalProperties: false properties: enabled: type: boolean default: false title: Enabled description: Toggles whether the process is enabled. process_every_seconds: type: integer title: Processing Interval (seconds) description: How often to run the consolidation process. default: 600 minimum: 300 maximum: 3600 requested_size_bytes: type: integer title: Requested File Size (bytes) description: The size of the requested files. default: 500000000 minimum: 50000000 maximum: 10000000000 base_path: type: string title: Base Path default: '' maxLength: 512 description: >- The path from the container to begin recursivelylooking for files to merge. A blank value indicates root.Merged files will only contain data from the respectivefolder. required: - auth - bucket - prefix - encoding - compression - region - if: properties: type: enum: - blackhole deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Blackhole description: Drop data arriving at this destination additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true - if: properties: type: enum: - datadog-logs deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Datadog Logs description: Publish log events to Datadog additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true api_key: type: string title: Datadog API Key description: Datadog log application API key minLength: 1 ui: display_type: secret compression: type: string description: >- The compression strategy used on the encoded data prior to sending. enum: - gzip - none title: Compression default: gzip site: type: string description: The Datadog site (region) to send logs to. enum: - us1 - us3 - us5 - eu1 title: Datadog Site default: us1 required: - api_key - compression - site - if: properties: type: enum: - datadog-metrics deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Datadog Metrics description: >- Publish metrics events to Datadog. Sending custom metrics to Datadog may result in extra charges on your Datadog bill. Please go to https://docs.datadoghq.com/metrics/custom_metrics/ for more information on Datadog custom metrics pricing. additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true api_key: type: string title: Datadog API Key description: Datadog metrics application API key minLength: 1 maxLength: 512 ui: display_type: secret site: type: string description: The Datadog site (region) to send metrics to. enum: - us1 - us3 - us5 - eu1 title: Datadog Site default: us1 required: - api_key - site - if: properties: type: enum: - clickhouse deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Clickhouse description: Deliver log data to the ClickHouse database additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true endpoint: type: string minLength: 1 maxLength: 2048 title: Endpoint description: Endpoint of the Clickhouse server format: uri errorMessage: format: Must be a valid URI auth: title: Auth Strategy description: Authentication strategy allOf: - if: properties: strategy: enum: - basic then: properties: user: type: string title: User minLength: 1 maxLength: 512 description: The basic authentication user password: type: string title: Password minLength: 1 maxLength: 512 description: The basic authentication password ui: display_type: secret required: - user - password - if: properties: strategy: enum: - bearer then: properties: token: type: string title: Token minLength: 1 maxLength: 2048 description: The bearer token to use for authentication ui: display_type: secret required: - token properties: strategy: type: string title: Strategy description: The ES authentication strategy to use enum: - basic - bearer required: - strategy table: type: string title: Table description: The table that data is inserted into pattern: ^[a-zA-Z0-9_]+$ errorMessage: Invalid table name ui: display_type: data-template database: type: string title: Database description: The database that contains the table that data is inserted into pattern: ^[a-zA-Z0-9_]+$ errorMessage: Invalid database name ui: display_type: data-template compression: type: string enum: - gzip - none - snappy - zlib - zstd title: Compression description: The compression algorithm to use default: gzip skip_unknown_fields: type: boolean title: Skip Unknown Fields description: >- Allows ClickHouse to discard fields not present in the table schema default: false required: - endpoint - auth - table - if: properties: type: enum: - opentelemetry deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: OpenTelemetry description: Publish data to OpenTelemetry additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true endpoint: type: string title: Collector Endpoint format: url errorMessage: format: Must be a valid URL minLength: 1 maxLength: 512 description: The OpenTelemetry Collector URL that receives the data auth: title: Authentication description: Configures HTTP authentication additionalProperties: type: string allOf: - if: properties: strategy: enum: - basic then: properties: user: type: string title: User minLength: 1 maxLength: 512 description: The basic authentication user password: type: string title: Password minLength: 1 maxLength: 512 description: The basic authentication password ui: display_type: secret required: - user - password - if: properties: strategy: enum: - bearer then: properties: token: type: string title: Token minLength: 1 maxLength: 1024 description: The bearer token ui: display_type: secret required: - token properties: strategy: type: string enum: - basic - bearer - none title: Strategy description: The authentication strategy to use default: none headers: type: array title: HTTP Headers description: >- An array of key/value objects for the headers to be sent with the request. items: type: object title: Headers description: A key/value object describing a header name and its value. additionalProperties: false properties: header_name: type: string title: Header Name minLength: 1 description: The name of the header. header_value: type: string title: Header Value minLength: 1 description: The value of the header. required: - header_name - header_value compression: type: string title: Compression description: The compression strategy (defaults to none). enum: - gzip - none default: none required: - endpoint - compression properties: title: oneOf: - type: string minLength: 1 maxLength: 512 pattern: ^[^<>"]+$ - type: 'null' errorMessage: invalid title name description: oneOf: - type: string minLength: 1 pattern: ^[^<>"]+$ - type: 'null' errorMessage: invalid description type: type: string enum: - mezmo - mezmo-archive - http - cloudwatch-logs - cloudwatch-metrics - sqs - notification-channel - azure-blob-storage - kinesis-firehose - kinesis-streams - elasticsearch - gcp-cloud-monitoring - gcp-cloud-storage - gcp-cloud-operations - gcp-cloud-pubsub - honeycomb-logs - indexed-search - kafka - loki - new-relic - prometheus-remote-write - pulsar - redis - splunk-hec-logs - sumo-logic-logs - sumo-logic-metrics - vector - s3 - blackhole - datadog-logs - datadog-metrics - clickhouse - opentelemetry user_config: type: object required: - user_config required: - sources - transforms - sinks required: true responses: '200': description: Default Response /v3/pipeline/with-linked-components: post: summary: >- Initialize a pipeline with provided components. The links for all components have to be determined by the caller. tags: - Pipeline Management description: Populates and publishes a new pipeline as specified. requestBody: content: application/json: schema: type: object additionalProperties: false properties: origin: type: string enum: - ui - system default: system pipeline: oneOf: - type: 'null' - type: object properties: id: type: string format: uuid title: oneOf: - type: string minLength: 1 maxLength: 512 pattern: ^[^<>"]+$ - type: 'null' errorMessage: invalid title name sources: type: array minItems: 1 items: type: object allOf: - if: properties: type: enum: - http deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: HTTP description: Receive data from incoming HTTP requests additionalProperties: false properties: capture_metadata: type: boolean title: Capture Metadata description: >- Enable the inclusion of all http headers and query string parameters that were sent from the source. ui: feature_flag: pipeline-capture-metadata display_type: hidden default: true auto_parse: type: object title: Configure automated parsing additionalProperties: false ui: display_type: hidden properties: enabled: type: boolean default: false enabled_event_types: type: array items: type: string enum: - HTTPD_COMBINEDLOG - HTTPD_COMMONLOG - HTTPD_ERRORLOG - SYSLOG5424LINE - SYSLOGLINE - SYSLOGPAMSESSION - CRONLOG - MONGO3_LOG - NAGIOSLOGLINE - POSTGRESQL - RAILS3 - RUBY_LOGGER - REDISLOG - REDISMONLOG - S3_ACCESS_LOG - ELB_ACCESS_LOG - CLOUDFRONT_ACCESS_LOG - CATALINALOG - TOMCATLOG - SQUID3 - BIND9 - HAPROXYTCP - HAPROXYHTTP - BACULA_LOGLINE - BRO_HTTP - BRO_DNS - BRO_CONN - BRO_FILES - NETSCREENSESSIONLOG - CISCO_TAGGED_SYSLOG - CISCOFW104001 - CISCOFW104002 - CISCOFW104003 - CISCOFW104004 - CISCOFW105003 - CISCOFW105004 - CISCOFW105005 - CISCOFW105008 - CISCOFW105009 - CISCOFW106001 - CISCOFW106006_106007_106010 - CISCOFW106014 - CISCOFW106015 - CISCOFW106021 - CISCOFW106023 - CISCOFW106100_2_3 - CISCOFW106100 - CISCOFW304001 - CISCOFW110002 - CISCOFW302010 - CISCOFW302013_302014_302015_302016 - CISCOFW302020_302021 - CISCOFW305011 - CISCOFW313001_313004_313008 - CISCOFW313005 - CISCOFW321001 - CISCOFW402117 - CISCOFW402119 - CISCOFW419001 - CISCOFW419002 - CISCOFW500004 - CISCOFW602303_602304 - CISCOFW710001_710002_710003_710005_710006 - CISCOFW713172 - CISCOFW733100 - SHOREWALL - SFW2 decoding: oneOf: - enum: - bytes - enum: - json - enum: - ndjson - enum: - auto ui: display_type: list errorMessage: must be equal to one of the allowed values description: The decoding method for converting frames into data events. title: Decoding Method default: auto required: - decoding - if: properties: type: enum: - splunk-hec deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Splunk HEC description: Receive Splunk logs additionalProperties: false properties: capture_metadata: type: boolean title: Capture Metadata description: >- Enable the inclusion of all http headers and query string parameters that were sent from the source. ui: feature_flag: pipeline-capture-metadata display_type: hidden default: true auto_parse: type: object title: Configure automated parsing additionalProperties: false ui: display_type: hidden properties: enabled: type: boolean default: false enabled_event_types: type: array items: type: string enum: - HTTPD_COMBINEDLOG - HTTPD_COMMONLOG - HTTPD_ERRORLOG - SYSLOG5424LINE - SYSLOGLINE - SYSLOGPAMSESSION - CRONLOG - MONGO3_LOG - NAGIOSLOGLINE - POSTGRESQL - RAILS3 - RUBY_LOGGER - REDISLOG - REDISMONLOG - S3_ACCESS_LOG - ELB_ACCESS_LOG - CLOUDFRONT_ACCESS_LOG - CATALINALOG - TOMCATLOG - SQUID3 - BIND9 - HAPROXYTCP - HAPROXYHTTP - BACULA_LOGLINE - BRO_HTTP - BRO_DNS - BRO_CONN - BRO_FILES - NETSCREENSESSIONLOG - CISCO_TAGGED_SYSLOG - CISCOFW104001 - CISCOFW104002 - CISCOFW104003 - CISCOFW104004 - CISCOFW105003 - CISCOFW105004 - CISCOFW105005 - CISCOFW105008 - CISCOFW105009 - CISCOFW106001 - CISCOFW106006_106007_106010 - CISCOFW106014 - CISCOFW106015 - CISCOFW106021 - CISCOFW106023 - CISCOFW106100_2_3 - CISCOFW106100 - CISCOFW304001 - CISCOFW110002 - CISCOFW302010 - CISCOFW302013_302014_302015_302016 - CISCOFW302020_302021 - CISCOFW305011 - CISCOFW313001_313004_313008 - CISCOFW313005 - CISCOFW321001 - CISCOFW402117 - CISCOFW402119 - CISCOFW419001 - CISCOFW419002 - CISCOFW500004 - CISCOFW602303_602304 - CISCOFW710001_710002_710003_710005_710006 - CISCOFW713172 - CISCOFW733100 - SHOREWALL - SFW2 - if: properties: type: enum: - s3 deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: AWS S3 description: Collect data from object contents in AWS S3 additionalProperties: false properties: sqs_queue_url: type: string title: SQS Queue URL description: >- The URL of a AWS SQS queue configured to receive S3 bucket notifications for the desired S3 buckets format: uri errorMessage: format: Must be a valid URI auth: type: object title: AWS Authentication description: Configures AWS authentication additionalProperties: false properties: access_key_id: type: string title: Access Key ID minLength: 1 description: The AWS access key id secret_access_key: type: string title: Secret Access Key minLength: 1 description: The AWS secret access key ui: display_type: secret required: - access_key_id - secret_access_key compression: type: string title: Compression description: The compression format of the S3 objects enum: - auto - gzip - none - zstd default: auto region: type: string title: Region minLength: 1 description: The name of the source AWS region. required: - sqs_queue_url - auth - compression - region - if: properties: type: enum: - sqs deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: AWS SQS description: Collect messages from AWS SQS additionalProperties: false properties: queue_url: type: string title: Queue URL description: The URL of an AWS SQS queue format: uri errorMessage: format: Must be a valid URI maxLength: 128 auth: type: object title: AWS Authentication description: Configures AWS authentication additionalProperties: false properties: access_key_id: type: string title: Access Key ID minLength: 1 description: The AWS access key id maxLength: 128 secret_access_key: type: string title: Secret Access Key minLength: 1 description: The AWS secret access key ui: display_type: secret maxLength: 128 required: - access_key_id - secret_access_key region: type: string title: Region minLength: 1 description: The name of the source AWS region. maxLength: 128 required: - queue_url - auth - region - if: properties: type: enum: - demo-logs deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: title: Demo Logs description: Generate log messages for testing and demonstration. additionalProperties: type: array title: Log Lines description: >- A list of user-provided log lines that will be sent through the pipeline infinitely. A maximum of 25 lines is allowed. The lines can be either plain text or ndjson. If ndjson is detected, the encoding for the source will be set to JSON, and lines that are not valid will show up as a visible component error. Otherwise, text encoding will be used. minItems: 1 maxItems: 25 items: type: string minLength: 1 maxLength: 4096 title: Line description: >- A single line of log data, either stringified JSON or plain text. ui: display_type: user-provided-lines allOf: - if: properties: format: enum: - user_provided required: - format then: properties: lines: type: array title: Log Lines description: >- A list of user-provided log lines that will be sent through the pipeline infinitely. A maximum of 25 lines is allowed. The lines can be either plain text or ndjson. If ndjson is detected, the encoding for the source will be set to JSON, and lines that are not valid will show up as a visible component error. Otherwise, text encoding will be used. minItems: 1 maxItems: 25 items: type: string minLength: 1 maxLength: 4096 title: Line description: >- A single line of log data, either stringified JSON or plain text. ui: display_type: user-provided-lines required: - lines - allOf: - errorMessage: '`lines` is only valid when `format` is `user_provided`' if: not: type: object properties: format: enum: - user_provided required: - format then: not: type: object properties: lines: type: array title: Log Lines description: >- A list of user-provided log lines that will be sent through the pipeline infinitely. A maximum of 25 lines is allowed. The lines can be either plain text or ndjson. If ndjson is detected, the encoding for the source will be set to JSON, and lines that are not valid will show up as a visible component error. Otherwise, text encoding will be used. minItems: 1 maxItems: 25 items: type: string minLength: 1 maxLength: 4096 title: Line description: >- A single line of log data, either stringified JSON or plain text. ui: display_type: user-provided-lines required: - lines properties: interval_ms: type: number title: Interval description: >- The amount of time to pause between each demo log. Fractions of a second are accepted to increase throughput. e.g. 500 ms == 120 lines per minute default: 1000 minimum: 500 maximum: 3600000 ui: display_type: milliseconds-to-human-readable format: title: Format description: The format of the log messages. All of the data is generated. ui: display_type: list errorMessage: must be one of the available types of demo data oneOf: - label: HTTP JSON description: HTTP request data in JSON format. enum: - json - label: Sensor JSON description: Data from environment sensors in JSON format. enum: - env_sensor - label: Financial JSON description: Financial data in JSON format. enum: - financial - label: Nginx Requests description: Nginx logs in text format. enum: - nginx - label: Apache Requests description: Apache request logs in text format. enum: - apache_common - label: Apache Errors description: Apache error logs in text format. enum: - apache_error - label: BSD Syslog description: Mock BSD syslog entries in rfc3164 text format. enum: - bsd_syslog - label: Syslog description: Mock syslog entries in rfc5424 text format. enum: - syslog - label: HTTP Metrics description: HTTP-based metrics in the Mezmo metrics JSON format. enum: - http_metrics - label: Generic Metrics description: Generic metrics in the Mezmo metrics JSON format enum: - generic_metrics - label: User-Provided Logs description: >- Custom logs provided by the user. These can either be plain text or serialized JSON. enum: - user_provided - label: Infrastructure Logs description: >- Infrastructure logs consisting of sysdig, nginx, HDFS and spark logs enum: - infrastructure - label: Kubernetes Logs description: Kubernetes logs in JSON format enum: - kubernetes default: json required: - format - if: properties: type: enum: - kinesis-firehose deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Kinesis Firehose description: Receive Kinesis Firehose data additionalProperties: false properties: capture_metadata: type: boolean title: Capture Metadata description: >- Enable the inclusion of all http headers and query string parameters that were sent from the source. ui: feature_flag: pipeline-capture-metadata display_type: hidden default: true auto_parse: type: object title: Configure automated parsing additionalProperties: false ui: display_type: hidden properties: enabled: type: boolean default: false enabled_event_types: type: array items: type: string enum: - HTTPD_COMBINEDLOG - HTTPD_COMMONLOG - HTTPD_ERRORLOG - SYSLOG5424LINE - SYSLOGLINE - SYSLOGPAMSESSION - CRONLOG - MONGO3_LOG - NAGIOSLOGLINE - POSTGRESQL - RAILS3 - RUBY_LOGGER - REDISLOG - REDISMONLOG - S3_ACCESS_LOG - ELB_ACCESS_LOG - CLOUDFRONT_ACCESS_LOG - CATALINALOG - TOMCATLOG - SQUID3 - BIND9 - HAPROXYTCP - HAPROXYHTTP - BACULA_LOGLINE - BRO_HTTP - BRO_DNS - BRO_CONN - BRO_FILES - NETSCREENSESSIONLOG - CISCO_TAGGED_SYSLOG - CISCOFW104001 - CISCOFW104002 - CISCOFW104003 - CISCOFW104004 - CISCOFW105003 - CISCOFW105004 - CISCOFW105005 - CISCOFW105008 - CISCOFW105009 - CISCOFW106001 - CISCOFW106006_106007_106010 - CISCOFW106014 - CISCOFW106015 - CISCOFW106021 - CISCOFW106023 - CISCOFW106100_2_3 - CISCOFW106100 - CISCOFW304001 - CISCOFW110002 - CISCOFW302010 - CISCOFW302013_302014_302015_302016 - CISCOFW302020_302021 - CISCOFW305011 - CISCOFW313001_313004_313008 - CISCOFW313005 - CISCOFW321001 - CISCOFW402117 - CISCOFW402119 - CISCOFW419001 - CISCOFW419002 - CISCOFW500004 - CISCOFW602303_602304 - CISCOFW710001_710002_710003_710005_710006 - CISCOFW713172 - CISCOFW733100 - SHOREWALL - SFW2 format: type: string enum: - json - text default: json title: Data Format description: >- This specifies what the data format will be after it is base64 decoded. If it is JSON, it will be automatically parsed. required: - format - if: properties: type: enum: - fluent deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Fluent (Fluentd, Fluent Bit) description: Receive data from Fluentd or Fluent Bit additionalProperties: false properties: capture_metadata: type: boolean title: Capture Metadata description: >- Enable the inclusion of all http headers and query string parameters that were sent from the source. ui: feature_flag: pipeline-capture-metadata display_type: hidden default: true auto_parse: type: object title: Configure automated parsing additionalProperties: false ui: display_type: hidden properties: enabled: type: boolean default: false enabled_event_types: type: array items: type: string enum: - HTTPD_COMBINEDLOG - HTTPD_COMMONLOG - HTTPD_ERRORLOG - SYSLOG5424LINE - SYSLOGLINE - SYSLOGPAMSESSION - CRONLOG - MONGO3_LOG - NAGIOSLOGLINE - POSTGRESQL - RAILS3 - RUBY_LOGGER - REDISLOG - REDISMONLOG - S3_ACCESS_LOG - ELB_ACCESS_LOG - CLOUDFRONT_ACCESS_LOG - CATALINALOG - TOMCATLOG - SQUID3 - BIND9 - HAPROXYTCP - HAPROXYHTTP - BACULA_LOGLINE - BRO_HTTP - BRO_DNS - BRO_CONN - BRO_FILES - NETSCREENSESSIONLOG - CISCO_TAGGED_SYSLOG - CISCOFW104001 - CISCOFW104002 - CISCOFW104003 - CISCOFW104004 - CISCOFW105003 - CISCOFW105004 - CISCOFW105005 - CISCOFW105008 - CISCOFW105009 - CISCOFW106001 - CISCOFW106006_106007_106010 - CISCOFW106014 - CISCOFW106015 - CISCOFW106021 - CISCOFW106023 - CISCOFW106100_2_3 - CISCOFW106100 - CISCOFW304001 - CISCOFW110002 - CISCOFW302010 - CISCOFW302013_302014_302015_302016 - CISCOFW302020_302021 - CISCOFW305011 - CISCOFW313001_313004_313008 - CISCOFW313005 - CISCOFW321001 - CISCOFW402117 - CISCOFW402119 - CISCOFW419001 - CISCOFW419002 - CISCOFW500004 - CISCOFW602303_602304 - CISCOFW710001_710002_710003_710005_710006 - CISCOFW713172 - CISCOFW733100 - SHOREWALL - SFW2 decoding: type: string description: The decoding method for converting frames into data events. enum: - bytes - json - ndjson title: Decoding Method default: json required: - decoding - if: properties: type: enum: - log-analysis deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Mezmo Log Analysis description: Receive data directly from your Mezmo Log Analysis account additionalProperties: false - if: properties: type: enum: - log-analysis-ingestion deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Mezmo Log Analysis Ingestion description: >- Receive unparsed source data directly from your Mezmo Log Analysis account. additionalProperties: type: string - if: properties: type: enum: - logstash deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Logstash description: Receive Logstash data additionalProperties: false properties: capture_metadata: type: boolean title: Capture Metadata description: >- Enable the inclusion of all http headers and query string parameters that were sent from the source. ui: feature_flag: pipeline-capture-metadata display_type: hidden default: true auto_parse: type: object title: Configure automated parsing additionalProperties: false ui: display_type: hidden properties: enabled: type: boolean default: false enabled_event_types: type: array items: type: string enum: - HTTPD_COMBINEDLOG - HTTPD_COMMONLOG - HTTPD_ERRORLOG - SYSLOG5424LINE - SYSLOGLINE - SYSLOGPAMSESSION - CRONLOG - MONGO3_LOG - NAGIOSLOGLINE - POSTGRESQL - RAILS3 - RUBY_LOGGER - REDISLOG - REDISMONLOG - S3_ACCESS_LOG - ELB_ACCESS_LOG - CLOUDFRONT_ACCESS_LOG - CATALINALOG - TOMCATLOG - SQUID3 - BIND9 - HAPROXYTCP - HAPROXYHTTP - BACULA_LOGLINE - BRO_HTTP - BRO_DNS - BRO_CONN - BRO_FILES - NETSCREENSESSIONLOG - CISCO_TAGGED_SYSLOG - CISCOFW104001 - CISCOFW104002 - CISCOFW104003 - CISCOFW104004 - CISCOFW105003 - CISCOFW105004 - CISCOFW105005 - CISCOFW105008 - CISCOFW105009 - CISCOFW106001 - CISCOFW106006_106007_106010 - CISCOFW106014 - CISCOFW106015 - CISCOFW106021 - CISCOFW106023 - CISCOFW106100_2_3 - CISCOFW106100 - CISCOFW304001 - CISCOFW110002 - CISCOFW302010 - CISCOFW302013_302014_302015_302016 - CISCOFW302020_302021 - CISCOFW305011 - CISCOFW313001_313004_313008 - CISCOFW313005 - CISCOFW321001 - CISCOFW402117 - CISCOFW402119 - CISCOFW419001 - CISCOFW419002 - CISCOFW500004 - CISCOFW602303_602304 - CISCOFW710001_710002_710003_710005_710006 - CISCOFW713172 - CISCOFW733100 - SHOREWALL - SFW2 format: type: string enum: - json - text default: json title: Data Format description: The format of the logstash data required: - format - if: properties: type: enum: - mezmo-agent deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Mezmo Agent description: Receive logs from the Mezmo Agent additionalProperties: false properties: capture_metadata: type: boolean title: Capture Metadata description: >- Enable the inclusion of all http headers and query string parameters that were sent from the source. ui: feature_flag: pipeline-capture-metadata display_type: hidden default: true auto_parse: type: object title: Configure automated parsing additionalProperties: false ui: display_type: hidden properties: enabled: type: boolean default: false enabled_event_types: type: array items: type: string enum: - HTTPD_COMBINEDLOG - HTTPD_COMMONLOG - HTTPD_ERRORLOG - SYSLOG5424LINE - SYSLOGLINE - SYSLOGPAMSESSION - CRONLOG - MONGO3_LOG - NAGIOSLOGLINE - POSTGRESQL - RAILS3 - RUBY_LOGGER - REDISLOG - REDISMONLOG - S3_ACCESS_LOG - ELB_ACCESS_LOG - CLOUDFRONT_ACCESS_LOG - CATALINALOG - TOMCATLOG - SQUID3 - BIND9 - HAPROXYTCP - HAPROXYHTTP - BACULA_LOGLINE - BRO_HTTP - BRO_DNS - BRO_CONN - BRO_FILES - NETSCREENSESSIONLOG - CISCO_TAGGED_SYSLOG - CISCOFW104001 - CISCOFW104002 - CISCOFW104003 - CISCOFW104004 - CISCOFW105003 - CISCOFW105004 - CISCOFW105005 - CISCOFW105008 - CISCOFW105009 - CISCOFW106001 - CISCOFW106006_106007_106010 - CISCOFW106014 - CISCOFW106015 - CISCOFW106021 - CISCOFW106023 - CISCOFW106100_2_3 - CISCOFW106100 - CISCOFW304001 - CISCOFW110002 - CISCOFW302010 - CISCOFW302013_302014_302015_302016 - CISCOFW302020_302021 - CISCOFW305011 - CISCOFW313001_313004_313008 - CISCOFW313005 - CISCOFW321001 - CISCOFW402117 - CISCOFW402119 - CISCOFW419001 - CISCOFW419002 - CISCOFW500004 - CISCOFW602303_602304 - CISCOFW710001_710002_710003_710005_710006 - CISCOFW713172 - CISCOFW733100 - SHOREWALL - SFW2 - if: properties: type: enum: - mezmo-datadog-source deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Datadog Agent description: Send logs and metrics data directly from an installed datadog agent additionalProperties: false properties: capture_metadata: type: boolean title: Capture Metadata description: >- Enable the inclusion of all http headers and query string parameters that were sent from the source. ui: feature_flag: pipeline-capture-metadata display_type: hidden default: true auto_parse: type: object title: Configure automated parsing additionalProperties: false ui: display_type: hidden properties: enabled: type: boolean default: false enabled_event_types: type: array items: type: string enum: - HTTPD_COMBINEDLOG - HTTPD_COMMONLOG - HTTPD_ERRORLOG - SYSLOG5424LINE - SYSLOGLINE - SYSLOGPAMSESSION - CRONLOG - MONGO3_LOG - NAGIOSLOGLINE - POSTGRESQL - RAILS3 - RUBY_LOGGER - REDISLOG - REDISMONLOG - S3_ACCESS_LOG - ELB_ACCESS_LOG - CLOUDFRONT_ACCESS_LOG - CATALINALOG - TOMCATLOG - SQUID3 - BIND9 - HAPROXYTCP - HAPROXYHTTP - BACULA_LOGLINE - BRO_HTTP - BRO_DNS - BRO_CONN - BRO_FILES - NETSCREENSESSIONLOG - CISCO_TAGGED_SYSLOG - CISCOFW104001 - CISCOFW104002 - CISCOFW104003 - CISCOFW104004 - CISCOFW105003 - CISCOFW105004 - CISCOFW105005 - CISCOFW105008 - CISCOFW105009 - CISCOFW106001 - CISCOFW106006_106007_106010 - CISCOFW106014 - CISCOFW106015 - CISCOFW106021 - CISCOFW106023 - CISCOFW106100_2_3 - CISCOFW106100 - CISCOFW304001 - CISCOFW110002 - CISCOFW302010 - CISCOFW302013_302014_302015_302016 - CISCOFW302020_302021 - CISCOFW305011 - CISCOFW313001_313004_313008 - CISCOFW313005 - CISCOFW321001 - CISCOFW402117 - CISCOFW402119 - CISCOFW419001 - CISCOFW419002 - CISCOFW500004 - CISCOFW602303_602304 - CISCOFW710001_710002_710003_710005_710006 - CISCOFW713172 - CISCOFW733100 - SHOREWALL - SFW2 - if: properties: type: enum: - pipeline-data-restoration deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Mezmo Pipeline Data Restoration description: Receive data directly from your Mezmo Pipline Data Restoration additionalProperties: false - if: properties: type: enum: - webhook deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Webhook (WebSub) description: Receive data from incoming webhooks using the WebSub protocol. additionalProperties: false properties: capture_metadata: type: boolean title: Capture Metadata description: >- Enable the inclusion of all http headers and query string parameters that were sent from the source. ui: feature_flag: pipeline-capture-metadata display_type: hidden default: true auto_parse: type: object title: Configure automated parsing additionalProperties: false ui: display_type: hidden properties: enabled: type: boolean default: false enabled_event_types: type: array items: type: string enum: - HTTPD_COMBINEDLOG - HTTPD_COMMONLOG - HTTPD_ERRORLOG - SYSLOG5424LINE - SYSLOGLINE - SYSLOGPAMSESSION - CRONLOG - MONGO3_LOG - NAGIOSLOGLINE - POSTGRESQL - RAILS3 - RUBY_LOGGER - REDISLOG - REDISMONLOG - S3_ACCESS_LOG - ELB_ACCESS_LOG - CLOUDFRONT_ACCESS_LOG - CATALINALOG - TOMCATLOG - SQUID3 - BIND9 - HAPROXYTCP - HAPROXYHTTP - BACULA_LOGLINE - BRO_HTTP - BRO_DNS - BRO_CONN - BRO_FILES - NETSCREENSESSIONLOG - CISCO_TAGGED_SYSLOG - CISCOFW104001 - CISCOFW104002 - CISCOFW104003 - CISCOFW104004 - CISCOFW105003 - CISCOFW105004 - CISCOFW105005 - CISCOFW105008 - CISCOFW105009 - CISCOFW106001 - CISCOFW106006_106007_106010 - CISCOFW106014 - CISCOFW106015 - CISCOFW106021 - CISCOFW106023 - CISCOFW106100_2_3 - CISCOFW106100 - CISCOFW304001 - CISCOFW110002 - CISCOFW302010 - CISCOFW302013_302014_302015_302016 - CISCOFW302020_302021 - CISCOFW305011 - CISCOFW313001_313004_313008 - CISCOFW313005 - CISCOFW321001 - CISCOFW402117 - CISCOFW402119 - CISCOFW419001 - CISCOFW419002 - CISCOFW500004 - CISCOFW602303_602304 - CISCOFW710001_710002_710003_710005_710006 - CISCOFW713172 - CISCOFW733100 - SHOREWALL - SFW2 - if: properties: type: enum: - kafka deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Kafka Source description: Collect events from Kafka compatible brokers additionalProperties: type: string properties: brokers: type: array title: Brokers uniqueItems: true minItems: 1 description: >- An array of key/val objects for each host/port of a Kafka broker. items: type: object title: Host/Port description: The host and port for the Kafka broker. additionalProperties: false properties: host: title: Host description: The hostname or IP address of the broker. anyOf: - type: string format: ipv4 errorMessage: format: Must be a valid IPv4 address - type: string format: hostname maxLength: 512 errorMessage: format: Must be a valid hostname port: type: number minimum: 1 maximum: 65535 title: Port description: The port that the broker listens on. required: - host - port topics: type: array title: Topics items: type: string title: Topic minLength: 1 maxLength: 256 description: The Kafka topic name. minItems: 1 description: List of Kafka topics names to read events from. group_id: type: string title: Consumer Group minLength: 1 description: The consumer group name to be used to consume events from Kafka. tls_enabled: type: boolean title: TLS enabled description: Whether or not to require TLS for outgoing connections. default: true sasl_enabled: type: boolean title: SASL/SCRAM Enabled description: Enable SASL/SCRAM authentication with Kafka default: true decoding_codec: type: string enum: - bytes - json title: Decoding description: Configures how events are decoded from raw bytes default: bytes if: properties: sasl_enabled: enum: - true then: properties: sasl_mechanism: type: string enum: - SCRAM-SHA-512 - SCRAM-SHA-256 - PLAIN title: SASL description: The Kafka SASL/SCRAM mechanism default: PLAIN sasl_username: type: string title: Username description: The SASL/SCRAM username minLength: 1 sasl_password: type: string title: Password description: The SASL/SCRAM username ui: display_type: secret minLength: 1 required: - sasl_mechanism - sasl_username - sasl_password required: - brokers - topics - group_id - decoding_codec - if: properties: type: enum: - azure-event-hub deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Azure Event Hub Source description: Collect events from Azure Event Hub using Azure Kafka Endpoint additionalProperties: false properties: connection_string: type: string title: Connection String minLength: 1 maxLength: 512 description: The Connection String as it appears in hub consumer SAS Policy ui: display_type: secret namespace: type: string title: Namespace minLength: 1 maxLength: 256 description: The Event Hub Namespace topics: type: array title: Event Hub Names items: type: string title: Name minLength: 1 maxLength: 256 description: Azure Event Hub instance name. minItems: 1 description: The list of Azure Event Hub name(s) to read events from. group_id: type: string title: Consumer Group minLength: 1 maxLength: 256 description: The consumer group name that this consumer belongs to. decoding_codec: type: string enum: - bytes - json title: Decoding description: Configures how events are decoded from raw bytes default: bytes required: - connection_string - namespace - topics - group_id - decoding_codec - if: properties: type: enum: - prometheus-remote-write deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Prometheus Remote Write description: Receive prometheus remote write protobuf data from HTTP requests additionalProperties: false properties: capture_metadata: type: boolean title: Capture Metadata description: >- Enable the inclusion of all http headers and query string parameters that were sent from the source. ui: feature_flag: pipeline-capture-metadata display_type: hidden default: true auto_parse: type: object title: Configure automated parsing additionalProperties: false ui: display_type: hidden properties: enabled: type: boolean default: false enabled_event_types: type: array items: type: string enum: - HTTPD_COMBINEDLOG - HTTPD_COMMONLOG - HTTPD_ERRORLOG - SYSLOG5424LINE - SYSLOGLINE - SYSLOGPAMSESSION - CRONLOG - MONGO3_LOG - NAGIOSLOGLINE - POSTGRESQL - RAILS3 - RUBY_LOGGER - REDISLOG - REDISMONLOG - S3_ACCESS_LOG - ELB_ACCESS_LOG - CLOUDFRONT_ACCESS_LOG - CATALINALOG - TOMCATLOG - SQUID3 - BIND9 - HAPROXYTCP - HAPROXYHTTP - BACULA_LOGLINE - BRO_HTTP - BRO_DNS - BRO_CONN - BRO_FILES - NETSCREENSESSIONLOG - CISCO_TAGGED_SYSLOG - CISCOFW104001 - CISCOFW104002 - CISCOFW104003 - CISCOFW104004 - CISCOFW105003 - CISCOFW105004 - CISCOFW105005 - CISCOFW105008 - CISCOFW105009 - CISCOFW106001 - CISCOFW106006_106007_106010 - CISCOFW106014 - CISCOFW106015 - CISCOFW106021 - CISCOFW106023 - CISCOFW106100_2_3 - CISCOFW106100 - CISCOFW304001 - CISCOFW110002 - CISCOFW302010 - CISCOFW302013_302014_302015_302016 - CISCOFW302020_302021 - CISCOFW305011 - CISCOFW313001_313004_313008 - CISCOFW313005 - CISCOFW321001 - CISCOFW402117 - CISCOFW402119 - CISCOFW419001 - CISCOFW419002 - CISCOFW500004 - CISCOFW602303_602304 - CISCOFW710001_710002_710003_710005_710006 - CISCOFW713172 - CISCOFW733100 - SHOREWALL - SFW2 - if: properties: type: enum: - open-telemetry-metrics deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: OpenTelemetry Metrics description: Receive metrics from OpenTelemetry over HTTP additionalProperties: false properties: capture_metadata: type: boolean title: Capture Metadata description: >- Enable the inclusion of all http headers and query string parameters that were sent from the source. ui: feature_flag: pipeline-capture-metadata display_type: hidden default: true auto_parse: type: object title: Configure automated parsing additionalProperties: false ui: display_type: hidden properties: enabled: type: boolean default: false enabled_event_types: type: array items: type: string enum: - HTTPD_COMBINEDLOG - HTTPD_COMMONLOG - HTTPD_ERRORLOG - SYSLOG5424LINE - SYSLOGLINE - SYSLOGPAMSESSION - CRONLOG - MONGO3_LOG - NAGIOSLOGLINE - POSTGRESQL - RAILS3 - RUBY_LOGGER - REDISLOG - REDISMONLOG - S3_ACCESS_LOG - ELB_ACCESS_LOG - CLOUDFRONT_ACCESS_LOG - CATALINALOG - TOMCATLOG - SQUID3 - BIND9 - HAPROXYTCP - HAPROXYHTTP - BACULA_LOGLINE - BRO_HTTP - BRO_DNS - BRO_CONN - BRO_FILES - NETSCREENSESSIONLOG - CISCO_TAGGED_SYSLOG - CISCOFW104001 - CISCOFW104002 - CISCOFW104003 - CISCOFW104004 - CISCOFW105003 - CISCOFW105004 - CISCOFW105005 - CISCOFW105008 - CISCOFW105009 - CISCOFW106001 - CISCOFW106006_106007_106010 - CISCOFW106014 - CISCOFW106015 - CISCOFW106021 - CISCOFW106023 - CISCOFW106100_2_3 - CISCOFW106100 - CISCOFW304001 - CISCOFW110002 - CISCOFW302010 - CISCOFW302013_302014_302015_302016 - CISCOFW302020_302021 - CISCOFW305011 - CISCOFW313001_313004_313008 - CISCOFW313005 - CISCOFW321001 - CISCOFW402117 - CISCOFW402119 - CISCOFW419001 - CISCOFW419002 - CISCOFW500004 - CISCOFW602303_602304 - CISCOFW710001_710002_710003_710005_710006 - CISCOFW713172 - CISCOFW733100 - SHOREWALL - SFW2 - if: properties: type: enum: - open-telemetry-logs deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: OpenTelemetry Logs description: Receive logs from OpenTelemetry over HTTP additionalProperties: false properties: capture_metadata: type: boolean title: Capture Metadata description: >- Enable the inclusion of all http headers and query string parameters that were sent from the source. ui: feature_flag: pipeline-capture-metadata display_type: hidden default: true auto_parse: type: object title: Configure automated parsing additionalProperties: false ui: display_type: hidden properties: enabled: type: boolean default: false enabled_event_types: type: array items: type: string enum: - HTTPD_COMBINEDLOG - HTTPD_COMMONLOG - HTTPD_ERRORLOG - SYSLOG5424LINE - SYSLOGLINE - SYSLOGPAMSESSION - CRONLOG - MONGO3_LOG - NAGIOSLOGLINE - POSTGRESQL - RAILS3 - RUBY_LOGGER - REDISLOG - REDISMONLOG - S3_ACCESS_LOG - ELB_ACCESS_LOG - CLOUDFRONT_ACCESS_LOG - CATALINALOG - TOMCATLOG - SQUID3 - BIND9 - HAPROXYTCP - HAPROXYHTTP - BACULA_LOGLINE - BRO_HTTP - BRO_DNS - BRO_CONN - BRO_FILES - NETSCREENSESSIONLOG - CISCO_TAGGED_SYSLOG - CISCOFW104001 - CISCOFW104002 - CISCOFW104003 - CISCOFW104004 - CISCOFW105003 - CISCOFW105004 - CISCOFW105005 - CISCOFW105008 - CISCOFW105009 - CISCOFW106001 - CISCOFW106006_106007_106010 - CISCOFW106014 - CISCOFW106015 - CISCOFW106021 - CISCOFW106023 - CISCOFW106100_2_3 - CISCOFW106100 - CISCOFW304001 - CISCOFW110002 - CISCOFW302010 - CISCOFW302013_302014_302015_302016 - CISCOFW302020_302021 - CISCOFW305011 - CISCOFW313001_313004_313008 - CISCOFW313005 - CISCOFW321001 - CISCOFW402117 - CISCOFW402119 - CISCOFW419001 - CISCOFW419002 - CISCOFW500004 - CISCOFW602303_602304 - CISCOFW710001_710002_710003_710005_710006 - CISCOFW713172 - CISCOFW733100 - SHOREWALL - SFW2 - if: properties: type: enum: - open-telemetry-traces deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: OpenTelemetry Traces description: Receive traces from OpenTelemetry over HTTP additionalProperties: false properties: capture_metadata: type: boolean title: Capture Metadata description: >- Enable the inclusion of all http headers and query string parameters that were sent from the source. ui: feature_flag: pipeline-capture-metadata display_type: hidden default: true auto_parse: type: object title: Configure automated parsing additionalProperties: false ui: display_type: hidden properties: enabled: type: boolean default: false enabled_event_types: type: array items: type: string enum: - HTTPD_COMBINEDLOG - HTTPD_COMMONLOG - HTTPD_ERRORLOG - SYSLOG5424LINE - SYSLOGLINE - SYSLOGPAMSESSION - CRONLOG - MONGO3_LOG - NAGIOSLOGLINE - POSTGRESQL - RAILS3 - RUBY_LOGGER - REDISLOG - REDISMONLOG - S3_ACCESS_LOG - ELB_ACCESS_LOG - CLOUDFRONT_ACCESS_LOG - CATALINALOG - TOMCATLOG - SQUID3 - BIND9 - HAPROXYTCP - HAPROXYHTTP - BACULA_LOGLINE - BRO_HTTP - BRO_DNS - BRO_CONN - BRO_FILES - NETSCREENSESSIONLOG - CISCO_TAGGED_SYSLOG - CISCOFW104001 - CISCOFW104002 - CISCOFW104003 - CISCOFW104004 - CISCOFW105003 - CISCOFW105004 - CISCOFW105005 - CISCOFW105008 - CISCOFW105009 - CISCOFW106001 - CISCOFW106006_106007_106010 - CISCOFW106014 - CISCOFW106015 - CISCOFW106021 - CISCOFW106023 - CISCOFW106100_2_3 - CISCOFW106100 - CISCOFW304001 - CISCOFW110002 - CISCOFW302010 - CISCOFW302013_302014_302015_302016 - CISCOFW302020_302021 - CISCOFW305011 - CISCOFW313001_313004_313008 - CISCOFW313005 - CISCOFW321001 - CISCOFW402117 - CISCOFW402119 - CISCOFW419001 - CISCOFW419002 - CISCOFW500004 - CISCOFW602303_602304 - CISCOFW710001_710002_710003_710005_710006 - CISCOFW713172 - CISCOFW733100 - SHOREWALL - SFW2 properties: title: oneOf: - type: string minLength: 1 maxLength: 512 pattern: ^[^<>"]+$ - type: 'null' errorMessage: invalid title name description: oneOf: - type: string minLength: 1 pattern: ^[^<>"]+$ - type: 'null' errorMessage: invalid description type: type: string enum: - http - splunk-hec - s3 - sqs - demo-logs - kinesis-firehose - fluent - log-analysis - log-analysis-ingestion - logstash - mezmo-agent - mezmo-datadog-source - pipeline-data-restoration - webhook - kafka - azure-event-hub - prometheus-remote-write - open-telemetry-metrics - open-telemetry-logs - open-telemetry-traces id: type: string pattern: ^[a-zA-Z0-9-_]+$ user_config: type: object required: - id - user_config transforms: type: array minItems: 1 items: type: object allOf: - if: properties: type: enum: - filter deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Filter description: Define condition(s) to include or exclude events from the pipeline additionalProperties: false properties: action: title: Action description: How to handle events matching this criteria oneOf: - label: Allow events matching this criteria enum: - allow - label: Drop events matching this criteria enum: - drop default: allow ui: display_type: radio-group conditional: allOf: - $ref: '#condition-group' title: Conditional description: A group of expressions joined together by an and/or operator. ui: display_type: rules-engine required: - action - conditional - if: properties: type: enum: - sample deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Sample description: >- Sample data at a given rate, retaining only a subset of data events for further processing. additionalProperties: false properties: rate: type: integer title: Rate description: >- The rate at which events will be forwarded, expressed as 1/N. For example, `rate = 10` means 1 out of every 10 events will be forwarded and the rest will be dropped minimum: 2 maximum: 10000 default: 10 always_include: title: Always Include description: >- Certain events may always want to be included (not sampled), regardless of the sample rate. Events matching this criteria will always show up in the results. additionalProperties: type: - string - number dependencies: field: - str_operator str_operator: - field allOf: - if: title: Operator properties: str_operator: type: string enum: - greater - greater_or_equal - less - less_or_equal ui: display_type: list-operators title: Operator description: The comparison operator required: - str_operator then: title: Value properties: value: errorMessage: pattern: Value must be a number type: - string - number title: Value description: The value used in the expression, ie the "operand" minLength: 1 pattern: ^\s*-?(\d*\.)?\d+\s*$ required: - value - if: title: Operator properties: str_operator: type: string enum: - greater - greater_or_equal - less - less_or_equal - equal - contains - is_ip_in_cidr_range - ends_with - starts_with - drain_matches - contains_token_term - contains_token_prefix - percent_change_greater - percent_change_greater_or_equal - percent_change_less - percent_change_less_or_equal - value_change_greater - value_change_greater_or_equal - value_change_less - value_change_less_or_equal ui: display_type: list-operators title: Operator description: The comparison operator required: - str_operator then: title: Value properties: value: type: - string - number title: Value description: The value used in the expression, ie the "operand" minLength: 1 required: - value - if: title: Operator properties: str_operator: type: string enum: - contains - equal - ends_with - starts_with - drain_matches - contains_token_term - contains_token_prefix ui: display_type: list-operators title: Operator required: - str_operator then: title: Case sensitive properties: case_sensitive: type: boolean title: Case sensitive description: Perform case sensitive comparison? default: true properties: field: type: string title: Field description: >- The JSON field to use in a condition to always include in sampling pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference str_operator: type: string enum: - greater - greater_or_equal - less - less_or_equal - equal - contains - is_ip_in_cidr_range - is_metric - is_array - is_boolean - is_empty - is_null - is_number - is_object - is_string - exists - ends_with - starts_with - contains_token_term - contains_token_prefix - regex_match ui: display_type: list-operators title: Operator description: The comparison operator negate: type: boolean default: false title: Negate description: Negate the comparison expression required: - rate - if: properties: type: enum: - drop-fields deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Remove Fields description: Remove fields additionalProperties: false properties: fields: type: array title: Fields description: A list of fields to be removed items: type: string title: Field pattern: >- ^(?:metadata|\.|(?:metadata|message)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ minLength: 1 errorMessage: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields). Note that dropping an event's message or timestamp is not allowed minItems: 1 required: - fields - if: properties: type: enum: - parse deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: title: Parse description: Parse a specified field using the chosen parser additionalProperties: type: object errorMessage: properties: target_field: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) allOf: - if: properties: parser: enum: - parse_key_value then: properties: options: $ref: '#key-value-parser-options' - if: properties: parser: enum: - parse_timestamp then: properties: options: $ref: '#timestamp-parser-options' required: - options - if: properties: parser: enum: - parse_grok then: properties: options: $ref: '#grok-pattern-parser-options' required: - options - if: properties: parser: enum: - parse_regex then: properties: options: allOf: - $ref: '#regex-parser-options' ui: display_type: regex-pattern required: - options - if: properties: parser: enum: - parse_apache_log then: properties: options: $ref: '#apache-parser-options' required: - options - if: properties: parser: enum: - parse_nginx_log then: properties: options: $ref: '#nginx-parser-options' required: - options - if: properties: parser: enum: - parse_cef then: properties: options: $ref: '#cef-parser-options' - if: properties: parser: enum: - parse_csv then: properties: options: $ref: '#csv-parser-options' properties: field: type: string title: Field description: The JSON field whose value should be parsed. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ default: . errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference target_field: oneOf: - type: 'null' - type: string pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ maxLength: 512 errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference title: Target field description: >- The field into which the parsed value should be inserted. Leave blank to insert the parsed data into the original field. default: null parser: oneOf: - label: AWS Cloudfront Access Log group: Line Parsers ui: {} enum: - aws_cloudfront_access_log - label: AWS Cloudwatch Log group: Line Parsers ui: {} enum: - parse_aws_cloudwatch_log_subscription_message - label: AWS Load Balancer Log group: Line Parsers ui: {} enum: - parse_aws_alb_log - label: AWS S3 Access Log group: Line Parsers ui: {} enum: - aws_s3_access_log - label: Apache Log group: Line Parsers ui: {} enum: - parse_apache_log - label: CSV group: Line Parsers ui: {} enum: - parse_csv - label: Common Event Format (CEF) Log group: Line Parsers ui: {} enum: - parse_cef - label: Common Log group: Line Parsers ui: {} enum: - parse_common_log - label: Elasticsearch Log group: Line Parsers ui: {} enum: - elasticsearch_log - label: Golang Log group: Line Parsers ui: {} enum: - golang_log - label: Google Log group: Line Parsers ui: {} enum: - parse_glog - label: Grok Pattern group: Line Parsers ui: {} enum: - parse_grok - label: Integer group: Field Parsers ui: {} enum: - parse_int - label: JSON group: Line Parsers ui: {} enum: - parse_json - label: Key/Value group: Line Parsers ui: {} enum: - parse_key_value - label: Kubernetes Log group: Line Parsers ui: {} enum: - parse_klog - label: Linux Authorization group: Line Parsers ui: {} enum: - parse_linux_authorization - label: Mongo Log group: Line Parsers ui: {} enum: - mongo_log - label: Mongo3 Log group: Line Parsers ui: {} enum: - mongo3_log - label: Nginx Log group: Line Parsers ui: {} enum: - parse_nginx_log - label: PSQL Log group: Line Parsers ui: {} enum: - psql_log - label: Query String group: Field Parsers ui: {} enum: - parse_query_string - label: Regular Expression group: Line Parsers ui: feature_flag: pipeline-parser-regex enum: - parse_regex - label: Syslog group: Line Parsers ui: {} enum: - parse_syslog - label: Timestamp group: Field Parsers ui: {} enum: - parse_timestamp - label: Tokens group: Field Parsers ui: {} enum: - parse_tokens - label: URL group: Field Parsers ui: {} enum: - parse_url - label: User Agent group: Field Parsers ui: {} enum: - parse_user_agent - label: VPC Flow Log group: Line Parsers ui: {} description: Parse VPC logs using default format for fields from v2 to v4 enum: - parse_aws_vpc_flow_log ui: display_type: list errorMessage: must be one of allowed parsers title: Parser description: The kind of parser to use against the input value from "field". required: - field - parser - if: properties: type: enum: - parse-sequentially deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Parse Sequentially description: >- Parse fields using one or more parsers in order, sending out the event on the first successful match. additionalProperties: false errorMessage: properties: target_field: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) properties: field: type: string title: Field description: The field whose value should be parsed. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ default: . maxLength: 512 errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference target_field: oneOf: - type: 'null' - type: string pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ maxLength: 512 errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference title: Target field description: >- The field into which the parsed value should be inserted. Leave blank to insert the parsed data into the original field. default: null parsers: type: array title: List of parsers description: >- The list of parsers to use in order against the input valuefrom "field", short-circuiting on the first successful match. items: type: object allOf: - if: properties: parser: enum: - parse_key_value then: properties: options: $ref: '#key-value-parser-options' - if: properties: parser: enum: - parse_timestamp then: properties: options: $ref: '#timestamp-parser-options' required: - options - if: properties: parser: enum: - parse_grok then: properties: options: $ref: '#grok-pattern-parser-options' required: - options - if: properties: parser: enum: - parse_regex then: properties: options: allOf: - $ref: '#regex-parser-options' ui: display_type: regex-pattern required: - options - if: properties: parser: enum: - parse_apache_log then: properties: options: $ref: '#apache-parser-options' required: - options - if: properties: parser: enum: - parse_nginx_log then: properties: options: $ref: '#nginx-parser-options' required: - options - if: properties: parser: enum: - parse_cef then: properties: options: $ref: '#cef-parser-options' - if: properties: parser: enum: - parse_csv then: properties: options: $ref: '#csv-parser-options' properties: label: oneOf: - type: 'null' - type: string maxLength: 20 title: Title description: >- An arbitrary name you choose to identify the results of this processor for use when connecting them to other components. parser: oneOf: - label: AWS Cloudfront Access Log group: Line Parsers ui: {} enum: - aws_cloudfront_access_log - label: AWS Cloudwatch Log group: Line Parsers ui: {} enum: - parse_aws_cloudwatch_log_subscription_message - label: AWS Load Balancer Log group: Line Parsers ui: {} enum: - parse_aws_alb_log - label: AWS S3 Access Log group: Line Parsers ui: {} enum: - aws_s3_access_log - label: Apache Log group: Line Parsers ui: {} enum: - parse_apache_log - label: CSV group: Line Parsers ui: {} enum: - parse_csv - label: Common Event Format (CEF) Log group: Line Parsers ui: {} enum: - parse_cef - label: Common Log group: Line Parsers ui: {} enum: - parse_common_log - label: Elasticsearch Log group: Line Parsers ui: {} enum: - elasticsearch_log - label: Golang Log group: Line Parsers ui: {} enum: - golang_log - label: Google Log group: Line Parsers ui: {} enum: - parse_glog - label: Grok Pattern group: Line Parsers ui: {} enum: - parse_grok - label: Integer group: Field Parsers ui: {} enum: - parse_int - label: JSON group: Line Parsers ui: {} enum: - parse_json - label: Key/Value group: Line Parsers ui: {} enum: - parse_key_value - label: Kubernetes Log group: Line Parsers ui: {} enum: - parse_klog - label: Linux Authorization group: Line Parsers ui: {} enum: - parse_linux_authorization - label: Mongo Log group: Line Parsers ui: {} enum: - mongo_log - label: Mongo3 Log group: Line Parsers ui: {} enum: - mongo3_log - label: Nginx Log group: Line Parsers ui: {} enum: - parse_nginx_log - label: PSQL Log group: Line Parsers ui: {} enum: - psql_log - label: Query String group: Field Parsers ui: {} enum: - parse_query_string - label: Regular Expression group: Line Parsers ui: feature_flag: pipeline-parser-regex enum: - parse_regex - label: Syslog group: Line Parsers ui: {} enum: - parse_syslog - label: Timestamp group: Field Parsers ui: {} enum: - parse_timestamp - label: Tokens group: Field Parsers ui: {} enum: - parse_tokens - label: URL group: Field Parsers ui: {} enum: - parse_url - label: User Agent group: Field Parsers ui: {} enum: - parse_user_agent - label: VPC Flow Log group: Line Parsers ui: {} description: >- Parse VPC logs using default format for fields from v2 to v4 enum: - parse_aws_vpc_flow_log ui: display_type: list errorMessage: must be one of allowed parsers title: Parser description: >- The kind of parser to use against the input value from "field". _output_name: type: string title: '[INTERNAL]' description: >- Internal-only field that holds a static id for the output_name. This way, labels for outputs can be updated independent of the id. default: '' required: - parser - _output_name additionalProperties: type: object minItems: 1 ui: display_type: multi-output-map unique_parser_and_options: true errorMessage: uniqueItemProperties: parser can not be defined more than once required: - field - parsers - if: properties: type: enum: - route deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Route description: >- Route data based on whether it matches (or doesn't match) logical comparisons additionalProperties: false properties: conditionals: type: array minItems: 1 title: Conditionals description: A list of complete conditionals, including expressions and label items: type: object additionalProperties: false title: Conditional description: A list of conditions and a label properties: conditional: $ref: '#condition-group' label: type: string minLength: 1 maxLength: 255 title: Label description: >- An arbitrary name you choose to identify the results of this processor for use when connecting them to other components. _output_name: type: string title: '[INTERNAL]' description: >- Internal-only field that holds a static id for the output_name. This way, labels for outputs can be updated independent of the id. default: '' required: - conditional - label - _output_name required: - conditionals - if: properties: type: enum: - stringify deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Stringify description: Stringify messages as JSON additionalProperties: false - if: properties: type: enum: - compact-fields deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Compact Fields description: Remove empty values from a list of fields additionalProperties: false properties: fields: type: array title: Fields description: A list of fields to remove empty values from items: type: string title: Field description: Field to remove empty value from pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference minItems: 1 options: type: object additionalProperties: false title: Compact Options description: Additional options for removing empty values from fields properties: compact_array: type: boolean title: Compact Array description: Remove empty arrays from a field default: true compact_object: type: boolean title: Compact Object description: Remove empty objects from a field default: true required: - fields - if: properties: type: enum: - flatten-fields deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Flatten Fields description: >- Flattens the object or array value of a field into a single level representation additionalProperties: false properties: fields: type: array title: Fields description: >- Define specific nested objects you wish to flatten. This will not flatten the entire object, just the specified fields. Leave blank to flatten the entire object items: type: string title: Field description: Field to flatten pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference options: type: object additionalProperties: false title: Flatten Options description: Additional options for flattening fields properties: delimiter: type: string default: _ title: Delimiter description: >- When flattening, the nested field labels will be appended to the higher level field names along with a delimiter to allow understanding of the original structure. The default is an underscore (_) to ensure no collisions with the . notation used for field identifiers minLength: 1 maxLength: 1 - if: properties: type: enum: - dedupe deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Dedupe description: Dedupe (remove duplicates) from the data stream additionalProperties: false properties: number_of_events: type: number description: Number of events to compare across title: Number of Events minimum: 2 maximum: 5000 default: 5000 comparison_type: type: string enum: - Ignore - Match description: >- Match: Only compares across the fields which are specified by the user.Ignore: Compares everything but the fields specified by the user. title: Comparison Type default: Match fields: type: array title: Fields description: A list of fields on which to base deduping items: type: string title: Field pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference minItems: 1 required: - number_of_events - comparison_type - fields - if: properties: type: enum: - encrypt-fields deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: title: Encrypt Field description: Encrypts the value of the provided field additionalProperties: false allOf: - if: properties: algorithm: enum: - AES-256-CFB required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CFB required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CFB required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-OFB required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-OFB required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-OFB required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-CTR required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CTR required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CTR required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-CBC-PKCS7 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CBC-PKCS7 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CBC-PKCS7 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-CBC-ANSIX923 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CBC-ANSIX923 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CBC-ANSIX923 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-CBC-ISO7816 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CBC-ISO7816 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CBC-ISO7816 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-CBC-ISO10126 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CBC-ISO10126 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CBC-ISO10126 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 16 maxLength: 16 required: - key properties: field: type: string title: Field description: >- Field to encrypt. The value of the field must be a primitive (string, number, boolean) pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference algorithm: oneOf: - label: AES-256-CFB (key = 32 characters, iv = 16 characters) description: AES-256-CFB (key = 32 characters, iv = 16 characters) enum: - AES-256-CFB - label: AES-192-CFB (key = 24 characters, iv = 16 characters) description: AES-192-CFB (key = 24 characters, iv = 16 characters) enum: - AES-192-CFB - label: AES-128-CFB (key = 16 characters, iv = 16 characters) description: AES-128-CFB (key = 16 characters, iv = 16 characters) enum: - AES-128-CFB - label: AES-256-OFB (key = 32 characters, iv = 16 characters) description: AES-256-OFB (key = 32 characters, iv = 16 characters) enum: - AES-256-OFB - label: AES-192-OFB (key = 24 characters, iv = 16 characters) description: AES-192-OFB (key = 24 characters, iv = 16 characters) enum: - AES-192-OFB - label: AES-128-OFB (key = 16 characters, iv = 16 characters) description: AES-128-OFB (key = 16 characters, iv = 16 characters) enum: - AES-128-OFB - label: AES-256-CTR (key = 32 characters, iv = 16 characters) description: AES-256-CTR (key = 32 characters, iv = 16 characters) enum: - AES-256-CTR - label: AES-192-CTR (key = 24 characters, iv = 16 characters) description: AES-192-CTR (key = 24 characters, iv = 16 characters) enum: - AES-192-CTR - label: AES-128-CTR (key = 16 characters, iv = 16 characters) description: AES-128-CTR (key = 16 characters, iv = 16 characters) enum: - AES-128-CTR - label: AES-256-CBC-PKCS7 (key = 32 characters, iv = 16 characters) description: AES-256-CBC-PKCS7 (key = 32 characters, iv = 16 characters) enum: - AES-256-CBC-PKCS7 - label: AES-192-CBC-PKCS7 (key = 24 characters, iv = 16 characters) description: AES-192-CBC-PKCS7 (key = 24 characters, iv = 16 characters) enum: - AES-192-CBC-PKCS7 - label: AES-128-CBC-PKCS7 (key = 16 characters, iv = 16 characters) description: AES-128-CBC-PKCS7 (key = 16 characters, iv = 16 characters) enum: - AES-128-CBC-PKCS7 - label: >- AES-256-CBC-ANSIX923 (key = 32 characters, iv = 16 characters) description: >- AES-256-CBC-ANSIX923 (key = 32 characters, iv = 16 characters) enum: - AES-256-CBC-ANSIX923 - label: >- AES-192-CBC-ANSIX923 (key = 24 characters, iv = 16 characters) description: >- AES-192-CBC-ANSIX923 (key = 24 characters, iv = 16 characters) enum: - AES-192-CBC-ANSIX923 - label: >- AES-128-CBC-ANSIX923 (key = 16 characters, iv = 16 characters) description: >- AES-128-CBC-ANSIX923 (key = 16 characters, iv = 16 characters) enum: - AES-128-CBC-ANSIX923 - label: >- AES-256-CBC-ISO7816 (key = 32 characters, iv = 16 characters) description: >- AES-256-CBC-ISO7816 (key = 32 characters, iv = 16 characters) enum: - AES-256-CBC-ISO7816 - label: >- AES-192-CBC-ISO7816 (key = 24 characters, iv = 16 characters) description: >- AES-192-CBC-ISO7816 (key = 24 characters, iv = 16 characters) enum: - AES-192-CBC-ISO7816 - label: >- AES-128-CBC-ISO7816 (key = 16 characters, iv = 16 characters) description: >- AES-128-CBC-ISO7816 (key = 16 characters, iv = 16 characters) enum: - AES-128-CBC-ISO7816 - label: >- AES-256-CBC-ISO10126 (key = 32 characters, iv = 16 characters) description: >- AES-256-CBC-ISO10126 (key = 32 characters, iv = 16 characters) enum: - AES-256-CBC-ISO10126 - label: >- AES-192-CBC-ISO10126 (key = 24 characters, iv = 16 characters) description: >- AES-192-CBC-ISO10126 (key = 24 characters, iv = 16 characters) enum: - AES-192-CBC-ISO10126 - label: >- AES-128-CBC-ISO10126 (key = 16 characters, iv = 16 characters) description: >- AES-128-CBC-ISO10126 (key = 16 characters, iv = 16 characters) enum: - AES-128-CBC-ISO10126 ui: display_type: list title: Encryption algorithm description: The encryption algorithm to use key: type: string title: Encryption key description: The encryption key minLength: 16 maxLength: 32 iv_field: type: string title: Initialization vector (IV) field description: >- The field in which to store the generated initialization vector, IV. Each encrypted value will have a unique IV pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference encode_raw_bytes: type: boolean title: Encode encrypted field and IV as Base 64 text description: >- Encode the encrypted value and generated initialization vector as Base64 text default: true required: - field - algorithm - key - iv_field - if: properties: type: enum: - decrypt-fields deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: title: Decrypt Field additionalProperties: false description: Decrypts the value of the provided field allOf: - if: properties: algorithm: enum: - AES-256-CFB required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CFB required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CFB required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-OFB required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-OFB required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-OFB required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-CTR required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CTR required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CTR required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-CBC-PKCS7 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CBC-PKCS7 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CBC-PKCS7 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-CBC-ANSIX923 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CBC-ANSIX923 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CBC-ANSIX923 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-CBC-ISO7816 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CBC-ISO7816 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CBC-ISO7816 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-CBC-ISO10126 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CBC-ISO10126 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CBC-ISO10126 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 16 maxLength: 16 required: - key properties: field: type: string title: Field description: Field to decrypt. The value of the field must be a string pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference algorithm: oneOf: - label: AES-256-CFB (key = 32 characters, iv = 16 characters) description: AES-256-CFB (key = 32 characters, iv = 16 characters) enum: - AES-256-CFB - label: AES-192-CFB (key = 24 characters, iv = 16 characters) description: AES-192-CFB (key = 24 characters, iv = 16 characters) enum: - AES-192-CFB - label: AES-128-CFB (key = 16 characters, iv = 16 characters) description: AES-128-CFB (key = 16 characters, iv = 16 characters) enum: - AES-128-CFB - label: AES-256-OFB (key = 32 characters, iv = 16 characters) description: AES-256-OFB (key = 32 characters, iv = 16 characters) enum: - AES-256-OFB - label: AES-192-OFB (key = 24 characters, iv = 16 characters) description: AES-192-OFB (key = 24 characters, iv = 16 characters) enum: - AES-192-OFB - label: AES-128-OFB (key = 16 characters, iv = 16 characters) description: AES-128-OFB (key = 16 characters, iv = 16 characters) enum: - AES-128-OFB - label: AES-256-CTR (key = 32 characters, iv = 16 characters) description: AES-256-CTR (key = 32 characters, iv = 16 characters) enum: - AES-256-CTR - label: AES-192-CTR (key = 24 characters, iv = 16 characters) description: AES-192-CTR (key = 24 characters, iv = 16 characters) enum: - AES-192-CTR - label: AES-128-CTR (key = 16 characters, iv = 16 characters) description: AES-128-CTR (key = 16 characters, iv = 16 characters) enum: - AES-128-CTR - label: AES-256-CBC-PKCS7 (key = 32 characters, iv = 16 characters) description: AES-256-CBC-PKCS7 (key = 32 characters, iv = 16 characters) enum: - AES-256-CBC-PKCS7 - label: AES-192-CBC-PKCS7 (key = 24 characters, iv = 16 characters) description: AES-192-CBC-PKCS7 (key = 24 characters, iv = 16 characters) enum: - AES-192-CBC-PKCS7 - label: AES-128-CBC-PKCS7 (key = 16 characters, iv = 16 characters) description: AES-128-CBC-PKCS7 (key = 16 characters, iv = 16 characters) enum: - AES-128-CBC-PKCS7 - label: >- AES-256-CBC-ANSIX923 (key = 32 characters, iv = 16 characters) description: >- AES-256-CBC-ANSIX923 (key = 32 characters, iv = 16 characters) enum: - AES-256-CBC-ANSIX923 - label: >- AES-192-CBC-ANSIX923 (key = 24 characters, iv = 16 characters) description: >- AES-192-CBC-ANSIX923 (key = 24 characters, iv = 16 characters) enum: - AES-192-CBC-ANSIX923 - label: >- AES-128-CBC-ANSIX923 (key = 16 characters, iv = 16 characters) description: >- AES-128-CBC-ANSIX923 (key = 16 characters, iv = 16 characters) enum: - AES-128-CBC-ANSIX923 - label: >- AES-256-CBC-ISO7816 (key = 32 characters, iv = 16 characters) description: >- AES-256-CBC-ISO7816 (key = 32 characters, iv = 16 characters) enum: - AES-256-CBC-ISO7816 - label: >- AES-192-CBC-ISO7816 (key = 24 characters, iv = 16 characters) description: >- AES-192-CBC-ISO7816 (key = 24 characters, iv = 16 characters) enum: - AES-192-CBC-ISO7816 - label: >- AES-128-CBC-ISO7816 (key = 16 characters, iv = 16 characters) description: >- AES-128-CBC-ISO7816 (key = 16 characters, iv = 16 characters) enum: - AES-128-CBC-ISO7816 - label: >- AES-256-CBC-ISO10126 (key = 32 characters, iv = 16 characters) description: >- AES-256-CBC-ISO10126 (key = 32 characters, iv = 16 characters) enum: - AES-256-CBC-ISO10126 - label: >- AES-192-CBC-ISO10126 (key = 24 characters, iv = 16 characters) description: >- AES-192-CBC-ISO10126 (key = 24 characters, iv = 16 characters) enum: - AES-192-CBC-ISO10126 - label: >- AES-128-CBC-ISO10126 (key = 16 characters, iv = 16 characters) description: >- AES-128-CBC-ISO10126 (key = 16 characters, iv = 16 characters) enum: - AES-128-CBC-ISO10126 ui: display_type: list title: Decryption algorithm description: The algorithm with which the data was encrypted key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 16 maxLength: 32 iv_field: type: string title: Initialization vector (IV) field description: The field from which to read the initialization vector, IV pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference decode_raw_bytes: type: boolean title: Decode the encrypted value and IV as Base 64 text description: >- Decode the encrypted value and initialization vector as Base64 text default: true required: - field - algorithm - key - iv_field - if: properties: type: enum: - js-script deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Script Execution description: >- Use JavaScript to reshape and transform your data. You can combine multiple actions like filtering, dropping, mapping, and casting inside of a single js script. additionalProperties: true properties: script: type: string title: Script ui: display_type: code-editor options: language: js description: The JavaScript representation of the event transformation. minLength: 1 maxLength: 10000 default: > // Modify the event using a subset of the JavaScript language. // The function must return the modified event function processEvent(message, metadata, timestamp, annotations) { return message } - if: properties: type: enum: - unroll deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Unroll description: Takes an array of events and emits them all as individual events. additionalProperties: false properties: field: type: string title: Field description: The JSON field name that contains an array of events pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference values_only: type: boolean title: Values only? description: >- When enabled, the values from the specified array field will be emitted as new events. Otherwise, the original event will be duplicated for each value in the array field, with the unrolled value present in the field specified. default: true required: - field - if: properties: type: enum: - vrl deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: VRL description: Modify your observability data using Vector Remap Language (VRL) additionalProperties: false properties: source: type: string title: VRL Source description: VRL code to execute for each event ui: display_type: code-editor options: language: vrl minLength: 1 default: .message = .message required: - source - if: properties: type: enum: - reduce deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Reduce description: >- Combine multiple events over time into one based on a set of criteria additionalProperties: false properties: duration_ms: type: number maximum: 7200000 title: Duration description: >- The amount of time (in milliseconds) to allow streaming events to accumulate into a single "reduced" event. The process repeats indefinitely, or until an "ends when" condition is satisfied. ui: display_type: milliseconds-to-human-readable default: 30000 max_events: type: number minimum: 1 maximum: 100000 title: Max Events description: >- The maximum number of events that can be included in a time window (specified by Duration above). The reduce operation will stop once it has reached this number of events, regardless of whether the Duration seconds have elapsed. group_by: type: array minItems: 1 title: Group By Field Path items: type: string title: Field description: The dot-notated property path errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ description: >- Before reducing, group events based on matching data from each of these field paths. Supports nesting via dot-notation. date_formats: type: array title: Date Formats description: >- Describes which root-level properties are dates, and their expected format. Dot-notation is supported, but nested field lookup paths will be an error. minItems: 1 items: type: object additionalProperties: false properties: field: type: string title: Date Field Path description: >- Specifies a root-level path property that contains a date value. maxLength: 200 errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ format: type: string title: Date Format description: The template describing the date format format: strftime_datetime errorMessage: must be a valid strftime datetime format maxLength: 200 required: - field - format merge_strategies: type: array title: Merge Strategy per Field minItems: 1 description: >- Specify merge strategies for individual root-level properties. Dot-notation is supported, but nested field lookup paths will be an error. items: type: object additionalProperties: false properties: field: type: string title: Field Path description: >- This is a root-level path property to apply a merge strategy to its value maxLength: 200 errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ strategy: oneOf: - label: Array description: Append each value onto an array enum: - array - label: Smallest array description: Keep the smallest array within each reduce time window enum: - shortest_array - label: Largest array description: Keep the largest array within each reduce time window enum: - longest_array - label: Unique Array description: All unique values go into a flattened array enum: - flat_unique - label: Join with space description: Concatenate into a single space-delimited string enum: - concat - label: Join with newline description: Concatenate into a single newline-delimited string enum: - concat_newline - label: Join description: >- Concatenate into a single string without using a delimiter enum: - concat_raw - label: Keep first description: Keep only the first value and discard the rest enum: - discard - label: Keep last description: Keep only the last non-null value enum: - retain - label: Min value description: >- Keep the minimum numeric value within each reduce time window enum: - min - label: Max value description: >- Keep the maximum numeric value within each reduce time window enum: - max - label: Sum description: Add numeric values together enum: - sum ui: display_type: list errorMessage: must be one of the allowed merge strategies title: Merge Strategy description: The merge strategy to be used for the specified property required: - field - strategy flush_condition: title: Event Flush Condition description: >- Force accumulated event reduction to flush the result when a conditional expression evaluates to true on an inbound event. allOf: - type: object properties: when: type: string enum: - none - starts_when - ends_when default: none ui: display_type: list errorMessage: '' title: Flush When required: - when - if: properties: when: not: enum: - none then: properties: conditional: allOf: - $ref: '#condition-group' ui: display_type: rules-engine required: - conditional - if: properties: type: enum: - map-fields deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Map Fields description: Maps data from one field to another, either by moving or copying. additionalProperties: false properties: mappings: type: array title: Mappings description: >- A list of field mappings. Mappings are applied in the order they are defined. minItems: 1 ui: display_type: sortable items: type: object properties: source_field: type: string title: Source field description: The field to copy data from pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference target_field: type: string title: Target field description: The field to copy data into pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference drop_source: type: boolean title: Drop source? description: >- When enabled, the source field is dropped after the data is copied to the target field. Otherwise, it is preserved. default: false overwrite_target: type: boolean title: Overwrite target? description: >- When enabled, any existing data in the target field is overwritten. Otherwise, the target field will be preserved and this mapping will have no effect. default: false required: - source_field - target_field - drop_source - overwrite_target if: properties: drop_source: enum: - true required: - drop_source then: properties: source_field: type: string title: Source field description: The field to copy data from pattern: >- ^(?:metadata|\.|(?:metadata|message)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ minLength: 1 errorMessage: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields). Note that dropping an event's message or timestamp is not allowed required: - source_field required: - mappings - if: properties: type: enum: - metrics-tag-cardinality-limit deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: title: Tag Cardinality Limit (Metrics) description: >- Limits the cardinality of metric events by either dropping events or tags that exceed a specified value limit additionalProperties: false allOf: - if: title: Mode properties: mode: type: string enum: - exact then: title: Value Limit properties: value_limit: type: integer title: Value Limit description: Maximum number of unique values for tags minimum: 1 maximum: 500 required: - value_limit - if: title: Mode properties: mode: type: string enum: - probabilistic then: title: Value Limit properties: value_limit: type: integer title: Value Limit description: Maximum number of unique values for tags minimum: 1 maximum: 5000 required: - value_limit properties: tags: type: array title: Tags description: >- A list of tags to apply cardinality limits. If none are provided, all tags will be considered. items: type: string title: Tag minLength: 1 maxLength: 100 minItems: 0 maxItems: 10 exclude_tags: type: array title: Exclude Tags description: A list of tags to explicitly exclude from cardinality limits. items: type: string title: Tag minLength: 1 maxLength: 100 minItems: 0 maxItems: 10 action: type: string title: Action description: >- The action to take when a tag's cardinality exceeds the value limit enum: - drop_tag - drop_event default: drop_event value_limit: type: integer title: Value Limit description: Maximum number of unique values for tags mode: type: string title: Mode description: The method to used to reduce tag value cardinality enum: - exact - probabilistic default: exact - if: properties: type: enum: - aggregate deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Aggregate description: >- Aggregates multiple events into a single event using either a tumbling interval window or a sliding interval window. additionalProperties: false properties: group_by: type: array minItems: 1 maxItems: 100 title: Group By Field Path description: >- When aggregating, group events based on matching values from each of these field paths. Supports nesting via dot-notation. items: type: string title: Field description: The dot-notated property path errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ ui: default_hint: - .name - .namespace - .tags evaluate: additionalProperties: type: string title: Evaluate description: How events should be aggregated within an interval. allOf: - if: properties: operation: enum: - CUSTOM then: properties: script: type: string title: Script ui: display_type: code-editor options: language: js description: The JavaScript representation of an event aggregation. minLength: 1 maxLength: 2048 default: > // Perform custom aggregation of the current event into the accumulated event. // The accumulated event is persisted across the entire aggregation window. // The function must return the accumulated event, which will be emitted when // the aggregation window elapses. function aggregateEvent(accum, event, metadata, timestamp, annotations) { return accum } required: - script properties: operation: title: Operation oneOf: - label: add description: >- Adds new values together if they are of the same type. For counters and gauges the values are added together. For sets, this is a union of the elements. For distribution, the samples are concatenated. For histograms, values in the same bucket will be added together. enum: - DEFAULT - label: sum description: >- Values are added together. Works for the GAUGE and COUNT metric types. enum: - SUM - label: minimum description: >- Get the minimum value for a series of events. Works for the GAUGE and COUNT metric types. enum: - MIN - label: maximum description: >- Get the maximum value for a series of events. Works for the GAUGE and COUNT metric types. enum: - MAX - label: average description: >- Values are averaged together. Works for the GAUGE and COUNT metric types. enum: - AVG - label: set intersection description: The aggregate produces a set of unique values. enum: - SET_INTERSECTION - label: distribution concatenation description: >- The aggregate produces a list of values combined together. enum: - DIST_CONCAT - label: custom description: Use a custom script to perform aggregation. enum: - CUSTOM ui: display_type: list required: - operation window: default: {} additionalProperties: type: integer title: Window Type description: >- Sliding windows can overlap, whereas tumbling windows are disjoint. For example, a tumbling window has a fixed time span and any events that fall within the "interval" will be used in the aggregate. Whereas in a sliding window, the aggregation occurs every "window duration" seconds after an event is encountered. allOf: - if: properties: type: enum: - sliding required: - type then: properties: window_min: type: integer title: Slide Duration (seconds) description: >- The amount of time to wait before creating a new sliding window for future events. minimum: 1 maximum: 200 default: 5 required: - window_min properties: type: type: string title: Type enum: - tumbling - sliding default: tumbling interval: type: integer title: Interval (seconds) description: The interval over which events are aggregated in seconds. default: 5 minimum: 1 maximum: 90000 required: - type - interval event_timestamp: type: string title: Event Timestamp description: >- The path to a field on the event that contains a epoch timestamp value. If an event does not have a timestamp field, events will be associated to the wall clock value when the event is processed. default: timestamp maxLength: 200 errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ conditional: ui: display_type: rules-engine title: Event Flush Condition description: >- Force accumulated event reduction to flush the result when a conditional expression evaluates to true on an inbound event. allOf: - $ref: '#condition-group' required: - evaluate - window - if: properties: type: enum: - event-to-metric deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Event to Metric description: Allows conversion between arbitrary events and a Metric. additionalProperties: false properties: metric_name: type: string title: Metric Name description: The machine name of the metric to emit. minLength: 1 maxLength: 128 pattern: ^[a-zA-Z][a-zA-Z0-9_:]*$ errorMessage: must be a valid metric name metric_kind: title: Kind description: >- The kind of metric to emit, Absolute or Incremental. Absolute metrics represent a complete value, and will generally replace an existing value for the metric in the target destination. Incremental metrics represent an additive value which is aggregated in the target destination to produce a new value. oneOf: - label: Incremental enum: - incremental - label: Absolute enum: - absolute default: incremental ui: display_type: radio-group metric_type: title: Type oneOf: - label: Counter enum: - counter - label: Sum enum: - sum - label: Gauge enum: - gauge default: counter ui: display_type: list value: title: Value description: The value of this metric. additionalProperties: false allOf: - if: properties: value_type: enum: - field then: title: Value properties: value: type: string title: Field value description: The field to use as the value. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference required: - value - if: properties: value_type: enum: - value then: title: Value properties: value: type: number title: Value description: The value. required: - value properties: value_type: title: Value type description: Either a value from an event field, or a new value input. oneOf: - label: Value from event field enum: - field - label: New value enum: - value default: field ui: display_type: radio-group value: type: - number - string required: - value_type namespace: title: Namespace description: The namespace for this metric. additionalProperties: false allOf: - if: properties: value_type: enum: - field then: title: Value properties: value: type: string title: Field value description: The field to use as the value. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference required: - value - if: properties: value_type: enum: - value required: - value_type then: title: Value properties: value: type: string title: Value description: The value. minLength: 1 maxLength: 128 required: - value properties: value_type: title: Value type description: >- Either a value from an event field, a new value input, or none to omit the namespace. oneOf: - label: Value from event field enum: - field - label: New value enum: - value - label: None enum: - none default: none ui: display_type: radio-group value: type: - string - 'null' required: - value_type tags: type: array maxItems: 10 title: Tags description: A set of tags (also called labels) to apply to the metric event. items: type: object title: Tag description: A tag to apply to the metric event. allOf: - if: properties: value_type: enum: - field then: title: Value properties: value: type: string title: Field value description: >- The field to use as the value of this tag. Note that fields with highly-variable values will result in high-cardinality metrics, which may impact storage or cost in downstream destinations. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference fallback: type: string title: Fallback value description: >- The value to use for this tag when the field value is missing. minLength: 1 maxLength: 512 required: - value - if: properties: value_type: enum: - value then: title: Value properties: value: type: string title: Value description: The value for this tag. minLength: 1 maxLength: 128 required: - value properties: name: type: string title: Name description: The tag name pattern: ^[a-zA-Z][a-zA-Z0-9_]*$ minLength: 1 maxLength: 128 errorMessage: must be a valid metric tag name value_type: title: Value type description: Either a value from an event field, or a new value input. oneOf: - label: Value from event field enum: - field - label: New value enum: - value default: field ui: display_type: radio-group value: type: string required: - name - value_type - value additionalProperties: type: string required: - metric_name - metric_kind - metric_type - value - namespace - if: properties: type: enum: - clustering deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Clustering description: Finds clusters of similar values additionalProperties: false properties: cluster_field: type: string title: Field description: Field to cluster values on pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference default: .message similarity_threshold: type: number title: Similarity threshold description: >- A threshold from 0.0 (not similar) to 1.0 (exactly similar) that determines how close two values need to be to be consider as being part of the same cluster default: 0.4 minimum: 0 maximum: 1 max_clusters: type: integer title: Maximum cluster description: Maximum number of clusters kept cached by the transform default: 1000 minimum: 1 maximum: 2000 max_node_depth: type: integer title: Maximum tree depth description: Maximum depth of the prefix tree default: 4 minimum: 2 maximum: 8 max_children: type: integer title: Maximum children default: 100 minimum: 2 maximum: 200 description: Maximum number of children per node of the prefix tree required: - cluster_field - if: properties: type: enum: - redact deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Redact description: Detect and anonymize sensitive data in logs and metrics additionalProperties: false properties: field: type: string title: Field description: >- Detect patterns in the specified field. If the field is empty, the processor will look for patterns across all the fields. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ default: . maxLength: 256 errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference mask_patterns: type: array title: Redact Patterns description: List of PII patterns to detect and mask minItems: 1 maxItems: 20 items: type: object additionalProperties: type: string title: Redact Pattern description: A PII pattern to detect and/or anonymize allOf: - if: properties: action: enum: - Replace required: - action then: properties: replacement: type: string title: Replacement description: >- The text to replace the detected pattern. Example: 000-00-0000 minLength: 1 maxLength: 512 required: - replacement - if: properties: action: enum: - Hash required: - action then: properties: algorithm: title: Algorithm description: >- Hash and encode (base16 or hex) matched PII data using the specified algorithm oneOf: - label: sha2 (sha-512_256) enum: - sha2 - label: sha3 (sha3-512) enum: - sha3 ui: display_type: list required: - algorithm - if: properties: name: enum: - custom required: - name then: properties: custom_options: allOf: - type: object additionalProperties: false properties: pattern: type: string title: Pattern description: The custom pattern minLength: 1 maxLength: 4096 case_sensitive: type: boolean title: Case sensitive description: Perform case sensitive matching default: true multiline: type: boolean title: Multiline mode description: >- ^ and $ match beginning and end of line respectively default: false match_newline: type: boolean title: Dot all mode description: Dot matches newline character default: false crlf_newline: type: boolean title: Carriage return (CRLF) mode description: "When multiline mode is enabled, newline character is interpreted as \r\n" default: false ignore_whitespace: type: boolean title: Verbose mode description: Ignore whitespace and allow comment lines default: false required: - pattern title: Replace Expression description: >- The regular expression used to find replacements. Named capture groups are supported, e.g. `(?P<key>\w+):(?P<value>.+)`. ui: display_type: regex-pattern required: - custom_options properties: name: title: Pattern description: The PII pattern to match oneOf: - label: Credit card number enum: - credit_card - label: Email address enum: - email_address - label: IPv4 address enum: - ipv4_address - label: US or Canada phone number enum: - us_canada_phone_number - label: US social security number enum: - us_social_security_number - label: Custom pattern ui: feature_flag: pipeline-parser-regex enum: - custom ui: display_type: list action: type: string title: Action description: Action to take on the detected PII data enum: - Hash - Replace - Detect Only default: Replace required: - name - action required: - mask_patterns - if: properties: type: enum: - throttle deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Throttle description: Throttle (rate-limit) events passing through this component additionalProperties: false properties: threshold: type: number minimum: 1 maximum: 10000000 title: Limit description: >- The number of events to allow over the time window. If a Key Field is specified, the limit will be applied to events for each unique value of that field. default: 1 window_ms: type: number minimum: 1000 maximum: 2592000000 title: Window description: The time window over which the configured limit is applied ui: display_type: milliseconds-to-human-readable default: 5000 key_field: type: string title: Key Field description: >- When configured, the value from this field will be used to group events into separate buckets for throttling. If left blank, or if the event does not contain a value in this field, the event will be throttled as part of the default bucket. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference exclude: allOf: - $ref: '#condition-group' title: Exclude description: >- When configured, events matching this condition will not be throttled. ui: display_type: rules-engine required: - threshold - window_ms - if: properties: type: enum: - set-timestamp deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Set Timestamp description: >- Parse a list of fields using a chosen time format. First match will override the default timestamp of the event. additionalProperties: false properties: parsers: type: array title: List of Fields and Parsers description: >- The list of fields and parsers to use in order of priority, short-circuiting on the first successful match. ui: display_type: multi-output-map options: sortable: false items: type: object properties: field: type: string title: Field description: >- The field whose value will be parsed by the chosen date pattern pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ default: . maxLength: 512 errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference options: $ref: '#timestamp-parser-options' required: - field - options minItems: 1 required: - parsers - if: properties: type: enum: - data-profiler deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Data Profiler description: >- Profile the data sent through your pipeline. The system will generate annotations such as event type and total bytes to be used downstream. It will also normalize the app/host/level/line fields before sending the event downstream. A profile of the data including total, events, event size, log composition, and message templates will be generated as well. additionalProperties: false properties: app_fields: type: array title: App Path(s) description: A list of paths to look for the app value minItems: 1 maxItems: 10 uniqueItems: true items: type: string title: App description: The path to the app maxLength: 512 pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference ui: default_hint: - .app - .application - .container host_fields: type: array title: Host Path(s) description: A list of paths to look for the host value minItems: 1 maxItems: 10 uniqueItems: true items: type: string title: Host description: The path to the host maxLength: 512 pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference ui: default_hint: - .host - .hostname level_fields: type: array title: Level Path(s) description: A list of paths to look for the log level minItems: 1 maxItems: 10 uniqueItems: true items: type: string title: Level description: The path to the level maxLength: 512 pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference ui: default_hint: - .level - .log_level line_fields: type: array title: Line Path(s) description: A list of paths to look for the event line minItems: 1 maxItems: 10 uniqueItems: true items: type: string title: Line description: The path to the line maxLength: 512 pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference ui: default_hint: - .line - .message label_fields: type: array title: Label Path(s) description: A list of paths to look for the event labels maxItems: 10 uniqueItems: true items: type: string title: Label description: The path to the labels maxLength: 512 pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference ui: default_hint: - .labels field_summary: type: boolean default: false title: Field Summary description: >- If enabled the profiler will attempt to summarize the values in fields of events passing through the node. ui: display_type: hidden ff_default_value: pipeline-node-transform-field-summary required: - app_fields - host_fields - level_fields - line_fields - if: properties: type: enum: - replace deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: title: Replace description: >- Replace parts of a string using a regular expression or static string additionalProperties: type: object allOf: - if: properties: replacement_type: enum: - regex required: - replacement_type then: properties: options: allOf: - type: object additionalProperties: false properties: pattern: type: string minLength: 1 maxLength: 4096 case_sensitive: type: boolean title: Case sensitive description: Perform case sensitive matching default: true multiline: type: boolean title: Multiline mode description: >- ^ and $ match beginning and end of line respectively default: false match_newline: type: boolean title: Dot all mode description: Dot matches newline character default: false crlf_newline: type: boolean title: Carriage return (CRLF) mode description: "When multiline mode is enabled, newline character is interpreted as \r\n" default: false ignore_whitespace: type: boolean title: Verbose mode description: Ignore whitespace and allow comment lines default: false required: - pattern title: Replace Expression description: >- The regular expression used to find replacements. Named capture groups are supported, e.g. `(?P<key>\w+):(?P<value>.+)`. ui: display_type: regex-pattern replace_with: type: string minLength: 1 maxLength: 4096 title: Replacement String description: >- The replacement string can be static or optionally contain capture group references. Capture groups can be positional or named. For example, to switch the position of two strings in the source string `onetwo`, the pattern could be `(?Pone)(?P.+)`, and the replacement would be `$two$one`. To do the same thing with positional capture groups, a pattern of `(\w{3})(\w{3})`, and a replacement of `$2$1` would have the same result.

To render a literal dollar sign `$` in the replacement, escape it with `$$`. So, if you had a capture group named `price`, to prepend a dollar sign to the replacement, you would use `$$$price`. required: - options - replace_with - if: properties: replacement_type: enum: - string required: - replacement_type then: properties: string_pattern: type: string title: String Pattern minLength: 1 maxLength: 4096 description: >- A static, case-sensitive string, which will be searched and replaced. replace_with: type: string minLength: 1 maxLength: 4096 title: Replacement String description: The static string used in each replacement. required: - string_pattern - replace_with properties: source_field: type: string title: Source Field description: The field that contains a string for replacement to operate on. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ default: . errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference max_times: type: number title: Maximum Replacements description: >- The maximum number of replacements to be made. If not specified, all occurrences will be replaced. minimum: 1 replacement_type: title: Replacement Type description: >- Specifies how the source string will be searched for replacement. ui: display_type: list errorMessage: must be one of the available pattern types oneOf: - label: String description: Replace a specified string one or more times. enum: - string - label: Regular Expression description: Replace using a regular expression. enum: - regex required: - source_field - replacement_type - if: properties: type: enum: - trace-sampling deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: title: Trace Sampling description: >- Performs sampling of traces per a given rate using the Trace ID as the unique identifier. If a trace is rate selected, all spans of the trace are forwarded. If the Trace ID cannot be found, the event is dropped. allOf: - if: properties: sample_type: enum: - head then: properties: rate: type: integer title: Rate description: >- The rate at which events will be forwarded, expressed as 1/N. For example, `rate = 10` means 1 out of every 10 unique traces will be forwarded and all others will be dropped default: 10 minimum: 2 maximum: 100000 parent_span_id_field: type: 'null' conditionals: type: 'null' required: - rate - if: properties: sample_type: enum: - tail then: properties: parent_span_id_field: type: string default: .parent_span_id title: Parent Span ID description: The field to use as the parent span identifier pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference conditionals: type: array minItems: 1 title: Conditionals description: >- A list of complete conditionals, including expressions and a sample rate to maintain items: type: object additionalProperties: false title: Conditional description: A list of conditions and a sample rate to maintain properties: rate: type: integer title: Rate description: >- The rate at which events will be forwarded, expressed as 1/N. For example, `rate = 10` means 1 out of every 10 unique traces will be forwarded and all others will be dropped. A `rate = 1` means keep all traces that match the condition. minimum: 1 maximum: 100000 default: 10 conditional: allOf: - $ref: '#condition-group' title: Conditional description: >- A group of expressions joined together by an and/or operator. ui: display_type: rules-engine _output_name: type: string title: '[INTERNAL]' description: >- Internal-only field that holds a static id for the output_name. This way, labels for outputs can be updated independent of the id. required: - rate - conditional required: - parent_span_id_field - conditionals properties: sample_type: type: string description: The type of trace sampling to apply. enum: - head title: Sample Type default: head trace_id_field: type: string default: .trace_id title: Trace ID description: The field to use as the identifier of a trace. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference required: - sample_type - trace_id_field properties: title: oneOf: - type: string minLength: 1 maxLength: 512 pattern: ^[^<>"]+$ - type: 'null' errorMessage: invalid title name description: oneOf: - type: string minLength: 1 pattern: ^[^<>"]+$ - type: 'null' errorMessage: invalid description type: type: string enum: - filter - sample - drop-fields - parse - parse-sequentially - route - stringify - compact-fields - flatten-fields - dedupe - encrypt-fields - decrypt-fields - js-script - unroll - vrl - reduce - map-fields - metrics-tag-cardinality-limit - aggregate - event-to-metric - clustering - redact - throttle - set-timestamp - data-profiler - replace - trace-sampling id: type: string pattern: ^[a-zA-Z0-9-_]+$ inputs: type: array minItems: 1 items: type: string user_config: type: object required: - id - inputs - user_config sinks: type: array minItems: 1 items: type: object additionalProperties: false allOf: - if: properties: type: enum: - mezmo deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Mezmo Log Analysis description: Send log data to Mezmo Log Analysis additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true mezmo_host: type: string title: Mezmo Host description: The host for your Log Analysis environment default: null minLength: 1 ingestion_key: type: string title: Ingestion Key description: Ingestion key ui: display_type: mezmo-ingestion-key minLength: 1 query: type: object title: Query Parameters default: hostname: mezmo properties: hostname: type: string title: Hostname description: >- Hostname string or template to extract event values to form a hostname. Depending on the type of ingestion, this value is usually set from other places so the default value is often sufficient. If the value in the event is null, empty, or doesn't exist, when using a template then it will be set to "mezmo" default: mezmo minLength: 1 ui: display_type: data-template tags: type: array title: Tags description: >- List of tag strings or templates to attach to logs. Template must resolve to either an array of strings, or a string. Commas are treated as tag delimiters. items: type: string title: Tag description: Tag string or template to attach to logs minLength: 1 maxLength: 512 ui: display_type: data-template ip: type: string title: IP description: IP address template to attach to logs minLength: 1 maxLength: 512 ui: display_type: data-template mac: type: string title: MAC description: MAC address template to attach to logs minLength: 1 maxLength: 512 ui: display_type: data-template message: title: Log Construction Scheme default: scheme: Explicit field selection allOf: - if: properties: scheme: enum: - Explicit field selection then: properties: line: type: string title: Line description: >- Template or field reference to use as the log line. Field reference can point to an object. minLength: 1 maxLength: 1024 meta_field: type: string title: Meta Field description: Field containing the meta object for the log pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference maxLength: 512 app: type: string title: App description: App name template to attach to logs minLength: 1 maxLength: 512 ui: display_type: data-template file: type: string title: File description: File name template to attach to logs minLength: 1 maxLength: 512 ui: display_type: data-template timestamp_field: type: string title: Timestamp description: Field containing the timestamp for the log pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference maxLength: 512 env: type: string title: Env description: Environment name template to attach to logs minLength: 1 maxLength: 512 ui: display_type: data-template properties: scheme: type: string title: Scheme description: How to construct the log message. enum: - Explicit field selection - Message pass-through default: Explicit field selection required: - scheme required: - mezmo_host - ingestion_key - if: properties: type: enum: - mezmo-archive deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Mezmo Archive Destination description: Store logs for future imports into mezmo pipeline or log analysis. additionalProperties: false properties: batch_timeout_secs: type: integer title: Batch Timeout (seconds) description: >- The maximum amount of time, in seconds, events will be buffered before being flushed to the destination enum: - 30 - 60 - 90 - 120 - 300 default: 300 ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true destination: title: Destination description: '' allOf: - if: properties: archive_provider: enum: - Azure then: properties: container_name: type: string title: Container Name minLength: 1 description: The name of the container for blob storage. connection_string: type: string title: Connection String minLength: 1 description: >- A connection string for the account that contains an access key required: - container_name - connection_string - if: properties: archive_provider: enum: - S3 then: properties: access_key_id: type: string title: Access Key ID minLength: 1 description: The AWS access key id secret_access_key: type: string title: Secret Access Key minLength: 1 description: The AWS secret access key ui: display_type: secret bucket: type: string title: Bucket pattern: >- (?!(^xn--|.+-s3alias$))^[a-z0-9][a-z0-9-]{1,61}[a-z0-9]$ description: >- The S3 bucket name. Do not include a leading s3:// or a trailing / region: type: string title: Region minLength: 1 description: The name of the AWS region that is targeted. required: - access_key_id - secret_access_key - bucket - region properties: archive_provider: type: string title: Archive Provider description: The storage destination for your data. enum: - Azure - S3 default: Azure required: - archive_provider required: - destination - if: properties: type: enum: - http deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: HTTP description: Send data to a HTTP endpoint additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true uri: type: string title: URI format: uri errorMessage: format: Must be a valid URI description: >- The full URI to make HTTP requests to. This should include the protocol and host, but can also include the port, path, and any other valid part of a URI. minLength: 1 encoding: type: string description: The encoding to apply to the data. enum: - json - ndjson - text title: Encoding default: text compression: type: string description: >- The compression strategy used on the encoded data prior to sending. enum: - gzip - none title: Compression default: none auth: title: Authentication description: Configures HTTP authentication allOf: - if: properties: strategy: enum: - basic then: properties: user: type: string title: User minLength: 1 description: The basic authentication user password: type: string title: Password minLength: 1 description: The basic authentication password ui: display_type: secret required: - user - password - if: properties: strategy: enum: - bearer then: properties: token: type: string title: Token minLength: 1 description: The bearer token ui: display_type: secret required: - token properties: strategy: type: string enum: - basic - bearer - none title: Strategy description: The authentication strategy to use default: none headers: type: array title: HTTP Headers description: >- An array of key/value objects for the headers to be sent with the request. items: type: object title: Headers description: A key/value object describing a header name and its value. additionalProperties: false properties: header_name: type: string title: Header Name minLength: 1 description: The name of the header. header_value: type: string title: Header Value minLength: 1 description: The value of the header. required: - header_name - header_value advanced_options: type: object ui: display_type: collapsable feature_flag: pipeline-node-sink-http-advanced-options title: Advanced Options description: Settings available for fine-tuning the destination behavior. additionalProperties: false dependencies: payload_prefix: - payload_suffix payload_suffix: - payload_prefix properties: max_bytes: type: integer title: Payload Size description: >- The maximum number of uncompressed bytes when batching data to the destination. minimum: 1 maximum: 2097152 timeout_secs: type: integer title: Timeout Seconds description: The number of seconds before a destination write timeout. minimum: 5 method: type: string enum: - post - put - patch - delete - get - head - options - trace title: HTTP Method description: The HTTP method to use for the destination. payload_prefix: type: string title: Payload Prefix description: >- Add a prefix to the payload. Only used for serialized JSON chunks. This option also requires "Payload Suffix" to form a valid JSON string. examples: - '{"event":"' - '{"extra_prop": true, "our_thing":"' minLength: 1 payload_suffix: type: string title: Payload Suffix description: >- Used in combination with "Payload Prefix" to form valid JSON from the payload. examples: - '}' - '"extra_prop": true }' minLength: 1 proxy: type: object additionalProperties: false title: Proxy Settings description: Configure proxy settings for the destination. properties: enabled: type: boolean title: Enabled description: Turns proxying on/off default: false endpoint: type: string minLength: 1 title: HTTP(S) Endpoint description: HTTP or HTTPS Endpoint to use for traffic. format: uri errorMessage: format: Must be a valid URI hosts_bypass_proxy: type: array minItems: 1 title: Hosts to Bypass description: >- A list of hosts to bypass proxying. Can be specified as a domain name, IP address, or CIDR block. Wildcards are supported as a dot (.) in domain names, or as a star (*) to match all hosts. examples: - 93.184.216.34 - ignorehost.example.com - .ignoreall.example.com - 93.184.216.0/24 items: type: string title: Host description: Domain name, IP address, or CIDR block. minLength: 1 tls_protocols: type: array title: TLS Protocols description: >- A list of ALPN protocols to use during TLS negotiation. They are attempted in the order they appear. items: type: string title: Protocol description: An ALPN protocol name minLength: 1 minItems: 1 rate_limiting: type: object additionalProperties: false title: Rate Limiting description: Settings for controlling rate limiting to the destination. properties: request_limit: type: integer minimum: 1 title: Request Limit description: >- The max number of requests allowed within the specified "Duration Seconds" setting. duration_secs: type: integer default: 1 title: Duration Seconds description: The window of time used to apply "Request Limit." minimum: 1 required: - uri - encoding - compression - if: properties: type: enum: - cloudwatch-logs deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: AWS Cloudwatch Logs description: Publish log events to AWS Cloudwatch Logs additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true auth: type: object title: AWS Authentication description: Configures AWS authentication additionalProperties: false properties: access_key_id: type: string title: Access Key ID minLength: 1 description: The AWS access key id secret_access_key: type: string title: Secret Access Key minLength: 1 description: The AWS secret access key ui: display_type: secret required: - access_key_id - secret_access_key encoding: type: string description: The encoding to apply to the data. enum: - json - text title: Encoding default: text compression: type: string description: >- The compression strategy used on the encoded data prior to sending. enum: - gzip - none title: Compression default: none group_name: type: string title: Group Name minLength: 1 description: The name of the log group for the targeted log stream. region: type: string title: Region minLength: 1 description: The name of the AWS region that is targeted. stream_name: type: string title: Stream Name minLength: 1 description: The name of the targeted log stream. required: - auth - encoding - compression - group_name - region - stream_name - if: properties: type: enum: - cloudwatch-metrics deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: AWS Cloudwatch Metrics description: Send metric events to AWS Cloudwatch Metrics additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true auth: type: object title: AWS Authentication description: Configures AWS authentication additionalProperties: false properties: access_key_id: type: string title: Access Key ID minLength: 1 description: The AWS access key id secret_access_key: type: string title: Secret Access Key minLength: 1 description: The AWS secret access key ui: display_type: secret required: - access_key_id - secret_access_key compression: type: string description: >- The compression strategy used on the encoded data prior to sending. enum: - gzip - none title: Compression default: none namespace: type: string title: Namespace minLength: 1 description: >- Name for the container that will isolate metrics from one another. region: type: string title: Region minLength: 1 description: The name of the AWS region that is targeted. required: - auth - compression - namespace - region - if: properties: type: enum: - sqs deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: AWS Simple Queue Service (SQS) description: Publish log events to AWS Simple Queue Service additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true auth: type: object title: AWS Authentication description: Configures AWS authentication additionalProperties: false properties: access_key_id: type: string title: Access Key ID minLength: 1 description: The AWS access key id secret_access_key: type: string title: Secret Access Key minLength: 1 description: The AWS secret access key ui: display_type: secret required: - access_key_id - secret_access_key encoding: type: string description: The encoding to apply to the data. enum: - json - text title: Encoding default: text sqs_queue_url: type: string title: SQS Queue URL format: url errorMessage: format: Must be a valid URL description: The URL of the SQS queue that the data is publishing to. region: type: string title: Region minLength: 1 description: The name of the AWS region that is targeted. required: - auth - encoding - sqs_queue_url - region - if: properties: type: enum: - notification-channel deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Notification Channel description: Send a Message to a 3rd Party Service. additionalProperties: false properties: service: title: Service description: >- Configure a service to receive the payload. Currently supports Slack, PagerDuty, Webhook or Mezmo (Log Analysis). Where allowed, messagetemplating can be used. To use a template, surround field names with double curly braces, e.g., {{.my_field}} or {{."my-object".thing}}. See [documentation](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) for more examples. additionalProperties: type: string title: URI format: uri errorMessage: format: Must be a valid URI description: >- The full URI where the payload is sent. This should include the protocol and host, but can also include the port, path, and any other valid part of a URI. minLength: 1 allOf: - if: properties: name: enum: - slack required: - name then: properties: uri: type: string title: URI format: uri errorMessage: format: Must be a valid URI description: >- The full URI where the payload is sent. This should include the protocol and host, but can also include the port, path, and any other valid part of a URI. minLength: 1 message_text: type: string title: Message minLength: 1 ui: display_type: data-template options: multiline: true description: The text to be sent as a Slack message. required: - uri - message_text - if: properties: name: enum: - pager_duty required: - name then: properties: uri: type: string title: URI format: uri errorMessage: format: Must be a valid URI description: >- The full URI where the payload is sent. This should include the protocol and host, but can also include the port, path, and any other valid part of a URI. minLength: 1 summary: type: string title: Summary description: Summarize the alert details for PagerDuty. minLength: 1 ui: display_type: data-template severity: type: string enum: - INFO - WARNING - ERROR - CRITICAL title: Severity description: The severity level of the alert. source: type: string title: Source description: The source of the alert. minLength: 1 ui: display_type: data-template routing_key: type: string title: Routing Key description: The service integration routing key. minLength: 1 ui: display_type: secret event_action: type: string enum: - trigger - acknowledge - resolve title: Event Action description: The event action associated with the alert. required: - uri - summary - severity - source - routing_key - event_action - if: properties: name: enum: - webhook required: - name then: properties: uri: type: string title: URI format: uri errorMessage: format: Must be a valid URI description: >- The full URI where the payload is sent. This should include the protocol and host, but can also include the port, path, and any other valid part of a URI. minLength: 1 method: type: string title: HTTP Method description: The HTTP method to use for the request. enum: - post - put - patch - delete - get - head - options - trace ui: default_hint: post message_text: type: string title: Message minLength: 1 ui: display_type: data-template options: multiline: true description: >- The Webhook payload to send. This can be a text string or stringified JSON. If the message can be parsed as JSON, it will be sent as such. headers: type: array title: HTTP Headers description: >- A list of key/value pairs to be included in the HTTP request. items: type: object title: Headers description: >- A key/value object describing a header name and its value. additionalProperties: false properties: header_name: type: string title: Header Name minLength: 1 description: The name of the header. header_value: type: string title: Header Value minLength: 1 maxLength: 1024 description: The value of the header. required: - header_name - header_value maxItems: 20 auth: title: Authentication Options description: Configures HTTP authentication allOf: - if: properties: strategy: enum: - basic then: properties: user: type: string title: User minLength: 1 description: The basic authentication user password: type: string title: Password minLength: 1 description: The basic authentication password ui: display_type: secret required: - user - password - if: properties: strategy: enum: - bearer then: properties: token: type: string title: Token minLength: 1 description: The bearer token ui: display_type: secret required: - token properties: strategy: type: string enum: - basic - bearer - none title: Strategy description: Choose basic or token-based authentication. default: none required: - uri - message_text - if: properties: name: enum: - log_analysis required: - name then: properties: severity: type: string enum: - INFO - WARNING - ERROR - CRITICAL title: Severity description: The severity level of the alert. default: INFO subject: type: string title: Subject description: A string to be used as the subject of the alert. minLength: 1 maxLength: 200 ui: display_type: data-template body: type: string title: Body description: A string to be used as the alert message body. minLength: 1 maxLength: 1024 ui: display_type: data-template ingestion_key: type: string title: Ingestion Key description: >- The key required to communicate to Log Analysis for sending the alert. ui: display_type: mezmo-ingestion-key minLength: 1 required: - severity - subject - body - ingestion_key properties: name: title: Name description: The name of the service that will receive the notification. ui: display_type: list errorMessage: must be a valid service name oneOf: - label: Slack enum: - slack - label: PagerDuty enum: - pager_duty - label: Webhook enum: - webhook - label: Mezmo Log Analysis enum: - log_analysis required: - name throttling: type: object title: Throttling description: >- Only allow a certain number of notifications to be sent within a given time window. properties: window_secs: type: integer title: Window Seconds description: >- The time frame during which the number of notifications (set by the Threshold) is permitted. default: 60 minimum: 1 threshold: type: integer title: Threshold description: >- The maximum number of notifications allowed over the given time window set by "Window Seconds" (default: 1). default: 1 minimum: 1 required: - service - throttling - if: properties: type: enum: - azure-blob-storage deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Azure Blob Storage description: Publish log events to Azure Blob Storage additionalProperties: false properties: batch_timeout_secs: type: integer title: Batch Timeout (seconds) description: >- The maximum amount of time, in seconds, events will be buffered before being flushed to the destination enum: - 30 - 60 - 90 - 120 - 300 default: 300 ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true encoding: type: string description: The encoding to apply to the data. enum: - json - ndjson - text title: Encoding default: text compression: type: string description: >- The compression strategy used on the encoded data prior to sending. enum: - gzip - none title: Compression default: none container_name: type: string title: Container Name minLength: 1 description: The name of the container for blob storage. connection_string: type: string title: Connection String minLength: 1 description: A connection string for the account that contains an access key prefix: type: string title: Prefix minLength: 1 description: A prefix to be applied to all object keys ui: display_type: data-template file_consolidation: type: object ui: display_type: collapsable feature_flag: pipeline-node-sink-azure-blob-file-consolidation title: File Consolidation description: >- This sink writes many small files out to azure blob storage. Enabling this process will allow the automatic consolidation of these small files into larger files of your choosing. This process will enable upon deployment and run on the chosen interval from `Processing Interval` creating files named `merged_[timestamp].log` where `timestamp` is the time since epoch when the actual file was created. The process will recursively access all files under the `Base Path` to handle merging sub-directory logging structures. additionalProperties: false properties: enabled: type: boolean default: false title: Enabled description: Toggles whether the process is enabled. process_every_seconds: type: integer title: Processing Interval (seconds) description: How often to run the consolidation process. default: 600 minimum: 300 maximum: 3600 requested_size_bytes: type: integer title: Requested File Size (bytes) description: The requested size of the consolidated files. default: 500000000 minimum: 50000000 maximum: 10000000000 base_path: type: string title: Base Path default: '' maxLength: 512 description: >- The path from the container to begin recursivelylooking for files to merge. A blank value indicates root.Merged files will only contain data from the respectivefolder. required: - encoding - compression - container_name - connection_string - if: properties: type: enum: - kinesis-firehose deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: AWS Kinesis Data Firehose description: Publish log events to AWS Kinesis Data Firehose additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true auth: type: object title: AWS Authentication description: Configures AWS authentication additionalProperties: false properties: access_key_id: type: string title: Access Key ID minLength: 1 description: The AWS access key id secret_access_key: type: string title: Secret Access Key minLength: 1 description: The AWS secret access key ui: display_type: secret required: - access_key_id - secret_access_key encoding: type: string description: The encoding to apply to the data. enum: - json - text title: Encoding default: text compression: type: string description: >- The compression strategy used on the encoded data prior to sending. enum: - gzip - none title: Compression default: none region: type: string title: Region minLength: 1 description: The name of the AWS region that is targeted. stream_name: type: string title: Stream Name minLength: 1 description: The name of the Kinesis Firehose stream required: - auth - encoding - compression - region - stream_name - if: properties: type: enum: - kinesis-streams deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: AWS Kinesis Streams description: Publish log events to AWS Kinesis Streams additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true auth: type: object title: AWS Authentication description: Configures AWS authentication additionalProperties: false properties: access_key_id: type: string title: Access Key ID minLength: 1 description: The AWS access key id secret_access_key: type: string title: Secret Access Key minLength: 1 description: The AWS secret access key ui: display_type: secret required: - access_key_id - secret_access_key encoding: type: string description: The encoding to apply to the data. enum: - json - text title: Encoding default: text compression: type: string description: >- The compression strategy used on the encoded data prior to sending. enum: - gzip - none title: Compression default: none partition_key_field: type: string title: Partition Key Field pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ description: >- The field in the log whose value is used as Kinesis' partition key. errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference region: type: string title: Region minLength: 1 description: The name of the AWS region that is targeted. stream_name: type: string title: Stream Name minLength: 1 description: The name of the targeted log stream. required: - auth - encoding - compression - region - stream_name - if: properties: type: enum: - elasticsearch deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Elasticsearch description: Publish log events to Elasticsearch additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true compression: type: string description: >- The compression strategy used on the encoded data prior to sending. enum: - gzip - none title: Compression default: none auth: title: ElasticSearch Auth Strategy description: ElasticSearch authentication strategy allOf: - if: properties: strategy: enum: - basic then: properties: user: type: string title: User minLength: 1 maxLength: 512 description: The basic authentication user password: type: string title: Password minLength: 1 maxLength: 512 description: The basic authentication password ui: display_type: secret required: - user - password - if: properties: strategy: enum: - aws then: properties: access_key_id: type: string title: Access Key ID minLength: 1 maxLength: 1024 description: AWS access key ID ui: display_type: secret secret_access_key: type: string title: Secrete Access Key minLength: 1 maxLength: 1024 description: AWS secrete access key ui: display_type: secret region: type: string title: Region minLength: 1 maxLength: 128 description: The name of the AWS region that is targeted. required: - access_key_id - secret_access_key - region properties: strategy: type: string title: Strategy description: The ES authentication strategy to use enum: - basic - aws required: - strategy endpoints: type: array title: Endpoints uniqueItems: true minItems: 1 description: An array of ElasticSearch endpoints items: type: string title: URL format: url errorMessage: format: Must be a valid URL description: The full Elasticsearch URL to send data to. minLength: 1 maxLength: 1024 pipeline: type: string title: Pipeline minLength: 1 description: Name of an ES ingest pipeline to include. index: type: string minLength: 1 maxLength: 512 title: Custom Index description: Index to use when writing ES events (default = mezmo-%Y.%m.%d) default: mezmo-%Y.%m.%d required: - compression - auth - endpoints - if: properties: type: enum: - gcp-cloud-monitoring deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: GCP Cloud Monitoring description: Publish metrics events to GCP Cloud Monitoring additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true credentials_json: type: string title: JSON Credentials minLength: 1 description: JSON Credentials ui: display_type: secret-file project_id: type: string title: Project ID minLength: 1 description: The Project ID as defined in Google Cloud. resource_type: type: string title: Resource Type minLength: 1 description: The `monitored-resource` type as defined in Monitoring. resource_labels: type: array title: Resource Labels description: >- An array of key/val objects describing labels of the Monitoring resource. items: type: object title: Label/Value description: A key/val object describing a label name and its value. additionalProperties: false properties: label_name: type: string title: Label Name minLength: 1 description: The name of the label. label_value: type: string title: Label Value minLength: 1 description: The value of the label. required: - label_name - label_value required: - credentials_json - project_id - resource_type - if: properties: type: enum: - gcp-cloud-storage deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: GCP Cloud Storage description: Publish log events to GCP Cloud Storage additionalProperties: false properties: batch_timeout_secs: type: integer title: Batch Timeout (seconds) description: >- The maximum amount of time, in seconds, events will be buffered before being flushed to the destination enum: - 30 - 60 - 90 - 120 - 300 default: 300 ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true credentials_json: type: string title: JSON Credentials minLength: 1 description: JSON Credentials ui: display_type: secret-file encoding: type: string description: Dictates how the data will be serialized before storing. enum: - json - ndjson - text title: Encoding default: text bucket: type: string title: Bucket minLength: 1 description: The name of the bucket in GCP where the data will be stored. compression: type: string description: >- The compression strategy used on the encoded data prior to sending. enum: - gzip - none title: Compression default: none bucket_prefix: type: string title: Bucket Prefix minLength: 1 description: >- The prefix applied to the bucket name, giving the appearance of having directories. required: - credentials_json - encoding - bucket - compression - if: properties: type: enum: - gcp-cloud-operations deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: GCP Cloud Operations description: Publish log events to GCP Cloud Operations additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true credentials_json: type: string title: JSON Credentials minLength: 1 description: JSON Credentials ui: display_type: secret-file log_id: type: string title: Log ID minLength: 1 description: Concise reference for the log stream name. project_id: type: string title: Project ID minLength: 1 description: The Project ID as defined in Google Cloud. resource_type: type: string title: Resource Type minLength: 1 description: The `monitored-resource` type as defined in Monitoring. resource_labels: type: array title: Resource Labels description: >- An array of key/val objects describing labels of the Monitoring resource. items: type: object title: Label/Value description: A key/val object describing a label name and its value. additionalProperties: false properties: label_name: type: string title: Label Name minLength: 1 description: The name of the label. label_value: type: string title: Label Value minLength: 1 description: The value of the label. required: - label_name - label_value required: - credentials_json - log_id - project_id - resource_type - if: properties: type: enum: - gcp-cloud-pubsub deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: GCP Cloud PubSub description: Publish log events to GCP Cloud PubSub additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true credentials_json: type: string title: JSON Credentials minLength: 1 description: JSON Credentials ui: display_type: secret-file encoding: type: string description: Dictates how the data will be serialized before storing. enum: - json - text title: Encoding default: text project_id: type: string title: Project ID minLength: 1 description: The Project ID as defined in Google Cloud. topic: type: string title: Topic minLength: 1 description: The name of the topic to publish messages to. required: - credentials_json - encoding - project_id - topic - if: properties: type: enum: - honeycomb-logs deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Honeycomb Logs description: Send log data to Honeycomb additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true api_key: type: string title: Honeycomb API key description: Your Honeycomb API key minLength: 1 maxLength: 512 ui: display_type: secret dataset: type: string title: Dataset description: The name of the targeted dataset. minLength: 1 maxLength: 512 required: - api_key - dataset - if: properties: type: enum: - indexed-search deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Mezmo Indexed Search description: >- Publish log events to Mezmo's data store for Indexed Search capabilities additionalProperties: true properties: search_field: type: string title: Full Text Search Field pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ default: .line maxLength: 256 description: >- If your data reaches this stage as a plain-text string, the entire payload will be indexed for full-text searching. If your data reaches this stage as an object, the field you enter here will be used as the full-text search field. Nested fields are allowed. errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference timestamp_field: type: string title: Timestamp Field pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ default: timestamp maxLength: 256 description: >- If your data reaches this stage as an object, this field represents the timestamp value to be cataloged for searching. If the field does not exists the current time will be used. We accept UNIX timestamps (in milliseconds), ISO8601/RFC3389, and RFC2822 formats. errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference required: - search_field - timestamp_field - if: properties: type: enum: - kafka deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: title: Kafka description: Publish log events to Kafka additionalProperties: type: string allOf: - if: properties: sasl_enabled: enum: - true then: properties: sasl_mechanism: type: string enum: - SCRAM-SHA-512 - SCRAM-SHA-256 - PLAIN title: SASL description: The Kafka SASL/SCRAM mechanism default: SCRAM-SHA-512 sasl_username: type: string title: Username description: The SASL/SCRAM username minLength: 1 sasl_password: type: string title: Password description: The SASL/SCRAM username ui: display_type: secret minLength: 1 required: - sasl_mechanism - sasl_username - sasl_password - if: properties: tls_enabled: enum: - true then: properties: tls_ca_certificate: type: string title: TLS CA Certificate Chain minLength: 1 description: The CA certificate chain in the PEM format ui: display_type: secret-file format: x509_certificate errorMessage: format: Must be a valid PEM or DER certificate tls_verify_certificate: type: boolean title: TLS Verify Certificate description: Verify the TLS certificates default: true required: - tls_verify_certificate properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true encoding: type: string description: The encoding to apply to the data. enum: - json - text title: Encoding default: text compression: type: string description: >- The compression strategy used on the encoded data prior to sending. enum: - gzip - lz4 - snappy - zstd - none title: Compression default: none event_key_field: type: string title: Event Key Field minLength: 1 description: The field in the log whose value is used as Kafka's event key. brokers: type: array title: Brokers uniqueItems: true minItems: 1 description: >- An array of key/val objects for each host/port of a Kafka broker. items: type: object title: Host/Port description: The host and port for the Kafka broker. additionalProperties: false properties: host: title: Host description: The hostname or IP address of the broker. anyOf: - type: string format: ipv4 errorMessage: format: Must be a valid IPv4 address - type: string format: hostname errorMessage: format: Must be a valid hostname port: type: number title: Port description: The port that the broker listens on. required: - host - port topic: type: string title: Topic minLength: 1 description: The name of the topic to publish to. group_id: type: string title: Group ID minLength: 1 description: The consumer group identifier sasl_enabled: type: boolean title: SASL/SCRAM Enabled description: Enable SASL/SCRAM authentication with Kafka default: false tls_enabled: type: boolean title: Use TLS description: Enable TLS during connections to the remote default: false required: - encoding - compression - brokers - topic - if: properties: type: enum: - loki deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Loki description: Publish log events to Loki additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true auth: title: Loki Auth Strategy description: Loki authentication strategy allOf: - if: properties: strategy: enum: - basic then: properties: user: type: string title: User minLength: 1 maxLength: 512 description: The basic authentication user password: type: string title: Password minLength: 1 maxLength: 512 description: The basic authentication password ui: display_type: secret required: - user - password - if: properties: strategy: enum: - bearer then: properties: token: type: string title: Token minLength: 1 maxLength: 1024 description: The bearer token (passed as-is) ui: display_type: secret required: - token properties: strategy: type: string title: Strategy description: The authentication strategy to use (only basic supported) enum: - basic - bearer required: - strategy encoding: type: object title: Encoding description: Configures how event are encoded properties: codec: type: string title: Codec enum: - json - text required: - codec endpoint: type: string title: Endpoint format: url errorMessage: format: Must be a valid URL minLength: 1 maxLength: 512 description: The Loki base URL path: type: string title: Path minLength: 1 maxLength: 512 description: >- The path appended to the Loki base URL(defaults to /loki/api/v1/push) default: /loki/api/v1/push labels: type: array title: Loki Labels description: Key/Value pair used to describe Loki data items: type: object title: Labels description: Label name and values additionalProperties: false properties: label_name: type: string title: Label Name description: The name of the label minLength: 1 maxLength: 512 label_value: type: string title: Label Value description: The value of the label minLength: 1 maxLength: 512 required: - label_name - label_value required: - auth - encoding - endpoint - labels - if: properties: type: enum: - new-relic deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: New Relic description: Publish to New Relic additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true account_id: type: string title: New Relic Account ID description: Account ID for your New Relic account minLength: 1 maxLength: 1024 ui: display_type: secret api: type: string title: API description: New Relic API endpoint (events not supported) enum: - logs - metrics default: logs license_key: type: string title: New Relic License Key description: License key for your New Relic account minLength: 1 maxLength: 1024 ui: display_type: secret required: - account_id - api - license_key - if: properties: type: enum: - prometheus-remote-write deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Prometheus Remote Write description: Publish metrics to Prometheus Remote Write endpoint additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true endpoint: type: string title: Endpoint format: uri errorMessage: format: Must be a valid URI description: >- The full URI to make HTTP requests to. This should include the protocol and host, but can also include the port, path, and any other valid part of a URI. Example: http://example.org:8080/api/v1/push minLength: 1 maxLength: 2000 auth: title: Authentication description: Configures HTTP authentication allOf: - if: properties: strategy: enum: - basic then: properties: user: type: string title: User minLength: 1 maxLength: 512 description: The basic authentication user password: type: string title: Password minLength: 1 maxLength: 512 description: The basic authentication password ui: display_type: secret required: - user - password - if: properties: strategy: enum: - bearer then: properties: token: type: string title: Token minLength: 1 maxLength: 1024 description: The bearer token ui: display_type: secret required: - token properties: strategy: type: string enum: - basic - bearer - none title: Strategy description: The authentication strategy to use default: none required: - strategy required: - endpoint - if: properties: type: enum: - pulsar deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Pulsar description: Publish log events to Apache Pulsar additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true encoding: type: string description: The encoding to apply to the data. enum: - json - text title: Encoding default: text url: type: string title: URL format: url errorMessage: format: Must be a valid URL description: The full Pulsar URL to send data to. topic: type: string title: Topic minLength: 1 description: The name of the Pulsar topic to publish to. required: - encoding - url - topic - if: properties: type: enum: - redis deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Redis description: Publish log events to Redis additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true encoding: type: string description: The encoding to apply to the data. enum: - json - text title: Encoding default: text url: type: string title: URL format: uri errorMessage: format: Must be a valid URL description: The full Redis URL to send data to. key: type: string title: Key minLength: 1 description: The name of the Redis key to write data to. required: - encoding - url - key - if: properties: type: enum: - splunk-hec-logs deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Splunk HTTP Event Collector description: Publish log events to Splunk HTTP Event Collector additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true compression: type: string description: >- The compression strategy used on the encoded data prior to sending. enum: - gzip - none title: Compression default: none url: type: string title: Endpoint format: url errorMessage: format: Must be a valid URL description: >- The base URL for the Splunk instance. The collector path, such as `/services/collector/events`, will be automatically inferred from the destination's configuration. token: type: string title: Token minLength: 1 ui: display_type: secret description: The default token to authenticate to Splunk HEC. tls_verify_certificate: type: boolean title: Verify TLS Certificate description: >- If unchecked, the TLS certificate of the remote host will not be verified. **This is insecure**, but may be useful for testing or when evaluating Splunk in a trial instance. default: true host_field: type: string title: Host Field description: The field that contains the hostname to include in the event. default: metadata.host timestamp_field: type: string title: Timestamp Field description: The field that contains the timestamp to include in the event. default: metadata.time pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ minLength: 1 maxLength: 512 errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: feature_flag: pipeline-splunk-sink-v2 display_type: field-reference source: title: Source description: >- The source of events sent to this sink. This is typically the filename the logs originated from. Use the field path ".metadata.source" to use the upstream source value from a HEC log source additionalProperties: false allOf: - if: properties: value_type: enum: - field then: title: Value properties: value: type: string title: Field value description: The field to use as the value. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference required: - value - if: properties: value_type: enum: - value required: - value_type then: title: Value properties: value: type: string title: Value description: The value. minLength: 1 maxLength: 512 required: - value properties: value_type: title: Value type description: >- Either a value from an event field, a new value input, or none to omit value. oneOf: - label: Value from event field enum: - field - label: New value enum: - value - label: None enum: - none default: none ui: display_type: radio-group value: ui: feature_flag: pipeline-splunk-sink-v2 type: - string - 'null' required: - value_type sourcetype: title: Source Type description: >- The sourcetype of events sent to this sink. Use the field path ".metadata.sourcetype" to use the upstream sourcetype value from a HEC log source additionalProperties: false allOf: - if: properties: value_type: enum: - field then: title: Value properties: value: type: string title: Field value description: The field to use as the value. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference required: - value - if: properties: value_type: enum: - value required: - value_type then: title: Value properties: value: type: string title: Value description: The value. minLength: 1 maxLength: 512 required: - value properties: value_type: title: Value type description: >- Either a value from an event field, a new value input, or none to omit value. oneOf: - label: Value from event field enum: - field - label: New value enum: - value - label: None enum: - none default: none ui: display_type: radio-group value: ui: feature_flag: pipeline-splunk-sink-v2 type: - string - 'null' required: - value_type index: title: Index description: >- The name of the index to send events to. Use the field path ".metadata.index" to use the upstream index value from a HEC log source additionalProperties: false allOf: - if: properties: value_type: enum: - field then: title: Value properties: value: type: string title: Field value description: The field to use as the value. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference required: - value - if: properties: value_type: enum: - value required: - value_type then: title: Value properties: value: type: string title: Value description: The value. minLength: 1 maxLength: 512 required: - value properties: value_type: title: Value type description: >- Either a value from an event field, a new value input, or none to omit value. oneOf: - label: Value from event field enum: - field - label: New value enum: - value - label: None enum: - none default: none ui: display_type: radio-group value: ui: feature_flag: pipeline-splunk-sink-v2 type: - string - 'null' required: - value_type required: - compression - url - token - source - sourcetype - index - if: properties: type: enum: - sumo-logic-logs deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Sumo Logic Logs description: Publish log data to Sumo Logic additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true endpoint: type: string title: Collector Endpoint format: url errorMessage: format: Must be a valid URL minLength: 1 maxLength: 512 description: The Sumo Logic Collector URL that receives the data ui: display_type: secret compression: type: string title: Compression description: The compression strategy (defaults to none). enum: - gzip - none default: none category: type: string minLength: 1 maxLength: 512 title: Category description: >- Category to use when publishing data to Sumo Logic (default = mezmo_default_category) default: mezmo_default_category required: - endpoint - compression - if: properties: type: enum: - sumo-logic-metrics deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Sumo Logic Metrics description: Publish metrics data to Sumo Logic additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true endpoint: type: string title: Collector Endpoint format: url errorMessage: format: Must be a valid URL minLength: 1 maxLength: 512 description: The Sumo Logic Collector URL that receives the data ui: display_type: secret compression: type: string title: Compression description: The compression strategy (defaults to none). enum: - gzip - none default: none category: type: string minLength: 1 maxLength: 512 title: Category description: >- Category to use when publishing data to Sumo Logic (default = mezmo_default_category) default: mezmo_default_category required: - endpoint - compression - if: properties: type: enum: - vector deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Vector description: Publish log events to Vector additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true compression: type: string description: >- The compression strategy used on the encoded data prior to sending. enum: - gzip - none title: Compression default: none vector_ip: type: string title: Vector IP format: ipv4 description: The IP portion of the Vector host vector_port: type: number title: Vector Port description: The port that the Vector host listens on. version: type: string enum: - '1' - '2' title: Version default: '2' description: >- The version of the Vector sink which must match the source version. required: - compression - vector_ip - vector_port - version - if: properties: type: enum: - s3 deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: AWS S3 description: Publish events as objects in AWS S3 additionalProperties: false errorMessage: properties: bucket: must be a valid bucket name tags: must be a valid tag identifier properties: batch_timeout_secs: type: integer title: Batch Timeout (seconds) description: >- The maximum amount of time, in seconds, events will be buffered before being flushed to the destination enum: - 30 - 60 - 90 - 120 - 300 default: 300 ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true auth: type: object title: AWS Authentication description: Configures AWS authentication additionalProperties: false properties: access_key_id: type: string title: Access Key ID minLength: 1 description: The AWS access key id secret_access_key: type: string title: Secret Access Key minLength: 1 description: The AWS secret access key ui: display_type: secret required: - access_key_id - secret_access_key bucket: type: string title: Bucket pattern: (?!(^xn--|.+-s3alias$))^[a-z0-9][a-z0-9-]{1,61}[a-z0-9]$ description: >- The S3 bucket name. Do not include a leading s3:// or a trailing / prefix: type: string title: Prefix description: A prefix to apply to all object key names. default: / ui: display_type: data-template tags: type: array maxItems: 10 title: Tags description: A set of tags (key/value pairs) to apply to the object in S3. items: type: object properties: key: type: string title: Name description: The tag name pattern: ^[A-Za-z0-9_.:=+@/\-\s]*$ minLength: 1 maxLength: 128 value: type: string title: Value description: The tag value default: '' pattern: ^[A-Za-z0-9_.:=+@/\-\s]*$ minLength: 0 maxLength: 256 required: - key - value encoding: type: string description: The encoding to apply to the data. enum: - json - ndjson - text title: Encoding default: text compression: type: string description: >- The compression strategy used on the encoded data prior to sending. enum: - gzip - none title: Compression default: none region: type: string title: Region minLength: 1 description: The name of the AWS region that is targeted. file_consolidation: type: object ui: display_type: collapsable feature_flag: pipeline-node-sink-s3-file-consolidation title: File Consolidation description: >- This sink writes many small files out to S3. Enabling this process will allow the automatic consolidation of these small files into larger files of your choosing. This process will enable upon deployment and run on the chosen interval from `Processing Interval` creating files named `merged_[timestamp].log` where `timestamp` is the time since epoch. *NOTE*: only `text` and `ndjson` files written by the sink are consolidated. additionalProperties: false properties: enabled: type: boolean default: false title: Enabled description: Toggles whether the process is enabled. process_every_seconds: type: integer title: Processing Interval (seconds) description: How often to run the consolidation process. default: 600 minimum: 300 maximum: 3600 requested_size_bytes: type: integer title: Requested File Size (bytes) description: The size of the requested files. default: 500000000 minimum: 50000000 maximum: 10000000000 base_path: type: string title: Base Path default: '' maxLength: 512 description: >- The path from the container to begin recursivelylooking for files to merge. A blank value indicates root.Merged files will only contain data from the respectivefolder. required: - auth - bucket - prefix - encoding - compression - region - if: properties: type: enum: - blackhole deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Blackhole description: Drop data arriving at this destination additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true - if: properties: type: enum: - datadog-logs deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Datadog Logs description: Publish log events to Datadog additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true api_key: type: string title: Datadog API Key description: Datadog log application API key minLength: 1 ui: display_type: secret compression: type: string description: >- The compression strategy used on the encoded data prior to sending. enum: - gzip - none title: Compression default: gzip site: type: string description: The Datadog site (region) to send logs to. enum: - us1 - us3 - us5 - eu1 title: Datadog Site default: us1 required: - api_key - compression - site - if: properties: type: enum: - datadog-metrics deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Datadog Metrics description: >- Publish metrics events to Datadog. Sending custom metrics to Datadog may result in extra charges on your Datadog bill. Please go to https://docs.datadoghq.com/metrics/custom_metrics/ for more information on Datadog custom metrics pricing. additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true api_key: type: string title: Datadog API Key description: Datadog metrics application API key minLength: 1 maxLength: 512 ui: display_type: secret site: type: string description: The Datadog site (region) to send metrics to. enum: - us1 - us3 - us5 - eu1 title: Datadog Site default: us1 required: - api_key - site - if: properties: type: enum: - clickhouse deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Clickhouse description: Deliver log data to the ClickHouse database additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true endpoint: type: string minLength: 1 maxLength: 2048 title: Endpoint description: Endpoint of the Clickhouse server format: uri errorMessage: format: Must be a valid URI auth: title: Auth Strategy description: Authentication strategy allOf: - if: properties: strategy: enum: - basic then: properties: user: type: string title: User minLength: 1 maxLength: 512 description: The basic authentication user password: type: string title: Password minLength: 1 maxLength: 512 description: The basic authentication password ui: display_type: secret required: - user - password - if: properties: strategy: enum: - bearer then: properties: token: type: string title: Token minLength: 1 maxLength: 2048 description: The bearer token to use for authentication ui: display_type: secret required: - token properties: strategy: type: string title: Strategy description: The ES authentication strategy to use enum: - basic - bearer required: - strategy table: type: string title: Table description: The table that data is inserted into pattern: ^[a-zA-Z0-9_]+$ errorMessage: Invalid table name ui: display_type: data-template database: type: string title: Database description: The database that contains the table that data is inserted into pattern: ^[a-zA-Z0-9_]+$ errorMessage: Invalid database name ui: display_type: data-template compression: type: string enum: - gzip - none - snappy - zlib - zstd title: Compression description: The compression algorithm to use default: gzip skip_unknown_fields: type: boolean title: Skip Unknown Fields description: >- Allows ClickHouse to discard fields not present in the table schema default: false required: - endpoint - auth - table - if: properties: type: enum: - opentelemetry deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: OpenTelemetry description: Publish data to OpenTelemetry additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true endpoint: type: string title: Collector Endpoint format: url errorMessage: format: Must be a valid URL minLength: 1 maxLength: 512 description: The OpenTelemetry Collector URL that receives the data auth: title: Authentication description: Configures HTTP authentication additionalProperties: type: string allOf: - if: properties: strategy: enum: - basic then: properties: user: type: string title: User minLength: 1 maxLength: 512 description: The basic authentication user password: type: string title: Password minLength: 1 maxLength: 512 description: The basic authentication password ui: display_type: secret required: - user - password - if: properties: strategy: enum: - bearer then: properties: token: type: string title: Token minLength: 1 maxLength: 1024 description: The bearer token ui: display_type: secret required: - token properties: strategy: type: string enum: - basic - bearer - none title: Strategy description: The authentication strategy to use default: none headers: type: array title: HTTP Headers description: >- An array of key/value objects for the headers to be sent with the request. items: type: object title: Headers description: A key/value object describing a header name and its value. additionalProperties: false properties: header_name: type: string title: Header Name minLength: 1 description: The name of the header. header_value: type: string title: Header Value minLength: 1 description: The value of the header. required: - header_name - header_value compression: type: string title: Compression description: The compression strategy (defaults to none). enum: - gzip - none default: none required: - endpoint - compression properties: title: oneOf: - type: string minLength: 1 maxLength: 512 pattern: ^[^<>"]+$ - type: 'null' errorMessage: invalid title name description: oneOf: - type: string minLength: 1 pattern: ^[^<>"]+$ - type: 'null' errorMessage: invalid description type: type: string enum: - mezmo - mezmo-archive - http - cloudwatch-logs - cloudwatch-metrics - sqs - notification-channel - azure-blob-storage - kinesis-firehose - kinesis-streams - elasticsearch - gcp-cloud-monitoring - gcp-cloud-storage - gcp-cloud-operations - gcp-cloud-pubsub - honeycomb-logs - indexed-search - kafka - loki - new-relic - prometheus-remote-write - pulsar - redis - splunk-hec-logs - sumo-logic-logs - sumo-logic-metrics - vector - s3 - blackhole - datadog-logs - datadog-metrics - clickhouse - opentelemetry inputs: type: array minItems: 1 items: type: string user_config: type: object required: - inputs - user_config required: - sources - transforms - sinks required: true responses: '200': description: Default Response /v3/pipeline/{pipeline_id}/sink/{pipeline_sink_id}: delete: summary: Delete sink tags: - Pipeline Nodes description: Remove sink from a pipeline parameters: - schema: type: string format: uuid in: path name: pipeline_id required: true - schema: type: string format: uuid in: path name: pipeline_sink_id required: true responses: '200': description: Default Response get: summary: Fetch sink tags: - Pipeline Nodes description: Fetch a sink configuration of a pipeline parameters: - schema: type: string format: uuid in: path name: pipeline_id required: true - schema: type: string format: uuid in: path name: pipeline_sink_id required: true responses: '200': description: Default Response put: summary: Update sink tags: - Pipeline Nodes description: Updates a sink of a pipeline requestBody: content: application/json: schema: type: object allOf: - if: properties: type: enum: - mezmo deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Mezmo Log Analysis description: Send log data to Mezmo Log Analysis additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true mezmo_host: type: string title: Mezmo Host description: The host for your Log Analysis environment default: null minLength: 1 ingestion_key: type: string title: Ingestion Key description: Ingestion key ui: display_type: mezmo-ingestion-key minLength: 1 query: type: object title: Query Parameters default: hostname: mezmo properties: hostname: type: string title: Hostname description: >- Hostname string or template to extract event values to form a hostname. Depending on the type of ingestion, this value is usually set from other places so the default value is often sufficient. If the value in the event is null, empty, or doesn't exist, when using a template then it will be set to "mezmo" default: mezmo minLength: 1 ui: display_type: data-template tags: type: array title: Tags description: >- List of tag strings or templates to attach to logs. Template must resolve to either an array of strings, or a string. Commas are treated as tag delimiters. items: type: string title: Tag description: Tag string or template to attach to logs minLength: 1 maxLength: 512 ui: display_type: data-template ip: type: string title: IP description: IP address template to attach to logs minLength: 1 maxLength: 512 ui: display_type: data-template mac: type: string title: MAC description: MAC address template to attach to logs minLength: 1 maxLength: 512 ui: display_type: data-template message: title: Log Construction Scheme default: scheme: Explicit field selection allOf: - if: properties: scheme: enum: - Explicit field selection then: properties: line: type: string title: Line description: >- Template or field reference to use as the log line. Field reference can point to an object. minLength: 1 maxLength: 1024 meta_field: type: string title: Meta Field description: Field containing the meta object for the log pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference maxLength: 512 app: type: string title: App description: App name template to attach to logs minLength: 1 maxLength: 512 ui: display_type: data-template file: type: string title: File description: File name template to attach to logs minLength: 1 maxLength: 512 ui: display_type: data-template timestamp_field: type: string title: Timestamp description: Field containing the timestamp for the log pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference maxLength: 512 env: type: string title: Env description: Environment name template to attach to logs minLength: 1 maxLength: 512 ui: display_type: data-template properties: scheme: type: string title: Scheme description: How to construct the log message. enum: - Explicit field selection - Message pass-through default: Explicit field selection required: - scheme required: - mezmo_host - ingestion_key - if: properties: type: enum: - mezmo-archive deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Mezmo Archive Destination description: Store logs for future imports into mezmo pipeline or log analysis. additionalProperties: false properties: batch_timeout_secs: type: integer title: Batch Timeout (seconds) description: >- The maximum amount of time, in seconds, events will be buffered before being flushed to the destination enum: - 30 - 60 - 90 - 120 - 300 default: 300 ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true destination: title: Destination description: '' allOf: - if: properties: archive_provider: enum: - Azure then: properties: container_name: type: string title: Container Name minLength: 1 description: The name of the container for blob storage. connection_string: type: string title: Connection String minLength: 1 description: >- A connection string for the account that contains an access key required: - container_name - connection_string - if: properties: archive_provider: enum: - S3 then: properties: access_key_id: type: string title: Access Key ID minLength: 1 description: The AWS access key id secret_access_key: type: string title: Secret Access Key minLength: 1 description: The AWS secret access key ui: display_type: secret bucket: type: string title: Bucket pattern: (?!(^xn--|.+-s3alias$))^[a-z0-9][a-z0-9-]{1,61}[a-z0-9]$ description: >- The S3 bucket name. Do not include a leading s3:// or a trailing / region: type: string title: Region minLength: 1 description: The name of the AWS region that is targeted. required: - access_key_id - secret_access_key - bucket - region properties: archive_provider: type: string title: Archive Provider description: The storage destination for your data. enum: - Azure - S3 default: Azure required: - archive_provider required: - destination - if: properties: type: enum: - http deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: HTTP description: Send data to a HTTP endpoint additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true uri: type: string title: URI format: uri errorMessage: format: Must be a valid URI description: >- The full URI to make HTTP requests to. This should include the protocol and host, but can also include the port, path, and any other valid part of a URI. minLength: 1 encoding: type: string description: The encoding to apply to the data. enum: - json - ndjson - text title: Encoding default: text compression: type: string description: The compression strategy used on the encoded data prior to sending. enum: - gzip - none title: Compression default: none auth: title: Authentication description: Configures HTTP authentication allOf: - if: properties: strategy: enum: - basic then: properties: user: type: string title: User minLength: 1 description: The basic authentication user password: type: string title: Password minLength: 1 description: The basic authentication password ui: display_type: secret required: - user - password - if: properties: strategy: enum: - bearer then: properties: token: type: string title: Token minLength: 1 description: The bearer token ui: display_type: secret required: - token properties: strategy: type: string enum: - basic - bearer - none title: Strategy description: The authentication strategy to use default: none headers: type: array title: HTTP Headers description: >- An array of key/value objects for the headers to be sent with the request. items: type: object title: Headers description: A key/value object describing a header name and its value. additionalProperties: false properties: header_name: type: string title: Header Name minLength: 1 description: The name of the header. header_value: type: string title: Header Value minLength: 1 description: The value of the header. required: - header_name - header_value advanced_options: type: object ui: display_type: collapsable feature_flag: pipeline-node-sink-http-advanced-options title: Advanced Options description: Settings available for fine-tuning the destination behavior. additionalProperties: false dependencies: payload_prefix: - payload_suffix payload_suffix: - payload_prefix properties: max_bytes: type: integer title: Payload Size description: >- The maximum number of uncompressed bytes when batching data to the destination. minimum: 1 maximum: 2097152 timeout_secs: type: integer title: Timeout Seconds description: The number of seconds before a destination write timeout. minimum: 5 method: type: string enum: - post - put - patch - delete - get - head - options - trace title: HTTP Method description: The HTTP method to use for the destination. payload_prefix: type: string title: Payload Prefix description: >- Add a prefix to the payload. Only used for serialized JSON chunks. This option also requires "Payload Suffix" to form a valid JSON string. examples: - '{"event":"' - '{"extra_prop": true, "our_thing":"' minLength: 1 payload_suffix: type: string title: Payload Suffix description: >- Used in combination with "Payload Prefix" to form valid JSON from the payload. examples: - '}' - '"extra_prop": true }' minLength: 1 proxy: type: object additionalProperties: false title: Proxy Settings description: Configure proxy settings for the destination. properties: enabled: type: boolean title: Enabled description: Turns proxying on/off default: false endpoint: type: string minLength: 1 title: HTTP(S) Endpoint description: HTTP or HTTPS Endpoint to use for traffic. format: uri errorMessage: format: Must be a valid URI hosts_bypass_proxy: type: array minItems: 1 title: Hosts to Bypass description: >- A list of hosts to bypass proxying. Can be specified as a domain name, IP address, or CIDR block. Wildcards are supported as a dot (.) in domain names, or as a star (*) to match all hosts. examples: - 93.184.216.34 - ignorehost.example.com - .ignoreall.example.com - 93.184.216.0/24 items: type: string title: Host description: Domain name, IP address, or CIDR block. minLength: 1 tls_protocols: type: array title: TLS Protocols description: >- A list of ALPN protocols to use during TLS negotiation. They are attempted in the order they appear. items: type: string title: Protocol description: An ALPN protocol name minLength: 1 minItems: 1 rate_limiting: type: object additionalProperties: false title: Rate Limiting description: Settings for controlling rate limiting to the destination. properties: request_limit: type: integer minimum: 1 title: Request Limit description: >- The max number of requests allowed within the specified "Duration Seconds" setting. duration_secs: type: integer default: 1 title: Duration Seconds description: The window of time used to apply "Request Limit." minimum: 1 required: - uri - encoding - compression - if: properties: type: enum: - cloudwatch-logs deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: AWS Cloudwatch Logs description: Publish log events to AWS Cloudwatch Logs additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true auth: type: object title: AWS Authentication description: Configures AWS authentication additionalProperties: false properties: access_key_id: type: string title: Access Key ID minLength: 1 description: The AWS access key id secret_access_key: type: string title: Secret Access Key minLength: 1 description: The AWS secret access key ui: display_type: secret required: - access_key_id - secret_access_key encoding: type: string description: The encoding to apply to the data. enum: - json - text title: Encoding default: text compression: type: string description: The compression strategy used on the encoded data prior to sending. enum: - gzip - none title: Compression default: none group_name: type: string title: Group Name minLength: 1 description: The name of the log group for the targeted log stream. region: type: string title: Region minLength: 1 description: The name of the AWS region that is targeted. stream_name: type: string title: Stream Name minLength: 1 description: The name of the targeted log stream. required: - auth - encoding - compression - group_name - region - stream_name - if: properties: type: enum: - cloudwatch-metrics deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: AWS Cloudwatch Metrics description: Send metric events to AWS Cloudwatch Metrics additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true auth: type: object title: AWS Authentication description: Configures AWS authentication additionalProperties: false properties: access_key_id: type: string title: Access Key ID minLength: 1 description: The AWS access key id secret_access_key: type: string title: Secret Access Key minLength: 1 description: The AWS secret access key ui: display_type: secret required: - access_key_id - secret_access_key compression: type: string description: The compression strategy used on the encoded data prior to sending. enum: - gzip - none title: Compression default: none namespace: type: string title: Namespace minLength: 1 description: Name for the container that will isolate metrics from one another. region: type: string title: Region minLength: 1 description: The name of the AWS region that is targeted. required: - auth - compression - namespace - region - if: properties: type: enum: - sqs deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: AWS Simple Queue Service (SQS) description: Publish log events to AWS Simple Queue Service additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true auth: type: object title: AWS Authentication description: Configures AWS authentication additionalProperties: false properties: access_key_id: type: string title: Access Key ID minLength: 1 description: The AWS access key id secret_access_key: type: string title: Secret Access Key minLength: 1 description: The AWS secret access key ui: display_type: secret required: - access_key_id - secret_access_key encoding: type: string description: The encoding to apply to the data. enum: - json - text title: Encoding default: text sqs_queue_url: type: string title: SQS Queue URL format: url errorMessage: format: Must be a valid URL description: The URL of the SQS queue that the data is publishing to. region: type: string title: Region minLength: 1 description: The name of the AWS region that is targeted. required: - auth - encoding - sqs_queue_url - region - if: properties: type: enum: - notification-channel deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Notification Channel description: Send a Message to a 3rd Party Service. additionalProperties: false properties: service: title: Service description: >- Configure a service to receive the payload. Currently supports Slack, PagerDuty, Webhook or Mezmo (Log Analysis). Where allowed, messagetemplating can be used. To use a template, surround field names with double curly braces, e.g., {{.my_field}} or {{."my-object".thing}}. See [documentation](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) for more examples. additionalProperties: type: string title: URI format: uri errorMessage: format: Must be a valid URI description: >- The full URI where the payload is sent. This should include the protocol and host, but can also include the port, path, and any other valid part of a URI. minLength: 1 allOf: - if: properties: name: enum: - slack required: - name then: properties: uri: type: string title: URI format: uri errorMessage: format: Must be a valid URI description: >- The full URI where the payload is sent. This should include the protocol and host, but can also include the port, path, and any other valid part of a URI. minLength: 1 message_text: type: string title: Message minLength: 1 ui: display_type: data-template options: multiline: true description: The text to be sent as a Slack message. required: - uri - message_text - if: properties: name: enum: - pager_duty required: - name then: properties: uri: type: string title: URI format: uri errorMessage: format: Must be a valid URI description: >- The full URI where the payload is sent. This should include the protocol and host, but can also include the port, path, and any other valid part of a URI. minLength: 1 summary: type: string title: Summary description: Summarize the alert details for PagerDuty. minLength: 1 ui: display_type: data-template severity: type: string enum: - INFO - WARNING - ERROR - CRITICAL title: Severity description: The severity level of the alert. source: type: string title: Source description: The source of the alert. minLength: 1 ui: display_type: data-template routing_key: type: string title: Routing Key description: The service integration routing key. minLength: 1 ui: display_type: secret event_action: type: string enum: - trigger - acknowledge - resolve title: Event Action description: The event action associated with the alert. required: - uri - summary - severity - source - routing_key - event_action - if: properties: name: enum: - webhook required: - name then: properties: uri: type: string title: URI format: uri errorMessage: format: Must be a valid URI description: >- The full URI where the payload is sent. This should include the protocol and host, but can also include the port, path, and any other valid part of a URI. minLength: 1 method: type: string title: HTTP Method description: The HTTP method to use for the request. enum: - post - put - patch - delete - get - head - options - trace ui: default_hint: post message_text: type: string title: Message minLength: 1 ui: display_type: data-template options: multiline: true description: >- The Webhook payload to send. This can be a text string or stringified JSON. If the message can be parsed as JSON, it will be sent as such. headers: type: array title: HTTP Headers description: >- A list of key/value pairs to be included in the HTTP request. items: type: object title: Headers description: A key/value object describing a header name and its value. additionalProperties: false properties: header_name: type: string title: Header Name minLength: 1 description: The name of the header. header_value: type: string title: Header Value minLength: 1 maxLength: 1024 description: The value of the header. required: - header_name - header_value maxItems: 20 auth: title: Authentication Options description: Configures HTTP authentication allOf: - if: properties: strategy: enum: - basic then: properties: user: type: string title: User minLength: 1 description: The basic authentication user password: type: string title: Password minLength: 1 description: The basic authentication password ui: display_type: secret required: - user - password - if: properties: strategy: enum: - bearer then: properties: token: type: string title: Token minLength: 1 description: The bearer token ui: display_type: secret required: - token properties: strategy: type: string enum: - basic - bearer - none title: Strategy description: Choose basic or token-based authentication. default: none required: - uri - message_text - if: properties: name: enum: - log_analysis required: - name then: properties: severity: type: string enum: - INFO - WARNING - ERROR - CRITICAL title: Severity description: The severity level of the alert. default: INFO subject: type: string title: Subject description: A string to be used as the subject of the alert. minLength: 1 maxLength: 200 ui: display_type: data-template body: type: string title: Body description: A string to be used as the alert message body. minLength: 1 maxLength: 1024 ui: display_type: data-template ingestion_key: type: string title: Ingestion Key description: >- The key required to communicate to Log Analysis for sending the alert. ui: display_type: mezmo-ingestion-key minLength: 1 required: - severity - subject - body - ingestion_key properties: name: title: Name description: The name of the service that will receive the notification. ui: display_type: list errorMessage: must be a valid service name oneOf: - label: Slack enum: - slack - label: PagerDuty enum: - pager_duty - label: Webhook enum: - webhook - label: Mezmo Log Analysis enum: - log_analysis required: - name throttling: type: object title: Throttling description: >- Only allow a certain number of notifications to be sent within a given time window. properties: window_secs: type: integer title: Window Seconds description: >- The time frame during which the number of notifications (set by the Threshold) is permitted. default: 60 minimum: 1 threshold: type: integer title: Threshold description: >- The maximum number of notifications allowed over the given time window set by "Window Seconds" (default: 1). default: 1 minimum: 1 required: - service - throttling - if: properties: type: enum: - azure-blob-storage deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Azure Blob Storage description: Publish log events to Azure Blob Storage additionalProperties: false properties: batch_timeout_secs: type: integer title: Batch Timeout (seconds) description: >- The maximum amount of time, in seconds, events will be buffered before being flushed to the destination enum: - 30 - 60 - 90 - 120 - 300 default: 300 ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true encoding: type: string description: The encoding to apply to the data. enum: - json - ndjson - text title: Encoding default: text compression: type: string description: The compression strategy used on the encoded data prior to sending. enum: - gzip - none title: Compression default: none container_name: type: string title: Container Name minLength: 1 description: The name of the container for blob storage. connection_string: type: string title: Connection String minLength: 1 description: A connection string for the account that contains an access key prefix: type: string title: Prefix minLength: 1 description: A prefix to be applied to all object keys ui: display_type: data-template file_consolidation: type: object ui: display_type: collapsable feature_flag: pipeline-node-sink-azure-blob-file-consolidation title: File Consolidation description: >- This sink writes many small files out to azure blob storage. Enabling this process will allow the automatic consolidation of these small files into larger files of your choosing. This process will enable upon deployment and run on the chosen interval from `Processing Interval` creating files named `merged_[timestamp].log` where `timestamp` is the time since epoch when the actual file was created. The process will recursively access all files under the `Base Path` to handle merging sub-directory logging structures. additionalProperties: false properties: enabled: type: boolean default: false title: Enabled description: Toggles whether the process is enabled. process_every_seconds: type: integer title: Processing Interval (seconds) description: How often to run the consolidation process. default: 600 minimum: 300 maximum: 3600 requested_size_bytes: type: integer title: Requested File Size (bytes) description: The requested size of the consolidated files. default: 500000000 minimum: 50000000 maximum: 10000000000 base_path: type: string title: Base Path default: '' maxLength: 512 description: >- The path from the container to begin recursivelylooking for files to merge. A blank value indicates root.Merged files will only contain data from the respectivefolder. required: - encoding - compression - container_name - connection_string - if: properties: type: enum: - kinesis-firehose deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: AWS Kinesis Data Firehose description: Publish log events to AWS Kinesis Data Firehose additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true auth: type: object title: AWS Authentication description: Configures AWS authentication additionalProperties: false properties: access_key_id: type: string title: Access Key ID minLength: 1 description: The AWS access key id secret_access_key: type: string title: Secret Access Key minLength: 1 description: The AWS secret access key ui: display_type: secret required: - access_key_id - secret_access_key encoding: type: string description: The encoding to apply to the data. enum: - json - text title: Encoding default: text compression: type: string description: The compression strategy used on the encoded data prior to sending. enum: - gzip - none title: Compression default: none region: type: string title: Region minLength: 1 description: The name of the AWS region that is targeted. stream_name: type: string title: Stream Name minLength: 1 description: The name of the Kinesis Firehose stream required: - auth - encoding - compression - region - stream_name - if: properties: type: enum: - kinesis-streams deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: AWS Kinesis Streams description: Publish log events to AWS Kinesis Streams additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true auth: type: object title: AWS Authentication description: Configures AWS authentication additionalProperties: false properties: access_key_id: type: string title: Access Key ID minLength: 1 description: The AWS access key id secret_access_key: type: string title: Secret Access Key minLength: 1 description: The AWS secret access key ui: display_type: secret required: - access_key_id - secret_access_key encoding: type: string description: The encoding to apply to the data. enum: - json - text title: Encoding default: text compression: type: string description: The compression strategy used on the encoded data prior to sending. enum: - gzip - none title: Compression default: none partition_key_field: type: string title: Partition Key Field pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ description: The field in the log whose value is used as Kinesis' partition key. errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference region: type: string title: Region minLength: 1 description: The name of the AWS region that is targeted. stream_name: type: string title: Stream Name minLength: 1 description: The name of the targeted log stream. required: - auth - encoding - compression - region - stream_name - if: properties: type: enum: - elasticsearch deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Elasticsearch description: Publish log events to Elasticsearch additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true compression: type: string description: The compression strategy used on the encoded data prior to sending. enum: - gzip - none title: Compression default: none auth: title: ElasticSearch Auth Strategy description: ElasticSearch authentication strategy allOf: - if: properties: strategy: enum: - basic then: properties: user: type: string title: User minLength: 1 maxLength: 512 description: The basic authentication user password: type: string title: Password minLength: 1 maxLength: 512 description: The basic authentication password ui: display_type: secret required: - user - password - if: properties: strategy: enum: - aws then: properties: access_key_id: type: string title: Access Key ID minLength: 1 maxLength: 1024 description: AWS access key ID ui: display_type: secret secret_access_key: type: string title: Secrete Access Key minLength: 1 maxLength: 1024 description: AWS secrete access key ui: display_type: secret region: type: string title: Region minLength: 1 maxLength: 128 description: The name of the AWS region that is targeted. required: - access_key_id - secret_access_key - region properties: strategy: type: string title: Strategy description: The ES authentication strategy to use enum: - basic - aws required: - strategy endpoints: type: array title: Endpoints uniqueItems: true minItems: 1 description: An array of ElasticSearch endpoints items: type: string title: URL format: url errorMessage: format: Must be a valid URL description: The full Elasticsearch URL to send data to. minLength: 1 maxLength: 1024 pipeline: type: string title: Pipeline minLength: 1 description: Name of an ES ingest pipeline to include. index: type: string minLength: 1 maxLength: 512 title: Custom Index description: Index to use when writing ES events (default = mezmo-%Y.%m.%d) default: mezmo-%Y.%m.%d required: - compression - auth - endpoints - if: properties: type: enum: - gcp-cloud-monitoring deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: GCP Cloud Monitoring description: Publish metrics events to GCP Cloud Monitoring additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true credentials_json: type: string title: JSON Credentials minLength: 1 description: JSON Credentials ui: display_type: secret-file project_id: type: string title: Project ID minLength: 1 description: The Project ID as defined in Google Cloud. resource_type: type: string title: Resource Type minLength: 1 description: The `monitored-resource` type as defined in Monitoring. resource_labels: type: array title: Resource Labels description: >- An array of key/val objects describing labels of the Monitoring resource. items: type: object title: Label/Value description: A key/val object describing a label name and its value. additionalProperties: false properties: label_name: type: string title: Label Name minLength: 1 description: The name of the label. label_value: type: string title: Label Value minLength: 1 description: The value of the label. required: - label_name - label_value required: - credentials_json - project_id - resource_type - if: properties: type: enum: - gcp-cloud-storage deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: GCP Cloud Storage description: Publish log events to GCP Cloud Storage additionalProperties: false properties: batch_timeout_secs: type: integer title: Batch Timeout (seconds) description: >- The maximum amount of time, in seconds, events will be buffered before being flushed to the destination enum: - 30 - 60 - 90 - 120 - 300 default: 300 ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true credentials_json: type: string title: JSON Credentials minLength: 1 description: JSON Credentials ui: display_type: secret-file encoding: type: string description: Dictates how the data will be serialized before storing. enum: - json - ndjson - text title: Encoding default: text bucket: type: string title: Bucket minLength: 1 description: The name of the bucket in GCP where the data will be stored. compression: type: string description: The compression strategy used on the encoded data prior to sending. enum: - gzip - none title: Compression default: none bucket_prefix: type: string title: Bucket Prefix minLength: 1 description: >- The prefix applied to the bucket name, giving the appearance of having directories. required: - credentials_json - encoding - bucket - compression - if: properties: type: enum: - gcp-cloud-operations deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: GCP Cloud Operations description: Publish log events to GCP Cloud Operations additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true credentials_json: type: string title: JSON Credentials minLength: 1 description: JSON Credentials ui: display_type: secret-file log_id: type: string title: Log ID minLength: 1 description: Concise reference for the log stream name. project_id: type: string title: Project ID minLength: 1 description: The Project ID as defined in Google Cloud. resource_type: type: string title: Resource Type minLength: 1 description: The `monitored-resource` type as defined in Monitoring. resource_labels: type: array title: Resource Labels description: >- An array of key/val objects describing labels of the Monitoring resource. items: type: object title: Label/Value description: A key/val object describing a label name and its value. additionalProperties: false properties: label_name: type: string title: Label Name minLength: 1 description: The name of the label. label_value: type: string title: Label Value minLength: 1 description: The value of the label. required: - label_name - label_value required: - credentials_json - log_id - project_id - resource_type - if: properties: type: enum: - gcp-cloud-pubsub deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: GCP Cloud PubSub description: Publish log events to GCP Cloud PubSub additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true credentials_json: type: string title: JSON Credentials minLength: 1 description: JSON Credentials ui: display_type: secret-file encoding: type: string description: Dictates how the data will be serialized before storing. enum: - json - text title: Encoding default: text project_id: type: string title: Project ID minLength: 1 description: The Project ID as defined in Google Cloud. topic: type: string title: Topic minLength: 1 description: The name of the topic to publish messages to. required: - credentials_json - encoding - project_id - topic - if: properties: type: enum: - honeycomb-logs deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Honeycomb Logs description: Send log data to Honeycomb additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true api_key: type: string title: Honeycomb API key description: Your Honeycomb API key minLength: 1 maxLength: 512 ui: display_type: secret dataset: type: string title: Dataset description: The name of the targeted dataset. minLength: 1 maxLength: 512 required: - api_key - dataset - if: properties: type: enum: - indexed-search deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Mezmo Indexed Search description: Publish log events to Mezmo's data store for Indexed Search capabilities additionalProperties: true properties: search_field: type: string title: Full Text Search Field pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ default: .line maxLength: 256 description: >- If your data reaches this stage as a plain-text string, the entire payload will be indexed for full-text searching. If your data reaches this stage as an object, the field you enter here will be used as the full-text search field. Nested fields are allowed. errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference timestamp_field: type: string title: Timestamp Field pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ default: timestamp maxLength: 256 description: >- If your data reaches this stage as an object, this field represents the timestamp value to be cataloged for searching. If the field does not exists the current time will be used. We accept UNIX timestamps (in milliseconds), ISO8601/RFC3389, and RFC2822 formats. errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference required: - search_field - timestamp_field - if: properties: type: enum: - kafka deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: title: Kafka description: Publish log events to Kafka additionalProperties: type: string allOf: - if: properties: sasl_enabled: enum: - true then: properties: sasl_mechanism: type: string enum: - SCRAM-SHA-512 - SCRAM-SHA-256 - PLAIN title: SASL description: The Kafka SASL/SCRAM mechanism default: SCRAM-SHA-512 sasl_username: type: string title: Username description: The SASL/SCRAM username minLength: 1 sasl_password: type: string title: Password description: The SASL/SCRAM username ui: display_type: secret minLength: 1 required: - sasl_mechanism - sasl_username - sasl_password - if: properties: tls_enabled: enum: - true then: properties: tls_ca_certificate: type: string title: TLS CA Certificate Chain minLength: 1 description: The CA certificate chain in the PEM format ui: display_type: secret-file format: x509_certificate errorMessage: format: Must be a valid PEM or DER certificate tls_verify_certificate: type: boolean title: TLS Verify Certificate description: Verify the TLS certificates default: true required: - tls_verify_certificate properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true encoding: type: string description: The encoding to apply to the data. enum: - json - text title: Encoding default: text compression: type: string description: The compression strategy used on the encoded data prior to sending. enum: - gzip - lz4 - snappy - zstd - none title: Compression default: none event_key_field: type: string title: Event Key Field minLength: 1 description: The field in the log whose value is used as Kafka's event key. brokers: type: array title: Brokers uniqueItems: true minItems: 1 description: An array of key/val objects for each host/port of a Kafka broker. items: type: object title: Host/Port description: The host and port for the Kafka broker. additionalProperties: false properties: host: title: Host description: The hostname or IP address of the broker. anyOf: - type: string format: ipv4 errorMessage: format: Must be a valid IPv4 address - type: string format: hostname errorMessage: format: Must be a valid hostname port: type: number title: Port description: The port that the broker listens on. required: - host - port topic: type: string title: Topic minLength: 1 description: The name of the topic to publish to. group_id: type: string title: Group ID minLength: 1 description: The consumer group identifier sasl_enabled: type: boolean title: SASL/SCRAM Enabled description: Enable SASL/SCRAM authentication with Kafka default: false tls_enabled: type: boolean title: Use TLS description: Enable TLS during connections to the remote default: false required: - encoding - compression - brokers - topic - if: properties: type: enum: - loki deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Loki description: Publish log events to Loki additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true auth: title: Loki Auth Strategy description: Loki authentication strategy allOf: - if: properties: strategy: enum: - basic then: properties: user: type: string title: User minLength: 1 maxLength: 512 description: The basic authentication user password: type: string title: Password minLength: 1 maxLength: 512 description: The basic authentication password ui: display_type: secret required: - user - password - if: properties: strategy: enum: - bearer then: properties: token: type: string title: Token minLength: 1 maxLength: 1024 description: The bearer token (passed as-is) ui: display_type: secret required: - token properties: strategy: type: string title: Strategy description: The authentication strategy to use (only basic supported) enum: - basic - bearer required: - strategy encoding: type: object title: Encoding description: Configures how event are encoded properties: codec: type: string title: Codec enum: - json - text required: - codec endpoint: type: string title: Endpoint format: url errorMessage: format: Must be a valid URL minLength: 1 maxLength: 512 description: The Loki base URL path: type: string title: Path minLength: 1 maxLength: 512 description: The path appended to the Loki base URL(defaults to /loki/api/v1/push) default: /loki/api/v1/push labels: type: array title: Loki Labels description: Key/Value pair used to describe Loki data items: type: object title: Labels description: Label name and values additionalProperties: false properties: label_name: type: string title: Label Name description: The name of the label minLength: 1 maxLength: 512 label_value: type: string title: Label Value description: The value of the label minLength: 1 maxLength: 512 required: - label_name - label_value required: - auth - encoding - endpoint - labels - if: properties: type: enum: - new-relic deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: New Relic description: Publish to New Relic additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true account_id: type: string title: New Relic Account ID description: Account ID for your New Relic account minLength: 1 maxLength: 1024 ui: display_type: secret api: type: string title: API description: New Relic API endpoint (events not supported) enum: - logs - metrics default: logs license_key: type: string title: New Relic License Key description: License key for your New Relic account minLength: 1 maxLength: 1024 ui: display_type: secret required: - account_id - api - license_key - if: properties: type: enum: - prometheus-remote-write deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Prometheus Remote Write description: Publish metrics to Prometheus Remote Write endpoint additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true endpoint: type: string title: Endpoint format: uri errorMessage: format: Must be a valid URI description: >- The full URI to make HTTP requests to. This should include the protocol and host, but can also include the port, path, and any other valid part of a URI. Example: http://example.org:8080/api/v1/push minLength: 1 maxLength: 2000 auth: title: Authentication description: Configures HTTP authentication allOf: - if: properties: strategy: enum: - basic then: properties: user: type: string title: User minLength: 1 maxLength: 512 description: The basic authentication user password: type: string title: Password minLength: 1 maxLength: 512 description: The basic authentication password ui: display_type: secret required: - user - password - if: properties: strategy: enum: - bearer then: properties: token: type: string title: Token minLength: 1 maxLength: 1024 description: The bearer token ui: display_type: secret required: - token properties: strategy: type: string enum: - basic - bearer - none title: Strategy description: The authentication strategy to use default: none required: - strategy required: - endpoint - if: properties: type: enum: - pulsar deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Pulsar description: Publish log events to Apache Pulsar additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true encoding: type: string description: The encoding to apply to the data. enum: - json - text title: Encoding default: text url: type: string title: URL format: url errorMessage: format: Must be a valid URL description: The full Pulsar URL to send data to. topic: type: string title: Topic minLength: 1 description: The name of the Pulsar topic to publish to. required: - encoding - url - topic - if: properties: type: enum: - redis deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Redis description: Publish log events to Redis additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true encoding: type: string description: The encoding to apply to the data. enum: - json - text title: Encoding default: text url: type: string title: URL format: uri errorMessage: format: Must be a valid URL description: The full Redis URL to send data to. key: type: string title: Key minLength: 1 description: The name of the Redis key to write data to. required: - encoding - url - key - if: properties: type: enum: - splunk-hec-logs deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Splunk HTTP Event Collector description: Publish log events to Splunk HTTP Event Collector additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true compression: type: string description: The compression strategy used on the encoded data prior to sending. enum: - gzip - none title: Compression default: none url: type: string title: Endpoint format: url errorMessage: format: Must be a valid URL description: >- The base URL for the Splunk instance. The collector path, such as `/services/collector/events`, will be automatically inferred from the destination's configuration. token: type: string title: Token minLength: 1 ui: display_type: secret description: The default token to authenticate to Splunk HEC. tls_verify_certificate: type: boolean title: Verify TLS Certificate description: >- If unchecked, the TLS certificate of the remote host will not be verified. **This is insecure**, but may be useful for testing or when evaluating Splunk in a trial instance. default: true host_field: type: string title: Host Field description: The field that contains the hostname to include in the event. default: metadata.host timestamp_field: type: string title: Timestamp Field description: The field that contains the timestamp to include in the event. default: metadata.time pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ minLength: 1 maxLength: 512 errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: feature_flag: pipeline-splunk-sink-v2 display_type: field-reference source: title: Source description: >- The source of events sent to this sink. This is typically the filename the logs originated from. Use the field path ".metadata.source" to use the upstream source value from a HEC log source additionalProperties: false allOf: - if: properties: value_type: enum: - field then: title: Value properties: value: type: string title: Field value description: The field to use as the value. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference required: - value - if: properties: value_type: enum: - value required: - value_type then: title: Value properties: value: type: string title: Value description: The value. minLength: 1 maxLength: 512 required: - value properties: value_type: title: Value type description: >- Either a value from an event field, a new value input, or none to omit value. oneOf: - label: Value from event field enum: - field - label: New value enum: - value - label: None enum: - none default: none ui: display_type: radio-group value: ui: feature_flag: pipeline-splunk-sink-v2 type: - string - 'null' required: - value_type sourcetype: title: Source Type description: >- The sourcetype of events sent to this sink. Use the field path ".metadata.sourcetype" to use the upstream sourcetype value from a HEC log source additionalProperties: false allOf: - if: properties: value_type: enum: - field then: title: Value properties: value: type: string title: Field value description: The field to use as the value. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference required: - value - if: properties: value_type: enum: - value required: - value_type then: title: Value properties: value: type: string title: Value description: The value. minLength: 1 maxLength: 512 required: - value properties: value_type: title: Value type description: >- Either a value from an event field, a new value input, or none to omit value. oneOf: - label: Value from event field enum: - field - label: New value enum: - value - label: None enum: - none default: none ui: display_type: radio-group value: ui: feature_flag: pipeline-splunk-sink-v2 type: - string - 'null' required: - value_type index: title: Index description: >- The name of the index to send events to. Use the field path ".metadata.index" to use the upstream index value from a HEC log source additionalProperties: false allOf: - if: properties: value_type: enum: - field then: title: Value properties: value: type: string title: Field value description: The field to use as the value. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference required: - value - if: properties: value_type: enum: - value required: - value_type then: title: Value properties: value: type: string title: Value description: The value. minLength: 1 maxLength: 512 required: - value properties: value_type: title: Value type description: >- Either a value from an event field, a new value input, or none to omit value. oneOf: - label: Value from event field enum: - field - label: New value enum: - value - label: None enum: - none default: none ui: display_type: radio-group value: ui: feature_flag: pipeline-splunk-sink-v2 type: - string - 'null' required: - value_type required: - compression - url - token - source - sourcetype - index - if: properties: type: enum: - sumo-logic-logs deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Sumo Logic Logs description: Publish log data to Sumo Logic additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true endpoint: type: string title: Collector Endpoint format: url errorMessage: format: Must be a valid URL minLength: 1 maxLength: 512 description: The Sumo Logic Collector URL that receives the data ui: display_type: secret compression: type: string title: Compression description: The compression strategy (defaults to none). enum: - gzip - none default: none category: type: string minLength: 1 maxLength: 512 title: Category description: >- Category to use when publishing data to Sumo Logic (default = mezmo_default_category) default: mezmo_default_category required: - endpoint - compression - if: properties: type: enum: - sumo-logic-metrics deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Sumo Logic Metrics description: Publish metrics data to Sumo Logic additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true endpoint: type: string title: Collector Endpoint format: url errorMessage: format: Must be a valid URL minLength: 1 maxLength: 512 description: The Sumo Logic Collector URL that receives the data ui: display_type: secret compression: type: string title: Compression description: The compression strategy (defaults to none). enum: - gzip - none default: none category: type: string minLength: 1 maxLength: 512 title: Category description: >- Category to use when publishing data to Sumo Logic (default = mezmo_default_category) default: mezmo_default_category required: - endpoint - compression - if: properties: type: enum: - vector deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Vector description: Publish log events to Vector additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true compression: type: string description: The compression strategy used on the encoded data prior to sending. enum: - gzip - none title: Compression default: none vector_ip: type: string title: Vector IP format: ipv4 description: The IP portion of the Vector host vector_port: type: number title: Vector Port description: The port that the Vector host listens on. version: type: string enum: - '1' - '2' title: Version default: '2' description: The version of the Vector sink which must match the source version. required: - compression - vector_ip - vector_port - version - if: properties: type: enum: - s3 deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: AWS S3 description: Publish events as objects in AWS S3 additionalProperties: false errorMessage: properties: bucket: must be a valid bucket name tags: must be a valid tag identifier properties: batch_timeout_secs: type: integer title: Batch Timeout (seconds) description: >- The maximum amount of time, in seconds, events will be buffered before being flushed to the destination enum: - 30 - 60 - 90 - 120 - 300 default: 300 ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true auth: type: object title: AWS Authentication description: Configures AWS authentication additionalProperties: false properties: access_key_id: type: string title: Access Key ID minLength: 1 description: The AWS access key id secret_access_key: type: string title: Secret Access Key minLength: 1 description: The AWS secret access key ui: display_type: secret required: - access_key_id - secret_access_key bucket: type: string title: Bucket pattern: (?!(^xn--|.+-s3alias$))^[a-z0-9][a-z0-9-]{1,61}[a-z0-9]$ description: The S3 bucket name. Do not include a leading s3:// or a trailing / prefix: type: string title: Prefix description: A prefix to apply to all object key names. default: / ui: display_type: data-template tags: type: array maxItems: 10 title: Tags description: A set of tags (key/value pairs) to apply to the object in S3. items: type: object properties: key: type: string title: Name description: The tag name pattern: ^[A-Za-z0-9_.:=+@/\-\s]*$ minLength: 1 maxLength: 128 value: type: string title: Value description: The tag value default: '' pattern: ^[A-Za-z0-9_.:=+@/\-\s]*$ minLength: 0 maxLength: 256 required: - key - value encoding: type: string description: The encoding to apply to the data. enum: - json - ndjson - text title: Encoding default: text compression: type: string description: The compression strategy used on the encoded data prior to sending. enum: - gzip - none title: Compression default: none region: type: string title: Region minLength: 1 description: The name of the AWS region that is targeted. file_consolidation: type: object ui: display_type: collapsable feature_flag: pipeline-node-sink-s3-file-consolidation title: File Consolidation description: >- This sink writes many small files out to S3. Enabling this process will allow the automatic consolidation of these small files into larger files of your choosing. This process will enable upon deployment and run on the chosen interval from `Processing Interval` creating files named `merged_[timestamp].log` where `timestamp` is the time since epoch. *NOTE*: only `text` and `ndjson` files written by the sink are consolidated. additionalProperties: false properties: enabled: type: boolean default: false title: Enabled description: Toggles whether the process is enabled. process_every_seconds: type: integer title: Processing Interval (seconds) description: How often to run the consolidation process. default: 600 minimum: 300 maximum: 3600 requested_size_bytes: type: integer title: Requested File Size (bytes) description: The size of the requested files. default: 500000000 minimum: 50000000 maximum: 10000000000 base_path: type: string title: Base Path default: '' maxLength: 512 description: >- The path from the container to begin recursivelylooking for files to merge. A blank value indicates root.Merged files will only contain data from the respectivefolder. required: - auth - bucket - prefix - encoding - compression - region - if: properties: type: enum: - blackhole deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Blackhole description: Drop data arriving at this destination additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true - if: properties: type: enum: - datadog-logs deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Datadog Logs description: Publish log events to Datadog additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true api_key: type: string title: Datadog API Key description: Datadog log application API key minLength: 1 ui: display_type: secret compression: type: string description: The compression strategy used on the encoded data prior to sending. enum: - gzip - none title: Compression default: gzip site: type: string description: The Datadog site (region) to send logs to. enum: - us1 - us3 - us5 - eu1 title: Datadog Site default: us1 required: - api_key - compression - site - if: properties: type: enum: - datadog-metrics deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Datadog Metrics description: >- Publish metrics events to Datadog. Sending custom metrics to Datadog may result in extra charges on your Datadog bill. Please go to https://docs.datadoghq.com/metrics/custom_metrics/ for more information on Datadog custom metrics pricing. additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true api_key: type: string title: Datadog API Key description: Datadog metrics application API key minLength: 1 maxLength: 512 ui: display_type: secret site: type: string description: The Datadog site (region) to send metrics to. enum: - us1 - us3 - us5 - eu1 title: Datadog Site default: us1 required: - api_key - site - if: properties: type: enum: - clickhouse deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Clickhouse description: Deliver log data to the ClickHouse database additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true endpoint: type: string minLength: 1 maxLength: 2048 title: Endpoint description: Endpoint of the Clickhouse server format: uri errorMessage: format: Must be a valid URI auth: title: Auth Strategy description: Authentication strategy allOf: - if: properties: strategy: enum: - basic then: properties: user: type: string title: User minLength: 1 maxLength: 512 description: The basic authentication user password: type: string title: Password minLength: 1 maxLength: 512 description: The basic authentication password ui: display_type: secret required: - user - password - if: properties: strategy: enum: - bearer then: properties: token: type: string title: Token minLength: 1 maxLength: 2048 description: The bearer token to use for authentication ui: display_type: secret required: - token properties: strategy: type: string title: Strategy description: The ES authentication strategy to use enum: - basic - bearer required: - strategy table: type: string title: Table description: The table that data is inserted into pattern: ^[a-zA-Z0-9_]+$ errorMessage: Invalid table name ui: display_type: data-template database: type: string title: Database description: The database that contains the table that data is inserted into pattern: ^[a-zA-Z0-9_]+$ errorMessage: Invalid database name ui: display_type: data-template compression: type: string enum: - gzip - none - snappy - zlib - zstd title: Compression description: The compression algorithm to use default: gzip skip_unknown_fields: type: boolean title: Skip Unknown Fields description: Allows ClickHouse to discard fields not present in the table schema default: false required: - endpoint - auth - table - if: properties: type: enum: - opentelemetry deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: OpenTelemetry description: Publish data to OpenTelemetry additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true endpoint: type: string title: Collector Endpoint format: url errorMessage: format: Must be a valid URL minLength: 1 maxLength: 512 description: The OpenTelemetry Collector URL that receives the data auth: title: Authentication description: Configures HTTP authentication additionalProperties: type: string allOf: - if: properties: strategy: enum: - basic then: properties: user: type: string title: User minLength: 1 maxLength: 512 description: The basic authentication user password: type: string title: Password minLength: 1 maxLength: 512 description: The basic authentication password ui: display_type: secret required: - user - password - if: properties: strategy: enum: - bearer then: properties: token: type: string title: Token minLength: 1 maxLength: 1024 description: The bearer token ui: display_type: secret required: - token properties: strategy: type: string enum: - basic - bearer - none title: Strategy description: The authentication strategy to use default: none headers: type: array title: HTTP Headers description: >- An array of key/value objects for the headers to be sent with the request. items: type: object title: Headers description: A key/value object describing a header name and its value. additionalProperties: false properties: header_name: type: string title: Header Name minLength: 1 description: The name of the header. header_value: type: string title: Header Value minLength: 1 description: The value of the header. required: - header_name - header_value compression: type: string title: Compression description: The compression strategy (defaults to none). enum: - gzip - none default: none required: - endpoint - compression properties: title: oneOf: - type: string minLength: 1 maxLength: 512 pattern: ^[^<>"]+$ - type: 'null' errorMessage: invalid title name description: oneOf: - type: string minLength: 1 pattern: ^[^<>"]+$ - type: 'null' errorMessage: invalid description type: type: string enum: - mezmo - mezmo-archive - http - cloudwatch-logs - cloudwatch-metrics - sqs - notification-channel - azure-blob-storage - kinesis-firehose - kinesis-streams - elasticsearch - gcp-cloud-monitoring - gcp-cloud-storage - gcp-cloud-operations - gcp-cloud-pubsub - honeycomb-logs - indexed-search - kafka - loki - new-relic - prometheus-remote-write - pulsar - redis - splunk-hec-logs - sumo-logic-logs - sumo-logic-metrics - vector - s3 - blackhole - datadog-logs - datadog-metrics - clickhouse - opentelemetry deploy_type: type: string enum: - saas - local default: saas user_config: type: object generation_id: type: integer inputs: type: array items: type: string required: - type - user_config - generation_id - inputs required: true parameters: - schema: type: string format: uuid in: path name: pipeline_id required: true - schema: type: string format: uuid in: path name: pipeline_sink_id required: true responses: '200': description: Default Response /v3/pipeline/{pipeline_id}/sink: post: summary: Add a sink to a pipeline tags: - Pipeline Nodes requestBody: content: application/json: schema: type: object allOf: - if: properties: type: enum: - mezmo deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Mezmo Log Analysis description: Send log data to Mezmo Log Analysis additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true mezmo_host: type: string title: Mezmo Host description: The host for your Log Analysis environment default: null minLength: 1 ingestion_key: type: string title: Ingestion Key description: Ingestion key ui: display_type: mezmo-ingestion-key minLength: 1 query: type: object title: Query Parameters default: hostname: mezmo properties: hostname: type: string title: Hostname description: >- Hostname string or template to extract event values to form a hostname. Depending on the type of ingestion, this value is usually set from other places so the default value is often sufficient. If the value in the event is null, empty, or doesn't exist, when using a template then it will be set to "mezmo" default: mezmo minLength: 1 ui: display_type: data-template tags: type: array title: Tags description: >- List of tag strings or templates to attach to logs. Template must resolve to either an array of strings, or a string. Commas are treated as tag delimiters. items: type: string title: Tag description: Tag string or template to attach to logs minLength: 1 maxLength: 512 ui: display_type: data-template ip: type: string title: IP description: IP address template to attach to logs minLength: 1 maxLength: 512 ui: display_type: data-template mac: type: string title: MAC description: MAC address template to attach to logs minLength: 1 maxLength: 512 ui: display_type: data-template message: title: Log Construction Scheme default: scheme: Explicit field selection allOf: - if: properties: scheme: enum: - Explicit field selection then: properties: line: type: string title: Line description: >- Template or field reference to use as the log line. Field reference can point to an object. minLength: 1 maxLength: 1024 meta_field: type: string title: Meta Field description: Field containing the meta object for the log pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference maxLength: 512 app: type: string title: App description: App name template to attach to logs minLength: 1 maxLength: 512 ui: display_type: data-template file: type: string title: File description: File name template to attach to logs minLength: 1 maxLength: 512 ui: display_type: data-template timestamp_field: type: string title: Timestamp description: Field containing the timestamp for the log pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference maxLength: 512 env: type: string title: Env description: Environment name template to attach to logs minLength: 1 maxLength: 512 ui: display_type: data-template properties: scheme: type: string title: Scheme description: How to construct the log message. enum: - Explicit field selection - Message pass-through default: Explicit field selection required: - scheme required: - mezmo_host - ingestion_key - if: properties: type: enum: - mezmo-archive deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Mezmo Archive Destination description: Store logs for future imports into mezmo pipeline or log analysis. additionalProperties: false properties: batch_timeout_secs: type: integer title: Batch Timeout (seconds) description: >- The maximum amount of time, in seconds, events will be buffered before being flushed to the destination enum: - 30 - 60 - 90 - 120 - 300 default: 300 ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true destination: title: Destination description: '' allOf: - if: properties: archive_provider: enum: - Azure then: properties: container_name: type: string title: Container Name minLength: 1 description: The name of the container for blob storage. connection_string: type: string title: Connection String minLength: 1 description: >- A connection string for the account that contains an access key required: - container_name - connection_string - if: properties: archive_provider: enum: - S3 then: properties: access_key_id: type: string title: Access Key ID minLength: 1 description: The AWS access key id secret_access_key: type: string title: Secret Access Key minLength: 1 description: The AWS secret access key ui: display_type: secret bucket: type: string title: Bucket pattern: (?!(^xn--|.+-s3alias$))^[a-z0-9][a-z0-9-]{1,61}[a-z0-9]$ description: >- The S3 bucket name. Do not include a leading s3:// or a trailing / region: type: string title: Region minLength: 1 description: The name of the AWS region that is targeted. required: - access_key_id - secret_access_key - bucket - region properties: archive_provider: type: string title: Archive Provider description: The storage destination for your data. enum: - Azure - S3 default: Azure required: - archive_provider required: - destination - if: properties: type: enum: - http deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: HTTP description: Send data to a HTTP endpoint additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true uri: type: string title: URI format: uri errorMessage: format: Must be a valid URI description: >- The full URI to make HTTP requests to. This should include the protocol and host, but can also include the port, path, and any other valid part of a URI. minLength: 1 encoding: type: string description: The encoding to apply to the data. enum: - json - ndjson - text title: Encoding default: text compression: type: string description: The compression strategy used on the encoded data prior to sending. enum: - gzip - none title: Compression default: none auth: title: Authentication description: Configures HTTP authentication allOf: - if: properties: strategy: enum: - basic then: properties: user: type: string title: User minLength: 1 description: The basic authentication user password: type: string title: Password minLength: 1 description: The basic authentication password ui: display_type: secret required: - user - password - if: properties: strategy: enum: - bearer then: properties: token: type: string title: Token minLength: 1 description: The bearer token ui: display_type: secret required: - token properties: strategy: type: string enum: - basic - bearer - none title: Strategy description: The authentication strategy to use default: none headers: type: array title: HTTP Headers description: >- An array of key/value objects for the headers to be sent with the request. items: type: object title: Headers description: A key/value object describing a header name and its value. additionalProperties: false properties: header_name: type: string title: Header Name minLength: 1 description: The name of the header. header_value: type: string title: Header Value minLength: 1 description: The value of the header. required: - header_name - header_value advanced_options: type: object ui: display_type: collapsable feature_flag: pipeline-node-sink-http-advanced-options title: Advanced Options description: Settings available for fine-tuning the destination behavior. additionalProperties: false dependencies: payload_prefix: - payload_suffix payload_suffix: - payload_prefix properties: max_bytes: type: integer title: Payload Size description: >- The maximum number of uncompressed bytes when batching data to the destination. minimum: 1 maximum: 2097152 timeout_secs: type: integer title: Timeout Seconds description: The number of seconds before a destination write timeout. minimum: 5 method: type: string enum: - post - put - patch - delete - get - head - options - trace title: HTTP Method description: The HTTP method to use for the destination. payload_prefix: type: string title: Payload Prefix description: >- Add a prefix to the payload. Only used for serialized JSON chunks. This option also requires "Payload Suffix" to form a valid JSON string. examples: - '{"event":"' - '{"extra_prop": true, "our_thing":"' minLength: 1 payload_suffix: type: string title: Payload Suffix description: >- Used in combination with "Payload Prefix" to form valid JSON from the payload. examples: - '}' - '"extra_prop": true }' minLength: 1 proxy: type: object additionalProperties: false title: Proxy Settings description: Configure proxy settings for the destination. properties: enabled: type: boolean title: Enabled description: Turns proxying on/off default: false endpoint: type: string minLength: 1 title: HTTP(S) Endpoint description: HTTP or HTTPS Endpoint to use for traffic. format: uri errorMessage: format: Must be a valid URI hosts_bypass_proxy: type: array minItems: 1 title: Hosts to Bypass description: >- A list of hosts to bypass proxying. Can be specified as a domain name, IP address, or CIDR block. Wildcards are supported as a dot (.) in domain names, or as a star (*) to match all hosts. examples: - 93.184.216.34 - ignorehost.example.com - .ignoreall.example.com - 93.184.216.0/24 items: type: string title: Host description: Domain name, IP address, or CIDR block. minLength: 1 tls_protocols: type: array title: TLS Protocols description: >- A list of ALPN protocols to use during TLS negotiation. They are attempted in the order they appear. items: type: string title: Protocol description: An ALPN protocol name minLength: 1 minItems: 1 rate_limiting: type: object additionalProperties: false title: Rate Limiting description: Settings for controlling rate limiting to the destination. properties: request_limit: type: integer minimum: 1 title: Request Limit description: >- The max number of requests allowed within the specified "Duration Seconds" setting. duration_secs: type: integer default: 1 title: Duration Seconds description: The window of time used to apply "Request Limit." minimum: 1 required: - uri - encoding - compression - if: properties: type: enum: - cloudwatch-logs deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: AWS Cloudwatch Logs description: Publish log events to AWS Cloudwatch Logs additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true auth: type: object title: AWS Authentication description: Configures AWS authentication additionalProperties: false properties: access_key_id: type: string title: Access Key ID minLength: 1 description: The AWS access key id secret_access_key: type: string title: Secret Access Key minLength: 1 description: The AWS secret access key ui: display_type: secret required: - access_key_id - secret_access_key encoding: type: string description: The encoding to apply to the data. enum: - json - text title: Encoding default: text compression: type: string description: The compression strategy used on the encoded data prior to sending. enum: - gzip - none title: Compression default: none group_name: type: string title: Group Name minLength: 1 description: The name of the log group for the targeted log stream. region: type: string title: Region minLength: 1 description: The name of the AWS region that is targeted. stream_name: type: string title: Stream Name minLength: 1 description: The name of the targeted log stream. required: - auth - encoding - compression - group_name - region - stream_name - if: properties: type: enum: - cloudwatch-metrics deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: AWS Cloudwatch Metrics description: Send metric events to AWS Cloudwatch Metrics additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true auth: type: object title: AWS Authentication description: Configures AWS authentication additionalProperties: false properties: access_key_id: type: string title: Access Key ID minLength: 1 description: The AWS access key id secret_access_key: type: string title: Secret Access Key minLength: 1 description: The AWS secret access key ui: display_type: secret required: - access_key_id - secret_access_key compression: type: string description: The compression strategy used on the encoded data prior to sending. enum: - gzip - none title: Compression default: none namespace: type: string title: Namespace minLength: 1 description: Name for the container that will isolate metrics from one another. region: type: string title: Region minLength: 1 description: The name of the AWS region that is targeted. required: - auth - compression - namespace - region - if: properties: type: enum: - sqs deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: AWS Simple Queue Service (SQS) description: Publish log events to AWS Simple Queue Service additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true auth: type: object title: AWS Authentication description: Configures AWS authentication additionalProperties: false properties: access_key_id: type: string title: Access Key ID minLength: 1 description: The AWS access key id secret_access_key: type: string title: Secret Access Key minLength: 1 description: The AWS secret access key ui: display_type: secret required: - access_key_id - secret_access_key encoding: type: string description: The encoding to apply to the data. enum: - json - text title: Encoding default: text sqs_queue_url: type: string title: SQS Queue URL format: url errorMessage: format: Must be a valid URL description: The URL of the SQS queue that the data is publishing to. region: type: string title: Region minLength: 1 description: The name of the AWS region that is targeted. required: - auth - encoding - sqs_queue_url - region - if: properties: type: enum: - notification-channel deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Notification Channel description: Send a Message to a 3rd Party Service. additionalProperties: false properties: service: title: Service description: >- Configure a service to receive the payload. Currently supports Slack, PagerDuty, Webhook or Mezmo (Log Analysis). Where allowed, messagetemplating can be used. To use a template, surround field names with double curly braces, e.g., {{.my_field}} or {{."my-object".thing}}. See [documentation](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) for more examples. additionalProperties: type: string title: URI format: uri errorMessage: format: Must be a valid URI description: >- The full URI where the payload is sent. This should include the protocol and host, but can also include the port, path, and any other valid part of a URI. minLength: 1 allOf: - if: properties: name: enum: - slack required: - name then: properties: uri: type: string title: URI format: uri errorMessage: format: Must be a valid URI description: >- The full URI where the payload is sent. This should include the protocol and host, but can also include the port, path, and any other valid part of a URI. minLength: 1 message_text: type: string title: Message minLength: 1 ui: display_type: data-template options: multiline: true description: The text to be sent as a Slack message. required: - uri - message_text - if: properties: name: enum: - pager_duty required: - name then: properties: uri: type: string title: URI format: uri errorMessage: format: Must be a valid URI description: >- The full URI where the payload is sent. This should include the protocol and host, but can also include the port, path, and any other valid part of a URI. minLength: 1 summary: type: string title: Summary description: Summarize the alert details for PagerDuty. minLength: 1 ui: display_type: data-template severity: type: string enum: - INFO - WARNING - ERROR - CRITICAL title: Severity description: The severity level of the alert. source: type: string title: Source description: The source of the alert. minLength: 1 ui: display_type: data-template routing_key: type: string title: Routing Key description: The service integration routing key. minLength: 1 ui: display_type: secret event_action: type: string enum: - trigger - acknowledge - resolve title: Event Action description: The event action associated with the alert. required: - uri - summary - severity - source - routing_key - event_action - if: properties: name: enum: - webhook required: - name then: properties: uri: type: string title: URI format: uri errorMessage: format: Must be a valid URI description: >- The full URI where the payload is sent. This should include the protocol and host, but can also include the port, path, and any other valid part of a URI. minLength: 1 method: type: string title: HTTP Method description: The HTTP method to use for the request. enum: - post - put - patch - delete - get - head - options - trace ui: default_hint: post message_text: type: string title: Message minLength: 1 ui: display_type: data-template options: multiline: true description: >- The Webhook payload to send. This can be a text string or stringified JSON. If the message can be parsed as JSON, it will be sent as such. headers: type: array title: HTTP Headers description: >- A list of key/value pairs to be included in the HTTP request. items: type: object title: Headers description: A key/value object describing a header name and its value. additionalProperties: false properties: header_name: type: string title: Header Name minLength: 1 description: The name of the header. header_value: type: string title: Header Value minLength: 1 maxLength: 1024 description: The value of the header. required: - header_name - header_value maxItems: 20 auth: title: Authentication Options description: Configures HTTP authentication allOf: - if: properties: strategy: enum: - basic then: properties: user: type: string title: User minLength: 1 description: The basic authentication user password: type: string title: Password minLength: 1 description: The basic authentication password ui: display_type: secret required: - user - password - if: properties: strategy: enum: - bearer then: properties: token: type: string title: Token minLength: 1 description: The bearer token ui: display_type: secret required: - token properties: strategy: type: string enum: - basic - bearer - none title: Strategy description: Choose basic or token-based authentication. default: none required: - uri - message_text - if: properties: name: enum: - log_analysis required: - name then: properties: severity: type: string enum: - INFO - WARNING - ERROR - CRITICAL title: Severity description: The severity level of the alert. default: INFO subject: type: string title: Subject description: A string to be used as the subject of the alert. minLength: 1 maxLength: 200 ui: display_type: data-template body: type: string title: Body description: A string to be used as the alert message body. minLength: 1 maxLength: 1024 ui: display_type: data-template ingestion_key: type: string title: Ingestion Key description: >- The key required to communicate to Log Analysis for sending the alert. ui: display_type: mezmo-ingestion-key minLength: 1 required: - severity - subject - body - ingestion_key properties: name: title: Name description: The name of the service that will receive the notification. ui: display_type: list errorMessage: must be a valid service name oneOf: - label: Slack enum: - slack - label: PagerDuty enum: - pager_duty - label: Webhook enum: - webhook - label: Mezmo Log Analysis enum: - log_analysis required: - name throttling: type: object title: Throttling description: >- Only allow a certain number of notifications to be sent within a given time window. properties: window_secs: type: integer title: Window Seconds description: >- The time frame during which the number of notifications (set by the Threshold) is permitted. default: 60 minimum: 1 threshold: type: integer title: Threshold description: >- The maximum number of notifications allowed over the given time window set by "Window Seconds" (default: 1). default: 1 minimum: 1 required: - service - throttling - if: properties: type: enum: - azure-blob-storage deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Azure Blob Storage description: Publish log events to Azure Blob Storage additionalProperties: false properties: batch_timeout_secs: type: integer title: Batch Timeout (seconds) description: >- The maximum amount of time, in seconds, events will be buffered before being flushed to the destination enum: - 30 - 60 - 90 - 120 - 300 default: 300 ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true encoding: type: string description: The encoding to apply to the data. enum: - json - ndjson - text title: Encoding default: text compression: type: string description: The compression strategy used on the encoded data prior to sending. enum: - gzip - none title: Compression default: none container_name: type: string title: Container Name minLength: 1 description: The name of the container for blob storage. connection_string: type: string title: Connection String minLength: 1 description: A connection string for the account that contains an access key prefix: type: string title: Prefix minLength: 1 description: A prefix to be applied to all object keys ui: display_type: data-template file_consolidation: type: object ui: display_type: collapsable feature_flag: pipeline-node-sink-azure-blob-file-consolidation title: File Consolidation description: >- This sink writes many small files out to azure blob storage. Enabling this process will allow the automatic consolidation of these small files into larger files of your choosing. This process will enable upon deployment and run on the chosen interval from `Processing Interval` creating files named `merged_[timestamp].log` where `timestamp` is the time since epoch when the actual file was created. The process will recursively access all files under the `Base Path` to handle merging sub-directory logging structures. additionalProperties: false properties: enabled: type: boolean default: false title: Enabled description: Toggles whether the process is enabled. process_every_seconds: type: integer title: Processing Interval (seconds) description: How often to run the consolidation process. default: 600 minimum: 300 maximum: 3600 requested_size_bytes: type: integer title: Requested File Size (bytes) description: The requested size of the consolidated files. default: 500000000 minimum: 50000000 maximum: 10000000000 base_path: type: string title: Base Path default: '' maxLength: 512 description: >- The path from the container to begin recursivelylooking for files to merge. A blank value indicates root.Merged files will only contain data from the respectivefolder. required: - encoding - compression - container_name - connection_string - if: properties: type: enum: - kinesis-firehose deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: AWS Kinesis Data Firehose description: Publish log events to AWS Kinesis Data Firehose additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true auth: type: object title: AWS Authentication description: Configures AWS authentication additionalProperties: false properties: access_key_id: type: string title: Access Key ID minLength: 1 description: The AWS access key id secret_access_key: type: string title: Secret Access Key minLength: 1 description: The AWS secret access key ui: display_type: secret required: - access_key_id - secret_access_key encoding: type: string description: The encoding to apply to the data. enum: - json - text title: Encoding default: text compression: type: string description: The compression strategy used on the encoded data prior to sending. enum: - gzip - none title: Compression default: none region: type: string title: Region minLength: 1 description: The name of the AWS region that is targeted. stream_name: type: string title: Stream Name minLength: 1 description: The name of the Kinesis Firehose stream required: - auth - encoding - compression - region - stream_name - if: properties: type: enum: - kinesis-streams deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: AWS Kinesis Streams description: Publish log events to AWS Kinesis Streams additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true auth: type: object title: AWS Authentication description: Configures AWS authentication additionalProperties: false properties: access_key_id: type: string title: Access Key ID minLength: 1 description: The AWS access key id secret_access_key: type: string title: Secret Access Key minLength: 1 description: The AWS secret access key ui: display_type: secret required: - access_key_id - secret_access_key encoding: type: string description: The encoding to apply to the data. enum: - json - text title: Encoding default: text compression: type: string description: The compression strategy used on the encoded data prior to sending. enum: - gzip - none title: Compression default: none partition_key_field: type: string title: Partition Key Field pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ description: The field in the log whose value is used as Kinesis' partition key. errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference region: type: string title: Region minLength: 1 description: The name of the AWS region that is targeted. stream_name: type: string title: Stream Name minLength: 1 description: The name of the targeted log stream. required: - auth - encoding - compression - region - stream_name - if: properties: type: enum: - elasticsearch deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Elasticsearch description: Publish log events to Elasticsearch additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true compression: type: string description: The compression strategy used on the encoded data prior to sending. enum: - gzip - none title: Compression default: none auth: title: ElasticSearch Auth Strategy description: ElasticSearch authentication strategy allOf: - if: properties: strategy: enum: - basic then: properties: user: type: string title: User minLength: 1 maxLength: 512 description: The basic authentication user password: type: string title: Password minLength: 1 maxLength: 512 description: The basic authentication password ui: display_type: secret required: - user - password - if: properties: strategy: enum: - aws then: properties: access_key_id: type: string title: Access Key ID minLength: 1 maxLength: 1024 description: AWS access key ID ui: display_type: secret secret_access_key: type: string title: Secrete Access Key minLength: 1 maxLength: 1024 description: AWS secrete access key ui: display_type: secret region: type: string title: Region minLength: 1 maxLength: 128 description: The name of the AWS region that is targeted. required: - access_key_id - secret_access_key - region properties: strategy: type: string title: Strategy description: The ES authentication strategy to use enum: - basic - aws required: - strategy endpoints: type: array title: Endpoints uniqueItems: true minItems: 1 description: An array of ElasticSearch endpoints items: type: string title: URL format: url errorMessage: format: Must be a valid URL description: The full Elasticsearch URL to send data to. minLength: 1 maxLength: 1024 pipeline: type: string title: Pipeline minLength: 1 description: Name of an ES ingest pipeline to include. index: type: string minLength: 1 maxLength: 512 title: Custom Index description: Index to use when writing ES events (default = mezmo-%Y.%m.%d) default: mezmo-%Y.%m.%d required: - compression - auth - endpoints - if: properties: type: enum: - gcp-cloud-monitoring deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: GCP Cloud Monitoring description: Publish metrics events to GCP Cloud Monitoring additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true credentials_json: type: string title: JSON Credentials minLength: 1 description: JSON Credentials ui: display_type: secret-file project_id: type: string title: Project ID minLength: 1 description: The Project ID as defined in Google Cloud. resource_type: type: string title: Resource Type minLength: 1 description: The `monitored-resource` type as defined in Monitoring. resource_labels: type: array title: Resource Labels description: >- An array of key/val objects describing labels of the Monitoring resource. items: type: object title: Label/Value description: A key/val object describing a label name and its value. additionalProperties: false properties: label_name: type: string title: Label Name minLength: 1 description: The name of the label. label_value: type: string title: Label Value minLength: 1 description: The value of the label. required: - label_name - label_value required: - credentials_json - project_id - resource_type - if: properties: type: enum: - gcp-cloud-storage deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: GCP Cloud Storage description: Publish log events to GCP Cloud Storage additionalProperties: false properties: batch_timeout_secs: type: integer title: Batch Timeout (seconds) description: >- The maximum amount of time, in seconds, events will be buffered before being flushed to the destination enum: - 30 - 60 - 90 - 120 - 300 default: 300 ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true credentials_json: type: string title: JSON Credentials minLength: 1 description: JSON Credentials ui: display_type: secret-file encoding: type: string description: Dictates how the data will be serialized before storing. enum: - json - ndjson - text title: Encoding default: text bucket: type: string title: Bucket minLength: 1 description: The name of the bucket in GCP where the data will be stored. compression: type: string description: The compression strategy used on the encoded data prior to sending. enum: - gzip - none title: Compression default: none bucket_prefix: type: string title: Bucket Prefix minLength: 1 description: >- The prefix applied to the bucket name, giving the appearance of having directories. required: - credentials_json - encoding - bucket - compression - if: properties: type: enum: - gcp-cloud-operations deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: GCP Cloud Operations description: Publish log events to GCP Cloud Operations additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true credentials_json: type: string title: JSON Credentials minLength: 1 description: JSON Credentials ui: display_type: secret-file log_id: type: string title: Log ID minLength: 1 description: Concise reference for the log stream name. project_id: type: string title: Project ID minLength: 1 description: The Project ID as defined in Google Cloud. resource_type: type: string title: Resource Type minLength: 1 description: The `monitored-resource` type as defined in Monitoring. resource_labels: type: array title: Resource Labels description: >- An array of key/val objects describing labels of the Monitoring resource. items: type: object title: Label/Value description: A key/val object describing a label name and its value. additionalProperties: false properties: label_name: type: string title: Label Name minLength: 1 description: The name of the label. label_value: type: string title: Label Value minLength: 1 description: The value of the label. required: - label_name - label_value required: - credentials_json - log_id - project_id - resource_type - if: properties: type: enum: - gcp-cloud-pubsub deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: GCP Cloud PubSub description: Publish log events to GCP Cloud PubSub additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true credentials_json: type: string title: JSON Credentials minLength: 1 description: JSON Credentials ui: display_type: secret-file encoding: type: string description: Dictates how the data will be serialized before storing. enum: - json - text title: Encoding default: text project_id: type: string title: Project ID minLength: 1 description: The Project ID as defined in Google Cloud. topic: type: string title: Topic minLength: 1 description: The name of the topic to publish messages to. required: - credentials_json - encoding - project_id - topic - if: properties: type: enum: - honeycomb-logs deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Honeycomb Logs description: Send log data to Honeycomb additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true api_key: type: string title: Honeycomb API key description: Your Honeycomb API key minLength: 1 maxLength: 512 ui: display_type: secret dataset: type: string title: Dataset description: The name of the targeted dataset. minLength: 1 maxLength: 512 required: - api_key - dataset - if: properties: type: enum: - indexed-search deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Mezmo Indexed Search description: Publish log events to Mezmo's data store for Indexed Search capabilities additionalProperties: true properties: search_field: type: string title: Full Text Search Field pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ default: .line maxLength: 256 description: >- If your data reaches this stage as a plain-text string, the entire payload will be indexed for full-text searching. If your data reaches this stage as an object, the field you enter here will be used as the full-text search field. Nested fields are allowed. errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference timestamp_field: type: string title: Timestamp Field pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ default: timestamp maxLength: 256 description: >- If your data reaches this stage as an object, this field represents the timestamp value to be cataloged for searching. If the field does not exists the current time will be used. We accept UNIX timestamps (in milliseconds), ISO8601/RFC3389, and RFC2822 formats. errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference required: - search_field - timestamp_field - if: properties: type: enum: - kafka deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: title: Kafka description: Publish log events to Kafka additionalProperties: type: string allOf: - if: properties: sasl_enabled: enum: - true then: properties: sasl_mechanism: type: string enum: - SCRAM-SHA-512 - SCRAM-SHA-256 - PLAIN title: SASL description: The Kafka SASL/SCRAM mechanism default: SCRAM-SHA-512 sasl_username: type: string title: Username description: The SASL/SCRAM username minLength: 1 sasl_password: type: string title: Password description: The SASL/SCRAM username ui: display_type: secret minLength: 1 required: - sasl_mechanism - sasl_username - sasl_password - if: properties: tls_enabled: enum: - true then: properties: tls_ca_certificate: type: string title: TLS CA Certificate Chain minLength: 1 description: The CA certificate chain in the PEM format ui: display_type: secret-file format: x509_certificate errorMessage: format: Must be a valid PEM or DER certificate tls_verify_certificate: type: boolean title: TLS Verify Certificate description: Verify the TLS certificates default: true required: - tls_verify_certificate properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true encoding: type: string description: The encoding to apply to the data. enum: - json - text title: Encoding default: text compression: type: string description: The compression strategy used on the encoded data prior to sending. enum: - gzip - lz4 - snappy - zstd - none title: Compression default: none event_key_field: type: string title: Event Key Field minLength: 1 description: The field in the log whose value is used as Kafka's event key. brokers: type: array title: Brokers uniqueItems: true minItems: 1 description: An array of key/val objects for each host/port of a Kafka broker. items: type: object title: Host/Port description: The host and port for the Kafka broker. additionalProperties: false properties: host: title: Host description: The hostname or IP address of the broker. anyOf: - type: string format: ipv4 errorMessage: format: Must be a valid IPv4 address - type: string format: hostname errorMessage: format: Must be a valid hostname port: type: number title: Port description: The port that the broker listens on. required: - host - port topic: type: string title: Topic minLength: 1 description: The name of the topic to publish to. group_id: type: string title: Group ID minLength: 1 description: The consumer group identifier sasl_enabled: type: boolean title: SASL/SCRAM Enabled description: Enable SASL/SCRAM authentication with Kafka default: false tls_enabled: type: boolean title: Use TLS description: Enable TLS during connections to the remote default: false required: - encoding - compression - brokers - topic - if: properties: type: enum: - loki deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Loki description: Publish log events to Loki additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true auth: title: Loki Auth Strategy description: Loki authentication strategy allOf: - if: properties: strategy: enum: - basic then: properties: user: type: string title: User minLength: 1 maxLength: 512 description: The basic authentication user password: type: string title: Password minLength: 1 maxLength: 512 description: The basic authentication password ui: display_type: secret required: - user - password - if: properties: strategy: enum: - bearer then: properties: token: type: string title: Token minLength: 1 maxLength: 1024 description: The bearer token (passed as-is) ui: display_type: secret required: - token properties: strategy: type: string title: Strategy description: The authentication strategy to use (only basic supported) enum: - basic - bearer required: - strategy encoding: type: object title: Encoding description: Configures how event are encoded properties: codec: type: string title: Codec enum: - json - text required: - codec endpoint: type: string title: Endpoint format: url errorMessage: format: Must be a valid URL minLength: 1 maxLength: 512 description: The Loki base URL path: type: string title: Path minLength: 1 maxLength: 512 description: The path appended to the Loki base URL(defaults to /loki/api/v1/push) default: /loki/api/v1/push labels: type: array title: Loki Labels description: Key/Value pair used to describe Loki data items: type: object title: Labels description: Label name and values additionalProperties: false properties: label_name: type: string title: Label Name description: The name of the label minLength: 1 maxLength: 512 label_value: type: string title: Label Value description: The value of the label minLength: 1 maxLength: 512 required: - label_name - label_value required: - auth - encoding - endpoint - labels - if: properties: type: enum: - new-relic deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: New Relic description: Publish to New Relic additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true account_id: type: string title: New Relic Account ID description: Account ID for your New Relic account minLength: 1 maxLength: 1024 ui: display_type: secret api: type: string title: API description: New Relic API endpoint (events not supported) enum: - logs - metrics default: logs license_key: type: string title: New Relic License Key description: License key for your New Relic account minLength: 1 maxLength: 1024 ui: display_type: secret required: - account_id - api - license_key - if: properties: type: enum: - prometheus-remote-write deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Prometheus Remote Write description: Publish metrics to Prometheus Remote Write endpoint additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true endpoint: type: string title: Endpoint format: uri errorMessage: format: Must be a valid URI description: >- The full URI to make HTTP requests to. This should include the protocol and host, but can also include the port, path, and any other valid part of a URI. Example: http://example.org:8080/api/v1/push minLength: 1 maxLength: 2000 auth: title: Authentication description: Configures HTTP authentication allOf: - if: properties: strategy: enum: - basic then: properties: user: type: string title: User minLength: 1 maxLength: 512 description: The basic authentication user password: type: string title: Password minLength: 1 maxLength: 512 description: The basic authentication password ui: display_type: secret required: - user - password - if: properties: strategy: enum: - bearer then: properties: token: type: string title: Token minLength: 1 maxLength: 1024 description: The bearer token ui: display_type: secret required: - token properties: strategy: type: string enum: - basic - bearer - none title: Strategy description: The authentication strategy to use default: none required: - strategy required: - endpoint - if: properties: type: enum: - pulsar deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Pulsar description: Publish log events to Apache Pulsar additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true encoding: type: string description: The encoding to apply to the data. enum: - json - text title: Encoding default: text url: type: string title: URL format: url errorMessage: format: Must be a valid URL description: The full Pulsar URL to send data to. topic: type: string title: Topic minLength: 1 description: The name of the Pulsar topic to publish to. required: - encoding - url - topic - if: properties: type: enum: - redis deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Redis description: Publish log events to Redis additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true encoding: type: string description: The encoding to apply to the data. enum: - json - text title: Encoding default: text url: type: string title: URL format: uri errorMessage: format: Must be a valid URL description: The full Redis URL to send data to. key: type: string title: Key minLength: 1 description: The name of the Redis key to write data to. required: - encoding - url - key - if: properties: type: enum: - splunk-hec-logs deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Splunk HTTP Event Collector description: Publish log events to Splunk HTTP Event Collector additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true compression: type: string description: The compression strategy used on the encoded data prior to sending. enum: - gzip - none title: Compression default: none url: type: string title: Endpoint format: url errorMessage: format: Must be a valid URL description: >- The base URL for the Splunk instance. The collector path, such as `/services/collector/events`, will be automatically inferred from the destination's configuration. token: type: string title: Token minLength: 1 ui: display_type: secret description: The default token to authenticate to Splunk HEC. tls_verify_certificate: type: boolean title: Verify TLS Certificate description: >- If unchecked, the TLS certificate of the remote host will not be verified. **This is insecure**, but may be useful for testing or when evaluating Splunk in a trial instance. default: true host_field: type: string title: Host Field description: The field that contains the hostname to include in the event. default: metadata.host timestamp_field: type: string title: Timestamp Field description: The field that contains the timestamp to include in the event. default: metadata.time pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ minLength: 1 maxLength: 512 errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: feature_flag: pipeline-splunk-sink-v2 display_type: field-reference source: title: Source description: >- The source of events sent to this sink. This is typically the filename the logs originated from. Use the field path ".metadata.source" to use the upstream source value from a HEC log source additionalProperties: false allOf: - if: properties: value_type: enum: - field then: title: Value properties: value: type: string title: Field value description: The field to use as the value. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference required: - value - if: properties: value_type: enum: - value required: - value_type then: title: Value properties: value: type: string title: Value description: The value. minLength: 1 maxLength: 512 required: - value properties: value_type: title: Value type description: >- Either a value from an event field, a new value input, or none to omit value. oneOf: - label: Value from event field enum: - field - label: New value enum: - value - label: None enum: - none default: none ui: display_type: radio-group value: ui: feature_flag: pipeline-splunk-sink-v2 type: - string - 'null' required: - value_type sourcetype: title: Source Type description: >- The sourcetype of events sent to this sink. Use the field path ".metadata.sourcetype" to use the upstream sourcetype value from a HEC log source additionalProperties: false allOf: - if: properties: value_type: enum: - field then: title: Value properties: value: type: string title: Field value description: The field to use as the value. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference required: - value - if: properties: value_type: enum: - value required: - value_type then: title: Value properties: value: type: string title: Value description: The value. minLength: 1 maxLength: 512 required: - value properties: value_type: title: Value type description: >- Either a value from an event field, a new value input, or none to omit value. oneOf: - label: Value from event field enum: - field - label: New value enum: - value - label: None enum: - none default: none ui: display_type: radio-group value: ui: feature_flag: pipeline-splunk-sink-v2 type: - string - 'null' required: - value_type index: title: Index description: >- The name of the index to send events to. Use the field path ".metadata.index" to use the upstream index value from a HEC log source additionalProperties: false allOf: - if: properties: value_type: enum: - field then: title: Value properties: value: type: string title: Field value description: The field to use as the value. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference required: - value - if: properties: value_type: enum: - value required: - value_type then: title: Value properties: value: type: string title: Value description: The value. minLength: 1 maxLength: 512 required: - value properties: value_type: title: Value type description: >- Either a value from an event field, a new value input, or none to omit value. oneOf: - label: Value from event field enum: - field - label: New value enum: - value - label: None enum: - none default: none ui: display_type: radio-group value: ui: feature_flag: pipeline-splunk-sink-v2 type: - string - 'null' required: - value_type required: - compression - url - token - source - sourcetype - index - if: properties: type: enum: - sumo-logic-logs deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Sumo Logic Logs description: Publish log data to Sumo Logic additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true endpoint: type: string title: Collector Endpoint format: url errorMessage: format: Must be a valid URL minLength: 1 maxLength: 512 description: The Sumo Logic Collector URL that receives the data ui: display_type: secret compression: type: string title: Compression description: The compression strategy (defaults to none). enum: - gzip - none default: none category: type: string minLength: 1 maxLength: 512 title: Category description: >- Category to use when publishing data to Sumo Logic (default = mezmo_default_category) default: mezmo_default_category required: - endpoint - compression - if: properties: type: enum: - sumo-logic-metrics deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Sumo Logic Metrics description: Publish metrics data to Sumo Logic additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true endpoint: type: string title: Collector Endpoint format: url errorMessage: format: Must be a valid URL minLength: 1 maxLength: 512 description: The Sumo Logic Collector URL that receives the data ui: display_type: secret compression: type: string title: Compression description: The compression strategy (defaults to none). enum: - gzip - none default: none category: type: string minLength: 1 maxLength: 512 title: Category description: >- Category to use when publishing data to Sumo Logic (default = mezmo_default_category) default: mezmo_default_category required: - endpoint - compression - if: properties: type: enum: - vector deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Vector description: Publish log events to Vector additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true compression: type: string description: The compression strategy used on the encoded data prior to sending. enum: - gzip - none title: Compression default: none vector_ip: type: string title: Vector IP format: ipv4 description: The IP portion of the Vector host vector_port: type: number title: Vector Port description: The port that the Vector host listens on. version: type: string enum: - '1' - '2' title: Version default: '2' description: The version of the Vector sink which must match the source version. required: - compression - vector_ip - vector_port - version - if: properties: type: enum: - s3 deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: AWS S3 description: Publish events as objects in AWS S3 additionalProperties: false errorMessage: properties: bucket: must be a valid bucket name tags: must be a valid tag identifier properties: batch_timeout_secs: type: integer title: Batch Timeout (seconds) description: >- The maximum amount of time, in seconds, events will be buffered before being flushed to the destination enum: - 30 - 60 - 90 - 120 - 300 default: 300 ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true auth: type: object title: AWS Authentication description: Configures AWS authentication additionalProperties: false properties: access_key_id: type: string title: Access Key ID minLength: 1 description: The AWS access key id secret_access_key: type: string title: Secret Access Key minLength: 1 description: The AWS secret access key ui: display_type: secret required: - access_key_id - secret_access_key bucket: type: string title: Bucket pattern: (?!(^xn--|.+-s3alias$))^[a-z0-9][a-z0-9-]{1,61}[a-z0-9]$ description: The S3 bucket name. Do not include a leading s3:// or a trailing / prefix: type: string title: Prefix description: A prefix to apply to all object key names. default: / ui: display_type: data-template tags: type: array maxItems: 10 title: Tags description: A set of tags (key/value pairs) to apply to the object in S3. items: type: object properties: key: type: string title: Name description: The tag name pattern: ^[A-Za-z0-9_.:=+@/\-\s]*$ minLength: 1 maxLength: 128 value: type: string title: Value description: The tag value default: '' pattern: ^[A-Za-z0-9_.:=+@/\-\s]*$ minLength: 0 maxLength: 256 required: - key - value encoding: type: string description: The encoding to apply to the data. enum: - json - ndjson - text title: Encoding default: text compression: type: string description: The compression strategy used on the encoded data prior to sending. enum: - gzip - none title: Compression default: none region: type: string title: Region minLength: 1 description: The name of the AWS region that is targeted. file_consolidation: type: object ui: display_type: collapsable feature_flag: pipeline-node-sink-s3-file-consolidation title: File Consolidation description: >- This sink writes many small files out to S3. Enabling this process will allow the automatic consolidation of these small files into larger files of your choosing. This process will enable upon deployment and run on the chosen interval from `Processing Interval` creating files named `merged_[timestamp].log` where `timestamp` is the time since epoch. *NOTE*: only `text` and `ndjson` files written by the sink are consolidated. additionalProperties: false properties: enabled: type: boolean default: false title: Enabled description: Toggles whether the process is enabled. process_every_seconds: type: integer title: Processing Interval (seconds) description: How often to run the consolidation process. default: 600 minimum: 300 maximum: 3600 requested_size_bytes: type: integer title: Requested File Size (bytes) description: The size of the requested files. default: 500000000 minimum: 50000000 maximum: 10000000000 base_path: type: string title: Base Path default: '' maxLength: 512 description: >- The path from the container to begin recursivelylooking for files to merge. A blank value indicates root.Merged files will only contain data from the respectivefolder. required: - auth - bucket - prefix - encoding - compression - region - if: properties: type: enum: - blackhole deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Blackhole description: Drop data arriving at this destination additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true - if: properties: type: enum: - datadog-logs deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Datadog Logs description: Publish log events to Datadog additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true api_key: type: string title: Datadog API Key description: Datadog log application API key minLength: 1 ui: display_type: secret compression: type: string description: The compression strategy used on the encoded data prior to sending. enum: - gzip - none title: Compression default: gzip site: type: string description: The Datadog site (region) to send logs to. enum: - us1 - us3 - us5 - eu1 title: Datadog Site default: us1 required: - api_key - compression - site - if: properties: type: enum: - datadog-metrics deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Datadog Metrics description: >- Publish metrics events to Datadog. Sending custom metrics to Datadog may result in extra charges on your Datadog bill. Please go to https://docs.datadoghq.com/metrics/custom_metrics/ for more information on Datadog custom metrics pricing. additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true api_key: type: string title: Datadog API Key description: Datadog metrics application API key minLength: 1 maxLength: 512 ui: display_type: secret site: type: string description: The Datadog site (region) to send metrics to. enum: - us1 - us3 - us5 - eu1 title: Datadog Site default: us1 required: - api_key - site - if: properties: type: enum: - clickhouse deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Clickhouse description: Deliver log data to the ClickHouse database additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true endpoint: type: string minLength: 1 maxLength: 2048 title: Endpoint description: Endpoint of the Clickhouse server format: uri errorMessage: format: Must be a valid URI auth: title: Auth Strategy description: Authentication strategy allOf: - if: properties: strategy: enum: - basic then: properties: user: type: string title: User minLength: 1 maxLength: 512 description: The basic authentication user password: type: string title: Password minLength: 1 maxLength: 512 description: The basic authentication password ui: display_type: secret required: - user - password - if: properties: strategy: enum: - bearer then: properties: token: type: string title: Token minLength: 1 maxLength: 2048 description: The bearer token to use for authentication ui: display_type: secret required: - token properties: strategy: type: string title: Strategy description: The ES authentication strategy to use enum: - basic - bearer required: - strategy table: type: string title: Table description: The table that data is inserted into pattern: ^[a-zA-Z0-9_]+$ errorMessage: Invalid table name ui: display_type: data-template database: type: string title: Database description: The database that contains the table that data is inserted into pattern: ^[a-zA-Z0-9_]+$ errorMessage: Invalid database name ui: display_type: data-template compression: type: string enum: - gzip - none - snappy - zlib - zstd title: Compression description: The compression algorithm to use default: gzip skip_unknown_fields: type: boolean title: Skip Unknown Fields description: Allows ClickHouse to discard fields not present in the table schema default: false required: - endpoint - auth - table - if: properties: type: enum: - opentelemetry deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: OpenTelemetry description: Publish data to OpenTelemetry additionalProperties: false properties: ack_enabled: type: boolean title: End-to-end Acknowledgement description: Sets end-to-end acknowledgement on the destination default: true endpoint: type: string title: Collector Endpoint format: url errorMessage: format: Must be a valid URL minLength: 1 maxLength: 512 description: The OpenTelemetry Collector URL that receives the data auth: title: Authentication description: Configures HTTP authentication additionalProperties: type: string allOf: - if: properties: strategy: enum: - basic then: properties: user: type: string title: User minLength: 1 maxLength: 512 description: The basic authentication user password: type: string title: Password minLength: 1 maxLength: 512 description: The basic authentication password ui: display_type: secret required: - user - password - if: properties: strategy: enum: - bearer then: properties: token: type: string title: Token minLength: 1 maxLength: 1024 description: The bearer token ui: display_type: secret required: - token properties: strategy: type: string enum: - basic - bearer - none title: Strategy description: The authentication strategy to use default: none headers: type: array title: HTTP Headers description: >- An array of key/value objects for the headers to be sent with the request. items: type: object title: Headers description: A key/value object describing a header name and its value. additionalProperties: false properties: header_name: type: string title: Header Name minLength: 1 description: The name of the header. header_value: type: string title: Header Value minLength: 1 description: The value of the header. required: - header_name - header_value compression: type: string title: Compression description: The compression strategy (defaults to none). enum: - gzip - none default: none required: - endpoint - compression properties: type: type: string enum: - mezmo - mezmo-archive - http - cloudwatch-logs - cloudwatch-metrics - sqs - notification-channel - azure-blob-storage - kinesis-firehose - kinesis-streams - elasticsearch - gcp-cloud-monitoring - gcp-cloud-storage - gcp-cloud-operations - gcp-cloud-pubsub - honeycomb-logs - indexed-search - kafka - loki - new-relic - prometheus-remote-write - pulsar - redis - splunk-hec-logs - sumo-logic-logs - sumo-logic-metrics - vector - s3 - blackhole - datadog-logs - datadog-metrics - clickhouse - opentelemetry deploy_type: type: string enum: - saas - local default: saas title: oneOf: - type: string minLength: 1 maxLength: 512 pattern: ^[^<>"]+$ - type: 'null' errorMessage: invalid title name description: oneOf: - type: string minLength: 1 pattern: ^[^<>"]+$ - type: 'null' errorMessage: invalid description user_config: type: object inputs: type: array items: type: string description: >- A list of inputs for the given sink. Typically this is empty when creating a new sink via the UI, but is leveraged by the terraform provider when managing pipeline state. required: - type - user_config required: true parameters: - schema: type: string format: uuid in: path name: pipeline_id required: true responses: '200': description: Default Response /v3/pipeline/{pipeline_id}/source/{pipeline_source_id}: delete: summary: Remove source tags: - Pipeline Nodes description: Removes a source from a pipeline parameters: - schema: type: string format: uuid in: path name: pipeline_id required: true - schema: type: string format: uuid in: path name: pipeline_source_id required: true responses: '200': description: Default Response get: summary: Retrieve source tags: - Pipeline Nodes description: Fetches configuration of a source parameters: - schema: type: string format: uuid in: path name: pipeline_id required: true - schema: type: string format: uuid in: path name: pipeline_source_id required: true responses: '200': description: Default Response put: summary: Update source tags: - Pipeline Nodes description: Updates a source in a pipeline requestBody: content: application/json: schema: type: object allOf: - if: properties: type: enum: - http deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: HTTP description: Receive data from incoming HTTP requests additionalProperties: false properties: capture_metadata: type: boolean title: Capture Metadata description: |- Enable the inclusion of all http headers and query string parameters that were sent from the source. ui: feature_flag: pipeline-capture-metadata display_type: hidden default: true auto_parse: type: object title: Configure automated parsing additionalProperties: false ui: display_type: hidden properties: enabled: type: boolean default: false enabled_event_types: type: array items: type: string enum: - HTTPD_COMBINEDLOG - HTTPD_COMMONLOG - HTTPD_ERRORLOG - SYSLOG5424LINE - SYSLOGLINE - SYSLOGPAMSESSION - CRONLOG - MONGO3_LOG - NAGIOSLOGLINE - POSTGRESQL - RAILS3 - RUBY_LOGGER - REDISLOG - REDISMONLOG - S3_ACCESS_LOG - ELB_ACCESS_LOG - CLOUDFRONT_ACCESS_LOG - CATALINALOG - TOMCATLOG - SQUID3 - BIND9 - HAPROXYTCP - HAPROXYHTTP - BACULA_LOGLINE - BRO_HTTP - BRO_DNS - BRO_CONN - BRO_FILES - NETSCREENSESSIONLOG - CISCO_TAGGED_SYSLOG - CISCOFW104001 - CISCOFW104002 - CISCOFW104003 - CISCOFW104004 - CISCOFW105003 - CISCOFW105004 - CISCOFW105005 - CISCOFW105008 - CISCOFW105009 - CISCOFW106001 - CISCOFW106006_106007_106010 - CISCOFW106014 - CISCOFW106015 - CISCOFW106021 - CISCOFW106023 - CISCOFW106100_2_3 - CISCOFW106100 - CISCOFW304001 - CISCOFW110002 - CISCOFW302010 - CISCOFW302013_302014_302015_302016 - CISCOFW302020_302021 - CISCOFW305011 - CISCOFW313001_313004_313008 - CISCOFW313005 - CISCOFW321001 - CISCOFW402117 - CISCOFW402119 - CISCOFW419001 - CISCOFW419002 - CISCOFW500004 - CISCOFW602303_602304 - CISCOFW710001_710002_710003_710005_710006 - CISCOFW713172 - CISCOFW733100 - SHOREWALL - SFW2 decoding: oneOf: - enum: - bytes - enum: - json - enum: - ndjson - enum: - auto ui: display_type: list errorMessage: must be equal to one of the allowed values description: The decoding method for converting frames into data events. title: Decoding Method default: auto required: - decoding - if: properties: type: enum: - splunk-hec deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Splunk HEC description: Receive Splunk logs additionalProperties: false properties: capture_metadata: type: boolean title: Capture Metadata description: |- Enable the inclusion of all http headers and query string parameters that were sent from the source. ui: feature_flag: pipeline-capture-metadata display_type: hidden default: true auto_parse: type: object title: Configure automated parsing additionalProperties: false ui: display_type: hidden properties: enabled: type: boolean default: false enabled_event_types: type: array items: type: string enum: - HTTPD_COMBINEDLOG - HTTPD_COMMONLOG - HTTPD_ERRORLOG - SYSLOG5424LINE - SYSLOGLINE - SYSLOGPAMSESSION - CRONLOG - MONGO3_LOG - NAGIOSLOGLINE - POSTGRESQL - RAILS3 - RUBY_LOGGER - REDISLOG - REDISMONLOG - S3_ACCESS_LOG - ELB_ACCESS_LOG - CLOUDFRONT_ACCESS_LOG - CATALINALOG - TOMCATLOG - SQUID3 - BIND9 - HAPROXYTCP - HAPROXYHTTP - BACULA_LOGLINE - BRO_HTTP - BRO_DNS - BRO_CONN - BRO_FILES - NETSCREENSESSIONLOG - CISCO_TAGGED_SYSLOG - CISCOFW104001 - CISCOFW104002 - CISCOFW104003 - CISCOFW104004 - CISCOFW105003 - CISCOFW105004 - CISCOFW105005 - CISCOFW105008 - CISCOFW105009 - CISCOFW106001 - CISCOFW106006_106007_106010 - CISCOFW106014 - CISCOFW106015 - CISCOFW106021 - CISCOFW106023 - CISCOFW106100_2_3 - CISCOFW106100 - CISCOFW304001 - CISCOFW110002 - CISCOFW302010 - CISCOFW302013_302014_302015_302016 - CISCOFW302020_302021 - CISCOFW305011 - CISCOFW313001_313004_313008 - CISCOFW313005 - CISCOFW321001 - CISCOFW402117 - CISCOFW402119 - CISCOFW419001 - CISCOFW419002 - CISCOFW500004 - CISCOFW602303_602304 - CISCOFW710001_710002_710003_710005_710006 - CISCOFW713172 - CISCOFW733100 - SHOREWALL - SFW2 - if: properties: type: enum: - s3 deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: AWS S3 description: Collect data from object contents in AWS S3 additionalProperties: false properties: sqs_queue_url: type: string title: SQS Queue URL description: >- The URL of a AWS SQS queue configured to receive S3 bucket notifications for the desired S3 buckets format: uri errorMessage: format: Must be a valid URI auth: type: object title: AWS Authentication description: Configures AWS authentication additionalProperties: false properties: access_key_id: type: string title: Access Key ID minLength: 1 description: The AWS access key id secret_access_key: type: string title: Secret Access Key minLength: 1 description: The AWS secret access key ui: display_type: secret required: - access_key_id - secret_access_key compression: type: string title: Compression description: The compression format of the S3 objects enum: - auto - gzip - none - zstd default: auto region: type: string title: Region minLength: 1 description: The name of the source AWS region. required: - sqs_queue_url - auth - compression - region - if: properties: type: enum: - sqs deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: AWS SQS description: Collect messages from AWS SQS additionalProperties: false properties: queue_url: type: string title: Queue URL description: The URL of an AWS SQS queue format: uri errorMessage: format: Must be a valid URI maxLength: 128 auth: type: object title: AWS Authentication description: Configures AWS authentication additionalProperties: false properties: access_key_id: type: string title: Access Key ID minLength: 1 description: The AWS access key id maxLength: 128 secret_access_key: type: string title: Secret Access Key minLength: 1 description: The AWS secret access key ui: display_type: secret maxLength: 128 required: - access_key_id - secret_access_key region: type: string title: Region minLength: 1 description: The name of the source AWS region. maxLength: 128 required: - queue_url - auth - region - if: properties: type: enum: - demo-logs deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: title: Demo Logs description: Generate log messages for testing and demonstration. additionalProperties: type: array title: Log Lines description: >- A list of user-provided log lines that will be sent through the pipeline infinitely. A maximum of 25 lines is allowed. The lines can be either plain text or ndjson. If ndjson is detected, the encoding for the source will be set to JSON, and lines that are not valid will show up as a visible component error. Otherwise, text encoding will be used. minItems: 1 maxItems: 25 items: type: string minLength: 1 maxLength: 4096 title: Line description: A single line of log data, either stringified JSON or plain text. ui: display_type: user-provided-lines allOf: - if: properties: format: enum: - user_provided required: - format then: properties: lines: type: array title: Log Lines description: >- A list of user-provided log lines that will be sent through the pipeline infinitely. A maximum of 25 lines is allowed. The lines can be either plain text or ndjson. If ndjson is detected, the encoding for the source will be set to JSON, and lines that are not valid will show up as a visible component error. Otherwise, text encoding will be used. minItems: 1 maxItems: 25 items: type: string minLength: 1 maxLength: 4096 title: Line description: >- A single line of log data, either stringified JSON or plain text. ui: display_type: user-provided-lines required: - lines - allOf: - errorMessage: '`lines` is only valid when `format` is `user_provided`' if: not: type: object properties: format: enum: - user_provided required: - format then: not: type: object properties: lines: type: array title: Log Lines description: >- A list of user-provided log lines that will be sent through the pipeline infinitely. A maximum of 25 lines is allowed. The lines can be either plain text or ndjson. If ndjson is detected, the encoding for the source will be set to JSON, and lines that are not valid will show up as a visible component error. Otherwise, text encoding will be used. minItems: 1 maxItems: 25 items: type: string minLength: 1 maxLength: 4096 title: Line description: >- A single line of log data, either stringified JSON or plain text. ui: display_type: user-provided-lines required: - lines properties: interval_ms: type: number title: Interval description: >- The amount of time to pause between each demo log. Fractions of a second are accepted to increase throughput. e.g. 500 ms == 120 lines per minute default: 1000 minimum: 500 maximum: 3600000 ui: display_type: milliseconds-to-human-readable format: title: Format description: The format of the log messages. All of the data is generated. ui: display_type: list errorMessage: must be one of the available types of demo data oneOf: - label: HTTP JSON description: HTTP request data in JSON format. enum: - json - label: Sensor JSON description: Data from environment sensors in JSON format. enum: - env_sensor - label: Financial JSON description: Financial data in JSON format. enum: - financial - label: Nginx Requests description: Nginx logs in text format. enum: - nginx - label: Apache Requests description: Apache request logs in text format. enum: - apache_common - label: Apache Errors description: Apache error logs in text format. enum: - apache_error - label: BSD Syslog description: Mock BSD syslog entries in rfc3164 text format. enum: - bsd_syslog - label: Syslog description: Mock syslog entries in rfc5424 text format. enum: - syslog - label: HTTP Metrics description: HTTP-based metrics in the Mezmo metrics JSON format. enum: - http_metrics - label: Generic Metrics description: Generic metrics in the Mezmo metrics JSON format enum: - generic_metrics - label: User-Provided Logs description: >- Custom logs provided by the user. These can either be plain text or serialized JSON. enum: - user_provided - label: Infrastructure Logs description: >- Infrastructure logs consisting of sysdig, nginx, HDFS and spark logs enum: - infrastructure - label: Kubernetes Logs description: Kubernetes logs in JSON format enum: - kubernetes default: json required: - format - if: properties: type: enum: - kinesis-firehose deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Kinesis Firehose description: Receive Kinesis Firehose data additionalProperties: false properties: capture_metadata: type: boolean title: Capture Metadata description: |- Enable the inclusion of all http headers and query string parameters that were sent from the source. ui: feature_flag: pipeline-capture-metadata display_type: hidden default: true auto_parse: type: object title: Configure automated parsing additionalProperties: false ui: display_type: hidden properties: enabled: type: boolean default: false enabled_event_types: type: array items: type: string enum: - HTTPD_COMBINEDLOG - HTTPD_COMMONLOG - HTTPD_ERRORLOG - SYSLOG5424LINE - SYSLOGLINE - SYSLOGPAMSESSION - CRONLOG - MONGO3_LOG - NAGIOSLOGLINE - POSTGRESQL - RAILS3 - RUBY_LOGGER - REDISLOG - REDISMONLOG - S3_ACCESS_LOG - ELB_ACCESS_LOG - CLOUDFRONT_ACCESS_LOG - CATALINALOG - TOMCATLOG - SQUID3 - BIND9 - HAPROXYTCP - HAPROXYHTTP - BACULA_LOGLINE - BRO_HTTP - BRO_DNS - BRO_CONN - BRO_FILES - NETSCREENSESSIONLOG - CISCO_TAGGED_SYSLOG - CISCOFW104001 - CISCOFW104002 - CISCOFW104003 - CISCOFW104004 - CISCOFW105003 - CISCOFW105004 - CISCOFW105005 - CISCOFW105008 - CISCOFW105009 - CISCOFW106001 - CISCOFW106006_106007_106010 - CISCOFW106014 - CISCOFW106015 - CISCOFW106021 - CISCOFW106023 - CISCOFW106100_2_3 - CISCOFW106100 - CISCOFW304001 - CISCOFW110002 - CISCOFW302010 - CISCOFW302013_302014_302015_302016 - CISCOFW302020_302021 - CISCOFW305011 - CISCOFW313001_313004_313008 - CISCOFW313005 - CISCOFW321001 - CISCOFW402117 - CISCOFW402119 - CISCOFW419001 - CISCOFW419002 - CISCOFW500004 - CISCOFW602303_602304 - CISCOFW710001_710002_710003_710005_710006 - CISCOFW713172 - CISCOFW733100 - SHOREWALL - SFW2 format: type: string enum: - json - text default: json title: Data Format description: >- This specifies what the data format will be after it is base64 decoded. If it is JSON, it will be automatically parsed. required: - format - if: properties: type: enum: - fluent deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Fluent (Fluentd, Fluent Bit) description: Receive data from Fluentd or Fluent Bit additionalProperties: false properties: capture_metadata: type: boolean title: Capture Metadata description: |- Enable the inclusion of all http headers and query string parameters that were sent from the source. ui: feature_flag: pipeline-capture-metadata display_type: hidden default: true auto_parse: type: object title: Configure automated parsing additionalProperties: false ui: display_type: hidden properties: enabled: type: boolean default: false enabled_event_types: type: array items: type: string enum: - HTTPD_COMBINEDLOG - HTTPD_COMMONLOG - HTTPD_ERRORLOG - SYSLOG5424LINE - SYSLOGLINE - SYSLOGPAMSESSION - CRONLOG - MONGO3_LOG - NAGIOSLOGLINE - POSTGRESQL - RAILS3 - RUBY_LOGGER - REDISLOG - REDISMONLOG - S3_ACCESS_LOG - ELB_ACCESS_LOG - CLOUDFRONT_ACCESS_LOG - CATALINALOG - TOMCATLOG - SQUID3 - BIND9 - HAPROXYTCP - HAPROXYHTTP - BACULA_LOGLINE - BRO_HTTP - BRO_DNS - BRO_CONN - BRO_FILES - NETSCREENSESSIONLOG - CISCO_TAGGED_SYSLOG - CISCOFW104001 - CISCOFW104002 - CISCOFW104003 - CISCOFW104004 - CISCOFW105003 - CISCOFW105004 - CISCOFW105005 - CISCOFW105008 - CISCOFW105009 - CISCOFW106001 - CISCOFW106006_106007_106010 - CISCOFW106014 - CISCOFW106015 - CISCOFW106021 - CISCOFW106023 - CISCOFW106100_2_3 - CISCOFW106100 - CISCOFW304001 - CISCOFW110002 - CISCOFW302010 - CISCOFW302013_302014_302015_302016 - CISCOFW302020_302021 - CISCOFW305011 - CISCOFW313001_313004_313008 - CISCOFW313005 - CISCOFW321001 - CISCOFW402117 - CISCOFW402119 - CISCOFW419001 - CISCOFW419002 - CISCOFW500004 - CISCOFW602303_602304 - CISCOFW710001_710002_710003_710005_710006 - CISCOFW713172 - CISCOFW733100 - SHOREWALL - SFW2 decoding: type: string description: The decoding method for converting frames into data events. enum: - bytes - json - ndjson title: Decoding Method default: json required: - decoding - if: properties: type: enum: - log-analysis deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Mezmo Log Analysis description: Receive data directly from your Mezmo Log Analysis account additionalProperties: false - if: properties: type: enum: - log-analysis-ingestion deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Mezmo Log Analysis Ingestion description: >- Receive unparsed source data directly from your Mezmo Log Analysis account. additionalProperties: type: string - if: properties: type: enum: - logstash deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Logstash description: Receive Logstash data additionalProperties: false properties: capture_metadata: type: boolean title: Capture Metadata description: |- Enable the inclusion of all http headers and query string parameters that were sent from the source. ui: feature_flag: pipeline-capture-metadata display_type: hidden default: true auto_parse: type: object title: Configure automated parsing additionalProperties: false ui: display_type: hidden properties: enabled: type: boolean default: false enabled_event_types: type: array items: type: string enum: - HTTPD_COMBINEDLOG - HTTPD_COMMONLOG - HTTPD_ERRORLOG - SYSLOG5424LINE - SYSLOGLINE - SYSLOGPAMSESSION - CRONLOG - MONGO3_LOG - NAGIOSLOGLINE - POSTGRESQL - RAILS3 - RUBY_LOGGER - REDISLOG - REDISMONLOG - S3_ACCESS_LOG - ELB_ACCESS_LOG - CLOUDFRONT_ACCESS_LOG - CATALINALOG - TOMCATLOG - SQUID3 - BIND9 - HAPROXYTCP - HAPROXYHTTP - BACULA_LOGLINE - BRO_HTTP - BRO_DNS - BRO_CONN - BRO_FILES - NETSCREENSESSIONLOG - CISCO_TAGGED_SYSLOG - CISCOFW104001 - CISCOFW104002 - CISCOFW104003 - CISCOFW104004 - CISCOFW105003 - CISCOFW105004 - CISCOFW105005 - CISCOFW105008 - CISCOFW105009 - CISCOFW106001 - CISCOFW106006_106007_106010 - CISCOFW106014 - CISCOFW106015 - CISCOFW106021 - CISCOFW106023 - CISCOFW106100_2_3 - CISCOFW106100 - CISCOFW304001 - CISCOFW110002 - CISCOFW302010 - CISCOFW302013_302014_302015_302016 - CISCOFW302020_302021 - CISCOFW305011 - CISCOFW313001_313004_313008 - CISCOFW313005 - CISCOFW321001 - CISCOFW402117 - CISCOFW402119 - CISCOFW419001 - CISCOFW419002 - CISCOFW500004 - CISCOFW602303_602304 - CISCOFW710001_710002_710003_710005_710006 - CISCOFW713172 - CISCOFW733100 - SHOREWALL - SFW2 format: type: string enum: - json - text default: json title: Data Format description: The format of the logstash data required: - format - if: properties: type: enum: - mezmo-agent deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Mezmo Agent description: Receive logs from the Mezmo Agent additionalProperties: false properties: capture_metadata: type: boolean title: Capture Metadata description: |- Enable the inclusion of all http headers and query string parameters that were sent from the source. ui: feature_flag: pipeline-capture-metadata display_type: hidden default: true auto_parse: type: object title: Configure automated parsing additionalProperties: false ui: display_type: hidden properties: enabled: type: boolean default: false enabled_event_types: type: array items: type: string enum: - HTTPD_COMBINEDLOG - HTTPD_COMMONLOG - HTTPD_ERRORLOG - SYSLOG5424LINE - SYSLOGLINE - SYSLOGPAMSESSION - CRONLOG - MONGO3_LOG - NAGIOSLOGLINE - POSTGRESQL - RAILS3 - RUBY_LOGGER - REDISLOG - REDISMONLOG - S3_ACCESS_LOG - ELB_ACCESS_LOG - CLOUDFRONT_ACCESS_LOG - CATALINALOG - TOMCATLOG - SQUID3 - BIND9 - HAPROXYTCP - HAPROXYHTTP - BACULA_LOGLINE - BRO_HTTP - BRO_DNS - BRO_CONN - BRO_FILES - NETSCREENSESSIONLOG - CISCO_TAGGED_SYSLOG - CISCOFW104001 - CISCOFW104002 - CISCOFW104003 - CISCOFW104004 - CISCOFW105003 - CISCOFW105004 - CISCOFW105005 - CISCOFW105008 - CISCOFW105009 - CISCOFW106001 - CISCOFW106006_106007_106010 - CISCOFW106014 - CISCOFW106015 - CISCOFW106021 - CISCOFW106023 - CISCOFW106100_2_3 - CISCOFW106100 - CISCOFW304001 - CISCOFW110002 - CISCOFW302010 - CISCOFW302013_302014_302015_302016 - CISCOFW302020_302021 - CISCOFW305011 - CISCOFW313001_313004_313008 - CISCOFW313005 - CISCOFW321001 - CISCOFW402117 - CISCOFW402119 - CISCOFW419001 - CISCOFW419002 - CISCOFW500004 - CISCOFW602303_602304 - CISCOFW710001_710002_710003_710005_710006 - CISCOFW713172 - CISCOFW733100 - SHOREWALL - SFW2 - if: properties: type: enum: - mezmo-datadog-source deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Datadog Agent description: Send logs and metrics data directly from an installed datadog agent additionalProperties: false properties: capture_metadata: type: boolean title: Capture Metadata description: |- Enable the inclusion of all http headers and query string parameters that were sent from the source. ui: feature_flag: pipeline-capture-metadata display_type: hidden default: true auto_parse: type: object title: Configure automated parsing additionalProperties: false ui: display_type: hidden properties: enabled: type: boolean default: false enabled_event_types: type: array items: type: string enum: - HTTPD_COMBINEDLOG - HTTPD_COMMONLOG - HTTPD_ERRORLOG - SYSLOG5424LINE - SYSLOGLINE - SYSLOGPAMSESSION - CRONLOG - MONGO3_LOG - NAGIOSLOGLINE - POSTGRESQL - RAILS3 - RUBY_LOGGER - REDISLOG - REDISMONLOG - S3_ACCESS_LOG - ELB_ACCESS_LOG - CLOUDFRONT_ACCESS_LOG - CATALINALOG - TOMCATLOG - SQUID3 - BIND9 - HAPROXYTCP - HAPROXYHTTP - BACULA_LOGLINE - BRO_HTTP - BRO_DNS - BRO_CONN - BRO_FILES - NETSCREENSESSIONLOG - CISCO_TAGGED_SYSLOG - CISCOFW104001 - CISCOFW104002 - CISCOFW104003 - CISCOFW104004 - CISCOFW105003 - CISCOFW105004 - CISCOFW105005 - CISCOFW105008 - CISCOFW105009 - CISCOFW106001 - CISCOFW106006_106007_106010 - CISCOFW106014 - CISCOFW106015 - CISCOFW106021 - CISCOFW106023 - CISCOFW106100_2_3 - CISCOFW106100 - CISCOFW304001 - CISCOFW110002 - CISCOFW302010 - CISCOFW302013_302014_302015_302016 - CISCOFW302020_302021 - CISCOFW305011 - CISCOFW313001_313004_313008 - CISCOFW313005 - CISCOFW321001 - CISCOFW402117 - CISCOFW402119 - CISCOFW419001 - CISCOFW419002 - CISCOFW500004 - CISCOFW602303_602304 - CISCOFW710001_710002_710003_710005_710006 - CISCOFW713172 - CISCOFW733100 - SHOREWALL - SFW2 - if: properties: type: enum: - pipeline-data-restoration deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Mezmo Pipeline Data Restoration description: Receive data directly from your Mezmo Pipline Data Restoration additionalProperties: false - if: properties: type: enum: - webhook deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Webhook (WebSub) description: Receive data from incoming webhooks using the WebSub protocol. additionalProperties: false properties: capture_metadata: type: boolean title: Capture Metadata description: |- Enable the inclusion of all http headers and query string parameters that were sent from the source. ui: feature_flag: pipeline-capture-metadata display_type: hidden default: true auto_parse: type: object title: Configure automated parsing additionalProperties: false ui: display_type: hidden properties: enabled: type: boolean default: false enabled_event_types: type: array items: type: string enum: - HTTPD_COMBINEDLOG - HTTPD_COMMONLOG - HTTPD_ERRORLOG - SYSLOG5424LINE - SYSLOGLINE - SYSLOGPAMSESSION - CRONLOG - MONGO3_LOG - NAGIOSLOGLINE - POSTGRESQL - RAILS3 - RUBY_LOGGER - REDISLOG - REDISMONLOG - S3_ACCESS_LOG - ELB_ACCESS_LOG - CLOUDFRONT_ACCESS_LOG - CATALINALOG - TOMCATLOG - SQUID3 - BIND9 - HAPROXYTCP - HAPROXYHTTP - BACULA_LOGLINE - BRO_HTTP - BRO_DNS - BRO_CONN - BRO_FILES - NETSCREENSESSIONLOG - CISCO_TAGGED_SYSLOG - CISCOFW104001 - CISCOFW104002 - CISCOFW104003 - CISCOFW104004 - CISCOFW105003 - CISCOFW105004 - CISCOFW105005 - CISCOFW105008 - CISCOFW105009 - CISCOFW106001 - CISCOFW106006_106007_106010 - CISCOFW106014 - CISCOFW106015 - CISCOFW106021 - CISCOFW106023 - CISCOFW106100_2_3 - CISCOFW106100 - CISCOFW304001 - CISCOFW110002 - CISCOFW302010 - CISCOFW302013_302014_302015_302016 - CISCOFW302020_302021 - CISCOFW305011 - CISCOFW313001_313004_313008 - CISCOFW313005 - CISCOFW321001 - CISCOFW402117 - CISCOFW402119 - CISCOFW419001 - CISCOFW419002 - CISCOFW500004 - CISCOFW602303_602304 - CISCOFW710001_710002_710003_710005_710006 - CISCOFW713172 - CISCOFW733100 - SHOREWALL - SFW2 - if: properties: type: enum: - kafka deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Kafka Source description: Collect events from Kafka compatible brokers additionalProperties: type: string properties: brokers: type: array title: Brokers uniqueItems: true minItems: 1 description: An array of key/val objects for each host/port of a Kafka broker. items: type: object title: Host/Port description: The host and port for the Kafka broker. additionalProperties: false properties: host: title: Host description: The hostname or IP address of the broker. anyOf: - type: string format: ipv4 errorMessage: format: Must be a valid IPv4 address - type: string format: hostname maxLength: 512 errorMessage: format: Must be a valid hostname port: type: number minimum: 1 maximum: 65535 title: Port description: The port that the broker listens on. required: - host - port topics: type: array title: Topics items: type: string title: Topic minLength: 1 maxLength: 256 description: The Kafka topic name. minItems: 1 description: List of Kafka topics names to read events from. group_id: type: string title: Consumer Group minLength: 1 description: The consumer group name to be used to consume events from Kafka. tls_enabled: type: boolean title: TLS enabled description: Whether or not to require TLS for outgoing connections. default: true sasl_enabled: type: boolean title: SASL/SCRAM Enabled description: Enable SASL/SCRAM authentication with Kafka default: true decoding_codec: type: string enum: - bytes - json title: Decoding description: Configures how events are decoded from raw bytes default: bytes if: properties: sasl_enabled: enum: - true then: properties: sasl_mechanism: type: string enum: - SCRAM-SHA-512 - SCRAM-SHA-256 - PLAIN title: SASL description: The Kafka SASL/SCRAM mechanism default: PLAIN sasl_username: type: string title: Username description: The SASL/SCRAM username minLength: 1 sasl_password: type: string title: Password description: The SASL/SCRAM username ui: display_type: secret minLength: 1 required: - sasl_mechanism - sasl_username - sasl_password required: - brokers - topics - group_id - decoding_codec - if: properties: type: enum: - azure-event-hub deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Azure Event Hub Source description: Collect events from Azure Event Hub using Azure Kafka Endpoint additionalProperties: false properties: connection_string: type: string title: Connection String minLength: 1 maxLength: 512 description: The Connection String as it appears in hub consumer SAS Policy ui: display_type: secret namespace: type: string title: Namespace minLength: 1 maxLength: 256 description: The Event Hub Namespace topics: type: array title: Event Hub Names items: type: string title: Name minLength: 1 maxLength: 256 description: Azure Event Hub instance name. minItems: 1 description: The list of Azure Event Hub name(s) to read events from. group_id: type: string title: Consumer Group minLength: 1 maxLength: 256 description: The consumer group name that this consumer belongs to. decoding_codec: type: string enum: - bytes - json title: Decoding description: Configures how events are decoded from raw bytes default: bytes required: - connection_string - namespace - topics - group_id - decoding_codec - if: properties: type: enum: - prometheus-remote-write deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Prometheus Remote Write description: Receive prometheus remote write protobuf data from HTTP requests additionalProperties: false properties: capture_metadata: type: boolean title: Capture Metadata description: |- Enable the inclusion of all http headers and query string parameters that were sent from the source. ui: feature_flag: pipeline-capture-metadata display_type: hidden default: true auto_parse: type: object title: Configure automated parsing additionalProperties: false ui: display_type: hidden properties: enabled: type: boolean default: false enabled_event_types: type: array items: type: string enum: - HTTPD_COMBINEDLOG - HTTPD_COMMONLOG - HTTPD_ERRORLOG - SYSLOG5424LINE - SYSLOGLINE - SYSLOGPAMSESSION - CRONLOG - MONGO3_LOG - NAGIOSLOGLINE - POSTGRESQL - RAILS3 - RUBY_LOGGER - REDISLOG - REDISMONLOG - S3_ACCESS_LOG - ELB_ACCESS_LOG - CLOUDFRONT_ACCESS_LOG - CATALINALOG - TOMCATLOG - SQUID3 - BIND9 - HAPROXYTCP - HAPROXYHTTP - BACULA_LOGLINE - BRO_HTTP - BRO_DNS - BRO_CONN - BRO_FILES - NETSCREENSESSIONLOG - CISCO_TAGGED_SYSLOG - CISCOFW104001 - CISCOFW104002 - CISCOFW104003 - CISCOFW104004 - CISCOFW105003 - CISCOFW105004 - CISCOFW105005 - CISCOFW105008 - CISCOFW105009 - CISCOFW106001 - CISCOFW106006_106007_106010 - CISCOFW106014 - CISCOFW106015 - CISCOFW106021 - CISCOFW106023 - CISCOFW106100_2_3 - CISCOFW106100 - CISCOFW304001 - CISCOFW110002 - CISCOFW302010 - CISCOFW302013_302014_302015_302016 - CISCOFW302020_302021 - CISCOFW305011 - CISCOFW313001_313004_313008 - CISCOFW313005 - CISCOFW321001 - CISCOFW402117 - CISCOFW402119 - CISCOFW419001 - CISCOFW419002 - CISCOFW500004 - CISCOFW602303_602304 - CISCOFW710001_710002_710003_710005_710006 - CISCOFW713172 - CISCOFW733100 - SHOREWALL - SFW2 - if: properties: type: enum: - open-telemetry-metrics deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: OpenTelemetry Metrics description: Receive metrics from OpenTelemetry over HTTP additionalProperties: false properties: capture_metadata: type: boolean title: Capture Metadata description: |- Enable the inclusion of all http headers and query string parameters that were sent from the source. ui: feature_flag: pipeline-capture-metadata display_type: hidden default: true auto_parse: type: object title: Configure automated parsing additionalProperties: false ui: display_type: hidden properties: enabled: type: boolean default: false enabled_event_types: type: array items: type: string enum: - HTTPD_COMBINEDLOG - HTTPD_COMMONLOG - HTTPD_ERRORLOG - SYSLOG5424LINE - SYSLOGLINE - SYSLOGPAMSESSION - CRONLOG - MONGO3_LOG - NAGIOSLOGLINE - POSTGRESQL - RAILS3 - RUBY_LOGGER - REDISLOG - REDISMONLOG - S3_ACCESS_LOG - ELB_ACCESS_LOG - CLOUDFRONT_ACCESS_LOG - CATALINALOG - TOMCATLOG - SQUID3 - BIND9 - HAPROXYTCP - HAPROXYHTTP - BACULA_LOGLINE - BRO_HTTP - BRO_DNS - BRO_CONN - BRO_FILES - NETSCREENSESSIONLOG - CISCO_TAGGED_SYSLOG - CISCOFW104001 - CISCOFW104002 - CISCOFW104003 - CISCOFW104004 - CISCOFW105003 - CISCOFW105004 - CISCOFW105005 - CISCOFW105008 - CISCOFW105009 - CISCOFW106001 - CISCOFW106006_106007_106010 - CISCOFW106014 - CISCOFW106015 - CISCOFW106021 - CISCOFW106023 - CISCOFW106100_2_3 - CISCOFW106100 - CISCOFW304001 - CISCOFW110002 - CISCOFW302010 - CISCOFW302013_302014_302015_302016 - CISCOFW302020_302021 - CISCOFW305011 - CISCOFW313001_313004_313008 - CISCOFW313005 - CISCOFW321001 - CISCOFW402117 - CISCOFW402119 - CISCOFW419001 - CISCOFW419002 - CISCOFW500004 - CISCOFW602303_602304 - CISCOFW710001_710002_710003_710005_710006 - CISCOFW713172 - CISCOFW733100 - SHOREWALL - SFW2 - if: properties: type: enum: - open-telemetry-logs deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: OpenTelemetry Logs description: Receive logs from OpenTelemetry over HTTP additionalProperties: false properties: capture_metadata: type: boolean title: Capture Metadata description: |- Enable the inclusion of all http headers and query string parameters that were sent from the source. ui: feature_flag: pipeline-capture-metadata display_type: hidden default: true auto_parse: type: object title: Configure automated parsing additionalProperties: false ui: display_type: hidden properties: enabled: type: boolean default: false enabled_event_types: type: array items: type: string enum: - HTTPD_COMBINEDLOG - HTTPD_COMMONLOG - HTTPD_ERRORLOG - SYSLOG5424LINE - SYSLOGLINE - SYSLOGPAMSESSION - CRONLOG - MONGO3_LOG - NAGIOSLOGLINE - POSTGRESQL - RAILS3 - RUBY_LOGGER - REDISLOG - REDISMONLOG - S3_ACCESS_LOG - ELB_ACCESS_LOG - CLOUDFRONT_ACCESS_LOG - CATALINALOG - TOMCATLOG - SQUID3 - BIND9 - HAPROXYTCP - HAPROXYHTTP - BACULA_LOGLINE - BRO_HTTP - BRO_DNS - BRO_CONN - BRO_FILES - NETSCREENSESSIONLOG - CISCO_TAGGED_SYSLOG - CISCOFW104001 - CISCOFW104002 - CISCOFW104003 - CISCOFW104004 - CISCOFW105003 - CISCOFW105004 - CISCOFW105005 - CISCOFW105008 - CISCOFW105009 - CISCOFW106001 - CISCOFW106006_106007_106010 - CISCOFW106014 - CISCOFW106015 - CISCOFW106021 - CISCOFW106023 - CISCOFW106100_2_3 - CISCOFW106100 - CISCOFW304001 - CISCOFW110002 - CISCOFW302010 - CISCOFW302013_302014_302015_302016 - CISCOFW302020_302021 - CISCOFW305011 - CISCOFW313001_313004_313008 - CISCOFW313005 - CISCOFW321001 - CISCOFW402117 - CISCOFW402119 - CISCOFW419001 - CISCOFW419002 - CISCOFW500004 - CISCOFW602303_602304 - CISCOFW710001_710002_710003_710005_710006 - CISCOFW713172 - CISCOFW733100 - SHOREWALL - SFW2 - if: properties: type: enum: - open-telemetry-traces deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: OpenTelemetry Traces description: Receive traces from OpenTelemetry over HTTP additionalProperties: false properties: capture_metadata: type: boolean title: Capture Metadata description: |- Enable the inclusion of all http headers and query string parameters that were sent from the source. ui: feature_flag: pipeline-capture-metadata display_type: hidden default: true auto_parse: type: object title: Configure automated parsing additionalProperties: false ui: display_type: hidden properties: enabled: type: boolean default: false enabled_event_types: type: array items: type: string enum: - HTTPD_COMBINEDLOG - HTTPD_COMMONLOG - HTTPD_ERRORLOG - SYSLOG5424LINE - SYSLOGLINE - SYSLOGPAMSESSION - CRONLOG - MONGO3_LOG - NAGIOSLOGLINE - POSTGRESQL - RAILS3 - RUBY_LOGGER - REDISLOG - REDISMONLOG - S3_ACCESS_LOG - ELB_ACCESS_LOG - CLOUDFRONT_ACCESS_LOG - CATALINALOG - TOMCATLOG - SQUID3 - BIND9 - HAPROXYTCP - HAPROXYHTTP - BACULA_LOGLINE - BRO_HTTP - BRO_DNS - BRO_CONN - BRO_FILES - NETSCREENSESSIONLOG - CISCO_TAGGED_SYSLOG - CISCOFW104001 - CISCOFW104002 - CISCOFW104003 - CISCOFW104004 - CISCOFW105003 - CISCOFW105004 - CISCOFW105005 - CISCOFW105008 - CISCOFW105009 - CISCOFW106001 - CISCOFW106006_106007_106010 - CISCOFW106014 - CISCOFW106015 - CISCOFW106021 - CISCOFW106023 - CISCOFW106100_2_3 - CISCOFW106100 - CISCOFW304001 - CISCOFW110002 - CISCOFW302010 - CISCOFW302013_302014_302015_302016 - CISCOFW302020_302021 - CISCOFW305011 - CISCOFW313001_313004_313008 - CISCOFW313005 - CISCOFW321001 - CISCOFW402117 - CISCOFW402119 - CISCOFW419001 - CISCOFW419002 - CISCOFW500004 - CISCOFW602303_602304 - CISCOFW710001_710002_710003_710005_710006 - CISCOFW713172 - CISCOFW733100 - SHOREWALL - SFW2 properties: title: oneOf: - type: string minLength: 1 maxLength: 512 pattern: ^[^<>"]+$ - type: 'null' errorMessage: invalid title name description: oneOf: - type: string minLength: 1 pattern: ^[^<>"]+$ - type: 'null' errorMessage: invalid description type: type: string enum: - http - splunk-hec - s3 - sqs - demo-logs - kinesis-firehose - fluent - log-analysis - log-analysis-ingestion - logstash - mezmo-agent - mezmo-datadog-source - pipeline-data-restoration - webhook - kafka - azure-event-hub - prometheus-remote-write - open-telemetry-metrics - open-telemetry-logs - open-telemetry-traces deploy_type: type: string enum: - saas - local default: saas user_config: type: object generation_id: type: integer required: - type - user_config - generation_id required: true parameters: - schema: type: string format: uuid in: path name: pipeline_id required: true - schema: type: string format: uuid in: path name: pipeline_source_id required: true responses: '200': description: Default Response /v3/pipeline/{pipeline_id}/source: post: summary: Add source tags: - Pipeline Nodes description: Adds a source to a pipeline requestBody: content: application/json: schema: type: object additionalProperties: false allOf: - if: properties: type: enum: - http deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: HTTP description: Receive data from incoming HTTP requests additionalProperties: false properties: capture_metadata: type: boolean title: Capture Metadata description: |- Enable the inclusion of all http headers and query string parameters that were sent from the source. ui: feature_flag: pipeline-capture-metadata display_type: hidden default: true auto_parse: type: object title: Configure automated parsing additionalProperties: false ui: display_type: hidden properties: enabled: type: boolean default: false enabled_event_types: type: array items: type: string enum: - HTTPD_COMBINEDLOG - HTTPD_COMMONLOG - HTTPD_ERRORLOG - SYSLOG5424LINE - SYSLOGLINE - SYSLOGPAMSESSION - CRONLOG - MONGO3_LOG - NAGIOSLOGLINE - POSTGRESQL - RAILS3 - RUBY_LOGGER - REDISLOG - REDISMONLOG - S3_ACCESS_LOG - ELB_ACCESS_LOG - CLOUDFRONT_ACCESS_LOG - CATALINALOG - TOMCATLOG - SQUID3 - BIND9 - HAPROXYTCP - HAPROXYHTTP - BACULA_LOGLINE - BRO_HTTP - BRO_DNS - BRO_CONN - BRO_FILES - NETSCREENSESSIONLOG - CISCO_TAGGED_SYSLOG - CISCOFW104001 - CISCOFW104002 - CISCOFW104003 - CISCOFW104004 - CISCOFW105003 - CISCOFW105004 - CISCOFW105005 - CISCOFW105008 - CISCOFW105009 - CISCOFW106001 - CISCOFW106006_106007_106010 - CISCOFW106014 - CISCOFW106015 - CISCOFW106021 - CISCOFW106023 - CISCOFW106100_2_3 - CISCOFW106100 - CISCOFW304001 - CISCOFW110002 - CISCOFW302010 - CISCOFW302013_302014_302015_302016 - CISCOFW302020_302021 - CISCOFW305011 - CISCOFW313001_313004_313008 - CISCOFW313005 - CISCOFW321001 - CISCOFW402117 - CISCOFW402119 - CISCOFW419001 - CISCOFW419002 - CISCOFW500004 - CISCOFW602303_602304 - CISCOFW710001_710002_710003_710005_710006 - CISCOFW713172 - CISCOFW733100 - SHOREWALL - SFW2 decoding: oneOf: - enum: - bytes - enum: - json - enum: - ndjson - enum: - auto ui: display_type: list errorMessage: must be equal to one of the allowed values description: The decoding method for converting frames into data events. title: Decoding Method default: auto required: - decoding - if: properties: type: enum: - splunk-hec deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Splunk HEC description: Receive Splunk logs additionalProperties: false properties: capture_metadata: type: boolean title: Capture Metadata description: |- Enable the inclusion of all http headers and query string parameters that were sent from the source. ui: feature_flag: pipeline-capture-metadata display_type: hidden default: true auto_parse: type: object title: Configure automated parsing additionalProperties: false ui: display_type: hidden properties: enabled: type: boolean default: false enabled_event_types: type: array items: type: string enum: - HTTPD_COMBINEDLOG - HTTPD_COMMONLOG - HTTPD_ERRORLOG - SYSLOG5424LINE - SYSLOGLINE - SYSLOGPAMSESSION - CRONLOG - MONGO3_LOG - NAGIOSLOGLINE - POSTGRESQL - RAILS3 - RUBY_LOGGER - REDISLOG - REDISMONLOG - S3_ACCESS_LOG - ELB_ACCESS_LOG - CLOUDFRONT_ACCESS_LOG - CATALINALOG - TOMCATLOG - SQUID3 - BIND9 - HAPROXYTCP - HAPROXYHTTP - BACULA_LOGLINE - BRO_HTTP - BRO_DNS - BRO_CONN - BRO_FILES - NETSCREENSESSIONLOG - CISCO_TAGGED_SYSLOG - CISCOFW104001 - CISCOFW104002 - CISCOFW104003 - CISCOFW104004 - CISCOFW105003 - CISCOFW105004 - CISCOFW105005 - CISCOFW105008 - CISCOFW105009 - CISCOFW106001 - CISCOFW106006_106007_106010 - CISCOFW106014 - CISCOFW106015 - CISCOFW106021 - CISCOFW106023 - CISCOFW106100_2_3 - CISCOFW106100 - CISCOFW304001 - CISCOFW110002 - CISCOFW302010 - CISCOFW302013_302014_302015_302016 - CISCOFW302020_302021 - CISCOFW305011 - CISCOFW313001_313004_313008 - CISCOFW313005 - CISCOFW321001 - CISCOFW402117 - CISCOFW402119 - CISCOFW419001 - CISCOFW419002 - CISCOFW500004 - CISCOFW602303_602304 - CISCOFW710001_710002_710003_710005_710006 - CISCOFW713172 - CISCOFW733100 - SHOREWALL - SFW2 - if: properties: type: enum: - s3 deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: AWS S3 description: Collect data from object contents in AWS S3 additionalProperties: false properties: sqs_queue_url: type: string title: SQS Queue URL description: >- The URL of a AWS SQS queue configured to receive S3 bucket notifications for the desired S3 buckets format: uri errorMessage: format: Must be a valid URI auth: type: object title: AWS Authentication description: Configures AWS authentication additionalProperties: false properties: access_key_id: type: string title: Access Key ID minLength: 1 description: The AWS access key id secret_access_key: type: string title: Secret Access Key minLength: 1 description: The AWS secret access key ui: display_type: secret required: - access_key_id - secret_access_key compression: type: string title: Compression description: The compression format of the S3 objects enum: - auto - gzip - none - zstd default: auto region: type: string title: Region minLength: 1 description: The name of the source AWS region. required: - sqs_queue_url - auth - compression - region - if: properties: type: enum: - sqs deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: AWS SQS description: Collect messages from AWS SQS additionalProperties: false properties: queue_url: type: string title: Queue URL description: The URL of an AWS SQS queue format: uri errorMessage: format: Must be a valid URI maxLength: 128 auth: type: object title: AWS Authentication description: Configures AWS authentication additionalProperties: false properties: access_key_id: type: string title: Access Key ID minLength: 1 description: The AWS access key id maxLength: 128 secret_access_key: type: string title: Secret Access Key minLength: 1 description: The AWS secret access key ui: display_type: secret maxLength: 128 required: - access_key_id - secret_access_key region: type: string title: Region minLength: 1 description: The name of the source AWS region. maxLength: 128 required: - queue_url - auth - region - if: properties: type: enum: - demo-logs deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: title: Demo Logs description: Generate log messages for testing and demonstration. additionalProperties: type: array title: Log Lines description: >- A list of user-provided log lines that will be sent through the pipeline infinitely. A maximum of 25 lines is allowed. The lines can be either plain text or ndjson. If ndjson is detected, the encoding for the source will be set to JSON, and lines that are not valid will show up as a visible component error. Otherwise, text encoding will be used. minItems: 1 maxItems: 25 items: type: string minLength: 1 maxLength: 4096 title: Line description: A single line of log data, either stringified JSON or plain text. ui: display_type: user-provided-lines allOf: - if: properties: format: enum: - user_provided required: - format then: properties: lines: type: array title: Log Lines description: >- A list of user-provided log lines that will be sent through the pipeline infinitely. A maximum of 25 lines is allowed. The lines can be either plain text or ndjson. If ndjson is detected, the encoding for the source will be set to JSON, and lines that are not valid will show up as a visible component error. Otherwise, text encoding will be used. minItems: 1 maxItems: 25 items: type: string minLength: 1 maxLength: 4096 title: Line description: >- A single line of log data, either stringified JSON or plain text. ui: display_type: user-provided-lines required: - lines - allOf: - errorMessage: '`lines` is only valid when `format` is `user_provided`' if: not: type: object properties: format: enum: - user_provided required: - format then: not: type: object properties: lines: type: array title: Log Lines description: >- A list of user-provided log lines that will be sent through the pipeline infinitely. A maximum of 25 lines is allowed. The lines can be either plain text or ndjson. If ndjson is detected, the encoding for the source will be set to JSON, and lines that are not valid will show up as a visible component error. Otherwise, text encoding will be used. minItems: 1 maxItems: 25 items: type: string minLength: 1 maxLength: 4096 title: Line description: >- A single line of log data, either stringified JSON or plain text. ui: display_type: user-provided-lines required: - lines properties: interval_ms: type: number title: Interval description: >- The amount of time to pause between each demo log. Fractions of a second are accepted to increase throughput. e.g. 500 ms == 120 lines per minute default: 1000 minimum: 500 maximum: 3600000 ui: display_type: milliseconds-to-human-readable format: title: Format description: The format of the log messages. All of the data is generated. ui: display_type: list errorMessage: must be one of the available types of demo data oneOf: - label: HTTP JSON description: HTTP request data in JSON format. enum: - json - label: Sensor JSON description: Data from environment sensors in JSON format. enum: - env_sensor - label: Financial JSON description: Financial data in JSON format. enum: - financial - label: Nginx Requests description: Nginx logs in text format. enum: - nginx - label: Apache Requests description: Apache request logs in text format. enum: - apache_common - label: Apache Errors description: Apache error logs in text format. enum: - apache_error - label: BSD Syslog description: Mock BSD syslog entries in rfc3164 text format. enum: - bsd_syslog - label: Syslog description: Mock syslog entries in rfc5424 text format. enum: - syslog - label: HTTP Metrics description: HTTP-based metrics in the Mezmo metrics JSON format. enum: - http_metrics - label: Generic Metrics description: Generic metrics in the Mezmo metrics JSON format enum: - generic_metrics - label: User-Provided Logs description: >- Custom logs provided by the user. These can either be plain text or serialized JSON. enum: - user_provided - label: Infrastructure Logs description: >- Infrastructure logs consisting of sysdig, nginx, HDFS and spark logs enum: - infrastructure - label: Kubernetes Logs description: Kubernetes logs in JSON format enum: - kubernetes default: json required: - format - if: properties: type: enum: - kinesis-firehose deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Kinesis Firehose description: Receive Kinesis Firehose data additionalProperties: false properties: capture_metadata: type: boolean title: Capture Metadata description: |- Enable the inclusion of all http headers and query string parameters that were sent from the source. ui: feature_flag: pipeline-capture-metadata display_type: hidden default: true auto_parse: type: object title: Configure automated parsing additionalProperties: false ui: display_type: hidden properties: enabled: type: boolean default: false enabled_event_types: type: array items: type: string enum: - HTTPD_COMBINEDLOG - HTTPD_COMMONLOG - HTTPD_ERRORLOG - SYSLOG5424LINE - SYSLOGLINE - SYSLOGPAMSESSION - CRONLOG - MONGO3_LOG - NAGIOSLOGLINE - POSTGRESQL - RAILS3 - RUBY_LOGGER - REDISLOG - REDISMONLOG - S3_ACCESS_LOG - ELB_ACCESS_LOG - CLOUDFRONT_ACCESS_LOG - CATALINALOG - TOMCATLOG - SQUID3 - BIND9 - HAPROXYTCP - HAPROXYHTTP - BACULA_LOGLINE - BRO_HTTP - BRO_DNS - BRO_CONN - BRO_FILES - NETSCREENSESSIONLOG - CISCO_TAGGED_SYSLOG - CISCOFW104001 - CISCOFW104002 - CISCOFW104003 - CISCOFW104004 - CISCOFW105003 - CISCOFW105004 - CISCOFW105005 - CISCOFW105008 - CISCOFW105009 - CISCOFW106001 - CISCOFW106006_106007_106010 - CISCOFW106014 - CISCOFW106015 - CISCOFW106021 - CISCOFW106023 - CISCOFW106100_2_3 - CISCOFW106100 - CISCOFW304001 - CISCOFW110002 - CISCOFW302010 - CISCOFW302013_302014_302015_302016 - CISCOFW302020_302021 - CISCOFW305011 - CISCOFW313001_313004_313008 - CISCOFW313005 - CISCOFW321001 - CISCOFW402117 - CISCOFW402119 - CISCOFW419001 - CISCOFW419002 - CISCOFW500004 - CISCOFW602303_602304 - CISCOFW710001_710002_710003_710005_710006 - CISCOFW713172 - CISCOFW733100 - SHOREWALL - SFW2 format: type: string enum: - json - text default: json title: Data Format description: >- This specifies what the data format will be after it is base64 decoded. If it is JSON, it will be automatically parsed. required: - format - if: properties: type: enum: - fluent deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Fluent (Fluentd, Fluent Bit) description: Receive data from Fluentd or Fluent Bit additionalProperties: false properties: capture_metadata: type: boolean title: Capture Metadata description: |- Enable the inclusion of all http headers and query string parameters that were sent from the source. ui: feature_flag: pipeline-capture-metadata display_type: hidden default: true auto_parse: type: object title: Configure automated parsing additionalProperties: false ui: display_type: hidden properties: enabled: type: boolean default: false enabled_event_types: type: array items: type: string enum: - HTTPD_COMBINEDLOG - HTTPD_COMMONLOG - HTTPD_ERRORLOG - SYSLOG5424LINE - SYSLOGLINE - SYSLOGPAMSESSION - CRONLOG - MONGO3_LOG - NAGIOSLOGLINE - POSTGRESQL - RAILS3 - RUBY_LOGGER - REDISLOG - REDISMONLOG - S3_ACCESS_LOG - ELB_ACCESS_LOG - CLOUDFRONT_ACCESS_LOG - CATALINALOG - TOMCATLOG - SQUID3 - BIND9 - HAPROXYTCP - HAPROXYHTTP - BACULA_LOGLINE - BRO_HTTP - BRO_DNS - BRO_CONN - BRO_FILES - NETSCREENSESSIONLOG - CISCO_TAGGED_SYSLOG - CISCOFW104001 - CISCOFW104002 - CISCOFW104003 - CISCOFW104004 - CISCOFW105003 - CISCOFW105004 - CISCOFW105005 - CISCOFW105008 - CISCOFW105009 - CISCOFW106001 - CISCOFW106006_106007_106010 - CISCOFW106014 - CISCOFW106015 - CISCOFW106021 - CISCOFW106023 - CISCOFW106100_2_3 - CISCOFW106100 - CISCOFW304001 - CISCOFW110002 - CISCOFW302010 - CISCOFW302013_302014_302015_302016 - CISCOFW302020_302021 - CISCOFW305011 - CISCOFW313001_313004_313008 - CISCOFW313005 - CISCOFW321001 - CISCOFW402117 - CISCOFW402119 - CISCOFW419001 - CISCOFW419002 - CISCOFW500004 - CISCOFW602303_602304 - CISCOFW710001_710002_710003_710005_710006 - CISCOFW713172 - CISCOFW733100 - SHOREWALL - SFW2 decoding: type: string description: The decoding method for converting frames into data events. enum: - bytes - json - ndjson title: Decoding Method default: json required: - decoding - if: properties: type: enum: - log-analysis deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Mezmo Log Analysis description: Receive data directly from your Mezmo Log Analysis account additionalProperties: false - if: properties: type: enum: - log-analysis-ingestion deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Mezmo Log Analysis Ingestion description: >- Receive unparsed source data directly from your Mezmo Log Analysis account. additionalProperties: type: string - if: properties: type: enum: - logstash deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Logstash description: Receive Logstash data additionalProperties: false properties: capture_metadata: type: boolean title: Capture Metadata description: |- Enable the inclusion of all http headers and query string parameters that were sent from the source. ui: feature_flag: pipeline-capture-metadata display_type: hidden default: true auto_parse: type: object title: Configure automated parsing additionalProperties: false ui: display_type: hidden properties: enabled: type: boolean default: false enabled_event_types: type: array items: type: string enum: - HTTPD_COMBINEDLOG - HTTPD_COMMONLOG - HTTPD_ERRORLOG - SYSLOG5424LINE - SYSLOGLINE - SYSLOGPAMSESSION - CRONLOG - MONGO3_LOG - NAGIOSLOGLINE - POSTGRESQL - RAILS3 - RUBY_LOGGER - REDISLOG - REDISMONLOG - S3_ACCESS_LOG - ELB_ACCESS_LOG - CLOUDFRONT_ACCESS_LOG - CATALINALOG - TOMCATLOG - SQUID3 - BIND9 - HAPROXYTCP - HAPROXYHTTP - BACULA_LOGLINE - BRO_HTTP - BRO_DNS - BRO_CONN - BRO_FILES - NETSCREENSESSIONLOG - CISCO_TAGGED_SYSLOG - CISCOFW104001 - CISCOFW104002 - CISCOFW104003 - CISCOFW104004 - CISCOFW105003 - CISCOFW105004 - CISCOFW105005 - CISCOFW105008 - CISCOFW105009 - CISCOFW106001 - CISCOFW106006_106007_106010 - CISCOFW106014 - CISCOFW106015 - CISCOFW106021 - CISCOFW106023 - CISCOFW106100_2_3 - CISCOFW106100 - CISCOFW304001 - CISCOFW110002 - CISCOFW302010 - CISCOFW302013_302014_302015_302016 - CISCOFW302020_302021 - CISCOFW305011 - CISCOFW313001_313004_313008 - CISCOFW313005 - CISCOFW321001 - CISCOFW402117 - CISCOFW402119 - CISCOFW419001 - CISCOFW419002 - CISCOFW500004 - CISCOFW602303_602304 - CISCOFW710001_710002_710003_710005_710006 - CISCOFW713172 - CISCOFW733100 - SHOREWALL - SFW2 format: type: string enum: - json - text default: json title: Data Format description: The format of the logstash data required: - format - if: properties: type: enum: - mezmo-agent deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Mezmo Agent description: Receive logs from the Mezmo Agent additionalProperties: false properties: capture_metadata: type: boolean title: Capture Metadata description: |- Enable the inclusion of all http headers and query string parameters that were sent from the source. ui: feature_flag: pipeline-capture-metadata display_type: hidden default: true auto_parse: type: object title: Configure automated parsing additionalProperties: false ui: display_type: hidden properties: enabled: type: boolean default: false enabled_event_types: type: array items: type: string enum: - HTTPD_COMBINEDLOG - HTTPD_COMMONLOG - HTTPD_ERRORLOG - SYSLOG5424LINE - SYSLOGLINE - SYSLOGPAMSESSION - CRONLOG - MONGO3_LOG - NAGIOSLOGLINE - POSTGRESQL - RAILS3 - RUBY_LOGGER - REDISLOG - REDISMONLOG - S3_ACCESS_LOG - ELB_ACCESS_LOG - CLOUDFRONT_ACCESS_LOG - CATALINALOG - TOMCATLOG - SQUID3 - BIND9 - HAPROXYTCP - HAPROXYHTTP - BACULA_LOGLINE - BRO_HTTP - BRO_DNS - BRO_CONN - BRO_FILES - NETSCREENSESSIONLOG - CISCO_TAGGED_SYSLOG - CISCOFW104001 - CISCOFW104002 - CISCOFW104003 - CISCOFW104004 - CISCOFW105003 - CISCOFW105004 - CISCOFW105005 - CISCOFW105008 - CISCOFW105009 - CISCOFW106001 - CISCOFW106006_106007_106010 - CISCOFW106014 - CISCOFW106015 - CISCOFW106021 - CISCOFW106023 - CISCOFW106100_2_3 - CISCOFW106100 - CISCOFW304001 - CISCOFW110002 - CISCOFW302010 - CISCOFW302013_302014_302015_302016 - CISCOFW302020_302021 - CISCOFW305011 - CISCOFW313001_313004_313008 - CISCOFW313005 - CISCOFW321001 - CISCOFW402117 - CISCOFW402119 - CISCOFW419001 - CISCOFW419002 - CISCOFW500004 - CISCOFW602303_602304 - CISCOFW710001_710002_710003_710005_710006 - CISCOFW713172 - CISCOFW733100 - SHOREWALL - SFW2 - if: properties: type: enum: - mezmo-datadog-source deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Datadog Agent description: Send logs and metrics data directly from an installed datadog agent additionalProperties: false properties: capture_metadata: type: boolean title: Capture Metadata description: |- Enable the inclusion of all http headers and query string parameters that were sent from the source. ui: feature_flag: pipeline-capture-metadata display_type: hidden default: true auto_parse: type: object title: Configure automated parsing additionalProperties: false ui: display_type: hidden properties: enabled: type: boolean default: false enabled_event_types: type: array items: type: string enum: - HTTPD_COMBINEDLOG - HTTPD_COMMONLOG - HTTPD_ERRORLOG - SYSLOG5424LINE - SYSLOGLINE - SYSLOGPAMSESSION - CRONLOG - MONGO3_LOG - NAGIOSLOGLINE - POSTGRESQL - RAILS3 - RUBY_LOGGER - REDISLOG - REDISMONLOG - S3_ACCESS_LOG - ELB_ACCESS_LOG - CLOUDFRONT_ACCESS_LOG - CATALINALOG - TOMCATLOG - SQUID3 - BIND9 - HAPROXYTCP - HAPROXYHTTP - BACULA_LOGLINE - BRO_HTTP - BRO_DNS - BRO_CONN - BRO_FILES - NETSCREENSESSIONLOG - CISCO_TAGGED_SYSLOG - CISCOFW104001 - CISCOFW104002 - CISCOFW104003 - CISCOFW104004 - CISCOFW105003 - CISCOFW105004 - CISCOFW105005 - CISCOFW105008 - CISCOFW105009 - CISCOFW106001 - CISCOFW106006_106007_106010 - CISCOFW106014 - CISCOFW106015 - CISCOFW106021 - CISCOFW106023 - CISCOFW106100_2_3 - CISCOFW106100 - CISCOFW304001 - CISCOFW110002 - CISCOFW302010 - CISCOFW302013_302014_302015_302016 - CISCOFW302020_302021 - CISCOFW305011 - CISCOFW313001_313004_313008 - CISCOFW313005 - CISCOFW321001 - CISCOFW402117 - CISCOFW402119 - CISCOFW419001 - CISCOFW419002 - CISCOFW500004 - CISCOFW602303_602304 - CISCOFW710001_710002_710003_710005_710006 - CISCOFW713172 - CISCOFW733100 - SHOREWALL - SFW2 - if: properties: type: enum: - pipeline-data-restoration deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Mezmo Pipeline Data Restoration description: Receive data directly from your Mezmo Pipline Data Restoration additionalProperties: false - if: properties: type: enum: - webhook deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Webhook (WebSub) description: Receive data from incoming webhooks using the WebSub protocol. additionalProperties: false properties: capture_metadata: type: boolean title: Capture Metadata description: |- Enable the inclusion of all http headers and query string parameters that were sent from the source. ui: feature_flag: pipeline-capture-metadata display_type: hidden default: true auto_parse: type: object title: Configure automated parsing additionalProperties: false ui: display_type: hidden properties: enabled: type: boolean default: false enabled_event_types: type: array items: type: string enum: - HTTPD_COMBINEDLOG - HTTPD_COMMONLOG - HTTPD_ERRORLOG - SYSLOG5424LINE - SYSLOGLINE - SYSLOGPAMSESSION - CRONLOG - MONGO3_LOG - NAGIOSLOGLINE - POSTGRESQL - RAILS3 - RUBY_LOGGER - REDISLOG - REDISMONLOG - S3_ACCESS_LOG - ELB_ACCESS_LOG - CLOUDFRONT_ACCESS_LOG - CATALINALOG - TOMCATLOG - SQUID3 - BIND9 - HAPROXYTCP - HAPROXYHTTP - BACULA_LOGLINE - BRO_HTTP - BRO_DNS - BRO_CONN - BRO_FILES - NETSCREENSESSIONLOG - CISCO_TAGGED_SYSLOG - CISCOFW104001 - CISCOFW104002 - CISCOFW104003 - CISCOFW104004 - CISCOFW105003 - CISCOFW105004 - CISCOFW105005 - CISCOFW105008 - CISCOFW105009 - CISCOFW106001 - CISCOFW106006_106007_106010 - CISCOFW106014 - CISCOFW106015 - CISCOFW106021 - CISCOFW106023 - CISCOFW106100_2_3 - CISCOFW106100 - CISCOFW304001 - CISCOFW110002 - CISCOFW302010 - CISCOFW302013_302014_302015_302016 - CISCOFW302020_302021 - CISCOFW305011 - CISCOFW313001_313004_313008 - CISCOFW313005 - CISCOFW321001 - CISCOFW402117 - CISCOFW402119 - CISCOFW419001 - CISCOFW419002 - CISCOFW500004 - CISCOFW602303_602304 - CISCOFW710001_710002_710003_710005_710006 - CISCOFW713172 - CISCOFW733100 - SHOREWALL - SFW2 - if: properties: type: enum: - kafka deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Kafka Source description: Collect events from Kafka compatible brokers additionalProperties: type: string properties: brokers: type: array title: Brokers uniqueItems: true minItems: 1 description: An array of key/val objects for each host/port of a Kafka broker. items: type: object title: Host/Port description: The host and port for the Kafka broker. additionalProperties: false properties: host: title: Host description: The hostname or IP address of the broker. anyOf: - type: string format: ipv4 errorMessage: format: Must be a valid IPv4 address - type: string format: hostname maxLength: 512 errorMessage: format: Must be a valid hostname port: type: number minimum: 1 maximum: 65535 title: Port description: The port that the broker listens on. required: - host - port topics: type: array title: Topics items: type: string title: Topic minLength: 1 maxLength: 256 description: The Kafka topic name. minItems: 1 description: List of Kafka topics names to read events from. group_id: type: string title: Consumer Group minLength: 1 description: The consumer group name to be used to consume events from Kafka. tls_enabled: type: boolean title: TLS enabled description: Whether or not to require TLS for outgoing connections. default: true sasl_enabled: type: boolean title: SASL/SCRAM Enabled description: Enable SASL/SCRAM authentication with Kafka default: true decoding_codec: type: string enum: - bytes - json title: Decoding description: Configures how events are decoded from raw bytes default: bytes if: properties: sasl_enabled: enum: - true then: properties: sasl_mechanism: type: string enum: - SCRAM-SHA-512 - SCRAM-SHA-256 - PLAIN title: SASL description: The Kafka SASL/SCRAM mechanism default: PLAIN sasl_username: type: string title: Username description: The SASL/SCRAM username minLength: 1 sasl_password: type: string title: Password description: The SASL/SCRAM username ui: display_type: secret minLength: 1 required: - sasl_mechanism - sasl_username - sasl_password required: - brokers - topics - group_id - decoding_codec - if: properties: type: enum: - azure-event-hub deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Azure Event Hub Source description: Collect events from Azure Event Hub using Azure Kafka Endpoint additionalProperties: false properties: connection_string: type: string title: Connection String minLength: 1 maxLength: 512 description: The Connection String as it appears in hub consumer SAS Policy ui: display_type: secret namespace: type: string title: Namespace minLength: 1 maxLength: 256 description: The Event Hub Namespace topics: type: array title: Event Hub Names items: type: string title: Name minLength: 1 maxLength: 256 description: Azure Event Hub instance name. minItems: 1 description: The list of Azure Event Hub name(s) to read events from. group_id: type: string title: Consumer Group minLength: 1 maxLength: 256 description: The consumer group name that this consumer belongs to. decoding_codec: type: string enum: - bytes - json title: Decoding description: Configures how events are decoded from raw bytes default: bytes required: - connection_string - namespace - topics - group_id - decoding_codec - if: properties: type: enum: - prometheus-remote-write deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Prometheus Remote Write description: Receive prometheus remote write protobuf data from HTTP requests additionalProperties: false properties: capture_metadata: type: boolean title: Capture Metadata description: |- Enable the inclusion of all http headers and query string parameters that were sent from the source. ui: feature_flag: pipeline-capture-metadata display_type: hidden default: true auto_parse: type: object title: Configure automated parsing additionalProperties: false ui: display_type: hidden properties: enabled: type: boolean default: false enabled_event_types: type: array items: type: string enum: - HTTPD_COMBINEDLOG - HTTPD_COMMONLOG - HTTPD_ERRORLOG - SYSLOG5424LINE - SYSLOGLINE - SYSLOGPAMSESSION - CRONLOG - MONGO3_LOG - NAGIOSLOGLINE - POSTGRESQL - RAILS3 - RUBY_LOGGER - REDISLOG - REDISMONLOG - S3_ACCESS_LOG - ELB_ACCESS_LOG - CLOUDFRONT_ACCESS_LOG - CATALINALOG - TOMCATLOG - SQUID3 - BIND9 - HAPROXYTCP - HAPROXYHTTP - BACULA_LOGLINE - BRO_HTTP - BRO_DNS - BRO_CONN - BRO_FILES - NETSCREENSESSIONLOG - CISCO_TAGGED_SYSLOG - CISCOFW104001 - CISCOFW104002 - CISCOFW104003 - CISCOFW104004 - CISCOFW105003 - CISCOFW105004 - CISCOFW105005 - CISCOFW105008 - CISCOFW105009 - CISCOFW106001 - CISCOFW106006_106007_106010 - CISCOFW106014 - CISCOFW106015 - CISCOFW106021 - CISCOFW106023 - CISCOFW106100_2_3 - CISCOFW106100 - CISCOFW304001 - CISCOFW110002 - CISCOFW302010 - CISCOFW302013_302014_302015_302016 - CISCOFW302020_302021 - CISCOFW305011 - CISCOFW313001_313004_313008 - CISCOFW313005 - CISCOFW321001 - CISCOFW402117 - CISCOFW402119 - CISCOFW419001 - CISCOFW419002 - CISCOFW500004 - CISCOFW602303_602304 - CISCOFW710001_710002_710003_710005_710006 - CISCOFW713172 - CISCOFW733100 - SHOREWALL - SFW2 - if: properties: type: enum: - open-telemetry-metrics deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: OpenTelemetry Metrics description: Receive metrics from OpenTelemetry over HTTP additionalProperties: false properties: capture_metadata: type: boolean title: Capture Metadata description: |- Enable the inclusion of all http headers and query string parameters that were sent from the source. ui: feature_flag: pipeline-capture-metadata display_type: hidden default: true auto_parse: type: object title: Configure automated parsing additionalProperties: false ui: display_type: hidden properties: enabled: type: boolean default: false enabled_event_types: type: array items: type: string enum: - HTTPD_COMBINEDLOG - HTTPD_COMMONLOG - HTTPD_ERRORLOG - SYSLOG5424LINE - SYSLOGLINE - SYSLOGPAMSESSION - CRONLOG - MONGO3_LOG - NAGIOSLOGLINE - POSTGRESQL - RAILS3 - RUBY_LOGGER - REDISLOG - REDISMONLOG - S3_ACCESS_LOG - ELB_ACCESS_LOG - CLOUDFRONT_ACCESS_LOG - CATALINALOG - TOMCATLOG - SQUID3 - BIND9 - HAPROXYTCP - HAPROXYHTTP - BACULA_LOGLINE - BRO_HTTP - BRO_DNS - BRO_CONN - BRO_FILES - NETSCREENSESSIONLOG - CISCO_TAGGED_SYSLOG - CISCOFW104001 - CISCOFW104002 - CISCOFW104003 - CISCOFW104004 - CISCOFW105003 - CISCOFW105004 - CISCOFW105005 - CISCOFW105008 - CISCOFW105009 - CISCOFW106001 - CISCOFW106006_106007_106010 - CISCOFW106014 - CISCOFW106015 - CISCOFW106021 - CISCOFW106023 - CISCOFW106100_2_3 - CISCOFW106100 - CISCOFW304001 - CISCOFW110002 - CISCOFW302010 - CISCOFW302013_302014_302015_302016 - CISCOFW302020_302021 - CISCOFW305011 - CISCOFW313001_313004_313008 - CISCOFW313005 - CISCOFW321001 - CISCOFW402117 - CISCOFW402119 - CISCOFW419001 - CISCOFW419002 - CISCOFW500004 - CISCOFW602303_602304 - CISCOFW710001_710002_710003_710005_710006 - CISCOFW713172 - CISCOFW733100 - SHOREWALL - SFW2 - if: properties: type: enum: - open-telemetry-logs deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: OpenTelemetry Logs description: Receive logs from OpenTelemetry over HTTP additionalProperties: false properties: capture_metadata: type: boolean title: Capture Metadata description: |- Enable the inclusion of all http headers and query string parameters that were sent from the source. ui: feature_flag: pipeline-capture-metadata display_type: hidden default: true auto_parse: type: object title: Configure automated parsing additionalProperties: false ui: display_type: hidden properties: enabled: type: boolean default: false enabled_event_types: type: array items: type: string enum: - HTTPD_COMBINEDLOG - HTTPD_COMMONLOG - HTTPD_ERRORLOG - SYSLOG5424LINE - SYSLOGLINE - SYSLOGPAMSESSION - CRONLOG - MONGO3_LOG - NAGIOSLOGLINE - POSTGRESQL - RAILS3 - RUBY_LOGGER - REDISLOG - REDISMONLOG - S3_ACCESS_LOG - ELB_ACCESS_LOG - CLOUDFRONT_ACCESS_LOG - CATALINALOG - TOMCATLOG - SQUID3 - BIND9 - HAPROXYTCP - HAPROXYHTTP - BACULA_LOGLINE - BRO_HTTP - BRO_DNS - BRO_CONN - BRO_FILES - NETSCREENSESSIONLOG - CISCO_TAGGED_SYSLOG - CISCOFW104001 - CISCOFW104002 - CISCOFW104003 - CISCOFW104004 - CISCOFW105003 - CISCOFW105004 - CISCOFW105005 - CISCOFW105008 - CISCOFW105009 - CISCOFW106001 - CISCOFW106006_106007_106010 - CISCOFW106014 - CISCOFW106015 - CISCOFW106021 - CISCOFW106023 - CISCOFW106100_2_3 - CISCOFW106100 - CISCOFW304001 - CISCOFW110002 - CISCOFW302010 - CISCOFW302013_302014_302015_302016 - CISCOFW302020_302021 - CISCOFW305011 - CISCOFW313001_313004_313008 - CISCOFW313005 - CISCOFW321001 - CISCOFW402117 - CISCOFW402119 - CISCOFW419001 - CISCOFW419002 - CISCOFW500004 - CISCOFW602303_602304 - CISCOFW710001_710002_710003_710005_710006 - CISCOFW713172 - CISCOFW733100 - SHOREWALL - SFW2 - if: properties: type: enum: - open-telemetry-traces deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: OpenTelemetry Traces description: Receive traces from OpenTelemetry over HTTP additionalProperties: false properties: capture_metadata: type: boolean title: Capture Metadata description: |- Enable the inclusion of all http headers and query string parameters that were sent from the source. ui: feature_flag: pipeline-capture-metadata display_type: hidden default: true auto_parse: type: object title: Configure automated parsing additionalProperties: false ui: display_type: hidden properties: enabled: type: boolean default: false enabled_event_types: type: array items: type: string enum: - HTTPD_COMBINEDLOG - HTTPD_COMMONLOG - HTTPD_ERRORLOG - SYSLOG5424LINE - SYSLOGLINE - SYSLOGPAMSESSION - CRONLOG - MONGO3_LOG - NAGIOSLOGLINE - POSTGRESQL - RAILS3 - RUBY_LOGGER - REDISLOG - REDISMONLOG - S3_ACCESS_LOG - ELB_ACCESS_LOG - CLOUDFRONT_ACCESS_LOG - CATALINALOG - TOMCATLOG - SQUID3 - BIND9 - HAPROXYTCP - HAPROXYHTTP - BACULA_LOGLINE - BRO_HTTP - BRO_DNS - BRO_CONN - BRO_FILES - NETSCREENSESSIONLOG - CISCO_TAGGED_SYSLOG - CISCOFW104001 - CISCOFW104002 - CISCOFW104003 - CISCOFW104004 - CISCOFW105003 - CISCOFW105004 - CISCOFW105005 - CISCOFW105008 - CISCOFW105009 - CISCOFW106001 - CISCOFW106006_106007_106010 - CISCOFW106014 - CISCOFW106015 - CISCOFW106021 - CISCOFW106023 - CISCOFW106100_2_3 - CISCOFW106100 - CISCOFW304001 - CISCOFW110002 - CISCOFW302010 - CISCOFW302013_302014_302015_302016 - CISCOFW302020_302021 - CISCOFW305011 - CISCOFW313001_313004_313008 - CISCOFW313005 - CISCOFW321001 - CISCOFW402117 - CISCOFW402119 - CISCOFW419001 - CISCOFW419002 - CISCOFW500004 - CISCOFW602303_602304 - CISCOFW710001_710002_710003_710005_710006 - CISCOFW713172 - CISCOFW733100 - SHOREWALL - SFW2 properties: type: type: string enum: - http - splunk-hec - s3 - sqs - demo-logs - kinesis-firehose - fluent - log-analysis - log-analysis-ingestion - logstash - mezmo-agent - mezmo-datadog-source - pipeline-data-restoration - webhook - kafka - azure-event-hub - prometheus-remote-write - open-telemetry-metrics - open-telemetry-logs - open-telemetry-traces deploy_type: type: string enum: - saas - local default: saas title: oneOf: - type: string minLength: 1 maxLength: 512 pattern: ^[^<>"]+$ - type: 'null' errorMessage: invalid title name description: oneOf: - type: string minLength: 1 pattern: ^[^<>"]+$ - type: 'null' errorMessage: invalid description user_config: type: object gateway_route_id: type: string format: uuid required: - type - user_config required: true parameters: - schema: type: string format: uuid in: path name: pipeline_id required: true responses: '200': description: Default Response /v3/pipeline/{pipeline_id}/transform/{pipeline_transform_id}: delete: summary: Remove transform tags: - Pipeline Nodes description: Removes transform from pipeline parameters: - schema: type: string format: uuid in: path name: pipeline_id required: true - schema: type: string format: uuid in: path name: pipeline_transform_id required: true responses: '200': description: Default Response get: summary: Fetch transform tags: - Pipeline Nodes description: Fetch transform configuration parameters: - schema: type: string format: uuid in: path name: pipeline_id required: true - schema: type: string format: uuid in: path name: pipeline_transform_id required: true responses: '200': description: Default Response put: summary: Update transform tags: - Pipeline Nodes description: Updates a transform of a pipeline requestBody: content: application/json: schema: type: object allOf: - if: properties: type: enum: - filter deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Filter description: Define condition(s) to include or exclude events from the pipeline additionalProperties: false properties: action: title: Action description: How to handle events matching this criteria oneOf: - label: Allow events matching this criteria enum: - allow - label: Drop events matching this criteria enum: - drop default: allow ui: display_type: radio-group conditional: allOf: - $ref: '#condition-group' title: Conditional description: A group of expressions joined together by an and/or operator. ui: display_type: rules-engine required: - action - conditional - if: properties: type: enum: - sample deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Sample description: >- Sample data at a given rate, retaining only a subset of data events for further processing. additionalProperties: false properties: rate: type: integer title: Rate description: >- The rate at which events will be forwarded, expressed as 1/N. For example, `rate = 10` means 1 out of every 10 events will be forwarded and the rest will be dropped minimum: 2 maximum: 10000 default: 10 always_include: title: Always Include description: >- Certain events may always want to be included (not sampled), regardless of the sample rate. Events matching this criteria will always show up in the results. additionalProperties: type: - string - number dependencies: field: - str_operator str_operator: - field allOf: - if: title: Operator properties: str_operator: type: string enum: - greater - greater_or_equal - less - less_or_equal ui: display_type: list-operators title: Operator description: The comparison operator required: - str_operator then: title: Value properties: value: errorMessage: pattern: Value must be a number type: - string - number title: Value description: The value used in the expression, ie the "operand" minLength: 1 pattern: ^\s*-?(\d*\.)?\d+\s*$ required: - value - if: title: Operator properties: str_operator: type: string enum: - greater - greater_or_equal - less - less_or_equal - equal - contains - is_ip_in_cidr_range - ends_with - starts_with - drain_matches - contains_token_term - contains_token_prefix - percent_change_greater - percent_change_greater_or_equal - percent_change_less - percent_change_less_or_equal - value_change_greater - value_change_greater_or_equal - value_change_less - value_change_less_or_equal ui: display_type: list-operators title: Operator description: The comparison operator required: - str_operator then: title: Value properties: value: type: - string - number title: Value description: The value used in the expression, ie the "operand" minLength: 1 required: - value - if: title: Operator properties: str_operator: type: string enum: - contains - equal - ends_with - starts_with - drain_matches - contains_token_term - contains_token_prefix ui: display_type: list-operators title: Operator required: - str_operator then: title: Case sensitive properties: case_sensitive: type: boolean title: Case sensitive description: Perform case sensitive comparison? default: true properties: field: type: string title: Field description: The JSON field to use in a condition to always include in sampling pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference str_operator: type: string enum: - greater - greater_or_equal - less - less_or_equal - equal - contains - is_ip_in_cidr_range - is_metric - is_array - is_boolean - is_empty - is_null - is_number - is_object - is_string - exists - ends_with - starts_with - contains_token_term - contains_token_prefix - regex_match ui: display_type: list-operators title: Operator description: The comparison operator negate: type: boolean default: false title: Negate description: Negate the comparison expression required: - rate - if: properties: type: enum: - drop-fields deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Remove Fields description: Remove fields additionalProperties: false properties: fields: type: array title: Fields description: A list of fields to be removed items: type: string title: Field pattern: >- ^(?:metadata|\.|(?:metadata|message)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ minLength: 1 errorMessage: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields). Note that dropping an event's message or timestamp is not allowed minItems: 1 required: - fields - if: properties: type: enum: - parse deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: title: Parse description: Parse a specified field using the chosen parser additionalProperties: type: object errorMessage: properties: target_field: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) allOf: - if: properties: parser: enum: - parse_key_value then: properties: options: $ref: '#key-value-parser-options' - if: properties: parser: enum: - parse_timestamp then: properties: options: $ref: '#timestamp-parser-options' required: - options - if: properties: parser: enum: - parse_grok then: properties: options: $ref: '#grok-pattern-parser-options' required: - options - if: properties: parser: enum: - parse_regex then: properties: options: allOf: - $ref: '#regex-parser-options' ui: display_type: regex-pattern required: - options - if: properties: parser: enum: - parse_apache_log then: properties: options: $ref: '#apache-parser-options' required: - options - if: properties: parser: enum: - parse_nginx_log then: properties: options: $ref: '#nginx-parser-options' required: - options - if: properties: parser: enum: - parse_cef then: properties: options: $ref: '#cef-parser-options' - if: properties: parser: enum: - parse_csv then: properties: options: $ref: '#csv-parser-options' properties: field: type: string title: Field description: The JSON field whose value should be parsed. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ default: . errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference target_field: oneOf: - type: 'null' - type: string pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ maxLength: 512 errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference title: Target field description: >- The field into which the parsed value should be inserted. Leave blank to insert the parsed data into the original field. default: null parser: oneOf: - label: AWS Cloudfront Access Log group: Line Parsers ui: {} enum: - aws_cloudfront_access_log - label: AWS Cloudwatch Log group: Line Parsers ui: {} enum: - parse_aws_cloudwatch_log_subscription_message - label: AWS Load Balancer Log group: Line Parsers ui: {} enum: - parse_aws_alb_log - label: AWS S3 Access Log group: Line Parsers ui: {} enum: - aws_s3_access_log - label: Apache Log group: Line Parsers ui: {} enum: - parse_apache_log - label: CSV group: Line Parsers ui: {} enum: - parse_csv - label: Common Event Format (CEF) Log group: Line Parsers ui: {} enum: - parse_cef - label: Common Log group: Line Parsers ui: {} enum: - parse_common_log - label: Elasticsearch Log group: Line Parsers ui: {} enum: - elasticsearch_log - label: Golang Log group: Line Parsers ui: {} enum: - golang_log - label: Google Log group: Line Parsers ui: {} enum: - parse_glog - label: Grok Pattern group: Line Parsers ui: {} enum: - parse_grok - label: Integer group: Field Parsers ui: {} enum: - parse_int - label: JSON group: Line Parsers ui: {} enum: - parse_json - label: Key/Value group: Line Parsers ui: {} enum: - parse_key_value - label: Kubernetes Log group: Line Parsers ui: {} enum: - parse_klog - label: Linux Authorization group: Line Parsers ui: {} enum: - parse_linux_authorization - label: Mongo Log group: Line Parsers ui: {} enum: - mongo_log - label: Mongo3 Log group: Line Parsers ui: {} enum: - mongo3_log - label: Nginx Log group: Line Parsers ui: {} enum: - parse_nginx_log - label: PSQL Log group: Line Parsers ui: {} enum: - psql_log - label: Query String group: Field Parsers ui: {} enum: - parse_query_string - label: Regular Expression group: Line Parsers ui: feature_flag: pipeline-parser-regex enum: - parse_regex - label: Syslog group: Line Parsers ui: {} enum: - parse_syslog - label: Timestamp group: Field Parsers ui: {} enum: - parse_timestamp - label: Tokens group: Field Parsers ui: {} enum: - parse_tokens - label: URL group: Field Parsers ui: {} enum: - parse_url - label: User Agent group: Field Parsers ui: {} enum: - parse_user_agent - label: VPC Flow Log group: Line Parsers ui: {} description: Parse VPC logs using default format for fields from v2 to v4 enum: - parse_aws_vpc_flow_log ui: display_type: list errorMessage: must be one of allowed parsers title: Parser description: The kind of parser to use against the input value from "field". required: - field - parser - if: properties: type: enum: - parse-sequentially deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Parse Sequentially description: >- Parse fields using one or more parsers in order, sending out the event on the first successful match. additionalProperties: false errorMessage: properties: target_field: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) properties: field: type: string title: Field description: The field whose value should be parsed. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ default: . maxLength: 512 errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference target_field: oneOf: - type: 'null' - type: string pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ maxLength: 512 errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference title: Target field description: >- The field into which the parsed value should be inserted. Leave blank to insert the parsed data into the original field. default: null parsers: type: array title: List of parsers description: >- The list of parsers to use in order against the input valuefrom "field", short-circuiting on the first successful match. items: type: object allOf: - if: properties: parser: enum: - parse_key_value then: properties: options: $ref: '#key-value-parser-options' - if: properties: parser: enum: - parse_timestamp then: properties: options: $ref: '#timestamp-parser-options' required: - options - if: properties: parser: enum: - parse_grok then: properties: options: $ref: '#grok-pattern-parser-options' required: - options - if: properties: parser: enum: - parse_regex then: properties: options: allOf: - $ref: '#regex-parser-options' ui: display_type: regex-pattern required: - options - if: properties: parser: enum: - parse_apache_log then: properties: options: $ref: '#apache-parser-options' required: - options - if: properties: parser: enum: - parse_nginx_log then: properties: options: $ref: '#nginx-parser-options' required: - options - if: properties: parser: enum: - parse_cef then: properties: options: $ref: '#cef-parser-options' - if: properties: parser: enum: - parse_csv then: properties: options: $ref: '#csv-parser-options' properties: label: oneOf: - type: 'null' - type: string maxLength: 20 title: Title description: >- An arbitrary name you choose to identify the results of this processor for use when connecting them to other components. parser: oneOf: - label: AWS Cloudfront Access Log group: Line Parsers ui: {} enum: - aws_cloudfront_access_log - label: AWS Cloudwatch Log group: Line Parsers ui: {} enum: - parse_aws_cloudwatch_log_subscription_message - label: AWS Load Balancer Log group: Line Parsers ui: {} enum: - parse_aws_alb_log - label: AWS S3 Access Log group: Line Parsers ui: {} enum: - aws_s3_access_log - label: Apache Log group: Line Parsers ui: {} enum: - parse_apache_log - label: CSV group: Line Parsers ui: {} enum: - parse_csv - label: Common Event Format (CEF) Log group: Line Parsers ui: {} enum: - parse_cef - label: Common Log group: Line Parsers ui: {} enum: - parse_common_log - label: Elasticsearch Log group: Line Parsers ui: {} enum: - elasticsearch_log - label: Golang Log group: Line Parsers ui: {} enum: - golang_log - label: Google Log group: Line Parsers ui: {} enum: - parse_glog - label: Grok Pattern group: Line Parsers ui: {} enum: - parse_grok - label: Integer group: Field Parsers ui: {} enum: - parse_int - label: JSON group: Line Parsers ui: {} enum: - parse_json - label: Key/Value group: Line Parsers ui: {} enum: - parse_key_value - label: Kubernetes Log group: Line Parsers ui: {} enum: - parse_klog - label: Linux Authorization group: Line Parsers ui: {} enum: - parse_linux_authorization - label: Mongo Log group: Line Parsers ui: {} enum: - mongo_log - label: Mongo3 Log group: Line Parsers ui: {} enum: - mongo3_log - label: Nginx Log group: Line Parsers ui: {} enum: - parse_nginx_log - label: PSQL Log group: Line Parsers ui: {} enum: - psql_log - label: Query String group: Field Parsers ui: {} enum: - parse_query_string - label: Regular Expression group: Line Parsers ui: feature_flag: pipeline-parser-regex enum: - parse_regex - label: Syslog group: Line Parsers ui: {} enum: - parse_syslog - label: Timestamp group: Field Parsers ui: {} enum: - parse_timestamp - label: Tokens group: Field Parsers ui: {} enum: - parse_tokens - label: URL group: Field Parsers ui: {} enum: - parse_url - label: User Agent group: Field Parsers ui: {} enum: - parse_user_agent - label: VPC Flow Log group: Line Parsers ui: {} description: Parse VPC logs using default format for fields from v2 to v4 enum: - parse_aws_vpc_flow_log ui: display_type: list errorMessage: must be one of allowed parsers title: Parser description: The kind of parser to use against the input value from "field". _output_name: type: string title: '[INTERNAL]' description: >- Internal-only field that holds a static id for the output_name. This way, labels for outputs can be updated independent of the id. default: '' required: - parser - _output_name additionalProperties: type: object minItems: 1 ui: display_type: multi-output-map unique_parser_and_options: true errorMessage: uniqueItemProperties: parser can not be defined more than once required: - field - parsers - if: properties: type: enum: - route deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Route description: >- Route data based on whether it matches (or doesn't match) logical comparisons additionalProperties: false properties: conditionals: type: array minItems: 1 title: Conditionals description: A list of complete conditionals, including expressions and label items: type: object additionalProperties: false title: Conditional description: A list of conditions and a label properties: conditional: $ref: '#condition-group' label: type: string minLength: 1 maxLength: 255 title: Label description: >- An arbitrary name you choose to identify the results of this processor for use when connecting them to other components. _output_name: type: string title: '[INTERNAL]' description: >- Internal-only field that holds a static id for the output_name. This way, labels for outputs can be updated independent of the id. default: '' required: - conditional - label - _output_name required: - conditionals - if: properties: type: enum: - stringify deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Stringify description: Stringify messages as JSON additionalProperties: false - if: properties: type: enum: - compact-fields deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Compact Fields description: Remove empty values from a list of fields additionalProperties: false properties: fields: type: array title: Fields description: A list of fields to remove empty values from items: type: string title: Field description: Field to remove empty value from pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference minItems: 1 options: type: object additionalProperties: false title: Compact Options description: Additional options for removing empty values from fields properties: compact_array: type: boolean title: Compact Array description: Remove empty arrays from a field default: true compact_object: type: boolean title: Compact Object description: Remove empty objects from a field default: true required: - fields - if: properties: type: enum: - flatten-fields deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Flatten Fields description: >- Flattens the object or array value of a field into a single level representation additionalProperties: false properties: fields: type: array title: Fields description: >- Define specific nested objects you wish to flatten. This will not flatten the entire object, just the specified fields. Leave blank to flatten the entire object items: type: string title: Field description: Field to flatten pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference options: type: object additionalProperties: false title: Flatten Options description: Additional options for flattening fields properties: delimiter: type: string default: _ title: Delimiter description: >- When flattening, the nested field labels will be appended to the higher level field names along with a delimiter to allow understanding of the original structure. The default is an underscore (_) to ensure no collisions with the . notation used for field identifiers minLength: 1 maxLength: 1 - if: properties: type: enum: - dedupe deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Dedupe description: Dedupe (remove duplicates) from the data stream additionalProperties: false properties: number_of_events: type: number description: Number of events to compare across title: Number of Events minimum: 2 maximum: 5000 default: 5000 comparison_type: type: string enum: - Ignore - Match description: >- Match: Only compares across the fields which are specified by the user.Ignore: Compares everything but the fields specified by the user. title: Comparison Type default: Match fields: type: array title: Fields description: A list of fields on which to base deduping items: type: string title: Field pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference minItems: 1 required: - number_of_events - comparison_type - fields - if: properties: type: enum: - encrypt-fields deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: title: Encrypt Field description: Encrypts the value of the provided field additionalProperties: false allOf: - if: properties: algorithm: enum: - AES-256-CFB required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CFB required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CFB required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-OFB required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-OFB required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-OFB required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-CTR required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CTR required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CTR required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-CBC-PKCS7 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CBC-PKCS7 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CBC-PKCS7 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-CBC-ANSIX923 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CBC-ANSIX923 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CBC-ANSIX923 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-CBC-ISO7816 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CBC-ISO7816 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CBC-ISO7816 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-CBC-ISO10126 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CBC-ISO10126 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CBC-ISO10126 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 16 maxLength: 16 required: - key properties: field: type: string title: Field description: >- Field to encrypt. The value of the field must be a primitive (string, number, boolean) pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference algorithm: oneOf: - label: AES-256-CFB (key = 32 characters, iv = 16 characters) description: AES-256-CFB (key = 32 characters, iv = 16 characters) enum: - AES-256-CFB - label: AES-192-CFB (key = 24 characters, iv = 16 characters) description: AES-192-CFB (key = 24 characters, iv = 16 characters) enum: - AES-192-CFB - label: AES-128-CFB (key = 16 characters, iv = 16 characters) description: AES-128-CFB (key = 16 characters, iv = 16 characters) enum: - AES-128-CFB - label: AES-256-OFB (key = 32 characters, iv = 16 characters) description: AES-256-OFB (key = 32 characters, iv = 16 characters) enum: - AES-256-OFB - label: AES-192-OFB (key = 24 characters, iv = 16 characters) description: AES-192-OFB (key = 24 characters, iv = 16 characters) enum: - AES-192-OFB - label: AES-128-OFB (key = 16 characters, iv = 16 characters) description: AES-128-OFB (key = 16 characters, iv = 16 characters) enum: - AES-128-OFB - label: AES-256-CTR (key = 32 characters, iv = 16 characters) description: AES-256-CTR (key = 32 characters, iv = 16 characters) enum: - AES-256-CTR - label: AES-192-CTR (key = 24 characters, iv = 16 characters) description: AES-192-CTR (key = 24 characters, iv = 16 characters) enum: - AES-192-CTR - label: AES-128-CTR (key = 16 characters, iv = 16 characters) description: AES-128-CTR (key = 16 characters, iv = 16 characters) enum: - AES-128-CTR - label: AES-256-CBC-PKCS7 (key = 32 characters, iv = 16 characters) description: AES-256-CBC-PKCS7 (key = 32 characters, iv = 16 characters) enum: - AES-256-CBC-PKCS7 - label: AES-192-CBC-PKCS7 (key = 24 characters, iv = 16 characters) description: AES-192-CBC-PKCS7 (key = 24 characters, iv = 16 characters) enum: - AES-192-CBC-PKCS7 - label: AES-128-CBC-PKCS7 (key = 16 characters, iv = 16 characters) description: AES-128-CBC-PKCS7 (key = 16 characters, iv = 16 characters) enum: - AES-128-CBC-PKCS7 - label: AES-256-CBC-ANSIX923 (key = 32 characters, iv = 16 characters) description: AES-256-CBC-ANSIX923 (key = 32 characters, iv = 16 characters) enum: - AES-256-CBC-ANSIX923 - label: AES-192-CBC-ANSIX923 (key = 24 characters, iv = 16 characters) description: AES-192-CBC-ANSIX923 (key = 24 characters, iv = 16 characters) enum: - AES-192-CBC-ANSIX923 - label: AES-128-CBC-ANSIX923 (key = 16 characters, iv = 16 characters) description: AES-128-CBC-ANSIX923 (key = 16 characters, iv = 16 characters) enum: - AES-128-CBC-ANSIX923 - label: AES-256-CBC-ISO7816 (key = 32 characters, iv = 16 characters) description: AES-256-CBC-ISO7816 (key = 32 characters, iv = 16 characters) enum: - AES-256-CBC-ISO7816 - label: AES-192-CBC-ISO7816 (key = 24 characters, iv = 16 characters) description: AES-192-CBC-ISO7816 (key = 24 characters, iv = 16 characters) enum: - AES-192-CBC-ISO7816 - label: AES-128-CBC-ISO7816 (key = 16 characters, iv = 16 characters) description: AES-128-CBC-ISO7816 (key = 16 characters, iv = 16 characters) enum: - AES-128-CBC-ISO7816 - label: AES-256-CBC-ISO10126 (key = 32 characters, iv = 16 characters) description: AES-256-CBC-ISO10126 (key = 32 characters, iv = 16 characters) enum: - AES-256-CBC-ISO10126 - label: AES-192-CBC-ISO10126 (key = 24 characters, iv = 16 characters) description: AES-192-CBC-ISO10126 (key = 24 characters, iv = 16 characters) enum: - AES-192-CBC-ISO10126 - label: AES-128-CBC-ISO10126 (key = 16 characters, iv = 16 characters) description: AES-128-CBC-ISO10126 (key = 16 characters, iv = 16 characters) enum: - AES-128-CBC-ISO10126 ui: display_type: list title: Encryption algorithm description: The encryption algorithm to use key: type: string title: Encryption key description: The encryption key minLength: 16 maxLength: 32 iv_field: type: string title: Initialization vector (IV) field description: >- The field in which to store the generated initialization vector, IV. Each encrypted value will have a unique IV pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference encode_raw_bytes: type: boolean title: Encode encrypted field and IV as Base 64 text description: >- Encode the encrypted value and generated initialization vector as Base64 text default: true required: - field - algorithm - key - iv_field - if: properties: type: enum: - decrypt-fields deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: title: Decrypt Field additionalProperties: false description: Decrypts the value of the provided field allOf: - if: properties: algorithm: enum: - AES-256-CFB required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CFB required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CFB required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-OFB required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-OFB required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-OFB required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-CTR required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CTR required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CTR required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-CBC-PKCS7 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CBC-PKCS7 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CBC-PKCS7 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-CBC-ANSIX923 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CBC-ANSIX923 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CBC-ANSIX923 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-CBC-ISO7816 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CBC-ISO7816 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CBC-ISO7816 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-CBC-ISO10126 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CBC-ISO10126 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CBC-ISO10126 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 16 maxLength: 16 required: - key properties: field: type: string title: Field description: Field to decrypt. The value of the field must be a string pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference algorithm: oneOf: - label: AES-256-CFB (key = 32 characters, iv = 16 characters) description: AES-256-CFB (key = 32 characters, iv = 16 characters) enum: - AES-256-CFB - label: AES-192-CFB (key = 24 characters, iv = 16 characters) description: AES-192-CFB (key = 24 characters, iv = 16 characters) enum: - AES-192-CFB - label: AES-128-CFB (key = 16 characters, iv = 16 characters) description: AES-128-CFB (key = 16 characters, iv = 16 characters) enum: - AES-128-CFB - label: AES-256-OFB (key = 32 characters, iv = 16 characters) description: AES-256-OFB (key = 32 characters, iv = 16 characters) enum: - AES-256-OFB - label: AES-192-OFB (key = 24 characters, iv = 16 characters) description: AES-192-OFB (key = 24 characters, iv = 16 characters) enum: - AES-192-OFB - label: AES-128-OFB (key = 16 characters, iv = 16 characters) description: AES-128-OFB (key = 16 characters, iv = 16 characters) enum: - AES-128-OFB - label: AES-256-CTR (key = 32 characters, iv = 16 characters) description: AES-256-CTR (key = 32 characters, iv = 16 characters) enum: - AES-256-CTR - label: AES-192-CTR (key = 24 characters, iv = 16 characters) description: AES-192-CTR (key = 24 characters, iv = 16 characters) enum: - AES-192-CTR - label: AES-128-CTR (key = 16 characters, iv = 16 characters) description: AES-128-CTR (key = 16 characters, iv = 16 characters) enum: - AES-128-CTR - label: AES-256-CBC-PKCS7 (key = 32 characters, iv = 16 characters) description: AES-256-CBC-PKCS7 (key = 32 characters, iv = 16 characters) enum: - AES-256-CBC-PKCS7 - label: AES-192-CBC-PKCS7 (key = 24 characters, iv = 16 characters) description: AES-192-CBC-PKCS7 (key = 24 characters, iv = 16 characters) enum: - AES-192-CBC-PKCS7 - label: AES-128-CBC-PKCS7 (key = 16 characters, iv = 16 characters) description: AES-128-CBC-PKCS7 (key = 16 characters, iv = 16 characters) enum: - AES-128-CBC-PKCS7 - label: AES-256-CBC-ANSIX923 (key = 32 characters, iv = 16 characters) description: AES-256-CBC-ANSIX923 (key = 32 characters, iv = 16 characters) enum: - AES-256-CBC-ANSIX923 - label: AES-192-CBC-ANSIX923 (key = 24 characters, iv = 16 characters) description: AES-192-CBC-ANSIX923 (key = 24 characters, iv = 16 characters) enum: - AES-192-CBC-ANSIX923 - label: AES-128-CBC-ANSIX923 (key = 16 characters, iv = 16 characters) description: AES-128-CBC-ANSIX923 (key = 16 characters, iv = 16 characters) enum: - AES-128-CBC-ANSIX923 - label: AES-256-CBC-ISO7816 (key = 32 characters, iv = 16 characters) description: AES-256-CBC-ISO7816 (key = 32 characters, iv = 16 characters) enum: - AES-256-CBC-ISO7816 - label: AES-192-CBC-ISO7816 (key = 24 characters, iv = 16 characters) description: AES-192-CBC-ISO7816 (key = 24 characters, iv = 16 characters) enum: - AES-192-CBC-ISO7816 - label: AES-128-CBC-ISO7816 (key = 16 characters, iv = 16 characters) description: AES-128-CBC-ISO7816 (key = 16 characters, iv = 16 characters) enum: - AES-128-CBC-ISO7816 - label: AES-256-CBC-ISO10126 (key = 32 characters, iv = 16 characters) description: AES-256-CBC-ISO10126 (key = 32 characters, iv = 16 characters) enum: - AES-256-CBC-ISO10126 - label: AES-192-CBC-ISO10126 (key = 24 characters, iv = 16 characters) description: AES-192-CBC-ISO10126 (key = 24 characters, iv = 16 characters) enum: - AES-192-CBC-ISO10126 - label: AES-128-CBC-ISO10126 (key = 16 characters, iv = 16 characters) description: AES-128-CBC-ISO10126 (key = 16 characters, iv = 16 characters) enum: - AES-128-CBC-ISO10126 ui: display_type: list title: Decryption algorithm description: The algorithm with which the data was encrypted key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 16 maxLength: 32 iv_field: type: string title: Initialization vector (IV) field description: The field from which to read the initialization vector, IV pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference decode_raw_bytes: type: boolean title: Decode the encrypted value and IV as Base 64 text description: Decode the encrypted value and initialization vector as Base64 text default: true required: - field - algorithm - key - iv_field - if: properties: type: enum: - js-script deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Script Execution description: >- Use JavaScript to reshape and transform your data. You can combine multiple actions like filtering, dropping, mapping, and casting inside of a single js script. additionalProperties: true properties: script: type: string title: Script ui: display_type: code-editor options: language: js description: The JavaScript representation of the event transformation. minLength: 1 maxLength: 10000 default: | // Modify the event using a subset of the JavaScript language. // The function must return the modified event function processEvent(message, metadata, timestamp, annotations) { return message } - if: properties: type: enum: - unroll deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Unroll description: Takes an array of events and emits them all as individual events. additionalProperties: false properties: field: type: string title: Field description: The JSON field name that contains an array of events pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference values_only: type: boolean title: Values only? description: >- When enabled, the values from the specified array field will be emitted as new events. Otherwise, the original event will be duplicated for each value in the array field, with the unrolled value present in the field specified. default: true required: - field - if: properties: type: enum: - vrl deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: VRL description: Modify your observability data using Vector Remap Language (VRL) additionalProperties: false properties: source: type: string title: VRL Source description: VRL code to execute for each event ui: display_type: code-editor options: language: vrl minLength: 1 default: .message = .message required: - source - if: properties: type: enum: - reduce deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Reduce description: Combine multiple events over time into one based on a set of criteria additionalProperties: false properties: duration_ms: type: number maximum: 7200000 title: Duration description: >- The amount of time (in milliseconds) to allow streaming events to accumulate into a single "reduced" event. The process repeats indefinitely, or until an "ends when" condition is satisfied. ui: display_type: milliseconds-to-human-readable default: 30000 max_events: type: number minimum: 1 maximum: 100000 title: Max Events description: >- The maximum number of events that can be included in a time window (specified by Duration above). The reduce operation will stop once it has reached this number of events, regardless of whether the Duration seconds have elapsed. group_by: type: array minItems: 1 title: Group By Field Path items: type: string title: Field description: The dot-notated property path errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ description: >- Before reducing, group events based on matching data from each of these field paths. Supports nesting via dot-notation. date_formats: type: array title: Date Formats description: >- Describes which root-level properties are dates, and their expected format. Dot-notation is supported, but nested field lookup paths will be an error. minItems: 1 items: type: object additionalProperties: false properties: field: type: string title: Date Field Path description: Specifies a root-level path property that contains a date value. maxLength: 200 errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ format: type: string title: Date Format description: The template describing the date format format: strftime_datetime errorMessage: must be a valid strftime datetime format maxLength: 200 required: - field - format merge_strategies: type: array title: Merge Strategy per Field minItems: 1 description: >- Specify merge strategies for individual root-level properties. Dot-notation is supported, but nested field lookup paths will be an error. items: type: object additionalProperties: false properties: field: type: string title: Field Path description: >- This is a root-level path property to apply a merge strategy to its value maxLength: 200 errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ strategy: oneOf: - label: Array description: Append each value onto an array enum: - array - label: Smallest array description: Keep the smallest array within each reduce time window enum: - shortest_array - label: Largest array description: Keep the largest array within each reduce time window enum: - longest_array - label: Unique Array description: All unique values go into a flattened array enum: - flat_unique - label: Join with space description: Concatenate into a single space-delimited string enum: - concat - label: Join with newline description: Concatenate into a single newline-delimited string enum: - concat_newline - label: Join description: Concatenate into a single string without using a delimiter enum: - concat_raw - label: Keep first description: Keep only the first value and discard the rest enum: - discard - label: Keep last description: Keep only the last non-null value enum: - retain - label: Min value description: >- Keep the minimum numeric value within each reduce time window enum: - min - label: Max value description: >- Keep the maximum numeric value within each reduce time window enum: - max - label: Sum description: Add numeric values together enum: - sum ui: display_type: list errorMessage: must be one of the allowed merge strategies title: Merge Strategy description: The merge strategy to be used for the specified property required: - field - strategy flush_condition: title: Event Flush Condition description: >- Force accumulated event reduction to flush the result when a conditional expression evaluates to true on an inbound event. allOf: - type: object properties: when: type: string enum: - none - starts_when - ends_when default: none ui: display_type: list errorMessage: '' title: Flush When required: - when - if: properties: when: not: enum: - none then: properties: conditional: allOf: - $ref: '#condition-group' ui: display_type: rules-engine required: - conditional - if: properties: type: enum: - map-fields deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Map Fields description: Maps data from one field to another, either by moving or copying. additionalProperties: false properties: mappings: type: array title: Mappings description: >- A list of field mappings. Mappings are applied in the order they are defined. minItems: 1 ui: display_type: sortable items: type: object properties: source_field: type: string title: Source field description: The field to copy data from pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference target_field: type: string title: Target field description: The field to copy data into pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference drop_source: type: boolean title: Drop source? description: >- When enabled, the source field is dropped after the data is copied to the target field. Otherwise, it is preserved. default: false overwrite_target: type: boolean title: Overwrite target? description: >- When enabled, any existing data in the target field is overwritten. Otherwise, the target field will be preserved and this mapping will have no effect. default: false required: - source_field - target_field - drop_source - overwrite_target if: properties: drop_source: enum: - true required: - drop_source then: properties: source_field: type: string title: Source field description: The field to copy data from pattern: >- ^(?:metadata|\.|(?:metadata|message)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ minLength: 1 errorMessage: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields). Note that dropping an event's message or timestamp is not allowed required: - source_field required: - mappings - if: properties: type: enum: - metrics-tag-cardinality-limit deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: title: Tag Cardinality Limit (Metrics) description: >- Limits the cardinality of metric events by either dropping events or tags that exceed a specified value limit additionalProperties: false allOf: - if: title: Mode properties: mode: type: string enum: - exact then: title: Value Limit properties: value_limit: type: integer title: Value Limit description: Maximum number of unique values for tags minimum: 1 maximum: 500 required: - value_limit - if: title: Mode properties: mode: type: string enum: - probabilistic then: title: Value Limit properties: value_limit: type: integer title: Value Limit description: Maximum number of unique values for tags minimum: 1 maximum: 5000 required: - value_limit properties: tags: type: array title: Tags description: >- A list of tags to apply cardinality limits. If none are provided, all tags will be considered. items: type: string title: Tag minLength: 1 maxLength: 100 minItems: 0 maxItems: 10 exclude_tags: type: array title: Exclude Tags description: A list of tags to explicitly exclude from cardinality limits. items: type: string title: Tag minLength: 1 maxLength: 100 minItems: 0 maxItems: 10 action: type: string title: Action description: The action to take when a tag's cardinality exceeds the value limit enum: - drop_tag - drop_event default: drop_event value_limit: type: integer title: Value Limit description: Maximum number of unique values for tags mode: type: string title: Mode description: The method to used to reduce tag value cardinality enum: - exact - probabilistic default: exact - if: properties: type: enum: - aggregate deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Aggregate description: >- Aggregates multiple events into a single event using either a tumbling interval window or a sliding interval window. additionalProperties: false properties: group_by: type: array minItems: 1 maxItems: 100 title: Group By Field Path description: >- When aggregating, group events based on matching values from each of these field paths. Supports nesting via dot-notation. items: type: string title: Field description: The dot-notated property path errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ ui: default_hint: - .name - .namespace - .tags evaluate: additionalProperties: type: string title: Evaluate description: How events should be aggregated within an interval. allOf: - if: properties: operation: enum: - CUSTOM then: properties: script: type: string title: Script ui: display_type: code-editor options: language: js description: The JavaScript representation of an event aggregation. minLength: 1 maxLength: 2048 default: > // Perform custom aggregation of the current event into the accumulated event. // The accumulated event is persisted across the entire aggregation window. // The function must return the accumulated event, which will be emitted when // the aggregation window elapses. function aggregateEvent(accum, event, metadata, timestamp, annotations) { return accum } required: - script properties: operation: title: Operation oneOf: - label: add description: >- Adds new values together if they are of the same type. For counters and gauges the values are added together. For sets, this is a union of the elements. For distribution, the samples are concatenated. For histograms, values in the same bucket will be added together. enum: - DEFAULT - label: sum description: >- Values are added together. Works for the GAUGE and COUNT metric types. enum: - SUM - label: minimum description: >- Get the minimum value for a series of events. Works for the GAUGE and COUNT metric types. enum: - MIN - label: maximum description: >- Get the maximum value for a series of events. Works for the GAUGE and COUNT metric types. enum: - MAX - label: average description: >- Values are averaged together. Works for the GAUGE and COUNT metric types. enum: - AVG - label: set intersection description: The aggregate produces a set of unique values. enum: - SET_INTERSECTION - label: distribution concatenation description: The aggregate produces a list of values combined together. enum: - DIST_CONCAT - label: custom description: Use a custom script to perform aggregation. enum: - CUSTOM ui: display_type: list required: - operation window: default: {} additionalProperties: type: integer title: Window Type description: >- Sliding windows can overlap, whereas tumbling windows are disjoint. For example, a tumbling window has a fixed time span and any events that fall within the "interval" will be used in the aggregate. Whereas in a sliding window, the aggregation occurs every "window duration" seconds after an event is encountered. allOf: - if: properties: type: enum: - sliding required: - type then: properties: window_min: type: integer title: Slide Duration (seconds) description: >- The amount of time to wait before creating a new sliding window for future events. minimum: 1 maximum: 200 default: 5 required: - window_min properties: type: type: string title: Type enum: - tumbling - sliding default: tumbling interval: type: integer title: Interval (seconds) description: The interval over which events are aggregated in seconds. default: 5 minimum: 1 maximum: 90000 required: - type - interval event_timestamp: type: string title: Event Timestamp description: >- The path to a field on the event that contains a epoch timestamp value. If an event does not have a timestamp field, events will be associated to the wall clock value when the event is processed. default: timestamp maxLength: 200 errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ conditional: ui: display_type: rules-engine title: Event Flush Condition description: >- Force accumulated event reduction to flush the result when a conditional expression evaluates to true on an inbound event. allOf: - $ref: '#condition-group' required: - evaluate - window - if: properties: type: enum: - event-to-metric deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Event to Metric description: Allows conversion between arbitrary events and a Metric. additionalProperties: false properties: metric_name: type: string title: Metric Name description: The machine name of the metric to emit. minLength: 1 maxLength: 128 pattern: ^[a-zA-Z][a-zA-Z0-9_:]*$ errorMessage: must be a valid metric name metric_kind: title: Kind description: >- The kind of metric to emit, Absolute or Incremental. Absolute metrics represent a complete value, and will generally replace an existing value for the metric in the target destination. Incremental metrics represent an additive value which is aggregated in the target destination to produce a new value. oneOf: - label: Incremental enum: - incremental - label: Absolute enum: - absolute default: incremental ui: display_type: radio-group metric_type: title: Type oneOf: - label: Counter enum: - counter - label: Sum enum: - sum - label: Gauge enum: - gauge default: counter ui: display_type: list value: title: Value description: The value of this metric. additionalProperties: false allOf: - if: properties: value_type: enum: - field then: title: Value properties: value: type: string title: Field value description: The field to use as the value. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference required: - value - if: properties: value_type: enum: - value then: title: Value properties: value: type: number title: Value description: The value. required: - value properties: value_type: title: Value type description: Either a value from an event field, or a new value input. oneOf: - label: Value from event field enum: - field - label: New value enum: - value default: field ui: display_type: radio-group value: type: - number - string required: - value_type namespace: title: Namespace description: The namespace for this metric. additionalProperties: false allOf: - if: properties: value_type: enum: - field then: title: Value properties: value: type: string title: Field value description: The field to use as the value. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference required: - value - if: properties: value_type: enum: - value required: - value_type then: title: Value properties: value: type: string title: Value description: The value. minLength: 1 maxLength: 128 required: - value properties: value_type: title: Value type description: >- Either a value from an event field, a new value input, or none to omit the namespace. oneOf: - label: Value from event field enum: - field - label: New value enum: - value - label: None enum: - none default: none ui: display_type: radio-group value: type: - string - 'null' required: - value_type tags: type: array maxItems: 10 title: Tags description: A set of tags (also called labels) to apply to the metric event. items: type: object title: Tag description: A tag to apply to the metric event. allOf: - if: properties: value_type: enum: - field then: title: Value properties: value: type: string title: Field value description: >- The field to use as the value of this tag. Note that fields with highly-variable values will result in high-cardinality metrics, which may impact storage or cost in downstream destinations. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference fallback: type: string title: Fallback value description: >- The value to use for this tag when the field value is missing. minLength: 1 maxLength: 512 required: - value - if: properties: value_type: enum: - value then: title: Value properties: value: type: string title: Value description: The value for this tag. minLength: 1 maxLength: 128 required: - value properties: name: type: string title: Name description: The tag name pattern: ^[a-zA-Z][a-zA-Z0-9_]*$ minLength: 1 maxLength: 128 errorMessage: must be a valid metric tag name value_type: title: Value type description: Either a value from an event field, or a new value input. oneOf: - label: Value from event field enum: - field - label: New value enum: - value default: field ui: display_type: radio-group value: type: string required: - name - value_type - value additionalProperties: type: string required: - metric_name - metric_kind - metric_type - value - namespace - if: properties: type: enum: - clustering deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Clustering description: Finds clusters of similar values additionalProperties: false properties: cluster_field: type: string title: Field description: Field to cluster values on pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference default: .message similarity_threshold: type: number title: Similarity threshold description: >- A threshold from 0.0 (not similar) to 1.0 (exactly similar) that determines how close two values need to be to be consider as being part of the same cluster default: 0.4 minimum: 0 maximum: 1 max_clusters: type: integer title: Maximum cluster description: Maximum number of clusters kept cached by the transform default: 1000 minimum: 1 maximum: 2000 max_node_depth: type: integer title: Maximum tree depth description: Maximum depth of the prefix tree default: 4 minimum: 2 maximum: 8 max_children: type: integer title: Maximum children default: 100 minimum: 2 maximum: 200 description: Maximum number of children per node of the prefix tree required: - cluster_field - if: properties: type: enum: - redact deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Redact description: Detect and anonymize sensitive data in logs and metrics additionalProperties: false properties: field: type: string title: Field description: >- Detect patterns in the specified field. If the field is empty, the processor will look for patterns across all the fields. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ default: . maxLength: 256 errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference mask_patterns: type: array title: Redact Patterns description: List of PII patterns to detect and mask minItems: 1 maxItems: 20 items: type: object additionalProperties: type: string title: Redact Pattern description: A PII pattern to detect and/or anonymize allOf: - if: properties: action: enum: - Replace required: - action then: properties: replacement: type: string title: Replacement description: >- The text to replace the detected pattern. Example: 000-00-0000 minLength: 1 maxLength: 512 required: - replacement - if: properties: action: enum: - Hash required: - action then: properties: algorithm: title: Algorithm description: >- Hash and encode (base16 or hex) matched PII data using the specified algorithm oneOf: - label: sha2 (sha-512_256) enum: - sha2 - label: sha3 (sha3-512) enum: - sha3 ui: display_type: list required: - algorithm - if: properties: name: enum: - custom required: - name then: properties: custom_options: allOf: - type: object additionalProperties: false properties: pattern: type: string title: Pattern description: The custom pattern minLength: 1 maxLength: 4096 case_sensitive: type: boolean title: Case sensitive description: Perform case sensitive matching default: true multiline: type: boolean title: Multiline mode description: >- ^ and $ match beginning and end of line respectively default: false match_newline: type: boolean title: Dot all mode description: Dot matches newline character default: false crlf_newline: type: boolean title: Carriage return (CRLF) mode description: "When multiline mode is enabled, newline character is interpreted as \r\n" default: false ignore_whitespace: type: boolean title: Verbose mode description: Ignore whitespace and allow comment lines default: false required: - pattern title: Replace Expression description: >- The regular expression used to find replacements. Named capture groups are supported, e.g. `(?P<key>\w+):(?P<value>.+)`. ui: display_type: regex-pattern required: - custom_options properties: name: title: Pattern description: The PII pattern to match oneOf: - label: Credit card number enum: - credit_card - label: Email address enum: - email_address - label: IPv4 address enum: - ipv4_address - label: US or Canada phone number enum: - us_canada_phone_number - label: US social security number enum: - us_social_security_number - label: Custom pattern ui: feature_flag: pipeline-parser-regex enum: - custom ui: display_type: list action: type: string title: Action description: Action to take on the detected PII data enum: - Hash - Replace - Detect Only default: Replace required: - name - action required: - mask_patterns - if: properties: type: enum: - throttle deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Throttle description: Throttle (rate-limit) events passing through this component additionalProperties: false properties: threshold: type: number minimum: 1 maximum: 10000000 title: Limit description: >- The number of events to allow over the time window. If a Key Field is specified, the limit will be applied to events for each unique value of that field. default: 1 window_ms: type: number minimum: 1000 maximum: 2592000000 title: Window description: The time window over which the configured limit is applied ui: display_type: milliseconds-to-human-readable default: 5000 key_field: type: string title: Key Field description: >- When configured, the value from this field will be used to group events into separate buckets for throttling. If left blank, or if the event does not contain a value in this field, the event will be throttled as part of the default bucket. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference exclude: allOf: - $ref: '#condition-group' title: Exclude description: When configured, events matching this condition will not be throttled. ui: display_type: rules-engine required: - threshold - window_ms - if: properties: type: enum: - set-timestamp deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Set Timestamp description: >- Parse a list of fields using a chosen time format. First match will override the default timestamp of the event. additionalProperties: false properties: parsers: type: array title: List of Fields and Parsers description: >- The list of fields and parsers to use in order of priority, short-circuiting on the first successful match. ui: display_type: multi-output-map options: sortable: false items: type: object properties: field: type: string title: Field description: The field whose value will be parsed by the chosen date pattern pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ default: . maxLength: 512 errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference options: $ref: '#timestamp-parser-options' required: - field - options minItems: 1 required: - parsers - if: properties: type: enum: - data-profiler deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Data Profiler description: >- Profile the data sent through your pipeline. The system will generate annotations such as event type and total bytes to be used downstream. It will also normalize the app/host/level/line fields before sending the event downstream. A profile of the data including total, events, event size, log composition, and message templates will be generated as well. additionalProperties: false properties: app_fields: type: array title: App Path(s) description: A list of paths to look for the app value minItems: 1 maxItems: 10 uniqueItems: true items: type: string title: App description: The path to the app maxLength: 512 pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference ui: default_hint: - .app - .application - .container host_fields: type: array title: Host Path(s) description: A list of paths to look for the host value minItems: 1 maxItems: 10 uniqueItems: true items: type: string title: Host description: The path to the host maxLength: 512 pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference ui: default_hint: - .host - .hostname level_fields: type: array title: Level Path(s) description: A list of paths to look for the log level minItems: 1 maxItems: 10 uniqueItems: true items: type: string title: Level description: The path to the level maxLength: 512 pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference ui: default_hint: - .level - .log_level line_fields: type: array title: Line Path(s) description: A list of paths to look for the event line minItems: 1 maxItems: 10 uniqueItems: true items: type: string title: Line description: The path to the line maxLength: 512 pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference ui: default_hint: - .line - .message label_fields: type: array title: Label Path(s) description: A list of paths to look for the event labels maxItems: 10 uniqueItems: true items: type: string title: Label description: The path to the labels maxLength: 512 pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference ui: default_hint: - .labels field_summary: type: boolean default: false title: Field Summary description: >- If enabled the profiler will attempt to summarize the values in fields of events passing through the node. ui: display_type: hidden ff_default_value: pipeline-node-transform-field-summary required: - app_fields - host_fields - level_fields - line_fields - if: properties: type: enum: - replace deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: title: Replace description: Replace parts of a string using a regular expression or static string additionalProperties: type: object allOf: - if: properties: replacement_type: enum: - regex required: - replacement_type then: properties: options: allOf: - type: object additionalProperties: false properties: pattern: type: string minLength: 1 maxLength: 4096 case_sensitive: type: boolean title: Case sensitive description: Perform case sensitive matching default: true multiline: type: boolean title: Multiline mode description: ^ and $ match beginning and end of line respectively default: false match_newline: type: boolean title: Dot all mode description: Dot matches newline character default: false crlf_newline: type: boolean title: Carriage return (CRLF) mode description: "When multiline mode is enabled, newline character is interpreted as \r\n" default: false ignore_whitespace: type: boolean title: Verbose mode description: Ignore whitespace and allow comment lines default: false required: - pattern title: Replace Expression description: >- The regular expression used to find replacements. Named capture groups are supported, e.g. `(?P<key>\w+):(?P<value>.+)`. ui: display_type: regex-pattern replace_with: type: string minLength: 1 maxLength: 4096 title: Replacement String description: >- The replacement string can be static or optionally contain capture group references. Capture groups can be positional or named. For example, to switch the position of two strings in the source string `onetwo`, the pattern could be `(?Pone)(?P.+)`, and the replacement would be `$two$one`. To do the same thing with positional capture groups, a pattern of `(\w{3})(\w{3})`, and a replacement of `$2$1` would have the same result.

To render a literal dollar sign `$` in the replacement, escape it with `$$`. So, if you had a capture group named `price`, to prepend a dollar sign to the replacement, you would use `$$$price`. required: - options - replace_with - if: properties: replacement_type: enum: - string required: - replacement_type then: properties: string_pattern: type: string title: String Pattern minLength: 1 maxLength: 4096 description: >- A static, case-sensitive string, which will be searched and replaced. replace_with: type: string minLength: 1 maxLength: 4096 title: Replacement String description: The static string used in each replacement. required: - string_pattern - replace_with properties: source_field: type: string title: Source Field description: The field that contains a string for replacement to operate on. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ default: . errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference max_times: type: number title: Maximum Replacements description: >- The maximum number of replacements to be made. If not specified, all occurrences will be replaced. minimum: 1 replacement_type: title: Replacement Type description: Specifies how the source string will be searched for replacement. ui: display_type: list errorMessage: must be one of the available pattern types oneOf: - label: String description: Replace a specified string one or more times. enum: - string - label: Regular Expression description: Replace using a regular expression. enum: - regex required: - source_field - replacement_type - if: properties: type: enum: - trace-sampling deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: title: Trace Sampling description: >- Performs sampling of traces per a given rate using the Trace ID as the unique identifier. If a trace is rate selected, all spans of the trace are forwarded. If the Trace ID cannot be found, the event is dropped. allOf: - if: properties: sample_type: enum: - head then: properties: rate: type: integer title: Rate description: >- The rate at which events will be forwarded, expressed as 1/N. For example, `rate = 10` means 1 out of every 10 unique traces will be forwarded and all others will be dropped default: 10 minimum: 2 maximum: 100000 parent_span_id_field: type: 'null' conditionals: type: 'null' required: - rate - if: properties: sample_type: enum: - tail then: properties: parent_span_id_field: type: string default: .parent_span_id title: Parent Span ID description: The field to use as the parent span identifier pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference conditionals: type: array minItems: 1 title: Conditionals description: >- A list of complete conditionals, including expressions and a sample rate to maintain items: type: object additionalProperties: false title: Conditional description: A list of conditions and a sample rate to maintain properties: rate: type: integer title: Rate description: >- The rate at which events will be forwarded, expressed as 1/N. For example, `rate = 10` means 1 out of every 10 unique traces will be forwarded and all others will be dropped. A `rate = 1` means keep all traces that match the condition. minimum: 1 maximum: 100000 default: 10 conditional: allOf: - $ref: '#condition-group' title: Conditional description: >- A group of expressions joined together by an and/or operator. ui: display_type: rules-engine _output_name: type: string title: '[INTERNAL]' description: >- Internal-only field that holds a static id for the output_name. This way, labels for outputs can be updated independent of the id. required: - rate - conditional required: - parent_span_id_field - conditionals properties: sample_type: type: string description: The type of trace sampling to apply. enum: - head title: Sample Type default: head trace_id_field: type: string default: .trace_id title: Trace ID description: The field to use as the identifier of a trace. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference required: - sample_type - trace_id_field properties: title: oneOf: - type: string minLength: 1 maxLength: 512 pattern: ^[^<>"]+$ - type: 'null' errorMessage: invalid title name description: oneOf: - type: string minLength: 1 pattern: ^[^<>"]+$ - type: 'null' errorMessage: invalid description type: type: string enum: - filter - sample - drop-fields - parse - parse-sequentially - route - stringify - compact-fields - flatten-fields - dedupe - encrypt-fields - decrypt-fields - js-script - unroll - vrl - reduce - map-fields - metrics-tag-cardinality-limit - aggregate - event-to-metric - clustering - redact - throttle - set-timestamp - data-profiler - replace - trace-sampling deploy_type: type: string enum: - saas - local default: saas user_config: type: object generation_id: type: integer inputs: type: array items: type: string required: - type - user_config - generation_id - inputs required: true parameters: - schema: type: string format: uuid in: path name: pipeline_id required: true - schema: type: string format: uuid in: path name: pipeline_transform_id required: true responses: '200': description: Default Response /v3/pipeline/{pipeline_id}/transform: post: summary: Add transform tags: - Pipeline Nodes description: Add a transform to a pipeline requestBody: content: application/json: schema: type: object allOf: - if: properties: type: enum: - filter deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Filter description: Define condition(s) to include or exclude events from the pipeline additionalProperties: false properties: action: title: Action description: How to handle events matching this criteria oneOf: - label: Allow events matching this criteria enum: - allow - label: Drop events matching this criteria enum: - drop default: allow ui: display_type: radio-group conditional: allOf: - $ref: '#condition-group' title: Conditional description: A group of expressions joined together by an and/or operator. ui: display_type: rules-engine required: - action - conditional - if: properties: type: enum: - sample deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Sample description: >- Sample data at a given rate, retaining only a subset of data events for further processing. additionalProperties: false properties: rate: type: integer title: Rate description: >- The rate at which events will be forwarded, expressed as 1/N. For example, `rate = 10` means 1 out of every 10 events will be forwarded and the rest will be dropped minimum: 2 maximum: 10000 default: 10 always_include: title: Always Include description: >- Certain events may always want to be included (not sampled), regardless of the sample rate. Events matching this criteria will always show up in the results. additionalProperties: type: - string - number dependencies: field: - str_operator str_operator: - field allOf: - if: title: Operator properties: str_operator: type: string enum: - greater - greater_or_equal - less - less_or_equal ui: display_type: list-operators title: Operator description: The comparison operator required: - str_operator then: title: Value properties: value: errorMessage: pattern: Value must be a number type: - string - number title: Value description: The value used in the expression, ie the "operand" minLength: 1 pattern: ^\s*-?(\d*\.)?\d+\s*$ required: - value - if: title: Operator properties: str_operator: type: string enum: - greater - greater_or_equal - less - less_or_equal - equal - contains - is_ip_in_cidr_range - ends_with - starts_with - drain_matches - contains_token_term - contains_token_prefix - percent_change_greater - percent_change_greater_or_equal - percent_change_less - percent_change_less_or_equal - value_change_greater - value_change_greater_or_equal - value_change_less - value_change_less_or_equal ui: display_type: list-operators title: Operator description: The comparison operator required: - str_operator then: title: Value properties: value: type: - string - number title: Value description: The value used in the expression, ie the "operand" minLength: 1 required: - value - if: title: Operator properties: str_operator: type: string enum: - contains - equal - ends_with - starts_with - drain_matches - contains_token_term - contains_token_prefix ui: display_type: list-operators title: Operator required: - str_operator then: title: Case sensitive properties: case_sensitive: type: boolean title: Case sensitive description: Perform case sensitive comparison? default: true properties: field: type: string title: Field description: The JSON field to use in a condition to always include in sampling pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference str_operator: type: string enum: - greater - greater_or_equal - less - less_or_equal - equal - contains - is_ip_in_cidr_range - is_metric - is_array - is_boolean - is_empty - is_null - is_number - is_object - is_string - exists - ends_with - starts_with - contains_token_term - contains_token_prefix - regex_match ui: display_type: list-operators title: Operator description: The comparison operator negate: type: boolean default: false title: Negate description: Negate the comparison expression required: - rate - if: properties: type: enum: - drop-fields deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Remove Fields description: Remove fields additionalProperties: false properties: fields: type: array title: Fields description: A list of fields to be removed items: type: string title: Field pattern: >- ^(?:metadata|\.|(?:metadata|message)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ minLength: 1 errorMessage: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields). Note that dropping an event's message or timestamp is not allowed minItems: 1 required: - fields - if: properties: type: enum: - parse deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: title: Parse description: Parse a specified field using the chosen parser additionalProperties: type: object errorMessage: properties: target_field: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) allOf: - if: properties: parser: enum: - parse_key_value then: properties: options: $ref: '#key-value-parser-options' - if: properties: parser: enum: - parse_timestamp then: properties: options: $ref: '#timestamp-parser-options' required: - options - if: properties: parser: enum: - parse_grok then: properties: options: $ref: '#grok-pattern-parser-options' required: - options - if: properties: parser: enum: - parse_regex then: properties: options: allOf: - $ref: '#regex-parser-options' ui: display_type: regex-pattern required: - options - if: properties: parser: enum: - parse_apache_log then: properties: options: $ref: '#apache-parser-options' required: - options - if: properties: parser: enum: - parse_nginx_log then: properties: options: $ref: '#nginx-parser-options' required: - options - if: properties: parser: enum: - parse_cef then: properties: options: $ref: '#cef-parser-options' - if: properties: parser: enum: - parse_csv then: properties: options: $ref: '#csv-parser-options' properties: field: type: string title: Field description: The JSON field whose value should be parsed. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ default: . errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference target_field: oneOf: - type: 'null' - type: string pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ maxLength: 512 errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference title: Target field description: >- The field into which the parsed value should be inserted. Leave blank to insert the parsed data into the original field. default: null parser: oneOf: - label: AWS Cloudfront Access Log group: Line Parsers ui: {} enum: - aws_cloudfront_access_log - label: AWS Cloudwatch Log group: Line Parsers ui: {} enum: - parse_aws_cloudwatch_log_subscription_message - label: AWS Load Balancer Log group: Line Parsers ui: {} enum: - parse_aws_alb_log - label: AWS S3 Access Log group: Line Parsers ui: {} enum: - aws_s3_access_log - label: Apache Log group: Line Parsers ui: {} enum: - parse_apache_log - label: CSV group: Line Parsers ui: {} enum: - parse_csv - label: Common Event Format (CEF) Log group: Line Parsers ui: {} enum: - parse_cef - label: Common Log group: Line Parsers ui: {} enum: - parse_common_log - label: Elasticsearch Log group: Line Parsers ui: {} enum: - elasticsearch_log - label: Golang Log group: Line Parsers ui: {} enum: - golang_log - label: Google Log group: Line Parsers ui: {} enum: - parse_glog - label: Grok Pattern group: Line Parsers ui: {} enum: - parse_grok - label: Integer group: Field Parsers ui: {} enum: - parse_int - label: JSON group: Line Parsers ui: {} enum: - parse_json - label: Key/Value group: Line Parsers ui: {} enum: - parse_key_value - label: Kubernetes Log group: Line Parsers ui: {} enum: - parse_klog - label: Linux Authorization group: Line Parsers ui: {} enum: - parse_linux_authorization - label: Mongo Log group: Line Parsers ui: {} enum: - mongo_log - label: Mongo3 Log group: Line Parsers ui: {} enum: - mongo3_log - label: Nginx Log group: Line Parsers ui: {} enum: - parse_nginx_log - label: PSQL Log group: Line Parsers ui: {} enum: - psql_log - label: Query String group: Field Parsers ui: {} enum: - parse_query_string - label: Regular Expression group: Line Parsers ui: feature_flag: pipeline-parser-regex enum: - parse_regex - label: Syslog group: Line Parsers ui: {} enum: - parse_syslog - label: Timestamp group: Field Parsers ui: {} enum: - parse_timestamp - label: Tokens group: Field Parsers ui: {} enum: - parse_tokens - label: URL group: Field Parsers ui: {} enum: - parse_url - label: User Agent group: Field Parsers ui: {} enum: - parse_user_agent - label: VPC Flow Log group: Line Parsers ui: {} description: Parse VPC logs using default format for fields from v2 to v4 enum: - parse_aws_vpc_flow_log ui: display_type: list errorMessage: must be one of allowed parsers title: Parser description: The kind of parser to use against the input value from "field". required: - field - parser - if: properties: type: enum: - parse-sequentially deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Parse Sequentially description: >- Parse fields using one or more parsers in order, sending out the event on the first successful match. additionalProperties: false errorMessage: properties: target_field: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) properties: field: type: string title: Field description: The field whose value should be parsed. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ default: . maxLength: 512 errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference target_field: oneOf: - type: 'null' - type: string pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ maxLength: 512 errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference title: Target field description: >- The field into which the parsed value should be inserted. Leave blank to insert the parsed data into the original field. default: null parsers: type: array title: List of parsers description: >- The list of parsers to use in order against the input valuefrom "field", short-circuiting on the first successful match. items: type: object allOf: - if: properties: parser: enum: - parse_key_value then: properties: options: $ref: '#key-value-parser-options' - if: properties: parser: enum: - parse_timestamp then: properties: options: $ref: '#timestamp-parser-options' required: - options - if: properties: parser: enum: - parse_grok then: properties: options: $ref: '#grok-pattern-parser-options' required: - options - if: properties: parser: enum: - parse_regex then: properties: options: allOf: - $ref: '#regex-parser-options' ui: display_type: regex-pattern required: - options - if: properties: parser: enum: - parse_apache_log then: properties: options: $ref: '#apache-parser-options' required: - options - if: properties: parser: enum: - parse_nginx_log then: properties: options: $ref: '#nginx-parser-options' required: - options - if: properties: parser: enum: - parse_cef then: properties: options: $ref: '#cef-parser-options' - if: properties: parser: enum: - parse_csv then: properties: options: $ref: '#csv-parser-options' properties: label: oneOf: - type: 'null' - type: string maxLength: 20 title: Title description: >- An arbitrary name you choose to identify the results of this processor for use when connecting them to other components. parser: oneOf: - label: AWS Cloudfront Access Log group: Line Parsers ui: {} enum: - aws_cloudfront_access_log - label: AWS Cloudwatch Log group: Line Parsers ui: {} enum: - parse_aws_cloudwatch_log_subscription_message - label: AWS Load Balancer Log group: Line Parsers ui: {} enum: - parse_aws_alb_log - label: AWS S3 Access Log group: Line Parsers ui: {} enum: - aws_s3_access_log - label: Apache Log group: Line Parsers ui: {} enum: - parse_apache_log - label: CSV group: Line Parsers ui: {} enum: - parse_csv - label: Common Event Format (CEF) Log group: Line Parsers ui: {} enum: - parse_cef - label: Common Log group: Line Parsers ui: {} enum: - parse_common_log - label: Elasticsearch Log group: Line Parsers ui: {} enum: - elasticsearch_log - label: Golang Log group: Line Parsers ui: {} enum: - golang_log - label: Google Log group: Line Parsers ui: {} enum: - parse_glog - label: Grok Pattern group: Line Parsers ui: {} enum: - parse_grok - label: Integer group: Field Parsers ui: {} enum: - parse_int - label: JSON group: Line Parsers ui: {} enum: - parse_json - label: Key/Value group: Line Parsers ui: {} enum: - parse_key_value - label: Kubernetes Log group: Line Parsers ui: {} enum: - parse_klog - label: Linux Authorization group: Line Parsers ui: {} enum: - parse_linux_authorization - label: Mongo Log group: Line Parsers ui: {} enum: - mongo_log - label: Mongo3 Log group: Line Parsers ui: {} enum: - mongo3_log - label: Nginx Log group: Line Parsers ui: {} enum: - parse_nginx_log - label: PSQL Log group: Line Parsers ui: {} enum: - psql_log - label: Query String group: Field Parsers ui: {} enum: - parse_query_string - label: Regular Expression group: Line Parsers ui: feature_flag: pipeline-parser-regex enum: - parse_regex - label: Syslog group: Line Parsers ui: {} enum: - parse_syslog - label: Timestamp group: Field Parsers ui: {} enum: - parse_timestamp - label: Tokens group: Field Parsers ui: {} enum: - parse_tokens - label: URL group: Field Parsers ui: {} enum: - parse_url - label: User Agent group: Field Parsers ui: {} enum: - parse_user_agent - label: VPC Flow Log group: Line Parsers ui: {} description: Parse VPC logs using default format for fields from v2 to v4 enum: - parse_aws_vpc_flow_log ui: display_type: list errorMessage: must be one of allowed parsers title: Parser description: The kind of parser to use against the input value from "field". _output_name: type: string title: '[INTERNAL]' description: >- Internal-only field that holds a static id for the output_name. This way, labels for outputs can be updated independent of the id. default: '' required: - parser - _output_name additionalProperties: type: object minItems: 1 ui: display_type: multi-output-map unique_parser_and_options: true errorMessage: uniqueItemProperties: parser can not be defined more than once required: - field - parsers - if: properties: type: enum: - route deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Route description: >- Route data based on whether it matches (or doesn't match) logical comparisons additionalProperties: false properties: conditionals: type: array minItems: 1 title: Conditionals description: A list of complete conditionals, including expressions and label items: type: object additionalProperties: false title: Conditional description: A list of conditions and a label properties: conditional: $ref: '#condition-group' label: type: string minLength: 1 maxLength: 255 title: Label description: >- An arbitrary name you choose to identify the results of this processor for use when connecting them to other components. _output_name: type: string title: '[INTERNAL]' description: >- Internal-only field that holds a static id for the output_name. This way, labels for outputs can be updated independent of the id. default: '' required: - conditional - label - _output_name required: - conditionals - if: properties: type: enum: - stringify deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Stringify description: Stringify messages as JSON additionalProperties: false - if: properties: type: enum: - compact-fields deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Compact Fields description: Remove empty values from a list of fields additionalProperties: false properties: fields: type: array title: Fields description: A list of fields to remove empty values from items: type: string title: Field description: Field to remove empty value from pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference minItems: 1 options: type: object additionalProperties: false title: Compact Options description: Additional options for removing empty values from fields properties: compact_array: type: boolean title: Compact Array description: Remove empty arrays from a field default: true compact_object: type: boolean title: Compact Object description: Remove empty objects from a field default: true required: - fields - if: properties: type: enum: - flatten-fields deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Flatten Fields description: >- Flattens the object or array value of a field into a single level representation additionalProperties: false properties: fields: type: array title: Fields description: >- Define specific nested objects you wish to flatten. This will not flatten the entire object, just the specified fields. Leave blank to flatten the entire object items: type: string title: Field description: Field to flatten pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference options: type: object additionalProperties: false title: Flatten Options description: Additional options for flattening fields properties: delimiter: type: string default: _ title: Delimiter description: >- When flattening, the nested field labels will be appended to the higher level field names along with a delimiter to allow understanding of the original structure. The default is an underscore (_) to ensure no collisions with the . notation used for field identifiers minLength: 1 maxLength: 1 - if: properties: type: enum: - dedupe deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Dedupe description: Dedupe (remove duplicates) from the data stream additionalProperties: false properties: number_of_events: type: number description: Number of events to compare across title: Number of Events minimum: 2 maximum: 5000 default: 5000 comparison_type: type: string enum: - Ignore - Match description: >- Match: Only compares across the fields which are specified by the user.Ignore: Compares everything but the fields specified by the user. title: Comparison Type default: Match fields: type: array title: Fields description: A list of fields on which to base deduping items: type: string title: Field pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference minItems: 1 required: - number_of_events - comparison_type - fields - if: properties: type: enum: - encrypt-fields deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: title: Encrypt Field description: Encrypts the value of the provided field additionalProperties: false allOf: - if: properties: algorithm: enum: - AES-256-CFB required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CFB required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CFB required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-OFB required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-OFB required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-OFB required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-CTR required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CTR required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CTR required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-CBC-PKCS7 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CBC-PKCS7 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CBC-PKCS7 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-CBC-ANSIX923 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CBC-ANSIX923 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CBC-ANSIX923 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-CBC-ISO7816 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CBC-ISO7816 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CBC-ISO7816 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-CBC-ISO10126 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CBC-ISO10126 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CBC-ISO10126 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 16 maxLength: 16 required: - key properties: field: type: string title: Field description: >- Field to encrypt. The value of the field must be a primitive (string, number, boolean) pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference algorithm: oneOf: - label: AES-256-CFB (key = 32 characters, iv = 16 characters) description: AES-256-CFB (key = 32 characters, iv = 16 characters) enum: - AES-256-CFB - label: AES-192-CFB (key = 24 characters, iv = 16 characters) description: AES-192-CFB (key = 24 characters, iv = 16 characters) enum: - AES-192-CFB - label: AES-128-CFB (key = 16 characters, iv = 16 characters) description: AES-128-CFB (key = 16 characters, iv = 16 characters) enum: - AES-128-CFB - label: AES-256-OFB (key = 32 characters, iv = 16 characters) description: AES-256-OFB (key = 32 characters, iv = 16 characters) enum: - AES-256-OFB - label: AES-192-OFB (key = 24 characters, iv = 16 characters) description: AES-192-OFB (key = 24 characters, iv = 16 characters) enum: - AES-192-OFB - label: AES-128-OFB (key = 16 characters, iv = 16 characters) description: AES-128-OFB (key = 16 characters, iv = 16 characters) enum: - AES-128-OFB - label: AES-256-CTR (key = 32 characters, iv = 16 characters) description: AES-256-CTR (key = 32 characters, iv = 16 characters) enum: - AES-256-CTR - label: AES-192-CTR (key = 24 characters, iv = 16 characters) description: AES-192-CTR (key = 24 characters, iv = 16 characters) enum: - AES-192-CTR - label: AES-128-CTR (key = 16 characters, iv = 16 characters) description: AES-128-CTR (key = 16 characters, iv = 16 characters) enum: - AES-128-CTR - label: AES-256-CBC-PKCS7 (key = 32 characters, iv = 16 characters) description: AES-256-CBC-PKCS7 (key = 32 characters, iv = 16 characters) enum: - AES-256-CBC-PKCS7 - label: AES-192-CBC-PKCS7 (key = 24 characters, iv = 16 characters) description: AES-192-CBC-PKCS7 (key = 24 characters, iv = 16 characters) enum: - AES-192-CBC-PKCS7 - label: AES-128-CBC-PKCS7 (key = 16 characters, iv = 16 characters) description: AES-128-CBC-PKCS7 (key = 16 characters, iv = 16 characters) enum: - AES-128-CBC-PKCS7 - label: AES-256-CBC-ANSIX923 (key = 32 characters, iv = 16 characters) description: AES-256-CBC-ANSIX923 (key = 32 characters, iv = 16 characters) enum: - AES-256-CBC-ANSIX923 - label: AES-192-CBC-ANSIX923 (key = 24 characters, iv = 16 characters) description: AES-192-CBC-ANSIX923 (key = 24 characters, iv = 16 characters) enum: - AES-192-CBC-ANSIX923 - label: AES-128-CBC-ANSIX923 (key = 16 characters, iv = 16 characters) description: AES-128-CBC-ANSIX923 (key = 16 characters, iv = 16 characters) enum: - AES-128-CBC-ANSIX923 - label: AES-256-CBC-ISO7816 (key = 32 characters, iv = 16 characters) description: AES-256-CBC-ISO7816 (key = 32 characters, iv = 16 characters) enum: - AES-256-CBC-ISO7816 - label: AES-192-CBC-ISO7816 (key = 24 characters, iv = 16 characters) description: AES-192-CBC-ISO7816 (key = 24 characters, iv = 16 characters) enum: - AES-192-CBC-ISO7816 - label: AES-128-CBC-ISO7816 (key = 16 characters, iv = 16 characters) description: AES-128-CBC-ISO7816 (key = 16 characters, iv = 16 characters) enum: - AES-128-CBC-ISO7816 - label: AES-256-CBC-ISO10126 (key = 32 characters, iv = 16 characters) description: AES-256-CBC-ISO10126 (key = 32 characters, iv = 16 characters) enum: - AES-256-CBC-ISO10126 - label: AES-192-CBC-ISO10126 (key = 24 characters, iv = 16 characters) description: AES-192-CBC-ISO10126 (key = 24 characters, iv = 16 characters) enum: - AES-192-CBC-ISO10126 - label: AES-128-CBC-ISO10126 (key = 16 characters, iv = 16 characters) description: AES-128-CBC-ISO10126 (key = 16 characters, iv = 16 characters) enum: - AES-128-CBC-ISO10126 ui: display_type: list title: Encryption algorithm description: The encryption algorithm to use key: type: string title: Encryption key description: The encryption key minLength: 16 maxLength: 32 iv_field: type: string title: Initialization vector (IV) field description: >- The field in which to store the generated initialization vector, IV. Each encrypted value will have a unique IV pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference encode_raw_bytes: type: boolean title: Encode encrypted field and IV as Base 64 text description: >- Encode the encrypted value and generated initialization vector as Base64 text default: true required: - field - algorithm - key - iv_field - if: properties: type: enum: - decrypt-fields deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: title: Decrypt Field additionalProperties: false description: Decrypts the value of the provided field allOf: - if: properties: algorithm: enum: - AES-256-CFB required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CFB required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CFB required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-OFB required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-OFB required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-OFB required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-CTR required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CTR required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CTR required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-CBC-PKCS7 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CBC-PKCS7 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CBC-PKCS7 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-CBC-ANSIX923 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CBC-ANSIX923 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CBC-ANSIX923 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-CBC-ISO7816 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CBC-ISO7816 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CBC-ISO7816 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-CBC-ISO10126 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CBC-ISO10126 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CBC-ISO10126 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 16 maxLength: 16 required: - key properties: field: type: string title: Field description: Field to decrypt. The value of the field must be a string pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference algorithm: oneOf: - label: AES-256-CFB (key = 32 characters, iv = 16 characters) description: AES-256-CFB (key = 32 characters, iv = 16 characters) enum: - AES-256-CFB - label: AES-192-CFB (key = 24 characters, iv = 16 characters) description: AES-192-CFB (key = 24 characters, iv = 16 characters) enum: - AES-192-CFB - label: AES-128-CFB (key = 16 characters, iv = 16 characters) description: AES-128-CFB (key = 16 characters, iv = 16 characters) enum: - AES-128-CFB - label: AES-256-OFB (key = 32 characters, iv = 16 characters) description: AES-256-OFB (key = 32 characters, iv = 16 characters) enum: - AES-256-OFB - label: AES-192-OFB (key = 24 characters, iv = 16 characters) description: AES-192-OFB (key = 24 characters, iv = 16 characters) enum: - AES-192-OFB - label: AES-128-OFB (key = 16 characters, iv = 16 characters) description: AES-128-OFB (key = 16 characters, iv = 16 characters) enum: - AES-128-OFB - label: AES-256-CTR (key = 32 characters, iv = 16 characters) description: AES-256-CTR (key = 32 characters, iv = 16 characters) enum: - AES-256-CTR - label: AES-192-CTR (key = 24 characters, iv = 16 characters) description: AES-192-CTR (key = 24 characters, iv = 16 characters) enum: - AES-192-CTR - label: AES-128-CTR (key = 16 characters, iv = 16 characters) description: AES-128-CTR (key = 16 characters, iv = 16 characters) enum: - AES-128-CTR - label: AES-256-CBC-PKCS7 (key = 32 characters, iv = 16 characters) description: AES-256-CBC-PKCS7 (key = 32 characters, iv = 16 characters) enum: - AES-256-CBC-PKCS7 - label: AES-192-CBC-PKCS7 (key = 24 characters, iv = 16 characters) description: AES-192-CBC-PKCS7 (key = 24 characters, iv = 16 characters) enum: - AES-192-CBC-PKCS7 - label: AES-128-CBC-PKCS7 (key = 16 characters, iv = 16 characters) description: AES-128-CBC-PKCS7 (key = 16 characters, iv = 16 characters) enum: - AES-128-CBC-PKCS7 - label: AES-256-CBC-ANSIX923 (key = 32 characters, iv = 16 characters) description: AES-256-CBC-ANSIX923 (key = 32 characters, iv = 16 characters) enum: - AES-256-CBC-ANSIX923 - label: AES-192-CBC-ANSIX923 (key = 24 characters, iv = 16 characters) description: AES-192-CBC-ANSIX923 (key = 24 characters, iv = 16 characters) enum: - AES-192-CBC-ANSIX923 - label: AES-128-CBC-ANSIX923 (key = 16 characters, iv = 16 characters) description: AES-128-CBC-ANSIX923 (key = 16 characters, iv = 16 characters) enum: - AES-128-CBC-ANSIX923 - label: AES-256-CBC-ISO7816 (key = 32 characters, iv = 16 characters) description: AES-256-CBC-ISO7816 (key = 32 characters, iv = 16 characters) enum: - AES-256-CBC-ISO7816 - label: AES-192-CBC-ISO7816 (key = 24 characters, iv = 16 characters) description: AES-192-CBC-ISO7816 (key = 24 characters, iv = 16 characters) enum: - AES-192-CBC-ISO7816 - label: AES-128-CBC-ISO7816 (key = 16 characters, iv = 16 characters) description: AES-128-CBC-ISO7816 (key = 16 characters, iv = 16 characters) enum: - AES-128-CBC-ISO7816 - label: AES-256-CBC-ISO10126 (key = 32 characters, iv = 16 characters) description: AES-256-CBC-ISO10126 (key = 32 characters, iv = 16 characters) enum: - AES-256-CBC-ISO10126 - label: AES-192-CBC-ISO10126 (key = 24 characters, iv = 16 characters) description: AES-192-CBC-ISO10126 (key = 24 characters, iv = 16 characters) enum: - AES-192-CBC-ISO10126 - label: AES-128-CBC-ISO10126 (key = 16 characters, iv = 16 characters) description: AES-128-CBC-ISO10126 (key = 16 characters, iv = 16 characters) enum: - AES-128-CBC-ISO10126 ui: display_type: list title: Decryption algorithm description: The algorithm with which the data was encrypted key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 16 maxLength: 32 iv_field: type: string title: Initialization vector (IV) field description: The field from which to read the initialization vector, IV pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference decode_raw_bytes: type: boolean title: Decode the encrypted value and IV as Base 64 text description: Decode the encrypted value and initialization vector as Base64 text default: true required: - field - algorithm - key - iv_field - if: properties: type: enum: - js-script deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Script Execution description: >- Use JavaScript to reshape and transform your data. You can combine multiple actions like filtering, dropping, mapping, and casting inside of a single js script. additionalProperties: true properties: script: type: string title: Script ui: display_type: code-editor options: language: js description: The JavaScript representation of the event transformation. minLength: 1 maxLength: 10000 default: | // Modify the event using a subset of the JavaScript language. // The function must return the modified event function processEvent(message, metadata, timestamp, annotations) { return message } - if: properties: type: enum: - unroll deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Unroll description: Takes an array of events and emits them all as individual events. additionalProperties: false properties: field: type: string title: Field description: The JSON field name that contains an array of events pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference values_only: type: boolean title: Values only? description: >- When enabled, the values from the specified array field will be emitted as new events. Otherwise, the original event will be duplicated for each value in the array field, with the unrolled value present in the field specified. default: true required: - field - if: properties: type: enum: - vrl deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: VRL description: Modify your observability data using Vector Remap Language (VRL) additionalProperties: false properties: source: type: string title: VRL Source description: VRL code to execute for each event ui: display_type: code-editor options: language: vrl minLength: 1 default: .message = .message required: - source - if: properties: type: enum: - reduce deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Reduce description: Combine multiple events over time into one based on a set of criteria additionalProperties: false properties: duration_ms: type: number maximum: 7200000 title: Duration description: >- The amount of time (in milliseconds) to allow streaming events to accumulate into a single "reduced" event. The process repeats indefinitely, or until an "ends when" condition is satisfied. ui: display_type: milliseconds-to-human-readable default: 30000 max_events: type: number minimum: 1 maximum: 100000 title: Max Events description: >- The maximum number of events that can be included in a time window (specified by Duration above). The reduce operation will stop once it has reached this number of events, regardless of whether the Duration seconds have elapsed. group_by: type: array minItems: 1 title: Group By Field Path items: type: string title: Field description: The dot-notated property path errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ description: >- Before reducing, group events based on matching data from each of these field paths. Supports nesting via dot-notation. date_formats: type: array title: Date Formats description: >- Describes which root-level properties are dates, and their expected format. Dot-notation is supported, but nested field lookup paths will be an error. minItems: 1 items: type: object additionalProperties: false properties: field: type: string title: Date Field Path description: Specifies a root-level path property that contains a date value. maxLength: 200 errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ format: type: string title: Date Format description: The template describing the date format format: strftime_datetime errorMessage: must be a valid strftime datetime format maxLength: 200 required: - field - format merge_strategies: type: array title: Merge Strategy per Field minItems: 1 description: >- Specify merge strategies for individual root-level properties. Dot-notation is supported, but nested field lookup paths will be an error. items: type: object additionalProperties: false properties: field: type: string title: Field Path description: >- This is a root-level path property to apply a merge strategy to its value maxLength: 200 errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ strategy: oneOf: - label: Array description: Append each value onto an array enum: - array - label: Smallest array description: Keep the smallest array within each reduce time window enum: - shortest_array - label: Largest array description: Keep the largest array within each reduce time window enum: - longest_array - label: Unique Array description: All unique values go into a flattened array enum: - flat_unique - label: Join with space description: Concatenate into a single space-delimited string enum: - concat - label: Join with newline description: Concatenate into a single newline-delimited string enum: - concat_newline - label: Join description: Concatenate into a single string without using a delimiter enum: - concat_raw - label: Keep first description: Keep only the first value and discard the rest enum: - discard - label: Keep last description: Keep only the last non-null value enum: - retain - label: Min value description: >- Keep the minimum numeric value within each reduce time window enum: - min - label: Max value description: >- Keep the maximum numeric value within each reduce time window enum: - max - label: Sum description: Add numeric values together enum: - sum ui: display_type: list errorMessage: must be one of the allowed merge strategies title: Merge Strategy description: The merge strategy to be used for the specified property required: - field - strategy flush_condition: title: Event Flush Condition description: >- Force accumulated event reduction to flush the result when a conditional expression evaluates to true on an inbound event. allOf: - type: object properties: when: type: string enum: - none - starts_when - ends_when default: none ui: display_type: list errorMessage: '' title: Flush When required: - when - if: properties: when: not: enum: - none then: properties: conditional: allOf: - $ref: '#condition-group' ui: display_type: rules-engine required: - conditional - if: properties: type: enum: - map-fields deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Map Fields description: Maps data from one field to another, either by moving or copying. additionalProperties: false properties: mappings: type: array title: Mappings description: >- A list of field mappings. Mappings are applied in the order they are defined. minItems: 1 ui: display_type: sortable items: type: object properties: source_field: type: string title: Source field description: The field to copy data from pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference target_field: type: string title: Target field description: The field to copy data into pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference drop_source: type: boolean title: Drop source? description: >- When enabled, the source field is dropped after the data is copied to the target field. Otherwise, it is preserved. default: false overwrite_target: type: boolean title: Overwrite target? description: >- When enabled, any existing data in the target field is overwritten. Otherwise, the target field will be preserved and this mapping will have no effect. default: false required: - source_field - target_field - drop_source - overwrite_target if: properties: drop_source: enum: - true required: - drop_source then: properties: source_field: type: string title: Source field description: The field to copy data from pattern: >- ^(?:metadata|\.|(?:metadata|message)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ minLength: 1 errorMessage: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields). Note that dropping an event's message or timestamp is not allowed required: - source_field required: - mappings - if: properties: type: enum: - metrics-tag-cardinality-limit deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: title: Tag Cardinality Limit (Metrics) description: >- Limits the cardinality of metric events by either dropping events or tags that exceed a specified value limit additionalProperties: false allOf: - if: title: Mode properties: mode: type: string enum: - exact then: title: Value Limit properties: value_limit: type: integer title: Value Limit description: Maximum number of unique values for tags minimum: 1 maximum: 500 required: - value_limit - if: title: Mode properties: mode: type: string enum: - probabilistic then: title: Value Limit properties: value_limit: type: integer title: Value Limit description: Maximum number of unique values for tags minimum: 1 maximum: 5000 required: - value_limit properties: tags: type: array title: Tags description: >- A list of tags to apply cardinality limits. If none are provided, all tags will be considered. items: type: string title: Tag minLength: 1 maxLength: 100 minItems: 0 maxItems: 10 exclude_tags: type: array title: Exclude Tags description: A list of tags to explicitly exclude from cardinality limits. items: type: string title: Tag minLength: 1 maxLength: 100 minItems: 0 maxItems: 10 action: type: string title: Action description: The action to take when a tag's cardinality exceeds the value limit enum: - drop_tag - drop_event default: drop_event value_limit: type: integer title: Value Limit description: Maximum number of unique values for tags mode: type: string title: Mode description: The method to used to reduce tag value cardinality enum: - exact - probabilistic default: exact - if: properties: type: enum: - aggregate deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Aggregate description: >- Aggregates multiple events into a single event using either a tumbling interval window or a sliding interval window. additionalProperties: false properties: group_by: type: array minItems: 1 maxItems: 100 title: Group By Field Path description: >- When aggregating, group events based on matching values from each of these field paths. Supports nesting via dot-notation. items: type: string title: Field description: The dot-notated property path errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ ui: default_hint: - .name - .namespace - .tags evaluate: additionalProperties: type: string title: Evaluate description: How events should be aggregated within an interval. allOf: - if: properties: operation: enum: - CUSTOM then: properties: script: type: string title: Script ui: display_type: code-editor options: language: js description: The JavaScript representation of an event aggregation. minLength: 1 maxLength: 2048 default: > // Perform custom aggregation of the current event into the accumulated event. // The accumulated event is persisted across the entire aggregation window. // The function must return the accumulated event, which will be emitted when // the aggregation window elapses. function aggregateEvent(accum, event, metadata, timestamp, annotations) { return accum } required: - script properties: operation: title: Operation oneOf: - label: add description: >- Adds new values together if they are of the same type. For counters and gauges the values are added together. For sets, this is a union of the elements. For distribution, the samples are concatenated. For histograms, values in the same bucket will be added together. enum: - DEFAULT - label: sum description: >- Values are added together. Works for the GAUGE and COUNT metric types. enum: - SUM - label: minimum description: >- Get the minimum value for a series of events. Works for the GAUGE and COUNT metric types. enum: - MIN - label: maximum description: >- Get the maximum value for a series of events. Works for the GAUGE and COUNT metric types. enum: - MAX - label: average description: >- Values are averaged together. Works for the GAUGE and COUNT metric types. enum: - AVG - label: set intersection description: The aggregate produces a set of unique values. enum: - SET_INTERSECTION - label: distribution concatenation description: The aggregate produces a list of values combined together. enum: - DIST_CONCAT - label: custom description: Use a custom script to perform aggregation. enum: - CUSTOM ui: display_type: list required: - operation window: default: {} additionalProperties: type: integer title: Window Type description: >- Sliding windows can overlap, whereas tumbling windows are disjoint. For example, a tumbling window has a fixed time span and any events that fall within the "interval" will be used in the aggregate. Whereas in a sliding window, the aggregation occurs every "window duration" seconds after an event is encountered. allOf: - if: properties: type: enum: - sliding required: - type then: properties: window_min: type: integer title: Slide Duration (seconds) description: >- The amount of time to wait before creating a new sliding window for future events. minimum: 1 maximum: 200 default: 5 required: - window_min properties: type: type: string title: Type enum: - tumbling - sliding default: tumbling interval: type: integer title: Interval (seconds) description: The interval over which events are aggregated in seconds. default: 5 minimum: 1 maximum: 90000 required: - type - interval event_timestamp: type: string title: Event Timestamp description: >- The path to a field on the event that contains a epoch timestamp value. If an event does not have a timestamp field, events will be associated to the wall clock value when the event is processed. default: timestamp maxLength: 200 errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ conditional: ui: display_type: rules-engine title: Event Flush Condition description: >- Force accumulated event reduction to flush the result when a conditional expression evaluates to true on an inbound event. allOf: - $ref: '#condition-group' required: - evaluate - window - if: properties: type: enum: - event-to-metric deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Event to Metric description: Allows conversion between arbitrary events and a Metric. additionalProperties: false properties: metric_name: type: string title: Metric Name description: The machine name of the metric to emit. minLength: 1 maxLength: 128 pattern: ^[a-zA-Z][a-zA-Z0-9_:]*$ errorMessage: must be a valid metric name metric_kind: title: Kind description: >- The kind of metric to emit, Absolute or Incremental. Absolute metrics represent a complete value, and will generally replace an existing value for the metric in the target destination. Incremental metrics represent an additive value which is aggregated in the target destination to produce a new value. oneOf: - label: Incremental enum: - incremental - label: Absolute enum: - absolute default: incremental ui: display_type: radio-group metric_type: title: Type oneOf: - label: Counter enum: - counter - label: Sum enum: - sum - label: Gauge enum: - gauge default: counter ui: display_type: list value: title: Value description: The value of this metric. additionalProperties: false allOf: - if: properties: value_type: enum: - field then: title: Value properties: value: type: string title: Field value description: The field to use as the value. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference required: - value - if: properties: value_type: enum: - value then: title: Value properties: value: type: number title: Value description: The value. required: - value properties: value_type: title: Value type description: Either a value from an event field, or a new value input. oneOf: - label: Value from event field enum: - field - label: New value enum: - value default: field ui: display_type: radio-group value: type: - number - string required: - value_type namespace: title: Namespace description: The namespace for this metric. additionalProperties: false allOf: - if: properties: value_type: enum: - field then: title: Value properties: value: type: string title: Field value description: The field to use as the value. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference required: - value - if: properties: value_type: enum: - value required: - value_type then: title: Value properties: value: type: string title: Value description: The value. minLength: 1 maxLength: 128 required: - value properties: value_type: title: Value type description: >- Either a value from an event field, a new value input, or none to omit the namespace. oneOf: - label: Value from event field enum: - field - label: New value enum: - value - label: None enum: - none default: none ui: display_type: radio-group value: type: - string - 'null' required: - value_type tags: type: array maxItems: 10 title: Tags description: A set of tags (also called labels) to apply to the metric event. items: type: object title: Tag description: A tag to apply to the metric event. allOf: - if: properties: value_type: enum: - field then: title: Value properties: value: type: string title: Field value description: >- The field to use as the value of this tag. Note that fields with highly-variable values will result in high-cardinality metrics, which may impact storage or cost in downstream destinations. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference fallback: type: string title: Fallback value description: >- The value to use for this tag when the field value is missing. minLength: 1 maxLength: 512 required: - value - if: properties: value_type: enum: - value then: title: Value properties: value: type: string title: Value description: The value for this tag. minLength: 1 maxLength: 128 required: - value properties: name: type: string title: Name description: The tag name pattern: ^[a-zA-Z][a-zA-Z0-9_]*$ minLength: 1 maxLength: 128 errorMessage: must be a valid metric tag name value_type: title: Value type description: Either a value from an event field, or a new value input. oneOf: - label: Value from event field enum: - field - label: New value enum: - value default: field ui: display_type: radio-group value: type: string required: - name - value_type - value additionalProperties: type: string required: - metric_name - metric_kind - metric_type - value - namespace - if: properties: type: enum: - clustering deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Clustering description: Finds clusters of similar values additionalProperties: false properties: cluster_field: type: string title: Field description: Field to cluster values on pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference default: .message similarity_threshold: type: number title: Similarity threshold description: >- A threshold from 0.0 (not similar) to 1.0 (exactly similar) that determines how close two values need to be to be consider as being part of the same cluster default: 0.4 minimum: 0 maximum: 1 max_clusters: type: integer title: Maximum cluster description: Maximum number of clusters kept cached by the transform default: 1000 minimum: 1 maximum: 2000 max_node_depth: type: integer title: Maximum tree depth description: Maximum depth of the prefix tree default: 4 minimum: 2 maximum: 8 max_children: type: integer title: Maximum children default: 100 minimum: 2 maximum: 200 description: Maximum number of children per node of the prefix tree required: - cluster_field - if: properties: type: enum: - redact deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Redact description: Detect and anonymize sensitive data in logs and metrics additionalProperties: false properties: field: type: string title: Field description: >- Detect patterns in the specified field. If the field is empty, the processor will look for patterns across all the fields. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ default: . maxLength: 256 errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference mask_patterns: type: array title: Redact Patterns description: List of PII patterns to detect and mask minItems: 1 maxItems: 20 items: type: object additionalProperties: type: string title: Redact Pattern description: A PII pattern to detect and/or anonymize allOf: - if: properties: action: enum: - Replace required: - action then: properties: replacement: type: string title: Replacement description: >- The text to replace the detected pattern. Example: 000-00-0000 minLength: 1 maxLength: 512 required: - replacement - if: properties: action: enum: - Hash required: - action then: properties: algorithm: title: Algorithm description: >- Hash and encode (base16 or hex) matched PII data using the specified algorithm oneOf: - label: sha2 (sha-512_256) enum: - sha2 - label: sha3 (sha3-512) enum: - sha3 ui: display_type: list required: - algorithm - if: properties: name: enum: - custom required: - name then: properties: custom_options: allOf: - type: object additionalProperties: false properties: pattern: type: string title: Pattern description: The custom pattern minLength: 1 maxLength: 4096 case_sensitive: type: boolean title: Case sensitive description: Perform case sensitive matching default: true multiline: type: boolean title: Multiline mode description: >- ^ and $ match beginning and end of line respectively default: false match_newline: type: boolean title: Dot all mode description: Dot matches newline character default: false crlf_newline: type: boolean title: Carriage return (CRLF) mode description: "When multiline mode is enabled, newline character is interpreted as \r\n" default: false ignore_whitespace: type: boolean title: Verbose mode description: Ignore whitespace and allow comment lines default: false required: - pattern title: Replace Expression description: >- The regular expression used to find replacements. Named capture groups are supported, e.g. `(?P<key>\w+):(?P<value>.+)`. ui: display_type: regex-pattern required: - custom_options properties: name: title: Pattern description: The PII pattern to match oneOf: - label: Credit card number enum: - credit_card - label: Email address enum: - email_address - label: IPv4 address enum: - ipv4_address - label: US or Canada phone number enum: - us_canada_phone_number - label: US social security number enum: - us_social_security_number - label: Custom pattern ui: feature_flag: pipeline-parser-regex enum: - custom ui: display_type: list action: type: string title: Action description: Action to take on the detected PII data enum: - Hash - Replace - Detect Only default: Replace required: - name - action required: - mask_patterns - if: properties: type: enum: - throttle deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Throttle description: Throttle (rate-limit) events passing through this component additionalProperties: false properties: threshold: type: number minimum: 1 maximum: 10000000 title: Limit description: >- The number of events to allow over the time window. If a Key Field is specified, the limit will be applied to events for each unique value of that field. default: 1 window_ms: type: number minimum: 1000 maximum: 2592000000 title: Window description: The time window over which the configured limit is applied ui: display_type: milliseconds-to-human-readable default: 5000 key_field: type: string title: Key Field description: >- When configured, the value from this field will be used to group events into separate buckets for throttling. If left blank, or if the event does not contain a value in this field, the event will be throttled as part of the default bucket. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference exclude: allOf: - $ref: '#condition-group' title: Exclude description: When configured, events matching this condition will not be throttled. ui: display_type: rules-engine required: - threshold - window_ms - if: properties: type: enum: - set-timestamp deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Set Timestamp description: >- Parse a list of fields using a chosen time format. First match will override the default timestamp of the event. additionalProperties: false properties: parsers: type: array title: List of Fields and Parsers description: >- The list of fields and parsers to use in order of priority, short-circuiting on the first successful match. ui: display_type: multi-output-map options: sortable: false items: type: object properties: field: type: string title: Field description: The field whose value will be parsed by the chosen date pattern pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ default: . maxLength: 512 errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference options: $ref: '#timestamp-parser-options' required: - field - options minItems: 1 required: - parsers - if: properties: type: enum: - data-profiler deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Data Profiler description: >- Profile the data sent through your pipeline. The system will generate annotations such as event type and total bytes to be used downstream. It will also normalize the app/host/level/line fields before sending the event downstream. A profile of the data including total, events, event size, log composition, and message templates will be generated as well. additionalProperties: false properties: app_fields: type: array title: App Path(s) description: A list of paths to look for the app value minItems: 1 maxItems: 10 uniqueItems: true items: type: string title: App description: The path to the app maxLength: 512 pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference ui: default_hint: - .app - .application - .container host_fields: type: array title: Host Path(s) description: A list of paths to look for the host value minItems: 1 maxItems: 10 uniqueItems: true items: type: string title: Host description: The path to the host maxLength: 512 pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference ui: default_hint: - .host - .hostname level_fields: type: array title: Level Path(s) description: A list of paths to look for the log level minItems: 1 maxItems: 10 uniqueItems: true items: type: string title: Level description: The path to the level maxLength: 512 pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference ui: default_hint: - .level - .log_level line_fields: type: array title: Line Path(s) description: A list of paths to look for the event line minItems: 1 maxItems: 10 uniqueItems: true items: type: string title: Line description: The path to the line maxLength: 512 pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference ui: default_hint: - .line - .message label_fields: type: array title: Label Path(s) description: A list of paths to look for the event labels maxItems: 10 uniqueItems: true items: type: string title: Label description: The path to the labels maxLength: 512 pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference ui: default_hint: - .labels field_summary: type: boolean default: false title: Field Summary description: >- If enabled the profiler will attempt to summarize the values in fields of events passing through the node. ui: display_type: hidden ff_default_value: pipeline-node-transform-field-summary required: - app_fields - host_fields - level_fields - line_fields - if: properties: type: enum: - replace deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: title: Replace description: Replace parts of a string using a regular expression or static string additionalProperties: type: object allOf: - if: properties: replacement_type: enum: - regex required: - replacement_type then: properties: options: allOf: - type: object additionalProperties: false properties: pattern: type: string minLength: 1 maxLength: 4096 case_sensitive: type: boolean title: Case sensitive description: Perform case sensitive matching default: true multiline: type: boolean title: Multiline mode description: ^ and $ match beginning and end of line respectively default: false match_newline: type: boolean title: Dot all mode description: Dot matches newline character default: false crlf_newline: type: boolean title: Carriage return (CRLF) mode description: "When multiline mode is enabled, newline character is interpreted as \r\n" default: false ignore_whitespace: type: boolean title: Verbose mode description: Ignore whitespace and allow comment lines default: false required: - pattern title: Replace Expression description: >- The regular expression used to find replacements. Named capture groups are supported, e.g. `(?P<key>\w+):(?P<value>.+)`. ui: display_type: regex-pattern replace_with: type: string minLength: 1 maxLength: 4096 title: Replacement String description: >- The replacement string can be static or optionally contain capture group references. Capture groups can be positional or named. For example, to switch the position of two strings in the source string `onetwo`, the pattern could be `(?Pone)(?P.+)`, and the replacement would be `$two$one`. To do the same thing with positional capture groups, a pattern of `(\w{3})(\w{3})`, and a replacement of `$2$1` would have the same result.

To render a literal dollar sign `$` in the replacement, escape it with `$$`. So, if you had a capture group named `price`, to prepend a dollar sign to the replacement, you would use `$$$price`. required: - options - replace_with - if: properties: replacement_type: enum: - string required: - replacement_type then: properties: string_pattern: type: string title: String Pattern minLength: 1 maxLength: 4096 description: >- A static, case-sensitive string, which will be searched and replaced. replace_with: type: string minLength: 1 maxLength: 4096 title: Replacement String description: The static string used in each replacement. required: - string_pattern - replace_with properties: source_field: type: string title: Source Field description: The field that contains a string for replacement to operate on. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ default: . errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference max_times: type: number title: Maximum Replacements description: >- The maximum number of replacements to be made. If not specified, all occurrences will be replaced. minimum: 1 replacement_type: title: Replacement Type description: Specifies how the source string will be searched for replacement. ui: display_type: list errorMessage: must be one of the available pattern types oneOf: - label: String description: Replace a specified string one or more times. enum: - string - label: Regular Expression description: Replace using a regular expression. enum: - regex required: - source_field - replacement_type - if: properties: type: enum: - trace-sampling deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: title: Trace Sampling description: >- Performs sampling of traces per a given rate using the Trace ID as the unique identifier. If a trace is rate selected, all spans of the trace are forwarded. If the Trace ID cannot be found, the event is dropped. allOf: - if: properties: sample_type: enum: - head then: properties: rate: type: integer title: Rate description: >- The rate at which events will be forwarded, expressed as 1/N. For example, `rate = 10` means 1 out of every 10 unique traces will be forwarded and all others will be dropped default: 10 minimum: 2 maximum: 100000 parent_span_id_field: type: 'null' conditionals: type: 'null' required: - rate - if: properties: sample_type: enum: - tail then: properties: parent_span_id_field: type: string default: .parent_span_id title: Parent Span ID description: The field to use as the parent span identifier pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference conditionals: type: array minItems: 1 title: Conditionals description: >- A list of complete conditionals, including expressions and a sample rate to maintain items: type: object additionalProperties: false title: Conditional description: A list of conditions and a sample rate to maintain properties: rate: type: integer title: Rate description: >- The rate at which events will be forwarded, expressed as 1/N. For example, `rate = 10` means 1 out of every 10 unique traces will be forwarded and all others will be dropped. A `rate = 1` means keep all traces that match the condition. minimum: 1 maximum: 100000 default: 10 conditional: allOf: - $ref: '#condition-group' title: Conditional description: >- A group of expressions joined together by an and/or operator. ui: display_type: rules-engine _output_name: type: string title: '[INTERNAL]' description: >- Internal-only field that holds a static id for the output_name. This way, labels for outputs can be updated independent of the id. required: - rate - conditional required: - parent_span_id_field - conditionals properties: sample_type: type: string description: The type of trace sampling to apply. enum: - head title: Sample Type default: head trace_id_field: type: string default: .trace_id title: Trace ID description: The field to use as the identifier of a trace. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference required: - sample_type - trace_id_field properties: type: type: string enum: - filter - sample - drop-fields - parse - parse-sequentially - route - stringify - compact-fields - flatten-fields - dedupe - encrypt-fields - decrypt-fields - js-script - unroll - vrl - reduce - map-fields - metrics-tag-cardinality-limit - aggregate - event-to-metric - clustering - redact - throttle - set-timestamp - data-profiler - replace - trace-sampling deploy_type: type: string enum: - saas - local default: saas title: oneOf: - type: string minLength: 1 maxLength: 512 pattern: ^[^<>"]+$ - type: 'null' errorMessage: invalid title name description: oneOf: - type: string minLength: 1 pattern: ^[^<>"]+$ - type: 'null' errorMessage: invalid description user_config: type: object inputs: type: array items: type: string description: >- A list of inputs for the given transform. Typically this is empty when creating a new transform via the UI, but is leveraged by the terraform provider when managing pipeline state. required: - type - user_config required: true parameters: - schema: type: string format: uuid in: path name: pipeline_id required: true responses: '200': description: Default Response /v3/pipeline/{pipeline_id}/transform-group/{pipeline_transform_group_id}: delete: tags: - Pipeline Nodes parameters: - schema: type: string format: uuid in: path name: pipeline_id required: true - schema: type: string format: uuid in: path name: pipeline_transform_group_id required: true responses: '200': description: Default Response get: tags: - Pipeline Nodes parameters: - schema: type: string format: uuid in: path name: pipeline_id required: true - schema: type: string format: uuid in: path name: pipeline_transform_group_id required: true responses: '200': description: Default Response put: tags: - Pipeline Nodes requestBody: content: application/json: schema: type: object properties: transform_group_revision_id: oneOf: - type: string format: uuid - enum: - latest inputs: type: array items: type: string description: >- A list of inputs for the given group. Typically this is empty when creating a new transform via the UI, but is leveraged by the terraform provider when managing pipeline state. required: - transform_group_revision_id required: true parameters: - schema: type: string format: uuid in: path name: pipeline_id required: true - schema: type: string format: uuid in: path name: pipeline_transform_group_id required: true responses: '200': description: Default Response /v3/pipeline/{pipeline_id}/transform-group: post: tags: - Pipeline Nodes requestBody: content: application/json: schema: type: object properties: transform_group_id: type: string format: uuid deploy_type: type: string enum: - saas - local default: saas inputs: type: array items: type: string description: >- A list of inputs for the given group. Typically this is empty when creating a new transform via the UI, but is leveraged by the terraform provider when managing pipeline state. required: - transform_group_id required: true parameters: - schema: type: string format: uuid in: path name: pipeline_id required: true responses: '200': description: Default Response /v3/transform-group/{transform_group_id}: delete: tags: - Processor Groups description: Delete a Processor Group parameters: - schema: type: string format: uuid in: path name: transform_group_id required: true responses: '200': description: Default Response get: tags: - Processor Groups description: Get a Processor Group parameters: - schema: type: string format: uuid in: path name: transform_group_id required: true responses: '200': description: Default Response put: tags: - Processor Groups description: Update a Processor Group requestBody: content: application/json: schema: type: object additionalProperties: false properties: title: type: string title: Title description: The title of the Processor Group pattern: ^[^<>"]+$ errorMessage: invalid title maxLength: 512 description: oneOf: - type: string minLength: 1 pattern: ^[^<>"]+$ - type: 'null' title: Description description: A description of this Processor Group's behavior or purpose. errorMessage: invalid description required: - title required: true parameters: - schema: type: string format: uuid in: path name: transform_group_id required: true responses: '200': description: Default Response /v3/transform-group: get: tags: - Processor Groups description: List all Processor Groups for the account responses: '200': description: Default Response post: tags: - Processor Groups description: Create a new Processor Group requestBody: content: application/json: schema: type: object additionalProperties: false properties: title: type: string title: Title description: The title of the Processor Group pattern: ^[^<>"]+$ errorMessage: invalid title maxLength: 512 description: oneOf: - type: string minLength: 1 pattern: ^[^<>"]+$ - type: 'null' title: Description description: A description of this Processor Group's behavior or purpose. errorMessage: invalid description deploy_type: type: string title: Deploy Type description: The type of deployment this Processor Group will be used in. enum: - saas - local default: saas required: - title required: true responses: '200': description: Default Response /v3/transform-group/{transform_group_id}/publish: post: tags: - Processor Groups description: Publish changes to a Processor Group parameters: - schema: type: boolean default: false in: query name: allow_unconnected_edges required: false - schema: type: string format: uuid in: path name: transform_group_id required: true responses: '200': description: Default Response /v3/transform-group/from-pipeline: post: tags: - Processor Groups description: Create a new Processor Group from a set of Pipeline components requestBody: content: application/json: schema: type: object additionalProperties: false properties: title: type: string title: Title description: The title of the Processor Group pattern: ^[^<>"]+$ errorMessage: invalid title maxLength: 512 description: oneOf: - type: string minLength: 1 pattern: ^[^<>"]+$ - type: 'null' title: Description description: A description of this Processor Group's behavior or purpose. errorMessage: invalid description deploy_type: type: string title: Deploy Type description: The type of deployment this Processor Group will be used in. enum: - saas - local default: saas pipeline_id: type: string format: uuid title: Pipeline ID description: The Pipeline ID to copy from. pipeline_transform_ids: type: array items: type: string format: uuid minItems: 1 title: Pipeline Transform IDs description: A list of IDs of transforms to copy into this new group. required: - title - pipeline_id - pipeline_transform_ids required: true responses: '200': description: Default Response /v3/transform-group/{transform_group_id}/revision/{transform_group_revision_lookup}: get: tags: - Processor Groups parameters: - schema: type: string format: uuid in: path name: transform_group_id required: true - schema: oneOf: - type: string format: uuid - enum: - latest in: path name: transform_group_revision_lookup required: true responses: '200': description: Default Response /v3/transform-group/{transform_group_id}/revision: get: tags: - Processor Groups parameters: - schema: type: integer default: 200 minimum: 1 maximum: 1000 in: query name: limit required: false description: The maximum number of revisions to return - schema: type: string format: uuid in: path name: transform_group_id required: true responses: '200': description: Default Response /v3/transform-group/{transform_group_id}/transform/{transform_group_transform_id}: delete: tags: - Processor Groups description: Delete a Processor from a Processor Group parameters: - schema: type: string format: uuid in: path name: transform_group_id required: true - schema: type: string format: uuid in: path name: transform_group_transform_id required: true responses: '200': description: Default Response get: tags: - Processor Groups description: Get a Processor from a Processor Group parameters: - schema: type: string format: uuid in: path name: transform_group_id required: true - schema: type: string format: uuid in: path name: transform_group_transform_id required: true responses: '200': description: Default Response put: tags: - Processor Groups description: Update a Processor in a Processor Group requestBody: content: application/json: schema: type: object allOf: - if: properties: type: enum: - filter deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Filter description: Define condition(s) to include or exclude events from the pipeline additionalProperties: false properties: action: title: Action description: How to handle events matching this criteria oneOf: - label: Allow events matching this criteria enum: - allow - label: Drop events matching this criteria enum: - drop default: allow ui: display_type: radio-group conditional: allOf: - $ref: '#condition-group' title: Conditional description: A group of expressions joined together by an and/or operator. ui: display_type: rules-engine required: - action - conditional - if: properties: type: enum: - sample deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Sample description: >- Sample data at a given rate, retaining only a subset of data events for further processing. additionalProperties: false properties: rate: type: integer title: Rate description: >- The rate at which events will be forwarded, expressed as 1/N. For example, `rate = 10` means 1 out of every 10 events will be forwarded and the rest will be dropped minimum: 2 maximum: 10000 default: 10 always_include: title: Always Include description: >- Certain events may always want to be included (not sampled), regardless of the sample rate. Events matching this criteria will always show up in the results. additionalProperties: type: - string - number dependencies: field: - str_operator str_operator: - field allOf: - if: title: Operator properties: str_operator: type: string enum: - greater - greater_or_equal - less - less_or_equal ui: display_type: list-operators title: Operator description: The comparison operator required: - str_operator then: title: Value properties: value: errorMessage: pattern: Value must be a number type: - string - number title: Value description: The value used in the expression, ie the "operand" minLength: 1 pattern: ^\s*-?(\d*\.)?\d+\s*$ required: - value - if: title: Operator properties: str_operator: type: string enum: - greater - greater_or_equal - less - less_or_equal - equal - contains - is_ip_in_cidr_range - ends_with - starts_with - drain_matches - contains_token_term - contains_token_prefix - percent_change_greater - percent_change_greater_or_equal - percent_change_less - percent_change_less_or_equal - value_change_greater - value_change_greater_or_equal - value_change_less - value_change_less_or_equal ui: display_type: list-operators title: Operator description: The comparison operator required: - str_operator then: title: Value properties: value: type: - string - number title: Value description: The value used in the expression, ie the "operand" minLength: 1 required: - value - if: title: Operator properties: str_operator: type: string enum: - contains - equal - ends_with - starts_with - drain_matches - contains_token_term - contains_token_prefix ui: display_type: list-operators title: Operator required: - str_operator then: title: Case sensitive properties: case_sensitive: type: boolean title: Case sensitive description: Perform case sensitive comparison? default: true properties: field: type: string title: Field description: The JSON field to use in a condition to always include in sampling pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference str_operator: type: string enum: - greater - greater_or_equal - less - less_or_equal - equal - contains - is_ip_in_cidr_range - is_metric - is_array - is_boolean - is_empty - is_null - is_number - is_object - is_string - exists - ends_with - starts_with - contains_token_term - contains_token_prefix - regex_match ui: display_type: list-operators title: Operator description: The comparison operator negate: type: boolean default: false title: Negate description: Negate the comparison expression required: - rate - if: properties: type: enum: - drop-fields deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Remove Fields description: Remove fields additionalProperties: false properties: fields: type: array title: Fields description: A list of fields to be removed items: type: string title: Field pattern: >- ^(?:metadata|\.|(?:metadata|message)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ minLength: 1 errorMessage: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields). Note that dropping an event's message or timestamp is not allowed minItems: 1 required: - fields - if: properties: type: enum: - parse deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: title: Parse description: Parse a specified field using the chosen parser additionalProperties: type: object errorMessage: properties: target_field: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) allOf: - if: properties: parser: enum: - parse_key_value then: properties: options: $ref: '#key-value-parser-options' - if: properties: parser: enum: - parse_timestamp then: properties: options: $ref: '#timestamp-parser-options' required: - options - if: properties: parser: enum: - parse_grok then: properties: options: $ref: '#grok-pattern-parser-options' required: - options - if: properties: parser: enum: - parse_regex then: properties: options: allOf: - $ref: '#regex-parser-options' ui: display_type: regex-pattern required: - options - if: properties: parser: enum: - parse_apache_log then: properties: options: $ref: '#apache-parser-options' required: - options - if: properties: parser: enum: - parse_nginx_log then: properties: options: $ref: '#nginx-parser-options' required: - options - if: properties: parser: enum: - parse_cef then: properties: options: $ref: '#cef-parser-options' - if: properties: parser: enum: - parse_csv then: properties: options: $ref: '#csv-parser-options' properties: field: type: string title: Field description: The JSON field whose value should be parsed. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ default: . errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference target_field: oneOf: - type: 'null' - type: string pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ maxLength: 512 errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference title: Target field description: >- The field into which the parsed value should be inserted. Leave blank to insert the parsed data into the original field. default: null parser: oneOf: - label: AWS Cloudfront Access Log group: Line Parsers ui: {} enum: - aws_cloudfront_access_log - label: AWS Cloudwatch Log group: Line Parsers ui: {} enum: - parse_aws_cloudwatch_log_subscription_message - label: AWS Load Balancer Log group: Line Parsers ui: {} enum: - parse_aws_alb_log - label: AWS S3 Access Log group: Line Parsers ui: {} enum: - aws_s3_access_log - label: Apache Log group: Line Parsers ui: {} enum: - parse_apache_log - label: CSV group: Line Parsers ui: {} enum: - parse_csv - label: Common Event Format (CEF) Log group: Line Parsers ui: {} enum: - parse_cef - label: Common Log group: Line Parsers ui: {} enum: - parse_common_log - label: Elasticsearch Log group: Line Parsers ui: {} enum: - elasticsearch_log - label: Golang Log group: Line Parsers ui: {} enum: - golang_log - label: Google Log group: Line Parsers ui: {} enum: - parse_glog - label: Grok Pattern group: Line Parsers ui: {} enum: - parse_grok - label: Integer group: Field Parsers ui: {} enum: - parse_int - label: JSON group: Line Parsers ui: {} enum: - parse_json - label: Key/Value group: Line Parsers ui: {} enum: - parse_key_value - label: Kubernetes Log group: Line Parsers ui: {} enum: - parse_klog - label: Linux Authorization group: Line Parsers ui: {} enum: - parse_linux_authorization - label: Mongo Log group: Line Parsers ui: {} enum: - mongo_log - label: Mongo3 Log group: Line Parsers ui: {} enum: - mongo3_log - label: Nginx Log group: Line Parsers ui: {} enum: - parse_nginx_log - label: PSQL Log group: Line Parsers ui: {} enum: - psql_log - label: Query String group: Field Parsers ui: {} enum: - parse_query_string - label: Regular Expression group: Line Parsers ui: feature_flag: pipeline-parser-regex enum: - parse_regex - label: Syslog group: Line Parsers ui: {} enum: - parse_syslog - label: Timestamp group: Field Parsers ui: {} enum: - parse_timestamp - label: Tokens group: Field Parsers ui: {} enum: - parse_tokens - label: URL group: Field Parsers ui: {} enum: - parse_url - label: User Agent group: Field Parsers ui: {} enum: - parse_user_agent - label: VPC Flow Log group: Line Parsers ui: {} description: Parse VPC logs using default format for fields from v2 to v4 enum: - parse_aws_vpc_flow_log ui: display_type: list errorMessage: must be one of allowed parsers title: Parser description: The kind of parser to use against the input value from "field". required: - field - parser - if: properties: type: enum: - parse-sequentially deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Parse Sequentially description: >- Parse fields using one or more parsers in order, sending out the event on the first successful match. additionalProperties: false errorMessage: properties: target_field: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) properties: field: type: string title: Field description: The field whose value should be parsed. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ default: . maxLength: 512 errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference target_field: oneOf: - type: 'null' - type: string pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ maxLength: 512 errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference title: Target field description: >- The field into which the parsed value should be inserted. Leave blank to insert the parsed data into the original field. default: null parsers: type: array title: List of parsers description: >- The list of parsers to use in order against the input valuefrom "field", short-circuiting on the first successful match. items: type: object allOf: - if: properties: parser: enum: - parse_key_value then: properties: options: $ref: '#key-value-parser-options' - if: properties: parser: enum: - parse_timestamp then: properties: options: $ref: '#timestamp-parser-options' required: - options - if: properties: parser: enum: - parse_grok then: properties: options: $ref: '#grok-pattern-parser-options' required: - options - if: properties: parser: enum: - parse_regex then: properties: options: allOf: - $ref: '#regex-parser-options' ui: display_type: regex-pattern required: - options - if: properties: parser: enum: - parse_apache_log then: properties: options: $ref: '#apache-parser-options' required: - options - if: properties: parser: enum: - parse_nginx_log then: properties: options: $ref: '#nginx-parser-options' required: - options - if: properties: parser: enum: - parse_cef then: properties: options: $ref: '#cef-parser-options' - if: properties: parser: enum: - parse_csv then: properties: options: $ref: '#csv-parser-options' properties: label: oneOf: - type: 'null' - type: string maxLength: 20 title: Title description: >- An arbitrary name you choose to identify the results of this processor for use when connecting them to other components. parser: oneOf: - label: AWS Cloudfront Access Log group: Line Parsers ui: {} enum: - aws_cloudfront_access_log - label: AWS Cloudwatch Log group: Line Parsers ui: {} enum: - parse_aws_cloudwatch_log_subscription_message - label: AWS Load Balancer Log group: Line Parsers ui: {} enum: - parse_aws_alb_log - label: AWS S3 Access Log group: Line Parsers ui: {} enum: - aws_s3_access_log - label: Apache Log group: Line Parsers ui: {} enum: - parse_apache_log - label: CSV group: Line Parsers ui: {} enum: - parse_csv - label: Common Event Format (CEF) Log group: Line Parsers ui: {} enum: - parse_cef - label: Common Log group: Line Parsers ui: {} enum: - parse_common_log - label: Elasticsearch Log group: Line Parsers ui: {} enum: - elasticsearch_log - label: Golang Log group: Line Parsers ui: {} enum: - golang_log - label: Google Log group: Line Parsers ui: {} enum: - parse_glog - label: Grok Pattern group: Line Parsers ui: {} enum: - parse_grok - label: Integer group: Field Parsers ui: {} enum: - parse_int - label: JSON group: Line Parsers ui: {} enum: - parse_json - label: Key/Value group: Line Parsers ui: {} enum: - parse_key_value - label: Kubernetes Log group: Line Parsers ui: {} enum: - parse_klog - label: Linux Authorization group: Line Parsers ui: {} enum: - parse_linux_authorization - label: Mongo Log group: Line Parsers ui: {} enum: - mongo_log - label: Mongo3 Log group: Line Parsers ui: {} enum: - mongo3_log - label: Nginx Log group: Line Parsers ui: {} enum: - parse_nginx_log - label: PSQL Log group: Line Parsers ui: {} enum: - psql_log - label: Query String group: Field Parsers ui: {} enum: - parse_query_string - label: Regular Expression group: Line Parsers ui: feature_flag: pipeline-parser-regex enum: - parse_regex - label: Syslog group: Line Parsers ui: {} enum: - parse_syslog - label: Timestamp group: Field Parsers ui: {} enum: - parse_timestamp - label: Tokens group: Field Parsers ui: {} enum: - parse_tokens - label: URL group: Field Parsers ui: {} enum: - parse_url - label: User Agent group: Field Parsers ui: {} enum: - parse_user_agent - label: VPC Flow Log group: Line Parsers ui: {} description: Parse VPC logs using default format for fields from v2 to v4 enum: - parse_aws_vpc_flow_log ui: display_type: list errorMessage: must be one of allowed parsers title: Parser description: The kind of parser to use against the input value from "field". _output_name: type: string title: '[INTERNAL]' description: >- Internal-only field that holds a static id for the output_name. This way, labels for outputs can be updated independent of the id. default: '' required: - parser - _output_name additionalProperties: type: object minItems: 1 ui: display_type: multi-output-map unique_parser_and_options: true errorMessage: uniqueItemProperties: parser can not be defined more than once required: - field - parsers - if: properties: type: enum: - route deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Route description: >- Route data based on whether it matches (or doesn't match) logical comparisons additionalProperties: false properties: conditionals: type: array minItems: 1 title: Conditionals description: A list of complete conditionals, including expressions and label items: type: object additionalProperties: false title: Conditional description: A list of conditions and a label properties: conditional: $ref: '#condition-group' label: type: string minLength: 1 maxLength: 255 title: Label description: >- An arbitrary name you choose to identify the results of this processor for use when connecting them to other components. _output_name: type: string title: '[INTERNAL]' description: >- Internal-only field that holds a static id for the output_name. This way, labels for outputs can be updated independent of the id. default: '' required: - conditional - label - _output_name required: - conditionals - if: properties: type: enum: - stringify deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Stringify description: Stringify messages as JSON additionalProperties: false - if: properties: type: enum: - compact-fields deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Compact Fields description: Remove empty values from a list of fields additionalProperties: false properties: fields: type: array title: Fields description: A list of fields to remove empty values from items: type: string title: Field description: Field to remove empty value from pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference minItems: 1 options: type: object additionalProperties: false title: Compact Options description: Additional options for removing empty values from fields properties: compact_array: type: boolean title: Compact Array description: Remove empty arrays from a field default: true compact_object: type: boolean title: Compact Object description: Remove empty objects from a field default: true required: - fields - if: properties: type: enum: - flatten-fields deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Flatten Fields description: >- Flattens the object or array value of a field into a single level representation additionalProperties: false properties: fields: type: array title: Fields description: >- Define specific nested objects you wish to flatten. This will not flatten the entire object, just the specified fields. Leave blank to flatten the entire object items: type: string title: Field description: Field to flatten pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference options: type: object additionalProperties: false title: Flatten Options description: Additional options for flattening fields properties: delimiter: type: string default: _ title: Delimiter description: >- When flattening, the nested field labels will be appended to the higher level field names along with a delimiter to allow understanding of the original structure. The default is an underscore (_) to ensure no collisions with the . notation used for field identifiers minLength: 1 maxLength: 1 - if: properties: type: enum: - dedupe deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Dedupe description: Dedupe (remove duplicates) from the data stream additionalProperties: false properties: number_of_events: type: number description: Number of events to compare across title: Number of Events minimum: 2 maximum: 5000 default: 5000 comparison_type: type: string enum: - Ignore - Match description: >- Match: Only compares across the fields which are specified by the user.Ignore: Compares everything but the fields specified by the user. title: Comparison Type default: Match fields: type: array title: Fields description: A list of fields on which to base deduping items: type: string title: Field pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference minItems: 1 required: - number_of_events - comparison_type - fields - if: properties: type: enum: - encrypt-fields deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: title: Encrypt Field description: Encrypts the value of the provided field additionalProperties: false allOf: - if: properties: algorithm: enum: - AES-256-CFB required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CFB required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CFB required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-OFB required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-OFB required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-OFB required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-CTR required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CTR required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CTR required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-CBC-PKCS7 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CBC-PKCS7 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CBC-PKCS7 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-CBC-ANSIX923 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CBC-ANSIX923 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CBC-ANSIX923 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-CBC-ISO7816 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CBC-ISO7816 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CBC-ISO7816 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-CBC-ISO10126 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CBC-ISO10126 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CBC-ISO10126 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 16 maxLength: 16 required: - key properties: field: type: string title: Field description: >- Field to encrypt. The value of the field must be a primitive (string, number, boolean) pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference algorithm: oneOf: - label: AES-256-CFB (key = 32 characters, iv = 16 characters) description: AES-256-CFB (key = 32 characters, iv = 16 characters) enum: - AES-256-CFB - label: AES-192-CFB (key = 24 characters, iv = 16 characters) description: AES-192-CFB (key = 24 characters, iv = 16 characters) enum: - AES-192-CFB - label: AES-128-CFB (key = 16 characters, iv = 16 characters) description: AES-128-CFB (key = 16 characters, iv = 16 characters) enum: - AES-128-CFB - label: AES-256-OFB (key = 32 characters, iv = 16 characters) description: AES-256-OFB (key = 32 characters, iv = 16 characters) enum: - AES-256-OFB - label: AES-192-OFB (key = 24 characters, iv = 16 characters) description: AES-192-OFB (key = 24 characters, iv = 16 characters) enum: - AES-192-OFB - label: AES-128-OFB (key = 16 characters, iv = 16 characters) description: AES-128-OFB (key = 16 characters, iv = 16 characters) enum: - AES-128-OFB - label: AES-256-CTR (key = 32 characters, iv = 16 characters) description: AES-256-CTR (key = 32 characters, iv = 16 characters) enum: - AES-256-CTR - label: AES-192-CTR (key = 24 characters, iv = 16 characters) description: AES-192-CTR (key = 24 characters, iv = 16 characters) enum: - AES-192-CTR - label: AES-128-CTR (key = 16 characters, iv = 16 characters) description: AES-128-CTR (key = 16 characters, iv = 16 characters) enum: - AES-128-CTR - label: AES-256-CBC-PKCS7 (key = 32 characters, iv = 16 characters) description: AES-256-CBC-PKCS7 (key = 32 characters, iv = 16 characters) enum: - AES-256-CBC-PKCS7 - label: AES-192-CBC-PKCS7 (key = 24 characters, iv = 16 characters) description: AES-192-CBC-PKCS7 (key = 24 characters, iv = 16 characters) enum: - AES-192-CBC-PKCS7 - label: AES-128-CBC-PKCS7 (key = 16 characters, iv = 16 characters) description: AES-128-CBC-PKCS7 (key = 16 characters, iv = 16 characters) enum: - AES-128-CBC-PKCS7 - label: AES-256-CBC-ANSIX923 (key = 32 characters, iv = 16 characters) description: AES-256-CBC-ANSIX923 (key = 32 characters, iv = 16 characters) enum: - AES-256-CBC-ANSIX923 - label: AES-192-CBC-ANSIX923 (key = 24 characters, iv = 16 characters) description: AES-192-CBC-ANSIX923 (key = 24 characters, iv = 16 characters) enum: - AES-192-CBC-ANSIX923 - label: AES-128-CBC-ANSIX923 (key = 16 characters, iv = 16 characters) description: AES-128-CBC-ANSIX923 (key = 16 characters, iv = 16 characters) enum: - AES-128-CBC-ANSIX923 - label: AES-256-CBC-ISO7816 (key = 32 characters, iv = 16 characters) description: AES-256-CBC-ISO7816 (key = 32 characters, iv = 16 characters) enum: - AES-256-CBC-ISO7816 - label: AES-192-CBC-ISO7816 (key = 24 characters, iv = 16 characters) description: AES-192-CBC-ISO7816 (key = 24 characters, iv = 16 characters) enum: - AES-192-CBC-ISO7816 - label: AES-128-CBC-ISO7816 (key = 16 characters, iv = 16 characters) description: AES-128-CBC-ISO7816 (key = 16 characters, iv = 16 characters) enum: - AES-128-CBC-ISO7816 - label: AES-256-CBC-ISO10126 (key = 32 characters, iv = 16 characters) description: AES-256-CBC-ISO10126 (key = 32 characters, iv = 16 characters) enum: - AES-256-CBC-ISO10126 - label: AES-192-CBC-ISO10126 (key = 24 characters, iv = 16 characters) description: AES-192-CBC-ISO10126 (key = 24 characters, iv = 16 characters) enum: - AES-192-CBC-ISO10126 - label: AES-128-CBC-ISO10126 (key = 16 characters, iv = 16 characters) description: AES-128-CBC-ISO10126 (key = 16 characters, iv = 16 characters) enum: - AES-128-CBC-ISO10126 ui: display_type: list title: Encryption algorithm description: The encryption algorithm to use key: type: string title: Encryption key description: The encryption key minLength: 16 maxLength: 32 iv_field: type: string title: Initialization vector (IV) field description: >- The field in which to store the generated initialization vector, IV. Each encrypted value will have a unique IV pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference encode_raw_bytes: type: boolean title: Encode encrypted field and IV as Base 64 text description: >- Encode the encrypted value and generated initialization vector as Base64 text default: true required: - field - algorithm - key - iv_field - if: properties: type: enum: - decrypt-fields deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: title: Decrypt Field additionalProperties: false description: Decrypts the value of the provided field allOf: - if: properties: algorithm: enum: - AES-256-CFB required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CFB required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CFB required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-OFB required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-OFB required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-OFB required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-CTR required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CTR required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CTR required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-CBC-PKCS7 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CBC-PKCS7 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CBC-PKCS7 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-CBC-ANSIX923 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CBC-ANSIX923 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CBC-ANSIX923 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-CBC-ISO7816 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CBC-ISO7816 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CBC-ISO7816 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-CBC-ISO10126 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CBC-ISO10126 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CBC-ISO10126 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 16 maxLength: 16 required: - key properties: field: type: string title: Field description: Field to decrypt. The value of the field must be a string pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference algorithm: oneOf: - label: AES-256-CFB (key = 32 characters, iv = 16 characters) description: AES-256-CFB (key = 32 characters, iv = 16 characters) enum: - AES-256-CFB - label: AES-192-CFB (key = 24 characters, iv = 16 characters) description: AES-192-CFB (key = 24 characters, iv = 16 characters) enum: - AES-192-CFB - label: AES-128-CFB (key = 16 characters, iv = 16 characters) description: AES-128-CFB (key = 16 characters, iv = 16 characters) enum: - AES-128-CFB - label: AES-256-OFB (key = 32 characters, iv = 16 characters) description: AES-256-OFB (key = 32 characters, iv = 16 characters) enum: - AES-256-OFB - label: AES-192-OFB (key = 24 characters, iv = 16 characters) description: AES-192-OFB (key = 24 characters, iv = 16 characters) enum: - AES-192-OFB - label: AES-128-OFB (key = 16 characters, iv = 16 characters) description: AES-128-OFB (key = 16 characters, iv = 16 characters) enum: - AES-128-OFB - label: AES-256-CTR (key = 32 characters, iv = 16 characters) description: AES-256-CTR (key = 32 characters, iv = 16 characters) enum: - AES-256-CTR - label: AES-192-CTR (key = 24 characters, iv = 16 characters) description: AES-192-CTR (key = 24 characters, iv = 16 characters) enum: - AES-192-CTR - label: AES-128-CTR (key = 16 characters, iv = 16 characters) description: AES-128-CTR (key = 16 characters, iv = 16 characters) enum: - AES-128-CTR - label: AES-256-CBC-PKCS7 (key = 32 characters, iv = 16 characters) description: AES-256-CBC-PKCS7 (key = 32 characters, iv = 16 characters) enum: - AES-256-CBC-PKCS7 - label: AES-192-CBC-PKCS7 (key = 24 characters, iv = 16 characters) description: AES-192-CBC-PKCS7 (key = 24 characters, iv = 16 characters) enum: - AES-192-CBC-PKCS7 - label: AES-128-CBC-PKCS7 (key = 16 characters, iv = 16 characters) description: AES-128-CBC-PKCS7 (key = 16 characters, iv = 16 characters) enum: - AES-128-CBC-PKCS7 - label: AES-256-CBC-ANSIX923 (key = 32 characters, iv = 16 characters) description: AES-256-CBC-ANSIX923 (key = 32 characters, iv = 16 characters) enum: - AES-256-CBC-ANSIX923 - label: AES-192-CBC-ANSIX923 (key = 24 characters, iv = 16 characters) description: AES-192-CBC-ANSIX923 (key = 24 characters, iv = 16 characters) enum: - AES-192-CBC-ANSIX923 - label: AES-128-CBC-ANSIX923 (key = 16 characters, iv = 16 characters) description: AES-128-CBC-ANSIX923 (key = 16 characters, iv = 16 characters) enum: - AES-128-CBC-ANSIX923 - label: AES-256-CBC-ISO7816 (key = 32 characters, iv = 16 characters) description: AES-256-CBC-ISO7816 (key = 32 characters, iv = 16 characters) enum: - AES-256-CBC-ISO7816 - label: AES-192-CBC-ISO7816 (key = 24 characters, iv = 16 characters) description: AES-192-CBC-ISO7816 (key = 24 characters, iv = 16 characters) enum: - AES-192-CBC-ISO7816 - label: AES-128-CBC-ISO7816 (key = 16 characters, iv = 16 characters) description: AES-128-CBC-ISO7816 (key = 16 characters, iv = 16 characters) enum: - AES-128-CBC-ISO7816 - label: AES-256-CBC-ISO10126 (key = 32 characters, iv = 16 characters) description: AES-256-CBC-ISO10126 (key = 32 characters, iv = 16 characters) enum: - AES-256-CBC-ISO10126 - label: AES-192-CBC-ISO10126 (key = 24 characters, iv = 16 characters) description: AES-192-CBC-ISO10126 (key = 24 characters, iv = 16 characters) enum: - AES-192-CBC-ISO10126 - label: AES-128-CBC-ISO10126 (key = 16 characters, iv = 16 characters) description: AES-128-CBC-ISO10126 (key = 16 characters, iv = 16 characters) enum: - AES-128-CBC-ISO10126 ui: display_type: list title: Decryption algorithm description: The algorithm with which the data was encrypted key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 16 maxLength: 32 iv_field: type: string title: Initialization vector (IV) field description: The field from which to read the initialization vector, IV pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference decode_raw_bytes: type: boolean title: Decode the encrypted value and IV as Base 64 text description: Decode the encrypted value and initialization vector as Base64 text default: true required: - field - algorithm - key - iv_field - if: properties: type: enum: - js-script deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Script Execution description: >- Use JavaScript to reshape and transform your data. You can combine multiple actions like filtering, dropping, mapping, and casting inside of a single js script. additionalProperties: true properties: script: type: string title: Script ui: display_type: code-editor options: language: js description: The JavaScript representation of the event transformation. minLength: 1 maxLength: 10000 default: | // Modify the event using a subset of the JavaScript language. // The function must return the modified event function processEvent(message, metadata, timestamp, annotations) { return message } - if: properties: type: enum: - unroll deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Unroll description: Takes an array of events and emits them all as individual events. additionalProperties: false properties: field: type: string title: Field description: The JSON field name that contains an array of events pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference values_only: type: boolean title: Values only? description: >- When enabled, the values from the specified array field will be emitted as new events. Otherwise, the original event will be duplicated for each value in the array field, with the unrolled value present in the field specified. default: true required: - field - if: properties: type: enum: - vrl deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: VRL description: Modify your observability data using Vector Remap Language (VRL) additionalProperties: false properties: source: type: string title: VRL Source description: VRL code to execute for each event ui: display_type: code-editor options: language: vrl minLength: 1 default: .message = .message required: - source - if: properties: type: enum: - reduce deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Reduce description: Combine multiple events over time into one based on a set of criteria additionalProperties: false properties: duration_ms: type: number maximum: 7200000 title: Duration description: >- The amount of time (in milliseconds) to allow streaming events to accumulate into a single "reduced" event. The process repeats indefinitely, or until an "ends when" condition is satisfied. ui: display_type: milliseconds-to-human-readable default: 30000 max_events: type: number minimum: 1 maximum: 100000 title: Max Events description: >- The maximum number of events that can be included in a time window (specified by Duration above). The reduce operation will stop once it has reached this number of events, regardless of whether the Duration seconds have elapsed. group_by: type: array minItems: 1 title: Group By Field Path items: type: string title: Field description: The dot-notated property path errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ description: >- Before reducing, group events based on matching data from each of these field paths. Supports nesting via dot-notation. date_formats: type: array title: Date Formats description: >- Describes which root-level properties are dates, and their expected format. Dot-notation is supported, but nested field lookup paths will be an error. minItems: 1 items: type: object additionalProperties: false properties: field: type: string title: Date Field Path description: Specifies a root-level path property that contains a date value. maxLength: 200 errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ format: type: string title: Date Format description: The template describing the date format format: strftime_datetime errorMessage: must be a valid strftime datetime format maxLength: 200 required: - field - format merge_strategies: type: array title: Merge Strategy per Field minItems: 1 description: >- Specify merge strategies for individual root-level properties. Dot-notation is supported, but nested field lookup paths will be an error. items: type: object additionalProperties: false properties: field: type: string title: Field Path description: >- This is a root-level path property to apply a merge strategy to its value maxLength: 200 errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ strategy: oneOf: - label: Array description: Append each value onto an array enum: - array - label: Smallest array description: Keep the smallest array within each reduce time window enum: - shortest_array - label: Largest array description: Keep the largest array within each reduce time window enum: - longest_array - label: Unique Array description: All unique values go into a flattened array enum: - flat_unique - label: Join with space description: Concatenate into a single space-delimited string enum: - concat - label: Join with newline description: Concatenate into a single newline-delimited string enum: - concat_newline - label: Join description: Concatenate into a single string without using a delimiter enum: - concat_raw - label: Keep first description: Keep only the first value and discard the rest enum: - discard - label: Keep last description: Keep only the last non-null value enum: - retain - label: Min value description: >- Keep the minimum numeric value within each reduce time window enum: - min - label: Max value description: >- Keep the maximum numeric value within each reduce time window enum: - max - label: Sum description: Add numeric values together enum: - sum ui: display_type: list errorMessage: must be one of the allowed merge strategies title: Merge Strategy description: The merge strategy to be used for the specified property required: - field - strategy flush_condition: title: Event Flush Condition description: >- Force accumulated event reduction to flush the result when a conditional expression evaluates to true on an inbound event. allOf: - type: object properties: when: type: string enum: - none - starts_when - ends_when default: none ui: display_type: list errorMessage: '' title: Flush When required: - when - if: properties: when: not: enum: - none then: properties: conditional: allOf: - $ref: '#condition-group' ui: display_type: rules-engine required: - conditional - if: properties: type: enum: - map-fields deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Map Fields description: Maps data from one field to another, either by moving or copying. additionalProperties: false properties: mappings: type: array title: Mappings description: >- A list of field mappings. Mappings are applied in the order they are defined. minItems: 1 ui: display_type: sortable items: type: object properties: source_field: type: string title: Source field description: The field to copy data from pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference target_field: type: string title: Target field description: The field to copy data into pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference drop_source: type: boolean title: Drop source? description: >- When enabled, the source field is dropped after the data is copied to the target field. Otherwise, it is preserved. default: false overwrite_target: type: boolean title: Overwrite target? description: >- When enabled, any existing data in the target field is overwritten. Otherwise, the target field will be preserved and this mapping will have no effect. default: false required: - source_field - target_field - drop_source - overwrite_target if: properties: drop_source: enum: - true required: - drop_source then: properties: source_field: type: string title: Source field description: The field to copy data from pattern: >- ^(?:metadata|\.|(?:metadata|message)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ minLength: 1 errorMessage: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields). Note that dropping an event's message or timestamp is not allowed required: - source_field required: - mappings - if: properties: type: enum: - metrics-tag-cardinality-limit deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: title: Tag Cardinality Limit (Metrics) description: >- Limits the cardinality of metric events by either dropping events or tags that exceed a specified value limit additionalProperties: false allOf: - if: title: Mode properties: mode: type: string enum: - exact then: title: Value Limit properties: value_limit: type: integer title: Value Limit description: Maximum number of unique values for tags minimum: 1 maximum: 500 required: - value_limit - if: title: Mode properties: mode: type: string enum: - probabilistic then: title: Value Limit properties: value_limit: type: integer title: Value Limit description: Maximum number of unique values for tags minimum: 1 maximum: 5000 required: - value_limit properties: tags: type: array title: Tags description: >- A list of tags to apply cardinality limits. If none are provided, all tags will be considered. items: type: string title: Tag minLength: 1 maxLength: 100 minItems: 0 maxItems: 10 exclude_tags: type: array title: Exclude Tags description: A list of tags to explicitly exclude from cardinality limits. items: type: string title: Tag minLength: 1 maxLength: 100 minItems: 0 maxItems: 10 action: type: string title: Action description: The action to take when a tag's cardinality exceeds the value limit enum: - drop_tag - drop_event default: drop_event value_limit: type: integer title: Value Limit description: Maximum number of unique values for tags mode: type: string title: Mode description: The method to used to reduce tag value cardinality enum: - exact - probabilistic default: exact - if: properties: type: enum: - aggregate deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Aggregate description: >- Aggregates multiple events into a single event using either a tumbling interval window or a sliding interval window. additionalProperties: false properties: group_by: type: array minItems: 1 maxItems: 100 title: Group By Field Path description: >- When aggregating, group events based on matching values from each of these field paths. Supports nesting via dot-notation. items: type: string title: Field description: The dot-notated property path errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ ui: default_hint: - .name - .namespace - .tags evaluate: additionalProperties: type: string title: Evaluate description: How events should be aggregated within an interval. allOf: - if: properties: operation: enum: - CUSTOM then: properties: script: type: string title: Script ui: display_type: code-editor options: language: js description: The JavaScript representation of an event aggregation. minLength: 1 maxLength: 2048 default: > // Perform custom aggregation of the current event into the accumulated event. // The accumulated event is persisted across the entire aggregation window. // The function must return the accumulated event, which will be emitted when // the aggregation window elapses. function aggregateEvent(accum, event, metadata, timestamp, annotations) { return accum } required: - script properties: operation: title: Operation oneOf: - label: add description: >- Adds new values together if they are of the same type. For counters and gauges the values are added together. For sets, this is a union of the elements. For distribution, the samples are concatenated. For histograms, values in the same bucket will be added together. enum: - DEFAULT - label: sum description: >- Values are added together. Works for the GAUGE and COUNT metric types. enum: - SUM - label: minimum description: >- Get the minimum value for a series of events. Works for the GAUGE and COUNT metric types. enum: - MIN - label: maximum description: >- Get the maximum value for a series of events. Works for the GAUGE and COUNT metric types. enum: - MAX - label: average description: >- Values are averaged together. Works for the GAUGE and COUNT metric types. enum: - AVG - label: set intersection description: The aggregate produces a set of unique values. enum: - SET_INTERSECTION - label: distribution concatenation description: The aggregate produces a list of values combined together. enum: - DIST_CONCAT - label: custom description: Use a custom script to perform aggregation. enum: - CUSTOM ui: display_type: list required: - operation window: default: {} additionalProperties: type: integer title: Window Type description: >- Sliding windows can overlap, whereas tumbling windows are disjoint. For example, a tumbling window has a fixed time span and any events that fall within the "interval" will be used in the aggregate. Whereas in a sliding window, the aggregation occurs every "window duration" seconds after an event is encountered. allOf: - if: properties: type: enum: - sliding required: - type then: properties: window_min: type: integer title: Slide Duration (seconds) description: >- The amount of time to wait before creating a new sliding window for future events. minimum: 1 maximum: 200 default: 5 required: - window_min properties: type: type: string title: Type enum: - tumbling - sliding default: tumbling interval: type: integer title: Interval (seconds) description: The interval over which events are aggregated in seconds. default: 5 minimum: 1 maximum: 90000 required: - type - interval event_timestamp: type: string title: Event Timestamp description: >- The path to a field on the event that contains a epoch timestamp value. If an event does not have a timestamp field, events will be associated to the wall clock value when the event is processed. default: timestamp maxLength: 200 errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ conditional: ui: display_type: rules-engine title: Event Flush Condition description: >- Force accumulated event reduction to flush the result when a conditional expression evaluates to true on an inbound event. allOf: - $ref: '#condition-group' required: - evaluate - window - if: properties: type: enum: - event-to-metric deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Event to Metric description: Allows conversion between arbitrary events and a Metric. additionalProperties: false properties: metric_name: type: string title: Metric Name description: The machine name of the metric to emit. minLength: 1 maxLength: 128 pattern: ^[a-zA-Z][a-zA-Z0-9_:]*$ errorMessage: must be a valid metric name metric_kind: title: Kind description: >- The kind of metric to emit, Absolute or Incremental. Absolute metrics represent a complete value, and will generally replace an existing value for the metric in the target destination. Incremental metrics represent an additive value which is aggregated in the target destination to produce a new value. oneOf: - label: Incremental enum: - incremental - label: Absolute enum: - absolute default: incremental ui: display_type: radio-group metric_type: title: Type oneOf: - label: Counter enum: - counter - label: Sum enum: - sum - label: Gauge enum: - gauge default: counter ui: display_type: list value: title: Value description: The value of this metric. additionalProperties: false allOf: - if: properties: value_type: enum: - field then: title: Value properties: value: type: string title: Field value description: The field to use as the value. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference required: - value - if: properties: value_type: enum: - value then: title: Value properties: value: type: number title: Value description: The value. required: - value properties: value_type: title: Value type description: Either a value from an event field, or a new value input. oneOf: - label: Value from event field enum: - field - label: New value enum: - value default: field ui: display_type: radio-group value: type: - number - string required: - value_type namespace: title: Namespace description: The namespace for this metric. additionalProperties: false allOf: - if: properties: value_type: enum: - field then: title: Value properties: value: type: string title: Field value description: The field to use as the value. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference required: - value - if: properties: value_type: enum: - value required: - value_type then: title: Value properties: value: type: string title: Value description: The value. minLength: 1 maxLength: 128 required: - value properties: value_type: title: Value type description: >- Either a value from an event field, a new value input, or none to omit the namespace. oneOf: - label: Value from event field enum: - field - label: New value enum: - value - label: None enum: - none default: none ui: display_type: radio-group value: type: - string - 'null' required: - value_type tags: type: array maxItems: 10 title: Tags description: A set of tags (also called labels) to apply to the metric event. items: type: object title: Tag description: A tag to apply to the metric event. allOf: - if: properties: value_type: enum: - field then: title: Value properties: value: type: string title: Field value description: >- The field to use as the value of this tag. Note that fields with highly-variable values will result in high-cardinality metrics, which may impact storage or cost in downstream destinations. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference fallback: type: string title: Fallback value description: >- The value to use for this tag when the field value is missing. minLength: 1 maxLength: 512 required: - value - if: properties: value_type: enum: - value then: title: Value properties: value: type: string title: Value description: The value for this tag. minLength: 1 maxLength: 128 required: - value properties: name: type: string title: Name description: The tag name pattern: ^[a-zA-Z][a-zA-Z0-9_]*$ minLength: 1 maxLength: 128 errorMessage: must be a valid metric tag name value_type: title: Value type description: Either a value from an event field, or a new value input. oneOf: - label: Value from event field enum: - field - label: New value enum: - value default: field ui: display_type: radio-group value: type: string required: - name - value_type - value additionalProperties: type: string required: - metric_name - metric_kind - metric_type - value - namespace - if: properties: type: enum: - clustering deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Clustering description: Finds clusters of similar values additionalProperties: false properties: cluster_field: type: string title: Field description: Field to cluster values on pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference default: .message similarity_threshold: type: number title: Similarity threshold description: >- A threshold from 0.0 (not similar) to 1.0 (exactly similar) that determines how close two values need to be to be consider as being part of the same cluster default: 0.4 minimum: 0 maximum: 1 max_clusters: type: integer title: Maximum cluster description: Maximum number of clusters kept cached by the transform default: 1000 minimum: 1 maximum: 2000 max_node_depth: type: integer title: Maximum tree depth description: Maximum depth of the prefix tree default: 4 minimum: 2 maximum: 8 max_children: type: integer title: Maximum children default: 100 minimum: 2 maximum: 200 description: Maximum number of children per node of the prefix tree required: - cluster_field - if: properties: type: enum: - redact deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Redact description: Detect and anonymize sensitive data in logs and metrics additionalProperties: false properties: field: type: string title: Field description: >- Detect patterns in the specified field. If the field is empty, the processor will look for patterns across all the fields. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ default: . maxLength: 256 errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference mask_patterns: type: array title: Redact Patterns description: List of PII patterns to detect and mask minItems: 1 maxItems: 20 items: type: object additionalProperties: type: string title: Redact Pattern description: A PII pattern to detect and/or anonymize allOf: - if: properties: action: enum: - Replace required: - action then: properties: replacement: type: string title: Replacement description: >- The text to replace the detected pattern. Example: 000-00-0000 minLength: 1 maxLength: 512 required: - replacement - if: properties: action: enum: - Hash required: - action then: properties: algorithm: title: Algorithm description: >- Hash and encode (base16 or hex) matched PII data using the specified algorithm oneOf: - label: sha2 (sha-512_256) enum: - sha2 - label: sha3 (sha3-512) enum: - sha3 ui: display_type: list required: - algorithm - if: properties: name: enum: - custom required: - name then: properties: custom_options: allOf: - type: object additionalProperties: false properties: pattern: type: string title: Pattern description: The custom pattern minLength: 1 maxLength: 4096 case_sensitive: type: boolean title: Case sensitive description: Perform case sensitive matching default: true multiline: type: boolean title: Multiline mode description: >- ^ and $ match beginning and end of line respectively default: false match_newline: type: boolean title: Dot all mode description: Dot matches newline character default: false crlf_newline: type: boolean title: Carriage return (CRLF) mode description: "When multiline mode is enabled, newline character is interpreted as \r\n" default: false ignore_whitespace: type: boolean title: Verbose mode description: Ignore whitespace and allow comment lines default: false required: - pattern title: Replace Expression description: >- The regular expression used to find replacements. Named capture groups are supported, e.g. `(?P<key>\w+):(?P<value>.+)`. ui: display_type: regex-pattern required: - custom_options properties: name: title: Pattern description: The PII pattern to match oneOf: - label: Credit card number enum: - credit_card - label: Email address enum: - email_address - label: IPv4 address enum: - ipv4_address - label: US or Canada phone number enum: - us_canada_phone_number - label: US social security number enum: - us_social_security_number - label: Custom pattern ui: feature_flag: pipeline-parser-regex enum: - custom ui: display_type: list action: type: string title: Action description: Action to take on the detected PII data enum: - Hash - Replace - Detect Only default: Replace required: - name - action required: - mask_patterns - if: properties: type: enum: - throttle deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Throttle description: Throttle (rate-limit) events passing through this component additionalProperties: false properties: threshold: type: number minimum: 1 maximum: 10000000 title: Limit description: >- The number of events to allow over the time window. If a Key Field is specified, the limit will be applied to events for each unique value of that field. default: 1 window_ms: type: number minimum: 1000 maximum: 2592000000 title: Window description: The time window over which the configured limit is applied ui: display_type: milliseconds-to-human-readable default: 5000 key_field: type: string title: Key Field description: >- When configured, the value from this field will be used to group events into separate buckets for throttling. If left blank, or if the event does not contain a value in this field, the event will be throttled as part of the default bucket. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference exclude: allOf: - $ref: '#condition-group' title: Exclude description: When configured, events matching this condition will not be throttled. ui: display_type: rules-engine required: - threshold - window_ms - if: properties: type: enum: - set-timestamp deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Set Timestamp description: >- Parse a list of fields using a chosen time format. First match will override the default timestamp of the event. additionalProperties: false properties: parsers: type: array title: List of Fields and Parsers description: >- The list of fields and parsers to use in order of priority, short-circuiting on the first successful match. ui: display_type: multi-output-map options: sortable: false items: type: object properties: field: type: string title: Field description: The field whose value will be parsed by the chosen date pattern pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ default: . maxLength: 512 errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference options: $ref: '#timestamp-parser-options' required: - field - options minItems: 1 required: - parsers - if: properties: type: enum: - data-profiler deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Data Profiler description: >- Profile the data sent through your pipeline. The system will generate annotations such as event type and total bytes to be used downstream. It will also normalize the app/host/level/line fields before sending the event downstream. A profile of the data including total, events, event size, log composition, and message templates will be generated as well. additionalProperties: false properties: app_fields: type: array title: App Path(s) description: A list of paths to look for the app value minItems: 1 maxItems: 10 uniqueItems: true items: type: string title: App description: The path to the app maxLength: 512 pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference ui: default_hint: - .app - .application - .container host_fields: type: array title: Host Path(s) description: A list of paths to look for the host value minItems: 1 maxItems: 10 uniqueItems: true items: type: string title: Host description: The path to the host maxLength: 512 pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference ui: default_hint: - .host - .hostname level_fields: type: array title: Level Path(s) description: A list of paths to look for the log level minItems: 1 maxItems: 10 uniqueItems: true items: type: string title: Level description: The path to the level maxLength: 512 pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference ui: default_hint: - .level - .log_level line_fields: type: array title: Line Path(s) description: A list of paths to look for the event line minItems: 1 maxItems: 10 uniqueItems: true items: type: string title: Line description: The path to the line maxLength: 512 pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference ui: default_hint: - .line - .message label_fields: type: array title: Label Path(s) description: A list of paths to look for the event labels maxItems: 10 uniqueItems: true items: type: string title: Label description: The path to the labels maxLength: 512 pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference ui: default_hint: - .labels field_summary: type: boolean default: false title: Field Summary description: >- If enabled the profiler will attempt to summarize the values in fields of events passing through the node. ui: display_type: hidden ff_default_value: pipeline-node-transform-field-summary required: - app_fields - host_fields - level_fields - line_fields - if: properties: type: enum: - replace deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: title: Replace description: Replace parts of a string using a regular expression or static string additionalProperties: type: object allOf: - if: properties: replacement_type: enum: - regex required: - replacement_type then: properties: options: allOf: - type: object additionalProperties: false properties: pattern: type: string minLength: 1 maxLength: 4096 case_sensitive: type: boolean title: Case sensitive description: Perform case sensitive matching default: true multiline: type: boolean title: Multiline mode description: ^ and $ match beginning and end of line respectively default: false match_newline: type: boolean title: Dot all mode description: Dot matches newline character default: false crlf_newline: type: boolean title: Carriage return (CRLF) mode description: "When multiline mode is enabled, newline character is interpreted as \r\n" default: false ignore_whitespace: type: boolean title: Verbose mode description: Ignore whitespace and allow comment lines default: false required: - pattern title: Replace Expression description: >- The regular expression used to find replacements. Named capture groups are supported, e.g. `(?P<key>\w+):(?P<value>.+)`. ui: display_type: regex-pattern replace_with: type: string minLength: 1 maxLength: 4096 title: Replacement String description: >- The replacement string can be static or optionally contain capture group references. Capture groups can be positional or named. For example, to switch the position of two strings in the source string `onetwo`, the pattern could be `(?Pone)(?P.+)`, and the replacement would be `$two$one`. To do the same thing with positional capture groups, a pattern of `(\w{3})(\w{3})`, and a replacement of `$2$1` would have the same result.

To render a literal dollar sign `$` in the replacement, escape it with `$$`. So, if you had a capture group named `price`, to prepend a dollar sign to the replacement, you would use `$$$price`. required: - options - replace_with - if: properties: replacement_type: enum: - string required: - replacement_type then: properties: string_pattern: type: string title: String Pattern minLength: 1 maxLength: 4096 description: >- A static, case-sensitive string, which will be searched and replaced. replace_with: type: string minLength: 1 maxLength: 4096 title: Replacement String description: The static string used in each replacement. required: - string_pattern - replace_with properties: source_field: type: string title: Source Field description: The field that contains a string for replacement to operate on. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ default: . errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference max_times: type: number title: Maximum Replacements description: >- The maximum number of replacements to be made. If not specified, all occurrences will be replaced. minimum: 1 replacement_type: title: Replacement Type description: Specifies how the source string will be searched for replacement. ui: display_type: list errorMessage: must be one of the available pattern types oneOf: - label: String description: Replace a specified string one or more times. enum: - string - label: Regular Expression description: Replace using a regular expression. enum: - regex required: - source_field - replacement_type - if: properties: type: enum: - trace-sampling deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: title: Trace Sampling description: >- Performs sampling of traces per a given rate using the Trace ID as the unique identifier. If a trace is rate selected, all spans of the trace are forwarded. If the Trace ID cannot be found, the event is dropped. allOf: - if: properties: sample_type: enum: - head then: properties: rate: type: integer title: Rate description: >- The rate at which events will be forwarded, expressed as 1/N. For example, `rate = 10` means 1 out of every 10 unique traces will be forwarded and all others will be dropped default: 10 minimum: 2 maximum: 100000 parent_span_id_field: type: 'null' conditionals: type: 'null' required: - rate - if: properties: sample_type: enum: - tail then: properties: parent_span_id_field: type: string default: .parent_span_id title: Parent Span ID description: The field to use as the parent span identifier pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference conditionals: type: array minItems: 1 title: Conditionals description: >- A list of complete conditionals, including expressions and a sample rate to maintain items: type: object additionalProperties: false title: Conditional description: A list of conditions and a sample rate to maintain properties: rate: type: integer title: Rate description: >- The rate at which events will be forwarded, expressed as 1/N. For example, `rate = 10` means 1 out of every 10 unique traces will be forwarded and all others will be dropped. A `rate = 1` means keep all traces that match the condition. minimum: 1 maximum: 100000 default: 10 conditional: allOf: - $ref: '#condition-group' title: Conditional description: >- A group of expressions joined together by an and/or operator. ui: display_type: rules-engine _output_name: type: string title: '[INTERNAL]' description: >- Internal-only field that holds a static id for the output_name. This way, labels for outputs can be updated independent of the id. required: - rate - conditional required: - parent_span_id_field - conditionals properties: sample_type: type: string description: The type of trace sampling to apply. enum: - head title: Sample Type default: head trace_id_field: type: string default: .trace_id title: Trace ID description: The field to use as the identifier of a trace. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference required: - sample_type - trace_id_field properties: type: type: string enum: - filter - sample - drop-fields - parse - parse-sequentially - route - stringify - compact-fields - flatten-fields - dedupe - encrypt-fields - decrypt-fields - js-script - unroll - vrl - reduce - map-fields - metrics-tag-cardinality-limit - aggregate - event-to-metric - clustering - redact - throttle - set-timestamp - data-profiler - replace - trace-sampling deploy_type: type: string enum: - saas - local default: saas title: type: string pattern: ^[^<>"]+$ errorMessage: invalid title maxLength: 512 description: oneOf: - type: string minLength: 1 pattern: ^[^<>"]+$ - type: 'null' errorMessage: invalid description user_config: type: object generation_id: type: integer inputs: type: array items: type: string description: >- A list of inputs for the given transform. Typically this is empty when creating a new transform via the UI, but is leveraged by the terraform provider when managing pipeline state. required: - type - title - user_config - generation_id required: true parameters: - schema: type: string format: uuid in: path name: transform_group_id required: true - schema: type: string format: uuid in: path name: transform_group_transform_id required: true responses: '200': description: Default Response /v3/transform-group/{transform_group_id}/transform: post: tags: - Processor Groups description: Create a Processor in a Processor Group requestBody: content: application/json: schema: type: object allOf: - if: properties: type: enum: - filter deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Filter description: Define condition(s) to include or exclude events from the pipeline additionalProperties: false properties: action: title: Action description: How to handle events matching this criteria oneOf: - label: Allow events matching this criteria enum: - allow - label: Drop events matching this criteria enum: - drop default: allow ui: display_type: radio-group conditional: allOf: - $ref: '#condition-group' title: Conditional description: A group of expressions joined together by an and/or operator. ui: display_type: rules-engine required: - action - conditional - if: properties: type: enum: - sample deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Sample description: >- Sample data at a given rate, retaining only a subset of data events for further processing. additionalProperties: false properties: rate: type: integer title: Rate description: >- The rate at which events will be forwarded, expressed as 1/N. For example, `rate = 10` means 1 out of every 10 events will be forwarded and the rest will be dropped minimum: 2 maximum: 10000 default: 10 always_include: title: Always Include description: >- Certain events may always want to be included (not sampled), regardless of the sample rate. Events matching this criteria will always show up in the results. additionalProperties: type: - string - number dependencies: field: - str_operator str_operator: - field allOf: - if: title: Operator properties: str_operator: type: string enum: - greater - greater_or_equal - less - less_or_equal ui: display_type: list-operators title: Operator description: The comparison operator required: - str_operator then: title: Value properties: value: errorMessage: pattern: Value must be a number type: - string - number title: Value description: The value used in the expression, ie the "operand" minLength: 1 pattern: ^\s*-?(\d*\.)?\d+\s*$ required: - value - if: title: Operator properties: str_operator: type: string enum: - greater - greater_or_equal - less - less_or_equal - equal - contains - is_ip_in_cidr_range - ends_with - starts_with - drain_matches - contains_token_term - contains_token_prefix - percent_change_greater - percent_change_greater_or_equal - percent_change_less - percent_change_less_or_equal - value_change_greater - value_change_greater_or_equal - value_change_less - value_change_less_or_equal ui: display_type: list-operators title: Operator description: The comparison operator required: - str_operator then: title: Value properties: value: type: - string - number title: Value description: The value used in the expression, ie the "operand" minLength: 1 required: - value - if: title: Operator properties: str_operator: type: string enum: - contains - equal - ends_with - starts_with - drain_matches - contains_token_term - contains_token_prefix ui: display_type: list-operators title: Operator required: - str_operator then: title: Case sensitive properties: case_sensitive: type: boolean title: Case sensitive description: Perform case sensitive comparison? default: true properties: field: type: string title: Field description: The JSON field to use in a condition to always include in sampling pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference str_operator: type: string enum: - greater - greater_or_equal - less - less_or_equal - equal - contains - is_ip_in_cidr_range - is_metric - is_array - is_boolean - is_empty - is_null - is_number - is_object - is_string - exists - ends_with - starts_with - contains_token_term - contains_token_prefix - regex_match ui: display_type: list-operators title: Operator description: The comparison operator negate: type: boolean default: false title: Negate description: Negate the comparison expression required: - rate - if: properties: type: enum: - drop-fields deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Remove Fields description: Remove fields additionalProperties: false properties: fields: type: array title: Fields description: A list of fields to be removed items: type: string title: Field pattern: >- ^(?:metadata|\.|(?:metadata|message)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ minLength: 1 errorMessage: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields). Note that dropping an event's message or timestamp is not allowed minItems: 1 required: - fields - if: properties: type: enum: - parse deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: title: Parse description: Parse a specified field using the chosen parser additionalProperties: type: object errorMessage: properties: target_field: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) allOf: - if: properties: parser: enum: - parse_key_value then: properties: options: $ref: '#key-value-parser-options' - if: properties: parser: enum: - parse_timestamp then: properties: options: $ref: '#timestamp-parser-options' required: - options - if: properties: parser: enum: - parse_grok then: properties: options: $ref: '#grok-pattern-parser-options' required: - options - if: properties: parser: enum: - parse_regex then: properties: options: allOf: - $ref: '#regex-parser-options' ui: display_type: regex-pattern required: - options - if: properties: parser: enum: - parse_apache_log then: properties: options: $ref: '#apache-parser-options' required: - options - if: properties: parser: enum: - parse_nginx_log then: properties: options: $ref: '#nginx-parser-options' required: - options - if: properties: parser: enum: - parse_cef then: properties: options: $ref: '#cef-parser-options' - if: properties: parser: enum: - parse_csv then: properties: options: $ref: '#csv-parser-options' properties: field: type: string title: Field description: The JSON field whose value should be parsed. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ default: . errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference target_field: oneOf: - type: 'null' - type: string pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ maxLength: 512 errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference title: Target field description: >- The field into which the parsed value should be inserted. Leave blank to insert the parsed data into the original field. default: null parser: oneOf: - label: AWS Cloudfront Access Log group: Line Parsers ui: {} enum: - aws_cloudfront_access_log - label: AWS Cloudwatch Log group: Line Parsers ui: {} enum: - parse_aws_cloudwatch_log_subscription_message - label: AWS Load Balancer Log group: Line Parsers ui: {} enum: - parse_aws_alb_log - label: AWS S3 Access Log group: Line Parsers ui: {} enum: - aws_s3_access_log - label: Apache Log group: Line Parsers ui: {} enum: - parse_apache_log - label: CSV group: Line Parsers ui: {} enum: - parse_csv - label: Common Event Format (CEF) Log group: Line Parsers ui: {} enum: - parse_cef - label: Common Log group: Line Parsers ui: {} enum: - parse_common_log - label: Elasticsearch Log group: Line Parsers ui: {} enum: - elasticsearch_log - label: Golang Log group: Line Parsers ui: {} enum: - golang_log - label: Google Log group: Line Parsers ui: {} enum: - parse_glog - label: Grok Pattern group: Line Parsers ui: {} enum: - parse_grok - label: Integer group: Field Parsers ui: {} enum: - parse_int - label: JSON group: Line Parsers ui: {} enum: - parse_json - label: Key/Value group: Line Parsers ui: {} enum: - parse_key_value - label: Kubernetes Log group: Line Parsers ui: {} enum: - parse_klog - label: Linux Authorization group: Line Parsers ui: {} enum: - parse_linux_authorization - label: Mongo Log group: Line Parsers ui: {} enum: - mongo_log - label: Mongo3 Log group: Line Parsers ui: {} enum: - mongo3_log - label: Nginx Log group: Line Parsers ui: {} enum: - parse_nginx_log - label: PSQL Log group: Line Parsers ui: {} enum: - psql_log - label: Query String group: Field Parsers ui: {} enum: - parse_query_string - label: Regular Expression group: Line Parsers ui: feature_flag: pipeline-parser-regex enum: - parse_regex - label: Syslog group: Line Parsers ui: {} enum: - parse_syslog - label: Timestamp group: Field Parsers ui: {} enum: - parse_timestamp - label: Tokens group: Field Parsers ui: {} enum: - parse_tokens - label: URL group: Field Parsers ui: {} enum: - parse_url - label: User Agent group: Field Parsers ui: {} enum: - parse_user_agent - label: VPC Flow Log group: Line Parsers ui: {} description: Parse VPC logs using default format for fields from v2 to v4 enum: - parse_aws_vpc_flow_log ui: display_type: list errorMessage: must be one of allowed parsers title: Parser description: The kind of parser to use against the input value from "field". required: - field - parser - if: properties: type: enum: - parse-sequentially deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Parse Sequentially description: >- Parse fields using one or more parsers in order, sending out the event on the first successful match. additionalProperties: false errorMessage: properties: target_field: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) properties: field: type: string title: Field description: The field whose value should be parsed. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ default: . maxLength: 512 errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference target_field: oneOf: - type: 'null' - type: string pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ maxLength: 512 errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference title: Target field description: >- The field into which the parsed value should be inserted. Leave blank to insert the parsed data into the original field. default: null parsers: type: array title: List of parsers description: >- The list of parsers to use in order against the input valuefrom "field", short-circuiting on the first successful match. items: type: object allOf: - if: properties: parser: enum: - parse_key_value then: properties: options: $ref: '#key-value-parser-options' - if: properties: parser: enum: - parse_timestamp then: properties: options: $ref: '#timestamp-parser-options' required: - options - if: properties: parser: enum: - parse_grok then: properties: options: $ref: '#grok-pattern-parser-options' required: - options - if: properties: parser: enum: - parse_regex then: properties: options: allOf: - $ref: '#regex-parser-options' ui: display_type: regex-pattern required: - options - if: properties: parser: enum: - parse_apache_log then: properties: options: $ref: '#apache-parser-options' required: - options - if: properties: parser: enum: - parse_nginx_log then: properties: options: $ref: '#nginx-parser-options' required: - options - if: properties: parser: enum: - parse_cef then: properties: options: $ref: '#cef-parser-options' - if: properties: parser: enum: - parse_csv then: properties: options: $ref: '#csv-parser-options' properties: label: oneOf: - type: 'null' - type: string maxLength: 20 title: Title description: >- An arbitrary name you choose to identify the results of this processor for use when connecting them to other components. parser: oneOf: - label: AWS Cloudfront Access Log group: Line Parsers ui: {} enum: - aws_cloudfront_access_log - label: AWS Cloudwatch Log group: Line Parsers ui: {} enum: - parse_aws_cloudwatch_log_subscription_message - label: AWS Load Balancer Log group: Line Parsers ui: {} enum: - parse_aws_alb_log - label: AWS S3 Access Log group: Line Parsers ui: {} enum: - aws_s3_access_log - label: Apache Log group: Line Parsers ui: {} enum: - parse_apache_log - label: CSV group: Line Parsers ui: {} enum: - parse_csv - label: Common Event Format (CEF) Log group: Line Parsers ui: {} enum: - parse_cef - label: Common Log group: Line Parsers ui: {} enum: - parse_common_log - label: Elasticsearch Log group: Line Parsers ui: {} enum: - elasticsearch_log - label: Golang Log group: Line Parsers ui: {} enum: - golang_log - label: Google Log group: Line Parsers ui: {} enum: - parse_glog - label: Grok Pattern group: Line Parsers ui: {} enum: - parse_grok - label: Integer group: Field Parsers ui: {} enum: - parse_int - label: JSON group: Line Parsers ui: {} enum: - parse_json - label: Key/Value group: Line Parsers ui: {} enum: - parse_key_value - label: Kubernetes Log group: Line Parsers ui: {} enum: - parse_klog - label: Linux Authorization group: Line Parsers ui: {} enum: - parse_linux_authorization - label: Mongo Log group: Line Parsers ui: {} enum: - mongo_log - label: Mongo3 Log group: Line Parsers ui: {} enum: - mongo3_log - label: Nginx Log group: Line Parsers ui: {} enum: - parse_nginx_log - label: PSQL Log group: Line Parsers ui: {} enum: - psql_log - label: Query String group: Field Parsers ui: {} enum: - parse_query_string - label: Regular Expression group: Line Parsers ui: feature_flag: pipeline-parser-regex enum: - parse_regex - label: Syslog group: Line Parsers ui: {} enum: - parse_syslog - label: Timestamp group: Field Parsers ui: {} enum: - parse_timestamp - label: Tokens group: Field Parsers ui: {} enum: - parse_tokens - label: URL group: Field Parsers ui: {} enum: - parse_url - label: User Agent group: Field Parsers ui: {} enum: - parse_user_agent - label: VPC Flow Log group: Line Parsers ui: {} description: Parse VPC logs using default format for fields from v2 to v4 enum: - parse_aws_vpc_flow_log ui: display_type: list errorMessage: must be one of allowed parsers title: Parser description: The kind of parser to use against the input value from "field". _output_name: type: string title: '[INTERNAL]' description: >- Internal-only field that holds a static id for the output_name. This way, labels for outputs can be updated independent of the id. default: '' required: - parser - _output_name additionalProperties: type: object minItems: 1 ui: display_type: multi-output-map unique_parser_and_options: true errorMessage: uniqueItemProperties: parser can not be defined more than once required: - field - parsers - if: properties: type: enum: - route deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Route description: >- Route data based on whether it matches (or doesn't match) logical comparisons additionalProperties: false properties: conditionals: type: array minItems: 1 title: Conditionals description: A list of complete conditionals, including expressions and label items: type: object additionalProperties: false title: Conditional description: A list of conditions and a label properties: conditional: $ref: '#condition-group' label: type: string minLength: 1 maxLength: 255 title: Label description: >- An arbitrary name you choose to identify the results of this processor for use when connecting them to other components. _output_name: type: string title: '[INTERNAL]' description: >- Internal-only field that holds a static id for the output_name. This way, labels for outputs can be updated independent of the id. default: '' required: - conditional - label - _output_name required: - conditionals - if: properties: type: enum: - stringify deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Stringify description: Stringify messages as JSON additionalProperties: false - if: properties: type: enum: - compact-fields deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Compact Fields description: Remove empty values from a list of fields additionalProperties: false properties: fields: type: array title: Fields description: A list of fields to remove empty values from items: type: string title: Field description: Field to remove empty value from pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference minItems: 1 options: type: object additionalProperties: false title: Compact Options description: Additional options for removing empty values from fields properties: compact_array: type: boolean title: Compact Array description: Remove empty arrays from a field default: true compact_object: type: boolean title: Compact Object description: Remove empty objects from a field default: true required: - fields - if: properties: type: enum: - flatten-fields deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Flatten Fields description: >- Flattens the object or array value of a field into a single level representation additionalProperties: false properties: fields: type: array title: Fields description: >- Define specific nested objects you wish to flatten. This will not flatten the entire object, just the specified fields. Leave blank to flatten the entire object items: type: string title: Field description: Field to flatten pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference options: type: object additionalProperties: false title: Flatten Options description: Additional options for flattening fields properties: delimiter: type: string default: _ title: Delimiter description: >- When flattening, the nested field labels will be appended to the higher level field names along with a delimiter to allow understanding of the original structure. The default is an underscore (_) to ensure no collisions with the . notation used for field identifiers minLength: 1 maxLength: 1 - if: properties: type: enum: - dedupe deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Dedupe description: Dedupe (remove duplicates) from the data stream additionalProperties: false properties: number_of_events: type: number description: Number of events to compare across title: Number of Events minimum: 2 maximum: 5000 default: 5000 comparison_type: type: string enum: - Ignore - Match description: >- Match: Only compares across the fields which are specified by the user.Ignore: Compares everything but the fields specified by the user. title: Comparison Type default: Match fields: type: array title: Fields description: A list of fields on which to base deduping items: type: string title: Field pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference minItems: 1 required: - number_of_events - comparison_type - fields - if: properties: type: enum: - encrypt-fields deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: title: Encrypt Field description: Encrypts the value of the provided field additionalProperties: false allOf: - if: properties: algorithm: enum: - AES-256-CFB required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CFB required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CFB required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-OFB required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-OFB required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-OFB required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-CTR required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CTR required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CTR required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-CBC-PKCS7 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CBC-PKCS7 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CBC-PKCS7 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-CBC-ANSIX923 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CBC-ANSIX923 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CBC-ANSIX923 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-CBC-ISO7816 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CBC-ISO7816 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CBC-ISO7816 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-CBC-ISO10126 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CBC-ISO10126 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CBC-ISO10126 required: - algorithm then: properties: key: type: string title: Encryption key description: The encryption key minLength: 16 maxLength: 16 required: - key properties: field: type: string title: Field description: >- Field to encrypt. The value of the field must be a primitive (string, number, boolean) pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference algorithm: oneOf: - label: AES-256-CFB (key = 32 characters, iv = 16 characters) description: AES-256-CFB (key = 32 characters, iv = 16 characters) enum: - AES-256-CFB - label: AES-192-CFB (key = 24 characters, iv = 16 characters) description: AES-192-CFB (key = 24 characters, iv = 16 characters) enum: - AES-192-CFB - label: AES-128-CFB (key = 16 characters, iv = 16 characters) description: AES-128-CFB (key = 16 characters, iv = 16 characters) enum: - AES-128-CFB - label: AES-256-OFB (key = 32 characters, iv = 16 characters) description: AES-256-OFB (key = 32 characters, iv = 16 characters) enum: - AES-256-OFB - label: AES-192-OFB (key = 24 characters, iv = 16 characters) description: AES-192-OFB (key = 24 characters, iv = 16 characters) enum: - AES-192-OFB - label: AES-128-OFB (key = 16 characters, iv = 16 characters) description: AES-128-OFB (key = 16 characters, iv = 16 characters) enum: - AES-128-OFB - label: AES-256-CTR (key = 32 characters, iv = 16 characters) description: AES-256-CTR (key = 32 characters, iv = 16 characters) enum: - AES-256-CTR - label: AES-192-CTR (key = 24 characters, iv = 16 characters) description: AES-192-CTR (key = 24 characters, iv = 16 characters) enum: - AES-192-CTR - label: AES-128-CTR (key = 16 characters, iv = 16 characters) description: AES-128-CTR (key = 16 characters, iv = 16 characters) enum: - AES-128-CTR - label: AES-256-CBC-PKCS7 (key = 32 characters, iv = 16 characters) description: AES-256-CBC-PKCS7 (key = 32 characters, iv = 16 characters) enum: - AES-256-CBC-PKCS7 - label: AES-192-CBC-PKCS7 (key = 24 characters, iv = 16 characters) description: AES-192-CBC-PKCS7 (key = 24 characters, iv = 16 characters) enum: - AES-192-CBC-PKCS7 - label: AES-128-CBC-PKCS7 (key = 16 characters, iv = 16 characters) description: AES-128-CBC-PKCS7 (key = 16 characters, iv = 16 characters) enum: - AES-128-CBC-PKCS7 - label: AES-256-CBC-ANSIX923 (key = 32 characters, iv = 16 characters) description: AES-256-CBC-ANSIX923 (key = 32 characters, iv = 16 characters) enum: - AES-256-CBC-ANSIX923 - label: AES-192-CBC-ANSIX923 (key = 24 characters, iv = 16 characters) description: AES-192-CBC-ANSIX923 (key = 24 characters, iv = 16 characters) enum: - AES-192-CBC-ANSIX923 - label: AES-128-CBC-ANSIX923 (key = 16 characters, iv = 16 characters) description: AES-128-CBC-ANSIX923 (key = 16 characters, iv = 16 characters) enum: - AES-128-CBC-ANSIX923 - label: AES-256-CBC-ISO7816 (key = 32 characters, iv = 16 characters) description: AES-256-CBC-ISO7816 (key = 32 characters, iv = 16 characters) enum: - AES-256-CBC-ISO7816 - label: AES-192-CBC-ISO7816 (key = 24 characters, iv = 16 characters) description: AES-192-CBC-ISO7816 (key = 24 characters, iv = 16 characters) enum: - AES-192-CBC-ISO7816 - label: AES-128-CBC-ISO7816 (key = 16 characters, iv = 16 characters) description: AES-128-CBC-ISO7816 (key = 16 characters, iv = 16 characters) enum: - AES-128-CBC-ISO7816 - label: AES-256-CBC-ISO10126 (key = 32 characters, iv = 16 characters) description: AES-256-CBC-ISO10126 (key = 32 characters, iv = 16 characters) enum: - AES-256-CBC-ISO10126 - label: AES-192-CBC-ISO10126 (key = 24 characters, iv = 16 characters) description: AES-192-CBC-ISO10126 (key = 24 characters, iv = 16 characters) enum: - AES-192-CBC-ISO10126 - label: AES-128-CBC-ISO10126 (key = 16 characters, iv = 16 characters) description: AES-128-CBC-ISO10126 (key = 16 characters, iv = 16 characters) enum: - AES-128-CBC-ISO10126 ui: display_type: list title: Encryption algorithm description: The encryption algorithm to use key: type: string title: Encryption key description: The encryption key minLength: 16 maxLength: 32 iv_field: type: string title: Initialization vector (IV) field description: >- The field in which to store the generated initialization vector, IV. Each encrypted value will have a unique IV pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference encode_raw_bytes: type: boolean title: Encode encrypted field and IV as Base 64 text description: >- Encode the encrypted value and generated initialization vector as Base64 text default: true required: - field - algorithm - key - iv_field - if: properties: type: enum: - decrypt-fields deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: title: Decrypt Field additionalProperties: false description: Decrypts the value of the provided field allOf: - if: properties: algorithm: enum: - AES-256-CFB required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CFB required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CFB required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-OFB required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-OFB required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-OFB required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-CTR required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CTR required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CTR required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-CBC-PKCS7 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CBC-PKCS7 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CBC-PKCS7 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-CBC-ANSIX923 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CBC-ANSIX923 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CBC-ANSIX923 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-CBC-ISO7816 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CBC-ISO7816 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CBC-ISO7816 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 16 maxLength: 16 required: - key - if: properties: algorithm: enum: - AES-256-CBC-ISO10126 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 32 maxLength: 32 required: - key - if: properties: algorithm: enum: - AES-192-CBC-ISO10126 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 24 maxLength: 24 required: - key - if: properties: algorithm: enum: - AES-128-CBC-ISO10126 required: - algorithm then: properties: key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 16 maxLength: 16 required: - key properties: field: type: string title: Field description: Field to decrypt. The value of the field must be a string pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference algorithm: oneOf: - label: AES-256-CFB (key = 32 characters, iv = 16 characters) description: AES-256-CFB (key = 32 characters, iv = 16 characters) enum: - AES-256-CFB - label: AES-192-CFB (key = 24 characters, iv = 16 characters) description: AES-192-CFB (key = 24 characters, iv = 16 characters) enum: - AES-192-CFB - label: AES-128-CFB (key = 16 characters, iv = 16 characters) description: AES-128-CFB (key = 16 characters, iv = 16 characters) enum: - AES-128-CFB - label: AES-256-OFB (key = 32 characters, iv = 16 characters) description: AES-256-OFB (key = 32 characters, iv = 16 characters) enum: - AES-256-OFB - label: AES-192-OFB (key = 24 characters, iv = 16 characters) description: AES-192-OFB (key = 24 characters, iv = 16 characters) enum: - AES-192-OFB - label: AES-128-OFB (key = 16 characters, iv = 16 characters) description: AES-128-OFB (key = 16 characters, iv = 16 characters) enum: - AES-128-OFB - label: AES-256-CTR (key = 32 characters, iv = 16 characters) description: AES-256-CTR (key = 32 characters, iv = 16 characters) enum: - AES-256-CTR - label: AES-192-CTR (key = 24 characters, iv = 16 characters) description: AES-192-CTR (key = 24 characters, iv = 16 characters) enum: - AES-192-CTR - label: AES-128-CTR (key = 16 characters, iv = 16 characters) description: AES-128-CTR (key = 16 characters, iv = 16 characters) enum: - AES-128-CTR - label: AES-256-CBC-PKCS7 (key = 32 characters, iv = 16 characters) description: AES-256-CBC-PKCS7 (key = 32 characters, iv = 16 characters) enum: - AES-256-CBC-PKCS7 - label: AES-192-CBC-PKCS7 (key = 24 characters, iv = 16 characters) description: AES-192-CBC-PKCS7 (key = 24 characters, iv = 16 characters) enum: - AES-192-CBC-PKCS7 - label: AES-128-CBC-PKCS7 (key = 16 characters, iv = 16 characters) description: AES-128-CBC-PKCS7 (key = 16 characters, iv = 16 characters) enum: - AES-128-CBC-PKCS7 - label: AES-256-CBC-ANSIX923 (key = 32 characters, iv = 16 characters) description: AES-256-CBC-ANSIX923 (key = 32 characters, iv = 16 characters) enum: - AES-256-CBC-ANSIX923 - label: AES-192-CBC-ANSIX923 (key = 24 characters, iv = 16 characters) description: AES-192-CBC-ANSIX923 (key = 24 characters, iv = 16 characters) enum: - AES-192-CBC-ANSIX923 - label: AES-128-CBC-ANSIX923 (key = 16 characters, iv = 16 characters) description: AES-128-CBC-ANSIX923 (key = 16 characters, iv = 16 characters) enum: - AES-128-CBC-ANSIX923 - label: AES-256-CBC-ISO7816 (key = 32 characters, iv = 16 characters) description: AES-256-CBC-ISO7816 (key = 32 characters, iv = 16 characters) enum: - AES-256-CBC-ISO7816 - label: AES-192-CBC-ISO7816 (key = 24 characters, iv = 16 characters) description: AES-192-CBC-ISO7816 (key = 24 characters, iv = 16 characters) enum: - AES-192-CBC-ISO7816 - label: AES-128-CBC-ISO7816 (key = 16 characters, iv = 16 characters) description: AES-128-CBC-ISO7816 (key = 16 characters, iv = 16 characters) enum: - AES-128-CBC-ISO7816 - label: AES-256-CBC-ISO10126 (key = 32 characters, iv = 16 characters) description: AES-256-CBC-ISO10126 (key = 32 characters, iv = 16 characters) enum: - AES-256-CBC-ISO10126 - label: AES-192-CBC-ISO10126 (key = 24 characters, iv = 16 characters) description: AES-192-CBC-ISO10126 (key = 24 characters, iv = 16 characters) enum: - AES-192-CBC-ISO10126 - label: AES-128-CBC-ISO10126 (key = 16 characters, iv = 16 characters) description: AES-128-CBC-ISO10126 (key = 16 characters, iv = 16 characters) enum: - AES-128-CBC-ISO10126 ui: display_type: list title: Decryption algorithm description: The algorithm with which the data was encrypted key: type: string title: Decryption key description: The key/secret used to encrypt the value minLength: 16 maxLength: 32 iv_field: type: string title: Initialization vector (IV) field description: The field from which to read the initialization vector, IV pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference decode_raw_bytes: type: boolean title: Decode the encrypted value and IV as Base 64 text description: Decode the encrypted value and initialization vector as Base64 text default: true required: - field - algorithm - key - iv_field - if: properties: type: enum: - js-script deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Script Execution description: >- Use JavaScript to reshape and transform your data. You can combine multiple actions like filtering, dropping, mapping, and casting inside of a single js script. additionalProperties: true properties: script: type: string title: Script ui: display_type: code-editor options: language: js description: The JavaScript representation of the event transformation. minLength: 1 maxLength: 10000 default: | // Modify the event using a subset of the JavaScript language. // The function must return the modified event function processEvent(message, metadata, timestamp, annotations) { return message } - if: properties: type: enum: - unroll deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Unroll description: Takes an array of events and emits them all as individual events. additionalProperties: false properties: field: type: string title: Field description: The JSON field name that contains an array of events pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference values_only: type: boolean title: Values only? description: >- When enabled, the values from the specified array field will be emitted as new events. Otherwise, the original event will be duplicated for each value in the array field, with the unrolled value present in the field specified. default: true required: - field - if: properties: type: enum: - vrl deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: VRL description: Modify your observability data using Vector Remap Language (VRL) additionalProperties: false properties: source: type: string title: VRL Source description: VRL code to execute for each event ui: display_type: code-editor options: language: vrl minLength: 1 default: .message = .message required: - source - if: properties: type: enum: - reduce deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Reduce description: Combine multiple events over time into one based on a set of criteria additionalProperties: false properties: duration_ms: type: number maximum: 7200000 title: Duration description: >- The amount of time (in milliseconds) to allow streaming events to accumulate into a single "reduced" event. The process repeats indefinitely, or until an "ends when" condition is satisfied. ui: display_type: milliseconds-to-human-readable default: 30000 max_events: type: number minimum: 1 maximum: 100000 title: Max Events description: >- The maximum number of events that can be included in a time window (specified by Duration above). The reduce operation will stop once it has reached this number of events, regardless of whether the Duration seconds have elapsed. group_by: type: array minItems: 1 title: Group By Field Path items: type: string title: Field description: The dot-notated property path errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ description: >- Before reducing, group events based on matching data from each of these field paths. Supports nesting via dot-notation. date_formats: type: array title: Date Formats description: >- Describes which root-level properties are dates, and their expected format. Dot-notation is supported, but nested field lookup paths will be an error. minItems: 1 items: type: object additionalProperties: false properties: field: type: string title: Date Field Path description: Specifies a root-level path property that contains a date value. maxLength: 200 errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ format: type: string title: Date Format description: The template describing the date format format: strftime_datetime errorMessage: must be a valid strftime datetime format maxLength: 200 required: - field - format merge_strategies: type: array title: Merge Strategy per Field minItems: 1 description: >- Specify merge strategies for individual root-level properties. Dot-notation is supported, but nested field lookup paths will be an error. items: type: object additionalProperties: false properties: field: type: string title: Field Path description: >- This is a root-level path property to apply a merge strategy to its value maxLength: 200 errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ strategy: oneOf: - label: Array description: Append each value onto an array enum: - array - label: Smallest array description: Keep the smallest array within each reduce time window enum: - shortest_array - label: Largest array description: Keep the largest array within each reduce time window enum: - longest_array - label: Unique Array description: All unique values go into a flattened array enum: - flat_unique - label: Join with space description: Concatenate into a single space-delimited string enum: - concat - label: Join with newline description: Concatenate into a single newline-delimited string enum: - concat_newline - label: Join description: Concatenate into a single string without using a delimiter enum: - concat_raw - label: Keep first description: Keep only the first value and discard the rest enum: - discard - label: Keep last description: Keep only the last non-null value enum: - retain - label: Min value description: >- Keep the minimum numeric value within each reduce time window enum: - min - label: Max value description: >- Keep the maximum numeric value within each reduce time window enum: - max - label: Sum description: Add numeric values together enum: - sum ui: display_type: list errorMessage: must be one of the allowed merge strategies title: Merge Strategy description: The merge strategy to be used for the specified property required: - field - strategy flush_condition: title: Event Flush Condition description: >- Force accumulated event reduction to flush the result when a conditional expression evaluates to true on an inbound event. allOf: - type: object properties: when: type: string enum: - none - starts_when - ends_when default: none ui: display_type: list errorMessage: '' title: Flush When required: - when - if: properties: when: not: enum: - none then: properties: conditional: allOf: - $ref: '#condition-group' ui: display_type: rules-engine required: - conditional - if: properties: type: enum: - map-fields deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Map Fields description: Maps data from one field to another, either by moving or copying. additionalProperties: false properties: mappings: type: array title: Mappings description: >- A list of field mappings. Mappings are applied in the order they are defined. minItems: 1 ui: display_type: sortable items: type: object properties: source_field: type: string title: Source field description: The field to copy data from pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference target_field: type: string title: Target field description: The field to copy data into pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference drop_source: type: boolean title: Drop source? description: >- When enabled, the source field is dropped after the data is copied to the target field. Otherwise, it is preserved. default: false overwrite_target: type: boolean title: Overwrite target? description: >- When enabled, any existing data in the target field is overwritten. Otherwise, the target field will be preserved and this mapping will have no effect. default: false required: - source_field - target_field - drop_source - overwrite_target if: properties: drop_source: enum: - true required: - drop_source then: properties: source_field: type: string title: Source field description: The field to copy data from pattern: >- ^(?:metadata|\.|(?:metadata|message)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ minLength: 1 errorMessage: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields). Note that dropping an event's message or timestamp is not allowed required: - source_field required: - mappings - if: properties: type: enum: - metrics-tag-cardinality-limit deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: title: Tag Cardinality Limit (Metrics) description: >- Limits the cardinality of metric events by either dropping events or tags that exceed a specified value limit additionalProperties: false allOf: - if: title: Mode properties: mode: type: string enum: - exact then: title: Value Limit properties: value_limit: type: integer title: Value Limit description: Maximum number of unique values for tags minimum: 1 maximum: 500 required: - value_limit - if: title: Mode properties: mode: type: string enum: - probabilistic then: title: Value Limit properties: value_limit: type: integer title: Value Limit description: Maximum number of unique values for tags minimum: 1 maximum: 5000 required: - value_limit properties: tags: type: array title: Tags description: >- A list of tags to apply cardinality limits. If none are provided, all tags will be considered. items: type: string title: Tag minLength: 1 maxLength: 100 minItems: 0 maxItems: 10 exclude_tags: type: array title: Exclude Tags description: A list of tags to explicitly exclude from cardinality limits. items: type: string title: Tag minLength: 1 maxLength: 100 minItems: 0 maxItems: 10 action: type: string title: Action description: The action to take when a tag's cardinality exceeds the value limit enum: - drop_tag - drop_event default: drop_event value_limit: type: integer title: Value Limit description: Maximum number of unique values for tags mode: type: string title: Mode description: The method to used to reduce tag value cardinality enum: - exact - probabilistic default: exact - if: properties: type: enum: - aggregate deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Aggregate description: >- Aggregates multiple events into a single event using either a tumbling interval window or a sliding interval window. additionalProperties: false properties: group_by: type: array minItems: 1 maxItems: 100 title: Group By Field Path description: >- When aggregating, group events based on matching values from each of these field paths. Supports nesting via dot-notation. items: type: string title: Field description: The dot-notated property path errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ ui: default_hint: - .name - .namespace - .tags evaluate: additionalProperties: type: string title: Evaluate description: How events should be aggregated within an interval. allOf: - if: properties: operation: enum: - CUSTOM then: properties: script: type: string title: Script ui: display_type: code-editor options: language: js description: The JavaScript representation of an event aggregation. minLength: 1 maxLength: 2048 default: > // Perform custom aggregation of the current event into the accumulated event. // The accumulated event is persisted across the entire aggregation window. // The function must return the accumulated event, which will be emitted when // the aggregation window elapses. function aggregateEvent(accum, event, metadata, timestamp, annotations) { return accum } required: - script properties: operation: title: Operation oneOf: - label: add description: >- Adds new values together if they are of the same type. For counters and gauges the values are added together. For sets, this is a union of the elements. For distribution, the samples are concatenated. For histograms, values in the same bucket will be added together. enum: - DEFAULT - label: sum description: >- Values are added together. Works for the GAUGE and COUNT metric types. enum: - SUM - label: minimum description: >- Get the minimum value for a series of events. Works for the GAUGE and COUNT metric types. enum: - MIN - label: maximum description: >- Get the maximum value for a series of events. Works for the GAUGE and COUNT metric types. enum: - MAX - label: average description: >- Values are averaged together. Works for the GAUGE and COUNT metric types. enum: - AVG - label: set intersection description: The aggregate produces a set of unique values. enum: - SET_INTERSECTION - label: distribution concatenation description: The aggregate produces a list of values combined together. enum: - DIST_CONCAT - label: custom description: Use a custom script to perform aggregation. enum: - CUSTOM ui: display_type: list required: - operation window: default: {} additionalProperties: type: integer title: Window Type description: >- Sliding windows can overlap, whereas tumbling windows are disjoint. For example, a tumbling window has a fixed time span and any events that fall within the "interval" will be used in the aggregate. Whereas in a sliding window, the aggregation occurs every "window duration" seconds after an event is encountered. allOf: - if: properties: type: enum: - sliding required: - type then: properties: window_min: type: integer title: Slide Duration (seconds) description: >- The amount of time to wait before creating a new sliding window for future events. minimum: 1 maximum: 200 default: 5 required: - window_min properties: type: type: string title: Type enum: - tumbling - sliding default: tumbling interval: type: integer title: Interval (seconds) description: The interval over which events are aggregated in seconds. default: 5 minimum: 1 maximum: 90000 required: - type - interval event_timestamp: type: string title: Event Timestamp description: >- The path to a field on the event that contains a epoch timestamp value. If an event does not have a timestamp field, events will be associated to the wall clock value when the event is processed. default: timestamp maxLength: 200 errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ conditional: ui: display_type: rules-engine title: Event Flush Condition description: >- Force accumulated event reduction to flush the result when a conditional expression evaluates to true on an inbound event. allOf: - $ref: '#condition-group' required: - evaluate - window - if: properties: type: enum: - event-to-metric deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Event to Metric description: Allows conversion between arbitrary events and a Metric. additionalProperties: false properties: metric_name: type: string title: Metric Name description: The machine name of the metric to emit. minLength: 1 maxLength: 128 pattern: ^[a-zA-Z][a-zA-Z0-9_:]*$ errorMessage: must be a valid metric name metric_kind: title: Kind description: >- The kind of metric to emit, Absolute or Incremental. Absolute metrics represent a complete value, and will generally replace an existing value for the metric in the target destination. Incremental metrics represent an additive value which is aggregated in the target destination to produce a new value. oneOf: - label: Incremental enum: - incremental - label: Absolute enum: - absolute default: incremental ui: display_type: radio-group metric_type: title: Type oneOf: - label: Counter enum: - counter - label: Sum enum: - sum - label: Gauge enum: - gauge default: counter ui: display_type: list value: title: Value description: The value of this metric. additionalProperties: false allOf: - if: properties: value_type: enum: - field then: title: Value properties: value: type: string title: Field value description: The field to use as the value. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference required: - value - if: properties: value_type: enum: - value then: title: Value properties: value: type: number title: Value description: The value. required: - value properties: value_type: title: Value type description: Either a value from an event field, or a new value input. oneOf: - label: Value from event field enum: - field - label: New value enum: - value default: field ui: display_type: radio-group value: type: - number - string required: - value_type namespace: title: Namespace description: The namespace for this metric. additionalProperties: false allOf: - if: properties: value_type: enum: - field then: title: Value properties: value: type: string title: Field value description: The field to use as the value. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference required: - value - if: properties: value_type: enum: - value required: - value_type then: title: Value properties: value: type: string title: Value description: The value. minLength: 1 maxLength: 128 required: - value properties: value_type: title: Value type description: >- Either a value from an event field, a new value input, or none to omit the namespace. oneOf: - label: Value from event field enum: - field - label: New value enum: - value - label: None enum: - none default: none ui: display_type: radio-group value: type: - string - 'null' required: - value_type tags: type: array maxItems: 10 title: Tags description: A set of tags (also called labels) to apply to the metric event. items: type: object title: Tag description: A tag to apply to the metric event. allOf: - if: properties: value_type: enum: - field then: title: Value properties: value: type: string title: Field value description: >- The field to use as the value of this tag. Note that fields with highly-variable values will result in high-cardinality metrics, which may impact storage or cost in downstream destinations. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference fallback: type: string title: Fallback value description: >- The value to use for this tag when the field value is missing. minLength: 1 maxLength: 512 required: - value - if: properties: value_type: enum: - value then: title: Value properties: value: type: string title: Value description: The value for this tag. minLength: 1 maxLength: 128 required: - value properties: name: type: string title: Name description: The tag name pattern: ^[a-zA-Z][a-zA-Z0-9_]*$ minLength: 1 maxLength: 128 errorMessage: must be a valid metric tag name value_type: title: Value type description: Either a value from an event field, or a new value input. oneOf: - label: Value from event field enum: - field - label: New value enum: - value default: field ui: display_type: radio-group value: type: string required: - name - value_type - value additionalProperties: type: string required: - metric_name - metric_kind - metric_type - value - namespace - if: properties: type: enum: - clustering deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Clustering description: Finds clusters of similar values additionalProperties: false properties: cluster_field: type: string title: Field description: Field to cluster values on pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference default: .message similarity_threshold: type: number title: Similarity threshold description: >- A threshold from 0.0 (not similar) to 1.0 (exactly similar) that determines how close two values need to be to be consider as being part of the same cluster default: 0.4 minimum: 0 maximum: 1 max_clusters: type: integer title: Maximum cluster description: Maximum number of clusters kept cached by the transform default: 1000 minimum: 1 maximum: 2000 max_node_depth: type: integer title: Maximum tree depth description: Maximum depth of the prefix tree default: 4 minimum: 2 maximum: 8 max_children: type: integer title: Maximum children default: 100 minimum: 2 maximum: 200 description: Maximum number of children per node of the prefix tree required: - cluster_field - if: properties: type: enum: - redact deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Redact description: Detect and anonymize sensitive data in logs and metrics additionalProperties: false properties: field: type: string title: Field description: >- Detect patterns in the specified field. If the field is empty, the processor will look for patterns across all the fields. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ default: . maxLength: 256 errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference mask_patterns: type: array title: Redact Patterns description: List of PII patterns to detect and mask minItems: 1 maxItems: 20 items: type: object additionalProperties: type: string title: Redact Pattern description: A PII pattern to detect and/or anonymize allOf: - if: properties: action: enum: - Replace required: - action then: properties: replacement: type: string title: Replacement description: >- The text to replace the detected pattern. Example: 000-00-0000 minLength: 1 maxLength: 512 required: - replacement - if: properties: action: enum: - Hash required: - action then: properties: algorithm: title: Algorithm description: >- Hash and encode (base16 or hex) matched PII data using the specified algorithm oneOf: - label: sha2 (sha-512_256) enum: - sha2 - label: sha3 (sha3-512) enum: - sha3 ui: display_type: list required: - algorithm - if: properties: name: enum: - custom required: - name then: properties: custom_options: allOf: - type: object additionalProperties: false properties: pattern: type: string title: Pattern description: The custom pattern minLength: 1 maxLength: 4096 case_sensitive: type: boolean title: Case sensitive description: Perform case sensitive matching default: true multiline: type: boolean title: Multiline mode description: >- ^ and $ match beginning and end of line respectively default: false match_newline: type: boolean title: Dot all mode description: Dot matches newline character default: false crlf_newline: type: boolean title: Carriage return (CRLF) mode description: "When multiline mode is enabled, newline character is interpreted as \r\n" default: false ignore_whitespace: type: boolean title: Verbose mode description: Ignore whitespace and allow comment lines default: false required: - pattern title: Replace Expression description: >- The regular expression used to find replacements. Named capture groups are supported, e.g. `(?P<key>\w+):(?P<value>.+)`. ui: display_type: regex-pattern required: - custom_options properties: name: title: Pattern description: The PII pattern to match oneOf: - label: Credit card number enum: - credit_card - label: Email address enum: - email_address - label: IPv4 address enum: - ipv4_address - label: US or Canada phone number enum: - us_canada_phone_number - label: US social security number enum: - us_social_security_number - label: Custom pattern ui: feature_flag: pipeline-parser-regex enum: - custom ui: display_type: list action: type: string title: Action description: Action to take on the detected PII data enum: - Hash - Replace - Detect Only default: Replace required: - name - action required: - mask_patterns - if: properties: type: enum: - throttle deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Throttle description: Throttle (rate-limit) events passing through this component additionalProperties: false properties: threshold: type: number minimum: 1 maximum: 10000000 title: Limit description: >- The number of events to allow over the time window. If a Key Field is specified, the limit will be applied to events for each unique value of that field. default: 1 window_ms: type: number minimum: 1000 maximum: 2592000000 title: Window description: The time window over which the configured limit is applied ui: display_type: milliseconds-to-human-readable default: 5000 key_field: type: string title: Key Field description: >- When configured, the value from this field will be used to group events into separate buckets for throttling. If left blank, or if the event does not contain a value in this field, the event will be throttled as part of the default bucket. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference exclude: allOf: - $ref: '#condition-group' title: Exclude description: When configured, events matching this condition will not be throttled. ui: display_type: rules-engine required: - threshold - window_ms - if: properties: type: enum: - set-timestamp deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Set Timestamp description: >- Parse a list of fields using a chosen time format. First match will override the default timestamp of the event. additionalProperties: false properties: parsers: type: array title: List of Fields and Parsers description: >- The list of fields and parsers to use in order of priority, short-circuiting on the first successful match. ui: display_type: multi-output-map options: sortable: false items: type: object properties: field: type: string title: Field description: The field whose value will be parsed by the chosen date pattern pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ default: . maxLength: 512 errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference options: $ref: '#timestamp-parser-options' required: - field - options minItems: 1 required: - parsers - if: properties: type: enum: - data-profiler deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Data Profiler description: >- Profile the data sent through your pipeline. The system will generate annotations such as event type and total bytes to be used downstream. It will also normalize the app/host/level/line fields before sending the event downstream. A profile of the data including total, events, event size, log composition, and message templates will be generated as well. additionalProperties: false properties: app_fields: type: array title: App Path(s) description: A list of paths to look for the app value minItems: 1 maxItems: 10 uniqueItems: true items: type: string title: App description: The path to the app maxLength: 512 pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference ui: default_hint: - .app - .application - .container host_fields: type: array title: Host Path(s) description: A list of paths to look for the host value minItems: 1 maxItems: 10 uniqueItems: true items: type: string title: Host description: The path to the host maxLength: 512 pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference ui: default_hint: - .host - .hostname level_fields: type: array title: Level Path(s) description: A list of paths to look for the log level minItems: 1 maxItems: 10 uniqueItems: true items: type: string title: Level description: The path to the level maxLength: 512 pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference ui: default_hint: - .level - .log_level line_fields: type: array title: Line Path(s) description: A list of paths to look for the event line minItems: 1 maxItems: 10 uniqueItems: true items: type: string title: Line description: The path to the line maxLength: 512 pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference ui: default_hint: - .line - .message label_fields: type: array title: Label Path(s) description: A list of paths to look for the event labels maxItems: 10 uniqueItems: true items: type: string title: Label description: The path to the labels maxLength: 512 pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference ui: default_hint: - .labels field_summary: type: boolean default: false title: Field Summary description: >- If enabled the profiler will attempt to summarize the values in fields of events passing through the node. ui: display_type: hidden ff_default_value: pipeline-node-transform-field-summary required: - app_fields - host_fields - level_fields - line_fields - if: properties: type: enum: - replace deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: title: Replace description: Replace parts of a string using a regular expression or static string additionalProperties: type: object allOf: - if: properties: replacement_type: enum: - regex required: - replacement_type then: properties: options: allOf: - type: object additionalProperties: false properties: pattern: type: string minLength: 1 maxLength: 4096 case_sensitive: type: boolean title: Case sensitive description: Perform case sensitive matching default: true multiline: type: boolean title: Multiline mode description: ^ and $ match beginning and end of line respectively default: false match_newline: type: boolean title: Dot all mode description: Dot matches newline character default: false crlf_newline: type: boolean title: Carriage return (CRLF) mode description: "When multiline mode is enabled, newline character is interpreted as \r\n" default: false ignore_whitespace: type: boolean title: Verbose mode description: Ignore whitespace and allow comment lines default: false required: - pattern title: Replace Expression description: >- The regular expression used to find replacements. Named capture groups are supported, e.g. `(?P<key>\w+):(?P<value>.+)`. ui: display_type: regex-pattern replace_with: type: string minLength: 1 maxLength: 4096 title: Replacement String description: >- The replacement string can be static or optionally contain capture group references. Capture groups can be positional or named. For example, to switch the position of two strings in the source string `onetwo`, the pattern could be `(?Pone)(?P.+)`, and the replacement would be `$two$one`. To do the same thing with positional capture groups, a pattern of `(\w{3})(\w{3})`, and a replacement of `$2$1` would have the same result.

To render a literal dollar sign `$` in the replacement, escape it with `$$`. So, if you had a capture group named `price`, to prepend a dollar sign to the replacement, you would use `$$$price`. required: - options - replace_with - if: properties: replacement_type: enum: - string required: - replacement_type then: properties: string_pattern: type: string title: String Pattern minLength: 1 maxLength: 4096 description: >- A static, case-sensitive string, which will be searched and replaced. replace_with: type: string minLength: 1 maxLength: 4096 title: Replacement String description: The static string used in each replacement. required: - string_pattern - replace_with properties: source_field: type: string title: Source Field description: The field that contains a string for replacement to operate on. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ default: . errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference max_times: type: number title: Maximum Replacements description: >- The maximum number of replacements to be made. If not specified, all occurrences will be replaced. minimum: 1 replacement_type: title: Replacement Type description: Specifies how the source string will be searched for replacement. ui: display_type: list errorMessage: must be one of the available pattern types oneOf: - label: String description: Replace a specified string one or more times. enum: - string - label: Regular Expression description: Replace using a regular expression. enum: - regex required: - source_field - replacement_type - if: properties: type: enum: - trace-sampling deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: title: Trace Sampling description: >- Performs sampling of traces per a given rate using the Trace ID as the unique identifier. If a trace is rate selected, all spans of the trace are forwarded. If the Trace ID cannot be found, the event is dropped. allOf: - if: properties: sample_type: enum: - head then: properties: rate: type: integer title: Rate description: >- The rate at which events will be forwarded, expressed as 1/N. For example, `rate = 10` means 1 out of every 10 unique traces will be forwarded and all others will be dropped default: 10 minimum: 2 maximum: 100000 parent_span_id_field: type: 'null' conditionals: type: 'null' required: - rate - if: properties: sample_type: enum: - tail then: properties: parent_span_id_field: type: string default: .parent_span_id title: Parent Span ID description: The field to use as the parent span identifier pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference conditionals: type: array minItems: 1 title: Conditionals description: >- A list of complete conditionals, including expressions and a sample rate to maintain items: type: object additionalProperties: false title: Conditional description: A list of conditions and a sample rate to maintain properties: rate: type: integer title: Rate description: >- The rate at which events will be forwarded, expressed as 1/N. For example, `rate = 10` means 1 out of every 10 unique traces will be forwarded and all others will be dropped. A `rate = 1` means keep all traces that match the condition. minimum: 1 maximum: 100000 default: 10 conditional: allOf: - $ref: '#condition-group' title: Conditional description: >- A group of expressions joined together by an and/or operator. ui: display_type: rules-engine _output_name: type: string title: '[INTERNAL]' description: >- Internal-only field that holds a static id for the output_name. This way, labels for outputs can be updated independent of the id. required: - rate - conditional required: - parent_span_id_field - conditionals properties: sample_type: type: string description: The type of trace sampling to apply. enum: - head title: Sample Type default: head trace_id_field: type: string default: .trace_id title: Trace ID description: The field to use as the identifier of a trace. pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference required: - sample_type - trace_id_field properties: type: type: string enum: - filter - sample - drop-fields - parse - parse-sequentially - route - stringify - compact-fields - flatten-fields - dedupe - encrypt-fields - decrypt-fields - js-script - unroll - vrl - reduce - map-fields - metrics-tag-cardinality-limit - aggregate - event-to-metric - clustering - redact - throttle - set-timestamp - data-profiler - replace - trace-sampling deploy_type: type: string enum: - saas - local default: saas title: type: string pattern: ^[^<>"]+$ errorMessage: invalid title maxLength: 512 description: oneOf: - type: string minLength: 1 pattern: ^[^<>"]+$ - type: 'null' errorMessage: invalid description user_config: type: object inputs: type: array items: type: string description: >- A list of inputs for the given transform. Typically this is empty when creating a new transform via the UI, but is leveraged by the terraform provider when managing pipeline state. required: - type - title - user_config required: true parameters: - schema: type: string format: uuid in: path name: transform_group_id required: true responses: '200': description: Default Response /v3/pipeline/grok-pattern: get: summary: Retrieve grok pattern tags: - Sampling and Simulation description: Retrieves a list of grok patterns available to use responses: '200': description: Default Response /v3/pipeline/grok-pattern/simulate: post: summary: Simulate grok pattern tags: - Sampling and Simulation description: Simulate data flowing through a grok pattern requestBody: content: application/json: schema: type: object properties: pattern: type: string errorMessage: must be a valid grok expression composed only by community patterns pattern: ^(?:%\{[A-Z_0-9]+(?::\w+){0,2}\}[\s,;:-]*)+$ values: type: array items: type: string minItems: 1 required: - pattern - values required: true responses: '200': description: Default Response /v3/pipeline/regex-pattern/simulate: post: summary: Simulate regex pattern tags: - Sampling and Simulation description: View results of sending data using a regular expression requestBody: content: application/json: schema: type: object properties: pattern: type: string values: type: array items: type: string minItems: 1 required: - pattern - values required: true responses: '200': description: Default Response /v3/pipeline/{pipeline_id}/simulate/{pipeline_source_id}: post: summary: Start simulation of pipeline tags: - Sampling and Simulation description: Sends data to a vector instance with a simulated pipeline requestBody: content: application/json: schema: type: object properties: pipeline_sample_id: type: string format: uuid required: - pipeline_sample_id required: true parameters: - schema: type: string format: uuid in: path name: pipeline_id required: true - schema: type: string format: uuid in: path name: pipeline_source_id required: true responses: '200': description: Default Response /v3/pipeline/{pipeline_id}/tap/{component_id}: post: summary: Start pipeline tap tags: - Sampling and Simulation description: Start a tap operation to view data as it flows through the pipeline. requestBody: content: application/json: schema: type: object additionalProperties: false properties: limit: type: integer default: 10 description: The maximum number of events to return minimum: 1 maximum: 100 timeout: type: integer default: 5 description: >- The amount of time, in seconds, to wait for `limit` events to return before timing out minimum: 1 maximum: 20 edge_id: type: string minLength: 1 description: Specific edge_id instance to tap filter: allOf: - $ref: '#condition-group' title: Conditional description: A group of expressions joined together by an and/or operator. ui: display_type: rules-engine parameters: - schema: type: string format: uuid in: path name: pipeline_id required: true - schema: type: string pattern: ^([^\.]+)(?:\.(.+))?$ in: path name: component_id required: true description: The component whose output is to be tapped responses: '200': description: Default Response /v3/pipeline/timestamp-format/simulate: post: summary: Simulate timestamp pattern tags: - Sampling and Simulation description: Simulate data with a timestamp format requestBody: content: application/json: schema: type: object properties: format: type: string format: strftime_datetime errorMessage: must be a valid strftime datetime format minLength: 1 values: type: array items: type: string minItems: 1 required: - format - values required: true responses: '200': description: Default Response /v3/pipeline/{container_id}/source/{source_id}/sample/{sample_id}: delete: summary: Delete data sample tags: - Sampling and Simulation description: Remove a data sample from the pipeline parameters: - schema: type: string title: Container ID format: uuid in: path name: container_id required: true description: The ID of the container entity, either a Pipeline or Processor Group - schema: title: Source ID oneOf: - type: string format: uuid - enum: - transform-group-input-source-id in: path name: source_id required: true description: The ID of the source to retrieve samples for - schema: type: string title: Sample ID format: uuid in: path name: sample_id required: true description: The ID of the sample responses: '200': description: Default Response get: summary: Fetch data sample tags: - Sampling and Simulation description: Returns a data sample parameters: - schema: type: string title: Container ID format: uuid in: path name: container_id required: true description: The ID of the container entity, either a Pipeline or Processor Group - schema: title: Source ID oneOf: - type: string format: uuid - enum: - transform-group-input-source-id in: path name: source_id required: true description: The ID of the source to retrieve samples for - schema: type: string title: Sample ID format: uuid in: path name: sample_id required: true description: The ID of the sample responses: '200': description: Default Response put: summary: Update data sample tags: - Sampling and Simulation description: Updates an existing data sample requestBody: content: application/json: schema: type: object properties: title: oneOf: - type: string minLength: 1 maxLength: 512 pattern: ^[^<>"]+$ errorMessage: invalid title name - type: 'null' parameters: - schema: type: string title: Container ID format: uuid in: path name: container_id required: true description: The ID of the container entity, either a Pipeline or Processor Group - schema: title: Source ID oneOf: - type: string format: uuid - enum: - transform-group-input-source-id in: path name: source_id required: true description: The ID of the source to retrieve samples for - schema: type: string title: Sample ID format: uuid in: path name: sample_id required: true description: The ID of the sample responses: '200': description: Default Response /v3/pipeline/{container_id}/source/{source_id}/sample: get: summary: List data samples tags: - Sampling and Simulation description: Returns a list of data samples that exist in the pipeline parameters: - schema: type: string title: Container ID format: uuid in: path name: container_id required: true description: The ID of the container entity, either a Pipeline or Processor Group - schema: title: Source ID oneOf: - type: string format: uuid - enum: - transform-group-input-source-id in: path name: source_id required: true description: The ID of the source to retrieve samples for responses: '200': description: Default Response post: summary: Create data sample tags: - Sampling and Simulation description: Create sample data for use in simulation requestBody: content: application/json: schema: type: object additionalProperties: false properties: title: oneOf: - type: string minLength: 1 maxLength: 512 pattern: ^[^<>"]+$ - type: 'null' errorMessage: invalid title name version: title: Sample Version default: v2 enum: - v2 container_type: type: string enum: - pipeline - transform_group title: Container Type description: Discriminator to identify the type of container referenced by the container_id default: pipeline data: type: array title: Sample data minItems: 1 maxItems: 100 items: $ref: '#sample-envelope' required: - data required: true parameters: - schema: type: string title: Container ID format: uuid in: path name: container_id required: true description: The ID of the container entity, either a Pipeline or Processor Group - schema: title: Source ID oneOf: - type: string format: uuid - enum: - transform-group-input-source-id in: path name: source_id required: true description: The ID of the source to retrieve samples for responses: '200': description: Default Response /v3/transform-group/{transform_group_id}/simulate: post: tags: - Sampling and Simulation description: Sends data to a vector instance with a simulated transform group requestBody: content: application/json: schema: type: object properties: sample_id: type: string format: uuid required: - sample_id required: true parameters: - schema: type: string format: uuid in: path name: transform_group_id required: true responses: '200': description: Default Response /v1/config/screen: get: tags: - Screens summary: List all screens description: List all screens in a short form that only includes the screen id and title. operationId: list-screens responses: '200': description: A list of screens is returned on success. content: application/json: schema: type: array items: $ref: '#/components/schemas/screenListResource' post: tags: - Screens summary: Creates a new screen description: Creates a new screen operationId: create-screen requestBody: description: Request body. content: application/json: schema: $ref: '#/components/schemas/screenResource' required: true responses: '200': description: A new screen was created successfully! Entire memberResource is returned. content: application/json: schema: $ref: '#/components/schemas/screenResource' '400': description: The supplied request is invalid or malformed. content: application/json: schema: $ref: '#/components/schemas/errResponse' example: error: '"ratio" is required' code: BadRequest status: error /v1/config/screen/{id}: get: tags: - Screens summary: Retrieve a screen description: Retrieve a specific screen operationId: get-screen parameters: - name: id in: path required: true schema: type: string responses: '200': description: A fully detailed screen resource for the matching id. content: application/json: schema: $ref: '#/components/schemas/screenResource' '404': description: The specified screen was not found. content: application/json: schema: $ref: '#/components/schemas/errResponse' example: error: Unable to find screen with this id on this account code: NotFound status: error delete: tags: - Screens summary: Delete a screen description: Deletes a screen operationId: delete-screen parameters: - name: id in: path required: true schema: type: string responses: '200': description: The screen was deleted successfully. content: application/json: schema: type: object properties: deleted: type: boolean description: Indicates whether the screen was deleted example: true '404': description: The specified screen was not found. content: application/json: schema: $ref: '#/components/schemas/errResponse' example: error: Screen Not Found code: NotFound status: error /v3/pipeline/gateway-route/{gateway_route_id}: delete: summary: Delete gateway route tags: - Shared Sources description: >- Delete an unused gateway route. If the route is used by any active pipeline, an error will be returned. parameters: - schema: type: string in: path name: gateway_route_id required: true responses: '200': description: Default Response get: summary: Retrieve gateway route tags: - Shared Sources description: Retreive a gateway route parameters: - schema: type: string in: path name: gateway_route_id required: true responses: '200': description: Default Response patch: summary: Update gateway route property tags: - Shared Sources description: Patches a gateway route requestBody: content: application/json: schema: type: object additionalProperties: false properties: title: type: string minLength: 1 maxLength: 512 pattern: ^[^<>"]+$ errorMessage: invalid title name description: oneOf: - type: string minLength: 1 pattern: ^[^<>"]+$ - type: 'null' errorMessage: invalid description origin: type: string enum: - ui - system description: The route origin legacy_routing: type: boolean required: - title required: true parameters: - schema: type: string in: path name: gateway_route_id required: true responses: '200': description: Default Response put: summary: Update gateway route tags: - Shared Sources description: Updates a gateway route requestBody: content: application/json: schema: type: object additionalProperties: false properties: title: type: string minLength: 1 maxLength: 512 pattern: ^[^<>"]+$ errorMessage: invalid title name description: oneOf: - type: string minLength: 1 - type: 'null' errorMessage: invalid description type: type: string enum: - http - splunk-hec - kinesis-firehose - fluent - logstash - mezmo-agent - mezmo-datadog-source - webhook - prometheus-remote-write - open-telemetry-metrics - open-telemetry-logs - open-telemetry-traces legacy_routing: type: boolean required: - title - type required: true parameters: - schema: type: string in: path name: gateway_route_id required: true responses: '200': description: Default Response /v3/pipeline/gateway-route: get: summary: List all gateway routes tags: - Shared Sources description: List all of the shareable gateway routes for an account parameters: - schema: type: string format: uuid in: query name: id required: false description: Optional gateway route id to retrieve. - schema: type: string enum: - ui - system in: query name: origin required: false description: Filters the list of routes by origin. responses: '200': description: Default Response post: summary: Create gateway route tags: - Shared Sources description: Create a gateway route requestBody: content: application/json: schema: type: object additionalProperties: false properties: title: type: string minLength: 1 maxLength: 512 pattern: ^[^<>"]+$ errorMessage: invalid title name description: oneOf: - type: string minLength: 1 pattern: ^[^<>"]+$ - type: 'null' errorMessage: invalid description type: type: string enum: - http - splunk-hec - kinesis-firehose - fluent - logstash - mezmo-agent - mezmo-datadog-source - webhook - prometheus-remote-write - open-telemetry-metrics - open-telemetry-logs - open-telemetry-traces start_classification: type: boolean default: true legacy_routing: type: boolean default: true required: - title - type required: true responses: '200': description: Default Response /v3/pipeline/gateway-route/{gateway_route_id}/access-key/{id}: delete: summary: Delete access key tags: - Shared Sources description: Delete an access key parameters: - schema: type: string format: uuid in: path name: gateway_route_id required: true - schema: type: string format: uuid in: path name: id required: true responses: '200': description: Default Response /v3/pipeline/gateway-route/{gateway_route_id}/access-key: post: summary: Create access key tags: - Shared Sources description: Create an access key associated with this gateway route requestBody: content: application/json: schema: type: object allOf: - if: title: Type properties: type: type: string enum: - user_defined required: - type then: title: Key properties: key: type: string description: The access key to use required: - key properties: type: type: string enum: - generated - user_defined title: type: string minLength: 1 maxLength: 512 pattern: ^[^<>"]+$ errorMessage: invalid title name key: type: string minLength: 1 maxLength: 512 required: - type - title required: true parameters: - schema: type: string format: uuid in: path name: gateway_route_id required: true responses: '200': description: Default Response /v3/pipeline/gateway-route/{gateway_route_id}/signing-key/{id}: delete: summary: Delete signing key tags: - Shared Sources description: Removes a signing key attached to a gateway route parameters: - schema: type: string format: uuid in: path name: gateway_route_id required: true - schema: type: string format: uuid in: path name: id required: true responses: '200': description: Default Response /v3/pipeline/gateway-route/{gateway_route_id}/signing-key: put: summary: Add signing key tags: - Shared Sources description: Adds a signing key to a gateway route requestBody: content: application/json: schema: type: object additionalProperties: false properties: key: type: string minLength: 1 maxLength: 512 required: - key required: true parameters: - schema: type: string format: uuid in: path name: gateway_route_id required: true responses: '200': description: Default Response /v1/config/ingestion/status: get: tags: - Suspensions summary: Get Ingestion Status description: Retrieve the status of ingestion operationId: get-v1-config-ingestion-status responses: '200': description: OK content: application/json: schema: type: object properties: isIngesting: type: boolean '500': description: Failed to fetch ingestion status /v1/config/ingestion/suspend: post: tags: - Suspensions summary: Initiate Ingestion Suspension description: >- First step in suspending ingestion for the instance. Returns a token for use in /suspend/confirm to actually stop ingestion operationId: post-v1-config-ingestion-suspend responses: '200': description: Token to be used in /suspend/confirm content: application/json: schema: type: object properties: token: $ref: '#/components/schemas/uuid4' /v1/config/ingestion/suspend/confirm: post: tags: - Suspensions summary: Confirm Ingestion Suspension description: After /suspend is called, a token is returned for use with this route as a confirmation. operationId: post-v1-config-ingestion-suspend-confirm requestBody: description: Request parameters content: application/json: schema: type: object properties: token: $ref: '#/components/schemas/uuid4' required: true responses: '200': $ref: '#/components/responses/standardResponse200' '400': description: Bad Suspend Token content: application/json: schema: $ref: '#/components/schemas/errPropsDefault' '500': description: Failed to fetch ingestion status /v1/config/ingestion/resume: post: tags: - Suspensions summary: Resume Ingestion description: Resumes ingestion if it has been previously suspended operationId: post-v1-config-ingestion-resume responses: '200': $ref: '#/components/responses/standardResponse200' '500': description: Failed to fetch ingestion status /v1/usage/apps: get: tags: - Usage v1 summary: List Usage By App description: Lists aggregated usage information for all apps during a time period. operationId: list-usage-app parameters: - name: from in: query description: Start date. Set as UNIX timestamp in seconds. required: true schema: type: integer - name: to in: query description: End date. Set as UNIX timestamp in seconds. required: true schema: type: integer - name: limit in: query description: Maximum amount of apps to retrieve. required: false schema: type: integer responses: '200': description: OK content: application/json: schema: type: object additionalProperties: type: object properties: {} '400': description: Bad request content: {} /v1/usage/apps/{name}: get: tags: - Usage v1 summary: Get Usage By App description: >- Gets the aggregated usage information for an app matching the name provided as a path parameter, during a time period. Returns null when not found. operationId: get-usage-app parameters: - name: name in: path description: The name of the app from which to get the aggregated usage data required: true schema: type: string - name: from in: query description: Start date. Set as UNIX timestamp in seconds. required: true schema: type: integer - name: to in: query description: End date. Set as UNIX timestamp in seconds. required: true schema: type: integer responses: '200': description: OK content: application/json: schema: type: object additionalProperties: type: object properties: {} '400': description: Bad request content: {} /v1/usage/tags: get: tags: - Usage v1 summary: List Usage by Tag description: Lists aggregated usage information for all tags during a time period. operationId: list-usage-tag parameters: - name: from in: query description: Start date. Set as UNIX timestamp in seconds. required: true schema: type: integer - name: to in: query description: End date. Set as UNIX timestamp in seconds. required: true schema: type: integer - name: limit in: query description: Maximum amount of apps to retrieve. required: false schema: type: integer responses: '200': description: OK content: application/json: schema: type: object additionalProperties: type: object properties: {} '400': description: Bad request content: {} /v1/usage/tags/{name}: get: tags: - Usage v1 summary: Get Usage by Tag description: >- Gets the aggregated usage information for a tag matching the name provided as a path parameter, during a time period. Returns null when not found. operationId: get-usage-tag parameters: - name: name in: path description: The name of the app from which to get the aggregated usage data required: true schema: type: string - name: from in: query description: Start date. Set as UNIX timestamp in seconds. required: true schema: type: integer - name: to in: query description: End date. Set as UNIX timestamp in seconds. required: true schema: type: integer responses: '200': description: OK content: application/json: schema: type: object additionalProperties: type: object properties: {} '400': description: Bad request content: {} /v1/usage/hosts: get: tags: - Usage v1 summary: List Usage by Host description: Lists aggregated usage information for all hosts during a time period. operationId: list-usage-host parameters: - name: from in: query description: Start date. Set as UNIX timestamp in seconds. required: true schema: type: integer - name: to in: query description: End date. Set as UNIX timestamp in seconds. required: true schema: type: integer - name: limit in: query description: Maximum amount of apps to retrieve. required: false schema: type: integer responses: '200': description: OK content: application/json: schema: type: object additionalProperties: type: object properties: {} '400': description: Bad request content: {} /v1/usage/hosts/{name}: get: tags: - Usage v1 summary: Get Usage by Host description: >- Gets the aggregated usage information for a host matching the name provided as a path parameter, during a time period. Returns null when not found. operationId: get-usage-host parameters: - name: name in: path description: The name of the app from which to get the aggregated usage data required: true schema: type: string - name: from in: query description: Start date. Set as UNIX timestamp in seconds. required: true schema: type: integer - name: to in: query description: End date. Set as UNIX timestamp in seconds. required: true schema: type: integer responses: '200': description: OK content: application/json: schema: type: object additionalProperties: type: object properties: {} '400': description: Bad request content: {} /v2/usage: get: tags: - Usage v2 summary: Retrieve account usage totals description: Get aggregated usage information for an account's data during a time period. operationId: list-usage-v2 parameters: - name: from in: query description: >- Start time. A date-time string as defined by the ISO 8601 date format. This date-time will be used to calculate the starting day; the exact time will not be used. required: true schema: type: string example: '2023-01-01T12:00:00-05:00' - name: to in: query description: >- End time. A date-time string as defined by the ISO 8601 date format. This date-time will be used to calculate the ending day; the exact time will not be used. required: true schema: type: string example: '2023-01-01T12:00:00-05:00' responses: '200': $ref: '#/components/responses/usageResponse200' '400': description: Bad request content: application/json: schema: type: object properties: code: type: string error: type: string description: Message indicating why the request is bad. status: type: string /v2/usage/apps: get: tags: - Usage v2 summary: List Usage By App description: Lists aggregated usage information for all apps during a time period. operationId: list-usage-app-v2 parameters: - name: from in: query description: >- Start time. A date-time string as defined by the ISO 8601 date format. This date-time will be used to calculate the starting day; the exact time will not be used. required: true schema: type: string example: '2023-01-01T12:00:00-05:00' - name: to in: query description: >- End time. A date-time string as defined by the ISO 8601 date format. This date-time will be used to calculate the ending day; the exact time will not be used. required: true schema: type: string example: '2023-01-01T12:00:00-05:00' - name: limit in: query description: Maximum amount of apps to retrieve. required: false schema: type: integer - name: retention in: query description: >- Limit report to apps that contribute to a variable retention pool. Value must be a string indicating the retention pool duration, e.g. `7d` for a seven day retention pool. required: false schema: type: string example: 7d responses: '200': description: OK content: application/json: schema: type: object properties: from: type: string description: The calculated starting time (exclusive) used internally. to: type: string description: The calculated ending time (exclusive) used internally. results: type: array items: type: object properties: name: type: string description: The name of the app. total_bytes: type: integer description: The number of bytes of disk the app consumes over the time range. '400': description: Bad request content: application/json: schema: type: object properties: code: type: string error: type: string description: Message indicating why the request is bad. status: type: string /v2/usage/apps/{name}: get: tags: - Usage v2 summary: Get Usage By App description: >- Gets the aggregated usage information for an app matching the name provided as a path parameter during a time period. The `results` will either have a singular item, or be an empty array. operationId: get-usage-app-v2 parameters: - name: name in: path description: The name of the app from which to get the aggregated usage data. required: true schema: type: string - name: from in: query description: >- Start time. A date-time string as defined by the ISO 8601 date format. This date-time will be used to calculate the starting day; the exact time will not be used. required: true schema: type: string example: '2023-01-01T12:00:00-05:00' - name: to in: query description: >- End time. A date-time string as defined by the ISO 8601 date format. This date-time will be used to calculate the ending day; the exact time will not be used. required: true schema: type: string example: '2023-01-01T12:00:00-05:00' - name: retention in: query description: >- Limit report to apps that contribute to a variable retention pool. Value must be a string indicating the retention pool duration, e.g. `7d` for a seven day retention pool. required: false schema: type: string example: 7d responses: '200': description: OK content: application/json: schema: type: object properties: from: type: string description: The calculated starting time (exclusive) used internally. to: type: string description: The calculated ending time (exclusive) used internally. results: type: array items: type: object properties: name: type: string description: The name of the app. total_bytes: type: integer description: The number of bytes of disk the app consumes over the time range. '400': description: Bad request content: application/json: schema: type: object properties: code: type: string error: type: string description: Message indicating why the request is bad. status: type: string /v2/usage/hosts: get: tags: - Usage v2 summary: List Usage by Host description: Lists aggregated usage information for all hosts during a time period. operationId: list-usage-host-v2 parameters: - name: from in: query description: >- Start time. A date-time string as defined by the ISO 8601 date format. This date-time will be used to calculate the starting day; the exact time will not be used. required: true schema: type: string example: '2023-01-01T12:00:00-05:00' - name: to in: query description: >- End time. A date-time string as defined by the ISO 8601 date format. This date-time will be used to calculate the ending day; the exact time will not be used. required: true schema: type: string example: '2023-01-01T12:00:00-05:00' - name: limit in: query description: Maximum amount of hosts to retrieve. required: false schema: type: integer - name: retention in: query description: >- Limit report to hosts that contribute to a variable retention pool. Value must be a string indicating the retention pool duration, e.g. `7d` for a seven day retention pool. required: false schema: type: string example: 7d responses: '200': description: OK content: application/json: schema: type: object properties: from: type: string description: The calculated starting time (exclusive) used internally. to: type: string description: The calculated ending time (exclusive) used internally. results: type: array items: type: object properties: name: type: string description: The name of the host. total_bytes: type: integer description: The number of bytes of disk the host consumes over the time range. '400': description: Bad request content: application/json: schema: type: object properties: code: type: string error: type: string description: Message indicating why the request is bad. status: type: string /v2/usage/hosts/{name}: get: tags: - Usage v2 summary: Get Usage by Host description: >- Gets the aggregated usage information for a host matching the name provided as a path parameter during a time period. The `results` will either have a singular item, or be an empty array. operationId: get-usage-host-v2 parameters: - name: name in: path description: The name of the host from which to get the aggregated usage data. required: true schema: type: string - name: from in: query description: >- Start time. A date-time string as defined by the ISO 8601 date format. This date-time will be used to calculate the starting day; the exact time will not be used. required: true schema: type: string example: '2023-01-01T12:00:00-05:00' - name: to in: query description: >- End time. A date-time string as defined by the ISO 8601 date format. This date-time will be used to calculate the ending day; the exact time will not be used. required: true schema: type: string example: '2023-01-01T12:00:00-05:00' - name: retention in: query description: >- Limit report to hosts that contribute to a variable retention pool. Value must be a string indicating the retention pool duration, e.g. `7d` for a seven day retention pool. required: false schema: type: string example: 7d responses: '200': description: OK content: application/json: schema: type: object properties: from: type: string description: The calculated starting time (exclusive) used internally. to: type: string description: The calculated ending time (exclusive) used internally. results: type: array items: type: object properties: name: type: string description: The name of the host. total_bytes: type: integer description: The number of bytes of disk the host consumes over the time range. '400': description: Bad request content: application/json: schema: type: object properties: code: type: string error: type: string description: Message indicating why the request is bad. status: type: string /v2/usage/tags: get: tags: - Usage v2 summary: List Usage by Tag description: Lists aggregated usage information for all tags during a time period. operationId: list-usage-tag-v2 parameters: - name: from in: query description: >- Start time. A date-time string as defined by the ISO 8601 date format. This date-time will be used to calculate the starting day; the exact time will not be used. required: true schema: type: string example: '2023-01-01T12:00:00-05:00' - name: to in: query description: >- End time. A date-time string as defined by the ISO 8601 date format. This date-time will be used to calculate the ending day; the exact time will not be used. required: true schema: type: string example: '2023-01-01T12:00:00-05:00' - name: limit in: query description: Maximum amount of tags to retrieve. required: false schema: type: integer - name: retention in: query description: >- Limit report to tags that contribute to a variable retention pool. Value must be a string indicating the retention pool duration, e.g. `7d` for a seven day retention pool. required: false schema: type: string example: 7d responses: '200': description: OK content: application/json: schema: type: object properties: from: type: string description: The calculated starting time (exclusive) used internally. to: type: string description: The calculated ending time (exclusive) used internally. results: type: array items: type: object properties: name: type: string description: The name of the tag. total_bytes: type: integer description: The number of bytes of disk the tag consumes over the time range. '400': description: Bad request content: application/json: schema: type: object properties: code: type: string error: type: string description: Message indicating why the request is bad. status: type: string /v2/usage/tags/{name}: get: tags: - Usage v2 summary: Get Usage by Tag description: >- Gets the aggregated usage information for a tag matching the name provided as a path parameter during a time period. The `results` will either have a singular item, or be an empty array. operationId: get-usage-tag-v2 parameters: - name: name in: path description: The name of the tag from which to get the aggregated usage data. required: true schema: type: string - name: from in: query description: >- Start time. A date-time string as defined by the ISO 8601 date format. This date-time will be used to calculate the starting day; the exact time will not be used. required: true schema: type: string example: '2023-01-01T12:00:00-05:00' - name: to in: query description: >- End time. A date-time string as defined by the ISO 8601 date format. This date-time will be used to calculate the ending day; the exact time will not be used. required: true schema: type: string example: '2023-01-01T12:00:00-05:00' - name: retention in: query description: >- Limit report to tags that contribute to a variable retention pool. Value must be a string indicating the retention pool duration, e.g. `7d` for a seven day retention pool. required: false schema: type: string example: 7d responses: '200': description: OK content: application/json: schema: type: object properties: from: type: string description: The calculated starting time (exclusive) used internally. to: type: string description: The calculated ending time (exclusive) used internally. results: type: array items: type: object properties: name: type: string description: The name of the tag. total_bytes: type: integer description: The number of bytes of disk the tag consumes over the time range. '400': description: Bad request content: application/json: schema: type: object properties: code: type: string error: type: string description: Message indicating why the request is bad. status: type: string components: securitySchemes: AccessToken: type: apiKey in: header name: Authorization description: |- The primary authentication method for the Mezmo API. Pass the key in the `Authorization` header in the format `Token `. For example: ``` curl -H 'Authorization: Token ' ``` This key is used for all APIs with the exception of ingest. servicekeyAuth: type: apiKey in: header name: servicekey description: | (Deprecated) This key is used for v1 and v2 apis with the exception of ingest. Service keys can be used by placing your service key in the `Authorization` header of your api call. *NOTE* Usage of the service key auth mechanism has been deprecated in favor of IAM Access Tokens. Service keys can no longer be created, but can still be used for v1 and v2 apis while the transition takes place enterpriseServiceKeyAuth: type: apiKey in: header name: enterprise-servicekey description: | When authorizing against our enterprise endpoints, they expect an enterprise service key instead of a regular organization service key. Some endpoints might expect both an enterprise service key and a regular organization service key. apikeyAuth: type: apiKey in: header name: apikey description: > (Deprecated) This key is used for ingestion. You can obtain it following [these instructions](https://docs.mezmo.com/docs/ingestion-key) and placing the key in the header of your call. *NOTE* Usage of the apikey auth mechanism has been deprecated in favor of IAM Access Tokens. Service keys can no longer be created, but can still be used for v1 and v2 apis while the transition takes place basicAuth: type: http scheme: basic description: > (Deprecated) As an alternative to authenticating with a header value, you can also use basic authentication. To use either `apiKey` or `servicekey` as basic authentication, simply pass the key as the username with no password. For example: ``` curl https://api.logdna.com/v1/config/view -u INSERT_API_KEY: ``` schemas: alert-config: type: object title: Alert Configuration description: Configure an alert to be triggered when conditions are met. additionalProperties: false properties: general: type: object title: General description: Basic properties that describe the alert. additionalProperties: false properties: name: type: string title: Name minLength: 1 maxLength: 200 description: The name of the alert description: type: string title: Description minLength: 1 maxLength: 1024 description: An optional description describing what the alert is for. required: - name evaluation: title: Evaluation description: >- Multiple events (Log or Metric) will be collected into a single event (via aggregation) over a specified time window. That event is what the alert conditions will evaluate against. This section specifies how that aggregation should work. For Log event types, a `custom` JavaScript operation is required because the event shape is not standardized. additionalProperties: type: string title: Operation description: Specifies the type of aggregation operation to use with the window type and duration. ui: display_type: list oneOf: - const: DEFAULT label: add description: >- Adds new values together if they are of the same type. For counters and gauges the values are added together. For sets, this is a union of the elements. For distribution, the samples are concatenated. For histograms, values in the same bucket will be added together. - const: SUM label: sum description: Values are added together. Works for the GAUGE and COUNT metric types. - const: MIN label: minimum description: >- Get the minimum value for a series of events. Works for the GAUGE and COUNT metric types. - const: MAX label: maximum description: >- Get the maximum value for a series of events. Works for the GAUGE and COUNT metric types. - const: AVG label: average description: Values are averaged together. Works for the GAUGE and COUNT metric types. - const: SET_INTERSECTION label: set intersection description: The aggregate produces a set of unique values. - const: DIST_CONCAT label: distribution concatenation description: The aggregate produces a list of values combined together. - const: CUSTOM label: custom description: Use a custom script to perform aggregation. allOf: - if: allOf: - type: object properties: event_type: enum: - log required: - event_type - type: object properties: alert_type: not: enum: - absence required: - alert_type then: properties: operation: title: Operation description: >- Specifies the type of aggregation operation to use with the window type and duration. ui: display_type: list oneOf: - label: custom description: Use a custom script to perform aggregation. enum: - CUSTOM default: CUSTOM group_by: type: array minItems: 1 maxItems: 50 uniqueItems: true title: Group By Field Paths description: >- When aggregating, group events based on matching values from each of these field paths. Supports nesting via dot-notation. items: type: string maxLength: 100 title: Field description: The dot-notated property path. errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ ui: default_hint: - .name - .namespace - .tags event_timestamp: type: string title: Event Timestamp description: >- The path to a field on the event that contains a epoch timestamp value. If an event does not have a timestamp field, events will be associated to the wall clock value when the event is processed. maxLength: 200 errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: default_hint: timestamp pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ required: - operation - if: allOf: - type: object properties: event_type: enum: - metric required: - event_type - type: object properties: alert_type: not: enum: - absence required: - alert_type then: properties: operation: title: Operation description: >- Specifies the type of aggregation operation to use with the window type and duration. ui: display_type: list oneOf: - label: add description: >- Adds new values together if they are of the same type. For counters and gauges the values are added together. For sets, this is a union of the elements. For distribution, the samples are concatenated. For histograms, values in the same bucket will be added together. enum: - DEFAULT - label: sum description: Values are added together. Works for the GAUGE and COUNT metric types. enum: - SUM - label: minimum description: >- Get the minimum value for a series of events. Works for the GAUGE and COUNT metric types. enum: - MIN - label: maximum description: >- Get the maximum value for a series of events. Works for the GAUGE and COUNT metric types. enum: - MAX - label: average description: Values are averaged together. Works for the GAUGE and COUNT metric types. enum: - AVG - label: set intersection description: The aggregate produces a set of unique values. enum: - SET_INTERSECTION - label: distribution concatenation description: The aggregate produces a list of values combined together. enum: - DIST_CONCAT - label: custom description: Use a custom script to perform aggregation. enum: - CUSTOM default: SUM event_timestamp: not: type: string title: Event Timestamp description: >- The path to a field on the event that contains a epoch timestamp value. If an event does not have a timestamp field, events will be associated to the wall clock value when the event is processed. maxLength: 200 errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ required: - operation - if: properties: operation: enum: - CUSTOM required: - operation then: properties: script: type: string title: Script description: >- A custom JavaScript function that will control the aggregation. At the time of flushing, this aggregation will become the emitted event. ui: display_type: code-editor options: language: js minLength: 1 maxLength: 5000 default: > // Receives the current event, metadata and an accumulator object. Logic can be performed // on the event properties and added to the accumulator object for later analysis. // The accumulator is persisted, and will become the emitted event when // conditions are true, OR if the time window naturally expires. Those same conditions // will also be evaluated against the emitted event to determine if the alert should // be triggered. function alertAggregation(accum, event, metadata) { // This function *must* return the accumulated event return accum } required: - script - if: properties: alert_type: enum: - threshold required: - alert_type then: properties: conditional: title: Alert Conditions description: >- Evaluate these conditions against the aggregated event, and trigger an alert if they are met. If the conditions are true, it will also flush the current aggregated event and end the time window. allOf: - $ref: '#/components/schemas/alert-condition-group-threshold' ui: display_type: rules-engine options: operators: alert-condition-group-threshold - if: properties: alert_type: enum: - change required: - alert_type then: properties: conditional: title: Alert Conditions description: >- Evaluate these conditions against the aggregated event, and trigger an alert if they are met. If the conditions are true, it will also flush the current aggregated event and end the time window. allOf: - $ref: '#/components/schemas/alert-condition-group-change' ui: display_type: rules-engine options: operators: alert-condition-group-change - if: properties: alert_type: enum: - absence required: - alert_type then: properties: conditional: title: Alert Conditions description: No conditions are necessary for alerting on the absence of events. not: anyOf: - $ref: '#/components/schemas/alert-condition-group-threshold' - $ref: '#/components/schemas/alert-condition-group-change' ui: display_type: rules-engine options: hidden: true operation: not: type: string title: Operation description: >- Specifies the type of aggregation operation to use with the window type and duration. ui: display_type: list oneOf: - label: add description: >- Adds new values together if they are of the same type. For counters and gauges the values are added together. For sets, this is a union of the elements. For distribution, the samples are concatenated. For histograms, values in the same bucket will be added together. enum: - DEFAULT - label: sum description: Values are added together. Works for the GAUGE and COUNT metric types. enum: - SUM - label: minimum description: >- Get the minimum value for a series of events. Works for the GAUGE and COUNT metric types. enum: - MIN - label: maximum description: >- Get the maximum value for a series of events. Works for the GAUGE and COUNT metric types. enum: - MAX - label: average description: >- Values are averaged together. Works for the GAUGE and COUNT metric types. enum: - AVG - label: set intersection description: The aggregate produces a set of unique values. enum: - SET_INTERSECTION - label: distribution concatenation description: The aggregate produces a list of values combined together. enum: - DIST_CONCAT - label: custom description: Use a custom script to perform aggregation. enum: - CUSTOM script: not: type: string title: Script description: >- A custom JavaScript function that will control the aggregation. At the time of flushing, this aggregation will become the emitted event. ui: display_type: code-editor options: language: js minLength: 1 maxLength: 5000 properties: alert_type: title: Alert Type description: Specifies what kind of alert this should be. ui: display_type: list errorMessage: must be one of the allowed alert types oneOf: - label: Threshold Alert description: Trigger an alert when the evaluation is higher or lower than a set value. enum: - threshold - label: Change Alert description: Trigger an alert when there is a change in percentage or value thresholds. enum: - change - label: Absence Alert description: Trigger an alert when there is an absence of an event. enum: - absence event_type: title: Event Type description: The type of event is either a Log event or a Metric event. ui: display_type: list errorMessage: must be one of the allowed event types oneOf: - label: Metric description: Alert configuration based on a Metric event enum: - metric - label: Log description: Alert configuration based on a Log event enum: - log group_by: type: array minItems: 1 maxItems: 50 uniqueItems: true title: Group By Field Paths description: >- When aggregating, group events based on matching values from each of these field paths. Supports nesting via dot-notation. items: type: string maxLength: 100 title: Field description: The dot-notated property path. errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ window_type: type: string title: Window Type description: >- Sliding windows can overlap, whereas tumbling windows are disjoint. For example, a tumbling window has a fixed time span and any events that fall within the "window duration" will be used in the aggregate. In a sliding window, the aggregation occurs every "window duration" seconds after an event is encountered. enum: - tumbling - sliding default: tumbling window_duration_minutes: type: integer title: Window Duration (minutes) description: The duration of the aggregation window in minutes. default: 5 minimum: 1 maximum: 1440 conditional: type: object additionalProperties: false properties: expressions: type: array items: type: object logical_operation: type: string enum: - AND - OR required: - alert_type - event_type - window_type - window_duration_minutes alert_payload: type: object title: Alert Payload description: Determine the content of the alert and where to send it. additionalProperties: false properties: service: title: Service description: >- Configure a service to receive the payload. Currently supports Slack, PagerDuty, Webhook or Mezmo (Log Analysis). Where allowed, messagetemplating can be used. To use a template, surround field names with double curly braces, e.g., {{.my_field}} or {{."my-object".thing}}. See [documentation](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) for more examples. additionalProperties: type: string title: URI format: uri errorMessage: format: Must be a valid URI description: >- The full URI where the payload is sent. This should include the protocol and host, but can also include the port, path, and any other valid part of a URI. minLength: 1 allOf: - if: properties: name: enum: - slack required: - name then: properties: uri: type: string title: URI format: uri errorMessage: format: Must be a valid URI description: >- The full URI where the payload is sent. This should include the protocol and host, but can also include the port, path, and any other valid part of a URI. minLength: 1 message_text: type: string title: Message minLength: 1 ui: display_type: data-template options: multiline: true description: The text to be sent as a Slack message. required: - uri - message_text - if: properties: name: enum: - pager_duty required: - name then: properties: uri: type: string title: URI format: uri errorMessage: format: Must be a valid URI description: >- The full URI where the payload is sent. This should include the protocol and host, but can also include the port, path, and any other valid part of a URI. minLength: 1 summary: type: string title: Summary description: Summarize the alert details for PagerDuty. minLength: 1 ui: display_type: data-template severity: type: string enum: - INFO - WARNING - ERROR - CRITICAL title: Severity description: The severity level of the alert. source: type: string title: Source description: The source of the alert. minLength: 1 ui: display_type: data-template routing_key: type: string title: Routing Key description: The service integration routing key. minLength: 1 ui: display_type: secret event_action: type: string enum: - trigger - acknowledge - resolve title: Event Action description: The event action associated with the alert. required: - uri - summary - severity - source - routing_key - event_action - if: properties: name: enum: - webhook required: - name then: properties: uri: type: string title: URI format: uri errorMessage: format: Must be a valid URI description: >- The full URI where the payload is sent. This should include the protocol and host, but can also include the port, path, and any other valid part of a URI. minLength: 1 method: type: string title: HTTP Method description: The HTTP method to use for the request. enum: - post - put - patch - delete - get - head - options - trace ui: default_hint: post message_text: type: string title: Message minLength: 1 ui: display_type: data-template options: multiline: true description: >- The Webhook payload to send. This can be a text string or stringified JSON. If the message can be parsed as JSON, it will be sent as such. headers: type: array title: HTTP Headers description: A list of key/value pairs to be included in the HTTP request. items: type: object title: Headers description: A key/value object describing a header name and its value. additionalProperties: false properties: header_name: type: string title: Header Name minLength: 1 description: The name of the header. header_value: type: string title: Header Value minLength: 1 maxLength: 1024 description: The value of the header. required: - header_name - header_value maxItems: 20 auth: title: Authentication Options description: Configures HTTP authentication allOf: - if: properties: strategy: enum: - basic then: properties: user: type: string title: User minLength: 1 description: The basic authentication user password: type: string title: Password minLength: 1 description: The basic authentication password ui: display_type: secret required: - user - password - if: properties: strategy: enum: - bearer then: properties: token: type: string title: Token minLength: 1 description: The bearer token ui: display_type: secret required: - token properties: strategy: type: string enum: - basic - bearer - none title: Strategy description: Choose basic or token-based authentication. default: none required: - uri - message_text - if: properties: name: enum: - log_analysis required: - name then: properties: severity: type: string enum: - INFO - WARNING - ERROR - CRITICAL title: Severity description: The severity level of the alert. default: INFO subject: type: string title: Subject description: A string to be used as the subject of the alert. minLength: 1 maxLength: 200 ui: display_type: data-template body: type: string title: Body description: A string to be used as the alert message body. minLength: 1 maxLength: 1024 ui: display_type: data-template ingestion_key: type: string title: Ingestion Key description: The key required to communicate to Log Analysis for sending the alert. ui: display_type: mezmo-ingestion-key minLength: 1 required: - severity - subject - body - ingestion_key properties: name: title: Name description: The name of the service that will receive the notification. ui: display_type: list errorMessage: must be a valid service name oneOf: - label: Slack enum: - slack - label: PagerDuty enum: - pager_duty - label: Webhook enum: - webhook - label: Mezmo Log Analysis enum: - log_analysis required: - name throttling: type: object title: Throttling description: Only allow a certain number of notifications to be sent within a given time window. properties: window_secs: type: integer title: Window Seconds description: >- The time frame during which the number of notifications (set by the Threshold) is permitted. default: 60 minimum: 1 threshold: type: integer title: Threshold description: >- The maximum number of notifications allowed over the given time window set by "Window Seconds" (default: 1). default: 1 minimum: 1 required: - service - throttling required: - general - evaluation - alert_payload alert-expression-threshold: title: Conditional Expression description: Define an expression for field comparison additionalProperties: type: - string - number allOf: - if: title: Operator properties: str_operator: type: string enum: - greater - greater_or_equal - less - less_or_equal ui: display_type: list-operators title: Operator description: The comparison operator required: - str_operator then: title: Value properties: value: errorMessage: pattern: Value must be a number or numeric string to use the provided operator type: - string - number title: Value description: The value used in the expression, ie the "operand" minLength: 1 pattern: ^\s*-?(\d*\.)?\d+\s*$ required: - value - if: title: Operator properties: str_operator: type: string enum: - greater - greater_or_equal - less - less_or_equal - equal - contains - is_ip_in_cidr_range - ends_with - starts_with - drain_matches - contains_token_term - contains_token_prefix - percent_change_greater - percent_change_greater_or_equal - percent_change_less - percent_change_less_or_equal - value_change_greater - value_change_greater_or_equal - value_change_less - value_change_less_or_equal ui: display_type: list-operators title: Operator description: The comparison operator required: - str_operator then: title: Value properties: value: type: - string - number title: Value description: The value used in the expression, ie the "operand" minLength: 1 required: - value - if: title: Operator properties: str_operator: type: string enum: - is_ip_in_cidr_range required: - str_operator then: properties: value: type: string minLength: 3 format: cidr_range errorMessage: Value must be an IPv4 or IPv6 CIDR range required: - value - if: title: Operator properties: str_operator: type: string enum: - is_metric required: - str_operator then: title: Report errors properties: report_errors: type: boolean description: Report diagnostic errors when a value is not a valid metric? - if: title: Operator properties: str_operator: type: string enum: - contains - equal - ends_with - starts_with - drain_matches - contains_token_term - contains_token_prefix ui: display_type: list-operators title: Operator required: - str_operator then: title: Case sensitive properties: case_sensitive: type: boolean title: Case sensitive description: Perform case sensitive comparison? - if: title: Operator properties: str_operator: type: string enum: - regex_match ui: display_type: list-operators title: Operator description: The comparison operator required: - str_operator then: title: Pattern properties: value: type: string title: Pattern description: The regular expression pattern to match against minLength: 1 case_sensitive: type: boolean title: Case sensitive description: Perform case sensitive matching crlf_newline: type: boolean title: Carriage return (CRLF) mode description: "When multiline mode is enabled, newline character is interpreted as \r\n" match_newline: type: boolean title: Dot all mode description: Dot matches newline character multiline: type: boolean title: Multiline mode description: ^ and $ match beginning and end of line respectively ignore_whitespace: type: boolean title: Verbose mode description: Ignore whitespace and allow comment lines required: - value properties: field: type: string title: Field description: The JSON field whose value will be used in the comparison pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference str_operator: type: string enum: - greater - greater_or_equal - less - less_or_equal - equal - contains - is_ip_in_cidr_range - is_metric - is_array - is_boolean - is_empty - is_null - is_number - is_object - is_string - exists - ends_with - starts_with - contains_token_term - contains_token_prefix - regex_match ui: display_type: list-operators title: Operator description: The comparison operator negate: type: boolean title: Negate description: Negate the comparison expression required: - field - str_operator alert-expression-change: title: Conditional Expression description: Define an expression for field comparison additionalProperties: type: - string - number allOf: - if: title: Operator properties: str_operator: type: string enum: - greater - greater_or_equal - less - less_or_equal ui: display_type: list-operators title: Operator description: The comparison operator required: - str_operator then: title: Value properties: value: errorMessage: pattern: Value must be a number or numeric string to use the provided operator type: - string - number title: Value description: The value used in the expression, ie the "operand" minLength: 1 pattern: ^\s*-?(\d*\.)?\d+\s*$ required: - value - if: title: Operator properties: str_operator: type: string enum: - greater - greater_or_equal - less - less_or_equal - equal - contains - is_ip_in_cidr_range - ends_with - starts_with - drain_matches - contains_token_term - contains_token_prefix - percent_change_greater - percent_change_greater_or_equal - percent_change_less - percent_change_less_or_equal - value_change_greater - value_change_greater_or_equal - value_change_less - value_change_less_or_equal ui: display_type: list-operators title: Operator description: The comparison operator required: - str_operator then: title: Value properties: value: type: - string - number title: Value description: The value used in the expression, ie the "operand" minLength: 1 required: - value - if: title: Operator properties: str_operator: type: string enum: - is_ip_in_cidr_range required: - str_operator then: properties: value: type: string minLength: 3 format: cidr_range errorMessage: Value must be an IPv4 or IPv6 CIDR range required: - value - if: title: Operator properties: str_operator: type: string enum: - is_metric required: - str_operator then: title: Report errors properties: report_errors: type: boolean description: Report diagnostic errors when a value is not a valid metric? - if: title: Operator properties: str_operator: type: string enum: - contains - equal - ends_with - starts_with - drain_matches - contains_token_term - contains_token_prefix ui: display_type: list-operators title: Operator required: - str_operator then: title: Case sensitive properties: case_sensitive: type: boolean title: Case sensitive description: Perform case sensitive comparison? - if: title: Operator properties: str_operator: type: string enum: - regex_match ui: display_type: list-operators title: Operator description: The comparison operator required: - str_operator then: title: Pattern properties: value: type: string title: Pattern description: The regular expression pattern to match against minLength: 1 case_sensitive: type: boolean title: Case sensitive description: Perform case sensitive matching crlf_newline: type: boolean title: Carriage return (CRLF) mode description: "When multiline mode is enabled, newline character is interpreted as \r\n" match_newline: type: boolean title: Dot all mode description: Dot matches newline character multiline: type: boolean title: Multiline mode description: ^ and $ match beginning and end of line respectively ignore_whitespace: type: boolean title: Verbose mode description: Ignore whitespace and allow comment lines required: - value properties: field: type: string title: Field description: The JSON field whose value will be used in the comparison pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference str_operator: type: string enum: - percent_change_greater - percent_change_greater_or_equal - percent_change_less - percent_change_less_or_equal - value_change_greater - value_change_greater_or_equal - value_change_less - value_change_less_or_equal ui: display_type: list-operators title: Operator description: The comparison operator negate: type: boolean title: Negate description: Negate the comparison expression required: - field - str_operator alert-condition-group-threshold: type: object title: Condition Group description: A group of expressions joined together by an and/or operator properties: expressions: type: array title: Expressions description: List of conditional expressions to be combined with the designated operator minItems: 1 items: anyOf: - $ref: '#/components/schemas/alert-condition-group-threshold' - $ref: '#/components/schemas/alert-expression-threshold' logical_operation: type: string enum: - AND - OR default: AND title: Logical Operation description: The logical operation (AND/OR) to be applied to the list of conditionals required: - expressions - logical_operation alert-condition-group-change: type: object title: Condition Group description: A group of expressions joined together by an and/or operator properties: expressions: type: array title: Expressions description: List of conditional expressions to be combined with the designated operator minItems: 1 items: anyOf: - $ref: '#/components/schemas/alert-condition-group-change' - $ref: '#/components/schemas/alert-expression-change' logical_operation: type: string enum: - AND - OR default: AND title: Logical Operation description: The logical operation (AND/OR) to be applied to the list of conditionals required: - expressions - logical_operation alert-evaluation: title: Evaluation description: >- Multiple events (Log or Metric) will be collected into a single event (via aggregation) over a specified time window. That event is what the alert conditions will evaluate against. This section specifies how that aggregation should work. For Log event types, a `custom` JavaScript operation is required because the event shape is not standardized. additionalProperties: type: string title: Operation description: Specifies the type of aggregation operation to use with the window type and duration. ui: display_type: list oneOf: - const: DEFAULT label: add description: >- Adds new values together if they are of the same type. For counters and gauges the values are added together. For sets, this is a union of the elements. For distribution, the samples are concatenated. For histograms, values in the same bucket will be added together. - const: SUM label: sum description: Values are added together. Works for the GAUGE and COUNT metric types. - const: MIN label: minimum description: >- Get the minimum value for a series of events. Works for the GAUGE and COUNT metric types. - const: MAX label: maximum description: >- Get the maximum value for a series of events. Works for the GAUGE and COUNT metric types. - const: AVG label: average description: Values are averaged together. Works for the GAUGE and COUNT metric types. - const: SET_INTERSECTION label: set intersection description: The aggregate produces a set of unique values. - const: DIST_CONCAT label: distribution concatenation description: The aggregate produces a list of values combined together. - const: CUSTOM label: custom description: Use a custom script to perform aggregation. allOf: - if: allOf: - type: object properties: event_type: enum: - log required: - event_type - type: object properties: alert_type: not: enum: - absence required: - alert_type then: properties: operation: title: Operation description: >- Specifies the type of aggregation operation to use with the window type and duration. ui: display_type: list oneOf: - label: custom description: Use a custom script to perform aggregation. enum: - CUSTOM default: CUSTOM group_by: type: array minItems: 1 maxItems: 50 uniqueItems: true title: Group By Field Paths description: >- When aggregating, group events based on matching values from each of these field paths. Supports nesting via dot-notation. items: type: string maxLength: 100 title: Field description: The dot-notated property path. errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ ui: default_hint: - .name - .namespace - .tags event_timestamp: type: string title: Event Timestamp description: >- The path to a field on the event that contains a epoch timestamp value. If an event does not have a timestamp field, events will be associated to the wall clock value when the event is processed. maxLength: 200 errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: default_hint: timestamp pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ required: - operation - if: allOf: - type: object properties: event_type: enum: - metric required: - event_type - type: object properties: alert_type: not: enum: - absence required: - alert_type then: properties: operation: title: Operation description: >- Specifies the type of aggregation operation to use with the window type and duration. ui: display_type: list oneOf: - label: add description: >- Adds new values together if they are of the same type. For counters and gauges the values are added together. For sets, this is a union of the elements. For distribution, the samples are concatenated. For histograms, values in the same bucket will be added together. enum: - DEFAULT - label: sum description: Values are added together. Works for the GAUGE and COUNT metric types. enum: - SUM - label: minimum description: >- Get the minimum value for a series of events. Works for the GAUGE and COUNT metric types. enum: - MIN - label: maximum description: >- Get the maximum value for a series of events. Works for the GAUGE and COUNT metric types. enum: - MAX - label: average description: Values are averaged together. Works for the GAUGE and COUNT metric types. enum: - AVG - label: set intersection description: The aggregate produces a set of unique values. enum: - SET_INTERSECTION - label: distribution concatenation description: The aggregate produces a list of values combined together. enum: - DIST_CONCAT - label: custom description: Use a custom script to perform aggregation. enum: - CUSTOM default: SUM event_timestamp: not: type: string title: Event Timestamp description: >- The path to a field on the event that contains a epoch timestamp value. If an event does not have a timestamp field, events will be associated to the wall clock value when the event is processed. maxLength: 200 errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ required: - operation - if: properties: operation: enum: - CUSTOM required: - operation then: properties: script: type: string title: Script description: >- A custom JavaScript function that will control the aggregation. At the time of flushing, this aggregation will become the emitted event. ui: display_type: code-editor options: language: js minLength: 1 maxLength: 5000 default: > // Receives the current event, metadata and an accumulator object. Logic can be performed // on the event properties and added to the accumulator object for later analysis. // The accumulator is persisted, and will become the emitted event when // conditions are true, OR if the time window naturally expires. Those same conditions // will also be evaluated against the emitted event to determine if the alert should // be triggered. function alertAggregation(accum, event, metadata) { // This function *must* return the accumulated event return accum } required: - script - if: properties: alert_type: enum: - threshold required: - alert_type then: properties: conditional: title: Alert Conditions description: >- Evaluate these conditions against the aggregated event, and trigger an alert if they are met. If the conditions are true, it will also flush the current aggregated event and end the time window. allOf: - $ref: '#/components/schemas/alert-condition-group-threshold' ui: display_type: rules-engine options: operators: alert-condition-group-threshold - if: properties: alert_type: enum: - change required: - alert_type then: properties: conditional: title: Alert Conditions description: >- Evaluate these conditions against the aggregated event, and trigger an alert if they are met. If the conditions are true, it will also flush the current aggregated event and end the time window. allOf: - $ref: '#/components/schemas/alert-condition-group-change' ui: display_type: rules-engine options: operators: alert-condition-group-change - if: properties: alert_type: enum: - absence required: - alert_type then: properties: conditional: title: Alert Conditions description: No conditions are necessary for alerting on the absence of events. not: anyOf: - $ref: '#/components/schemas/alert-condition-group-threshold' - $ref: '#/components/schemas/alert-condition-group-change' ui: display_type: rules-engine options: hidden: true operation: not: type: string title: Operation description: >- Specifies the type of aggregation operation to use with the window type and duration. ui: display_type: list oneOf: - label: add description: >- Adds new values together if they are of the same type. For counters and gauges the values are added together. For sets, this is a union of the elements. For distribution, the samples are concatenated. For histograms, values in the same bucket will be added together. enum: - DEFAULT - label: sum description: Values are added together. Works for the GAUGE and COUNT metric types. enum: - SUM - label: minimum description: >- Get the minimum value for a series of events. Works for the GAUGE and COUNT metric types. enum: - MIN - label: maximum description: >- Get the maximum value for a series of events. Works for the GAUGE and COUNT metric types. enum: - MAX - label: average description: Values are averaged together. Works for the GAUGE and COUNT metric types. enum: - AVG - label: set intersection description: The aggregate produces a set of unique values. enum: - SET_INTERSECTION - label: distribution concatenation description: The aggregate produces a list of values combined together. enum: - DIST_CONCAT - label: custom description: Use a custom script to perform aggregation. enum: - CUSTOM script: not: type: string title: Script description: >- A custom JavaScript function that will control the aggregation. At the time of flushing, this aggregation will become the emitted event. ui: display_type: code-editor options: language: js minLength: 1 maxLength: 5000 properties: alert_type: title: Alert Type description: Specifies what kind of alert this should be. ui: display_type: list errorMessage: must be one of the allowed alert types oneOf: - label: Threshold Alert description: Trigger an alert when the evaluation is higher or lower than a set value. enum: - threshold - label: Change Alert description: Trigger an alert when there is a change in percentage or value thresholds. enum: - change - label: Absence Alert description: Trigger an alert when there is an absence of an event. enum: - absence event_type: title: Event Type description: The type of event is either a Log event or a Metric event. ui: display_type: list errorMessage: must be one of the allowed event types oneOf: - label: Metric description: Alert configuration based on a Metric event enum: - metric - label: Log description: Alert configuration based on a Log event enum: - log group_by: type: array minItems: 1 maxItems: 50 uniqueItems: true title: Group By Field Paths description: >- When aggregating, group events based on matching values from each of these field paths. Supports nesting via dot-notation. items: type: string maxLength: 100 title: Field description: The dot-notated property path. errorMessage: pattern: >- Must be a valid data access syntax. [Syntax Reference](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) ui: display_type: field-reference pattern: >- ^(?:timestamp|annotations|metadata|message|\.|(?:metadata|message|annotations)?(?:\.\w+\w*(?:\[\d+\])?|(?:\."(?:.+?)+"(:?\[\d+\])?))+)$ window_type: type: string title: Window Type description: >- Sliding windows can overlap, whereas tumbling windows are disjoint. For example, a tumbling window has a fixed time span and any events that fall within the "window duration" will be used in the aggregate. In a sliding window, the aggregation occurs every "window duration" seconds after an event is encountered. enum: - tumbling - sliding default: tumbling window_duration_minutes: type: integer title: Window Duration (minutes) description: The duration of the aggregation window in minutes. default: 5 minimum: 1 maximum: 1440 conditional: type: object additionalProperties: false properties: expressions: type: array items: type: object logical_operation: type: string enum: - AND - OR required: - alert_type - event_type - window_type - window_duration_minutes notification-channel-sink: type: object title: Alert Payload description: Determine the content of the alert and where to send it. additionalProperties: false properties: service: title: Service description: >- Configure a service to receive the payload. Currently supports Slack, PagerDuty, Webhook or Mezmo (Log Analysis). Where allowed, messagetemplating can be used. To use a template, surround field names with double curly braces, e.g., {{.my_field}} or {{."my-object".thing}}. See [documentation](https://docs.mezmo.com/telemetry-pipelines/syntax-for-editing-pipeline-component-configuration-values#data-fields) for more examples. additionalProperties: type: string title: URI format: uri errorMessage: format: Must be a valid URI description: >- The full URI where the payload is sent. This should include the protocol and host, but can also include the port, path, and any other valid part of a URI. minLength: 1 allOf: - if: properties: name: enum: - slack required: - name then: properties: uri: type: string title: URI format: uri errorMessage: format: Must be a valid URI description: >- The full URI where the payload is sent. This should include the protocol and host, but can also include the port, path, and any other valid part of a URI. minLength: 1 message_text: type: string title: Message minLength: 1 ui: display_type: data-template options: multiline: true description: The text to be sent as a Slack message. required: - uri - message_text - if: properties: name: enum: - pager_duty required: - name then: properties: uri: type: string title: URI format: uri errorMessage: format: Must be a valid URI description: >- The full URI where the payload is sent. This should include the protocol and host, but can also include the port, path, and any other valid part of a URI. minLength: 1 summary: type: string title: Summary description: Summarize the alert details for PagerDuty. minLength: 1 ui: display_type: data-template severity: type: string enum: - INFO - WARNING - ERROR - CRITICAL title: Severity description: The severity level of the alert. source: type: string title: Source description: The source of the alert. minLength: 1 ui: display_type: data-template routing_key: type: string title: Routing Key description: The service integration routing key. minLength: 1 ui: display_type: secret event_action: type: string enum: - trigger - acknowledge - resolve title: Event Action description: The event action associated with the alert. required: - uri - summary - severity - source - routing_key - event_action - if: properties: name: enum: - webhook required: - name then: properties: uri: type: string title: URI format: uri errorMessage: format: Must be a valid URI description: >- The full URI where the payload is sent. This should include the protocol and host, but can also include the port, path, and any other valid part of a URI. minLength: 1 method: type: string title: HTTP Method description: The HTTP method to use for the request. enum: - post - put - patch - delete - get - head - options - trace ui: default_hint: post message_text: type: string title: Message minLength: 1 ui: display_type: data-template options: multiline: true description: >- The Webhook payload to send. This can be a text string or stringified JSON. If the message can be parsed as JSON, it will be sent as such. headers: type: array title: HTTP Headers description: A list of key/value pairs to be included in the HTTP request. items: type: object title: Headers description: A key/value object describing a header name and its value. additionalProperties: false properties: header_name: type: string title: Header Name minLength: 1 description: The name of the header. header_value: type: string title: Header Value minLength: 1 maxLength: 1024 description: The value of the header. required: - header_name - header_value maxItems: 20 auth: title: Authentication Options description: Configures HTTP authentication allOf: - if: properties: strategy: enum: - basic then: properties: user: type: string title: User minLength: 1 description: The basic authentication user password: type: string title: Password minLength: 1 description: The basic authentication password ui: display_type: secret required: - user - password - if: properties: strategy: enum: - bearer then: properties: token: type: string title: Token minLength: 1 description: The bearer token ui: display_type: secret required: - token properties: strategy: type: string enum: - basic - bearer - none title: Strategy description: Choose basic or token-based authentication. default: none required: - uri - message_text - if: properties: name: enum: - log_analysis required: - name then: properties: severity: type: string enum: - INFO - WARNING - ERROR - CRITICAL title: Severity description: The severity level of the alert. default: INFO subject: type: string title: Subject description: A string to be used as the subject of the alert. minLength: 1 maxLength: 200 ui: display_type: data-template body: type: string title: Body description: A string to be used as the alert message body. minLength: 1 maxLength: 1024 ui: display_type: data-template ingestion_key: type: string title: Ingestion Key description: The key required to communicate to Log Analysis for sending the alert. ui: display_type: mezmo-ingestion-key minLength: 1 required: - severity - subject - body - ingestion_key properties: name: title: Name description: The name of the service that will receive the notification. ui: display_type: list errorMessage: must be a valid service name oneOf: - label: Slack enum: - slack - label: PagerDuty enum: - pager_duty - label: Webhook enum: - webhook - label: Mezmo Log Analysis enum: - log_analysis required: - name throttling: type: object title: Throttling description: Only allow a certain number of notifications to be sent within a given time window. properties: window_secs: type: integer title: Window Seconds description: >- The time frame during which the number of notifications (set by the Threshold) is permitted. default: 60 minimum: 1 threshold: type: integer title: Threshold description: >- The maximum number of notifications allowed over the given time window set by "Window Seconds" (default: 1). default: 1 minimum: 1 required: - service - throttling password: type: string description: Password provided in credentials for authorization format: password hostName: type: string description: Host name of the source. mac: type: string description: The network mac address of the host computer. pattern: ^([0-9A-Fa-f]{2}[:-]){5}([0-9A-Fa-f]{2})$ example: 01:02:03:04:ab:cd ip: type: string description: The local IP address of the host computer (ipv4 or ipv6) example: 10.0.1.111 now: type: string description: >- Field format in date-time notation. The source UNIX timestamp in milliseconds at the time of the request. Used to calculate time drift. tags: type: string description: Tags that are used to dynamically group hosts. line: type: string description: Text of the log line. timestamp: type: string description: UNIX timestamp, including milliseconds, when the log entry was recorded. app: type: string description: Name of the application that generates the log line. level: type: string description: >- Set a value for the level. For example, sample values for this parameter are INFO, WARNING, ERROR. meta: type: string description: >- This field is reserved for custom information that is associated with a log line. To add metadata to an API call, specify the meta field under the lines object. Metadata can be viewed inside that line's context. viewId: type: string description: ID of a view. name: type: string description: Name of a view. from: type: string description: Start time. Set as UNIX timestamp in seconds or milliseconds. to: type: string description: End time. Set as UNIX timestamp in seconds or milliseconds. size: type: string description: Number of log lines to include in the export. hosts: type: string description: Comma-separated list of hosts. apps: type: string description: Comma-separated list of applications. levels: type: string description: Comma-separated list of log levels. query: type: string description: Search query. prefer: type: string description: >- Defines the log lines that you want to export. Valid values are head, first log lines, and tail, last log lines. If not specified, defaults to tail. email: type: string description: >- Specifies the email with the downloadable link of your export. By default, the log lines are streamed. ownerEmail: type: string description: The email of a user to be given owner privileges of an account. example: owner@example.org plan: type: string description: The name of a plan. example: enterprise emailSubject: type: string description: >- Use to set the subject of the email. Use to represent a space. For example, a sample value is Export logs. integration: type: string description: >- Defines the type of notification channel. Valid values are: `email`, `webhook`, `slack`, and `pagerduty` archiveintegration: type: string description: >- Defines the type of archive destination. Valid values are: `ibm`, `s3`, `gcs`, `dos`, `azblob`, and `swift` pagination_id: type: string description: >- ID that indicates which page of results to be retrieved. Leave empty for the initial export request. emails: type: string description: Comma-separated list of email addresses. url: type: string description: Webhook URL. key: type: string description: PagerDuty key. triggerlimit: type: string description: >- Specify the number of log lines that match the view's filtering and search criteria. When the number of log lines is reached, an alert is triggered. triggerinterval: type: string description: >- Specify how often to trigger an alert. Valid values are: 30 seconds, 1 minute, 5 minutes, 15 minutes, 30 minutes, 1 hour, 6 hours, 12 hours, 24 hours autoresolve: type: boolean description: >- Set to true if you want the set a condition to resolve the incident that was raised by this alert. This property only applies to PagerDuty. autoresolvelimit: type: string description: >- Specify the number of log lines that match the view's filtering and search criteria. When the number of log lines is reached, this incident will be set to resolved in PagerDuty. autoresolveinterval: type: string description: >- Interval of time to aggregate and check # of matched lines against the auto resolve limit. Valid values are: 30 seconds, 1 minute, 5 minutes, 15 minutes, 30 minutes, 1 hour, 6 hours, 12 hours, 24 hours, 25 hours. immediate: type: boolean description: >- Set to true if you want the trigger condition to be evaluated as soon as the triggerlimit is reached. terminal: type: boolean description: >- Set to true if you want the trigger condition to be evaluated after the time that you specify in the triggerinterval field is reached. operator: type: string description: 'Type of alert. Valid values are: `presence`and `absence`.' timezone: type: string description: >- Timezone used to report timestamps. This can be any value defined in the tz database https://en.wikipedia.org/wiki/List_of_tz_database_time_zones. example: America/Chicago, EST, Etc/UTC, etc... method: type: string description: WebHook method such as POST. enum: - POST - PUT - PATCH presetid: type: string description: ID of a preset alert. category: type: string description: ' Logical name that you can use to group views.' groupname: type: string description: Name of a log group. groupId: type: string description: ID a log group. ruleId: type: string description: ID an exclusion rule. accessscope: type: string description: Query that determines the data that is visible to members of a group. keyId: type: string description: The ID of the key example: 629e3f7e62295be60488fb26 keyValue: type: string description: A secret key which can be used to authenticate requests to Mezmo. example: 7b273eb8adc962f2711ad760730debb0 keyCreated: type: integer description: The time the key was created as a Unix timestamp in milliseconds example: 1654538253244 keyType: type: string description: The type of the key enum: - ingestion - service example: ingestion keyName: type: string description: A user friendly name for the key maxLength: 30 example: servergroup_0_ingestion memberEmail: type: string description: Email of a member example: testemail@gmail.com role: type: string description: role of member. Owner shall not be created or set through api enum: - owner - admin - readonly - member example: member rbacGroupId: type: string description: The ID of one of RBAC groups that belong to the account accountId: type: string description: Account number of the organization example: '0000000000' enterpriseId: type: string description: Unique Id of the enterprise organization example: '0000000000' retention: type: integer description: Current retention period selected for the child organization example: 7 retentionPeriods: type: array items: $ref: '#/components/schemas/retention' description: Available log retention options for the child organization headers: type: object properties: timestamp: $ref: '#/components/schemas/X-MY-HEADER' bodyTemplate: type: object properties: timestamp: $ref: '#/components/schemas/my_log_lines' X-MY-HEADER: type: string description: Header value my_log_lines: type: string description: Define the webhook body. channel: type: object properties: integration: $ref: '#/components/schemas/integration' emails: type: array items: $ref: '#/components/schemas/emails' url: $ref: '#/components/schemas/url' key: $ref: '#/components/schemas/key' triggerlimit: $ref: '#/components/schemas/triggerlimit' triggerinterval: $ref: '#/components/schemas/triggerinterval' immediate: $ref: '#/components/schemas/immediate' terminal: $ref: '#/components/schemas/terminal' operator: $ref: '#/components/schemas/operator' timezone: $ref: '#/components/schemas/timezone' method: $ref: '#/components/schemas/method' headers: $ref: '#/components/schemas/headers' bodyTemplate: $ref: '#/components/schemas/bodyTemplate' autoresolve: $ref: '#/components/schemas/autoresolve' autoresolveinterval: $ref: '#/components/schemas/autoresolveinterval' autoresolvelimit: $ref: '#/components/schemas/autoresolvelimit' configRequest: type: object properties: name: $ref: '#/components/schemas/name' query: $ref: '#/components/schemas/query' hosts: type: array items: $ref: '#/components/schemas/hosts' apps: type: array items: $ref: '#/components/schemas/apps' levels: type: array items: $ref: '#/components/schemas/levels' tags: type: array items: $ref: '#/components/schemas/tags' category: type: array items: $ref: '#/components/schemas/category' presetid: $ref: '#/components/schemas/presetid' channels: type: array items: $ref: '#/components/schemas/channel' required: - name x-examples: example-1: name: string query: string hosts: string apps: string levels: string tags: string category: - string presetid: string channels: - integration: string email: string key: string triggerlimit: string triggerinterval: string immediate: true terminal: true operator: string timezone: string method: string headers: timestamp: string bodyTemplate: timestamp: string presetRequest: type: object properties: name: $ref: '#/components/schemas/name' channels: type: array items: $ref: '#/components/schemas/channel' categoryName: type: object properties: name: $ref: '#/components/schemas/name' archivingConfigRequest: type: object properties: integration: $ref: '#/components/schemas/integration' bucket: $ref: '#/components/schemas/bucket' endpoint: $ref: '#/components/schemas/endpoint' apikey: $ref: '#/components/schemas/apikey' resourceinstanceid: $ref: '#/components/schemas/resourceinstanceid' accountname: $ref: '#/components/schemas/accountname' accountkey: $ref: '#/components/schemas/accountkey' projectid: $ref: '#/components/schemas/projectid' space: $ref: '#/components/schemas/space' accesskey: $ref: '#/components/schemas/accesskey' secretkey: $ref: '#/components/schemas/secretkey' authurl: $ref: '#/components/schemas/authurl' expires: $ref: '#/components/schemas/expires' username: $ref: '#/components/schemas/username' password: $ref: '#/components/schemas/password' tenantname: $ref: '#/components/schemas/tenantname' x-examples: example-1: integration: string bucket: string endpoint: string apikey: string resourceinstanceid: string brokers: minItems: 1 type: array description: List of URLs referencing Kafka broker connections via SASL_SSL items: minLength: 1 type: string topic: minLength: 1 type: string description: Name of the relevant Kafka topic category storing log records stream_user: minLength: 1 type: string description: Username in provided credentials to authorize access to brokers stream_password: minLength: 1 type: string description: Password in provided credentials to authorize access to brokers format: password streamConfigCommon: required: - brokers - topic - user type: object properties: brokers: $ref: '#/components/schemas/brokers' topic: $ref: '#/components/schemas/topic' user: $ref: '#/components/schemas/stream_user' status: type: string description: Enumerated (binary) string representing whether or not the stream config is active readOnly: true enum: - active - inactive streamConfigRequest: description: Contains settings needed to set up connection(s) to Kafka brokers allOf: - $ref: '#/components/schemas/streamConfigCommon' - required: - password type: object properties: password: $ref: '#/components/schemas/stream_password' streamExclusionRequest: required: - title type: object properties: hosts: type: array items: $ref: '#/components/schemas/hosts' apps: type: array items: $ref: '#/components/schemas/apps' title: minLength: 1 type: string active: type: boolean default: false indexonly: type: boolean default: true query: $ref: '#/components/schemas/query' id: minLength: 1 type: string readOnly: true description: '' x-examples: example-1: hosts: - host-1 - host-2 apps: - app-1 - app-2 title: title active: false query: query id: '0000000' groupsRequest: type: object properties: name: $ref: '#/components/schemas/groupname' accessScopes: $ref: '#/components/schemas/accessscope' ingestlines: type: object properties: lines: type: array items: type: object required: - line properties: timestamp: $ref: '#/components/schemas/timestamp' line: $ref: '#/components/schemas/line' app: $ref: '#/components/schemas/app' level: $ref: '#/components/schemas/level' meta: $ref: '#/components/schemas/meta' indexRateAlertWebhookConfig: type: object properties: url: $ref: '#/components/schemas/url' method: $ref: '#/components/schemas/method' headers: $ref: '#/components/schemas/headers' bodyTemplate: $ref: '#/components/schemas/bodyTemplate' indexRateAlertConfig: type: object properties: max_lines: type: integer description: The number of lines required in order to trigger the alert. max_z_score: type: integer description: >- The number of standard deviations above the 30 day average lines in order to trigger the alert. threshold_alert: type: string description: >- Determines if you want alerts to be triggered if both the max lines and standard deviation have been triggered, or if one of the thresholds has been reached. enum: - separate - both frequency: type: string description: >- Notify recipients once per hour, or once per day, (starting when the threshold is passed the first time) until the index rate drops below the thresholds. When the index rate drops below the thresholds, alerting stops. enum: - hourly - daily channels: type: object properties: email: type: array items: $ref: '#/components/schemas/emails' pagerduty: type: array items: $ref: '#/components/schemas/key' slack: type: array items: $ref: '#/components/schemas/url' webhook: type: array items: $ref: '#/components/schemas/indexRateAlertWebhookConfig' enabled: type: boolean indexRateAlertWebhookConfigResponse: type: object properties: url: $ref: '#/components/schemas/url' method: $ref: '#/components/schemas/method' headers: $ref: '#/components/schemas/headers' bodyTemplate: type: string indexRateAlertConfigResponse: type: object properties: max_lines: type: integer description: The number of lines required in order to trigger the alert. max_z_score: type: integer description: >- The number of standard deviations above the 30 day average lines in order to trigger the alert. threshold_alert: type: string description: >- Determines if you want alerts to be triggered if both the max lines and standard deviation have been triggered, or if one of the thresholds has been reached. enum: - separate - both frequency: type: string description: >- Notify recipients once per hour, or once per day, (starting when the threshold is passed the first time) until the index rate drops below the thresholds. When the index rate drops below the thresholds, alerting stops. enum: - hourly - daily channels: type: object properties: email: type: array items: $ref: '#/components/schemas/emails' pagerduty: type: array items: $ref: '#/components/schemas/key' slack: type: array items: $ref: '#/components/schemas/url' webhook: type: array items: $ref: '#/components/schemas/indexRateAlertWebhookConfigResponse' enabled: type: boolean errCode: type: string description: Entity used as an arbitrary classifier of the error encountered errMsg: type: string description: Error information surfaced to end user(s) in the response errValidation: type: object properties: message: type: string description: Message providing context on the rule / property that failed validation key: type: string description: Relevant property or rule in the schema associated with failure message description: Specific error details related to invalid / missing request parameters errPropsDefault: type: object properties: code: $ref: '#/components/schemas/errCode' error: $ref: '#/components/schemas/errMsg' errPropsValidation: type: object properties: code: $ref: '#/components/schemas/errCode' details: $ref: '#/components/schemas/errValidation' integrationArchiving: type: string description: Set to `ibm` to archive to IBM Cloud Object Storage bucket: type: string description: Name of the bucket. endpoint: type: string description: Private endpoint associated with the bucket. apikey: type: string description: API key that is needed to authenticate with the Cloud Object Storage service. resourceinstanceid: type: string description: CRN of the Cloud Object Storage instance ID where the bucket is available. accountname: type: string description: Account name accountkey: type: string description: Account key projectid: type: string description: Project ID space: type: string description: Space accesskey: type: string description: Access key secretkey: type: string description: Secret key authurl: type: string description: Auth URL expires: type: string description: Expires username: type: string description: Username tenantname: type: string description: Tenant name uuid4: type: string format: uuid4 example: 03290da0-79a9-4090-9189-efb853d049ec corsHeaders: type: object properties: access-control-allow-origin: type: string access-control-allow-methods: type: string access-control-allow-headers: type: string access-control-max-age: type: string keyResource: type: object properties: id: $ref: '#/components/schemas/keyId' key: $ref: '#/components/schemas/keyValue' name: $ref: '#/components/schemas/keyName' type: $ref: '#/components/schemas/keyType' created: $ref: '#/components/schemas/keyCreated' keysPut: type: object properties: name: $ref: '#/components/schemas/keyName' additionalProperties: false minProperties: 1 memberResource: type: object properties: email: $ref: '#/components/schemas/memberEmail' role: $ref: '#/components/schemas/role' groups: type: array items: $ref: '#/components/schemas/rbacGroupId' membersPut: type: object properties: role: $ref: '#/components/schemas/role' groups: type: array items: $ref: '#/components/schemas/rbacGroupId' additionalProperties: false minProperties: 1 enterpriseResource: type: object properties: account: $ref: '#/components/schemas/accountId' enterprise: $ref: '#/components/schemas/enterpriseId' retention: $ref: '#/components/schemas/retention' owner: $ref: '#/components/schemas/ownerEmail' retentionPeriods: $ref: '#/components/schemas/retentionPeriods' plan: $ref: '#/components/schemas/plan' servicekey: $ref: '#/components/schemas/keyValue' enterprisePut: type: object properties: retention: $ref: '#/components/schemas/retention' owner: $ref: '#/components/schemas/ownerEmail' additionalProperties: false minProperties: 1 boardRequest: type: object properties: account: $ref: '#/components/schemas/accountId' title: type: string category: type: array items: type: string errResponse: type: object properties: error: type: string description: A human-readable description of the error code: type: string description: A unique code for the type of error status: type: string description: The status of the request screenResource: type: object properties: id: type: string readOnly: true description: The identifier of this screen. account: type: string readOnly: true description: The account this screen belongs to. title: type: string description: The title of this screen. ratio: type: string enum: - '3:2' - '4:3' - '5:4' - '16:9' - '16:10' - '21:9' description: The width to height aspect ratio of this screen. rotate90Degrees: type: boolean description: >- Whether the screen is rotated 90 degrees. Setting to true effectively swaps the width to height ratio. background: type: string enum: - dark - light description: The background color to use when viewing the screen. categories: type: array items: type: string description: A list of string categories the screen belongs to. widgets: type: array items: type: object properties: id: type: string description: The id of this widget. type: type: string enum: - count - gauge - table - time-shifted-graph description: The type for this widget. operation: type: string enum: - count - min - max - avg - cum - diff - percentile-75 - percentile-85 - percentile-95 - percentile-99 description: >- The type of operation to perform on the field indicated in the “field” option. If the "field" option is set to all lines, only "count" can be used and other values may not display any results. duration: type: string oneOf: - enum: - hourly-compare - daily-compare - weekly-compare - iso-weekly-compare - enum: - hourly - daily - weekly - iso-weekly - 1m - 30m - 1h - 2h - 4h - 8h - 12h - 1d - 2d - 4d - 1w - 2w description: >- How far into the past to graph this widget. Time shifted graph widgets must use time shifted durations e.g. "iso-weekly-compare". All other widgets must use regular durations e.g. "hourly". field: type: string description: >- Which field is being operated on and displayed in this widget. Use the special field “__ALL__LINES__” to perform the operation on all lines. match: type: string description: >- A special matching operation done on each value of the field. If the field is a string type then this value is used for equivalence matching. For number types, the supported operators are ["*", ">N", "=N", "<=N"]. If the field is set the special type "__ALL_LINES__", this option is disabled. filter: type: string description: >- An extra query filter to run to match lines. Must be specified as a Mezmo line query e.g. “statusCode:>=400”. label: type: string description: A label that is displayed on the widget. position: type: object properties: start: type: array items: {} description: The top left corner of the widget as an [x, y] coordinate. end: type: array items: {} description: The bottom right corner of the widget as an [x, y] coordinate. required: - start - end description: >- The positional coordinates of a widget. Screens coordinates start at at the top left at [1, 1] and increase down and to the right of a screen. additionalProperties: false sort: type: string oneOf: - enum: - asc - desc x-required: true description: >- The order in which the rows of a table are sorted. This option is specific to the "table" widget. rows: type: integer oneOf: - enum: - 3 - 5 - 10 - 15 - 20 - 25 x-required: true description: >- How many rows to include for the table. This option is specific to the "table" widget. leftLabel: type: string oneOf: - {} description: >- Label to display on the left side of a table. This option is specific to the "table" widget. rightLabel: type: string oneOf: - {} description: >- Label to display on the right side of a table. This option is specific to the "table" widget. showBarChart: type: boolean oneOf: - x-required: true description: >- Whether the rows of a table should also be displayed as a bar chart. This option is specific to the "table" widget. groupBy: type: string oneOf: - x-required: true description: >- An additional field with which to group results. This option is specific to the "table" widget. min: type: number format: float oneOf: - x-required: true description: The minimum bounds of the gauge. This option is specific to the "gauge" widget. max: type: number format: float oneOf: - x-required: true description: The maximum bounds of the gauge. This option is specific to the "gauge" widget. units: type: string oneOf: - {} description: >- A label for the units of this widget. This option is specific to the "count" and "gauge" widgets. format: type: string oneOf: - enum: - number-integer - number-tenths - number-ten-thousandths - number-unformatted - currency-prefix - currency-postfix - bytes-si - percent-integer - percent-thousandths - time-HH:mm:ss x-required: true description: >- How to format the results of this widget. This option is specific to the "count", "gauge", and "table" widgets. required: - id - type - operation - duration - field - position additionalProperties: false required: - id - title - ratio - rotate90Degrees - background additionalProperties: false screenListResource: type: object properties: id: type: string readOnly: true description: The identifier of this screen. account: type: string readOnly: true description: The account this screen belongs to. title: type: string description: The title of this screen. templateOperators: type: array items: type: object properties: type: type: string key: type: string keys: type: array items: type: string field: type: string start: type: string end: type: string ignoreEmptyElements: type: boolean delimiter: type: string preservePairs: type: array items: type: string operators: $ref: '#/components/schemas/templateOperators' example: - type: string key: string keys: - string field: string start: string end: string ignoreEmptyElements: true delimiter: string preservePairs: - string operators: - type: string key: string keys: - string field: string start: string end: string ignoreEmptyElements: true delimiter: string preservePairs: - string operators: - {} account: type: object title: Account description: |- Represents a platform level account registration, authentication and authorization. properties: account_id: type: string maxLength: 24 description: A value that uniquely identifies the account. example: 8aca3b99b4ee99db2c0fb6b7 enterprise_id: type: string maxLength: 24 description: A value that uniquely identifies the enterprise to which the account belongs. example: c0c41da6c8 account: type: string maxLength: 20 description: A shorthand identifier for the associated account example: b35eda6673 company: type: string maxLength: 250 description: Name of the company which owns the account example: Snippet Factory deactivated: type: string maxLength: 35 format: date-time nullable: true description: Indicates if the account has been deactivated or not owneremail: type: string maxLength: 320 format: email description: Email address of account owner example: admin@snippetfactory.com external_id: type: string maxLength: 255 default: null nullable: true description: And identifier that can be used to reference and id from a different system example: d615396c40 plan: type: object additionalProperties: true description: Billing plan and associated feature information pertaining properties: type: type: string minLength: 4 maxLength: 50 description: The new billing plan type to assign the target account once attached example: enterprise quota: type: object additionalProperties: true description: Quota information pertaining to the account properties: retention: type: integer minimum: 0 maximum: 90 format: int32 description: Data retention policy information pertaining to the account example: 14 retention: type: integer minimum: 0 maximum: 90 format: int32 description: Data retention policy information pertaining to the account status: type: string enum: - active - deactivated - pending description: Indication the account's current activation state created: type: string maxLength: 35 format: date-time description: Indicates when the account was created meta: type: object description: Metadata about the account additionalProperties: true required: - account_id - enterprise_id enterprise: type: object additionalProperties: false title: Enterprise description: An Enterprise represents a grouping of platform Accounts under a single entity. properties: enterprise_id: type: string maxLength: 32 description: A value that uniquely identifies the Enterprise. example: 6399e4ab6f66431300e1cf9a name: type: string maxLength: 255 description: A machine-friendly name for the Enterprise. example: Acme Inc. parent_enterprise_id: type: string maxLength: 32 description: The ID of the parent Enterprise, if any. example: e4b1070eae partner: type: string maxLength: 255 description: Name of the partner which owns the Enterprise. example: Example Inc. status: type: string minLength: 1 maxLength: 25 description: Indicates the status of the Enterprise. example: active accounts: type: array items: $ref: '#/components/schemas/account' description: The list of Accounts that belong to this Enterprise. maxItems: 3000 required: - enterprise_id - name - status account-owner: type: object title: Account Owner description: Represents the user that is considered the sole owner of an account properties: user_id: type: string maxLength: 24 description: Unique identifier which can be used to reference a specific user. example: 8602b169800306cb2122207e firstname: type: string maxLength: 50 description: Name of user given to them upon birth. example: John lastname: type: string maxLength: 50 description: Common family or surname of the user. example: Doe email: type: string format: email maxLength: 320 description: |- Used primarily as a means of authentication during login. Additionally, this is the address where platform emails intended for the user will be sent. example: member@snippetfactory.com name: type: string maxLength: 125 description: Full name of the user. Generally the combination of firstname and lastname. example: John Doe accounts: type: array items: type: string maxLength: 20 maxItems: 200 description: |- List of short account id which indicate all accounts under which a user is a member of. example: - 9fc4097ce3 - f4aba3b697 enterprise_ids: type: array items: type: string maxLength: 20 maxItems: 50 description: |- List of ids which indicate all enterprise accounts under which a user is a member of. example: - 1905a829ed - 810d90f7d9 verified: type: boolean description: |- Indication of wheather the user has completed the registration process and confirmed to be a legitimate permison. example: true account-bulk-operations-request: type: object title: Account Bulk Operations Request description: Request body for bulk operations on accounts properties: operations: type: array minItems: 1 maxItems: 100 description: List of bulk operations items: type: object title: Bulk Operation description: Represents a single bulk operation on an entity allOf: - if: properties: action: enum: - UPDATE - DELETE type: string then: required: - entity_id - if: properties: action: enum: - CREATE - UPDATE type: string then: required: - entity properties: operation_id: type: string title: Client generated id to associate operation with result description: |- Used for navigation only, not persisted in database. If not provided, the array index (as string) will be used in the response example: 4b1a1504fa47866dafe8b5d4 action: type: string enum: - CREATE - UPDATE - DELETE description: Operation type (enum, not extensible) entity_id: type: string title: Entity unique identifier description: |- The identifier of the entity for UPDATE and DELETE operations. MUST be null or omitted for CREATE operations example: 471ef1c4154b987a94509213 entity: $ref: '#/components/schemas/account-create-request' required: - action required: - operations bulk-operation-account: type: object title: Bulk Operation description: Represents a single bulk operation on an entity allOf: - if: properties: action: enum: - UPDATE - DELETE type: string then: required: - entity_id - if: properties: action: enum: - CREATE - UPDATE type: string then: required: - entity properties: operation_id: type: string title: Client generated id to associate operation with result description: |- Used for navigation only, not persisted in database. If not provided, the array index (as string) will be used in the response example: 4b1a1504fa47866dafe8b5d4 action: type: string enum: - CREATE - UPDATE - DELETE description: Operation type (enum, not extensible) entity_id: type: string title: Entity unique identifier description: |- The identifier of the entity for UPDATE and DELETE operations. MUST be null or omitted for CREATE operations example: 471ef1c4154b987a94509213 entity: $ref: '#/components/schemas/account-create-request' required: - action account-bulk-operations-response: type: object title: Account Bulk Operations Response description: Response body for bulk operations on accounts properties: status: type: string enum: - SUCCEEDED - FAILED - PARTIAL title: Overall status of the bulk operation operations: type: array title: List of bulk operations description: >- Results for each operation from the request. Should be in the same order as the request operations items: type: object title: Bulk Operation Result description: Represents the result of a single bulk operation on an entity properties: operation_id: type: string title: Matching operation_id or index from request description: |- Matches the operation_id from the request if provided, otherwise the index of the operation as a string example: 408efce8aa57bd0d8a861298 action: type: string enum: - CREATE - UPDATE - DELETE description: Operation type (enum, not extensible) entity_id: type: string title: Entity identifier description: Entity identifier for UPDATE / DELETE operations example: cb7b7c1f5100d2d5d31ad4f1 entity: $ref: '#/components/schemas/account' result: type: object title: Operation result description: Result details for a single operation properties: status: type: string enum: - SUCCEEDED - FAILED title: Operation result status detail: type: string title: Human readable message description: Detailed message about the operation result example: Account created successfully context: type: array title: Detailed error information description: |- List of objects providing additional context for errors or failures items: type: object title: Detailed error information properties: message: type: string title: Specific error details example: Account not found code: type: string title: Machine readable error code example: ECONTRACT required: - message - code required: - status - detail - context required: - operation_id - action required: - status - operations bulk-operation-result-account: type: object title: Bulk Operation Result description: Represents the result of a single bulk operation on an entity properties: operation_id: type: string title: Matching operation_id or index from request description: |- Matches the operation_id from the request if provided, otherwise the index of the operation as a string example: 408efce8aa57bd0d8a861298 action: type: string enum: - CREATE - UPDATE - DELETE description: Operation type (enum, not extensible) entity_id: type: string title: Entity identifier description: Entity identifier for UPDATE / DELETE operations example: cb7b7c1f5100d2d5d31ad4f1 entity: $ref: '#/components/schemas/account' result: type: object title: Operation result description: Result details for a single operation properties: status: type: string enum: - SUCCEEDED - FAILED title: Operation result status detail: type: string title: Human readable message description: Detailed message about the operation result example: Account created successfully context: type: array title: Detailed error information description: |- List of objects providing additional context for errors or failures items: type: object title: Detailed error information properties: message: type: string title: Specific error details example: Account not found code: type: string title: Machine readable error code example: ECONTRACT required: - message - code required: - status - detail - context required: - operation_id - action account-create-request: type: object title: Account description: |- Represents a platform level account registration, authentication and authorization. properties: company: type: string maxLength: 250 description: Name of the company which owns the account example: Snippet Factory owneremail: type: string maxLength: 320 format: email description: Email address of account owner example: admin@snippetfactory.com external_id: type: string maxLength: 255 default: null nullable: true description: And identifier that can be used to reference and id from a different system example: d615396c40 plan: type: object additionalProperties: true description: Billing plan and associated feature information pertaining properties: type: type: string minLength: 4 maxLength: 50 description: The new billing plan type to assign the target account once attached example: enterprise retention: type: integer minimum: 0 maximum: 90 format: int32 description: Data retention policy information pertaining to the account example: 7 enable_syslog: type: boolean default: false title: Enable Syslog Ingestion description: Enables syslog ingestion for the new account via custom ports. include_admins: type: boolean default: false title: Auto join all enterprise admins description: If true, all enterprise admins will be added as members of the child account pipeline: type: object additionalProperties: false title: Pipeline description: A Pipeline record represents a pipeline and its configuration for a given account. properties: id: type: string maxLength: 36 description: A value that uniquely identifies the Pipeline. example: 8b7ef846-a781-44da-a6f2-e42ade30a01b account_id: type: string maxLength: 10 description: The account ID associated with the Pipeline. example: '3243046218' config: type: object additionalProperties: true description: The configuration for the Pipeline. title: type: string maxLength: 255 description: A human-friendly name for the Pipeline. partition_id: type: string maxLength: 36 description: The partition ID associated with the Pipeline. created_at: type: string format: date-time description: The date and time when the Pipeline was created. updated_at: type: string format: date-time description: The date and time when the Pipeline was last updated. published_at: type: string format: date-time description: The date and time when the Pipeline was published. deployment_groups: type: array items: type: string maxLength: 32 description: The deployment groups associated with the Pipeline. published_revision_id: type: string maxLength: 36 description: The ID of the published revision of the Pipeline. example: b53844f6-b0ff-4754-84a7-19929bf19d92 deployed_revision_id: type: string maxLength: 36 description: The ID of the deployed revision of the Pipeline. example: 4b7d4aff-9356-4406-a06c-08a11e8e5378 loaded_revision_id: type: string maxLength: 36 description: The ID of the loaded revision of the Pipeline. example: be429863-0d54-476a-9b62-5e380bfeee40 processing_status: type: string maxLength: 25 description: The processing status of the Pipeline. origin: type: string maxLength: 25 description: The origin of the Pipeline. deploy_type: type: string maxLength: 25 description: The deployment type of the Pipeline. replay_config: type: array items: type: object additionalProperties: true description: The replay configuration for the Pipeline. required: - id - account_id pipeline-metric-usage: type: object additionalProperties: true title: PipelineMetricUsage description: >- A PipelineMetricUsage record represents the usage metric for a pipeline node for a given account. properties: edge_id: oneOf: - type: string maxLength: 36 examples: - c7e27454-e4c2-448d-be49-c9ab5e3187ec - type: 'null' description: The edge ID associated with the usage metric. pipeline_id: type: string maxLength: 36 description: A value that uniquely identifies the Pipeline. example: 02a0d09f-a32f-402f-95f9-4aaca45817ad pipeline_title: type: string description: The title of the Pipeline. account_id: type: string maxLength: 10 description: The account ID associated with the usage metric. example: b58fce2734 node_id: type: string maxLength: 36 description: The node ID associated with the usage metric. example: 072a04ec-31c1-4256-97d4-101d5bfd3ff9 node_title: type: string description: The title of the node. node_kind: type: string enum: - source - transform - sink description: The kind of the node (e.g. sink, source, transform). node_type: type: string description: The type of the node. deleted: type: boolean description: Whether the pipeline has been deleted. traffic: type: string description: The traffic direction (e.g. egress, ingress). at: type: string format: date-time description: The timestamp of the usage metric. byte_size: type: string description: The byte size of the usage metric. pipeline-metric-health: type: object additionalProperties: true title: PipelineMetricHealth description: >- A PipelineMetricHealth record represents the health metric for a pipeline for a given account. properties: pipeline_id: type: string maxLength: 36 description: A value that uniquely identifies the Pipeline. example: 53b12f99-1b0c-474c-a79b-4ad8fc2db362 pipeline_title: type: string description: The title of the Pipeline. account_id: type: string maxLength: 10 description: The account ID associated with the health metric. example: bbcccdcf15 sources: type: array items: type: object additionalProperties: true properties: node_id: type: string maxLength: 36 description: The node ID. example: 5e260cfa-d2ca-413a-93b0-f79c83509c5f node_title: oneOf: - type: string - type: 'null' description: The title of the node. node_kind: type: string description: The kind of the node. node_type: type: string description: The type of the node. traffic: type: string description: The traffic direction. logs: type: array items: type: object additionalProperties: true properties: level: type: string description: The log level (e.g. ERROR, WARN). edge_id: oneOf: - type: string maxLength: 36 examples: - 47fd24e6-9108-4eeb-a358-9e46ffd481d2 - type: 'null' description: The edge ID associated with the log entry. message: type: string description: The log message. is_stale: type: boolean description: Whether the log entry is stale. last_seen_at: type: string format: date-time description: The timestamp when this log entry was last seen. captured_data: oneOf: - type: object additionalProperties: true - type: 'null' description: Captured data associated with the log entry. description: Health log entries for the node. description: Source nodes of the pipeline. transforms: type: array items: type: object additionalProperties: true properties: node_id: type: string maxLength: 36 description: The node ID. example: 5e260cfa-d2ca-413a-93b0-f79c83509c5f node_title: oneOf: - type: string - type: 'null' description: The title of the node. node_kind: type: string description: The kind of the node. node_type: type: string description: The type of the node. traffic: type: string description: The traffic direction. logs: type: array items: type: object additionalProperties: true properties: level: type: string description: The log level (e.g. ERROR, WARN). edge_id: oneOf: - type: string maxLength: 36 examples: - 47fd24e6-9108-4eeb-a358-9e46ffd481d2 - type: 'null' description: The edge ID associated with the log entry. message: type: string description: The log message. is_stale: type: boolean description: Whether the log entry is stale. last_seen_at: type: string format: date-time description: The timestamp when this log entry was last seen. captured_data: oneOf: - type: object additionalProperties: true - type: 'null' description: Captured data associated with the log entry. description: Health log entries for the node. description: Transform nodes of the pipeline. transform_groups: type: array description: Transform group nodes of the pipeline. sinks: type: array items: type: object additionalProperties: true properties: node_id: type: string maxLength: 36 description: The node ID. example: 5e260cfa-d2ca-413a-93b0-f79c83509c5f node_title: oneOf: - type: string - type: 'null' description: The title of the node. node_kind: type: string description: The kind of the node. node_type: type: string description: The type of the node. traffic: type: string description: The traffic direction. logs: type: array items: type: object additionalProperties: true properties: level: type: string description: The log level (e.g. ERROR, WARN). edge_id: oneOf: - type: string maxLength: 36 examples: - 47fd24e6-9108-4eeb-a358-9e46ffd481d2 - type: 'null' description: The edge ID associated with the log entry. message: type: string description: The log message. is_stale: type: boolean description: Whether the log entry is stale. last_seen_at: type: string format: date-time description: The timestamp when this log entry was last seen. captured_data: oneOf: - type: object additionalProperties: true - type: 'null' description: Captured data associated with the log entry. description: Health log entries for the node. description: Sink nodes of the pipeline. pipeline-node: type: object additionalProperties: true title: PipelineNode description: >- A PipelineNode record represents the available node types, operators and alert schemas for building pipelines. properties: alert_schema: type: object additionalProperties: true node_alert_schema: type: object additionalProperties: true nodes: type: array items: type: object additionalProperties: true operators: type: object additionalProperties: true surge_alert_schema: type: object additionalProperties: true usage_alert_schema: type: object additionalProperties: true local_deploy_nodes: type: array items: type: object additionalProperties: true http-error-item: type: object title: HTTP Error Item additionalProperties: true properties: keyword: type: string maxLength: 30 description: An indication of the type of rule or error that caused the request to fail instance_path: type: string maxLength: 200 nullable: true default: null description: JSON Pointer to the location in the data instance schema_path: type: string maxLength: 200 nullable: true default: null description: JSON Pointer to the location of the failing keyword in the schema params: type: object additionalProperties: true description: Additional information about the error that can be used to generate error messages property_name: type: string maxLength: 100 nullable: true default: null description: Set for errors in `propertyNames` keyword schema message: type: string maxLength: 200 description: A message describing the nature of the error or failure required: - keyword - params - message http-error: type: object title: HTTP Error description: A normalized representation of an error that is returned from an http request properties: status: type: integer minimum: 300 maximum: 599 format: int32 description: The HTTP status code the error is associated with context: type: string description: The portion of the inbound request which triggered the validation failure enum: - body - querystring - params - headers code: type: string maxLength: 35 description: A textual classifier of the error or failure message: type: string maxLength: 200 description: A message describing the nature of the error or failure errors: type: array maxItems: 200 description: List of metadata objects describing validation failures or contextual information items: $ref: '#/components/schemas/http-error-item' required: - status - code - message validation-error: type: object title: Validation Error description: |- A normalized representation of an error that is returned from an http request in response to failed input validation properties: status: type: integer minimum: 300 maximum: 599 format: int32 description: The HTTP status code the error is associated with context: type: string description: The portion of the inbound request which triggered the validation failure enum: - body - querystring - params - headers code: type: string maxLength: 35 description: A textual classifier of the error or failure message: type: string maxLength: 200 description: A message describing the nature of the error or failure errors: type: array maxItems: 200 description: List of metadata objects describing validation failures or contextual information items: $ref: '#/components/schemas/http-error-item' required: - status - code - message legacy-validation-error: description: A normalized representation of an error that is returned from an http request type: object title: Validation Error (Legacy) properties: status: type: integer minimum: 300 maximum: 599 format: int32 description: The HTTP status code the error is associated with context: type: string description: The portion of the inbound request which triggered the validation failure enum: - body - querystring - params - headers code: type: string maxLength: 35 description: A textual classifier of the error or failure message: type: string maxLength: 200 description: A message describing the nature of the error or failure errors: type: array maxItems: 200 description: List of metadata objects describing validation failures items: additionalProperties: true type: object title: HTTP Error Item (Legacy) properties: keyword: type: string maxLength: 30 description: An indication of the type of rule or error that caused the request to fail params: type: object additionalProperties: true description: Additional information about the error that can be used to generate error messages message: type: string maxLength: 200 description: A message describing the nature of the error or failure schemaPath: type: string maxLength: 200 nullable: true default: null description: JSON Pointer to the location of the failing keyword in the schema instancePath: type: string maxLength: 200 nullable: true default: null description: JSON Pointer to the location in the data instance propertyName: type: string maxLength: 100 nullable: true default: null description: where the validation error occurred required: - keyword - params - message required: - status - code - message legacy-http-error-item: additionalProperties: true type: object title: HTTP Error Item (Legacy) properties: keyword: type: string maxLength: 30 description: An indication of the type of rule or error that caused the request to fail params: type: object additionalProperties: true description: Additional information about the error that can be used to generate error messages message: type: string maxLength: 200 description: A message describing the nature of the error or failure schemaPath: type: string maxLength: 200 nullable: true default: null description: JSON Pointer to the location of the failing keyword in the schema instancePath: type: string maxLength: 200 nullable: true default: null description: JSON Pointer to the location in the data instance propertyName: type: string maxLength: 100 nullable: true default: null description: where the validation error occurred required: - keyword - params - message responses: streamErrResponse401: description: Not Authorized content: '*/*': schema: $ref: '#/components/schemas/errPropsDefault' application/json: example: code: NotAuthorized error: Missing Credentials streamErrResponse500: description: Internal Server Error content: '*/*': schema: $ref: '#/components/schemas/errPropsDefault' application/json: example: code: ServerError error: Database operation failed to get stream configuration. streamResponse200: description: OK content: '*/*': schema: $ref: '#/components/schemas/streamConfigCommon' application/json: example: brokers: - broker-1.kafka.svc00.env.eventstreams.cloud.ibm.com:9093 topic: topic user: token status: active standardResponse200: description: OK content: application/json: schema: type: object properties: status: type: string example: OK usageResponse200: description: OK content: application/json: example: from: '2022-12-31T23:59:59.999Z' to: '2023-02-01T00:00:00.000Z' results: plan_bytes: 123000 restored_bytes: 1000 7d_vr_bytes: 1000 schema: type: object properties: from: type: string description: The calculated starting time (exclusive) used internally. to: type: string description: The calculated ending time (exclusive) used internally. results: type: object properties: plan_bytes: type: number restored_bytes: type: number xd_vr_bytes: type: number description: >- The number of bytes of logs stored for this variable retention type where x is 3, 7, 14, or 30 day usageResponseWORestored200: description: OK content: application/json: example: from: '2022-12-31T23:59:59.999Z' to: '2023-02-01T00:00:00.000Z' results: plan_bytes: 123000 schema: type: object properties: from: type: string description: The calculated starting time (exclusive) used internally. to: type: string description: The calculated ending time (exclusive) used internally. results: type: object properties: plan_bytes: type: number