> ## Documentation Index > Fetch the complete documentation index at: https://docs.mezmo.com/llms.txt > Use this file to discover all available pages before exploring further. # Update source > Updates a source in a pipeline ## OpenAPI ````yaml /apis/combined-mezmo-api.yaml put /v3/pipeline/{pipeline_id}/source/{pipeline_source_id} openapi: 3.1.0 info: title: Mezmo API description: >- *REST API for the Mezmo platform: send and export log data, build and manage telemetry pipelines, and administer enterprise accounts.* To use the inline testing functionality here, go into your Mezmo account and add the following address to your CORS Origins page: `https://docs.mezmo.com`. version: 3.0.0 contact: name: Mezmo email: support@mezmo.com url: https://mezmo.com/support servers: - url: https://api.mezmo.com security: - AccessToken: [] tags: - name: Account description: Perform account level actions - name: Alerting description: API to control alerts on pipelines - name: Archiving description: > >*Authentication*: You can find instructions on authentication [here](/log-analysis-api/ref#authentication). The LogDNA Archiving API enables you to programmatically configure your archiving instances via code. Specifically you can set up cold storage buckets to send your logs for long term storage. - name: Board description: > >*Authentication*: You can find instructions on authentication [here](/log-analysis-api/ref#authentication). The LogDNA's API's Board endpoints are used to create, get, list, and delete boards. - name: Configuration description: > >*Authentication*: You can find instructions on authentication [here](/log-analysis-api/ref#authentication). The LogDNA API's Configuration endpoints are used to create, update, and delete views, alerts, and categories. - name: Data Classification & Profiling description: API for classifying and profiling data - name: Edge - Local Deploy description: User routes that are available to use with local Edge instances - name: Enterprise description: Routes for management enterprise level resources - name: Enterprise v1 description: Legacy enterprise management endpoints, superseded by the Enterprise API. - name: Exclusions description: > >*Authentication*: You can find instructions on authentication [here](/log-analysis-api/ref#authentication). Control what you store by creating exclusion rules. New lines that match an exclusion rule will not be stored and will not count toward your storage quota. - name: Export description: > LogDNA provides two endpoints for exporting your log lines. Read the sections below to learn about each, and determine which is appropriate for your use case. ## Export API v1 The Export API v1 is used to export log lines in [JSONL](https://jsonlines.org/) format. Based on your specific plan, the maximum number of logs returned is limited to 10,000 / 20,000. There are two distinct functionalities: 1. Streaming desired log lines directly from the response as raw text data 2. Specifying an email to receive a link that can be used to the download the desired log lines > Note > When an email address is specified, the desired log lines will not be streamed to the response body. > The response will instead send a notification indicating that the request was successful and that the results > will be sent to the email address provided in the request. This entails archiving and uploading the desired logs > to a storage provider. The email includes a URI to download the logs. Be aware that access to this URI will expire > after 24-48 hours. ## Export API v2 The Export API v2 provides an enhancement over v1, allowing you to export any number of log lines using pagination and by-passing the plan-specific limit imposed in v1. By making (multiple) subsequent requests using the `pagination_id` parameter, you can retrieve logs in batches of JSON until all matching log lines have been returned. Each Export API v2 request is limited to 10,000 lines. ### Pagination Functionality Pagination is only necessary when the desired set of log lines is larger than 10,000, since this is the maximum number of results that can be returned in a given page. If an export does not exceed this size, the behavior is similar to v1 -- all of the logs will be returned in a single "page" and no further requests are needed. When an export does exceed this size, refer to the steps outlined below. This is the expected workflow to export a set (cardinality > 10,000) of log lines that requires pagination: 1. Send an initial request with valid query parameters to retrieve logs. Pass a `null` value for the `pagination_id` in the first request. 2. The JSON response contains 2 fields: `lines` and `pagination_id`. The former is an array containing the corresponding batch (the first batch in this case) of logs and the latter is a token used to retrieve the next page of results. 3. Send a subsequent request using the same initial query parameters and also passing in the token received in the response to the `pagination_id` parameter. This will retrieve the next page of results. 4. Repeat this request-response cycle, updating the `pagination_id` parameter with the token returned in every new response, until the response eventually returns a `null` value for the token. This indicates that the corresponding batch of logs is the last and that you have reached the final page. > Important Considerations > In subsequent requests to paginate through results, make sure none of the query parameters & values are modified from the initial request > (excluding `pagination_id`) > The tokens used in the `pagination_id` parameter are tied to a specific export and page so they can be used repeatedly in requests to return > deterministic results > "0" can be passed in to the `from` and `to` parameters which will be resolved to timestamps for a plan's retention boundary and the current > time respectively. This behavior is independent so the value can be passed interchangeably to the parameters -- resulting in 4 possible scenarios: > - [`from`: \, `to`: \] > - [`from`: 0 (retention boundary), `to`: \] > - [`from`: \, `to`: 0 (current time)] > - [`from`: 0 (retention boundary), `to`: 0 (current time)] - name: Groups description: >- Manage log groups that scope which data is accessible to members in that group. - name: Incident Mode description: Change or read state of pipeline (incident mode on/off) - name: Index Rate Alert description: Configure alerts that fire when the log index rate crosses a threshold. - name: Ingestion description: Ingestion API - name: Keys description: Create and manage ingestion and service keys for an account. - name: Log Volume Reduction description: CRUD operations for log volume reduction - name: Members description: Manage organization members, their roles, and RBAC group assignments. - name: Metrics description: Usage metrics for pipelines - name: Parsing Template description: Manage custom log parsing templates. - name: Pipeline Export description: Export pipeline to terraform - name: Pipeline Management description: CRUD Operations for pipeline - name: Pipeline Nodes description: CRUD operations for sources, processors, and destinations - name: Processor Groups description: CRUD operations for Processor Groups - name: Sampling and Simulation description: >- Create samples of data and run them through a simulated pipeline without affecting your published pipeline - name: Screens description: Create and manage custom UI layouts for display on large format screens. - name: Shared Sources description: Create and manage sources for use across multiple pipelines - name: Suspensions description: > >*Authentication*: You can find instructions on authentication [here](/log-analysis-api/ref#authentication). The LogDNA Start/Stop Ingestion API allows users to programmatically manage their data ingestion status. Users can get the current status of their ingestion, suspend their ingestion, and resume their ingestion. To prevent an accidental API call, the suspension of ingestion requires two subsequent API calls. > You can find instructions on authentication [here](/log-analysis-api/ref#authentication). - name: Usage v1 description: > >*Authentication*: You can find instructions on authentication [here](/log-analysis-api/ref#authentication). The Usage API provides endpoints for retrieving aggregated usage information for applications, hosts and tags during a time period. While the `to` and `from` times are unix timestamps, the report is granular by day. Therefore only the day portion is used to establish a date range. ### How is **Usage** defined? This API defines usage as a percentage of the total number of lines ingested by the account within the defined period of time. For example, an app called `myapp` that returns a percentage of `15` in the response object has 15 percent of their logs coming from `myapp`. - name: Usage v2 description: > >*Authentication*: You can find instructions on authentication [here](/log-analysis-api/ref#authentication). The Usage API provides endpoints for retrieving aggregated usage information for applications, hosts and tags during a time period. While the `to` and `from` times are unix timestamps, the report is granular by day. Therefore only the day portion is used to establish a date range. ### How is **Usage** defined? This API defines usage as the number of bytes used by the log lines stored on disk by the account within the defined period of time. For example, an app called `myapp` that returns `150,000` total bytes in the response object indicates that `myapp` is consuming `150,000` bytes of disk storage over the time range. externalDocs: url: https://docs.mezmo.com paths: /v3/pipeline/{pipeline_id}/source/{pipeline_source_id}: put: tags: - Pipeline Nodes summary: Update source description: Updates a source in a pipeline parameters: - schema: type: string format: uuid in: path name: pipeline_id required: true - schema: type: string format: uuid in: path name: pipeline_source_id required: true requestBody: content: application/json: schema: type: object allOf: - if: properties: type: enum: - http deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: HTTP description: Receive data from incoming HTTP requests additionalProperties: false properties: capture_metadata: type: boolean title: Capture Metadata description: >- Enable the inclusion of all http headers and query string parameters that were sent from the source. ui: feature_flag: pipeline-capture-metadata display_type: hidden default: true auto_parse: type: object title: Configure automated parsing additionalProperties: false ui: display_type: hidden properties: enabled: type: boolean default: false enabled_event_types: type: array items: type: string enum: - HTTPD_COMBINEDLOG - HTTPD_COMMONLOG - HTTPD_ERRORLOG - SYSLOG5424LINE - SYSLOGLINE - SYSLOGPAMSESSION - CRONLOG - MONGO3_LOG - NAGIOSLOGLINE - POSTGRESQL - RAILS3 - RUBY_LOGGER - REDISLOG - REDISMONLOG - S3_ACCESS_LOG - ELB_ACCESS_LOG - CLOUDFRONT_ACCESS_LOG - CATALINALOG - TOMCATLOG - SQUID3 - BIND9 - HAPROXYTCP - HAPROXYHTTP - BACULA_LOGLINE - BRO_HTTP - BRO_DNS - BRO_CONN - BRO_FILES - NETSCREENSESSIONLOG - CISCO_TAGGED_SYSLOG - CISCOFW104001 - CISCOFW104002 - CISCOFW104003 - CISCOFW104004 - CISCOFW105003 - CISCOFW105004 - CISCOFW105005 - CISCOFW105008 - CISCOFW105009 - CISCOFW106001 - CISCOFW106006_106007_106010 - CISCOFW106014 - CISCOFW106015 - CISCOFW106021 - CISCOFW106023 - CISCOFW106100_2_3 - CISCOFW106100 - CISCOFW304001 - CISCOFW110002 - CISCOFW302010 - CISCOFW302013_302014_302015_302016 - CISCOFW302020_302021 - CISCOFW305011 - CISCOFW313001_313004_313008 - CISCOFW313005 - CISCOFW321001 - CISCOFW402117 - CISCOFW402119 - CISCOFW419001 - CISCOFW419002 - CISCOFW500004 - CISCOFW602303_602304 - CISCOFW710001_710002_710003_710005_710006 - CISCOFW713172 - CISCOFW733100 - SHOREWALL - SFW2 decoding: oneOf: - enum: - bytes - enum: - json - enum: - ndjson - enum: - auto ui: display_type: list errorMessage: must be equal to one of the allowed values description: >- The decoding method for converting frames into data events. title: Decoding Method default: auto required: - decoding - if: properties: type: enum: - splunk-hec deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Splunk HEC description: Receive Splunk logs additionalProperties: false properties: capture_metadata: type: boolean title: Capture Metadata description: >- Enable the inclusion of all http headers and query string parameters that were sent from the source. ui: feature_flag: pipeline-capture-metadata display_type: hidden default: true auto_parse: type: object title: Configure automated parsing additionalProperties: false ui: display_type: hidden properties: enabled: type: boolean default: false enabled_event_types: type: array items: type: string enum: - HTTPD_COMBINEDLOG - HTTPD_COMMONLOG - HTTPD_ERRORLOG - SYSLOG5424LINE - SYSLOGLINE - SYSLOGPAMSESSION - CRONLOG - MONGO3_LOG - NAGIOSLOGLINE - POSTGRESQL - RAILS3 - RUBY_LOGGER - REDISLOG - REDISMONLOG - S3_ACCESS_LOG - ELB_ACCESS_LOG - CLOUDFRONT_ACCESS_LOG - CATALINALOG - TOMCATLOG - SQUID3 - BIND9 - HAPROXYTCP - HAPROXYHTTP - BACULA_LOGLINE - BRO_HTTP - BRO_DNS - BRO_CONN - BRO_FILES - NETSCREENSESSIONLOG - CISCO_TAGGED_SYSLOG - CISCOFW104001 - CISCOFW104002 - CISCOFW104003 - CISCOFW104004 - CISCOFW105003 - CISCOFW105004 - CISCOFW105005 - CISCOFW105008 - CISCOFW105009 - CISCOFW106001 - CISCOFW106006_106007_106010 - CISCOFW106014 - CISCOFW106015 - CISCOFW106021 - CISCOFW106023 - CISCOFW106100_2_3 - CISCOFW106100 - CISCOFW304001 - CISCOFW110002 - CISCOFW302010 - CISCOFW302013_302014_302015_302016 - CISCOFW302020_302021 - CISCOFW305011 - CISCOFW313001_313004_313008 - CISCOFW313005 - CISCOFW321001 - CISCOFW402117 - CISCOFW402119 - CISCOFW419001 - CISCOFW419002 - CISCOFW500004 - CISCOFW602303_602304 - CISCOFW710001_710002_710003_710005_710006 - CISCOFW713172 - CISCOFW733100 - SHOREWALL - SFW2 - if: properties: type: enum: - s3 deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: AWS S3 description: Collect data from object contents in AWS S3 additionalProperties: false properties: sqs_queue_url: type: string title: SQS Queue URL description: >- The URL of a AWS SQS queue configured to receive S3 bucket notifications for the desired S3 buckets format: uri errorMessage: format: Must be a valid URI auth: type: object title: AWS Authentication description: Configures AWS authentication additionalProperties: false properties: access_key_id: type: string title: Access Key ID minLength: 1 description: The AWS access key id secret_access_key: type: string title: Secret Access Key minLength: 1 description: The AWS secret access key ui: display_type: secret required: - access_key_id - secret_access_key compression: type: string title: Compression description: The compression format of the S3 objects enum: - auto - gzip - none - zstd default: auto region: type: string title: Region minLength: 1 description: The name of the source AWS region. required: - sqs_queue_url - auth - compression - region - if: properties: type: enum: - sqs deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: AWS SQS description: Collect messages from AWS SQS additionalProperties: false properties: queue_url: type: string title: Queue URL description: The URL of an AWS SQS queue format: uri errorMessage: format: Must be a valid URI maxLength: 128 auth: type: object title: AWS Authentication description: Configures AWS authentication additionalProperties: false properties: access_key_id: type: string title: Access Key ID minLength: 1 description: The AWS access key id maxLength: 128 secret_access_key: type: string title: Secret Access Key minLength: 1 description: The AWS secret access key ui: display_type: secret maxLength: 128 required: - access_key_id - secret_access_key region: type: string title: Region minLength: 1 description: The name of the source AWS region. maxLength: 128 required: - queue_url - auth - region - if: properties: type: enum: - demo-logs deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: title: Demo Logs description: Generate log messages for testing and demonstration. additionalProperties: type: array title: Log Lines description: >- A list of user-provided log lines that will be sent through the pipeline infinitely. A maximum of 25 lines is allowed. The lines can be either plain text or ndjson. If ndjson is detected, the encoding for the source will be set to JSON, and lines that are not valid will show up as a visible component error. Otherwise, text encoding will be used. minItems: 1 maxItems: 25 items: type: string minLength: 1 maxLength: 4096 title: Line description: >- A single line of log data, either stringified JSON or plain text. ui: display_type: user-provided-lines allOf: - if: properties: format: enum: - user_provided required: - format then: properties: lines: type: array title: Log Lines description: >- A list of user-provided log lines that will be sent through the pipeline infinitely. A maximum of 25 lines is allowed. The lines can be either plain text or ndjson. If ndjson is detected, the encoding for the source will be set to JSON, and lines that are not valid will show up as a visible component error. Otherwise, text encoding will be used. minItems: 1 maxItems: 25 items: type: string minLength: 1 maxLength: 4096 title: Line description: >- A single line of log data, either stringified JSON or plain text. ui: display_type: user-provided-lines required: - lines - allOf: - errorMessage: >- `lines` is only valid when `format` is `user_provided` if: not: type: object properties: format: enum: - user_provided required: - format then: not: type: object properties: lines: type: array title: Log Lines description: >- A list of user-provided log lines that will be sent through the pipeline infinitely. A maximum of 25 lines is allowed. The lines can be either plain text or ndjson. If ndjson is detected, the encoding for the source will be set to JSON, and lines that are not valid will show up as a visible component error. Otherwise, text encoding will be used. minItems: 1 maxItems: 25 items: type: string minLength: 1 maxLength: 4096 title: Line description: >- A single line of log data, either stringified JSON or plain text. ui: display_type: user-provided-lines required: - lines properties: interval_ms: type: number title: Interval description: >- The amount of time to pause between each demo log. Fractions of a second are accepted to increase throughput. e.g. 500 ms == 120 lines per minute default: 1000 minimum: 500 maximum: 3600000 ui: display_type: milliseconds-to-human-readable format: title: Format description: >- The format of the log messages. All of the data is generated. ui: display_type: list errorMessage: must be one of the available types of demo data oneOf: - label: HTTP JSON description: HTTP request data in JSON format. enum: - json - label: Sensor JSON description: Data from environment sensors in JSON format. enum: - env_sensor - label: Financial JSON description: Financial data in JSON format. enum: - financial - label: Nginx Requests description: Nginx logs in text format. enum: - nginx - label: Apache Requests description: Apache request logs in text format. enum: - apache_common - label: Apache Errors description: Apache error logs in text format. enum: - apache_error - label: BSD Syslog description: >- Mock BSD syslog entries in rfc3164 text format. enum: - bsd_syslog - label: Syslog description: Mock syslog entries in rfc5424 text format. enum: - syslog - label: HTTP Metrics description: >- HTTP-based metrics in the Mezmo metrics JSON format. enum: - http_metrics - label: Generic Metrics description: >- Generic metrics in the Mezmo metrics JSON format enum: - generic_metrics - label: User-Provided Logs description: >- Custom logs provided by the user. These can either be plain text or serialized JSON. enum: - user_provided - label: Infrastructure Logs description: >- Infrastructure logs consisting of sysdig, nginx, HDFS and spark logs enum: - infrastructure - label: Kubernetes Logs description: Kubernetes logs in JSON format enum: - kubernetes default: json required: - format - if: properties: type: enum: - kinesis-firehose deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Kinesis Firehose description: Receive Kinesis Firehose data additionalProperties: false properties: capture_metadata: type: boolean title: Capture Metadata description: >- Enable the inclusion of all http headers and query string parameters that were sent from the source. ui: feature_flag: pipeline-capture-metadata display_type: hidden default: true auto_parse: type: object title: Configure automated parsing additionalProperties: false ui: display_type: hidden properties: enabled: type: boolean default: false enabled_event_types: type: array items: type: string enum: - HTTPD_COMBINEDLOG - HTTPD_COMMONLOG - HTTPD_ERRORLOG - SYSLOG5424LINE - SYSLOGLINE - SYSLOGPAMSESSION - CRONLOG - MONGO3_LOG - NAGIOSLOGLINE - POSTGRESQL - RAILS3 - RUBY_LOGGER - REDISLOG - REDISMONLOG - S3_ACCESS_LOG - ELB_ACCESS_LOG - CLOUDFRONT_ACCESS_LOG - CATALINALOG - TOMCATLOG - SQUID3 - BIND9 - HAPROXYTCP - HAPROXYHTTP - BACULA_LOGLINE - BRO_HTTP - BRO_DNS - BRO_CONN - BRO_FILES - NETSCREENSESSIONLOG - CISCO_TAGGED_SYSLOG - CISCOFW104001 - CISCOFW104002 - CISCOFW104003 - CISCOFW104004 - CISCOFW105003 - CISCOFW105004 - CISCOFW105005 - CISCOFW105008 - CISCOFW105009 - CISCOFW106001 - CISCOFW106006_106007_106010 - CISCOFW106014 - CISCOFW106015 - CISCOFW106021 - CISCOFW106023 - CISCOFW106100_2_3 - CISCOFW106100 - CISCOFW304001 - CISCOFW110002 - CISCOFW302010 - CISCOFW302013_302014_302015_302016 - CISCOFW302020_302021 - CISCOFW305011 - CISCOFW313001_313004_313008 - CISCOFW313005 - CISCOFW321001 - CISCOFW402117 - CISCOFW402119 - CISCOFW419001 - CISCOFW419002 - CISCOFW500004 - CISCOFW602303_602304 - CISCOFW710001_710002_710003_710005_710006 - CISCOFW713172 - CISCOFW733100 - SHOREWALL - SFW2 format: type: string enum: - json - text default: json title: Data Format description: >- This specifies what the data format will be after it is base64 decoded. If it is JSON, it will be automatically parsed. required: - format - if: properties: type: enum: - fluent deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Fluent (Fluentd, Fluent Bit) description: Receive data from Fluentd or Fluent Bit additionalProperties: false properties: capture_metadata: type: boolean title: Capture Metadata description: >- Enable the inclusion of all http headers and query string parameters that were sent from the source. ui: feature_flag: pipeline-capture-metadata display_type: hidden default: true auto_parse: type: object title: Configure automated parsing additionalProperties: false ui: display_type: hidden properties: enabled: type: boolean default: false enabled_event_types: type: array items: type: string enum: - HTTPD_COMBINEDLOG - HTTPD_COMMONLOG - HTTPD_ERRORLOG - SYSLOG5424LINE - SYSLOGLINE - SYSLOGPAMSESSION - CRONLOG - MONGO3_LOG - NAGIOSLOGLINE - POSTGRESQL - RAILS3 - RUBY_LOGGER - REDISLOG - REDISMONLOG - S3_ACCESS_LOG - ELB_ACCESS_LOG - CLOUDFRONT_ACCESS_LOG - CATALINALOG - TOMCATLOG - SQUID3 - BIND9 - HAPROXYTCP - HAPROXYHTTP - BACULA_LOGLINE - BRO_HTTP - BRO_DNS - BRO_CONN - BRO_FILES - NETSCREENSESSIONLOG - CISCO_TAGGED_SYSLOG - CISCOFW104001 - CISCOFW104002 - CISCOFW104003 - CISCOFW104004 - CISCOFW105003 - CISCOFW105004 - CISCOFW105005 - CISCOFW105008 - CISCOFW105009 - CISCOFW106001 - CISCOFW106006_106007_106010 - CISCOFW106014 - CISCOFW106015 - CISCOFW106021 - CISCOFW106023 - CISCOFW106100_2_3 - CISCOFW106100 - CISCOFW304001 - CISCOFW110002 - CISCOFW302010 - CISCOFW302013_302014_302015_302016 - CISCOFW302020_302021 - CISCOFW305011 - CISCOFW313001_313004_313008 - CISCOFW313005 - CISCOFW321001 - CISCOFW402117 - CISCOFW402119 - CISCOFW419001 - CISCOFW419002 - CISCOFW500004 - CISCOFW602303_602304 - CISCOFW710001_710002_710003_710005_710006 - CISCOFW713172 - CISCOFW733100 - SHOREWALL - SFW2 decoding: type: string description: >- The decoding method for converting frames into data events. enum: - bytes - json - ndjson title: Decoding Method default: json required: - decoding - if: properties: type: enum: - log-analysis deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Mezmo Log Analysis description: >- Receive data directly from your Mezmo Log Analysis account additionalProperties: false - if: properties: type: enum: - log-analysis-ingestion deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Mezmo Log Analysis Ingestion description: >- Receive unparsed source data directly from your Mezmo Log Analysis account. additionalProperties: type: string - if: properties: type: enum: - logstash deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Logstash description: Receive Logstash data additionalProperties: false properties: capture_metadata: type: boolean title: Capture Metadata description: >- Enable the inclusion of all http headers and query string parameters that were sent from the source. ui: feature_flag: pipeline-capture-metadata display_type: hidden default: true auto_parse: type: object title: Configure automated parsing additionalProperties: false ui: display_type: hidden properties: enabled: type: boolean default: false enabled_event_types: type: array items: type: string enum: - HTTPD_COMBINEDLOG - HTTPD_COMMONLOG - HTTPD_ERRORLOG - SYSLOG5424LINE - SYSLOGLINE - SYSLOGPAMSESSION - CRONLOG - MONGO3_LOG - NAGIOSLOGLINE - POSTGRESQL - RAILS3 - RUBY_LOGGER - REDISLOG - REDISMONLOG - S3_ACCESS_LOG - ELB_ACCESS_LOG - CLOUDFRONT_ACCESS_LOG - CATALINALOG - TOMCATLOG - SQUID3 - BIND9 - HAPROXYTCP - HAPROXYHTTP - BACULA_LOGLINE - BRO_HTTP - BRO_DNS - BRO_CONN - BRO_FILES - NETSCREENSESSIONLOG - CISCO_TAGGED_SYSLOG - CISCOFW104001 - CISCOFW104002 - CISCOFW104003 - CISCOFW104004 - CISCOFW105003 - CISCOFW105004 - CISCOFW105005 - CISCOFW105008 - CISCOFW105009 - CISCOFW106001 - CISCOFW106006_106007_106010 - CISCOFW106014 - CISCOFW106015 - CISCOFW106021 - CISCOFW106023 - CISCOFW106100_2_3 - CISCOFW106100 - CISCOFW304001 - CISCOFW110002 - CISCOFW302010 - CISCOFW302013_302014_302015_302016 - CISCOFW302020_302021 - CISCOFW305011 - CISCOFW313001_313004_313008 - CISCOFW313005 - CISCOFW321001 - CISCOFW402117 - CISCOFW402119 - CISCOFW419001 - CISCOFW419002 - CISCOFW500004 - CISCOFW602303_602304 - CISCOFW710001_710002_710003_710005_710006 - CISCOFW713172 - CISCOFW733100 - SHOREWALL - SFW2 format: type: string enum: - json - text default: json title: Data Format description: The format of the logstash data required: - format - if: properties: type: enum: - mezmo-agent deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Mezmo Agent description: Receive logs from the Mezmo Agent additionalProperties: false properties: capture_metadata: type: boolean title: Capture Metadata description: >- Enable the inclusion of all http headers and query string parameters that were sent from the source. ui: feature_flag: pipeline-capture-metadata display_type: hidden default: true auto_parse: type: object title: Configure automated parsing additionalProperties: false ui: display_type: hidden properties: enabled: type: boolean default: false enabled_event_types: type: array items: type: string enum: - HTTPD_COMBINEDLOG - HTTPD_COMMONLOG - HTTPD_ERRORLOG - SYSLOG5424LINE - SYSLOGLINE - SYSLOGPAMSESSION - CRONLOG - MONGO3_LOG - NAGIOSLOGLINE - POSTGRESQL - RAILS3 - RUBY_LOGGER - REDISLOG - REDISMONLOG - S3_ACCESS_LOG - ELB_ACCESS_LOG - CLOUDFRONT_ACCESS_LOG - CATALINALOG - TOMCATLOG - SQUID3 - BIND9 - HAPROXYTCP - HAPROXYHTTP - BACULA_LOGLINE - BRO_HTTP - BRO_DNS - BRO_CONN - BRO_FILES - NETSCREENSESSIONLOG - CISCO_TAGGED_SYSLOG - CISCOFW104001 - CISCOFW104002 - CISCOFW104003 - CISCOFW104004 - CISCOFW105003 - CISCOFW105004 - CISCOFW105005 - CISCOFW105008 - CISCOFW105009 - CISCOFW106001 - CISCOFW106006_106007_106010 - CISCOFW106014 - CISCOFW106015 - CISCOFW106021 - CISCOFW106023 - CISCOFW106100_2_3 - CISCOFW106100 - CISCOFW304001 - CISCOFW110002 - CISCOFW302010 - CISCOFW302013_302014_302015_302016 - CISCOFW302020_302021 - CISCOFW305011 - CISCOFW313001_313004_313008 - CISCOFW313005 - CISCOFW321001 - CISCOFW402117 - CISCOFW402119 - CISCOFW419001 - CISCOFW419002 - CISCOFW500004 - CISCOFW602303_602304 - CISCOFW710001_710002_710003_710005_710006 - CISCOFW713172 - CISCOFW733100 - SHOREWALL - SFW2 - if: properties: type: enum: - mezmo-datadog-source deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Datadog Agent description: >- Send logs and metrics data directly from an installed datadog agent additionalProperties: false properties: capture_metadata: type: boolean title: Capture Metadata description: >- Enable the inclusion of all http headers and query string parameters that were sent from the source. ui: feature_flag: pipeline-capture-metadata display_type: hidden default: true auto_parse: type: object title: Configure automated parsing additionalProperties: false ui: display_type: hidden properties: enabled: type: boolean default: false enabled_event_types: type: array items: type: string enum: - HTTPD_COMBINEDLOG - HTTPD_COMMONLOG - HTTPD_ERRORLOG - SYSLOG5424LINE - SYSLOGLINE - SYSLOGPAMSESSION - CRONLOG - MONGO3_LOG - NAGIOSLOGLINE - POSTGRESQL - RAILS3 - RUBY_LOGGER - REDISLOG - REDISMONLOG - S3_ACCESS_LOG - ELB_ACCESS_LOG - CLOUDFRONT_ACCESS_LOG - CATALINALOG - TOMCATLOG - SQUID3 - BIND9 - HAPROXYTCP - HAPROXYHTTP - BACULA_LOGLINE - BRO_HTTP - BRO_DNS - BRO_CONN - BRO_FILES - NETSCREENSESSIONLOG - CISCO_TAGGED_SYSLOG - CISCOFW104001 - CISCOFW104002 - CISCOFW104003 - CISCOFW104004 - CISCOFW105003 - CISCOFW105004 - CISCOFW105005 - CISCOFW105008 - CISCOFW105009 - CISCOFW106001 - CISCOFW106006_106007_106010 - CISCOFW106014 - CISCOFW106015 - CISCOFW106021 - CISCOFW106023 - CISCOFW106100_2_3 - CISCOFW106100 - CISCOFW304001 - CISCOFW110002 - CISCOFW302010 - CISCOFW302013_302014_302015_302016 - CISCOFW302020_302021 - CISCOFW305011 - CISCOFW313001_313004_313008 - CISCOFW313005 - CISCOFW321001 - CISCOFW402117 - CISCOFW402119 - CISCOFW419001 - CISCOFW419002 - CISCOFW500004 - CISCOFW602303_602304 - CISCOFW710001_710002_710003_710005_710006 - CISCOFW713172 - CISCOFW733100 - SHOREWALL - SFW2 - if: properties: type: enum: - pipeline-data-restoration deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Mezmo Pipeline Data Restoration description: >- Receive data directly from your Mezmo Pipline Data Restoration additionalProperties: false - if: properties: type: enum: - webhook deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Webhook (WebSub) description: >- Receive data from incoming webhooks using the WebSub protocol. additionalProperties: false properties: capture_metadata: type: boolean title: Capture Metadata description: >- Enable the inclusion of all http headers and query string parameters that were sent from the source. ui: feature_flag: pipeline-capture-metadata display_type: hidden default: true auto_parse: type: object title: Configure automated parsing additionalProperties: false ui: display_type: hidden properties: enabled: type: boolean default: false enabled_event_types: type: array items: type: string enum: - HTTPD_COMBINEDLOG - HTTPD_COMMONLOG - HTTPD_ERRORLOG - SYSLOG5424LINE - SYSLOGLINE - SYSLOGPAMSESSION - CRONLOG - MONGO3_LOG - NAGIOSLOGLINE - POSTGRESQL - RAILS3 - RUBY_LOGGER - REDISLOG - REDISMONLOG - S3_ACCESS_LOG - ELB_ACCESS_LOG - CLOUDFRONT_ACCESS_LOG - CATALINALOG - TOMCATLOG - SQUID3 - BIND9 - HAPROXYTCP - HAPROXYHTTP - BACULA_LOGLINE - BRO_HTTP - BRO_DNS - BRO_CONN - BRO_FILES - NETSCREENSESSIONLOG - CISCO_TAGGED_SYSLOG - CISCOFW104001 - CISCOFW104002 - CISCOFW104003 - CISCOFW104004 - CISCOFW105003 - CISCOFW105004 - CISCOFW105005 - CISCOFW105008 - CISCOFW105009 - CISCOFW106001 - CISCOFW106006_106007_106010 - CISCOFW106014 - CISCOFW106015 - CISCOFW106021 - CISCOFW106023 - CISCOFW106100_2_3 - CISCOFW106100 - CISCOFW304001 - CISCOFW110002 - CISCOFW302010 - CISCOFW302013_302014_302015_302016 - CISCOFW302020_302021 - CISCOFW305011 - CISCOFW313001_313004_313008 - CISCOFW313005 - CISCOFW321001 - CISCOFW402117 - CISCOFW402119 - CISCOFW419001 - CISCOFW419002 - CISCOFW500004 - CISCOFW602303_602304 - CISCOFW710001_710002_710003_710005_710006 - CISCOFW713172 - CISCOFW733100 - SHOREWALL - SFW2 - if: properties: type: enum: - kafka deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Kafka Source description: Collect events from Kafka compatible brokers additionalProperties: type: string properties: brokers: type: array title: Brokers uniqueItems: true minItems: 1 description: >- An array of key/val objects for each host/port of a Kafka broker. items: type: object title: Host/Port description: The host and port for the Kafka broker. additionalProperties: false properties: host: title: Host description: The hostname or IP address of the broker. anyOf: - type: string format: ipv4 errorMessage: format: Must be a valid IPv4 address - type: string format: hostname maxLength: 512 errorMessage: format: Must be a valid hostname port: type: number minimum: 1 maximum: 65535 title: Port description: The port that the broker listens on. required: - host - port topics: type: array title: Topics items: type: string title: Topic minLength: 1 maxLength: 256 description: The Kafka topic name. minItems: 1 description: List of Kafka topics names to read events from. group_id: type: string title: Consumer Group minLength: 1 description: >- The consumer group name to be used to consume events from Kafka. tls_enabled: type: boolean title: TLS enabled description: >- Whether or not to require TLS for outgoing connections. default: true sasl_enabled: type: boolean title: SASL/SCRAM Enabled description: Enable SASL/SCRAM authentication with Kafka default: true decoding_codec: type: string enum: - bytes - json title: Decoding description: Configures how events are decoded from raw bytes default: bytes if: properties: sasl_enabled: enum: - true then: properties: sasl_mechanism: type: string enum: - SCRAM-SHA-512 - SCRAM-SHA-256 - PLAIN title: SASL description: The Kafka SASL/SCRAM mechanism default: PLAIN sasl_username: type: string title: Username description: The SASL/SCRAM username minLength: 1 sasl_password: type: string title: Password description: The SASL/SCRAM username ui: display_type: secret minLength: 1 required: - sasl_mechanism - sasl_username - sasl_password required: - brokers - topics - group_id - decoding_codec - if: properties: type: enum: - azure-event-hub deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Azure Event Hub Source description: >- Collect events from Azure Event Hub using Azure Kafka Endpoint additionalProperties: false properties: connection_string: type: string title: Connection String minLength: 1 maxLength: 512 description: >- The Connection String as it appears in hub consumer SAS Policy ui: display_type: secret namespace: type: string title: Namespace minLength: 1 maxLength: 256 description: The Event Hub Namespace topics: type: array title: Event Hub Names items: type: string title: Name minLength: 1 maxLength: 256 description: Azure Event Hub instance name. minItems: 1 description: >- The list of Azure Event Hub name(s) to read events from. group_id: type: string title: Consumer Group minLength: 1 maxLength: 256 description: >- The consumer group name that this consumer belongs to. decoding_codec: type: string enum: - bytes - json title: Decoding description: Configures how events are decoded from raw bytes default: bytes required: - connection_string - namespace - topics - group_id - decoding_codec - if: properties: type: enum: - prometheus-remote-write deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: Prometheus Remote Write description: >- Receive prometheus remote write protobuf data from HTTP requests additionalProperties: false properties: capture_metadata: type: boolean title: Capture Metadata description: >- Enable the inclusion of all http headers and query string parameters that were sent from the source. ui: feature_flag: pipeline-capture-metadata display_type: hidden default: true auto_parse: type: object title: Configure automated parsing additionalProperties: false ui: display_type: hidden properties: enabled: type: boolean default: false enabled_event_types: type: array items: type: string enum: - HTTPD_COMBINEDLOG - HTTPD_COMMONLOG - HTTPD_ERRORLOG - SYSLOG5424LINE - SYSLOGLINE - SYSLOGPAMSESSION - CRONLOG - MONGO3_LOG - NAGIOSLOGLINE - POSTGRESQL - RAILS3 - RUBY_LOGGER - REDISLOG - REDISMONLOG - S3_ACCESS_LOG - ELB_ACCESS_LOG - CLOUDFRONT_ACCESS_LOG - CATALINALOG - TOMCATLOG - SQUID3 - BIND9 - HAPROXYTCP - HAPROXYHTTP - BACULA_LOGLINE - BRO_HTTP - BRO_DNS - BRO_CONN - BRO_FILES - NETSCREENSESSIONLOG - CISCO_TAGGED_SYSLOG - CISCOFW104001 - CISCOFW104002 - CISCOFW104003 - CISCOFW104004 - CISCOFW105003 - CISCOFW105004 - CISCOFW105005 - CISCOFW105008 - CISCOFW105009 - CISCOFW106001 - CISCOFW106006_106007_106010 - CISCOFW106014 - CISCOFW106015 - CISCOFW106021 - CISCOFW106023 - CISCOFW106100_2_3 - CISCOFW106100 - CISCOFW304001 - CISCOFW110002 - CISCOFW302010 - CISCOFW302013_302014_302015_302016 - CISCOFW302020_302021 - CISCOFW305011 - CISCOFW313001_313004_313008 - CISCOFW313005 - CISCOFW321001 - CISCOFW402117 - CISCOFW402119 - CISCOFW419001 - CISCOFW419002 - CISCOFW500004 - CISCOFW602303_602304 - CISCOFW710001_710002_710003_710005_710006 - CISCOFW713172 - CISCOFW733100 - SHOREWALL - SFW2 - if: properties: type: enum: - open-telemetry-metrics deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: OpenTelemetry Metrics description: Receive metrics from OpenTelemetry over HTTP additionalProperties: false properties: capture_metadata: type: boolean title: Capture Metadata description: >- Enable the inclusion of all http headers and query string parameters that were sent from the source. ui: feature_flag: pipeline-capture-metadata display_type: hidden default: true auto_parse: type: object title: Configure automated parsing additionalProperties: false ui: display_type: hidden properties: enabled: type: boolean default: false enabled_event_types: type: array items: type: string enum: - HTTPD_COMBINEDLOG - HTTPD_COMMONLOG - HTTPD_ERRORLOG - SYSLOG5424LINE - SYSLOGLINE - SYSLOGPAMSESSION - CRONLOG - MONGO3_LOG - NAGIOSLOGLINE - POSTGRESQL - RAILS3 - RUBY_LOGGER - REDISLOG - REDISMONLOG - S3_ACCESS_LOG - ELB_ACCESS_LOG - CLOUDFRONT_ACCESS_LOG - CATALINALOG - TOMCATLOG - SQUID3 - BIND9 - HAPROXYTCP - HAPROXYHTTP - BACULA_LOGLINE - BRO_HTTP - BRO_DNS - BRO_CONN - BRO_FILES - NETSCREENSESSIONLOG - CISCO_TAGGED_SYSLOG - CISCOFW104001 - CISCOFW104002 - CISCOFW104003 - CISCOFW104004 - CISCOFW105003 - CISCOFW105004 - CISCOFW105005 - CISCOFW105008 - CISCOFW105009 - CISCOFW106001 - CISCOFW106006_106007_106010 - CISCOFW106014 - CISCOFW106015 - CISCOFW106021 - CISCOFW106023 - CISCOFW106100_2_3 - CISCOFW106100 - CISCOFW304001 - CISCOFW110002 - CISCOFW302010 - CISCOFW302013_302014_302015_302016 - CISCOFW302020_302021 - CISCOFW305011 - CISCOFW313001_313004_313008 - CISCOFW313005 - CISCOFW321001 - CISCOFW402117 - CISCOFW402119 - CISCOFW419001 - CISCOFW419002 - CISCOFW500004 - CISCOFW602303_602304 - CISCOFW710001_710002_710003_710005_710006 - CISCOFW713172 - CISCOFW733100 - SHOREWALL - SFW2 - if: properties: type: enum: - open-telemetry-logs deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: OpenTelemetry Logs description: Receive logs from OpenTelemetry over HTTP additionalProperties: false properties: capture_metadata: type: boolean title: Capture Metadata description: >- Enable the inclusion of all http headers and query string parameters that were sent from the source. ui: feature_flag: pipeline-capture-metadata display_type: hidden default: true auto_parse: type: object title: Configure automated parsing additionalProperties: false ui: display_type: hidden properties: enabled: type: boolean default: false enabled_event_types: type: array items: type: string enum: - HTTPD_COMBINEDLOG - HTTPD_COMMONLOG - HTTPD_ERRORLOG - SYSLOG5424LINE - SYSLOGLINE - SYSLOGPAMSESSION - CRONLOG - MONGO3_LOG - NAGIOSLOGLINE - POSTGRESQL - RAILS3 - RUBY_LOGGER - REDISLOG - REDISMONLOG - S3_ACCESS_LOG - ELB_ACCESS_LOG - CLOUDFRONT_ACCESS_LOG - CATALINALOG - TOMCATLOG - SQUID3 - BIND9 - HAPROXYTCP - HAPROXYHTTP - BACULA_LOGLINE - BRO_HTTP - BRO_DNS - BRO_CONN - BRO_FILES - NETSCREENSESSIONLOG - CISCO_TAGGED_SYSLOG - CISCOFW104001 - CISCOFW104002 - CISCOFW104003 - CISCOFW104004 - CISCOFW105003 - CISCOFW105004 - CISCOFW105005 - CISCOFW105008 - CISCOFW105009 - CISCOFW106001 - CISCOFW106006_106007_106010 - CISCOFW106014 - CISCOFW106015 - CISCOFW106021 - CISCOFW106023 - CISCOFW106100_2_3 - CISCOFW106100 - CISCOFW304001 - CISCOFW110002 - CISCOFW302010 - CISCOFW302013_302014_302015_302016 - CISCOFW302020_302021 - CISCOFW305011 - CISCOFW313001_313004_313008 - CISCOFW313005 - CISCOFW321001 - CISCOFW402117 - CISCOFW402119 - CISCOFW419001 - CISCOFW419002 - CISCOFW500004 - CISCOFW602303_602304 - CISCOFW710001_710002_710003_710005_710006 - CISCOFW713172 - CISCOFW733100 - SHOREWALL - SFW2 - if: properties: type: enum: - open-telemetry-traces deploy_type: anyOf: - enum: - saas - type: 'null' required: - type then: properties: user_config: type: object title: OpenTelemetry Traces description: Receive traces from OpenTelemetry over HTTP additionalProperties: false properties: capture_metadata: type: boolean title: Capture Metadata description: >- Enable the inclusion of all http headers and query string parameters that were sent from the source. ui: feature_flag: pipeline-capture-metadata display_type: hidden default: true auto_parse: type: object title: Configure automated parsing additionalProperties: false ui: display_type: hidden properties: enabled: type: boolean default: false enabled_event_types: type: array items: type: string enum: - HTTPD_COMBINEDLOG - HTTPD_COMMONLOG - HTTPD_ERRORLOG - SYSLOG5424LINE - SYSLOGLINE - SYSLOGPAMSESSION - CRONLOG - MONGO3_LOG - NAGIOSLOGLINE - POSTGRESQL - RAILS3 - RUBY_LOGGER - REDISLOG - REDISMONLOG - S3_ACCESS_LOG - ELB_ACCESS_LOG - CLOUDFRONT_ACCESS_LOG - CATALINALOG - TOMCATLOG - SQUID3 - BIND9 - HAPROXYTCP - HAPROXYHTTP - BACULA_LOGLINE - BRO_HTTP - BRO_DNS - BRO_CONN - BRO_FILES - NETSCREENSESSIONLOG - CISCO_TAGGED_SYSLOG - CISCOFW104001 - CISCOFW104002 - CISCOFW104003 - CISCOFW104004 - CISCOFW105003 - CISCOFW105004 - CISCOFW105005 - CISCOFW105008 - CISCOFW105009 - CISCOFW106001 - CISCOFW106006_106007_106010 - CISCOFW106014 - CISCOFW106015 - CISCOFW106021 - CISCOFW106023 - CISCOFW106100_2_3 - CISCOFW106100 - CISCOFW304001 - CISCOFW110002 - CISCOFW302010 - CISCOFW302013_302014_302015_302016 - CISCOFW302020_302021 - CISCOFW305011 - CISCOFW313001_313004_313008 - CISCOFW313005 - CISCOFW321001 - CISCOFW402117 - CISCOFW402119 - CISCOFW419001 - CISCOFW419002 - CISCOFW500004 - CISCOFW602303_602304 - CISCOFW710001_710002_710003_710005_710006 - CISCOFW713172 - CISCOFW733100 - SHOREWALL - SFW2 properties: title: oneOf: - type: string minLength: 1 maxLength: 512 pattern: ^[^<>"]+$ - type: 'null' errorMessage: invalid title name description: oneOf: - type: string minLength: 1 pattern: ^[^<>"]+$ - type: 'null' errorMessage: invalid description type: type: string enum: - http - splunk-hec - s3 - sqs - demo-logs - kinesis-firehose - fluent - log-analysis - log-analysis-ingestion - logstash - mezmo-agent - mezmo-datadog-source - pipeline-data-restoration - webhook - kafka - azure-event-hub - prometheus-remote-write - open-telemetry-metrics - open-telemetry-logs - open-telemetry-traces deploy_type: type: string enum: - saas - local default: saas user_config: type: object generation_id: type: integer required: - type - user_config - generation_id required: true responses: '200': description: Default Response components: securitySchemes: AccessToken: type: apiKey in: header name: Authorization description: >- The primary authentication method for the Mezmo API. Pass the key in the `Authorization` header in the format `Token `. For example: ``` curl -H 'Authorization: Token ' ``` This key is used for all APIs with the exception of ingest. ````