OpenTelemetry Collector
Description
You can export your log data directly to Mezmo with the OpenTelemetry Exporter. If you are already using OpenTelemetry Collector, you can start sending log data by adding the Mezmo Exporter to your existing Pipelines. If you are new to OTEL, you can use this quick start example to get you up and running.
Example OpenTelemetry Exporter Setup
- Download the appropriate OTEL collector for your environment from the OpenTelemetry website.
- Create a new pipeline in Mezmo.
- Add an OpenTelemetry Logs Source node. Note the Pipeline URL and access key.
- Add an OpenTelemetry Metrics Source node. Note the Pipeline URL and access key.
- Add an OpenTelemetry Traces Source node. Note the Pipeline URL and access key.
- Create a configuration file
config.yaml
with these contents:
receivers
filelog#This will collect new lines from log files in /var/log
include /var/log/*.log
hostmetrics#This will collect metrics about the host on which the collector is running.
scrapers
cpu
disk
filesystem
load
memory
jaeger#This will expose an endpoint where you can send traces
protocols
grpc
endpoint"0.0.0.0:14250"
exporters
otlphttp/logs
endpoint"<endpoint from Logs node>"
headers
Authorization"<access key from Logs node>"
otlphttp/metrics
endpoint"<endpoint from Metrics node>"
headers
Authorization"<access key from Metrics node>"
otlphttp/traces
endpoint"<endpoint from Traces node>"
headers
Authorization"<access key from Traces node>"
processors
batch
service
pipelines
logs
receivers filelog
exporters otlphttp/logs
traces
receivers jaeger
processors batch
exporters otlphttp/traces
metrics
receivers hostmetrics
processors batch
exporters otlphttp/metrics
The endpoints in the exporters config must exactly match the URL provided by the OpenTelemetry Metrics / Logs / Traces sources, You don't need to specify the /v1/metrics
/v1/logs
/v1/traces
paths, which will otherwise be ignored by the Sources.
- Start the Collector with this command:
./otelcol-contrib --config /path/to/config.yaml
- The Collector will start sending logs, metrics and traces to your Pipeline.
For complete information on setting up and configuring the OpenTelemetry Collector, visit https://opentelemetry.io/docs/collector/configuration
Installing the Collector via a Helm Chart
You can install the OpenTelemetry collector directly in a Kubernetes cluster using a Helm chart.
If you don't have the OpenTelemetry repo added to your Helm list, you must add it first.
helm repo add open-telemetry https://open-telemetry.github.io/opentelemetry-helm-charts
Update your repos once you've added the OpenTelemetry repo.
helm repo update
Then you can install the chart to deploy the Collector. The example shown here installs as a single global endpoint for all data with a Splunk HEC format as indicated in the provided .yaml
configuration. You may also provide a configuration of your own like the one above.
To utilize this chart, create a Splunk HEC Source, and note the Endpoint and Token. As shown in this command, replace <YOUR PIPELINE INGEST ID HERE>
with the ID portion of endpoint (the part that follows /v1/
). Replace <YOUR TOKEN HERE>
with your provisioned token.
helm upgrade -i --create-namespace -n mezmo \
mezmo-otel-collector open-telemetry/opentelemetry-collector \
--values https://repo.mezmo.com/otel-collector/values.yaml \
--set global.mezmo.endpoint="<YOUR ROUTE ID>" \
--set global.mezmo.token="<YOUR TOKEN HERE>"
--set image.repository="otel/opentelemetry-collector-contrib"