AWS S3 Source Configuration

AWS S3 Source Configuration

1. Navigate to Pipelines: Go to the Pipelines page in the left-hand sidebar and select the pipeline you wish to edit.
2. Add Source: Click Add Source and choose the AWS S3 option.
3. Configuration Fields:

• Title: A descriptive name for the source (e.g., "S3-Logs-Prod").
• Description: Optional field for additional context.
• AWS SQS URL: The AWS SQS queue URL where data is stored (required for ingestion).
• AWS Access Key: Your AWS credentials (Access Key ID and Secret Access Key) from the AWS Dashboard.
• Compression: Default is Auto; can be set to GZipNone or ZSTD.
• Region: Choose an AWS region (default is US East 1).

4. Save: Click Save to activate the source.

Note: AWS SQS is required for S3 integration, as data is read from the queue.