> ## Documentation Index
> Fetch the complete documentation index at: https://docs.mezmo.com/llms.txt
> Use this file to discover all available pages before exploring further.

# AWS SQS

> Stream log data from an Amazon SQS queue as a pipeline source, including SQS setup, the required access policy, and IAM permissions.

## Description

The AWS SQS Source enables you to stream log data from an SQS topic as a Pipeline source via Amazon's Simple Queue Service (SQS).

## Configure SQS

1. Log in to the AWS console and navigate to **Simple Queue Service**.
2. On the Amazon SQS landing page, click **Create queue**.
3. For **Type**, make sure **Standard** is selected.
4. Enter a **Name** for the service, for example `mezmo-pipeline`.
5. Leave all the other settings at the default, and at the bottom of the page click **Create queue**.
6. On the **Queue details** page, select the **Access policy** tab.
7. Click **Edit** for the Access policy, and enter the JSON definition at the end of these instructions.
8. In the JSON definition, enter these values, then click **Save**.
   1. For `Resource`, enter the value for `ARN` in the **Details** section for SQS.
   2. For `aws:SourceARN`, navigate to your S3 bucket, open the **Properties** tab, and enter the value for `Amazon Resource Name (ARN).`
   3. For `aws:SourceAccount,` enter the value for your AWS user account. You can find your **Account ID** by selecting your account name in the upper-right corner of the AWS console.

### SQS Access Policy JSON Definition

```json theme={null}
{
    "Version": "2012-10-17",
    "Id": "example-ID",
    "Statement": [
        {
            "Sid": "example-statement-ID",
            "Effect": "Allow",
            "Principal": {
                "Service": "s3.amazonaws.com"
            },
            "Action": [
                "SQS:SendMessage"
            ],
            "Resource": "SQS-queue-ARN",
            "Condition": {
                "ArnLike": {
                    "aws:SourceArn": "arn:aws:s3:*:*:awsexamplebucket1"
                },
                "StringEquals": {
                    "aws:SourceAccount": "bucket-owner-account-id"
                 }
            }
        }
    ]
}

```

## AWS Access Key Required IAM Permissions

| Name                                                                                                                  | Purpose   |
| --------------------------------------------------------------------------------------------------------------------- | --------- |
| [`sqs:ReceiveMessage`](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/APIReference/API_ReceiveMessage.html) | Operation |
| [`sqs:DeleteMessage`](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/APIReference/API_DeleteMessage.html)   | Operation |

## Mezmo Source [Configuration](/telemetry-pipelines/s3-source#configuration)

1. Select the Amazon SQS Source in your Mezmo Telemetry Pipeline and select **Edit config.**
2. Navigate to your SQS queue and copy the **URL** in the **Details** section.
3. In the Amazon SWS Source configuration, copy the URL into the **SQS Queue URL** field.
4. Enter the information for your **AWS Authentication**, and the **Region** for your SQS Queue.
5. Select the Compression method to use for your SQS event notifications. The `Compression` setting of `auto` will attempt to use the `Content Encoding`, `Content-Type`, and `key` suffix to determine which encoding is in use. If the compression setting cannot be determined, it will default to `none`.

### Mezmo Configuration Options

| Option                                 | Description                                                                                                                    |
| -------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------ |
| SQS Queue URL                          | The URL of an AWS SQS queue configured to receive S3 bucket notifications for the S3 buckets where your source data is stored. |
| AWS Authentication - Access Key ID     | The access key ID for your AWS account.                                                                                        |
| AWS Authentication - Secret Access Key | The secret access key for your AWS account.                                                                                    |
| Compression                            | The compression format for the S3 objects.                                                                                     |
| Region                                 | The name of the region for the S3 bucket.                                                                                      |

### Included Metadata

| Field       | Format | Description                               |
| ----------- | ------ | ----------------------------------------- |
| `file_name` | String | This is the name of the file being pulled |
| `size`      | Number | Number of bytes to be pulled              |