Skip to content
Ayhan Sipahi Ayhan Sipahi

SNS/SQS Cross-Account Fan-Out: Building Multi-Account Event Distribution in AWS

Implement secure cross-account event distribution with Amazon SNS and SQS: IAM policies, KMS encryption, AWS CDK, and common production pitfalls.

Cross-account SNS/SQS fan-out lets one SNS topic in a publisher account deliver every event to SQS queues owned by other accounts, without either side handing over credentials. Most of the work is permissions: a topic policy, a queue policy, and, once encryption is in play, a KMS key policy that agrees with both.

Three defaults carry the design. Let the queue owner create the subscription, give every encrypted queue a customer-managed KMS key, and filter at the SNS subscription so consumers never pay for messages they would discard. CDK wiring, FIFO ordering, and cross-account dashboards all hang off those three.

What the Pattern Buys

In a multi-account AWS Organization, the events one team produces are rarely interesting only to that team. Once accounts are split by team, service, or environment, the work becomes sharing those events without punching a hole in the boundary that justified the split.

The pattern earns its place on four counts:

Administrative isolation: Each account maintains independent control over its resources. The billing team can’t accidentally delete the fulfillment team’s infrastructure, even though they both receive events from the same source.

Independent scaling: Consumer accounts scale their SQS processing independently. A slow consumer only backs up its own queue, in its own account, while the others keep processing.

Cost efficiency: SNS to SQS delivery is free, so you pay for SNS publishes and SQS operations only. Nothing is billed for the delivery leg itself, which is where HTTP/S subscribers are charged per delivery.

Security boundaries: Each account implements its own encryption, access policies, and compliance controls. A security team can enforce strict key management in its own account without asking the publisher to change anything.

Architecture Overview

Here’s how the cross-account fan-out pattern works:

Consumer Account C (333333333333)

Consumer Account B (222222222222)

Consumer Account A (111111111111)

Publisher Account (999999999999)

Subscribe

Subscribe

Subscribe

SNS Topic central-events

SQS Queue service-a-queue

Lambda Processor

SQS Queue service-b-queue

Lambda Processor

SQS Queue service-c-queue

Lambda Processor

The pattern requires proper configuration at three levels:

  1. SNS topic policy: Grants cross-account sns:Subscribe permission
  2. SQS queue policy: Allows SNS service principal to sqs:SendMessage
  3. KMS key policy (if encrypted): Permits SNS to encrypt/decrypt messages

IAM Policies and Permissions

Getting cross-account permissions right is critical. The following patterns work reliably in production.

SNS Topic Policy (Publisher Account)

The SNS topic must explicitly grant sns:Subscribe permission to target accounts:

import * as sns from 'aws-cdk-lib/aws-sns';
import * as iam from 'aws-cdk-lib/aws-iam';
import { Stack, StackProps } from 'aws-cdk-lib';
import { Construct } from 'constructs';

export class PublisherStack extends Stack {
  public readonly topic: sns.Topic;

  constructor(scope: Construct, id: string, props?: StackProps) {
    super(scope, id, props);

    // Create SNS topic
    this.topic = new sns.Topic(this, 'CentralEventTopic', {
      topicName: 'central-events',
      displayName: 'Central Event Distribution Topic',
    });

    // Grant cross-account subscribe permissions
    this.topic.addToResourcePolicy(
      new iam.PolicyStatement({
        sid: 'AllowCrossAccountSubscribe',
        effect: iam.Effect.ALLOW,
        principals: [
          new iam.AccountPrincipal('111111111111'), // Account A
          new iam.AccountPrincipal('222222222222'), // Account B
          new iam.AccountPrincipal('333333333333'), // Account C
        ],
        actions: ['sns:Subscribe'],
        resources: [this.topic.topicArn],
      })
    );

    // Optionally allow specific IAM roles instead of entire accounts
    // This is more restrictive and follows least-privilege principle
    this.topic.addToResourcePolicy(
      new iam.PolicyStatement({
        sid: 'AllowSpecificRoleSubscribe',
        effect: iam.Effect.ALLOW,
        principals: [
          new iam.ArnPrincipal('arn:aws:iam::111111111111:role/ServiceARole'),
        ],
        actions: ['sns:Subscribe'],
        resources: [this.topic.topicArn],
      })
    );
  }
}

Key considerations:

  • Use AccountPrincipal for organization-wide access or ArnPrincipal for specific roles
  • The sns:Subscribe action is required for creating subscriptions
  • This policy covers subscription creation only; publishing to the topic is a separate permission
  • You can add conditions to restrict by source VPC, IP range, or other factors

SQS Queue Policy (Consumer Account)

Each consumer account needs a queue policy allowing the SNS service principal to send messages:

import * as sqs from 'aws-cdk-lib/aws-sqs';
import * as sns from 'aws-cdk-lib/aws-sns';
import * as subscriptions from 'aws-cdk-lib/aws-sns-subscriptions';
import * as iam from 'aws-cdk-lib/aws-iam';
import { Duration, Stack, StackProps } from 'aws-cdk-lib';
import { Construct } from 'constructs';

interface ConsumerStackProps extends StackProps {
  centralTopicArn: string; // ARN from publisher account
}

export class ConsumerStack extends Stack {
  constructor(scope: Construct, id: string, props: ConsumerStackProps) {
    super(scope, id, props);

    // Create dead letter queue for failed messages
    const dlq = new sqs.Queue(this, 'EventDLQ', {
      queueName: 'events-dlq',
      retentionPeriod: Duration.days(14),
    });

    // Create main event queue
    const queue = new sqs.Queue(this, 'EventQueue', {
      queueName: 'service-events',
      visibilityTimeout: Duration.seconds(30),
      receiveMessageWaitTime: Duration.seconds(20), // Enable long polling
      deadLetterQueue: {
        queue: dlq,
        maxReceiveCount: 3,
      },
    });

    // Add queue policy allowing SNS to send messages
    queue.addToResourcePolicy(
      new iam.PolicyStatement({
        sid: 'AllowSNSPublish',
        effect: iam.Effect.ALLOW,
        principals: [new iam.ServicePrincipal('sns.amazonaws.com')],
        actions: ['sqs:SendMessage'],
        resources: [queue.queueArn],
        conditions: {
          ArnEquals: {
            'aws:SourceArn': props.centralTopicArn,
          },
        },
      })
    );

    // Import the cross-account SNS topic
    const centralTopic = sns.Topic.fromTopicArn(
      this,
      'CentralTopic',
      props.centralTopicArn
    );

    // Subscribe queue to topic
    centralTopic.addSubscription(
      new subscriptions.SqsSubscription(queue, {
        rawMessageDelivery: false, // Set to true to receive just the message body
      })
    );
  }
}

Important details:

  • The Condition with aws:SourceArn prevents other SNS topics from sending to your queue
  • rawMessageDelivery: false wraps the message in SNS metadata (recommended for debugging)
  • Set rawMessageDelivery: true if you only want the message body without SNS envelope
  • Long polling (receiveMessageWaitTime) reduces empty receives and costs
  • A subscription’s dead-letter queue must sit in the same account and Region as the subscription, so every consumer account owns its own DLQ

The Two-Way Handshake

Cross-account subscriptions require both accounts to agree:

  1. Publisher permits subscription: SNS topic policy grants sns:Subscribe to consumer account
  2. Consumer accepts messages: SQS queue policy allows SNS service principal to send messages
  3. Consumer creates subscription: Queue owner calls sns:Subscribe using the topic ARN

This handshake is not optional. A missing policy on either side surfaces the same “Access Denied”, so troubleshooting a dead subscription always means reading both accounts, not just the one that reported the error.

KMS Encryption Configuration

Encryption adds complexity to cross-account setups. AWS-managed keys don’t work across account boundaries, so an encrypted queue needs a customer-managed key.

Why AWS-Managed Keys Don’t Work

When you create an SQS queue with encryption using the AWS-managed key (alias/aws/sqs), the key policy only grants permissions within that account. The SNS service in the publisher account can’t use a consumer account’s AWS-managed key.

Customer-Managed Key Setup

Here’s a working pattern for encrypted queues:

import * as kms from 'aws-cdk-lib/aws-kms';
import * as sqs from 'aws-cdk-lib/aws-sqs';
import * as sns from 'aws-cdk-lib/aws-sns';
import * as subscriptions from 'aws-cdk-lib/aws-sns-subscriptions';
import * as iam from 'aws-cdk-lib/aws-iam';
import * as cdk from 'aws-cdk-lib';
import { Duration, Stack, StackProps } from 'aws-cdk-lib';
import { Construct } from 'constructs';

interface EncryptedConsumerStackProps extends StackProps {
  centralTopicArn: string;
}

export class EncryptedConsumerStack extends Stack {
  constructor(scope: Construct, id: string, props: EncryptedConsumerStackProps) {
    super(scope, id, props);

    // Create customer-managed KMS key
    const queueKey = new kms.Key(this, 'QueueEncryptionKey', {
      description: 'KMS key for cross-account SQS queue encryption',
      enableKeyRotation: true,
      removalPolicy: cdk.RemovalPolicy.RETAIN, // Don't delete keys
    });

    // Grant SNS service permission to use the key
    queueKey.addToResourcePolicy(
      new iam.PolicyStatement({
        sid: 'AllowSNSToUseKey',
        effect: iam.Effect.ALLOW,
        principals: [new iam.ServicePrincipal('sns.amazonaws.com')],
        actions: [
          'kms:Decrypt',
          'kms:GenerateDataKey',
        ],
        resources: ['*'],
        conditions: {
          StringEquals: {
            // Ensure SNS uses this key only for SQS in this region
            'kms:ViaService': `sqs.${this.region}.amazonaws.com`,
          },
        },
      })
    );

    // Create encrypted dead letter queue
    const dlq = new sqs.Queue(this, 'EncryptedEventDLQ', {
      queueName: 'encrypted-events-dlq',
      encryptionMasterKey: queueKey,
      retentionPeriod: Duration.days(14),
    });

    // Create encrypted main queue
    const queue = new sqs.Queue(this, 'EncryptedEventQueue', {
      queueName: 'encrypted-service-events',
      encryptionMasterKey: queueKey,
      visibilityTimeout: Duration.seconds(30),
      receiveMessageWaitTime: Duration.seconds(20),
      deadLetterQueue: {
        queue: dlq,
        maxReceiveCount: 3,
      },
    });

    // Queue policy allowing SNS to send messages
    queue.addToResourcePolicy(
      new iam.PolicyStatement({
        sid: 'AllowSNSPublish',
        effect: iam.Effect.ALLOW,
        principals: [new iam.ServicePrincipal('sns.amazonaws.com')],
        actions: ['sqs:SendMessage'],
        resources: [queue.queueArn],
        conditions: {
          ArnEquals: {
            'aws:SourceArn': props.centralTopicArn,
          },
        },
      })
    );

    // Import cross-account topic and subscribe
    const centralTopic = sns.Topic.fromTopicArn(
      this,
      'CentralTopic',
      props.centralTopicArn
    );

    centralTopic.addSubscription(
      new subscriptions.SqsSubscription(queue)
    );
  }
}

Key policy requirements:

  • kms:Decrypt: SNS needs this to decrypt messages when sending to the queue
  • kms:GenerateDataKey: Required for envelope encryption
  • kms:ViaService condition: Restricts key usage to SQS service in specific region
  • Enable key rotation for security best practices

Cost Consideration

Customer-managed KMS keys cost $1/month per key, plus $0.03 per 10,000 requests. Encrypted cross-account delivery has no free tier to fall back on, so budget one key per consumer account.

Message Filtering for Cost Optimization

SNS subscription filters reduce costs by preventing unwanted messages from reaching queues. Filtering happens at the SNS level before SQS charges apply.

Attribute-Based Filtering

Message attributes provide simple, efficient filtering:

// Publisher: Publish with attributes
import { SNSClient, PublishCommand } from '@aws-sdk/client-sns';

const sns = new SNSClient({ region: 'us-east-1' });

await sns.send(
  new PublishCommand({
    TopicArn: 'arn:aws:sns:us-east-1:999999999999:central-events',
    Message: JSON.stringify({
      orderId: '12345',
      amount: 1500,
      region: 'us-east-1',
    }),
    MessageAttributes: {
      eventType: {
        DataType: 'String',
        StringValue: 'OrderCreated',
      },
      priority: {
        DataType: 'String',
        StringValue: 'high',
      },
      amount: {
        DataType: 'Number',
        StringValue: '1500',
      },
      region: {
        DataType: 'String',
        StringValue: 'us-east-1',
      },
    },
  })
);
// Consumer: Subscribe with filter policy
import * as sns from 'aws-cdk-lib/aws-sns';
import * as subscriptions from 'aws-cdk-lib/aws-sns-subscriptions';

centralTopic.addSubscription(
  new subscriptions.SqsSubscription(highPriorityQueue, {
    filterPolicy: {
      // Only high priority events
      priority: sns.SubscriptionFilter.stringFilter({
        allowlist: ['high', 'critical'],
      }),
      // Only large orders
      amount: sns.SubscriptionFilter.numericFilter({
        greaterThan: 1000,
      }),
    },
  })
);

centralTopic.addSubscription(
  new subscriptions.SqsSubscription(regionalQueue, {
    filterPolicy: {
      // Only specific regions
      region: sns.SubscriptionFilter.stringFilter({
        allowlist: ['us-east-1', 'eu-west-1'],
      }),
    },
  })
);

// Analytics queue receives everything (no filter)
centralTopic.addSubscription(
  new subscriptions.SqsSubscription(analyticsQueue)
);

Payload-Based Filtering

Payload-based filtering matches on the message body itself, so the publisher doesn’t have to duplicate routing fields into message attributes:

import { SNSClient, SubscribeCommand } from '@aws-sdk/client-sns';

const sns = new SNSClient({ region: 'us-east-1' });

await sns.send(
  new SubscribeCommand({
    TopicArn: 'arn:aws:sns:us-east-1:999999999999:central-events',
    Protocol: 'sqs',
    Endpoint: 'arn:aws:sqs:us-east-1:111111111111:service-events',
    Attributes: {
      FilterPolicyScope: 'MessageBody',
      FilterPolicy: JSON.stringify({
        order: {
          status: ['completed', 'shipped'],
          amount: [{ numeric: ['>', 1000] }],
        },
      }),
    },
  })
);

Filter policy benefits:

  • Each subscriber receives only the messages matching its policy, and pays SQS request charges for those alone
  • Savings scale with how much of the topic volume a subscriber discards, so a narrow consumer on a busy topic gains the most
  • Filter changes take up to 15 minutes to propagate
  • The quota is 200 filter policies per topic and 10,000 per account

FIFO Topics and Queues

FIFO (First-In-First-Out) topics provide strict ordering and deduplication. Use them when message order matters.

When to Use FIFO

FIFO makes sense for:

  • Order processing workflows where sequence matters
  • Financial transactions requiring exactly-once processing
  • State machine transitions that must occur in order
  • Inventory updates where order impacts final state

FIFO Setup Requirements

import * as sns from 'aws-cdk-lib/aws-sns';
import * as sqs from 'aws-cdk-lib/aws-sqs';
import * as subscriptions from 'aws-cdk-lib/aws-sns-subscriptions';

// Create FIFO topic (publisher account)
const fifoTopic = new sns.Topic(this, 'OrderEventTopic', {
  topicName: 'order-events.fifo',
  fifo: true,
  contentBasedDeduplication: true,
  // Throughput counted per message group, 300 messages/second each
  fifoThroughputScope: sns.FifoThroughputScope.MESSAGE_GROUP,
});

// Create FIFO queue (consumer account)
const fifoQueue = new sqs.Queue(this, 'OrderQueue', {
  queueName: 'orders.fifo',
  fifo: true,
  contentBasedDeduplication: true,
});

// FIFO topic can only subscribe FIFO queues
fifoTopic.addSubscription(
  new subscriptions.SqsSubscription(fifoQueue)
);

Publishing to FIFO topics:

import { SNSClient, PublishCommand } from '@aws-sdk/client-sns';

const sns = new SNSClient({ region: 'us-east-1' });

await sns.send(
  new PublishCommand({
    TopicArn: 'arn:aws:sns:us-east-1:999999999999:order-events.fifo',
    Message: JSON.stringify({ orderId: '12345', status: 'created' }),
    MessageGroupId: 'order-region-us-east-1', // Required for ordering
    MessageDeduplicationId: 'order-12345-created', // Optional if contentBasedDeduplication enabled
  })
);

Throughput scope considerations:

  • FifoThroughputScope: Topic, the default, caps a FIFO topic at 3,000 messages per second or 20 MB per second, whichever comes first
  • FifoThroughputScope: MessageGroup counts throughput per group with a ceiling of 300 messages per second per group, so topic throughput grows with the number of active groups
  • The per-account publish quota applies on top and varies by Region: 30,000 messages per second in us-east-1, 3,000 in most others
  • Partition message groups on a field that already has natural concurrency (customer, region, tenant); a single global group ID serializes the whole topic

Common Pitfalls and Solutions

The following issues appear consistently in production, along with their solutions.

Pitfall 1: Subscription Shows “PendingConfirmation”

Symptom: Subscription created but stuck in “PendingConfirmation” status. Messages never flow.

Root cause: When the topic owner creates the subscription (rather than the queue owner), SNS sends a confirmation message that must be manually confirmed.

Solution: Always have the queue owner create the subscription:

// PREFERRED: Queue owner subscribes (in consumer account)
const centralTopic = sns.Topic.fromTopicArn(
  this,
  'CentralTopic',
  'arn:aws:sns:us-east-1:999999999999:central-events'
);

centralTopic.addSubscription(
  new subscriptions.SqsSubscription(queue)
);
// Queue owner subscribes, so confirmation is automatic

If you must have the topic owner create subscriptions, automate confirmation:

# Python script to auto-confirm subscriptions
import boto3
import json

sqs = boto3.client('sqs')
queue_url = 'https://sqs.us-east-1.amazonaws.com/111111111111/service-events'

# Poll for confirmation message
response = sqs.receive_message(
    QueueUrl=queue_url,
    MaxNumberOfMessages=1,
    WaitTimeSeconds=10
)

for message in response.get('Messages', []):
    body = json.loads(message['Body'])

    if 'SubscribeURL' in body:
        # Confirm subscription by visiting URL
        import urllib.request
        urllib.request.urlopen(body['SubscribeURL'])
        print(f"Confirmed subscription: {body['SubscribeURL']}")

        # Delete confirmation message
        sqs.delete_message(
            QueueUrl=queue_url,
            ReceiptHandle=message['ReceiptHandle']
        )

Confirming through the URL carries a side effect worth knowing. AuthenticateOnUnsubscribe is set automatically only when the queue owner creates the subscription, and it cannot be turned on afterwards for a link that was opened without credentials. An unauthenticated unsubscribe request can then tear the subscription down.

Pitfall 2: KMS Key Access Denied

Symptom: Messages published to SNS but never appear in encrypted SQS queue. No errors in SNS metrics.

Root cause: SNS service lacks permission to use the KMS key for encryption.

Solution: Verify KMS key policy grants SNS the required permissions:

// Check your KMS key policy includes this
queueKey.addToResourcePolicy(
  new iam.PolicyStatement({
    effect: iam.Effect.ALLOW,
    principals: [new iam.ServicePrincipal('sns.amazonaws.com')],
    actions: [
      'kms:Decrypt',
      'kms:GenerateDataKey',
    ],
    resources: ['*'],
    conditions: {
      StringEquals: {
        'kms:ViaService': `sqs.${this.region}.amazonaws.com`,
      },
    },
  })
);

Troubleshooting tip: Check CloudTrail logs for KMS AccessDenied errors:

aws cloudtrail lookup-events \
  --lookup-attributes AttributeKey=EventName,AttributeValue=Decrypt \
  --max-results 50 \
  --region us-east-1 \
  --query 'Events[?ErrorCode==`AccessDenied`]'

Pitfall 3: Region Mismatch

Symptom: Messages never arrive, or the subscription is refused, even though both policies read correctly.

Root cause: The topic and the queue sit in different Regions while something in the chain is still pinned to one of them. The usual culprits are an aws:SourceArn condition carrying the topic’s Region, and a subscription dead-letter queue, which has to live in the same account and Region as the subscription itself.

Solution: Keeping the topic and its queues in one Region is the simpler default, and it avoids inter-Region delivery latency and data transfer charges as a side benefit. When a consumer genuinely has to run elsewhere, forward through a Lambda that republishes into a topic in the target Region:

// Region 1 (us-east-1): Original topic
const sourceTopicEast = new sns.Topic(this, 'SourceTopicEast', {
  topicName: 'events-us-east-1',
});

// Lambda forwarder publishes to region 2 topic
const forwarder = new lambda.Function(this, 'RegionForwarder', {
  runtime: lambda.Runtime.NODEJS_20_X,
  handler: 'index.handler',
  code: lambda.Code.fromInline(`
    const { SNSClient, PublishCommand } = require('@aws-sdk/client-sns');
    const sns = new SNSClient({ region: 'eu-west-1' });

    exports.handler = async (event) => {
      for (const record of event.Records) {
        await sns.send(new PublishCommand({
          TopicArn: process.env.TARGET_TOPIC_ARN,
          Message: record.Sns.Message,
          MessageAttributes: record.Sns.MessageAttributes,
        }));
      }
    };
  `),
  environment: {
    TARGET_TOPIC_ARN: 'arn:aws:sns:eu-west-1:999999999999:events-eu-west-1',
  },
});

sourceTopicEast.addSubscription(
  new subscriptions.LambdaSubscription(forwarder)
);

Pitfall 4: Message Size Limits

Symptom: Some messages delivered successfully, others silently disappear.

Root cause: SNS and SQS both have 256 KB message size limits. Messages exceeding this are dropped without notification.

Solution: Keep messages under 256 KB or use S3 for large payloads:

import { S3Client, PutObjectCommand } from '@aws-sdk/client-s3';
import { SNSClient, PublishCommand } from '@aws-sdk/client-sns';

const s3 = new S3Client({ region: 'us-east-1' });
const sns = new SNSClient({ region: 'us-east-1' });

async function publishLargeMessage(topicArn: string, payload: any) {
  const payloadSize = Buffer.byteLength(JSON.stringify(payload));

  if (payloadSize > 200_000) {
    // Store large payload in S3
    const messageId = crypto.randomUUID();
    const s3Key = `messages/${messageId}.json`;

    await s3.send(
      new PutObjectCommand({
        Bucket: 'large-message-payloads',
        Key: s3Key,
        Body: JSON.stringify(payload),
      })
    );

    // Publish reference to S3 object
    await sns.send(
      new PublishCommand({
        TopicArn: topicArn,
        Message: JSON.stringify({
          type: 'S3Reference',
          bucket: 'large-message-payloads',
          key: s3Key,
        }),
      })
    );
  } else {
    // Direct publish for small messages
    await sns.send(
      new PublishCommand({
        TopicArn: topicArn,
        Message: JSON.stringify(payload),
      })
    );
  }
}

Pitfall 5: Filter Policies Not Taking Effect

Symptom: Messages still delivered despite filter policy.

Root cause: Filter policies take up to 15 minutes to propagate, or message attributes don’t match filter format.

Solution: Wait for propagation and verify attribute format:

// Verify message attributes match filter expectations
await sns.send(
  new PublishCommand({
    TopicArn: topicArn,
    Message: JSON.stringify({ orderId: '12345' }),
    MessageAttributes: {
      eventType: {
        DataType: 'String',
        StringValue: 'OrderCreated', // Must match filter exactly
      },
      priority: {
        DataType: 'Number', // Use Number type for numeric filters
        StringValue: '5',
      },
    },
  })
);

// Monitor filtering effectiveness
const filterMetric = new cloudwatch.Metric({
  namespace: 'AWS/SNS',
  metricName: 'NumberOfNotificationsFilteredOut',
  dimensionsMap: {
    TopicName: 'central-events',
  },
});

Monitoring and Observability

Effective monitoring is essential for cross-account messaging. You need visibility into both publisher and consumer sides.

Key SNS Metrics

import * as cloudwatch from 'aws-cdk-lib/aws-cloudwatch';
import * as cloudwatch_actions from 'aws-cdk-lib/aws-cloudwatch-actions';
import * as sns from 'aws-cdk-lib/aws-sns';

// Monitor SNS delivery failures
new cloudwatch.Alarm(this, 'SNSDeliveryFailures', {
  metric: topic.metricNumberOfNotificationsFailed({
    statistic: 'Sum',
    period: Duration.minutes(5),
  }),
  threshold: 10,
  evaluationPeriods: 2,
  treatMissingData: cloudwatch.TreatMissingData.NOT_BREACHING,
  alarmDescription: 'Alert when SNS message delivery fails',
});

// Monitor filter effectiveness
new cloudwatch.Alarm(this, 'HighFilterRate', {
  metric: topic.metricNumberOfNotificationsFilteredOut({
    statistic: 'Sum',
    period: Duration.minutes(5),
  }),
  threshold: 1000,
  evaluationPeriods: 1,
  comparisonOperator: cloudwatch.ComparisonOperator.GREATER_THAN_THRESHOLD,
  alarmDescription: 'Alert when filter rate is unusually high',
});

Key SQS Metrics

// Monitor queue depth
new cloudwatch.Alarm(this, 'QueueDepthAlarm', {
  metric: queue.metricApproximateNumberOfMessagesVisible({
    statistic: 'Average',
    period: Duration.minutes(5),
  }),
  threshold: 1000,
  evaluationPeriods: 2,
  alarmDescription: 'Alert when queue depth grows too large',
});

// Monitor processing lag
new cloudwatch.Alarm(this, 'OldMessageAlarm', {
  metric: queue.metricApproximateAgeOfOldestMessage({
    statistic: 'Maximum',
    period: Duration.minutes(1),
  }),
  threshold: 300, // 5 minutes
  evaluationPeriods: 3,
  alarmDescription: 'Alert when messages are not processed timely',
});

// Monitor DLQ
new cloudwatch.Alarm(this, 'DLQMessages', {
  metric: dlq.metricApproximateNumberOfMessagesVisible({
    statistic: 'Sum',
    period: Duration.minutes(5),
  }),
  threshold: 1,
  evaluationPeriods: 1,
  alarmDescription: 'Alert on any messages in DLQ',
});

Cross-Account CloudWatch Observability

For unified monitoring across accounts, use CloudWatch Observability Access Manager:

import * as oam from 'aws-cdk-lib/aws-oam';

// In monitoring account: Create sink
const sink = new oam.CfnSink(this, 'MonitoringSink', {
  name: 'central-monitoring-sink',
  policy: {
    Version: '2012-10-17',
    Statement: [
      {
        Effect: 'Allow',
        Principal: {
          AWS: [
            'arn:aws:iam::111111111111:root',
            'arn:aws:iam::222222222222:root',
            'arn:aws:iam::333333333333:root',
          ],
        },
        Action: ['oam:CreateLink', 'oam:UpdateLink'],
        Resource: '*',
      },
    ],
  },
});

// In each source account: Create link to sink
const link = new oam.CfnLink(this, 'MonitoringLink', {
  resourceTypes: ['AWS::CloudWatch::Metric', 'AWS::Logs::LogGroup'],
  sinkIdentifier: 'arn:aws:oam:us-east-1:999999999999:sink/sink-id',
});

This enables a single dashboard showing metrics from all accounts without data transfer costs (within same region).

Cost Analysis

Understanding costs helps optimize your architecture.

Pricing Breakdown (2025)

SNS costs:

  • First 1 million requests/month: FREE
  • Beyond free tier: $0.50 per million publishes
  • SNS to SQS deliveries: FREE (major cost advantage)

SQS costs:

  • First 1 million requests/month: FREE
  • Standard queue: $0.40 per million requests
  • FIFO queue: $0.50 per million requests
  • Each 64 KB chunk = 1 request (256 KB message = 4 requests)

Fan-out cost example (1 million messages to 4 queues, free tier excluded):

  • SNS publishes: 1M × $0.50 = $0.50
  • SNS to SQS delivery: FREE
  • SQS receives: 4M × $0.40 = $1.60
  • SQS deletes: 4M × $0.40 = $1.60
  • Total: $3.70

With 50% message filtering:

  • SNS publishes: 1M × $0.50 = $0.50
  • Filtered deliveries: 2M messages delivered
  • SQS receives: 2M × $0.40 = $0.80
  • SQS deletes: 2M × $0.40 = $0.80
  • Total: $2.10 (43% cost reduction)

KMS costs (for encrypted queues):

  • Customer-managed key: $1/month per key
  • KMS requests: $0.03 per 10,000 requests
  • Request volume follows KmsDataKeyReusePeriodSeconds: a longer reuse window spreads one data key across many messages

Cost Optimization Strategies

  1. Filter at the subscription: consumers stop paying receive and delete charges for messages they would discard
  2. Enable SQS long polling: one paid receive covers a wait of up to 20 seconds instead of returning empty
  3. Use batch operations: up to 10 messages per API call
  4. Keep messages under 64 KB: every 64 KB chunk bills as a separate request
  5. Use Standard queues when ordering isn’t critical: $0.40 against $0.50 per million requests

Alternative Approaches

Three neighbouring services take over where fan-out runs out: routing complexity, replay, and synchronous work.

EventBridge

When to use:

  • Need complex event routing (100+ rules)
  • Schema registry and validation required
  • Event replay capability essential
  • Integration with 30+ AWS services

Trade-offs:

  • $1.00 per million events (vs SNS $0.50)
  • More powerful filtering with JSONPath-like syntax
  • Built-in schema discovery and validation
  • Native cross-account event buses
import * as events from 'aws-cdk-lib/aws-events';
import * as targets from 'aws-cdk-lib/aws-events-targets';

const bus = new events.EventBus(this, 'CentralBus', {
  eventBusName: 'central-events',
});

// Grant cross-account access
bus.grantPutEventsTo(new iam.AccountPrincipal('111111111111'));

// Complex filtering
new events.Rule(this, 'HighValueOrders', {
  eventBus: bus,
  eventPattern: {
    source: ['com.myapp.orders'],
    detailType: ['OrderCreated'],
    detail: {
      amount: [{ numeric: ['>', 1000] }],
      region: ['us-east-1', 'us-west-2'],
      status: ['pending', 'processing'],
    },
  },
  targets: [new targets.SqsQueue(queue)],
});

Kinesis Data Streams

When to use:

  • Ordered stream processing required
  • Need replay capability (up to 365 days)
  • Multiple consumers reading at different speeds
  • Real-time analytics use cases

Trade-offs:

  • More expensive ($0.015 per shard hour + PUT costs)
  • Complex shard management
  • Better for streaming analytics than discrete events
  • Higher operational overhead

Direct Lambda Invocation

When to use:

  • Synchronous processing acceptable
  • Event volume under Lambda concurrent execution limits
  • No need for queue management
  • Simple, fast processing logic

Trade-offs:

  • No built-in retry queues
  • Cold start considerations
  • Limited by Lambda concurrency
  • Less flexible than queues for scaling

End-to-End Multi-Account Setup

Publisher and consumer stacks, wired together:

// publisher-stack.ts
import * as cdk from 'aws-cdk-lib';
import * as sns from 'aws-cdk-lib/aws-sns';
import * as iam from 'aws-cdk-lib/aws-iam';
import { Construct } from 'constructs';

export class PublisherStack extends cdk.Stack {
  public readonly topicArn: string;

  constructor(scope: Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    const topic = new sns.Topic(this, 'CentralEvents', {
      topicName: 'central-events',
      displayName: 'Central Event Distribution',
    });

    // Allow multiple consumer accounts
    topic.addToResourcePolicy(
      new iam.PolicyStatement({
        sid: 'AllowConsumerSubscribe',
        principals: [
          new iam.AccountPrincipal('111111111111'),
          new iam.AccountPrincipal('222222222222'),
          new iam.AccountPrincipal('333333333333'),
        ],
        actions: ['sns:Subscribe'],
        resources: [topic.topicArn],
      })
    );

    this.topicArn = topic.topicArn;

    // Output for cross-stack references
    new cdk.CfnOutput(this, 'TopicArnOutput', {
      value: topic.topicArn,
      exportName: 'CentralEventTopicArn',
    });
  }
}

// consumer-stack.ts
import * as cdk from 'aws-cdk-lib';
import * as sqs from 'aws-cdk-lib/aws-sqs';
import * as sns from 'aws-cdk-lib/aws-sns';
import * as subscriptions from 'aws-cdk-lib/aws-sns-subscriptions';
import * as lambda from 'aws-cdk-lib/aws-lambda';
import * as lambdaEventSources from 'aws-cdk-lib/aws-lambda-event-sources';
import * as iam from 'aws-cdk-lib/aws-iam';
import * as kms from 'aws-cdk-lib/aws-kms';
import * as cloudwatch from 'aws-cdk-lib/aws-cloudwatch';
import { Construct } from 'constructs';

interface ConsumerStackProps extends cdk.StackProps {
  centralTopicArn: string;
  serviceName: string;
}

export class ConsumerStack extends cdk.Stack {
  constructor(scope: Construct, id: string, props: ConsumerStackProps) {
    super(scope, id, props);

    // Create KMS key for encryption
    const encryptionKey = new kms.Key(this, 'EncryptionKey', {
      description: `Encryption key for ${props.serviceName} events`,
      enableKeyRotation: true,
    });

    // Grant SNS permission to use key
    encryptionKey.addToResourcePolicy(
      new iam.PolicyStatement({
        principals: [new iam.ServicePrincipal('sns.amazonaws.com')],
        actions: ['kms:Decrypt', 'kms:GenerateDataKey'],
        resources: ['*'],
        conditions: {
          StringEquals: {
            'kms:ViaService': `sqs.${this.region}.amazonaws.com`,
          },
        },
      })
    );

    // Create DLQ
    const dlq = new sqs.Queue(this, 'DLQ', {
      queueName: `${props.serviceName}-dlq`,
      encryptionMasterKey: encryptionKey,
      retentionPeriod: cdk.Duration.days(14),
    });

    // Create main queue
    const queue = new sqs.Queue(this, 'EventQueue', {
      queueName: `${props.serviceName}-events`,
      encryptionMasterKey: encryptionKey,
      visibilityTimeout: cdk.Duration.seconds(30),
      receiveMessageWaitTime: cdk.Duration.seconds(20),
      deadLetterQueue: {
        queue: dlq,
        maxReceiveCount: 3,
      },
    });

    // Allow SNS to send messages
    queue.addToResourcePolicy(
      new iam.PolicyStatement({
        principals: [new iam.ServicePrincipal('sns.amazonaws.com')],
        actions: ['sqs:SendMessage'],
        resources: [queue.queueArn],
        conditions: {
          ArnEquals: {
            'aws:SourceArn': props.centralTopicArn,
          },
        },
      })
    );

    // Subscribe to central topic
    const centralTopic = sns.Topic.fromTopicArn(
      this,
      'CentralTopic',
      props.centralTopicArn
    );

    centralTopic.addSubscription(
      new subscriptions.SqsSubscription(queue, {
        rawMessageDelivery: false,
      })
    );

    // Create processor Lambda
    const processor = new lambda.Function(this, 'EventProcessor', {
      runtime: lambda.Runtime.NODEJS_20_X,
      handler: 'index.handler',
      code: lambda.Code.fromInline(`
        exports.handler = async (event) => {
          for (const record of event.Records) {
            const snsMessage = JSON.parse(record.body);
            const message = JSON.parse(snsMessage.Message);

            console.log('Processing message:', message);

            // Your business logic here

            // Message automatically deleted if handler succeeds
          }
        };
      `),
      timeout: cdk.Duration.seconds(30),
      environment: {
        SERVICE_NAME: props.serviceName,
      },
    });

    // Connect queue to Lambda
    processor.addEventSource(
      new lambdaEventSources.SqsEventSource(queue, {
        batchSize: 10,
        reportBatchItemFailures: true,
      })
    );

    // CloudWatch alarms
    new cloudwatch.Alarm(this, 'QueueDepthAlarm', {
      metric: queue.metricApproximateNumberOfMessagesVisible(),
      threshold: 1000,
      evaluationPeriods: 2,
    });

    new cloudwatch.Alarm(this, 'DLQMessagesAlarm', {
      metric: dlq.metricApproximateNumberOfMessagesVisible(),
      threshold: 1,
      evaluationPeriods: 1,
    });

    new cloudwatch.Alarm(this, 'ProcessorErrorsAlarm', {
      metric: processor.metricErrors(),
      threshold: 10,
      evaluationPeriods: 2,
    });
  }
}

When the Defaults Hold

Queue-owner subscriptions, customer-managed keys, and subscription-level filtering cover the ordinary case: a handful of accounts, one Region, and events small enough to travel as messages rather than S3 pointers. The first default breaks when the publisher has to bootstrap subscriptions for accounts it doesn’t control, and that is where the confirmation automation earns its place, together with the unsubscribe caveat attached to it. A compliance boundary can push the second one further, towards a separate key per queue with a narrower kms:ViaService scope. Filtering is the one to skip outright when a consumer genuinely wants every message, since the policy buys nothing and still adds a propagation delay to reason about.

Two constraints reshape the design before any of that. Ordering moves both ends to FIFO and turns message-group partitioning into the throughput lever. A second Region moves delivery behind a forwarder, since a subscription’s dead-letter queue can’t follow the topic across Regions. Routing rules that outgrow attribute and payload matching, or a requirement to replay history, point at EventBridge or Kinesis instead.

References

Related posts