How to publish a message in SQS from SNS

Creating a AWS CloudFormation template to publish to a topic and send it to Amazon SQS queues


May 10, 2021
featured image

Overview

Let say you want to process images in background or any other task that requires heavy processing and you don't want to tie this operation to any other core operation or service. Suppose service A deals with users:
  • Sign in
  • Sign up
  • Forgot password
  • Profile update
  • etc

Every time a user uploads an image to his profile you want to resize it and generate multiple thumbnails for multiple platforms or devices. You can do this operation under the same umbrella of service A but soon this business logic will grow up making our microservice a larger service. So, you decide to separate this kind of operations to a new microservice (service B) but how you communicate with each other? One option is to call the service directly but sounds you are tying bot services. Another option (here we go with this post) is to broadcast an event from service A called "profile updated".

Now we have to see how service B is notified to start processing the image and here is where SNS and SQS come.

In the following example, I show you have to deploy an SNS topic that writes a message in an SQS queue. After that, you could have a Lambda function that is triggered by this queue but this is out of this scope.

The Cloudformation template

      
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: >
  Sample SAM Template SNS and SQS

Parameters:
  Environment:
    Type: String
    Description: example, staging
Resources:
  MyQueue:
    Type: AWS::SQS::Queue
    Properties:
      QueueName: !Sub "${Environment}-my-queue.fifo"
      FifoQueue: true
  MyTopic:
    Type: AWS::SNS::Topic
    Properties:
      ContentBasedDeduplication: true
      FifoTopic: true
      Subscription:
        - Endpoint:
            Fn::GetAtt:
              - MyQueue
              - Arn
          Protocol: sqs
      TopicName: !Sub "${Environment}-my-topic.fifo"
  MyQueuePolicy:
    Type: AWS::SQS::QueuePolicy
    DependsOn:
      - MyQueue
    Properties:
      PolicyDocument:
        Version: "2012-10-17"
        Statement:
          - Effect: Allow
            Principal:
              Service:
                - "events.amazonaws.com"
                - "sqs.amazonaws.com"
                - "sns.amazonaws.com"
            Action:
              - "sqs:SendMessage"
              - "sqs:ReceiveMessage"
            Resource:
              Fn::GetAtt:
                - MyQueue
                - Arn
            Condition:
              ArnEquals:
                aws:SourceArn:
                  Ref: MyTopic
      Queues:
        - Ref: MyQueue
Outputs:
  MyTopicTopicARN:
    Value:
      Ref: MyTopic
  MyQueue:
    Value:
      Fn::Join:
        - " "
        - - 'ARN:'
          - Fn::GetAtt:
              - MyQueue
              - Arn
          - 'URL:'
          - Ref: MyQueue

      
    

Deploying using Sam

      
sam build
      
    
      
sam deploy --guided
      
    

Resources

Photo by Luís Perdigão on Unsplash


aws sns sqs cloudformation