Event Driven Architecture with Amazon API Gateway, Event Bridge, Lambda,SNS,SQS in Multi-Account Setup

🎉 Introduction to Microservices, Event-Driven, and Serverless Architectures 🚀

Microservices are a way of building applications where you split them into small, independent services that communicate via well-defined APIs. 📦 Each service does its own thing but works together like pieces of a puzzle 🧩. Event-driven architecture makes communication between these decoupled services super smooth by triggering actions based on specific events. 🎯 And to make things even more awesome, serverless architecture allows you to build and run these services without worrying about infrastructure! 🌐 Together, they create a perfect recipe for modern, scalable applications! 🍰

🎉 Let’s Dive into Event-Driven Architecture (Multi-Account Edition)! 🎯

In this article, we’re diving into the world of event-driven architecture, and we’re taking it up a notch with a multi-account setup! 🚀 We’ll use an API Gateway to send events to an EventBridge (aka event bus) in one account, which will then trigger a Lambda function in another account to do some awesome custom tasks! 🛠️💡

By leveraging multiple AWS accounts, we’ll show the full power of event-driven architecture, making your applications more secure, scalable, and efficient! While I’m using API Gateway to send events to EventBridge, you can easily tweak it to fit your needs and send events in other ways too! 🌐

Let’s get started! 🎉

Diagram: The image depicts an AWS architectural diagram completed at the end of this article.

🛠️ Step 1: Create API Gateway (Account A)

1. Go to Account A and set up a REST API using API Gateway.

2. Create a Lambda function and copy its ARN (we’ll need it soon). Also, note the IAM role ARN tied to this function.

3. Return to the API Gateway and create a method.

4. Paste the copied Lambda ARN into the Lambda section and save it. 💾

🚀 Step 2: Set Up EventBridge & SNS (Account B)

Now, let’s create the event bus that will handle our events:

1. In Account B, go to EventBridge and create an Event Bus.

2. Next, create an SNS Topic in Account B for event notifications. 📬

3. Go back to EventBridge and create a new Rule:

• Set the event pattern as follows:

{
  "detail-type": ["demo-event"],
  "source": ["api.gateway"]
}

• For the target, select the SNS topic created earlier.

4. Make sure the SNS topic gets the following resource policy to allow EventBridge to publish messages:

{
      "Sid": "AWSEvents_apigateway-eventxxxxxxxx",
      "Effect": "Allow",
      "Principal": {
        "Service": "events.amazonaws.com"
      },
      "Action": "sns:Publish",
      "Resource": "arn:aws:sns:ap-southeast-2:xxxxxxx:demo-topic"
    }
##xxxxxxx->should corrrespond to account-id of SNS (account B id)

📌 Note: Replace xxxxx with the Account B ID.

5. In EventBridge permissions, allow the Lambda function created in Account A to send events to the EventBridge in Account B with this policy:

{
  "Version": "2012-10-17",
  "Statement": [{
    "Sid": "AllowEvent",
    "Effect": "Allow",
    "Principal": {
      "AWS": "arn:aws:iam::yyyyy:root"
    },
    "Action": "events:PutEvents",
    "Resource": "arn:aws:events:ap-southeast-2:xxxxx:event-bus/demo-eventbus"
  }]
}

📌 Note: xxxxx is the EventBridge Account B ID, and yyyyy is the Lambda Account A ID.

Ok now that setup is done lets go to account c to setup the flow to receive these event from SNS and perform some function.

💥 Step 3: Setting Up SQS and Lambda (Account C)

To receive events from SNS we will be using SQS queue. And use that SQS to trigger lambda. Why to directly trigger lambda from SNS?

Well here are few reasons why …

Batching 🛠️
SQS bundles multiple events into batches before triggering a Lambda function, boosting performance by reusing hot code 🔥 and minimizing function invocations. This reduces cold start overhead ⏳ and helps you avoid hitting Lambda concurrency limits 🚦. But, there’s a small tradeoff: if events trickle in slowly, SQS might wait up to 20 seconds ⏰ before sending the messages to Lambda.

Observability 👀
With SQS, it’s easy to keep an eye on the backlog of unprocessed events or failed messages. 📊 Monitoring the queue length can be super helpful to spot if something’s going wrong in your system. 🚨

1. First lets allow SNS topic to be subscribed by services in account C. to allow this add resource policy to SNS topic.

{
      "Sid": "AllowtoSubscription",
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::zzzz:root"
      },
      "Action": "SNS:Subscribe",
      "Resource": "arn:aws:sns:ap-southeast-2:xxxxx:demo-topic"
    }
##zzzz->should corrrespond to account-id of sqs account (account C id)
##xxxx->should corrrespond to account-id of SNS (account B id)

📌 Note: zzzz is Account C, and xxxxx is the SNS Account B ID.

2. Now lets go to Account C. Create a SQS.

3. In Account C, create a subscription to SNS, with SQS as the protocol and the endpoint as the SQS queue. 🎯

Give arn of topic created in previous step ,protocol as SQS and endpoint as sqs topic created earlier. Then subscription will be created for the SNS topic.

If you go into SQS queue and go to SNS subscription section it will show the subscription.

🛠️ Step 4: Sending Events to Lambda (Account C)

Now lets create a lambda function which use SQS trigger and print out messages to the console.

For the lambda role give nessasary permissions to receive mesages from SQS queue. Add Role policy such as below.

{
            "Action": [
                "sqs:ReceiveMessage",
                "sqs:GetQueueAttributes",
                "sqs:DeleteMessage",
                "sqs:ChangeMessageVisibility"
            ],
            "Effect": "Allow",
            "Resource": "arn:aws:sqs:ap-southeast-2:zzzzz:demo-queue",
            "Sid": "AllowSQSPermissions"
        }
##zzzz->should corrrespond to account-id of lambda account (account C id)

Then add a trigger to lambda function with SQS queue we created.

Update the Lambda code to receive events from EventBridge:

import json

def lambda_handler(event, context):
    for record in event['Records']:
        message_body = record['body']
        print(f"Received message: {message_body}")
        
    return {
        'statusCode': 200,
        'body': json.dumps('Processed all messages')
    }

🎉 Step 5: Sending Events from Lambda (Account A)

Now lets go to account a and lambda function created in step 1.

Add a policy to allow the Lambda to send events to EventBridge in Account B:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Statement1",
            "Effect": "Allow",
            "Action": [
                "events:PutEvents"
            ],
            "Resource": [
                "arn:aws:events:ap-southeast-2:xxxxx:event-bus/demo-eventbus"
            ]
        }
    ]
}

Update the Lambda code to send events to EventBridge:

import json
import boto3
import os

eventbridge_client = boto3.client('events')
def lambda_handler(event, context):
    detail_type = "Content-View-Event"
    source = "api.gateway.content.get"
    
    event_entry = {
        'Source': source,
        'DetailType': detail_type,
        'Detail': json.dumps({
            'message': 'Content view event triggered'
        }),
        'EventBusName': os.environ.get('EVENT_BUS_NAME', 'default')  # Use environment variable or default bus
    }
    try:
        response = eventbridge_client.put_events(
            Entries=[event_entry]
        )
        print(f"Event sent successfully: {response}")
    except Exception as e:
        print(f"Error sending event: {e}")
    return {
        'statusCode': 200,
        'body': json.dumps('Event sent to EventBridge')
    }

Now all the configurations are done. Now lets test it.

🎉 Step 6: Test It!

Send a request to the API Gateway created in Step 1 using the test console.

You’ll get a response, and the event will travel through the pipeline — API Gateway → EventBridge → SNS → SQS → Lambda! 🌊

Check the Lambda logs in Account C to verify that the event was successfully received! 🎉

And that’s it! You’ve just created an event-driven, multi-account architecture using AWS services! 🚀 Now, go check those logs and enjoy the seamless event flow across accounts! 😎