Why you should archive AWS EventBridge events for your custom event bus?

Introduction

WS EventBridge is a fully managed event bus that makes it easy to process, store and analyze events generated by cloud resources, applications and custom sources. One of the powerful features of EventBridge is the ability to archive events, which allows you to store event data for later analysis and replay.

In this section, we'll take a closer look at how you can take advantage of the archive feature in EventBridge to save you in situations like mitigating data loss in case of failed events and the ability to replay them later.

Benefits of Archiving Events in EventBridge

• Data Loss Prevention: Archiving events in EventBridge provides a reliable backup of the events, allowing you to recover lost data in case of failed events. This is especially important for critical business processes where data loss can have a significant impact.
• Replaying Failed Events: In the event of a failed event, you can easily replay the archived event to recover from the failure and ensure that your business processes continue to run smoothly.
• Debugging and Troubleshooting: Archived events can be used for debugging and troubleshooting purposes, allowing you to diagnose and resolve issues more quickly and efficiently.
• Compliance: Archiving events can also help you meet compliance requirements by providing an audit trail of all events processed by your event bus.

We'll focus on the first 2 points for this article.

Mitigating Data Loss

One of the most significant benefits of archiving EventBridge events is that it helps you to mitigate data loss in case of failed events. When an event is sent to a custom event bus in EventBridge, it is processed by the target. If the target fails to process the event, the event is not lost and can be replayed later.

To illustrate this, let's consider a scenario where you have a custom event bus in EventBridge and you are sending events to a Lambda function as the target. The Lambda function is responsible for processing the events and storing the data in a database. If the Lambda function fails to process the event, the event will be lost, and the data will not be stored in the database.

However, by archiving the events, you can ensure that the events are not lost and can be replayed later. In this way, you can mitigate the risk of data loss and ensure that your data is always up-to-date.

Replay of Failed Events

Another benefit of archiving EventBridge events is that you can replay the events later. This is useful when you need to reprocess the events or when you need to perform an analysis on the data.

For example, let's consider a scenario where you have a custom event bus in EventBridge and you are sending events to a Lambda function as the target. The Lambda function is responsible for processing the events and storing the data in a database. If you need to perform an analysis on the data, you can replay the events and process them again.

How to Archive EventBridge Events

First we need to create a Custom EventBridge Event Bus. In CloudFormation, you can create a custom event bus by defining a resource of type `AWS::Events::EventBus`.

Resources:
  EventBus:
    Type: AWS::Events::EventBus
    Properties:
      Name: custom-event-bus
      EventBusPolicy:
        Version: '2012-10-17'
        Statement:
          - Effect: Allow
            Principal: '*'
            Action: events:PutEvents
            Resource: !GetAtt EventBus.Arn

Then to enable archiving for your custom event bus, you need to create the archive for this bus

  EventBusArchive:
    Type: AWS::Events::Archive
    Properties:
      Description: Archive for custom-event-bus
      EventBusName: custom-event-bus
      RetentionDays: 7

Finally we create an event pattern for this event bus

  EventPattern:
    Type: AWS::Events::EventPattern
    Properties:
      EventPattern:
        source:
          - custom-event-bus

Now you can simply send events to your event bus, and they'll automatically get archived. See the example below for sending and retrieving events from archive. (also possible via AWS console):

import { EventBridge, CloudFormation } from 'aws-sdk';

const eventBridge = new EventBridge();
const cloudFormation = new CloudFormation();

const archiveEvents = async (eventBusName: string, events: EventBridge.Types.PutEventsRequestEntry[]) => {
  try {
    const result = await eventBridge.putEvents({
      Entries: events,
      EventBusName: eventBusName
    }).promise();
    console.log(`Successfully archived events: ${JSON.stringify(result)}`);
  } catch (error) {
    console.error(`Error archiving events: ${error}`);
  }
};

const getArchivedEvents = async (eventBusName: string, eventSourceArn: string) => {
  try {
    const result = await eventBridge.listArchivedRules({
      EventBusName: eventBusName,
      EventSourceArn: eventSourceArn
    }).promise();
    console.log(`Successfully retrieved archived events: ${JSON.stringify(result)}`);
  } catch (error) {
    console.error(`Error retrieving archived events: ${error}`);
  }
};

export { archiveEvents, getArchivedEvents };

And you can make use of the SDK events.replayEvents function to replay them (also possible via AWS console):

const AWS = require('aws-sdk');

const events = new AWS.EventBridge({apiVersion: '2015-10-07'});

const replayArchivedEvents = async () => {
  const result = await events.replayEvents({
    EventBusName: 'custom-event-bus',
    EventSourceArn: 'arn:aws:events:eu-west-1:xxxxxxxxx:event-bus/custom-event-bus',
    ReplayName: 'replay-events-1',
    RuleArns: [
      'arn:aws:events:eu-west-1:xxxxxxxxx:rule/example-rule',
    ],
  }).promise();

  console.log(result);
};

replayArchivedEvents();

That's all for this one. Thanks for reading Devspedia. I love you and I'll see you the next time! :)