Create Extract Email Function

This is the first function, with missions:

  • Extract email content from MIME type to JSON data
  • Send that data to SQS.

Why i need this function?

Because many reasons:

  • I want to save money when calling Amazon Bedrock and lambda function in Generate email function (If a email with wrong data, we can parse them to our formatted data first before calling AI agent, reduce the tokens send to LLM model)
  • Easier for debugging
  • In real life, maybe some hater will attack this endpoint with spamming, so we can put a process function here.

Let’s create the lambda function

  1. In the AWS Console, search “lambda” and choose Lambda
lambda1
  1. Choose Create function
lambda2
  1. In the AWS Console, choose Author from scratch, Function name is ExtractEmailLambda, because i our code is use nodejs so we choose Nodejs 22.x. Then we choose existing role ExtractEmailFunctionRole then click Create function
lambda3
  1. Next, we need to upload our source code to this function, you can download it index.zip or you can build them with my github repository here. After got that file, you need to upload it to lambda Code > Upload from > .zip file
lambda4
  1. Click Save to upload it
lambda5
  1. Upload successful lambda6

  2. Next, we need to config environment variable for our function to interact with other service

lambda7
  1. We have 2 variables
KeyValueDescription
QUEUE_NAMEsqs-generate-email-content-queueThis is the queue name we will send message (email content) to generate response
BUCKET_NAMEai-powered-email-auto-repliesThis bucket for getting email content, because SES not emit email content, it just emit the messageId (the key of the email object in our bucket)

Click Save

lambda8
  1. Update environment variables successfully
lambda9

Details about our function

ConfigurationValueDescription
function_nameExtractEmailLambdaThe unique name identifier for the Lambda function
roleExtractEmailFunctionRoleIAM role that defines the permissions and access policies for the function
runtimenodejs20.xThe execution environment and version for the Lambda function (Node.js 20)
handlerindex.handlerThe entry point for the function
memory_size128Amount of memory allocated to the function in MB (128MB)
timeout30Maximum execution time allowed for the function in seconds
  1. Received event from SES and get email content from S3
// Helper function to extract email data from SES event
const extractEmailData = async (event: SESEvent) => {
  const sesRecord = event.Records[0].ses;
  const messageId = sesRecord.mail.messageId;
  const emailData = {
    sender: sesRecord.mail.source,
    key: messageId,
  };

  // Get email content from S3
  const getObjectCommand = new GetObjectCommand({
    Bucket: env.bucketName,
    Key: `received-email/${messageId}`
  });

  const response = await s3Client.send(getObjectCommand);
  const emailContent = await response.Body?.transformToString();

  // Parsing MIME type to JSON object
  const { text, subject } = await simpleParser(emailContent);
  return {
    ...emailData,
    subject,
    content: text || "No email data"
  };
};
  1. Sending parsed content message to SQS
const body = await extractEmailData(event);
const command = new GetQueueUrlCommand({ // GetQueueUrlRequest
    QueueName: env.queueName, // required
});
const queueUrlResponse = await sqsClient.send(command);
const params = {
    QueueUrl: queueUrlResponse.QueueUrl,
    MessageBody: JSON.stringify(body),
};
await sqsClient.send(new SendMessageCommand(params));

Full source code

import { Handler, SESEvent } from 'aws-lambda';
import { GetQueueUrlCommand, SQSClient, SendMessageCommand } from "@aws-sdk/client-sqs";
import { GetObjectCommand, S3Client } from "@aws-sdk/client-s3";
import { Logger } from '@aws-lambda-powertools/logger';
import * as mailparser from 'mailparser';

require('dotenv').config();

const simpleParser = mailparser.simpleParser;
const logger = new Logger();

const env = {
  queueName: process.env.QUEUE_NAME || "",
  bucketName: process.env.BUCKET_NAME || ""
}

const sqsClient = new SQSClient();
const s3Client = new S3Client();

// Helper function to extract email data from SES event
const extractEmailData = async (event: SESEvent) => {
  const sesRecord = event.Records[0].ses;
  const messageId = sesRecord.mail.messageId;
  const emailData = {
    sender: sesRecord.mail.source,
    key: messageId,
  };
  // logger.info("Email Data: ", JSON.stringify(emailData, null, 2));

  // Get email content from S3
  const getObjectCommand = new GetObjectCommand({
    Bucket: env.bucketName,
    Key: `received-email/${messageId}`
  });

  const response = await s3Client.send(getObjectCommand);
  const emailContent = await response.Body?.transformToString();
  const { text, subject } = await simpleParser(emailContent);
  return {
    ...emailData,
    subject,
    content: text || "No email data"
  };
};

export const handler: Handler = async (event: SESEvent): Promise<void> => {
  try {
    const body = await extractEmailData(event);
    // logger.info({
    //     message: "Extracted email data",
    //     body: body
    // });
    const command = new GetQueueUrlCommand({ // GetQueueUrlRequest
      QueueName: env.queueName, // required
    });
    const queueUrlResponse = await sqsClient.send(command);
    const params = {
      QueueUrl: queueUrlResponse.QueueUrl,
      MessageBody: JSON.stringify(body),
    };
    await sqsClient.send(new SendMessageCommand(params));
  }
  catch (error) {
    logger.error(error as string);
  }
};

Create SES rule for trigger extract email function

  1. Go to SES dashboard and scroll down at the left panel Configuration > Email receiving > Create rule set
lambda17a
  1. Enter rule set name TriggerEmailIncoming and choose Create rule set
lambda18a
  1. Then choose “Create rule”
lambda19
  1. We enter any name ExtractAndSaveToS3 and choose Next
lambda20
  1. At this step, as you can see, you can add many Recipient conditions look like the Guidelines, for simple i just want to trigger Lambda function and save email to S3. Then choose Next
lambda21
  1. Choose Add new action > Invoke AWS lambda function
lambda22
  1. Then we choose our function ExtractEmailLambda and add another action to save email to S3
lambda23
  1. Next we need to choose existing S3 bucket and folder received-email. We also provide them a policy to allow SES put object to S3, choose Create IAM Role
lambda24
  1. Enter our role name SESSaveEmailToS3Role and Create role
lambda25
  1. We need to attach our policy we created from step 4.4, Choose View role
lambda26
  1. Then we will attach our policy to this role. Choose Attach policies
lambda27
  1. Please type SESSaveEmailToS3Policy from the search box > SESSaveEmailToS3Policy > Add permissions
lambda28
  1. Attach success
lambda29
  1. Then we go back to SES, choose Next
lambda30
  1. Choose Create rule
lambda31
  1. Final, we need to active that rule, choose the rule and Set as active
lambda32
  1. Confirm set that rule
lambda33
  1. Setup receive email complete
lambda34