Automate email responses using Amazon Comprehend custom classification and entity detection

In this post, we demonstrate how to create an automated email response solution using Amazon Comprehend.

Organizations spend lots of resources, effort, and money on running their customer care operations to answer customer questions and provide solutions. Your customers may ask questions via various channels, such as email, chat, or phone, and deploying a workforce to answer those queries can be resource intensive, time-consuming, and even unproductive if the answers to those questions are repetitive.

During the COVID-19 pandemic, many organizations couldn’t adequately support their customers due to the shutdown of customer care and agent facilities, and customer queries were piling up. Some organizations struggled to reply to queries promptly, which can cause a poor customer experience. This in turn can result in customer dissatisfaction, and can impact an organization’s reputation and revenue in the long term.

Although your organization might have the data assets for customer queries and answers, you may still struggle to implement an automated process to reply to your customers. Challenges might include unstructured data, different languages, and a lack of expertise in artificial intelligence (AI) and machine learning (ML) technologies.

You can overcome such challenges by using Amazon Comprehend to automate email responses to customer queries. With our solution, you can identify the intent of customer emails send an automated response if the intent matches your existing knowledge base. If the intent doesn’t have a match, the email goes to the support team for a manual response. The following are some common customer intents when contacting customer care:

Transaction status (for example, status of a money transfer)
Password reset
Promo code or discount
Hours of operation
Find an agent location
Report fraud
Unlock account
Close account

Amazon Comprehend can help you perform classification and entity detection on emails for any of the intents above. For this solution, we show how to classify customer emails for the first three intents. You can also use Amazon Comprehend to detect key information from emails, so you can automate your business processes. For example you can use Amazon Comprehend to automate the reply to a customer request with specific information related to that query.

Solution overview

To build our customer email response flow, we use the following services:

Amazon Comprehend
AWS Lambda
Amazon Simple Email Service (Amazon SES)
Amazon Simple Notification Service (Amazon SNS)
Amazon WorkMail

The following architecture diagram highlights the end-to-end solution:

The solution workflow includes the following steps:

A customer sends an email to the customer support email created in WorkMail.
WorkMail invokes a Lambda function upon receiving the email.
The function sends the email content to a custom classification model endpoint.
The custom classification endpoint returns with a classified value and confidence level (over 80%, but you can configure this as needed).
If the classification value is MONEYTRANSFER, the Lambda function calls the entity detection endpoint to find the money transfer ID.
If the money transfer ID is returned, the function returns the money transfer status randomly (in real-world scenario, you can call the database via API to fetch the actual transfer status).
Based on the classified value returned, a predefined email template in Amazon SES is chosen, and a reply email is sent to the customer.
If the confidence level is less than 80%, a classified value is not returned, or entity detection doesn’t find the money transfer ID, the customer email is pushed to an SNS topic. You can subscribe to Amazon SNS to push the message to your ticketing system.

Prerequisites

Refer to the README.md file in the GitHub repo to make sure you meet the prerequisites to deploy this solution.

Deploy the solution

Solution deployment consists of the following high-level steps:

Complete manual configurations using the AWS Management Console.
Run scripts in an Amazon SageMaker notebook instance using the provided notebook file.
Deploy the solution using the AWS Cloud Development Kit (AWS CDK).

For full instructions, refer to the README.md file in the GitHub repo.

Test the solution

To test the solution, send an email from your personal email to the support email created as part of the AWS CDK deployment (for this post, we use support@mydomain.com). We use the following three intents in our sample data for custom classification training:

MONEYTRANSFER – The customer wants to know the status of a money transfer
PASSRESET – The customer has a login, account locked, or password request
PROMOCODE – The customer wants to know about a discount or promo code available for a money transfer

The following screenshot shows a sample customer email:

If the customer email is not classified or confidence levels are below 80%, the content of the email is forwarded to an SNS topic. Whoever is subscribed to the topic receives the email content as a message. We subscribed to this SNS topic with the email that we passed with the human_workflow_email parameter during the deployment.

Clean up

To avoid incurring ongoing costs, delete the resources you created as part of this solution when you’re done.

Conclusion

In this post, you learned how to configure an automated email response system using Amazon Comprehend customer classification and entity detection and other AWS services. This solution can provide the following benefits:

Improved email response time
Improved customer satisfaction
Cost savings regarding time and resources
Ability to focus on key customer issues

You can also expand this solution to other areas in your business and to other industries.

With the current architecture, the emails that are classified with a low confidence score are routed to a human loop for manual verification and response. You can use the inputs from the manual review process to further improve the Amazon Comprehend model and increase the automated classification rate. Amazon Augmented AI (Amazon A2I) provides built-in human review workflows for common ML use cases, such as NLP-based entity recognition in documents. This allows you to easily review predictions from Amazon Comprehend.

As we get more data for every intent, we will retrain and deploy the custom classification model and update the email response flow accordingly in the GitHub repo.

About the Author

Godwin Sahayaraj Vincent is an Enterprise Solutions Architect at AWS who is passionate about Machine Learning and providing guidance to customers to design, deploy and manage their AWS workloads and architectures. In his spare time, he loves to play cricket with his friends and tennis with his three kids.

Shamika Ariyawansa is an AI/ML Specialist Solutions Architect on the Global Healthcare and Life Sciences team at Amazon Web Services. He works with customers to advance their ML journey with a combination of AWS ML offerings and his ML domain knowledge. He is based out of Denver, Colorado. In his spare time, he enjoys off-roading adventures in the Colorado mountains and competing in machine learning competitions.