Unlocking language barriers: Translate application logs with Amazon Translate for seamless support

Application logs are an essential piece of information that provides crucial insights into the inner workings of an application. This includes valuable information such as events, errors, and user interactions that would aid an application developer or an operations support engineer to debug and provide support. However, when these logs are presented in languages other than English, it creates a significant hurdle for developers who can’t read the content, and hinders the support team’s ability to identify and address issues promptly.

In this post, we explore a solution on how you can unlock language barriers using Amazon Translate, a fully managed neural machine translation service for translating text to and from English across a wide range of supported languages. The solution will complement your existing logging workflows by automatically translating all your applications logs in Amazon CloudWatch in real time, which can alleviate the challenges posed by non-English application logs.

Solution overview

This solution shows you how you can use three key services to automate the translation of your application logs in an event-driven manner:

CloudWatch Logs is used to monitor, store, and access your log files generated from various sources such as AWS services and your applications
Amazon Translate is used to perform the translation of text to and from English
AWS Lambda is a compute service that lets you run codes to retrieve application logs and translate them through the use of the Amazon Translate SDK

The following diagram illustrates the solution architecture.

The workflow consists of the following steps:

A custom or third-party application is hosted on an Amazon Elastic Compute Cloud (Amazon EC2) instance and the generated application logs are uploaded to CloudWatch Logs via the CloudWatch Logs agent.
Each log entry written to CloudWatch Logs triggers the Lambda function subscribed to the CloudWatch log group.
The function processes the contents of the log entry and uses Amazon Translate SDK translate_text to translate the log content.
The translated log content is returned to the function.
The function writes the translated log content back to CloudWatch Logs in a different log group.

The entire process happens automatically in real time, and your developers will be able to access the translated application logs from the CloudWatch log groups with no change in how your existing application writes logs to CloudWatch.

Prerequisites

To follow through the instructions in this solution, you need an AWS account with an AWS Identity and Access Management (IAM) user who has permission to AWS CloudFormation, Amazon Translate, CloudWatch, Lambda, and IAM.

Deploy the solution

To get started, launch the following CloudFormation template to create a Lambda function, two CloudWatch log groups, and IAM role. Proceed to deploy with the default settings. This template takes about 1 minute to complete.

After the stack is created successfully, you can review the Lambda function by navigating to the Lambda console and locating the function translate-application-logs.

You can observe that there is a CloudWatch Logs trigger added to the function.

You can view the details of the trigger configuration by navigating to the Configuration tab and choosing Triggers in the navigation pane.

You can confirm that the trigger has been configured to subscribe to log events from the log group /applicationlogs. This is where your non-English application logs will be written to.

Next, choose Environment variables in the navigation pane.

Two environment variables are provided here:

source_language – The original language that the application log is in (for example, ja for Japanese)
target_language – The target language to translate the application log to (for example, en for English)

For a list of supported languages, refer to Supported languages and language codes.

Next, go to the Code tab and review the function logic:

import json, boto3, gzip, base64, os

translate = boto3.client(service_name='translate', region_name=os.environ['AWS_REGION'], use_ssl=True)
logs = boto3.client('logs')
    
def lambda_handler(event, context):
    # retrieve log messages
    encoded_zipped_data = event['awslogs']['data']
    zipped_data = base64.b64decode(encoded_zipped_data)
    data = gzip.decompress(zipped_data)
    json_log = json.loads(data)
    logGroup = json_log['logGroup']+'-'+os.environ['target_language']
    logStream = json_log['logStream']
    
    # check  if log group exists, create if not    
    dlg = logs.describe_log_groups(logGroupNamePrefix=logGroup)
    if len(dlg['logGroups']) == 0:
        logs.create_log_group(logGroupName=logGroup)

    # check if log stream exists, create if not    
    dls = logs.describe_log_streams(logGroupName=logGroup, logStreamNamePrefix=logStream)
    if len(dls['logStreams']) == 0:
        logs.create_log_stream(logGroupName=logGroup, logStreamName=logStream)

    # translate log event messages from source language to target language
    for logevent in json_log['logEvents']:
        logevent['message'] = translate.translate_text(Text=logevent['message'], SourceLanguageCode=os.environ['source_language'], TargetLanguageCode=os.environ['target_language']).get('TranslatedText')
        del logevent['id']

    # write translated log events back to a different log group in CloudWatch
    logs.put_log_events(
        logGroupName = logGroup,
        logStreamName = logStream,
        logEvents = json_log['logEvents']
    )
    
    # return success
    return {
        'statusCode': 200,
        'body': 'Translation success!'
    }

Test the solution

Finally, to test the solution, you can create a log message through the CloudWatch console and choose the created log group and log stream.

After creating your log messages, you will be able to see it translated immediately.

Clean up

To clean up the resources created in this post, delete the CloudFormation stack via the CloudFormation console.

Conclusion

This post addressed the challenge faced by developers and support teams when application logs are presented in languages other than English, making it difficult for them to debug and provide support. The proposed solution uses Amazon Translate to automatically translate non-English logs in CloudWatch, and provides step-by-step guidance on deploying the solution in your environment. Through this implementation, developers can now seamlessly bridge the language barrier, empowering them to address issues swiftly and effectively.

Try out this implementation and let us know your thoughts in the comments.

About the author

Xan Huang is a Senior Solutions Architect with AWS and is based in Singapore. He works with major financial institutions to design and build secure, scalable, and highly available solutions in the cloud. Outside of work, Xan spends most of his free time with his family and documenting his daughter’s growing up journey.

View Original Source (aws.amazon.com) Here.

← previous - next →