Create forecasting systems faster with automated workflows and notifications in Amazon Forecast

You can now enable notifications for workflow status changes while using Amazon Forecast, allowing you to work seamlessly without the disruption of having to check if a particular workflow has completed. Additionally, you can now automate workflows through the notifications to increase work efficiency. Forecast uses machine learning (ML) to generate more accurate demand forecasts, without requiring any prior ML experience. Forecast brings the same technology used at Amazon.com to developers as a fully managed service, removing the need to manage resources or rebuild your systems.

Previously, you had to proactively check to see if a job was complete at the end of each stage, whether it was importing your data, training the predictor, or generating the forecast. The time needed to import your data or train a predictor can vary depending on the size and contents of your data. The wait time can feel even longer when you have to constantly check the status before being able to proceed to the next task. The work flow disruption can negatively impact the entire day’s work. Additionally, if you were integrating Forecast into software solutions, you had to build notifications yourself, creating additional work.

Now, with a one-time setup of workflow notifications, you can choose to either be notified when a specific step is complete or set up sequential workflow tasks after the preceding workflow is complete, which eliminates administrative overhead. Forecast enables notifications by onboarding to Amazon EventBridge, which lets you activate these notifications either directly through the Forecast console or through APIs. You can customize the notification based on your preference of rules and selected events. You can also use EventBridge notifications to fully automate the forecasting cycle end to end, allowing for an even more streamlined experience using Forecast. Software as a service (SaaS) providers can set up routing rules to determine where to send generated forecasts to build applications that react in real time to the data that is received.

EventBridge allows you to build event-driven Forecast workflows. For example, you can create a rule that when data has been imported into Forecast, the completion of this event triggers the next step of training a predictor through AWS Lambda functions. We explore using Lambda functions to automate the Forecast workflow through events in the next section. Or, after the predictor has been trained, you can set up a new rule to receive an SMS text message notification through Amazon Simple Notification Service (Amazon SNS), reminding you to return to Forecast to evaluate the accuracy metrics of the predictor before proceeding to the next step. For this post, we use Lambda with Amazon Simple Email Service (Amazon SES) to send notification messages. For more information, see How do I send email using Lambda and Amazon SES?

Solution overview

In this section, we provide an example of how you can automate Forecast workflows using EventBridge notifications, from importing data, training a predictor, and generating forecasts.

It starts by creating rules in EventBridge that can be accessed through the API, SDK, CLI, and the Forecast console. You can also see the demonstration in the next section. For this use case, we select the target for all the rules as a Lambda function. For instructions on creating the functions and adding the necessary permissions, see Steps 1 and 2 in Tutorial: Schedule AWS Lambda Functions Using EventBridge.

You create rules for the following:

Dataset import – Checks whether the status field in the event is ACTIVE and invokes the Forecast Create Predictor
Predictor – Checks whether the status field in the event is ACTIVE and invokes the Forecast Create Forecast
Forecast – Checks whether the status field in the event is ACTIVE and invokes the Forecast Create Forecast Export
Forecast Export – Checks whether the status field in the event is ACTIVE and it invokes Amazon SES to send an email. At this point, the forecast export results are already exported to your Amazon Simple Storage Service (Amazon S3) bucket.

After you set up the rules, you can start with your first workflow of calling the dataset import job API. Forecast starts sending status change events with statuses like CREATE_IN_PROGRESS, ACTIVE, CREATE_FAILED, and CREATE_STOPPED to your account. After the event gets matched to the rule, it invokes the target Lambda function configured on the rule, and moves to the next steps of training a predictor, creating a forecast, and finally exporting the forecasts. After the forecasts are exported, you receive an email notification.

The following diagram illustrates this architecture.

Create rules for Forecast notifications through EventBridge

To create your rules for notifications, complete the following steps:

On the Forecast console, choose your dataset.
In the Dataset imports section, choose Configure notifications.

Links to additional information about setting up notifications is available in the help pane.

You’re redirected to the EventBridge console, where you now create your notification.

In the navigation pane, under Events, choose Rules.
Choose Create rule.
For Name, enter a name.
Under Define pattern, select Event pattern.
For Event matching patterns, select Pre-defined pattern by service.
For Event type, choose your event on the drop-down menu.

For this post, we choose Forecast Dataset Import Job State Change because we’re interested in knowing when the dataset import is complete.

When you choose your event, the appropriate event pattern is populated in the Event pattern section.

Under Select event bus, select AWS default event bus.
Confirm that Enable the rule on the select event bus is enabled.
For Target, choose Lambda function.
For Function, choose the function you created.
Choose Create.

Make sure that the rule and targets are in the same Region.

You’re redirected to the Rules page on the EventBridge console, where you can see a confirmation that your rule was created successfully.

Conclusion

You can now enable notifications for workflow status changes while using Forecast. With a one-time setup of workflow notifications, you can choose to either get notified or set up sequential workflow tasks after the preceding workflow has completed, eliminating administrative overhead.

To get started with this capability, see Setting Up Job Status Notifications. You can use this capability in all Regions where Forecast is publicly available. For more information about Region availability, see AWS Regional Services.

About the Authors

Alex Kim is a Sr. Product Manager for Amazon Forecast. His mission is to deliver AI/ML solutions to all customers who can benefit from it. In his free time, he enjoys all types of sports and discovering new places to eat.

Ranjith Kumar Bodla is an SDE in the Amazon Forecast team. He works as a backend developer within a distributed environment with a focus on AI/ML and leadership. During his spare time, he enjoys playing table tennis, traveling, and reading.

Raj Vippagunta is a Senior SDE at AWS AI Services. He leverages his vast experience in large-scale distributed systems and his passion for machine learning to build practical service offerings in the AI space. He has helped build various solutions for AWS and Amazon. In his spare time, he likes reading books and watching travel and cuisine vlogs from across the world.

Shannon Killingsworth is a UX Designer for Amazon Forecast and Amazon Personalize. His current work is creating console experiences that are usable by anyone, and integrating new features into the console experience. In his spare time, he is a fitness and automobile enthusiast.