AWS Panorama now supports NVIDIA JetPack SDK 4.6.2

AWS Panorama is a collection of machine learning (ML) devices and a software development kit (SDK) that brings computer vision to on-premises internet protocol (IP) cameras. AWS Panorama device options include the AWS Panorama Appliance and the Lenovo ThinkEdge SE70, powered by AWS Panorama. These device options provide you choices in price and performance, depending on your unique use case. Both AWS Panorama devices are built on NVIDIA® Jetson System on Modules (SOMs) and use the NVIDIA JetPack SDK.

AWS has released a new software update for AWS Panorama that supports NVIDIA Jetpack SDK version 4.6.2. You can download this software update and apply it to the AWS Panorama device via an over-the-air (OTA) upgrade process. For more details, see Managing an AWS Panorama Appliance.

This release is not backward compatible with previous software releases for AWS Panorama; you must rebuild and redeploy your applications. This post provides a step-by step guide to updating your application software libraries to the latest supported versions.

Overview of update

The NVIDIA Jetpack SDK version 4.6.2 includes support for newer versions of CUDA 10.2, cuDNN 8.2.1, and TensorRT 8.2.1. Other notable libraries now supported include DeepStream 6.0 and Triton Inference Server 21.07. In addition, TensorRT 8.2.1 includes an expanded list of DNN operator support, Sigmoid/Tanh INT8 support for DLA, and better integration with TensorFlow and PyTorch. Torch to TensorRT conversion is now supported, as well as TensorFlow to TensorRT, without the need to convert to ONNX as an intermediate step. For additional details, refer to NVIDIA Announces TensorRT 8.2 and Integrations with PyTorch and TensorFlow.

You can redeploy your applications by following the steps in the following sections.

Prerequisites

As a prerequisite, you need an AWS account and an AWS Panorama device.

Upgrade your AWS Panorama device

First, you upgrade your AWS Panorama device to the latest version.

On the AWS Panorama console, choose Devices in the navigation pane.
Choose an Appliance.
Choose Settings.
Under System software, choose View Software update.
Choose System Software version 5.0 or above and then proceed to install this software.

Redeploy your application

If you do not use the Open GPU access feature as part of your application, you use the Replace feature on the AWS Panorama console. The Replace function rebuilds your model for the latest software.

On the AWS Panorama console, choose Deployed applications in the navigation pane.
Select an application.
Choose Replace.

For applications using the Open GPU access feature, upgrading typically involves allowing your container access to the underlying GPU hardware and deploying and managing your own models and runtime. We recommend using NVIDIA TensorRT in your application, but you’re not limited to this.

You also need to update the libraries of your Dockerfile. Typical libraries to update include CUDA 10.2, cuDNN 8.2.1, TensorRT 8.2.1, DeepStream 6.0, OpenCV 4.1.1, and VPI 1.1. As a note, all related CUDA/NVIDIA changes in the software stack can be found at JetPack SDK 4.6.2.

Now you rebuild the models for TensorRT 8.2.1 and update your package.json file with the updated assets. You can now build your container with the updated dependencies and models and deploy the application container to your Appliance using the AWS Panorama console or APIs.

At this point your, AWS Panorama applications should be able to use the Jetpack SDK version 4.6.2. AWS Panorama’s sample applications that are compatible with this version are located on GitHub.

Conclusion

With the new update to AWS Panorama, you must rebuild and redeploy your applications. This post walked you through the steps to update your AWS Panorama application software libraries to the latest supported versions.

Please reach out to AWS Panorama with any questions at AWS re:Post.