Recognize celebrities in images and videos using Amazon Rekognition

The celebrity recognition feature in Amazon Rekognition automatically recognizes tens of thousands of well-known personalities in images and videos using machine learning (ML). Celebrity recognition significantly reduces the repetitive manual effort required to tag produced media content and make it readily searchable. Starting today, we’re updating our models to provide higher accuracy (lower false detections and rejections) and increased coverage of global celebrities. In addition, you get three new attributes for each celebrity recognized: presentation of gender, expression, and smile. This metadata helps further refine content search and filtering workflows. For example, you can now search for images where a particular actor is smiling, or measure the coverage of female versus male celebrities in event photos to ensure fair representation.

With the rapid proliferation in the volume of image and video content available on video on demand (VOD), streaming, and social media platforms, media companies are struggling to organize, search, and fully use their catalogs at scale. Similarly, news channels often need to rapidly locate images and videos of a particular celebrity in response to current events, but insufficient metadata makes it tedious to search their libraries for the right clips. Sports broadcasters also find it hard to quickly find the right footage from games and interviews to create highlights, shorts, and special programming. With Amazon Rekognition, you can automatically tag large volumes of new or archival content to make it instantly searchable for a comprehensive set of international, widely known celebrities such as actors, sportspeople, and online content creators.

The celebrity recognition feature for Amazon Rekognition is designed to be exclusively used in cases where you expect to find a known celebrity in a produced image or a video. Feature updates for Amazon Rekognition Image are available today in all supported Regions. Celebrity recognition updates for Amazon Rekognition Video will be made available at a later point in time.

How it works

You can add celebrity recognition capabilities to your applications by downloading the latest AWS SDK. The celebrity recognition feature can accept an image file stored in Amazon Simple Storage Service (Amazon S3) up to 15 MB in size or directly as a base-64 string up to 5 MB. You get back a JSON response with the recognized celebrities, along with other useful attributes such as bounding box, pose, and expressions. Let’s look at the API request and response in more detail.

The following example code is the image API request:

{
   "Image": { 
     "Bytes": blob,
     "S3Object": { 
       "Bucket": "string",
       "Name": "string",
       "Version": "string"
     }
   }
}

We get the following response:

{
   "CelebrityFaces": [ 
      { 

         "MatchConfidence": 99.87883758544922,
         "Name": "Werner Vogels",
         "Id": "23iZ1oP",
         "KnownGender": { 
            "Type": “Male”
         },
         "Urls": [
            "en.wikipedia.org/wiki/Werner_Vogels”
	],
         "Face": { 
            "BoundingBox": { 
               "Height": 0.288828045129776,
               "Left": 0.6104722023010254,
               "Top": 0.3479453921318054,
               "Width": 0.13631291687488556
            },
            "Confidence": 99.98967742919922,
            "Emotions": [ 
               { 
                  "Confidence": 79.22383117675781,
                  "Type": "CALM"
                  …
               }
            ],
            "Landmarks": [ 
             { 
               "Type": "mouthLeft", 
               "X": 0.4502509534358978,
               "Y": 0.6147698163986206 
             }
             …
            ],
            "Pose": { 
               "Pitch": 1.636465311050415,
               "Roll": -2.2817516326904297,
               "Yaw": 3.821389675140381
            },
            "Quality": { 
               "Brightness": 72.9759750366211,
               "Sharpness": 92.22801208496094
            },
            "Smile": { 
               "Confidence": 98.39344024658203,
               "Value": false
            }
         },
      }
   ],
   "UnrecognizedFaces": []
}

You can filter further or organize the recognized celebrities based on certain attributes. For example, you may wish to only look for smiling, front-facing images of a famous sportsperson. The following are some key response elements:

• MatchConfidence – A match confidence score that can be used to control API behavior. We recommend applying a suitable threshold to this score in your application to choose your preferred operating point. For example, by setting a threshold of 99%, you can eliminate false alarms but may miss some potential matches.
• Name, ID, and URLs – Celebrity name, a unique Amazon Rekognition ID, and list of URLs such as the celebrity’s IMDb or Wikipedia link for further information.
• BoundingBox – Coordinates of the rectangular bounding box location for each recognized celebrity face.
• Known Gender – Known gender identity for each recognized celebrity.
• Emotions – Emotion expressed on the celebrity’s face, for example, happy, sad or angry.
• Pose – Pose of the celebrity face, using three axes of roll, pitch, and yaw.
• Smile – Whether the celebrity is smiling or not.

Celebrity Recognition in videos

Celebrity recognition for Amazon Rekognition Video is an asynchronous API. To recognize celebrities in a stored video, use StartCelebrityRecognition to start video analysis. Amazon Rekognition Video publishes the job completion status of the video analysis to an Amazon Simple Notification Service (Amazon SNS) topic. If the video analysis is successful, call GetCelebrityRecognition to get the analysis results. For more information about starting video analysis and getting the results, see Calling Amazon Rekognition Video operations.

The latest quality updates and new attributes described in this post will be added to Amazon Rekognition Video at a later date, but you can still use the previous model today. Streaming live video is not yet supported. In this case, we recommend sampling video frames at a suitable rate to generate images and running them through Amazon Rekognition image APIs.

Getting started

You can try out the celebrity recognition feature with your images or extracted video frames on the Amazon Rekognition console.

To help you to visualize the results created by the celebrity recognition API in conjunction with other APIs such as label detection, check out the Content Analysis solution, an open-source code project on GitHub. This demo application catalogs videos and images with data generated by AWS ML services for computer vision and speech detection. A graphical user interface (GUI) enables users to search through the catalog to find videos or images containing certain content and analyze it.

Summary

In this post, we talked about how media companies (broadcasters, VOD platforms, sports and news channels, and social media platforms) can use the Amazon Rekognition celebrity recognition feature to easily tag produced images or video content with tens of thousands of well-known celebrities from across the globe. To get started, you can test your images on the Amazon Rekognition console, or use AWS solutions such as the Content Analysis solution. You can start incorporating celebrity recognition into your applications by downloading the latest AWS SDK, checking out Recognizing celebrities in an image, and referring to the Amazon Rekognition pricing page for billing.

View Original Source (aws.amazon.com) Here.