How Tata Elxsi detects industrial safety risks in seconds on AWS

Favorite

Detecting industrial safety risks in seconds, not minutes, is what keeps workers safe on an active plant floor. This is what Tata Elxsi set out to deliver by building IRIS, a real-time industrial safety platform on AWS.

In this post, we show how Tata Elxsi built IRIS (Industrial Real-Time Intelligence System). We cover the architecture decisions, the implementation approach, and the measurable results you can expect from a similar build. Whether you operate a handful of cameras or thousands across multiple sites, this blueprint provides patterns you can adapt for your organization.

About Tata Elxsi

Tata Elxsi is a global provider of design and technology services across industries including automotive, manufacturing, broadcast, communications, healthcare, and transportation. Its teams combine engineering depth with AI and computer vision to help enterprises modernize safety-critical physical operations. IRIS is Tata Elxsi’s industrial vision platform, built for organizations that already operate camera infrastructure but cannot yet turn those feeds into real-time, actionable intelligence.

The customer challenge

Industrial organizations have invested heavily in automated safety over the past decade. Manufacturing plants, warehouses, logistics hubs, and chemical facilities operate hundreds to thousands of cameras. These cover production lines, hazardous zones, vehicle corridors, loading areas, and restricted-access locations. Yet most of this footage is recorded and rarely acted on in real time. Safety teams face a common set of constraints:

• Reactive monitoring — Closed-circuit television (CCTV) functions as a recording system rather than a prevention system, so incidents surface only after they occur.
• Human monitoring limits — A control-room operator cannot reliably watch hundreds of feeds at once. Detection of unsafe conditions typically takes 15–45 minutes, depending on operator availability.
• Inconsistent compliance — Policy enforcement varies across shifts and sites, with audit coverage limited to two or three manual walkthroughs per shift.
• Uneconomical scaling — Adding cameras increases monitoring cost without a proportional improvement in safety outcomes.

These aren’t failures of any single tool. They were signals that safety monitoring needs to evolve from passive recording to continuous, automated detection that scales with the number of cameras.

Why real-time computer vision?

Computer vision represents the next step in workplace safety. It doesn’t replace existing safety programs. It augments them with continuous, automated monitoring that runs around the clock. The design goal for IRIS was to analyze video as it’s produced, detect unsafe conditions automatically, and generate actionable alerts in near real time, without streaming raw video to the cloud. IRIS runs in the Asia Pacific (Mumbai) AWS Region, chosen for data-residency requirements and low-latency proximity to customer facilities in India.

Solution overview

Tata Elxsi built IRIS as a serverless, event-driven pipeline that follows a repeatable pattern: observe at the edge, detect with computer vision, analyze for context, alert the right people, store for compliance, and learn from production data. Video is analyzed at the edge, only safety-relevant frames and structured metadata move to the cloud, and a correlation layer turns raw detections into high-confidence safety events. The following diagram shows how the components fit together.

Edge acquisition and processing: Filtering at the source

The workflow begins at the edge. IRIS deploys a dedicated edge-compute tier using AWS IoT Greengrass on industrial-grade, GPU-equipped edge servers, for example NVIDIA Jetson AGX Orin or equivalent. Each server is installed at the facility and connected to the camera network over RTSP/ONVIF.

At the edge, IRIS extracts frames at a configurable rate of 2-5 frames per second and applies motion-based filtering. It then runs a lightweight first-pass model to identify frames that contain people, vehicles, or equipment. Frames that pass these filters are uploaded to Amazon Simple Storage Service (Amazon S3). With AWS IoT Greengrass, you can manage secure device communication and deliver updated models to devices as Greengrass components from Amazon S3.

Decoupling the image path from the metadata path is central to the design. Extracted frames are written to a dedicated Amazon S3 bucket, partitioned by camera, date, and hour. The streaming event that flows through the pipeline carries only the Amazon S3 object key and context such as camera ID, plant, zone, and an NTP-synchronized timestamp. This keeps each event under 1 KB. A downstream consumer retrieves the referenced frame from Amazon S3 and runs the model. This keeps the streaming layer lightweight while the models retain full access to the visual data. In Tata Elxsi’s production deployments, filtering at the edge reduces the volume of frames sent to the cloud by roughly 70–80 percent, based on the customer’s production measurements.

Real-time event streaming: The event backbone

After edge processing, safety-relevant metadata and events are streamed into Amazon Kinesis Data Streams, which serves as the real-time event backbone of the platform. The stream carries frame metadata (the Amazon S3 object key), motion events, edge detection candidates, camera telemetry, and contextual safety information. It does not carry video.

Because IRIS performs frame extraction at the edge, the cloud payload is structured event data with Amazon S3 references rather than continuous video. Amazon Kinesis Data Streams is purpose-built for this event-driven, metadata-first pattern, where sub-second latency on structured records is the priority.

The stream runs in on-demand capacity mode, which removes manual shard management and scales throughput automatically with event volume during shift changes or multi-incident bursts. In Tata Elxsi’s production deployments, sustained throughput is 2,000–5,000 events per second per deployment, with burst capacity to roughly 15,000 events per second. Measured event-ingestion latency is under 200 milliseconds at p95.

Vision AI inference: The intelligence engine

Events are consumed by custom computer vision models deployed on Amazon SageMaker AI, the intelligence engine of IRIS. Separate real-time endpoints are provisioned per model family so each can scale independently:

• Personal protective equipment (PPE) compliance uses custom YOLOv8 object detection fine-tuned on industrial datasets to detect helmets, reflective jackets, gloves, and safety glasses.
• Restricted-zone monitoring combines object detection with polygon-based spatial geofencing for intrusion and boundary violations.
• Worker safety analytics uses SlowFast-based temporal action recognition for unsafe posture, movement, and interactions.
• Vehicle and equipment proximity uses multi-object tracking with monocular depth estimation for forklift and machine-proximity risks.

Endpoints run on ml.g5.xlarge instances (NVIDIA A10G GPU). AWS Application Auto Scaling applies a target-tracking scaling policy that scales out at 70 percent GPU utilization and scales in at 30 percent, with a minimum of two instances per endpoint for high availability (HA). To smooth traffic bursts across hundreds of concurrent streams, IRIS places an Amazon Simple Queue Service (Amazon SQS) queue between the stream consumers and the endpoints. Application Auto Scaling then adds instances when queue depth exceeds a configured threshold. Requests are processed in micro-batches of 4–8 frames to maximize GPU utilization. In production, each ml.g5.xlarge endpoint handles roughly 40-60 inference requests per second, and per-frame inference latency is under 300 milliseconds at p95.

Event correlation: From detections to high-confidence events

A single detection is often not enough to act on. A worker briefly crossing a boundary might not warrant escalation, whereas repeated violations in a short window might require immediate intervention. IRIS therefore adds a correlation layer, implemented as AWS Lambda functions that maintain short-term state in Amazon DynamoDB using time-to-live (TTL) entries for sliding-window evaluation. It combines detections with camera location, zone criticality, temporal patterns (configurable 30-second to 5-minute windows), and historical behavior, then evaluates violation frequency, duration, and severity. In Tata Elxsi’s production deployments, this correlation step reduces spurious alerts by an estimated 40–50 percent compared with passing detections through directly. This is based on the customer’s internal benchmarking of alert volumes before and after correlation.

Alert generation and automated response

After a high-confidence event is identified, AWS Lambda functions run event-driven response workflows and AWS Step Functions manage multi-step escalation. Events are de-duplicated with a sliding window. The same detection type from the same camera within a configurable window (default 60 seconds) is consolidated into one alert. Events are then classified by severity based on zone criticality, confidence, and duration. Escalation follows defined service-level agreements:

• Critical — Alert within 5 seconds, escalate if unacknowledged within 2 minutes.
• High — Alert within 10 seconds, escalate if unacknowledged within 5 minutes.
• Medium — Batched into digest notifications.
• Low — Logged for trend analysis, with no real-time alert.

Amazon EventBridge Scheduler triggers escalation checks, and AWS Step Functions advance the state machine through supervisor, plant-manager, and safety-director levels as needed. Alerts are delivered through a real-time safety dashboard, email and SMS by severity and recipient group, webhook integration with enterprise IT service management systems, and mobile push notifications for supervisors and safety officers.

Persistent storage and compliance: The system of record

Every event, including detection results, alert records, metadata, and investigation evidence, is stored in Amazon S3, the system of record for the platform. Organizations use this repository for safety audits, compliance reporting, root-cause investigation, and regulatory review.

Lifecycle policies manage cost as data ages. Active event data stays in S3 Standard for 30 days. Historical events move to S3 Standard-Infrequent Access from 30 to 90 days. Compliance and investigation records transition to S3 Glacier Instant Retrieval from 90 days to 1 year, which allows millisecond retrieval for audits. Long-term archival moves to S3 Glacier Flexible Retrieval beyond 1 year, with expiration configurable per customer retention requirements. Extracted frames tied to confirmed events are retained for 1 year and then archived, and frames with no or below-threshold detections are purged after 7 days.

Continuous learning: Improving with production data

IRIS improves as it runs. Production data in Amazon S3 feeds model-improvement workflows through Amazon SageMaker AI training pipelines. Training data is roughly 80 percent real-world annotated data collected from production environments under customer data agreements. The remaining 20 percent is synthetic data generated for rare cases such as uncommon PPE, unusual lighting, and atypical camera angles. Annotation combines Amazon SageMaker Ground Truth for large-scale labeling with an in-house Tata Elxsi review team for edge cases. An active-learning loop routes low-confidence production predictions for human review.

Re-training runs on three separate triggers:

• Scheduled — A quarterly baseline retraining cycle using accumulated production data.
• Drift-based — Model monitoring in Amazon SageMaker AI detects accuracy degradation and triggers re-training when accuracy drops below a configured threshold.
• Feedback-driven — Newly annotated samples above a threshold volume trigger an incremental training job.

Re-trained models are evaluated against a held-out evaluation set using the Amazon SageMaker AI model registry. Only models that meet or exceed current production accuracy are promoted, through blue/green deployment. As measured by Tata Elxsi on held-out production validation sets refreshed quarterly, PPE detection reaches 94.2 percent precision and 91.8 percent recall (mAP@0.5 of 92.7 percent). Restricted-zone intrusion reaches 96.1 percent precision and 93.4 percent recall. The post-correlation false-positive rate is under 3 percent across detection categories.

Security, privacy, and compliance

Security is enforced across a multi-account structure that separates model training, production inference, and analytics. Amazon S3 buckets use server-side encryption with customer-managed keys in AWS Key Management Service (AWS KMS), and inter-service communication uses TLS 1.2 or higher. Fine-grained AWS Identity and Access Management (IAM) policies scope each service role to least privilege, and human access uses AWS IAM Identity Center with roles aligned to job function.

Inference and data-processing workloads run inside a dedicated Amazon Virtual Private Cloud (Amazon VPC) with private subnets and no public internet exposure. VPC endpoints keep Amazon S3, Amazon Kinesis Data Streams, and Amazon SageMaker AI traffic on the AWS network. AWS CloudTrail records API activity, and Amazon GuardDuty monitors for anomalous access.

The results: Measurable business impact

Across production deployments, IRIS moved customers from reactive surveillance to proactive safety management. Tata Elxsi reports the following outcomes.

Dimension	Before IRIS	With IRIS (reported by Tata Elxsi)
Unsafe-condition detection	Manual review, 15–45 minutes	Under 5 seconds, end to end
Safety audit coverage	2–3 manual walkthroughs per shift	Continuous, automated 24×7 coverage
Recordable safety incidents	Baseline	15–20% reduction in the first 6 months
Manual surveillance operating cost	Baseline	Approximately 30% reduction
Scaling model	Cost grows with each added camera	Hundreds of concurrent streams per site

Key takeaways: Lessons for real-time safety platforms

1. Filter at the edge, stream metadata, not video — Extracting frames at the edge and streaming only Amazon S3 references keeps the cloud pipeline lightweight and cuts data-movement cost, while the models still get full access to the image.
2. Correlation is what makes alerts trustworthy — Raw detections produce noise. A temporal correlation layer turns them into high-confidence events and prevents the alert fatigue that causes teams to stop trusting the system.
3. Design for the model that will change — A retraining loop driven by production data, drift monitoring, and active learning is what keeps accuracy high as sites, lighting, and camera angles vary.
4. Governance and privacy are day-one decisions — For footage of identifiable people, anonymization, retention, and access control belong in the first design review, not the last.

Conclusion

IRIS shows how existing camera infrastructure can become a real-time safety system on AWS, detecting unsafe conditions in seconds rather than minutes. By filtering at the edge, streaming metadata, running purpose-built models on Amazon SageMaker AI, and adding a correlation layer, Tata Elxsi built a platform that scales across hundreds of concurrent streams per site while keeping raw video out of the cloud. The same event-driven foundation extends to quality inspection, perimeter monitoring, and process observation, with new domain models and zone rules layered on without re-architecting the pipeline.

To explore building a similar solution, review the AWS IoT Greengrass and Amazon SageMaker AI documentation. To discuss a proof of concept for your facilities, contact Tata Elxsi or your AWS account team.

---

About the authors

View Original Source (aws.amazon.com) Here.