Favorite Mixture-of-Experts (MoE) is a sparsely-activated architecture that scales large language model (LLM) capacity while keeping per-token compute low. Because its operations account for a large share of total compute time, inefficiencies there directly raise request latency and cost. Therefore, optimizing MoE operations through kernel engineering offers the largest opportunity
Read More
Shared by AWS Machine Learning October 13, 2026
Favorite SpaceXAI’s Grok 4.7 is now available on Amazon Bedrock, which offers a choice of foundation models (FMs) from multiple providers. According to SpaceXAI, Grok 4.7 is trained to work longer on more difficult tasks and knows when to check its own work. That matters most for agents, which carry
Read More
Shared by AWS Machine Learning October 13, 2026
Favorite This post is co-written with Kenneth Salmon, Edward Boswell, Jack Armstrong, Robert Bahn, and Nick Ciubotariu from ShipStation Global. Answering the same IT questions and clearing routine tickets drains time that IT teams could spend elsewhere. To reclaim it, the ShipStation Global Engineering team built Deckhand, an AI-powered IT
Read More
Shared by AWS Machine Learning October 13, 2026
Favorite Security analysts need fast answers about their cloud infrastructure without deep familiarity with domain-specific query languages. Orca Security’s tool covers over 900 cloud models (the resource types Sonar tracks, such as an Amazon Elastic Compute Cloud (Amazon EC2) instance or an Amazon Simple Storage Service (Amazon S3) bucket). Querying
Read More
Shared by AWS Machine Learning October 13, 2026
Favorite Managing AI model access across dozens or hundreds of AWS accounts creates operational friction when each workload account must subscribe and accept AWS Marketplace Private Pricing Offers (MPPOs) individually. Amazon Bedrock with managed entitlements simplifies multi-account access to third-party AI models so you can subscribe once from a central
Read More
Shared by AWS Machine Learning October 13, 2026
Favorite We’re giving developers, enterprises, scientists, and consumers access to Gemini 4 Argon, our latest frontier model. View Original Source (blog.google/technology/ai/) Here.
Favorite This post shows how to deploy one large language model (LLM) across two GPU nodes on Amazon Elastic Kubernetes Service (Amazon EKS) with the multi-node Ray LLM Deep Learning Containers. It uses separate prefill and decode nodes behind one OpenAI-compatible endpoint. This kind of multi-node setup is common for
Read More
Shared by AWS Machine Learning October 12, 2026
Favorite Applications that route requests, select agent actions, or assign policy outcomes often need scores over a fixed set of choices instead of free-form text. Deploying these decision models with llama.cpp on Amazon SageMaker AI gives you a managed path for serving quantized models on CPU or GPU infrastructure. This
Read More
Shared by AWS Machine Learning October 12, 2026
Favorite Single-Region inference can throttle when demand exceeds local capacity. Teams in Thailand, Malaysia, Singapore, Indonesia, and Taiwan need access to the newest OpenAI models beyond that capacity. Amazon Bedrock offers OpenAI GPT-6 Sol and GPT-6 Luna through global cross-Region inference from all five AWS Regions, and GPT-6 Astra from
Read More
Shared by AWS Machine Learning October 12, 2026
Favorite When an AI agent runs, it often needs to buy something to finish a task: a model inference, an API response, access web content, or a call to another agent. These purchases are small and frequent, sometimes a fraction of a cent each, and they happen inside the agent’s
Read More
Shared by AWS Machine Learning October 9, 2026
Shared by AWS Machine Learning October 13, 2026
