In an AI factory, high bandwidth memory (HBM)is the fastest, most expensive, and most contested memory tier. Every byte spent on a redundant weight copy, an idle session’s KV cache, or a recomputed prefix is a byte that is not being used to produce tokens. The question is no longer how much memory a GPU has, but how much useful work each byte delivers.
Official announcementNVIDIA Developer Blog
Five Ways NVIDIA Co-Design Maximizes the Value of Every Byte in the AI Factory
From the official release
The generational data in Figure 1 below shows why. From GB300 to NVIDIA Rubin, capacity held at 288 GB while bandwidth rose to 19.2 TB/s.
agentic throughput per megawatt grew far faster. On SemiAnalysis AgentX, which replays recorded coding-agent sessions with real tool-call gaps and growing context, NVIDIA Vera Rubin NVL72 then delivers up to 30x the throughput per megawatt of GB300 NVL72 at 160 tokens per second per user.
This is a short excerpt. Read the full announcement on the official source.
Continue on developer.nvidia.com → (opens in a new window)Related stories
Official announcement
Amazon Bedrock AgentCore now offers a Web Search API
AWS What’s New
Today, AWS announces the general availability of the Web Search API on Amazon Bedrock AgentCore, which lets you search the web with a single API call. Web Search on AgentCore has been available as an MCP tool through AgentCore Gateway. The new InvokeWebSearch API gives applicatio
Official announcement
Amazon CloudWatch Logs announces account-level retention policies
AWS What’s New
Amazon CloudWatch Logs now supports account-level retention policies, so you can use a single policy to set how long log data is kept across all or subset of log groups in an AWS account. Account-level policies allow platform and security teams to bring each log group in line wit
Official announcement
AWS Glue adds 100 new native connectors for popular SaaS applications
AWS What’s New
AWS Glue now supports 100 new native connectors that simplify access to data in popular SaaS and REST API-based applications. Previously, bringing data from SaaS applications into AWS required writing custom code. Now, teams can create a connection once in the AWS Glue Data Catal
Official announcement
Databricks' Commitment to European Sovereignty
Databricks Blog
Over a Decade of Investment in EuropeDatabricks has been building in Europe since...
Official announcement
Make a Robot Arm Feel Weightless with NVIDIA Holoscan 5
NVIDIA Developer Blog
Picture guiding a robot arm by hand while its motors supply just enough torque to support its weight and keep the robot upright. This simple interaction, a... Picture guiding a robot arm by hand while its motors supply just enough torque to support its weight and keep the robot u
Official announcement
Accelerating Reinforcement Learning Weight Refit with the Reshard Collective
NVIDIA Developer Blog
During reinforcement learning (RL) post-training, training trajectories are generated on the fly by running inference with the current policy model. For... During reinforcement learning (RL) post-training, training trajectories are generated on the fly by running inference with t