Key takeaways for CIOs from AWS re:Invent 2024

AWS’s annual re:Invent developer conference concluded last week. In addition to technical advancements, the event highlighted strategic initiatives that resonate with CIOs, including cost optimization, workflow efficiency, and accelerated AI application development.

The cloud giant showcased several new features and updates to its popular offerings including Amazon SageMaker, Amazon Q Developer, Amazon Bedrock, databases, infrastructure, and Q Business.

While there was a long list of announcements, here are the key ones that directly impact CIOs.

Optimizing IT expenditure

On the infrastructure front, AWS announced that Amazon EC2 P6 instances will feature the new Blackwell chips from Nvidia. It made Trainium2-powered EC2 instances generally available and revealed plans for the Trainium3 chip. The announcements focused not only on providing more compute capacity but also better price performance when compared to rivals.

For instance, AWS Trainium2-powered EC2 instances are said to be four times faster, with four times the memory bandwidth, and three times more memory capability than the previous generation powered by Tranium1 — all of which combined are expected to offer 35% lower latency than Trainium1-powered instances. 

Another effort in the price-performance-efficiency direction was the Tranium3 chip — which is expected to provide two times more compute than Tranium2 and 40% more energy efficiency. 

On the model and model development front, AWS CEO Matt Garman unveiled new Nova foundational models, which he said would offer better value, especially in terms of cost, than most rival models.

The Bedrock updates included Model Distillation — a managed service aimed at helping enterprises bring down their cost of running LLMs, and prompt caching on Bedrock — designed to reduce the cost of prompting an LLM by storing prompts in a cached memory.

Amazon SageMaker’s AI module also received cost-saving features, including SageMaker Hyperpod’s Flexible Training Plans and Task Governance. The updates were designed to allow enterprises to utilize at least 90% of their purchased clusters and instances while reducing the time needed to complete LLM training, LLM inferencing, and AI model development tasks.

Streamlining workflows and boosting productivity

The next set of re:Invent announcements focused on streamlining workflows for enterprises and helping businesses boost the productivity of developers and data professionals.

AWS announced that it will unify analytics and AI services under its SageMaker service. This unification could help enterprises reduce IT integration overhead, and complexity.

This unification is perhaps best exemplified by a new offering inside Amazon SageMaker, Unified Studio, which combines SQL analytics, data processing, AI development, data streaming, business intelligence, and search analytics.

Another offering that AWS announced to support the integration is the SageMaker Data Lakehouse, aimed at helping enterprises unify data across Amazon S3 data lakes and Amazon Redshift data warehouses. 

AWS has also rolled out five new storage tools and services with the goal of making its offerings easier to work with, if not cheaper or better. 

Accelerating AI app development

As adopting AI is no more a question, getting there fast enough is often a challenge. To address this, Amazon announced a series of updates during the developer conference.

The first series of updates designed to ease developer tasks were made to Amazon Q Developer — the company’s answer to Microsoft’s GPT-driven Copilot generative AI assistant.

The new expanded capabilities for Q Developer include automating code reviews, unit tests, and generating documentation — all of which are aimed at helping developers accelerate their tasks.

Further, AWS unveiled several code translation capabilities for Q in preview, including the ability to modernize .Net apps from Windows to Linux, mainframe code modernization, and the ability to help migrate VMware workloads.

With Q Developer taking up the translation tasks, developers would be free to focus on other tasks, boosting productivity, Garman said.

Other updates added to AWS’ generative AI platform — Bedrock — included Bedrock Intelligent Prompt Routing, Amazon Kendra GenAI Index, Bedrock Knowledge Bases support for structured data, GraphRAG, and Bedrock Data Automation for unstructured data retrieval.

On the storage front, AWS unveiled S3 Table Buckets and the S3 Metadata features.

While S3 Table Buckets aims to ease the implementation of Apache Iceber tables inside the storage service, the S3 Metadata capability will allow enterprises to automatically generate metadata of objects stored in S3 so that finding data becomes easier, eventually reducing time for a variety of workloads, including analytics and AI.