While training will continue to be pivotal in the production of robust AI models, the future will be focused on inference — the art of deploying these models to deliver real-time, actionable insights and outcomes for businesses and consumers alike — while injecting dynamic feedback loops from the edge back into the training process, fostering a cycle of continuous model improvement and adaptation.

AI inference is the point at which AI transforms from the promise of potential into practical application with real-world impact.  Our customers are employing AI inference across a spectrum of industries and use cases. These include:

• Smart cities: Optimizing traffic flow to reduce congestion and enhance safety; improving public security through intelligent surveillance
• Autonomous vehicles: Enabling split-second decision-making and facilitating efficient vehicle platooning
• Industrial Internet of Things (IoT) and manufacturing: Implementing predictive maintenance to prevent downtime; enhancing quality control through real-time video analysis
• Smart retail: Delivering hyperpersonalized shopping experiences and streamlining operations with smart checkouts and inventory management
• Healthcare and telemedicine: Monitoring patients in real time, accelerating medical diagnoses through image processing, and powering advanced wearable devices
• Media and entertainment: Curating personalized content, enabling real-time video transcoding and live stream enhancement

These examples merely scratch the surface of what customers can achieve with AI inferencing. As edge computing continues to evolve, we anticipate even more innovative applications across diverse sectors.

The delivery of such innovation has created some common challenges, including latency, cost, and scalability. At Akamai, we’ve been solving these issues in various contexts for decades.

The approach of consolidating swathes of generalized, overpowered GPUs into centralized data centers is no longer sufficient to deliver the outputs of well-trained AI models at the scale and with the responsiveness that the masses demand. We need to adopt an entirely new paradigm to enable inference architectures to be closer to users; that is, via a distributed cloud model.

There are unique considerations when delivering AI inference via a distributed cloud model, including:

• Latency and responsiveness: Transferring data back and forth between centralized cloud data centers negatively impacts user experience and can cause lost business opportunities. Decentralized, distributed architectures improve inference response times.
• Resource constraints: Edge devices face constraints on power, storage, and compute capabilities. Deploying lightweight, efficient AI models that deliver robust performance within these limitations is crucial.
• Security and data privacy: Local data processing enhances security by reducing data exposure during transit. This is particularly important for industries like healthcare, finance, and government that must comply with strict data locality and privacy regulations.
• Scalability and distributed architecture: As the number of distributed locations that host an AI application grows, managing and updating AI models across the network becomes increasingly complex. Scalable solutions for model deployment and maintenance are essential.
• Bandwidth and cost efficiency: AI inference running in a decentralized manner significantly reduces the amount of data transmitted to centralized cloud servers. This not only alleviates network congestion but also leads to substantial cost savings in data transfer and storage.

These considerations are critical aspects of deploying AI on distributed, decentralized cloud infrastructure and are drawing the focus of organizations looking to use AI effectively in their businesses.

At Akamai, we’re building the world’s most distributed cloud. Our extensive global infrastructure, developed over nearly 30 years, includes 25+ core compute regions, a rapidly expanding set of distributed compute locations, and more than 4,000 edge points of presence. This robust ecosystem is primed to meet the AI inference needs of organizations, for today and tomorrow.

We recognize that while customers demand high performance, they’re increasingly wary of the exorbitant cost overruns that are common with traditional cloud vendors. Akamai Cloud is designed to address this growing concern.

Instead of stockpiling expensive, generalized GPUs that are overkill for AI inference tasks, we’ve opted to provide customers with a balanced GPU alternative: Nvidia’s RTX 4000 Ada series GPUs which offer a blend of performance and cost efficiency, making them ideal for AI inferencing and running small language models, and for specialized workloads like media transcoding.

This approach allows us to deliver superior AI capabilities closer to users, while maintaining cost-effectiveness for our customers. Our testing has shown more than 80% cost savings when running a generative AI Stable Diffusion model, when compared with equivalent GPU alternatives that are available on traditional public cloud providers.

We believe this approach yields the most powerful and cost-efficient outcomes, and can encourage novel AI use cases.

As we continue to enhance AI’s usefulness, we believe that distributed inferencing is more than just a technological advancement — it’s a fundamental reimagining of how we use AI. The shift from centralized, resource-intensive computing to a continuum of distributed, efficient edge computing isn’t just inevitable, it’s already underway.

At Akamai, we’re not just observing the transformation — we’re actively shaping it. By combining the strength of our global distributed network, strategic cloud computing investments (including inference-optimized GPUs), and a deep understanding of performance and cost-efficiency, we’re focused on enabling organizations to unlock the true potential of AI inference.

Organizations have recognized that it isn’t a question of if but rather how they should embrace AI inference. The edge is no longer just a destination for data — it’s becoming the primary arena where AI delivers its most impactful, real-time insights. Welcome to the next generation of computing.