By: Sanjay Basu, Ph.D, Akshai Parthasarathy

Generative AI (GenAI) is predicted to deliver value comparable to that of the Internet. While AI technology can be considered a solution to critical challenges faced by public and private organizations, the increasing complexity and scale of AI models are driving the need for high-performance, secure, and compliant solutions that meet the needs of large enterprises.

Organizations, including those in the public sector, must navigate regulatory requirements and applicable data privacy regulations, emphasizing the importance of sovereign clouds for data control and compliance. Meanwhile, for private enterprises, productivity can be hampered by time-consuming and error-prone tasks, and existing AI techniques may be limited in their ability to handle diverse data types and complex scenarios, highlighting the need for advancements that push the boundaries of AI capabilities.

Five key focus areas will shape the future of GenAI and tackle its challenges: AI infrastructure, sovereign clouds, agentic workflows, retrieval augmented fine-tuning (RAFT), and multi-modal AI. Solutions from these areas will help GenAI's computational power, data privacy, productivity, accuracy, and capability to handle diverse types of data.

AI Infrastructure: Running the most Demanding AI Workloads Faster

According to Goldman Sachs, AI is poised to drive 160 percent of all datacenter power demand. Behind models like Open AI GPT 4o and Meta Llama 3 are massive, scale-out infrastructures that consist of high-performance compute, storage, network, and software for AI. Datacenter power demand remained flat until 2019 and has steadily increased since then to meet the accelerated increase in workload demand.

While a significant amount of datacenter infrastructure is currently being used for AI training, with the advent of more capable, trained foundation models, we expect that AI inference may play an increasing role in datacenter power and workload consumption.

Sovereign AI: Achieving Digital Sovereignty and Control of AI Data

Sovereignty in the cloud is an important consideration for the public sector and for private enterprises. Cloud providers have already set up sovereign regions to help address these considerations. Sovereign clouds can control how the AI technologies are deployed and operated, including the hardware and software infrastructure used to build and operate the AI technologies, as well as the policies and personnel used to manage the AI technologies and protect the data. Sovereign clouds are powering key concepts like Sovereign AI and Sovereign LLMs.

The importance of Sovereign AI lies in its ability to address data residency. Sovereign AI also promotes innovation within local ecosystems by helping to enable countries to harness the potential of AI technologies while maintaining stronger operational control. This is particularly important for sensitive sectors such as the public sector and national security, where data integrity, security, and governance are paramount. As the world of AI evolves, sovereign clouds and Sovereign AI represent a new flexible choice for customers to maximize foundational and emerging technology where they need it.

AI-driven Agents: a Paradigm Shift in how we Compute

We are witnessing a paradigm shift in how we compute. Large language-based inference software now serves as both the interface and the computing platform, combining software and hardware accelerators. Users, programmers, and application developers interact with these new platforms using natural language. Applications developed on this new paradigm utilize foundational large language models for general interactions and route task-specific prompts to smaller language models trained on domain-specific data. This end-to-end interaction is known as an agentic workflow.

Agentic workflows powered by AI encompass a variety of tasks, such as helping enable automated financial management, where an AI application can be implemented to help scan bank statements, credit card expenses, and other financial data, extract and summarize relevant fields, and help generate an automated budget tracking spreadsheet without manual data entry. AI agents can also be implemented to help assist with targeted web searches, helping users find a new home in a neighborhood that suits their preferences or locate the best tutorials and materials for home projects. Some agentic workflows are also already in use, such as code generation and testing. AI can be implemented to help convert a developer’s specifications into code, perform documentation, and handle basic testing, eliminating the need for a developer to manage these tasks, thus streamlining the software development process by efficiently linking access to information with cognitive insights.