NVIDIA AI Foundry Builds Custom Llama 3.1 Generative AI Models for the World’s Enterprises

NVIDIA announced a new NVIDIA AI Foundry service and NVIDIA NIM inference microservices to supercharge generative AI for the world’s enterprises with the Llama 3.1 collection of openly available models.

With NVIDIA AI Foundry, enterprises and nations can now create custom “supermodels” for their domain-specific industry use cases using Llama 3.1 and NVIDIA software, computing and expertise. Enterprises can train these supermodels with proprietary data as well as synthetic data generated from Llama 3.1 405B and the NVIDIA Nemotron™ Reward model.

NVIDIA AI Foundry is powered by the NVIDIA DGX™ Cloud AI platform, which is co-engineered with the world’s leading public clouds, to give enterprises significant compute resources that easily scale as AI demands change.

The new offerings come at a time when enterprises, as well as nations developing sovereign AI strategies, want to build custom large language models with domain-specific knowledge for generative AI applications that reflect their unique business or culture.

“Meta’s openly available Llama 3.1 models mark a pivotal moment for the adoption of generative AI within the world’s enterprises,” said Jensen Huang, founder and CEO of NVIDIA. “Llama 3.1 opens the floodgates for every enterprise and industry to build state-of-the-art generative AI applications. NVIDIA AI Foundry has integrated Llama 3.1 throughout and is ready to help enterprises build and deploy custom Llama supermodels.”

“The new Llama 3.1 models are a super-important step for open source AI,” said Mark Zuckerberg, founder and CEO of Meta. “With NVIDIA AI Foundry, companies can easily create and customize the state-of-the-art AI services people want and deploy them with NVIDIA NIM. I’m excited to get this in people’s hands.”

To supercharge enterprise deployments of Llama 3.1 models for production AI, NVIDIA NIM inference microservices for Llama 3.1 models are now available for download from ai.nvidia.com. NIM microservices are the fastest way to deploy Llama 3.1 models in production and power up to 2.5x higher throughput than running inference without NIM.

Enterprises can pair Llama 3.1 NIM microservices with new NVIDIA NeMo Retriever NIM microservices to create state-of-the-art retrieval pipelines for AI copilots, assistants and digital human avatars.

Global professional services firm Accenture is first to adopt NVIDIA AI Foundry to build custom Llama 3.1 models using the Accenture AI Refinery™ framework, both for its own use as well as for clients seeking to deploy generative AI applications that reflect their culture, languages and industries.

“The world’s leading enterprises see how generative AI is transforming every industry and are eager to deploy applications powered by custom models,” said Julie Sweet, chair and CEO of Accenture. “Accenture has been working with NVIDIA NIM inference microservices for our internal AI applications, and now, using NVIDIA AI Foundry, we can help clients quickly create and deploy custom Llama 3.1 models to power transformative AI applications for their own business priorities.”

NVIDIA AI Foundry provides an end-to-end service for quickly building custom supermodels. It combines NVIDIA software, infrastructure and expertise with open community models, technology and support from the NVIDIA AI ecosystem.

With NVIDIA AI Foundry, enterprises can create custom models using Llama 3.1 models and the NVIDIA NeMo platform — including the NVIDIA Nemotron-4 340B Reward model, ranked first on the Hugging Face RewardBench.

Once custom models are created, enterprises can create NVIDIA NIM inference microservices to run them in production using their preferred MLOps and AIOps platforms on their preferred cloud platforms and NVIDIA-Certified Systems™ from global server manufacturers.

NVIDIA AI Enterprise experts and global system integrator partners work with AI Foundry customers to accelerate the entire process, from development to deployment.

Enterprises that need additional training data for creating a domain-specific model can use Llama 3.1 405B and Nemotron-4 340B together to generate synthetic data to boost model accuracy when creating custom Llama supermodels.

Customers that have their own training data can customize Llama 3.1 models with NVIDIA NeMo for domain-adaptive pretraining, or DAPT, to further increase model accuracy.

NVIDIA and Meta have also teamed to provide a distillation recipe for Llama 3.1 that developers can use to build smaller custom Llama 3.1 models for generative AI applications. This enables enterprises to run Llama-powered AI applications on a broader range of accelerated infrastructure, such as AI workstations and laptops.

Companies across healthcare, energy, financial services, retail, transportation and telecommunications are already working with NVIDIA NIM microservices for Llama. Among the first to access the new NIM microservices for Llama 3.1 are Aramco, AT&T and Uber.

Trained on over 16,000 NVIDIA H100 Tensor Core GPUs and optimized for NVIDIA accelerated computing and software — in the data center, in the cloud and locally on workstations with NVIDIA RTX™ GPUs or PCs with GeForce RTX GPUs — the Llama 3.1 collection of multilingual LLMs is a collection of generative AI models in 8B-, 70B- and 405B-parameter sizes.

Using new NVIDIA NeMo Retriever NIM inference microservices for retrieval-augmented generation (RAG), organizations can enhance response accuracy when deploying customized Llama supermodels and Llama NIM microservices in production.

Combined with NVIDIA NIM inference microservices for Llama 3.1 405B, NeMo Retriever NIM microservices deliver the highest open and commercial text Q&A retrieval accuracy for RAG pipelines.

Hundreds of NVIDIA NIM partners providing enterprise, data and infrastructure platforms can now integrate the new microservices in their AI solutions to supercharge generative AI for the NVIDIA community of more than 5 million developers and 19,000 startups.

Production support for Llama 3.1 NIM and NeMo Retriever NIM microservices is available through NVIDIA AI Enterprise. Members of the NVIDIA Developer Program will soon be able to access NIM microservices for free for research, development and testing on their preferred infrastructure.

Source: NVIDIA media announcement