Red Hat Expands Enterprise AI Portfolio with New Inference Server, Validated Models, and llama Stack Integration
Table of Contents
Red Hat announced significant updates to it’s enterprise AI portfolio at Red Hat Summit 2024, including the Red Hat AI Inference Server, a program for validating third-party AI models, and integration of the Llama Stack with the model Context Protocol (MCP) API. Thes advancements aim to provide businesses with more options for deploying generative AI solutions in hybrid cloud environments.
Red Hat AI Inference Server for Scalable and Consistent AI
The red Hat AI Inference Server is designed to deliver faster, more consistent, and cost-effective AI inference at scale across hybrid cloud environments. It is indeed now integrated into the latest releases of Red Hat OpenShift AI and Red Hat Enterprise Linux AI (RHEL AI). The server is also available as a standalone solution, offering increased deployment flexibility for bright applications.
Third-Party Validation Models Enhance Trust and Performance
Red Hat is offering a collection of validated AI models through Hugging Face. This program helps organizations easily identify models suitable for their specific needs. Red Hat AI provides validated models and deployment guides to increase confidence in model performance and reproducibility. Some models are optimized using model compression technology to reduce size, increase inference speed, and minimize resource consumption and operating costs.
Llama Stack and MCP Simplify Generative AI Agent Progress
Red Hat AI integrates Meta’s Llama Stack with Antropic’s Model Context Protocol (MCP).This provides a standardized API for building and deploying AI applications and agents. Llama Stack, currently in developer preview on Red Hat AI, offers a unified API to access vLLM inference, Retrieval-Augmented Generation (RAG), model evaluation, guardrails, and agent functions across various generative AI models. MCP facilitates integration with external tools in agent workflows by providing standard interfaces for connecting apis, plugins, and data sources.
Industry Perspective on Open Source AI
According to Forrester, open source software is crucial for accelerating enterprise AI initiatives. Michelle Rosen, a research manager at IDC, stated, “enterprises are moving beyond the initial AI exploration phase and focusing on practical deployments. The key to their continued success will depend on their ability to flexibly adapt their AI strategies to different environments and needs. The future of AI requires not just powerful models, but models that can be deployed proactively and cost-effectively. This flexibility is essential for companies looking to scale their AI initiatives and realize business value.”
Key Takeaways
- Scalable Inference: Red Hat AI Inference Server delivers faster,more consistent,and cost-effective AI inference.
- Trusted Models: Validated models through Hugging Face provide confidence in performance and reproducibility.
- simplified Development: Llama Stack and MCP integration streamlines the creation of AI agents and applications.
- Hybrid Cloud Flexibility: The portfolio supports deployment across hybrid cloud environments.