AT&T Scales AI with Multi-Agent Stack, Sees 90% Cost Savings
As daily token usage soared to 8 billion, AT&T re-architected its AI orchestration layer, adopting a multi-agent system built on LangChain to dramatically improve efficiency and reduce costs.
The Challenge of Scale
AT&T faced a significant challenge with the increasing demands of its AI applications. Chief Data Officer Andy Markus and his team determined that processing everything through large reasoning models was becoming unsustainable, both financially and in terms of performance. VentureBeat reported on this shift in strategy on February 26, 2026.
A Multi-Agent Approach
The solution involved reconstructing the orchestration layer to create a multi-agent stack leveraging LangChain. This architecture utilizes large language model “super agents” to direct smaller, specialized “worker” agents, each focused on concise, purpose-driven tasks. This approach has led to substantial improvements in latency, speed, and response times.
Significant Cost Savings
The new orchestration layer has delivered impressive results, with AT&T realizing up to 90% cost savings. Markus believes the future of agentic AI lies in utilizing numerous small language models (SLMs), finding them to be as accurate, if not more so, than larger models within specific domains.
Question AT&T Workflows: Empowering Employees
AT&T recently deployed Ask AT&T Workflows, a graphical drag-and-drop agent builder, powered by this re-architected stack and Microsoft Azure. This tool allows employees to automate tasks using agents that access proprietary AT&T tools for document processing, natural language-to-SQL conversion, and image analysis. The focus is on leveraging AT&T’s own data to drive decision-making.
Human Oversight and Security
Despite the autonomous nature of the agents, human oversight remains a critical component. All agent actions are logged, data is isolated, and role-based access controls are enforced to ensure security and maintain a check and balance throughout the process.
Iterative Development and Model Selection
AT&T avoids a “build everything from scratch” mentality, instead prioritizing models that are “interchangeable and selectable.” The company plans to deprecate homegrown tools as industry standards mature, allowing them to quickly pilot, plug in, and plug out different components. VentureBeat highlights this agile approach to development.
AI-Fueled Coding
This principle of breaking down complex tasks extends to AT&T’s software development process. Markus describes “AI-fueled coding,” where developers use agile methods and function-specific build archetypes to generate production-ready code with minimal iteration. This technique is as well being used by non-technical teams to rapidly prototype software.
Employee Adoption and Productivity Gains
Ask AT&T Workflows has been rolled out to over 100,000 employees, with more than half using it daily. Active users report productivity gains as high as 90%. Interestingly, even technically proficient users are gravitating towards the no-code, drag-and-drop interface. Andy Markus’s LinkedIn post from November 13, 2025, details the positive reception of the platform.
Key Takeaways
- AT&T successfully scaled its AI initiatives by adopting a multi-agent system.
- The new architecture resulted in up to 90% cost savings.
- Employee empowerment through tools like Ask AT&T Workflows is driving productivity gains.
- Human oversight remains crucial for ensuring security and responsible AI deployment.
- A flexible, iterative approach to model selection and development is key to staying ahead in the rapidly evolving AI landscape.