Chapter 20 of 75

Building and Leading AI Teams

AI teams fail for organizational reasons more often than technical ones. The roles, the structures, and the leadership behaviors that produce AI teams that ship.

4 min read

Part II — Strategy and Leadership

Building and Leading AI Teams

The skills required to build AI systems at enterprise scale are genuinely scarce, genuinely interdisciplinary, and genuinely different from the skills required to build traditional software. An organization that tries to staff an AI program with software engineers who have read a few LLM tutorials, product managers who attended a two-day AI course, and data analysts who have Excel skills will fail — not because those people are not talented, but because the work requires specific capabilities that take time to develop.

20.1 The Core AI Team Roles

ML Engineer / AI Engineer: Builds, trains, evaluates, and deploys AI models. Writes the code that turns data into predictions. Understands model architectures, training dynamics, evaluation methodology, and deployment infrastructure. This is not the same role as a software engineer, and it is not the same role as a data scientist — it combines both with production engineering discipline.

Data Engineer: Builds and maintains the pipelines that deliver clean, reliable data to the AI system. Responsible for the data infrastructure that determines whether the AI has what it needs to train and operate. Without excellent data engineering, excellent models cannot be built.

AI Product Manager: Owns the product definition for AI systems — the problem the AI solves, the users it serves, the metrics that define success, and the roadmap. Must understand AI capabilities and limitations well enough to write meaningful specifications, work with ML engineers to define accuracy requirements, and evaluate AI outputs against user needs. Different from traditional product management in the depth of technical understanding required.

Domain Expert / Subject Matter Expert: The person who knows the domain the AI is being applied to — the compliance analyst, the underwriter, the clinical specialist. This person defines what correct looks like, provides labeled training data, validates model outputs, and owns the human review process. Domain expertise is often the scarcest resource in enterprise AI programs.

ML Ops Engineer: Owns the infrastructure that runs AI systems in production — model serving, monitoring, deployment pipelines, feature stores. As AI programs scale from one model to many, ML Ops becomes the critical function that determines whether the program can operate reliably.

20.2 Organizational Structures

Two organizational structures work for enterprise AI programs, depending on the organization's scale and maturity:

Centralized AI team (hub model): A dedicated AI team serves the entire organization. Domain experts are embedded from business units for specific projects. This model works when AI is new to the organization, when the talent pool is small, and when the AI program is building shared infrastructure. Risk: the AI team becomes a bottleneck, business units feel unsupported, and the team becomes disconnected from operational realities.

Federated AI (hub-and-spoke model): A central AI platform team builds and maintains the infrastructure. Embedded AI teams in each major business unit build domain-specific solutions on the platform. Domain experts are permanent members of the business unit teams. This model works when the AI program has matured, when business units have sufficient AI work to sustain dedicated teams, and when the platform is stable enough for business unit teams to build on reliably.

20.3 Sourcing AI Talent

AI talent is scarce. The organizations that source it effectively use a combination of strategies:

Hire for learning velocity, not current skills. AI is evolving faster than any hiring process can track. The candidates who will be most valuable in three years are the ones who learn quickly and deeply, not the ones who have the most current skills today.

Upskill adjacent roles. Software engineers, data analysts, and domain experts who are motivated to develop AI skills can be upskilled into AI roles faster and more reliably than external hiring in a hot market. This also produces AI practitioners who already understand the organization's systems and domain.

Partner with academia and bootcamps. Universities and AI training programs are producing AI talent that lacks enterprise experience but has strong technical foundations. Structured internship and rotation programs that provide enterprise context alongside technical work are effective sourcing channels.

Build the domain expert pipeline. Domain experts are the rarest AI resource and the one most organizations ignore. Clinical specialists, compliance analysts, and underwriters who understand how to work with AI systems are a competitive advantage. Making domain experts AI-capable is higher-leverage than any external hiring strategy.

20.4 Leading AI Teams

AI teams operate under conditions that require specific leadership approaches:

Protect time for exploration. AI development requires experimentation, and experimentation requires time that is not committed to delivery. Leaders who fill every sprint with committed features will produce AI teams that ship slowly and learn nothing.

Celebrate learning from failure. Models that do not work as expected, hypotheses that do not hold, approaches that are abandoned — these are the normal outputs of AI development. Teams that are punished for honest reporting of negative results learn to hide them.

Bridge the domain gap actively. The gap between ML engineers who understand models and domain experts who understand the problem is the most common cause of AI that performs well in evaluation and fails in production. Leaders who actively bridge this gap — structuring collaboration, requiring joint problem definition, involving domain experts in output evaluation — produce AI that works.

Measure team health alongside model health. AI teams that are burning out produce worse models. Sustainable pace, manageable scope, and clear priorities are not soft management concerns — they are determinants of AI output quality.