Part 3 — Enterprise Application

Hybrid Architectures — LLMs and LCMs Together

The LCM adoption question is not binary. The correct question is never "should we replace our LLMs with LCMs?" It is: "for which components of which workflows does the concept-level architecture produce better results than the token-level architecture, and how do we connect the two?"

This chapter answers that question with three patterns. Each pattern has a specific use case profile, an integration contract, a failure mode analysis, and a decision tree for choosing between them. Together, they constitute the hybrid architecture vocabulary that enterprise AI teams need to design systems that use the right tool for each component.

What You Will Learn

• Design three hybrid LLM-LCM patterns: concept router, concept elevator, and concept pipeline
• Define the integration contract at each LLM-LCM boundary
• Identify the failure modes specific to each hybrid pattern and their mitigations
• Apply a decision tree to select the appropriate pattern for a given use case
• Estimate the operational overhead of hybrid architectures relative to single-architecture systems

12.1 The Three Hybrid Patterns

The three patterns differ in where the LLM-LCM boundary sits and what crosses the boundary.

Pattern 1: The Concept Router. An LLM classifier analyzes incoming requests and routes them to either an LLM execution layer or an LCM execution layer, based on the Task Unit Test from Chapter 5. The router makes a binary or multi-class routing decision; the execution layer handles the actual task. Use this pattern when you have a mixed-task application — a system that handles both token-level and concept-level requests — and want a single entry point.

Pattern 2: The Concept Elevator. An LLM handles user-facing interaction and short-form output generation. An LCM handles the reasoning core for concept-level tasks. The LLM "elevates" the user's query into concept space (by formatting it for the LCM's encoder), and "lowers" the LCM's concept-level output back into natural language (by polishing and contextualizing the decoded output). Use this pattern when you have a conversational application that occasionally needs concept-level reasoning.

Pattern 3: The Concept Pipeline. LCM output feeds directly into an LLM as context. The LCM performs the concept-level reasoning (multi-document synthesis, cross-lingual comparison, contradiction detection), and its decoded output — a structured synthesis in natural language — is passed to an LLM that formats, contextualizes, or responds to user questions about it. Use this pattern when you need concept-level reasoning as a preprocessing step for a downstream LLM application.

12.2 Pattern 1: The Concept Router

Figure 12.1 — The Concept Router. An LLM classifier applies the Task Unit Test to incoming requests and routes to either the LLM or LCM execution path. Cost is paid for LCM only when the task genuinely warrants it.

When to use it. A knowledge management application serves two types of user requests: short-document Q&A (LLM-appropriate) and cross-document synthesis (LCM-appropriate). A single entry point must route both.

Architecture:

User Request
    ↓
[LLM Router] ← Task Unit Test classification
    ↓               ↓
[LLM Path]     [LCM Path]
    ↓               ↓
[LLM Response]  [LCM Response]
    ↓               ↓
[Response to User]

Integration contract. The router LLM classifies the request using a structured output schema. The classification includes: the identified task type (token-level or concept-level), the confidence score, and the parameters needed by the selected execution layer (which documents to include, what the synthesis goal is, which languages are involved).

from pydantic import BaseModel
from typing import Literal

class RoutingDecision(BaseModel):
    task_type: Literal["token_level", "concept_level"]
    confidence: float
    execution_params: dict
    routing_reason: str

ROUTER_PROMPT = """
Classify this request as requiring token-level or concept-level processing.

Token-level: short document Q&A, code generation, conversational, single-document tasks.
Concept-level: cross-document comparison, cross-lingual synthesis, long-form generation,
hierarchical planning, contradiction detection.

Return a RoutingDecision JSON object.

Request: {user_request}
"""

def route_request(user_request: str, llm) -> RoutingDecision:
    response = llm.invoke(
        ROUTER_PROMPT.format(user_request=user_request),
        output_schema=RoutingDecision
    )
    return response

Failure modes.

• Misclassification: The router incorrectly sends a concept-level request to the LLM path. The LLM produces a plausible but globally incoherent response that the user accepts without realizing the concept-level task was not handled correctly. Mitigation: include a confidence threshold below which both paths are run and results compared; route low-confidence decisions to the LCM path by default (concept-level over-routing is less harmful than concept-level under-routing).
• Latency asymmetry: The LCM path is significantly slower than the LLM path. Users who receive LCM responses wait longer. Mitigation: stream LCM responses where possible; set user expectations with progress indicators for concept-level tasks.

Decision tree entry point. Use the concept router when: your application handles mixed task types, you have a single conversational entry point, and users should not need to know whether their request is being handled by an LLM or an LCM.

12.3 Pattern 2: The Concept Elevator

Figure 12.2 — The Concept Elevator. An LLM manages the conversation at the token layer; an LCM handles heavy reasoning in concept space. The elevator and descender components bridge the two layers. Users experience a chat interface backed by concept-level reasoning.

When to use it. A customer-facing research assistant has a conversational interface but occasionally needs to synthesize across large document corpora in response to complex user queries. The base user experience is conversational (LLM-appropriate), but specific query types require concept-level reasoning.

Architecture:

User Message (conversational)
    ↓
[LLM — Conversation Manager]
    ↓ (detects concept-level need)
[LLM — Query Elevator: formats query for LCM encoder]
    ↓
[LCM — Concept Reasoning: encodes, reasons, decodes]
    ↓
[LLM — Response Polisher: contextualizes LCM output for conversation]
    ↓
[Response to User (conversational format)]

The elevator and polisher roles. The "elevator" LLM takes the user's natural language query and transforms it into a form optimized for the LCM encoder: explicit task instructions, document scope specification, output format requirements. The "polisher" LLM takes the LCM's decoded output — which is semantically correct but may be terse, structured, or not conversationally framed — and reformats it for the conversation context.

ELEVATOR_PROMPT = """
Transform this user request into a structured query for the document synthesis system.
The synthesis system needs: a clear synthesis goal, the document scope, and the output format.

User request: {user_request}
Available documents: {document_list}

Return a structured synthesis task specification.
"""

POLISHER_PROMPT = """
You are a research assistant. The user asked: "{user_request}"

Our document synthesis system produced this analysis:
{lcm_output}

Rephrase this analysis as a conversational response that:
1. Directly answers the user's question
2. Cites the most important findings
3. Flags any uncertainties or limitations
4. Invites follow-up questions
"""

def concept_elevator_pipeline(user_request: str, documents, llm, lcm):
    # Elevate query
    elevated_query = llm.invoke(
        ELEVATOR_PROMPT.format(
            user_request=user_request,
            document_list=[d.title for d in documents]
        )
    )

    # LCM reasoning
    lcm_output = lcm.synthesize(elevated_query, documents)

    # Polish response
    conversational_response = llm.invoke(
        POLISHER_PROMPT.format(
            user_request=user_request,
            lcm_output=lcm_output
        )
    )

    return conversational_response

Integration contract. The LLM-LCM boundary is the synthesis task specification (elevator output) and the synthesized analysis (LCM output). Both should be structured: the elevator output specifies what the LCM must produce; the LCM output is a decoded natural language synthesis that the polisher can work with.

Failure modes.

• Elevator distortion: The elevator LLM transforms the user's query in a way that changes its meaning before it reaches the LCM. The LCM answers a different question than the user asked. Mitigation: include the original user request alongside the elevated query in the LCM's context; validate elevator output against the original intent before passing to LCM.
• Polisher hallucination: The polisher LLM adds content that was not in the LCM's output, in service of making the response more conversational. Mitigation: instruct the polisher explicitly not to add information not present in the LCM output; include a citation validation step that confirms every claim in the polished response traces to the LCM output.

Decision tree entry point. Use the concept elevator when: your primary application is conversational, concept-level reasoning is a minority of requests, and user experience continuity across LLM and LCM tasks matters.

12.4 Pattern 3: The Concept Pipeline

Figure 12.3 — The Concept Pipeline. LCM batch-processes the document corpus offline, producing a structured analysis report. The LLM serves fast, interactive Q&A against the pre-reasoned output. LCM cost is amortized across all subsequent queries.

When to use it. A regulatory compliance platform uses the LCM to perform cross-jurisdiction contradiction detection (Chapter 9) and feeds the structured results to an LLM that answers analyst questions about specific contradictions. The LCM runs as a batch process; the LLM provides the interactive Q&A layer.

Architecture:

[Document Corpus] → [LCM — Batch Analysis] → [Structured Analysis Report]
                                                        ↓
                                               [LLM — Interactive Q&A]
                                                        ↓
                                               [Analyst Questions / Answers]

Integration contract. The LCM produces a structured analysis report — a document that is the LCM's decoded output, organized by finding type (equivalences, contradictions, gaps) with source document citations. This report becomes the LLM's knowledge base for the interactive Q&A session. The LLM does not access the raw document corpus — only the LCM's structured analysis.

Failure modes.

• LLM confabulation on LCM output: The analyst asks about a contradiction that the LCM identified, and the LLM elaborates on it with information not in the LCM's analysis. Mitigation: instruct the LLM explicitly to answer only from the provided analysis; include a "source" citation requirement for every claim.
• Stale LCM analysis: The documents change after the LCM batch run; the LLM answers questions based on outdated LCM output. Mitigation: timestamp the LCM analysis; warn users when the analysis is older than a configurable threshold; trigger re-analysis when documents change.

Decision tree entry point. Use the concept pipeline when: the concept-level reasoning is a batch preprocessing step, the LCM's output is a structured artifact that humans and downstream systems consume, and interactive access to the LCM's findings is needed.

12.5 Pattern Selection Decision Tree

Does the application serve mixed (token-level + concept-level) requests
from a single entry point?
    YES → Is the entry point conversational?
              YES → Concept Elevator
              NO  → Concept Router
    NO  → Is concept-level reasoning a batch preprocessing step?
              YES → Concept Pipeline
              NO  → Is concept-level reasoning the primary function?
                        YES → Pure LCM (no LLM wrapper needed)
                        NO  → Reconsider whether LCMs are needed

12.6 Operational Overhead of Hybrid Architectures

Hybrid architectures are more complex to build, monitor, and debug than single-architecture systems. Before committing to a hybrid design, account for:

Development overhead. Each pattern requires building and maintaining two inference pipelines (LLM and LCM), the boundary contracts between them, and the orchestration logic that moves data across the boundary. This is approximately 2-3x the development effort of a single-architecture system.

Observability complexity. A token-level trace from an LLM captures the full reasoning chain in readable text. A hybrid system trace must capture both the LLM's token-level reasoning and the LCM's concept-level operations (encoding, similarity scores, generation). These require different instrumentation and cannot be combined in a single trace format.

Error attribution. When a hybrid system produces a wrong answer, determining whether the error originated in the LLM component or the LCM component requires replaying the pipeline with intermediate outputs captured at each stage.

For early LCM adopters in 2026, hybrid architectures are the realistic production architecture. Pure LCM systems are appropriate for batch workflows where the full task is concept-level. For user-facing applications, the LLM handles everything the LCM cannot do well (conversational wrapping, local fluency, short-form Q&A), and the LCM handles everything the LLM cannot do well (concept-level reasoning, cross-lingual synthesis, global coherence).

Summary

Three hybrid patterns address the common use cases where LLMs and LCMs must work together: the concept router (mixed task types, single entry point), the concept elevator (conversational primary with concept-level reasoning secondary), and the concept pipeline (batch concept-level preprocessing with LLM interactive access). Each has a defined integration contract, characteristic failure modes, and a mitigation strategy.

• The router classifies; the elevator wraps; the pipeline sequences. Three distinct integration patterns for three distinct use case profiles.
• Integration contracts must be explicit. The boundary between LLM and LCM components is where most hybrid system failures originate. Define inputs, outputs, and validation rules for each boundary.
• Hybrid architectures add operational overhead. Account for 2-3x development effort and more complex observability requirements before committing to a hybrid design.
• The decision tree is the first step. Apply it before designing. Many applications that appear to need hybrid architectures are better served by a pure LLM or pure LCM approach.

Exercises