Part 2 — The Comparison Layer

LLMs vs. LCMs — An Architectural Comparison

Enterprise architects compare technology options by mapping them against the same dimensions: input requirements, processing model, output characteristics, and operational profile. Applying that discipline to LLMs and LCMs produces a precise picture of where they are the same, where they differ, and why the differences matter for specific task types.

This chapter puts the two architectures side by side with the rigor of an infrastructure comparison. It does not produce a winner — that is Chapters 6 and 7's job. It produces the framework that makes the comparison honest.

Figure 5.1 — Side-by-side architectural comparison. Same transformer foundation, different representational unit. LLM weaknesses (red) map directly to LCM strengths (green) for concept-level tasks.

5.1 Input Representation

LLMs: token sequences. Input text is converted to tokens by a tokenizer, then to token IDs, then to token embeddings by an embedding lookup table. The model receives a sequence of token embeddings. A 500-word document becomes approximately 650–700 token embeddings. The embedding for each token is a learned vector that encodes the statistical properties of that token in the training corpus — primarily co-occurrence patterns with other tokens.

Token embeddings are context-free: the embedding for "bank" is the same whether the sentence is about a financial institution or a river bank (though attention mechanisms create contextual representations during processing). Token embeddings are language-specific: the embedding for "bank" in English and "banque" in French are different vectors in different parts of the embedding space, even though they mean the same thing.

LCMs: sentence-level concept embeddings. Input text is segmented into sentences, each sentence is encoded by SONAR into a 1,024-dimensional concept embedding. A 500-word document (approximately 25–35 sentences) becomes 25–35 concept embeddings. Each embedding encodes the meaning of its sentence in a shared, cross-lingual semantic space. The embedding for "The committee approved the proposal" is close to the embedding for "Le comité a approuvé la proposition" and to "The board voted to accept the recommendation."

The compression ratio is significant: 650 token embeddings become 30 concept embeddings for the same 500-word document. But this compression is not lossless — intra-sentence structure is lost. The trade is token-level detail for semantic-level compression.

5.2 Internal Computation

Both LLMs and LCMs use transformer architectures, but they attend to different things.

LLMs: token-to-token attention. The transformer attention mechanism computes a weighted sum of token embeddings for each token, where weights reflect the relevance of each token to the current token. For autoregressive generation, each new token can attend to all previous tokens in the context window. Attention patterns are dense and position-influenced: nearby tokens receive higher weights from each other than distant tokens, all else being equal.

The context window defines the attention horizon: tokens beyond the window boundary receive zero weight. Within the window, attention degrades for content in the middle of long sequences.

LCMs: concept-to-concept attention. The concept model's attention mechanism computes weighted sums of concept embeddings for each concept embedding position in the output sequence. Attention weights reflect semantic relevance — proximity in concept space — rather than positional proximity. A concept embedding representing a relevant constraint from early in a long document can receive high attention weight from a concept embedding representing a later plan step, regardless of their sequential distance.

The concept model has no vocabulary constraint on its internal representations — it reasons over a continuous vector space rather than a discrete token vocabulary. Its internal states are 1,024-dimensional vectors that can express arbitrary semantic content without being constrained to token-plausible representations.

5.3 Output Generation

LLMs: token-by-token autoregressive generation. The model generates output one token at a time, each token conditioned on the prompt and all previously generated tokens. Generation is left-to-right, and each token must be plausible given the token sequence that preceded it. This produces locally fluent text — token-by-token plausibility enforces grammaticality and style — but does not guarantee global consistency across long outputs.

LCMs: concept-sequence generation followed by sentence-level decoding. The concept model generates a sequence of concept embeddings (one per output sentence, approximately). Each concept embedding is then decoded independently by the SONAR decoder into a natural language sentence. The decoder attends to the concept embedding and generates the sentence token-by-token, but it is constrained to express the meaning encoded in the concept embedding rather than freely generate plausible tokens.

The output language is a decoder parameter: the same concept embedding sequence can be decoded into English, French, or any other SONAR-supported language. The generation process separates the reasoning step (concept model) from the fluency step (decoder), which is why global coherence and local fluency are semi-independent in LCM outputs.

5.4 Context Handling

LLMs: fixed context window. State-of-the-art LLMs have context windows of 128,000 to 1,000,000 tokens, sufficient for most individual documents. Cross-document reasoning requires loading all documents into the same context window simultaneously, which becomes expensive and attention-degraded for large corpora.

LCMs: concept-level sequence with semantic attention. The concept model's sequence length limit is measured in concept embeddings, not tokens. Because each concept embedding encodes a full sentence, the concept model can attend over sequences representing much longer documents than an equivalent token budget would allow. For very large corpora, similarity-based retrieval in concept space allows the model to retrieve the most semantically relevant concept embeddings rather than loading all embeddings simultaneously.

5.5 Architectural Comparison Table

5.6 The Task Unit Test: Applied

The Task Unit Test from Chapter 1 maps directly onto the architectural dimensions above.

Test 1 — Natural unit: What is the natural unit of the task? If the task requires reasoning about sub-word structure, word choice, or short passages, the LLM's token-level representation is appropriate. If the task requires reasoning about complete propositions, semantic equivalences across vocabulary, or relationships between ideas regardless of surface form, the LCM's concept-level representation is more appropriate.

Test 2 — Semantic equivalence: Does the task need to recognize that two differently-worded statements mean the same thing? LLMs handle this reasonably well within a single document. They handle it poorly across documents and across languages. LCMs handle it via concept-space proximity, which is language-agnostic and vocabulary-independent.

Test 3 — Global consistency: Does the task require maintaining consistency across a long output or across multiple source documents? LLMs fail this test for sufficiently long outputs due to positional attention bias. LCMs maintain consistency through semantic attention.

Test 4 — Cross-lingual: Does the task require reasoning across documents in multiple languages, without translation as a preprocessing step? LLMs require translation or multilingual fine-tuning. LCMs operate natively in a shared concept space.

A task that passes three or four tests is a strong LCM candidate. A task that passes zero or one is a strong LLM candidate. Tasks that pass two are genuinely ambiguous — hybrid architectures (Chapter 12) or LLM-with-retrieval patterns may serve them better than a pure LCM approach.

5.7 Boundary Conditions

Long document Q&A. A user asks a question about a 200-page technical specification. The natural unit is a passage (LLM-appropriate), but the question may require reasoning across multiple sections (LCM-appropriate). Resolution: use LLM RAG if the answer lives in a single section; use LCM concept retrieval if the answer requires synthesizing across sections.

Structured data extraction from documents. Extract a table of obligations from a 100-page contract. The extraction task is passage-level (LLM-appropriate), but consistency across the full contract is concept-level (LCM-appropriate). Resolution: extract with LLM, validate consistency with LCM.

Multilingual classification. Classify 10,000 customer feedback items (in five languages) into ten thematic categories. The classification unit is a sentence (borderline). The thematic categories are concept-level. The cross-lingual requirement pushes toward LCM. Resolution: SONAR encoding + concept-space clustering + LLM for category labeling.

Exercises