Framework Quick Reference
The entire LegacyForward.ai framework — stages, gates, anti-patterns, key questions, and how the pillars connect. A practitioner’s quick reference.
Preparing PDF…Where to Start
“We have AI ideas but don’t know which are worth it.”
1Signal Capture →
“Our AI projects stall or deliver inconsistent results.”
2Grounded Delivery →
“We can’t replace legacy systems but need AI to work with them.”
3Legacy Coexistence →
Signal Capture
Identify where AI creates value impossible by other means
Value Hypothesis
Articulate where AI creates net new value before any technical work
GATE: Clear, measurable, worth pursuing; deterministic alternatives ruled out
Value Validation
Validate across data, feasibility, organizational, and economic dimensions
GATE: All four pass — data, feasibility, adoption, economics
Value Tracking
Measure value continuously in production with leading/lagging indicators
GATE: Thresholds trigger review or kill if progress stalls
◆ Key Questions
- › Where does this create net new value we cannot achieve any other way?
- › Remove the AI and use unlimited human effort — could we achieve the same result?
- › What is success in operational terms — revenue, cost, risk, time-to-insight — with specific targets?
- › Does the required data actually exist, and is it accessible?
- › Will the organization trust and actually use the output?
- › Does the value justify the full cost — development, integration, governance, monitoring, retraining?
⚠ Anti-Patterns
- The Adoption Trap — Measuring success by deployment volume (users, queries/day) instead of value delivered
- Solutions Looking for Problems — Starting with “we need an AI strategy” instead of “we have a problem only AI can solve”
- Automation as Transformation — Making a broken process faster is not transformation. Test: remove the AI, could humans achieve the same?
- The Vibe-Coded Commitment — AI-assisted dev builds compelling demo in days; leadership commits before validating the value hypothesis
- The Perpetual Pilot — Initiatives in “pilot” indefinitely, avoiding accountability. Every pilot needs a kill date.
- The Sunk Cost Spiral — Continuing to fund initiatives that have consumed resources but failed to demonstrate value
Automation vs. Transformation
| Automation | Transformation | |
|---|---|---|
| Definition | AI performs a task previously done by humans, faster or cheaper | AI produces an outcome previously impossible at any cost |
| The Test | Remove the AI. Could enough humans achieve the same result? | Remove the AI. The outcome ceases to exist entirely. |
| Example | AI reads invoices, extracts fields into a spreadsheet | AI analyzes 15 years of invoices to identify systematic 4% overcharging |
| Value Ceiling | Bounded by cost of labor it replaces | Unbounded — net new value that didn’t exist before |
Grounded Delivery
Deliver AI through phases designed for non-deterministic systems
Frame
Value hypothesis, boundaries, probabilistic success criteria, governance model
GO / NO-GO
Explore
Structured experiments, parallel approaches, build evaluation dataset
GO / PIVOT / KILL
Shape
Production architecture, integration contracts, fallback paths, ops model
GO / REVISIT
Harden
Production code, eval suite, adversarial testing, human evaluation
GO / ITERATE
Operate
Deploy, monitor drift, collect feedback, retrain/re-prompt continuously
ONGOING
◆ Key Questions by Phase
- Frame: Does the problem genuinely require non-deterministic components? What does “good enough” look like in probabilistic terms?
- Explore: What explicit hypothesis are we testing? How will we evaluate? What is the kill criterion?
- Shape: Where is the boundary between deterministic and non-deterministic components? What are our fallback paths?
- Harden: Does production eval meet probabilistic criteria with statistical confidence? What are the adversarial vectors?
- Operate: How do we detect model drift, data drift, concept drift? Is the eval dataset a living artifact?
⚠ Anti-Patterns
- Frame Skipped — Team wants to “start building” without defining probabilistic success criteria
- Velocity as Progress — Code production throughput is not the bottleneck; evaluation throughput is
- Test Generation Illusion — AI-generated tests that assert on specific strings provide false confidence for non-deterministic systems
- Sunk Cost at Gates — Evidence shows approach is marginal, but team pushes forward because of time already invested
- Ship and Forget — Deploying without budget for ongoing evaluation, monitoring, and retraining
Agile vs. Grounded Delivery
| Construct | Agile (Deterministic) | Grounded Delivery (Non-Deterministic) |
|---|---|---|
| Work Unit | User Story | Value Hypothesis |
| Success Criteria | Binary pass/fail acceptance criteria | Probabilistic thresholds (e.g., 92% acceptable ±3%) |
| Planning | Estimate in story points, commit to scope | Time-box investigation, go/no-go on evidence |
| Testing | Regression (what passed yesterday passes today) | Continuous evaluation (quality distribution shifts) |
| Done | Feature meets spec | Quality exceeds threshold with statistical confidence |
| Experimentation | Spikes — second-class, grudgingly tolerated | Full phase (Explore) with artifacts and funding |
| Post-Deploy | Ship and stabilize | Permanent investment in evaluation and retraining |
Legacy Coexistence
Make AI work alongside the systems that run the enterprise
Data Exhaust
Legacy produces data AI can analyze without real-time access
Batch latency; decades of unanalyzed data
Sidecar
AI augments legacy near-real-time without modifying it
Observes events; supplementary outputs; legacy is SoR
Gateway
Controlled interface translating modern & legacy protocols
Encapsulates legacy complexity; deep interface knowledge
Shadow Pipeline
AI replaces legacy gradually with validated parallel runs
Both run; outputs compared; confidence before cutover
Legacy-Aware Agent
Autonomous agents across modern + legacy systems
Explicit legacy constraints; first-class in planning
◆ Key Questions
- › What interface types does the legacy system expose? (API, batch, terminal, file)
- › What are the extraction constraints? Real-time API vs. nightly batch vs. quarterly exports?
- › How stale can the data be before the value hypothesis becomes infeasible?
- › When the AI and legacy system disagree, who wins per use case?
- › What are the known data quality issues? Field inconsistencies? Missing data?
- › How does this legacy system fail? What is the error recovery model?
⚠ Anti-Patterns
- The Greenfield Fantasy — “Once we modernize, we can deploy AI properly” is a strategy for never deploying AI
- The Wrapper Illusion — API wrappers hide legacy complexity but don’t eliminate batch limits, format constraints, or failure modes
- Integration Afterthought — “We’ll figure out legacy integration later” — integration determines feasibility
- Screen Scraping Default — Works for demos; breaks in production. Last resort, not a pattern.
- The Strangler Fig Misconception — Valid for deterministic modernization, dangerous for AI. AI creates net new capabilities, not function-for-function replacement.
Rip-and-Replace vs. Legacy Coexistence
| Aspect | Rip-and-Replace | Legacy Coexistence |
|---|---|---|
| Premise | Replace legacy, then deploy AI | Deploy AI alongside legacy, deliberately designed for hybrid |
| Timeline | Years of multi-system migration | Weeks to deploy AI, no modernization prerequisite |
| Risk | Existential — entire business depends on switchover | Bounded — AI augments, legacy remains primary |
| Data Access | Assumes modern APIs and clean data | Works with data exhaust, batch extracts, file formats |
| Economics | Hundreds of millions, long ROI | Proportional to AI value capture |
How the Pillars Connect
The framework is a cycle, not a sequence
Signal Capture → Grounded Delivery
- Value Hypothesis becomes the primary input to the Frame phase
- Value Tracking feeds the probabilistic quality gates — quality is measured by progress toward value, not feature completion
Signal Capture → Legacy Coexistence
- Highest-value AI opportunities often exist because legacy systems contain decades of unanalyzed data
- Data validation must account for legacy constraints — formats, access patterns, extraction limitations
- Coexistence patterns determine whether a value hypothesis is technically feasible
Grounded Delivery → Legacy Coexistence
- Explore phase must include legacy integration discovery — don’t defer to Harden
- Shadow Pipeline maps to the Harden phase — quality gates evaluate AI against legacy baselines
- Dual-track governance: deterministic components use conventional rigor, non-deterministic use probabilistic evaluation
The feedback loop: Signal Capture identifies what to build. Grounded Delivery defines how. Legacy Coexistence ensures it works where it has to. The Operate phase feeds back into Signal Capture — production data reveals whether the value hypothesis was correct, informing the next round.
Red Flags — When Something Is Going Wrong
Signal Capture
- • Cannot articulate value hypothesis in one sentence
- • Initiative labeled “transformation” but is really automation
- • No measurable outcome; success criteria are subjective
- • 3-6 months in production, leading indicators haven’t materialized
- • 80%+ of portfolio is automation; nothing converting from experimental
Grounded Delivery
- • Frame phase skipped — team wants to “start building”
- • Evaluation dataset doesn’t exist, is tiny, or unrepresentative
- • Team completing stories but can’t articulate progress toward value
- • Quality defined as “it works” instead of probabilistic thresholds
- • Deployed without production evaluation suite running
Legacy Coexistence
- • “We’ll modernize first, then deploy AI”
- • Legacy integration discovered late in Harden phase
- • Integration discussed in abstract — no one tested actual legacy behavior
- • Trust boundaries between AI and legacy outputs left ambiguous
- • No contingency plan if legacy system becomes unavailable
Core Principles
Kill early. Every gate is a chance to stop spending on what won’t work.
Non-deterministic by default. AI outputs are distributions, not binaries. Design for it.
Legacy is a feature. Decades of data and process logic are assets, not obstacles.
Value before technology. If you can’t articulate the value, you can’t build the system.
Operate forever. Non-deterministic systems need permanent monitoring and investment.
Coexist deliberately. Not rip-and-replace. Not wrappers. Intentional integration patterns.