Why AI projects stall at the data layer

AI initiatives rarely fail because the model was inadequate. They stall because the material the model needs is scattered, inconsistently permissioned, undocumented, or stale, and nobody discovered that until the build was underway.

'AI-ready' is not a product you buy. It is a set of properties your data either has or does not, and each one maps to a specific failure if absent.

Structure: content the system can actually use

Models read text, but they cannot infer what a document is or when it applies. A folder of PDFs is not a knowledge base; it is a folder.

  • Format matters. Scanned images need OCR before anything can read them.
  • Chunking preserves or destroys meaning. A clause separated from its definition is misleading rather than merely incomplete.
  • Metadata carries the context. Document type, owner, effective date, superseded status.
  • Version clarity. If three versions exist and none is marked current, a system will quote whichever it finds.

Permissions that follow the user

This is the requirement most often discovered late, and the most expensive to retrofit.

If access is enforced only in the application that normally displays a document, then indexing that document for AI has quietly removed the control. The retrieval layer must filter by the requesting user's entitlements at query time, not after results are returned, and not by assuming everyone with system access may see everything.

Worth saying plainly

Filtering after retrieval is not access control. By that point the model has already read material the user was not entitled to see, and it may have influenced the answer.

Freshness and the retirement path

A confidently quoted obsolete policy is worse than no answer at all. An AI-ready source needs change detection, re-indexing on update, and, the step usually forgotten, removal when a document is retired.

Freshness should also be visible in the output. Showing when a source was last updated lets a reader apply their own judgement about whether it still holds.

Definitions before models

For anything numerical, the governed metric layer is the prerequisite. If revenue is defined three ways across the business, a forecast built on it inherits that ambiguity and produces a precise answer to an unclear question.

This is why the connected data work and the AI work are not separate programmes. The second depends on the first.

A realistic sequence

  • Inventory. What sources exist, who owns them, how current they are.
  • Prioritise. One domain that would answer a real question, not the whole estate.
  • Govern. Definitions, owners, access model.
  • Structure. Extract, chunk, attach metadata, index.
  • Prove. Build one grounded use case and evaluate it honestly.
  • Extend. Add domains against a pattern that already works.

This sequence is slower to start and considerably faster to finish than attempting everything simultaneously.

Key takeaways

  • AI projects stall on data readiness, not model capability.
  • Structure and metadata determine what retrieval can find.
  • Permissions must be enforced inside retrieval, per user.
  • Stale sources are worse than missing ones.
  • Governed definitions are a prerequisite for anything numerical.

Frequently asked questions

What does AI-ready data mean?

Data that is structured and chunked so meaning survives retrieval, carries metadata such as owner and effective date, enforces per-user permissions at query time, is kept current with a retirement path, and has governed definitions for anything numerical.

Do we need to fix all our data before starting with AI?

No. Prioritise one domain that would answer a genuinely valuable question, get that governed and structured properly, prove a use case on it, then extend the same pattern to further domains.

Assess Your Data Readiness?

Assess Your Data Readiness
Share this insight