Skip to content
KELTON PARTNERSINTELLIGENCE

AI where it adds leverage. Human judgement where it matters.

A four-layer operating model for combining rules, AI execution, specialist review and client authority.

AI is most valuable in investment operations when it changes the unit economics of repetitive, information-heavy work without obscuring who remains accountable.

That requires a more precise operating model than simply placing a “human in the loop”.

Divide the workflow into four layers

1. Rules and data

The workflow begins with approved inputs, defined permissions and known business rules. The system should know which source is authoritative, which fields are required and which users or services may access the information.

Conventional controls still matter. Required-field checks, reconciliation tolerances, entity mappings and deadline rules often provide a more dependable foundation than asking an AI model to infer everything.

2. AI execution

AI can add leverage where work contains variation or unstructured information. Examples include:

  • classifying inbound documents;
  • extracting and normalising fields;
  • drafting explanations for exceptions;
  • summarising changes between reporting periods;
  • retrieving relevant procedures or governing-document clauses;
  • preparing a first version of a recurring communication;
  • prioritising items for review.

The AI action should have a bounded purpose and structured output. “Review this fund” is not a workable instruction. “Extract these fields, identify missing evidence and route exceptions according to these rules” is much closer to an operating process.

3. Specialist review

Review should be based on the risk and uncertainty of the output.

A specialist may need to assess:

  • an unusual accounting treatment;
  • an unresolved reconciliation difference;
  • a document that does not match known formats;
  • an investor or regulatory communication;
  • a low-confidence extraction;
  • a case that requires interpretation rather than classification.

The reviewer needs the source evidence, model output, relevant rules and changes in one place. Requiring a human to reconstruct all context removes much of the benefit of automation.

4. Client approval

Material actions should remain subject to agreed authority. Approval may be required before releasing a report, instructing a provider, accepting an adjustment or communicating with an investor.

The client should be able to see:

  • what the system did;
  • what the specialist changed;
  • which exceptions remain;
  • who approved the final result;
  • which version was released.

Use risk-based review thresholds

Not every output requires the same review depth. A practical design may combine:

  • full review for material or externally released outputs;
  • exception-only review for stable, rule-bound processes;
  • sampling for high-volume, low-impact classifications;
  • automatic rejection or escalation for prohibited data, missing evidence or low confidence;
  • periodic control testing across all categories.

Thresholds should be documented and monitored. If reviewer corrections rise, the process may need retraining, new rules or a return to manual handling.

Make decisions traceable

Traceability is broader than storing a model response. The evidence should connect:

  • source document or dataset;
  • workflow and model version;
  • rules and prompts used;
  • AI output and confidence or exception status;
  • reviewer changes and comments;
  • approval and release status.

This supports investigation, control testing and continuous improvement. It also helps the firm understand whether an issue came from source data, workflow design, model behaviour or human review.

Keep the fallback path operational

Every AI-enabled workflow needs a route for cases the system should not handle. That may include:

  • unsupported document type;
  • missing or contradictory source data;
  • model or integration outage;
  • material policy exception;
  • suspected confidentiality or permission issue;
  • new transaction or structure outside the approved scope.

The fallback should not be “ask the operations team to figure it out”. It should specify the owner, required evidence and expected resolution path.

Measure the combined system

Model accuracy alone does not show whether the operating process is better. Measure:

  • end-to-end cycle time;
  • reviewer effort and correction rate;
  • exceptions detected and missed;
  • unresolved ageing;
  • rework after approval;
  • user adoption;
  • incidents and permission breaches;
  • quality of the final deliverable.

The unit of evaluation is the workflow, not only the model.

The operating principle

AI should do more of the preparation, pattern recognition and routing. Specialists should apply judgement to exceptions and material decisions. Clients should retain clear authority over the final outcome.

That is how AI adds leverage without making accountability disappear.

Sources and further reading

  • NIST AI Risk Management Framework: Source
  • NIST Generative AI Profile: Source
  • Citco on an AI-plus-human approach to document intelligence: Source
  • SS&C on intelligent automation combined with human review: Source

This article is general information and is not legal, accounting, regulatory or investment advice.

CONTINUE READING

Building control without duplicating your fund administrator

Read next