A Successful Data Load Is Not a Successful Integration

Reliable investment-system integration requires more than a clean data load. Agree on meaning, test lifecycle events, and make exceptions accountable.

A Successful Data Load Is Not a Successful Integration

An interface can load every row without answering the question an investment operations leader actually cares about: can the business trust the result?

For firms connecting trading, portfolio management, accounting and reporting systems, technical acceptance is an incomplete definition of success. A file may arrive on time and pass structural checks while its dates, identifiers or missing values still require interpretation. The result deserves scrutiny before it feeds another decision or report.

The stronger goal is an integration that preserves business meaning, makes exceptions visible and has a clear owner when something goes wrong.

Agree on meaning before mapping fields

A field mapping should explain more than where a value goes. It should record what the value means, when it is required, which source is authoritative, and how the receiving system should interpret it.

“Optional” deserves particular attention. A value that is optional for one system may be essential for a downstream calculation. Teams should distinguish an unknown value from an inapplicable field, an empty string, and a genuine zero.

This is a recognised data-modelling issue. W3C’s tabular data standards provide mechanisms for declaring data types, null markers, and validation metadata.[1][2] They do not supply an investment firm’s business rules; those still need agreement between technology and operations.

Before development starts, approve one shared specification covering definitions, required conditions, defaults, identifiers, and change control. Resolve conflicting versions before they become competing implementations.

Test the business lifecycle

A sample file demonstrates what happened in that sample. It does not establish that an integration handles every event within its agreed scope.

Build a coverage matrix around the instruments and events the business expects to process. Depending on scope, this could include new trades, amendments, cancellations, settlements, and relevant corporate actions. Mark exclusions explicitly and identify the process that will handle them.

Consider a hypothetical integration that successfully loads a new trade but has no agreed treatment for a later cancellation. Its initial load can pass while the receiving system continues to show a position that should have been removed.

The acceptance test should therefore connect an incoming event to its expected position, cash, or reporting effect. Business users should approve those outcomes alongside the technical team.

Reconcile outcomes, then investigate differences

Row counts are useful, but they answer a narrow question. Matching numbers of records does not show that the correct accounts, securities, dates, and amounts arrived.

Define checks at several levels: file completeness, unique identifiers, field validity and business totals. Choose measures appropriate to the instrument and workflow, with documented tolerances and an agreed valuation basis where relevant.

The Basel Committee’s risk data principles arose partly from banks’ difficulties aggregating exposures fully, quickly and accurately.[3] Their banking context matters: they are not a blanket requirement for investment managers. They nevertheless offer a useful reference for treating timely, accurate information as a management concern.

Every material difference should have a disposition: correct it, accept it with a documented reason, or stop the affected process.

Make recovery part of acceptance

Daily reliability also depends on what happens after a late file, a rejected record or an interrupted run.

Define who receives an alert, who can authorise a correction, and when unresolved exceptions must be escalated. Test whether a rerun creates duplicates and whether corrected data reaches every affected downstream output. Preserve enough evidence to reconstruct what was received, transformed, rejected and approved.

Before signing off, ask four questions: Are the data definitions agreed? Are in-scope lifecycle events tested? Do business outcomes reconcile? Can the team recover from a failed run?

Those questions turn go-live approval into an assessment of operational readiness.

Which of those four would be hardest to answer for your most important integration today? If the question is timely for your team, connect with IEVERS to continue the conversation.

References:

[1] W3C, Model for Tabular Data and Metadata on the Web: https://www.w3.org/TR/tabular-data-model/
[2] W3C, Metadata Vocabulary for Tabular Data: https://www.w3.org/TR/tabular-metadata/
[3] Basel Committee, Principles for effective risk data aggregation and risk reporting, 9 January 2013: https://www.bis.org/publ/bcbs239.htm