What “ETF data infrastructure” actually means

Every ETF issuer runs data infrastructure, whether they call it that or not. A spreadsheet emailed to a web agency every morning is data infrastructure — just slow, manual, and fragile. What separates an industrial pipeline from the spreadsheet is not technology for its own sake; it is four stages, each of which exists because something specific goes wrong without it.

Ingestion: meeting data where it is produced

Fund data originates with administrators, custodians, index providers, and exchanges, each with their own formats, schedules, and quirks. An ingestion layer accepts those formats as they are, records exactly what arrived and when, and retries sensibly when a file is late. The alternative — asking every provider to standardize — has been tried by every generation of the industry and has never worked.

Validation: the unglamorous stage that pays for everything

Raw fund data arrives wrong often enough that publishing it directly is negligent. NAVs appear with the wrong sign or an extra zero; holdings files arrive truncated; a share count reflects yesterday's split and the NAV does not. Validation checks each observation against expectations — tolerance against the prior value, consistency across related fields, completeness against the expected universe — and quarantines what fails rather than letting it reach a public page.

Equally important is what happens when a provider corrects yesterday's data. A serious pipeline versions the correction: the original value, the corrected value, and when each was known are all retained. Pipelines that overwrite in place cannot answer the question “what did we publish on Tuesday?” — a question auditors and compliance teams do ask.

Identity: the hardest problem in fund data

An ETF is not one thing. A fund has share classes; share classes have listings; listings have tickers that get reused and ISINs that get reissued. A pipeline that keys everything by ticker will eventually attach one fund's history to another fund. Industrial pipelines maintain stable internal identifiers with effective-dated mappings to tickers, ISINs, and exchanges, so that a ticker change is an event in the data rather than a corruption of it.

Distribution: one record, many surfaces

The payoff of the first three stages is that every downstream surface — product pages, APIs, file feeds, comparison tools, internal analytics — reads from the same validated record. When the website, the factsheet, and the sales deck disagree about a fund's AUM, the cause is almost always that they are fed by different pipelines. The fix is architectural, not editorial.

None of these stages is exotic. What they require is the discipline to run them every day, monitor them, and treat a late file at 6 a.m. as an operations problem rather than a surprise at 9. That operational layer is the part of the infrastructure you cannot see on a website — and the reason the website is right.