Financial research is often presented from the model outward: a signal, a chart, then a conclusion. We are building AtlasSignals in the opposite direction. Before a macro score can be credible, every observation beneath it needs a stable definition, traceable source, explicit transformation, and honest account of what was unavailable.
Atlas Data is the active alpha foundation for that work. Its current scope is deliberately narrow: acquire official source data, retain the raw response, normalise observations into a canonical contract, report incomplete loads, and expose a reproducible research view. It does not yet claim to be a production point-in-time archive, and it does not own macro scores or investment decisions.
A signal is only as trustworthy as the history, definitions, and failure states beneath it.
Begin with product boundaries
Governance starts with deciding which layer owns which meaning. In the current Atlas architecture, atlas.data owns catalogues, ingestion, observations, provenance, and quality. Future signal and regime interpretation belongs to atlas.macro; portfolio construction belongs to atlas.allocation; risk models and stress behaviour belong to atlas.risk.
That separation matters for more than code organisation. It prevents a provider connector from silently becoming an economic model, and it prevents a downstream model from depending on an undocumented source convention.
The governed data boundary
Illustrative architectureThe dependency direction is simple:
- source connectors satisfy a typed request contract;
- Atlas Data converts successful responses into canonical observations;
- downstream products consume only the public Atlas Data result;
- later interpretation remains outside the ingestion layer.
A canonical observation is more than a date and value
A two-column time series is convenient, but it is insufficient for governed research. The present long-form observation contract retains the economic identity and the source identity separately.
| Field group | Examples | Why it matters |
|---|---|---|
| Observation | series_id, observation_date, value | Identifies the Atlas concept and measured value. |
| Source | provider, provider_series_id, original_owner | Preserves where the observation came from. |
| Meaning | jurisdiction, source_units, units, observation_frequency | Makes transformations and comparisons explicit. |
| Time | retrieval_timestamp, release_timestamp, vintage_date | Separates what was observed from when it became knowable. |
| Quality | quality_status, is_preliminary, source_revision_identifier | Carries uncertainty instead of hiding it. |
The stable Atlas identifier describes the economic concept. A provider code remains source metadata. This means an upstream naming change does not need to leak into every model, while a researcher can still trace the exact series that supplied a value.
from atlas.data import DataPipeline, FailureMode
pipeline = DataPipeline()
result = pipeline.load_dataset(
"rates_mvp",
failure_mode=FailureMode.STRICT,
)
observations = result.observations # canonical long-form records
research_view = result.to_wide() # derived convenience view
The wide frame is useful for analysis. It is not the record of provenance. Treating the convenient view as the system of record would discard the context needed to reproduce and audit a result.
Four clocks, not one
Economic data does not have a single timestamp. At minimum, a credible historical reconstruction must distinguish:
- the observation date—the period the value describes;
- the release time—when the owner first made it available;
- the vintage—which revision of the observation is represented;
- the retrieval time—when Atlas obtained that payload.
The clocks behind an observation
Illustrative timelineThe current implementation records these fields when a source supplies them, but nullable metadata is not fabricated. That is an important constraint: “unknown” is more truthful than an inferred timestamp presented as fact.
Retain the source before transforming it
Successful remote responses are retained locally as immutable, content-addressed payloads with a sidecar describing the connector, request, retrieval time, status, and parser version. Normalisation then applies source mappings and units to produce canonical observations.
This creates a practical chain:
catalogue definition
-> typed provider request
-> immutable raw payload + request metadata
-> deterministic normalisation
-> canonical observation
-> derived research view
Content addressing is useful because the stored object is identified by its bytes rather than an arbitrary filename. Re-running a parser against the same payload can therefore be distinguished from retrieving different source content.
Failure is part of the data
Market and economic feeds fail unevenly. One jurisdiction can be available while another times out; one series can change format while its neighbours continue to parse. Hiding that distinction creates false confidence.
Atlas Data models partial results explicitly. A load result reports requested, loaded, and missing series alongside series-level and provider-level errors. Two policies make the caller’s intent clear:
- strict mode raises when any requested series is absent;
- best-effort mode retains healthy observations and records the gaps;
- both modes fail if no usable observations can be produced.
This is not merely operational detail. A macro signal calculated on reduced coverage may need lower confidence, a different explanation, or no publication at all. Failure metadata is therefore an input to future intelligence, not log noise to discard.
Cache is not history
The current SQLite cache is a replaceable performance layer. Its time-to-live governs refresh behaviour, and a fingerprint invalidates an entry when its upstream mapping, units, or source configuration changes.
That makes local iteration faster. It does not create a point-in-time database. A cache can expire, be replaced, and omit prior revisions; it cannot answer what a model genuinely knew on a historical date.
The next major boundary is an append-only observation repository that can:
- connect each normalised row to a retained response and parser version;
- preserve release, revision, and retrieval history;
- execute
as_ofqueries that cannot see later information; - record explicit quality and quarantine outcomes;
- reproduce a historical dataset deterministically.
Only after that boundary is proven should new macro signal implementations rely on the data foundation.
What this foundation makes possible
Governed data does not make a model correct. It makes the model’s inputs inspectable and its historical claims testable. That distinction is foundational for the Atlas direction: explainable macro intelligence should be able to show not only a signal and its contributors, but also the observations, source mappings, transformations, missing-data policy, and information set that produced it.
The immediate work remains intentionally modest: strengthen the data contract, prove point-in-time storage, and keep the product boundary clear. More sophisticated charts can wait. Trust cannot.
Sources and scope
- AtlasSignals repository: Layer One Architecture
- AtlasSignals repository: Atlas Data Pipeline
- AtlasSignals repository: Layer One Data Dictionary
This article describes the repository state at publication and is for research and information only. It is not investment advice.