FXMacroData's Singer tap is listed in Meltano Hub and emits announcements, release calendars, indicator catalogue metadata and daily FX reference rates. Start with a public USD configuration, discover the schema and inspect the emitted records before connecting a destination.
Who this guide is for: Data engineers using Singer or Meltano who need macro evidence in a research warehouse.
Outcome: A repeatable macro extraction job with Singer schemas, stable record IDs and a chosen destination.
A warehouse makes recurring macro work easier when every dataset enters through the same observable extraction process. Singer separates extraction from loading, which lets the team choose a destination while preserving stable identifiers and source fields. The valuable outcome is a repeatable research input whose missing values, access scope and date meanings remain visible after it leaves the API.
Use USD policy-rate evidence and the release calendar to orient the first session. The goal is to preserve the data's meaning as it moves into the research workflow.
1. Connect the accepted integration
The official Hub entry identifies the FXMacroData variant, its configuration fields and the source repository. Its current install reference follows the Git default branch rather than an immutable release. Record the installed commit in the pipeline's reproducibility record. Start with indicator discovery so configuration uses an actually published slug for the chosen currency.
Use the official listing for discovery and the FXMacroData integration README for its version-specific instructions.
meltano add tap-fxmacrodata --from-ref https://raw.githubusercontent.com/meltano/hub/main/_data/meltano/extractors/tap-fxmacrodata/fxmacrodata.yml
meltano config tap-fxmacrodata set indicators '["policy_rate"]'
meltano invoke tap-fxmacrodata --discoverThe public Hub definition works with Meltano's --from-ref route. Ordinary Hub discovery also supports meltano add tap-fxmacrodata after Meltano Cloud login. Invoke the extractor through Meltano so it uses the plugin's managed environment; adding it does not place a global tap-fxmacrodata command on your shell path. See the official Meltano CLI reference.
Research workflow
- 01Add the Hub extractor
Use the public Hub definition within the Meltano project.
- 02Discover catalogue and schema
Invoke discovery through Meltano's managed plugin environment.
- 03Inspect a bounded USD extraction
Select policy-rate observations and inspect the emitted schema and records.
- 04Validate the destination and state
Configure loader keys and review the job's scope before scheduling a recurring pipeline.
2. Run the native research workflow
Use a small USD configuration for the first extraction, then inspect both the Singer schema and the records emitted through Meltano. Announcements and FX rates are partitioned by their series or pair; catalogue and calendar streams provide discovery and planning context. The loader should retain stable keys and use a deliberate upsert policy rather than appending indistinguishable copies every time a job runs.
{
"currencies": ["USD"],
"indicators": ["policy_rate"]
}Choose the right result surface
| Native surface | Useful research output | Scope to check |
|---|---|---|
| announcements | One indicator's releases per currency | 0.1.0 uses reference-date incrementals and one history page |
| release_calendar / data_catalogue | Schedules and currency-specific indicator metadata | Full-table extraction within the endpoint's returned scope |
| forex_rates | Daily reference-rate records for configured pairs | Pairs and appropriate protected access must be configured |
3. Interpret the result before using it
Treat schema discovery as a research control, not just an installation step. Check nullable values at the destination and preserve announcement identifiers, reference dates and source links. Publication status and capture basis qualify the interpretation of announcement_datetime. They belong beside the value when the downstream question depends on what could have been known at a past cutoff.
Four meanings that should survive extraction
- Reference period
- The period described by an observation.
- Observed publication
- A publisher instant only when the response's quality fields establish it.
- Official schedule
- A future time or date-only announcement, retained with its status.
- Client receipt
- When a client received the response, separate from all three above.
4. Review scope, access and quality
Version 0.1.0 requests at most one page of 100 history rows per dated partition. A reference-date watermark also cannot detect every revision to an older period. State is therefore not evidence that a complete archive or vintage ledger has been captured. Its fixed projections and four streams do not cover every FXMacroData operation. Design the job around the selected data scope and verify pagination, revision policy and metadata retention before treating it as a warehouse backfill.
Public USD macro data can be evaluated without an API key, subject to the documented recent window, delay and fair-use limits. A start date does not override those rules. Optional api_key configuration widens access according to the dataset and subscription and is sent as an X-API-Key header. Keep credentials in the platform's private configuration, not shared JSON examples or command history.
A repeatable review checklist
- Compare emitted counts with page metadata
Compare rows with the endpoint's pagination scope before calling an extraction a backfill.
- Verify stable-key upserts
Upsert announcement_id and the FX pair/date key so reruns do not create accidental duplicates.
- Test a null value at the destination
Inspect a null at the destination; SQL defaults and transformations must preserve missingness.
- Recheck older-period revisions
A change to an older reference period may fall behind a date bookmark; evaluate revision capture separately.
Keep the original response or a reproducible local snapshot alongside the analysis. Measure row coverage, the treatment of missing values and the ability to reproduce the same window. If those checks fail, diagnose the extraction or interpretation before attributing a difference to the market.
Troubleshooting a first session
Unset indicators skip announcement selection, and unset fx_pairs skip FX selection as documented. Use the catalogue to resolve missing slugs. In 0.1.0, access failures and missing series can be logged and skipped; inspect the logs before accepting an empty output as a successful complete extraction. Discover a fresh schema when changing stream selection, and retain the installed commit and state separately for reproducibility.
Frequently asked questions
Is the tap officially listed?
Yes. The FXMacroData variant is present in the official Meltano Hub extractor catalogue.
Does a 2020 start date grant public history back to 2020?
No. Provider access windows still apply, and the 0.1.0 tap reads one history page.
Can a date watermark capture all revisions?
No. A revision to an older reference period can fall before that watermark; revision capture needs a separate, verified extraction policy.
Sources and next steps
Version and acceptance evidence: official integration listing and public adapter source. Data behaviour and optional access: FXMacroData API reference, hosted MCP guide and subscription access.
