Cloud ETL
Fivetran — planned
Will read sync history and schema change events via API, plus the _fivetran_synced column in every target. This integration is planned: documented here as reference, not connectable from the admin today.
Fivetran is a managed pipeline, which means the failure modes move rather than disappear. You will not lose rows to a mapping bug you wrote, but you can absolutely have a connector paused, a schema change silently applied, a table excluded from the sync, or a history-mode table growing rows nobody expected.
The compensating strength is that every destination table carries
_fivetran_synced, which makes freshness and reconciliation straightforward
without any additional instrumentation.
Fivetran today
Decim does not call cloud ETL service APIs today. Job scripts kept in a connected GitHub repository are read as code; a definition that exists only in a service console is not seen yet. Run logs can be read from Google Cloud Logging where they are shipped there, or exported and attached to the incident as files.
The sources that ship today are GitHub, Google Cloud Logging, Jira Cloud and the optional Windows agent. Everything below this point describes the planned Fivetran integration: what it is designed to read, and why.
What Decim will read from Fivetran
The intended read-only surface for this integration. It is reference material for the roadmap — and useful on its own, because these are the views and APIs worth knowing during an incident whether or not any tool reads them for you.
| What | From | Why it matters |
|---|---|---|
| Connector state | GET /v1/connectors/{id} | Paused state, sync frequency, last sync and failure |
| Sync history | GET /v1/connectors/{id}/logs | Start, finish and outcome per sync |
| Schema config | GET /v1/connectors/{id}/schemas | Which tables and columns are enabled or blocked |
| Row freshness | _fivetran_synced column | Present in every destination table — freshness for free |
| Soft deletes | _fivetran_deleted column | Rows removed upstream but retained downstream |
| Destination | GET /v1/destinations/{id} | Where the connector writes |
| Transformations | GET /v1/dbt/transformations | dbt runs orchestrated by Fivetran |
Permissions it is designed around
Written out in full, because "read-only access" is not a specification. These are the grants the planned integration is designed around and may change before it ships. Nothing below grants the ability to write, and row access would be requested only on the specific tables you name. You do not need to create any of this today.
-- Fivetran API: a key/secret pair, used with HTTP Basic auth.
-- Read endpoints only; Decim would never call resync, pause or modify.
-- GET /v1/connectors/{connector_id}
-- GET /v1/connectors/{connector_id}/schemas
-- In the destination warehouse, freshness needs only the metadata column.
GRANT SELECT (_fivetran_synced, _fivetran_deleted, id)
ON raw.transactions TO decim_agent;
-- Which is enough for the check that matters most:
SELECT
max(_fivetran_synced) AS last_sync,
count(*) AS rows_total,
count(*) FILTER (WHERE _fivetran_deleted) AS soft_deleted,
count(*) FILTER (WHERE _fivetran_synced > now() - interval '1 day') AS synced_today
FROM raw.transactions;What it would build
What this source would contribute to the pipeline topology and to the evidence available during an investigation:
- Source-to-destination edges per connector, with sync cadence
- Which tables and columns are excluded from the sync — often the real finding
- Schema change events, correlatable against a downstream break
- Freshness per table without any added instrumentation
Today, topology is built from code in GitHub, and each connection is marked direct or inferred. The design direction for sources like this one is a provenance model: a step found in a definition is declared, a step seen running is observed, one that is both is verified, and where two sources disagree the disagreement is kept as a drift note rather than resolved silently. That model is planned, not shipped.
Failure modes it would surface
What this source is uniquely good at proving — and, just as usefully, at disproving. An investigation that can refute a hypothesis cheaply is worth as much as one that confirms it. Until the integration ships, the same signals can be checked by hand, or requested by Decim as a check your team runs.
| Failure mode | The signal |
|---|---|
| Connector paused | Paused state with no alert configured |
| Table excluded from sync | Enabled flag false on a table downstream depends on |
| Schema change broke downstream | Column added or retyped, then a dbt model failing |
| Soft deletes misread as live rows | _fivetran_deleted not filtered in a model |
Limits
What this source cannot tell you, even once it is read. Stated because an investigation that overstates its sources produces confident wrong answers, which is worse than an honest cause unclear.
- Connector log retention through the API is limited; export to a destination for longer history
- You cannot see inside a connector's extraction logic — it is a managed black box by design
- History-mode tables have entirely different row-count semantics and must not be reconciled the same way
Related integrations
- Snowflake — ACCOUNT_USAGE gives a year of query, copy and task history — including rows loaded and rows rejected.
- dbt — manifest.json is the model graph; run_results.json is what actually happened. Both are artefacts, not APIs.
- BigQuery — INFORMATION_SCHEMA.JOBS gives every statement, its referenced tables and its output row count.
See all integrations, or how the sources are combined into one graph.
Get started
Investigating a Fivetran pipeline?
Bring an incident you already know the answer to. If Decim gets it wrong, that is a more useful demo than one where it doesn't.