Solutions
Your stack, your pipelines, the same evidence
Monitoring, a scheduler or a person on the finance team has already noticed the problem. Decim starts from that report and works out the cause, citing the code, log lines and ticket behind every conclusion. The method is the same on every stack; what changes is where the evidence lives.
The same investigation, on any stack
Decim does not need a particular platform underneath the pipeline. It needs the code, the logs and a description of what went wrong.
An incident starts as a plain-language description, such as “transaction volume dropped by 38% in the nightly ledger load”, optionally with the affected period, a linked Jira ticket and attached files. Decim matches it to a pipeline it discovered from your GitHub repository and names the step it will start from, citing the file and line range that made the match.
It then reads that pipeline's logs step by step, most suspicious first, and compares the failing run with successful runs from up to 30 days earlier. Each step ends as “looked fine”, “found something” or an honest gap such as “no logs”. From there it weighs up to five competing explanations, revises them as evidence arrives, and confirms a cause only when something corroborates it.
When the deciding evidence sits in a database Decim cannot read, it writes a read-only query for your team to run and takes the result back in. A fix arrives as a draft pull request; nothing is merged automatically. The pages below go through what that looks like on each kind of stack, along with the queries worth running by hand either way.
By stack
Four common shapes of data pipeline, and how an investigation reaches each one today.
Custom ETL services and scheduled jobs
Python, .NET, Java and SQL: queue consumers, Hangfire and cron jobs, file-arrival loaders, mappers and bulk inserts. Decim discovers each pipeline from its trigger in the repository and reads the code behind every step.
ReadSQL Server shops
SQL Agent jobs, SSIS packages and stored procedures. Procedures kept in git become steps in the discovered pipeline; evidence that lives in msdb or SSISDB is gathered through read-only queries your team runs.
ReadOrchestrated pipelines
Airflow DAGs and dbt projects, where every task is green and the number is still wrong. DAG files and models in the repository are read as code; the metadata database is reached through checks.
ReadCloud data teams
AWS Glue jobs, Bedrock workflows and similar services. Scripts kept in GitHub are read as code; logs reach the incident through Google Cloud Logging or as attached exports. Native AWS connectors are planned.
Read| Evidence | How it reaches the investigation | Status |
|---|---|---|
| Code, commits and pull requests | GitHub App, on the repositories you choose | Today |
| Pipeline structure | Discovered from code: trigger, steps, sources and destinations, with the lines behind each | Today |
| Logs | Google Cloud Logging sources you define; files and Windows Event Logs through the optional Windows agent | Today |
| The ticket | Jira Cloud issue imported as a snapshot: description, comments, attachments | Today |
| Exports and screenshots | Files attached to the incident, up to 500 MB each | Today |
| Evidence inside a database | A read-only query Decim writes and your team runs; Decim never executes it | Via a check |
| Database, scheduler and orchestrator connectors | SQL Server, PostgreSQL, SQL Agent, SSIS, Airflow, dbt, AWS Glue, Azure Data Factory and others | Planned |
By failure mode
The shapes that recur, whatever the stack, and what tends to identify each one.
Silent data loss
The run reported success and rows went missing: filtered out, diverted to a reject table, or never extracted. Nothing threw, so nothing alerted.
Duplicate rows
A retry re-ran a step that had already committed. The duplicate count matches the retried batch exactly.
Missing upstream input
The source never delivered. The pipeline short-circuited and exited clean.
Contention and deadlocks
Two jobs owned by different teams write to the same table in the same window, and neither definition mentions the other.
Late arrival and SLA breach
Row counts correct, timing wrong. Alerting on failure alone never catches it.
A change nobody connected
A parameter default, a mapping file or a filter changed in a release. The diff is small, the step still reports success, and the effect shows up days later in a report.
By role
Data engineers
Start from a located step and cited log lines instead of a blank terminal. Hypotheses already ruled out stay on the record with the evidence that ruled them out.
Platform and data leads
Every incident ends in a report: root cause with its confidence, the evidence, how it was verified and the fix. Exports as a PDF or JSON; an in-progress incident exports as an interim report.
Security and compliance
You choose the repositories, the log filters and what the agent is pointed at. Every user action is in an exportable audit log. Evidence is processed by a third-party language-model provider, with no redaction today.
For the method in detail, see how investigations work and what counts as evidence. For data handling and the current security posture, see security.
Get started
Bring us a pipeline that broke last week
The fastest way to evaluate this is a real incident you already know the answer to. If Decim gets it wrong, that is a far more useful demo than one where it doesn't.