Scheduled Job Troubleshooting: When a Job Fails Silently or Runs and Does Nothing

The scheduled jobs that cost the most are not the ones that fail loudly. Those page someone, and someone reads the error. The expensive ones are the job that never started and nobody noticed, the job that exited 0 in three seconds instead of twelve minutes, and the job that ran twice because a retry and the next schedule overlapped. Every dashboard that watches exit codes shows green.

Scheduled job troubleshooting is mostly about asking the right questions in the right order: did it start, what exactly ran, what did it see, what did it write, how long did it take, and how does that compare with a night that worked. The questions are the same on every platform. Where the answers live is not. Cron writes to syslog or nowhere, systemd to the journal, Kubernetes to events and pods that are garbage-collected, Cloud Run to Cloud Logging under two different resources, Airflow to its metadata database and task logs, and SQL Server Agent to msdb.

This guide gives the platform-independent method first, then the specific places to look on each of six common schedulers, then the causes behind the most common silent failure: a job that ran and did nothing.

Table of Contents

• Five ways a scheduled job goes wrong
• The questions to ask on any platform
• Cron
  • Did it run at all
  • The environment is not your shell
  • Missed runs and overlapping runs
• systemd timers
• Kubernetes CronJobs
  • Where the evidence goes
  • The fields that decide whether a Job starts
• Cloud Run jobs and Cloud Scheduler
• Airflow and other orchestrators
• SQL Server Agent and SSIS
• Jobs that succeed and do nothing
• Making the next failure visible
• Where Decim fits
• Frequently asked questions
• Related reading

Five ways a scheduled job goes wrong

Before reading any logs, decide which of these you are looking at. Each one sends you to a different place first.

Failure shape	What you see	Look first at
Did not start	No output, no log lines, no record of a run	The scheduler’s own record: is the job enabled, did the trigger fire, was it suppressed
Started and failed	Non-zero exit, error in logs, possibly retried	The first error in time order, not the last
Ran and did nothing	Success status, short duration, no output	What the job read as input and how it decided there was nothing to do
Ran twice or overlapped	Duplicate records, two runs in history, locks	Retry settings, concurrency policy, manual reruns
Ran late or too long	Downstream consumers saw stale data	Duration history, queueing, timeouts

The third shape, ran and did nothing, is the one people search for at 9am when a report is empty. It is also the hardest to see, because by the scheduler’s standards nothing went wrong. It gets its own section below.

The questions to ask on any platform

The method is the same as any root cause analysis: state the symptom precisely, find where behaviour diverged, compare with a healthy run, test rival explanations. For scheduled jobs that turns into six concrete questions.

1. Did it start? Find the scheduler’s record of the trigger firing, separate from the job’s own logs. A missing job log with a present trigger record means the job started and died before logging, or logged somewhere else.
2. What exactly ran? The command line, image tag, version, parameters and environment variables of this run, not of the definition you are looking at today. Definitions change; you need the one that was in force.
3. What did it see? The input it found: files, rows, messages, the watermark it read. “Found 0 items” is the most important log line a job can write.
4. What did it write? Outputs, rows, uploads, messages sent. Check the destination directly, not the job’s claim.
5. How long did it take? Compare with the same job’s history. A job that finished in a tenth of its usual time did not do its usual work.
6. How does this run differ from the last good one? Put the two runs side by side. Lines present in the good run and missing from the bad one are usually where the story is.

Question	Cron	systemd	Kubernetes	Cloud Run	Airflow	SQL Agent
Did it start	syslog / journal CRON lines	systemctl list-timers, journal	CronJob events, Job list	Execution list, Scheduler logs	DAG run list	sysjobhistory step 0 row
What ran	crontab, script	unit files	Job spec, image tag	Execution’s job revision	DAG version, task params	sysjobsteps.command
Output and errors	Mail or redirect, else lost	Journal for the unit	Pod logs (while pod exists)	Cloud Logging, cloud_run_job	Task logs	sysjobhistory.message, SSIS catalog
Duration history	Only if you log it	Journal timestamps	Job start and completion times	Execution start and end	Task instance durations	run_duration

Cron

Classic cron is the simplest scheduler and keeps the least evidence. It records that it started a command, and that is all. Everything else is up to the job.

Did it run at all

Most cron implementations log each command they start through syslog. On systemd-based distributions that ends up in the journal; on others in /var/log/syslog or /var/log/cron. The daemon’s name and log destination vary by distribution, so check both.

# Did cron start the job? (Debian/Ubuntu often name the unit "cron", Fedora/RHEL "crond".)
journalctl -u cron -u crond --since "2026-11-04 01:50" --until "2026-11-04 02:30" | grep -i export
grep -i export /var/log/syslog /var/log/cron 2>/dev/null | tail -20

# Which crontab is actually installed for the user the job runs as?
sudo crontab -l -u svc-export
ls -l /etc/cron.d/ && cat /etc/cron.d/export-invoices

If there is no line at all, the job did not start. Common reasons: the crontab was edited but not installed (a file in a repository is not a crontab until crontab or the package manager puts it in place), the file in /etc/cron.d has the wrong permissions or a name with a dot that some cron implementations ignore, the last line lacks a trailing newline, or the machine was off at the scheduled time.

That last point about the newline is documented: the crontab(5) manual notes that cron requires each entry to end in a newline character; if the last entry is missing one, cron considers the crontab at least partially broken and writes a warning to syslog.

The environment is not your shell

The most common cron failure is a command that works when you run it by hand and fails under cron. The reason is almost always the environment. According to crontab(5), cron sets SHELL to /bin/sh and takes LOGNAME and HOME from the passwd entry. It does not read your shell profile, so anything set there, a longer PATH, a virtualenv, a cloud credential variable, is missing. The cron(8) manual for the cronie implementation documents a -P flag precisely because cron sets its own PATH by default instead of inheriting one.

Two other details from crontab(5) catch people:

• A % in the command is turned into a newline unless escaped with a backslash, and everything after the first % is sent to the command as standard input. A date +%F in a crontab line silently becomes a different command.
• Output is mailed, or lost. Cron mails the command’s output to the crontab owner or to MAILTO. With MAILTO="" or no working mail setup, stdout and stderr go nowhere. Many “silent” cron failures logged a perfectly clear error to a mailbox nobody reads.

Also from the same page: when both the day-of-month and day-of-week fields are restricted, the job runs when either matches, not both. 0 2 1 * 1 runs on the first of the month and on every Monday.

The fix for most of this is a wrapper that makes the environment explicit and keeps the evidence:

# /etc/cron.d/export-invoices  (note the trailing newline)
SHELL=/bin/bash
PATH=/usr/local/bin:/usr/bin:/bin
MAILTO=""
0 2 * * * svc-export flock -n /run/lock/export-invoices.lock /opt/export/run.sh >> /var/log/export-invoices.log 2>&1

#!/usr/bin/env bash
# /opt/export/run.sh — fail loudly, log what was seen and done.
set -euo pipefail
echo "$(date -Is) start version=$(cat /opt/export/VERSION) watermark=$(cat /var/lib/export/watermark)"
count=$(/opt/export/bin/export --since "$(cat /var/lib/export/watermark)" --dry-count)
echo "$(date -Is) found ${count} invoices to export"
if [ "$count" -eq 0 ] && [ "$(date +%u)" -le 5 ]; then
  echo "$(date -Is) ERROR zero invoices on a weekday" >&2
  exit 3
fi
/opt/export/bin/export --since "$(cat /var/lib/export/watermark)"
echo "$(date -Is) done"

Note that the % signs inside the script are fine. Only the crontab line itself is subject to the % rule.

Missed runs and overlapping runs

Plain cron does not catch up. If the machine was down or the daemon stopped at 02:00, the 02:00 run simply does not happen, and nothing records that it was missed. Tools such as anacron exist for machines that are not always on, and systemd timers have Persistent= (below).

Cron also does not prevent overlap. If a run takes longer than the interval, the next one starts anyway, and two copies process the same input. The flock -n in the example above makes the second copy exit immediately. Whether that exit should be a failure is a decision worth making on purpose.

systemd timers

A systemd timer triggers a service unit, and both are recorded in the journal. That makes timers far easier to investigate than cron, as long as you look at the service rather than the timer.

# When did each timer last fire, and when will it fire next?
systemctl list-timers --all

# The service's last result, exit code and recent log lines.
systemctl status export-invoices.service

# Everything the job wrote, for the night in question.
journalctl -u export-invoices.service --since "2026-11-04 01:55" --until "2026-11-04 03:00" -o short-iso

# Check what an OnCalendar expression actually means, including the next trigger times.
systemd-analyze calendar --iterations=3 "Mon..Fri *-*-* 02:00:00"

Three settings explain most surprises, all documented in systemd.timer(5):

Setting	What it does	Why it matters in an incident
Persistent=	If true, a run missed while the timer was inactive (for example, the machine was off) is triggered when it becomes active again. Only applies to OnCalendar= timers; defaults to false	Without it, missed runs are skipped, as with cron. With it, a run may happen at boot time, hours late
AccuracySec=	Lets systemd coalesce wake-ups within a window; the default is one minute	A job scheduled for 02:00:00 may start a little later, which matters if it races another job
RandomizedDelaySec=	Adds a random delay to each trigger	Two runs of the same job on different nights can start at noticeably different times

The calendar syntax itself is in systemd.time(7). systemd-analyze calendar is the quickest way to check an expression; a timer that “never fires” often has an expression that means something other than what its author intended.

If the service unit is Type=oneshot, its exit status is the job’s exit status, and systemctl status shows it. A script that swallows its own errors still reports success here, so the same discipline as the cron wrapper applies: fail on unexpected conditions and log counts.

Kubernetes CronJobs

A Kubernetes CronJob creates a Job on schedule, and the Job creates pods. Each layer keeps different evidence, and the evidence disappears on a timer of its own.

Where the evidence goes

The CronJob documentation sets successfulJobsHistoryLimit to 3 and failedJobsHistoryLimit to 1 by default. When older Jobs are removed, their pods and the pod logs on the node go with them. On a job that runs every few minutes, the run you care about may have been garbage-collected before anyone looked. If your cluster ships container logs to a central store, that store is the real history; if it does not, there may be none.

# The CronJob's view: schedule, suspend flag, last schedule and last successful time.
kubectl get cronjob export-invoices -o wide
kubectl describe cronjob export-invoices      # includes recent events

# Jobs it created, newest last, with completion status.
kubectl get jobs --sort-by=.metadata.creationTimestamp | grep export-invoices

# Pods for one Job, and their logs (including the previous container if it restarted).
kubectl get pods -l job-name=export-invoices-29371440
kubectl logs job/export-invoices-29371440 --all-containers
kubectl logs export-invoices-29371440-abcde --previous

# Events are retained only briefly; read them early.
kubectl get events --field-selector involvedObject.name=export-invoices --sort-by=.lastTimestamp

Read events early in the investigation. They hold the reasons a Job was not created, a pod could not be scheduled, an image could not be pulled or a container was killed for exceeding its memory limit, and the API server keeps them only for a short time.

The fields that decide whether a Job starts

Field	Default	What to check
suspend	false	A suspended CronJob creates no Jobs and records no error. Someone may have suspended it during an earlier incident
concurrencyPolicy	Allow	Allow permits overlapping runs; Forbid skips a new run while the previous one is still running; Replace cancels the running one
startingDeadlineSeconds	unset (no deadline)	If set, a run that cannot start within this many seconds of its scheduled time is counted as missed
timeZone	the controller’s local time zone	A schedule written for local time running in UTC is off by hours, and shifts with daylight saving
successfulJobsHistoryLimit / failedJobsHistoryLimit	3 / 1	How much evidence survives

The same page states two behaviours worth knowing before an incident. First, scheduling is approximate: in some circumstances two Jobs might be created for one schedule, or none, so the Jobs you define should be idempotent. Second, if the controller finds more than 100 missed schedules, it records a TooManyMissedTimes warning event and starts only the most recent one; earlier missed runs are never started. With concurrencyPolicy: Forbid, a long-running Job counts the schedules it blocked as missed.

Inside the Job, the pod’s retries are governed by the Job’s backoffLimit and activeDeadlineSeconds. Every retry runs the container from the beginning. If the container had already written half its output, the retry writes it again unless the code is idempotent. A Job that reports Complete after a retry has run its logic more than once.

Cloud Run jobs and Cloud Scheduler

On Google Cloud, a common setup is a Cloud Run job triggered by Cloud Scheduler. That gives you two records for every run, and they answer different questions.

The scheduling guide has Cloud Scheduler send an authenticated HTTP POST to the Cloud Run Admin API’s jobs/JOB:run endpoint. The Scheduler job’s status records the outcome of that call, which starts an execution. Whether the execution then succeeded is recorded on the execution, not on the Scheduler job. A green Scheduler history is evidence that runs were requested, not that they completed.

Within the execution, two defaults matter. Per the task timeout page, each task runs for a maximum of 10 minutes by default, configurable up to 168 hours. A job that grew past ten minutes of work is cut off, and the logs show it stopping mid-run. Per the retries page, each task retries up to 3 times by default, and Google’s own advice is to make jobs idempotent so a restart does not produce corrupt or duplicate output.

# Executions of the job, with status and timing.
gcloud run jobs executions list --job export-invoices --region europe-west1

# One execution in detail: task counts, failures, timeout, retries.
gcloud run jobs executions describe export-invoices-x7k2p --region europe-west1

# The Scheduler side: did the trigger fire and get a successful response?
gcloud scheduler jobs describe export-invoices-nightly --location europe-west1

Logs for both are in Cloud Logging. Cloud Run job logs use the cloud_run_job resource type, and each entry carries labels for the execution name, the task index and the task attempt, documented on the Cloud Run logging page. Filtering by execution name gives you one run; adding the attempt label separates a first attempt from its retries.

resource.type="cloud_run_job"
resource.labels.job_name="export-invoices"
labels."run.googleapis.com/execution_name"="export-invoices-x7k2p"
severity>=DEFAULT

Label key formats in the query language can differ from how they are listed in the documentation, so if the filter returns nothing, open one entry from the execution in Logs Explorer and copy the label key from it. The Google Cloud Logging debugging queries article covers comparing two runs side by side.

Airflow and other orchestrators

Orchestrators such as Airflow, Dagster and Prefect keep much richer run history than cron, which helps, and introduce their own notion of success, which can mislead.

In Airflow, the DAG run documentation defines a run’s final state by its leaf tasks, the tasks with no children. A run is marked success when all leaf tasks are success or skipped, and failed when any leaf task is failed or upstream failed. Two consequences follow:

• Skips propagate quietly. A branch or a sensor that skips can skip everything downstream, and the run is still green. A DAG run where the load task was skipped looks the same in the run list as one where it loaded.
• Trigger rules change the meaning of success. The same page notes that a leaf task with the all_done trigger rule runs regardless of upstream states. If that leaf is a cleanup or notification task and it succeeds, the DAG run can be marked success even though earlier tasks failed.

Time is the other trap. The logical date of a run denotes the start of its data interval, not when it executed, and a run is usually scheduled after its interval has ended. A daily run labelled 3 November typically executes early on 4 November and processes 3 November’s data. Queries built on “today” instead of the run’s data interval read the wrong window, and the run succeeds with nothing to do.

Catchup also differs from what many teams remember. Current Airflow documentation describes catchup as off by default (catchup_by_default=False), so after a pause or an outage the scheduler creates a run only for the latest interval; the missed ones are not backfilled unless you ask. Older deployments may have it on. Check the setting for your version and the DAG’s own catchup argument before assuming missed intervals were processed.

Question	Where to look in Airflow
Did the run start	DAG runs list for the logical date; scheduler logs if there is no run
Which tasks actually did work	Task instance states: look for skipped and upstream_failed, not only failed
What window did it process	The run’s data interval, and the task’s rendered template fields
How long each task took	Task instance durations compared with previous runs
Was it retried	Try number on the task instance; each try has its own log

The same questions apply to Dagster, Prefect and other orchestrators with different vocabulary. Find out how each one defines a successful run, how it treats skipped work, and which time window a run is supposed to process.

SQL Server Agent and SSIS

SQL Server Agent keeps its job definitions and history in msdb, and SSIS packages deployed to the catalog keep their execution history in SSISDB. Both are readable with plain queries, and both have formats that mislead.

The dbo.sysjobhistory reference documents the traps. run_date and run_time are integers in yyyyMMdd and HHmmss form, and run_duration is also HHmmss packed into an integer: 13000 means 1 hour 30 minutes, not 13,000 seconds. A row with step_id = 0 is the job outcome, not a step. message is nvarchar(4000), so long errors, particularly from SSIS steps, are truncated. History is also pruned by size, per job and in total (resize the job history log), so a frequently running job may have lost the night you care about.

-- SQL Agent step history for one job, last 14 days, with real datetimes and durations.
SELECT  j.name AS job_name, h.step_id, h.step_name,
        DATETIMEFROMPARTS(h.run_date / 10000, h.run_date / 100 % 100, h.run_date % 100,
                          h.run_time / 10000, h.run_time / 100 % 100, h.run_time % 100, 0)
                                                        AS started_at,
        (h.run_duration / 10000) * 3600 + (h.run_duration / 100 % 100) * 60
          + (h.run_duration % 100)                      AS duration_s,
        h.run_status,          -- 0 failed, 1 succeeded, 2 retry, 3 cancelled
        h.retries_attempted,
        LEFT(h.message, 400)                            AS message_start
FROM    msdb.dbo.sysjobhistory AS h
JOIN    msdb.dbo.sysjobs       AS j ON j.job_id = h.job_id
WHERE   j.name = N'Nightly invoice export'
  AND   h.run_date >= CONVERT(int, FORMAT(DATEADD(day, -14, GETDATE()), 'yyyyMMdd'))
ORDER   BY started_at DESC, h.step_id;

A job can also report success when a step failed. Each step has an on_fail_action, and per the dbo.sysjobsteps reference the values are 1 quit with success, 2 quit with failure (the default), 3 go to next step and 4 go to a specific step. A step set to “go to next step” lets the job carry on past a failure, and the outcome row can read “The job succeeded.” The command column in the same table is the exact command the step ran, including parameters, which is where you see a parameter the step never passes.

-- Steps whose failure does not fail the job. Each one should be a deliberate choice.
SELECT j.name AS job_name, s.step_id, s.step_name, s.on_fail_action, s.retry_attempts
FROM   msdb.dbo.sysjobsteps AS s
JOIN   msdb.dbo.sysjobs     AS j ON j.job_id = s.job_id
WHERE  s.on_fail_action IN (1, 3, 4)
ORDER  BY j.name, s.step_id;

For SSIS packages run from the catalog, the full messages are in catalog.event_messages for the execution. SSIS failures cascade, and the last message is the least informative one: read errors in time order and start from the first.

Jobs that succeed and do nothing

This is the failure shape behind most “scheduled job failed silently” searches. The job started, exited cleanly, and wrote nothing. The causes are few, and they are the same on every platform.

Cause	What the run looks like	Evidence that confirms it
Empty input	Very short duration; “found 0” or no per-item log lines	Input listing at run time; upstream producer’s logs
Wrong time window	Normal-looking run that processed the wrong day	Window values in the log; the scheduler’s run date vs the code’s notion of “today”
Watermark or checkpoint already advanced	Reads “nothing newer than X”	Control table or state file value and who last wrote it
Wrong environment or target	Did the work, somewhere else	Environment variables and config of the run; the other target’s contents
Swallowed error	Error logged at warning level, then exit 0	The code’s error handling; the log line before the end
Feature flag or kill switch	Early return with a short message	Flag history; the code path that checks it
Path or glob matched nothing	“No files” or silent	The pattern, the directory listing at that time, file naming changes upstream
Permissions changed	Read returns empty rather than failing	Service account or role changes; access logs on the source

The pattern behind all of these is that the job treated “nothing to do” as a normal outcome. Sometimes it is. Often it is a fault in disguise, and the code has no way to tell the difference unless someone told it what normal looks like. The articles on ETL jobs that succeed with no data loaded and pipelines that succeed while rows go missing go deeper into the data-loading variants.

When you find a cause, write down the alternatives you ruled out and what ruled them out. “The producer did create invoices: its logs show 4,102 at 01:40” is the sentence that stops the next person re-investigating the same theory. Hypothesis-driven debugging has a template for keeping that record.

Making the next failure visible

Most of the pain in troubleshooting scheduled jobs comes from evidence that was never kept and from jobs that cannot tell success from nothing. A few changes fix most of it, on any platform:

• Log what the job saw and what it did, in the same format every run: the window or watermark read, the count of input items, the count of outputs written. Comparison across runs becomes a text diff.
• Fail on unexpected emptiness. If zero input is abnormal on a weekday, exit non-zero. A loud false alarm is cheaper than a silent week.
• Record the version and parameters of each run in the first log line: image tag or commit, arguments, environment name.
• Make every job idempotent, or protect it with a lock and a run ID. Kubernetes and Cloud Run both document that runs may be retried or duplicated; cron and timers do not stop overlaps by themselves.
• Keep history longer than your detection gap. Raise Kubernetes history limits or ship pod logs centrally; widen SQL Agent history; keep job logs for at least as long as it usually takes someone to notice a problem.
• Version the schedule. Crontabs, unit files, CronJob manifests, Scheduler jobs and Agent job definitions should all be in source control, so “what changed?” has a diff.
• Watch for absence separately. A heartbeat or “dead man’s switch” check, which alerts when an expected success signal does not arrive, catches the “did not start” shape that no job log can.

Where Decim fits

Everything above is the evidence-gathering part of an investigation: the scheduler’s record, the job’s code and its history, the logs of the bad run and a good one, and the ticket where someone reported the empty report. Assembling it is where most of the time goes, and the comparison with a healthy run is the step most often skipped under pressure.

Decim is an investigation tool for your own system. It does not monitor jobs or raise alerts; it starts after someone has noticed something. You describe what happened in plain language, optionally with the affected period, the scope, a Jira ticket (imported as a snapshot with comments and attachments) and files. Decim discovers flows from your GitHub code by their triggers, including scheduled ones defined in the repository, and maps each flow’s steps to the code behind them. It cites the relevant code by line range, reads logs from Google Cloud Logging step by step, compares the failing run with successful runs up to 30 days back, and cites the log lines it relies on. It says where a step looked fine, where it found something, and where the evidence had a gap.

It then works through up to five hypotheses, critiques them and records each revision. When the deciding evidence is somewhere it cannot read, such as a control table, a file share or a scheduler’s own history, it gives your team checks to run: read-only SQL validated against an allowlist, file checks or guided steps. Decim never runs SQL itself. A cause is confirmed only with corroboration; otherwise the outcome says likely cause, cause unclear or needs log access. Fixes come as a patch, a repair script your team runs, or a draft pull request on GitHub, never merged automatically. The rejected hypotheses stay with the evidence that ruled them out, so the team learns how its jobs actually behave.

Today the live integrations are GitHub, Google Cloud Logging and Jira, and more are added as customers need them. A job whose definition lives only in a scheduler outside the repository is not visible to discovery from code; its logs in Cloud Logging and the checks Decim proposes are how that evidence enters the investigation.

Frequently asked questions

Why does my cron job work manually but not from cron?

Almost always the environment. Cron runs commands with /bin/sh, its own short PATH and no shell profile, so commands, credentials and variables available in your interactive shell are missing. Unescaped % characters in the crontab line are converted to newlines. Set SHELL and PATH explicitly in the crontab, call scripts by absolute path, and redirect output to a log file so you can read the error.

How do I know if a cron job ran?

Check the system log for the cron daemon’s record of starting the command: the journal on systemd-based systems, or /var/log/syslog or /var/log/cron elsewhere. That proves the start, not the outcome. For the outcome you need the job’s own output, which cron mails or discards unless you redirect it.

Why did my Kubernetes CronJob not create a Job?

Check, in order: whether the CronJob is suspended; its recent events, including TooManyMissedTimes; whether concurrencyPolicy: Forbid skipped the run because the previous Job was still running; whether startingDeadlineSeconds expired before the controller could start it; and whether timeZone means the schedule fires at a different hour than you expect.

My Cloud Scheduler job shows success but the Cloud Run job failed. How?

The Scheduler job calls the Cloud Run Admin API to start an execution, and its status reflects that call. The execution’s own result is recorded on the execution. Check gcloud run jobs executions list and the cloud_run_job logs for the execution, including task timeouts and retries.

Why does Airflow mark a DAG run as success when a task did not run?

A DAG run’s state is decided by its leaf tasks. If those are success or skipped, the run is success, so a skipped branch reads as green. A leaf task with the all_done trigger rule can also make the run succeed after an upstream failure. Check task instance states, not just the DAG run’s.

How do I stop a scheduled job from running twice?

Make the job idempotent first: give each run an ID, write outputs keyed by it, and make reruns replace rather than append. Then prevent overlap at the scheduler where you can: flock for cron, concurrencyPolicy: Forbid for Kubernetes, max_active_runs for Airflow. A lock alone does not protect against retries of the same run.

Related reading

• Root cause analysis for software and data incidents: the general method this guide applies to scheduled jobs.
• ETL job succeeded but no data loaded: the data-loading variants of “ran and did nothing”.
• Google Cloud Logging debugging queries: comparing two runs in Cloud Logging.
• Hypothesis-driven debugging: keeping a record of the explanations you ruled out.
• Incident postmortem template: writing up what you found.

Decim investigates incidents in your own system and returns a diagnosis where every claim cites its evidence: the line of code, the log line, the commit, the check. It reads code from GitHub, logs from Google Cloud Logging and tickets from Jira, never runs SQL itself and never merges a fix. See how it works at decim.dev, or read about how investigations work.