ETL Job Succeeded but No Data Loaded? 9 Causes and How to Find Yours

The scheduler says green. Airflow marks the task as success, the Cloud Run job execution completed, cron exited with code 0, or SQL Agent wrote “The job succeeded.” The step log ends with “completed successfully.” And the target table has no rows for last night, or a fraction of what it should.

The usual first move is to rerun the job and see if it happens again. That is the wrong first move, because a rerun overwrites the evidence of what went wrong and frequently “fixes” the symptom without touching the cause. When an ETL job succeeded but no data loaded, the job did exactly what it was told. Something in what it was told changed, and your task is to find out what before the next run hides it.

There is a short list of mechanisms that produce this symptom, and each leaves a different trace. This article covers nine of them, with the check that confirms or rules out each. The checks are written as plain SQL in PostgreSQL syntax and work with small changes on most warehouses; the scheduler can be anything. If your problem is that some rows are missing rather than all of them, and one category vanished while others were fine, start with the companion piece, your pipeline succeeded and the rows are missing, which works through the filter-and-parameter case in depth. This one is about the wider family.

Table of Contents

• Before you rerun anything
• Why “succeeded” means so little
• The nine causes at a glance
• Causes on the read side
  • 1. The source really was empty for that window
  • 2. The watermark moved past the data
  • 3. The job read the wrong file
• Causes in the transform
  • 4. A filter or parameter excluded everything
  • 5. An inner join to an empty or stale lookup
  • 6. Rows were skipped or sent to a reject path
• Causes on the write side
  • 7. The write was rolled back or never committed
  • 8. The data went somewhere else
  • 9. Something removed it afterwards
• A triage order that saves time
• Making success mean something
• Frequently asked questions
• The pattern behind all nine
• Where Decim fits
• Related reading

Before you rerun anything

Spend five minutes preserving evidence. Most of what you need to diagnose an empty load disappears on the next run.

1. Copy the run’s logs somewhere they will not rotate: the scheduler’s task output, the application log, the container logs for that execution. Check how long your log store keeps them.
2. Record the control-table state: watermark values, last-processed file names, batch IDs. A rerun will move them.
3. Snapshot staging if it still holds the run’s data. Most staging tables are truncated at the start of the next run.
4. Note the exact input: file names, sizes and modification times in the landing folder or bucket; the source query’s time window.
5. Write down the job definition as it ran, including parameters, environment variables and the deployed version, because someone may “fix” it before you look.

Then decide whether a rerun is safe at all. If the job is not idempotent, and many incremental loads are not, a rerun after a partial write can produce duplicates on top of the missing rows.

Why “succeeded” means so little

A scheduler’s success status means the process exited with a success code, or the task function returned without raising. Airflow, cron, Kubernetes CronJobs, Cloud Run jobs, SQL Agent and every other scheduler work the same way: they record what the process said about itself.

The process, in turn, reports success unless something raised an error that reached the top. A query that returns zero rows raises nothing. An insert of zero rows raises nothing. An except block that logs and continues raises nothing. So “success” is really “nothing threw all the way out,” which is a much weaker claim than “the data arrived.”

One check is worth running before anything else, because it narrows the search immediately: how long did the step take compared with a normal night? A load that usually runs for 20 minutes and finished in 4 seconds had nothing to process, or skipped its work. A load that ran for the usual 20 minutes and still produced nothing did the work and lost the result. That one comparison splits the nine causes roughly in half.

Every scheduler keeps start and end times somewhere: Airflow’s task instance history, gcloud run jobs executions list for Cloud Run jobs, the Job objects a Kubernetes CronJob leaves behind, SQL Agent’s job history. Plain cron keeps nothing, which is a good reason to write your own run log. If you have one, the comparison is one query:

-- Step durations for the last 14 runs of one pipeline, from your own run log.
SELECT run_id,
       step,
       started_at,
       EXTRACT(EPOCH FROM (finished_at - started_at)) AS duration_seconds,
       rows_in,
       rows_out,
       status
FROM   etl.run_log
WHERE  pipeline   = 'nightly_sales'
  AND  started_at >= CURRENT_TIMESTAMP - INTERVAL '14 days'
ORDER  BY step, started_at DESC;

If your pipeline does not write a table like etl.run_log, the logs from each step usually hold the same information in a less convenient shape. The Google Cloud Logging queries guide shows how to pull one step across many runs.

The nine causes at a glance

#	Cause	Step duration	First check
1	Source empty for the window	Short	Count the source for the exact window the job used
2	Watermark moved past the data	Short	Control-table value before and after the run
3	Wrong or empty file read	Short	File name, size and timestamp the job picked up
4	Filter or parameter excluded everything	Normal or short	Parameter values the step ran with; code diff
5	Inner join to empty or stale lookup	Normal	Row count of the lookup table for that date
6	Rows skipped or sent to a reject path	Normal	Reject table, loader error count, skip settings
7	Write rolled back or never committed	Normal	Error handling, commit calls, transaction scope
8	Data written to the wrong place	Normal	Connection settings, schema resolution, target name
9	Something removed it afterwards	Normal	Later steps, cleanup jobs, retention policies

Causes on the read side

1. The source really was empty for that window

Sometimes the honest answer is that there was nothing to load. The source system had an outage, an upstream export failed, or the business genuinely had no activity: a holiday, a closed region. The job read an empty set and wrote an empty set.

The question is whether the source was empty for the window the job asked for, which is not the same as “empty”. Re-run the extract query by hand with the exact parameters the job used, which you recorded before rerunning, and compare it with the same query for the previous day. If the source has data now but didn’t at extract time, you are looking at a late-arriving source, which is a timing problem rather than a code problem. The fix is in scheduling or in a dependency check, not in the transform.

2. The watermark moved past the data

Incremental loads keep a bookmark: the last modified_at, the highest ID, the last change-data-capture position. Each run reads everything after the bookmark and then advances it. If the bookmark advances without the rows being loaded, those rows are skipped for good, and every subsequent run reports success.

Common ways it happens:

• The bookmark is updated before the load commits, and the load then fails or rolls back.
• The bookmark is set to “now” rather than to the maximum value actually read, so rows committed in the source during the extract are skipped.
• Source and pipeline disagree on time zone, so a watermark in UTC is compared with local-time timestamps.
• A source backfill writes old modified_at values that are already behind the bookmark.

-- Compare the watermark the run started from with what it actually loaded.
SELECT r.run_id,
       r.watermark_from,
       r.watermark_to,
       MIN(s.modified_at) AS min_loaded,
       MAX(s.modified_at) AS max_loaded,
       COUNT(s.order_id)  AS rows_loaded
FROM   etl.run_log AS r
LEFT   JOIN stg.orders AS s ON s.run_id = r.run_id
WHERE  r.pipeline   = 'orders_incremental'
  AND  r.started_at >= CURRENT_TIMESTAMP - INTERVAL '7 days'
GROUP  BY r.run_id, r.watermark_from, r.watermark_to
ORDER  BY r.run_id DESC;

If watermark_to jumped forward while rows_loaded was zero, you have found it. The fix is to set the watermark from the data you actually committed, in the same transaction as the load.

3. The job read the wrong file

File-driven pipelines have their own version of this. The job picked up a zero-byte file, a header-only file, yesterday’s file, or a file from a different feed that matched the same wildcard. Or it found no file at all and treated “nothing to do” as success.

Check the file name, size and modification time the job logged against what was in the landing location at the time. If the job does not log which file it read, that is the first improvement to make after the incident. Header-only files deserve a special mention: they are often produced by an upstream export that ran but returned no rows, and they look perfectly valid to a parser. A loader that counts data rows and refuses a file with none would have turned this into a failure on the night.

Causes in the transform

4. A filter or parameter excluded everything

A WHERE clause that excludes every row does not complain. This is the mechanism behind the EUR example: a transform gained a parameter with a default, the scheduled step never passed it, and the default excluded a whole currency. When the default excludes everything, you get an empty table instead of a short one.

Check the parameter values the step actually ran with, not the ones you think it uses. Look at the task’s rendered arguments in your scheduler, the environment variables on the deployed job, and the code that was deployed, compared with the last version that loaded correctly. If the transform lives in the database as a stored procedure or a view, compare the definition in the database with the one in source control, because the two drift. Environment-specific config values, a date parameter computed in the wrong time zone, and a feature flag that switched a branch off all produce the same symptom.

5. An inner join to an empty or stale lookup

Many transforms join the incoming rows to a reference table: currency rates, product master, store list, customer mapping. With an inner join, a row with no match is dropped silently. If the reference table was empty or had not been refreshed for the run date, every row is dropped.

-- How many incoming rows have no match in the lookup the transform joins to?
SELECT COUNT(*)                                     AS incoming,
       COUNT(*) FILTER (WHERE r.rate_date IS NULL)  AS no_rate
FROM   stg.transactions AS t
LEFT   JOIN ref.fx_rates AS r
       ON  r.currency  = t.currency
       AND r.rate_date = CAST(t.posted_at AS date)
WHERE  t.posted_at >= DATE '2026-11-02'
  AND  t.posted_at <  DATE '2026-11-03';

COUNT(*) FILTER (...) is PostgreSQL; on warehouses without it, use SUM(CASE WHEN r.rate_date IS NULL THEN 1 ELSE 0 END). If no_rate equals incoming, the lookup is the culprit, and the real question becomes why the reference load was late or empty. The step duration for this cause is usually normal: the work happened, the join just matched nothing.

6. Rows were skipped or sent to a reject path

Most loaders have a way to keep going when a row is bad. Snowflake’s COPY INTO has an ON_ERROR option whose CONTINUE setting keeps loading the file when errors are found. BigQuery load jobs accept a maxBadRecords setting that lets the job ignore up to that many bad records. Hand-written Python loaders often catch a parse or constraint exception per row and write the row to a reject table. ETL tools such as SSIS can redirect failing rows to an error output instead of failing the component.

Each of these is a reasonable design. Each also means that a format change upstream, such as a new date format, a new code or a column that became nullable, can route every row to the reject path while the job reports success.

Look for tables with reject, error, quarantine or invalid in the name, and count them for the run. Read the loader’s own result, not just the exit code: the number of rows loaded, the number of errors seen, and any warnings about truncation or conversion. Then check the skip setting itself. A tolerance of a few bad records that someone raised to a very large number during an earlier incident is a common find.

Causes on the write side

7. The write was rolled back or never committed

This is the cause that surprises experienced engineers, because the logs often show the insert happening.

In application code, the usual version is a missing commit. Python’s database API specification, PEP 249, says that if a database supports auto-commit, it “must be initially off”, and that closing a connection without committing “will cause an implicit rollback to be performed.” Drivers vary in their defaults, but many follow this. A loader that inserts, logs “inserted 52,000 rows” and exits without calling commit() produces exactly this incident: a log line that says the rows were written, and a table that says they were not.

# The pattern that hides failures: the rows are inserted, logged, and never committed,
# or the error is logged and swallowed.
def load(conn, rows):
    cur = conn.cursor()
    try:
        cur.executemany("INSERT INTO dw.fact_sales VALUES (%s, %s, %s)", rows)
        log.info("inserted %d rows", len(rows))   # true for this transaction, not for the table
        # Missing: conn.commit()
    except Exception as exc:
        log.error("load failed: %s", exc)
        # Missing: raise. Without it, the scheduler sees success.
    finally:
        conn.close()                               # uncommitted work is rolled back here

The same shape exists inside the database. In PostgreSQL’s PL/pgSQL, a block with an EXCEPTION clause that catches an error rolls back the changes made in that block, as the error-trapping documentation describes, and if the handler only raises a notice, the function returns normally. A T-SQL TRY...CATCH that logs the error without rethrowing it behaves the same way. In each case the caller, and the scheduler, see success.

To confirm: search the step’s code for except and catch blocks that do not re-raise, check every code path for a commit, and look at the application’s own error log table for the run window. The error is usually sitting there, recorded and ignored.

8. The data went somewhere else

The load worked. It wrote to a different place from the one you are looking at. Variations:

• Wrong environment: a connection string or secret pointed at staging, a test database or an old replica after an infrastructure change.
• Name resolution: an unqualified INSERT INTO sales resolved to a different schema because the service account’s default schema or PostgreSQL’s search_path changed.
• Wrong partition or table suffix: a date-suffixed table or partition computed from the wrong date or time zone.
• A view in between: the report reads a view that filters by a status or date that the new rows do not meet.
• Reading a replica: the report queries a read replica that is behind the primary.

Query the expected key range across every candidate target. If you find the rows in staging.sales instead of analytics.sales, the cause is configuration, and the diff you need is in deployment settings, not in code.

9. Something removed it afterwards

The load committed, and a later process removed the rows. A truncate-and-load step for the next stage that ran before the previous stage finished; a retention job with an off-by-one date boundary; a dedupe step that treated new rows as duplicates of old ones; a “cleanup” step added during a previous incident and forgotten.

A close relative is the insert that quietly does nothing. PostgreSQL’s INSERT ... ON CONFLICT DO NOTHING, and MERGE statements with no WHEN NOT MATCHED branch, report zero rows inserted without an error when every incoming key already exists, for example because a previous partial run left them there with stale values.

Check what else ran against the target between the load and the moment you looked: every scheduled job touching that table, not just the one that failed. Audit tables, change data capture or table history, where they exist, show deletes directly. If the table has none of these, a run-log row per step with rows in and out would have made this a two-minute check, which is an argument for adding one.

A triage order that saves time

The nine causes are not equally likely, and they are not equally cheap to check. A sensible order:

Order	Check	Rules in or out
1	Step duration against previous successful runs	Short: causes 1 to 4. Normal: causes 5 to 9
2	Counts in the run’s own logs: read, written, rejected	The stage where the count drops
3	Reject and error tables, loader error counts	Cause 6
4	Control table: watermark, last file, batch ID	Causes 2 and 3
5	Diff of code, parameters and config between the last good run and the first bad one	Causes 4 and 8
6	Write path and later processes	Causes 7 and 9

Step five deserves care. Include the job definitions, the transform code wherever it lives (repository, database, orchestrator), and the environment settings, not just the application repository. A changed environment variable leaves no commit.

This is the general method from the root cause analysis guide, applied to one symptom: localise first, compare with a healthy run, and write down which causes you have ruled out and how. The written list of ruled-out causes is the part people skip, and it is the part that saves the next person from checking them again.

Making success mean something

Every cause above is survivable if the pipeline notices on the night it happens. The fix is to make each step assert something about its own output, so that “succeeded” means “succeeded and produced a plausible result.”

# At the end of a load step: fail loudly if the result is implausible.
loaded = cur.rowcount            # read immediately after the INSERT; -1 if the driver cannot tell
conn.commit()

cur.execute("""
    SELECT AVG(rows_out) * 0.5
    FROM   etl.run_log
    WHERE  pipeline = %s AND status = 'ok'
      AND  started_at >= CURRENT_TIMESTAMP - INTERVAL '14 days'
""", ("nightly_sales",))
expected_min = cur.fetchone()[0] or 0

if loaded <= 0 or loaded < expected_min:
    raise RuntimeError(f"nightly_sales loaded {loaded} rows, below half the 14-day average")

Read the count straight after the statement you care about. PEP 249 defines cursor.rowcount as the rows affected by the last execute, and as -1 when it cannot be determined, so a later statement on the same cursor replaces it. If your driver returns -1, count the target for the run’s key range instead.

A few more guards that are cheap to add:

• Zero is never a valid row count for a feed that always has activity. Treat it as a failure, not a quiet night, and let a human override it on known holidays.
• Compare against a trailing baseline, not a fixed number, so volume growth does not cause false alarms.
• Break the count down by one key dimension (region, currency, source system) so a partial loss shows up as well as a total one.
• Write rows in, rows out and rows rejected per step to a run log in the same transaction as the load.
• Re-raise in every handler unless the step genuinely can continue, and make the decision explicit in a comment when it can.

These checks detect. They do not explain. When one fires at 02:15, you still need to work out which of the nine causes it was, which is where the triage order comes back in. For how detection and investigation divide the work, see observability vs root cause analysis.

Frequently asked questions

Why does my ETL job show success when it loaded nothing?

Because the scheduler records the exit status of the process, and a process that read zero rows, inserted zero rows or caught its own error exits normally. Success means nothing raised an error all the way to the top. It says nothing about how many rows arrived.

Should I just rerun the job?

Not first. Preserve the logs, the control-table state, staging and the input files, then check whether the job is idempotent. A rerun can move the watermark, overwrite staging and add duplicates on top of the missing rows, which removes the evidence and creates a second problem.

How do I tell a quiet day from a broken load?

Compare with a trailing baseline for the same weekday and, if possible, a breakdown by one key dimension. A genuine quiet day lowers every segment a little. A broken load usually removes one segment entirely, or all of them at once, and the step duration often changes with it.

The logs say rows were inserted. Why is the table empty?

The insert happened inside a transaction that was never committed or was rolled back. Check for a missing commit, a handler that catches and swallows the error, and connection or pool settings that changed the default commit behaviour. Then check whether the rows went to a different schema, database or environment.

Which cause is most common?

It varies by stack, and there is no reliable public data to quote. That is why the triage order above is arranged by cost rather than by likelihood: the first checks take minutes and rule out several causes at once, whichever one it turns out to be.

The pattern behind all nine

Every cause on this list has the same shape: a step did exactly what its code and configuration told it to do, and what it was told had changed. The source window, the bookmark, the file, the filter, the lookup, the skip setting, the commit, the target, the cleanup. None of them is an error in the sense that a scheduler understands.

So the investigation is always the same comparison: this run against the last one that worked, step by step, until you find the first place where they differ. It is slow by hand because the evidence is spread across a scheduler, a repository, a log store and the database. It is quicker when the evidence is gathered in one place and the comparison is laid out step by step.

Where Decim fits

Decim is a root cause analysis tool for your own system, whatever it runs on. A data load that succeeded with nothing in the table is one kind of incident it investigates; a service that started behaving differently after a deploy, or a scheduled job that started doing less, are others. The method is the same in each case, and it is the method in this article.

You describe the problem in plain language, optionally with the affected period, the scope and a Jira ticket, which Decim snapshots with its comments and attachments. Decim reads your code, commits and pull requests through a GitHub App, finds each flow by its trigger and maps its steps to lines of code. It then reads the Google Cloud Logging sources you name, read-only, step by step, noting what looked fine, what it found and where the logs had gaps, and compares the failing run with successful runs from up to 30 days back. Every finding cites the code by line range and the log lines it relied on.

It weighs up to five hypotheses, critiques them, and revises them in numbered steps. Rejected explanations are kept with the evidence that ruled them out, so the team learns why “the source was empty” or “the lookup was stale” was dropped as well as why the real cause was kept. When the deciding fact is a row count, Decim writes a read-only SQL check for your team to run, validated against an allowlist; it never connects to the database or runs SQL itself. A cause is marked as found only when the evidence corroborates it. Otherwise the outcome says “likely cause”, “cause unclear” or “needs log access”, which is more useful than a confident guess.

A fix comes as a patch, a SQL repair script for the team to review and run, or a draft pull request on GitHub, never merged automatically. Decim is not monitoring and raises no alerts: it starts after someone has noticed. Today it connects to GitHub, Google Cloud Logging and Jira, and integrations expand with what customers need. The investigations page walks through an example, and the evidence page lists what it collects.

Related reading

• Your pipeline succeeded and the rows are missing: the partial-loss case, worked through end to end.
• Scheduled job troubleshooting: when the job itself misbehaves, not just its output.
• Google Cloud Logging queries for debugging production incidents: pulling one step across many runs and diffing two runs.
• Root cause analysis guide: the general method behind the triage order.
• Incident postmortem template: writing the incident up, including the causes you ruled out.

---

Decim helps your team find out why something happened in your own system, from the evidence it already leaves behind, and keeps the reasoning so the next incident starts from what you learned. It reads code from GitHub, logs from Google Cloud Logging and tickets from Jira, read-only, and never runs SQL or merges a fix. Learn more at decim.dev, or see pricing.