Fix infinite loop in ORDER BY + LIMIT queries with Electric sync #1173

KyleAMathews · 2026-01-22T16:47:17Z

Summary

ELI5: Query asks for "top 10 where category='rare'" but only 3 rare items exist locally. System keeps asking "give me more!" but local index has nothing else → infinite loop. Fix: remember when the index is empty and stop asking. Reset when genuinely new data arrives.

Technical: Fixes an infinite loop that occurs when ORDER BY + LIMIT queries can't fill the TopK because WHERE filters out most data, while the sync layer continuously sends updates. Adds localIndexExhausted flag to prevent repeated load attempts, plus safety iteration limits as backstops.

User impact: Prevents app freezes when using live queries with ORDER BY + LIMIT + WHERE combinations that match few items.

Root Cause

The infinite loop occurs in maybeRunGraph's while loop:

Query has ORDER BY + LIMIT (e.g., LIMIT 10) + WHERE that matches few items (e.g., 2 items)
TopK operator needs more data (dataNeeded() returns 8)
loadMoreIfNeeded calls loadNextItems, which exhausts the local index
Sync layer sends an update
Bug 1: splitUpdates converts updates to delete+insert pairs for D2, and the hasInserts check was seeing those fake inserts, resetting localIndexExhausted
Bug 2: Without tracking index exhaustion, loadMoreIfNeeded keeps trying to load from the exhausted index
Loop continues indefinitely

Approach

Primary fix (localIndexExhausted flag in collection-subscriber.ts):

if (this.localIndexExhausted) {
  return true  // Don't try to load more
}

const foundLocalData = this.loadNextItems(n, subscription)
if (!foundLocalData) {
  this.localIndexExhausted = true  // Mark exhausted
}

Secondary fix (check ORIGINAL inserts, not split inserts):

// In sendChangesInRange - check BEFORE splitUpdates
const hasOriginalInserts = changesArray.some((c) => c.type === `insert`)
const splittedChanges = splitUpdates(changesArray)
this.sendChangesToPipelineWithTracking(splittedChanges, subscription, hasOriginalInserts)

This ensures the flag only resets for genuine new inserts from the sync layer, not for updates converted to delete+insert by splitUpdates.

Safety backstops (iteration limits):

D2.run(): 100,000 iterations
maybeRunGraph: 10,000 iterations
requestLimitedSnapshot: 10,000 iterations

Key Invariants

localIndexExhausted must only reset on original inserts, not fake inserts from splitUpdates
requestLimitedSnapshot must return boolean indicating if local data was found
Safety limits must be high enough to never trigger for legitimate workloads

Non-goals

Fixing the underlying TopK "greediness" (it will always want limit items)
Changing how splitUpdates works (it's correct for D2 processing)
Optimizing the number of loadSubset calls (just preventing infinite calls)

Verification

# Run the specific tests
pnpm vitest run packages/db/tests/infinite-loop-prevention.test.ts

# Key test: "should limit requestLimitedSnapshot calls when index is exhausted"
# - Sends 20 updates after initial load
# - With fix: requestLimitedSnapshot called 0-4 times after initial load
# - Without fix (before this PR): called ~19 times (nearly once per update)

Files Changed

File	Changes
`packages/db/src/query/live/collection-subscriber.ts`	Added `localIndexExhausted` flag, check original inserts before `splitUpdates`, pass flag to reset logic
`packages/db/src/collection/subscription.ts`	Changed `requestLimitedSnapshot` to return boolean, added iteration limit
`packages/db/src/query/live/collection-config-builder.ts`	Added iteration limit to `maybeRunGraph`, proper error handling with `transitionToError`
`packages/db-ivm/src/d2.ts`	Added iteration limit to `D2.run()`
`packages/db/tests/infinite-loop-prevention.test.ts`	6 tests including one that counts `requestLimitedSnapshot` calls to verify the fix

Add iteration safeguards to prevent infinite loops that can occur when using Electric with large datasets and ORDER BY/LIMIT queries: 1. `maybeRunGraph` while loop (collection-config-builder.ts): - Can loop infinitely when data loading triggers graph updates - Happens when WHERE filters out most data, causing dataNeeded() > 0 - Loading more data triggers updates that get filtered out - Added 10,000 iteration limit with error logging 2. `requestLimitedSnapshot` while loop (subscription.ts): - Can loop if index iteration has issues - Added 10,000 iteration limit with error logging - Removed unused `insertedKeys` tracking 3. `D2.run()` while loop (d2.ts): - Can loop infinitely on circular data flow bugs - Added 100,000 iteration limit with error logging The safeguards log errors to help debug the root cause while preventing the app from freezing.

changeset-bot · 2026-01-22T16:47:22Z

🦋 Changeset detected

Latest commit: 3a5f05d

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 13 packages

Name	Type
@tanstack/db	Patch
@tanstack/db-ivm	Patch
@tanstack/angular-db	Patch
@tanstack/electric-db-collection	Patch
@tanstack/offline-transactions	Patch
@tanstack/powersync-db-collection	Patch
@tanstack/query-db-collection	Patch
@tanstack/react-db	Patch
@tanstack/rxdb-db-collection	Patch
@tanstack/solid-db	Patch
@tanstack/svelte-db	Patch
@tanstack/trailbase-db-collection	Patch
@tanstack/vue-db	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

pkg-pr-new · 2026-01-22T16:49:04Z

More templates

@tanstack/angular-db

npm i https://pkg.pr.new/@tanstack/angular-db@1173

@tanstack/db

npm i https://pkg.pr.new/@tanstack/db@1173

@tanstack/db-ivm

npm i https://pkg.pr.new/@tanstack/db-ivm@1173

@tanstack/electric-db-collection

npm i https://pkg.pr.new/@tanstack/electric-db-collection@1173

@tanstack/offline-transactions

npm i https://pkg.pr.new/@tanstack/offline-transactions@1173

@tanstack/powersync-db-collection

npm i https://pkg.pr.new/@tanstack/powersync-db-collection@1173

@tanstack/query-db-collection

npm i https://pkg.pr.new/@tanstack/query-db-collection@1173

@tanstack/react-db

npm i https://pkg.pr.new/@tanstack/react-db@1173

@tanstack/rxdb-db-collection

npm i https://pkg.pr.new/@tanstack/rxdb-db-collection@1173

@tanstack/solid-db

npm i https://pkg.pr.new/@tanstack/solid-db@1173

@tanstack/svelte-db

npm i https://pkg.pr.new/@tanstack/svelte-db@1173

@tanstack/trailbase-db-collection

npm i https://pkg.pr.new/@tanstack/trailbase-db-collection@1173

@tanstack/vue-db

npm i https://pkg.pr.new/@tanstack/vue-db@1173

commit: 3a5f05d

github-actions · 2026-01-22T16:49:42Z

Size Change: +451 B (+0.5%)

Total Size: 91.4 kB

Filename	Size	Change
`./packages/db/dist/esm/collection/subscription.js`	3.8 kB	+179 B (+4.94%)	🔍
`./packages/db/dist/esm/query/live/collection-config-builder.js`	5.56 kB	+146 B (+2.7%)
`./packages/db/dist/esm/query/live/collection-subscriber.js`	2.06 kB	+126 B (+6.52%)	🔍

ℹ️ View Unchanged

Filename	Size
`./packages/db/dist/esm/collection/change-events.js`	1.39 kB
`./packages/db/dist/esm/collection/changes.js`	1.19 kB
`./packages/db/dist/esm/collection/events.js`	388 B
`./packages/db/dist/esm/collection/index.js`	3.32 kB
`./packages/db/dist/esm/collection/indexes.js`	1.1 kB
`./packages/db/dist/esm/collection/lifecycle.js`	1.68 kB
`./packages/db/dist/esm/collection/mutations.js`	2.34 kB
`./packages/db/dist/esm/collection/state.js`	3.49 kB
`./packages/db/dist/esm/collection/sync.js`	2.41 kB
`./packages/db/dist/esm/deferred.js`	207 B
`./packages/db/dist/esm/errors.js`	4.7 kB
`./packages/db/dist/esm/event-emitter.js`	748 B
`./packages/db/dist/esm/index.js`	2.69 kB
`./packages/db/dist/esm/indexes/auto-index.js`	742 B
`./packages/db/dist/esm/indexes/base-index.js`	766 B
`./packages/db/dist/esm/indexes/btree-index.js`	1.93 kB
`./packages/db/dist/esm/indexes/lazy-index.js`	1.1 kB
`./packages/db/dist/esm/indexes/reverse-index.js`	513 B
`./packages/db/dist/esm/local-only.js`	837 B
`./packages/db/dist/esm/local-storage.js`	2.1 kB
`./packages/db/dist/esm/optimistic-action.js`	359 B
`./packages/db/dist/esm/paced-mutations.js`	496 B
`./packages/db/dist/esm/proxy.js`	3.75 kB
`./packages/db/dist/esm/query/builder/functions.js`	733 B
`./packages/db/dist/esm/query/builder/index.js`	4.08 kB
`./packages/db/dist/esm/query/builder/ref-proxy.js`	1.05 kB
`./packages/db/dist/esm/query/compiler/evaluators.js`	1.42 kB
`./packages/db/dist/esm/query/compiler/expressions.js`	430 B
`./packages/db/dist/esm/query/compiler/group-by.js`	1.81 kB
`./packages/db/dist/esm/query/compiler/index.js`	2.02 kB
`./packages/db/dist/esm/query/compiler/joins.js`	2.07 kB
`./packages/db/dist/esm/query/compiler/order-by.js`	1.45 kB
`./packages/db/dist/esm/query/compiler/select.js`	1.06 kB
`./packages/db/dist/esm/query/expression-helpers.js`	1.43 kB
`./packages/db/dist/esm/query/ir.js`	673 B
`./packages/db/dist/esm/query/live-query-collection.js`	360 B
`./packages/db/dist/esm/query/live/collection-registry.js`	264 B
`./packages/db/dist/esm/query/live/internal.js`	145 B
`./packages/db/dist/esm/query/optimizer.js`	2.56 kB
`./packages/db/dist/esm/query/predicate-utils.js`	2.97 kB
`./packages/db/dist/esm/query/subset-dedupe.js`	921 B
`./packages/db/dist/esm/scheduler.js`	1.3 kB
`./packages/db/dist/esm/SortedMap.js`	1.3 kB
`./packages/db/dist/esm/strategies/debounceStrategy.js`	247 B
`./packages/db/dist/esm/strategies/queueStrategy.js`	428 B
`./packages/db/dist/esm/strategies/throttleStrategy.js`	246 B
`./packages/db/dist/esm/transactions.js`	2.9 kB
`./packages/db/dist/esm/utils.js`	924 B
`./packages/db/dist/esm/utils/browser-polyfills.js`	304 B
`./packages/db/dist/esm/utils/btree.js`	5.61 kB
`./packages/db/dist/esm/utils/comparison.js`	852 B
`./packages/db/dist/esm/utils/cursor.js`	457 B
`./packages/db/dist/esm/utils/index-optimization.js`	1.51 kB
`./packages/db/dist/esm/utils/type-guards.js`	157 B

_{compressed-size-action::db-package-size}

github-actions · 2026-01-22T16:50:19Z

Size Change: 0 B

Total Size: 3.7 kB

ℹ️ View Unchanged

Filename	Size
`./packages/react-db/dist/esm/index.js`	225 B
`./packages/react-db/dist/esm/useLiveInfiniteQuery.js`	1.17 kB
`./packages/react-db/dist/esm/useLiveQuery.js`	1.34 kB
`./packages/react-db/dist/esm/useLiveSuspenseQuery.js`	559 B
`./packages/react-db/dist/esm/usePacedMutations.js`	401 B

_{compressed-size-action::react-db-package-size}

The infinite loop occurred because `loadMoreIfNeeded` kept trying to load data even when the local index was exhausted. This happened when: 1. TopK had fewer items than limit (WHERE filtered out data) 2. loadMoreIfNeeded tried to load more → no local data found 3. Loop continued indefinitely since TopK still needed data Root cause fix: - Add `localIndexExhausted` flag to CollectionSubscriber - Track when local index has no more data for current cursor - Stop calling loadMoreIfNeeded when exhausted - Reset flag when new data arrives from sync layer (inserts) - requestLimitedSnapshot now returns boolean indicating if data was found Error handling improvements (per review feedback): - D2.run() now throws Error when iteration limit exceeded - Caller catches and calls transitionToError() for proper error state - requestLimitedSnapshot returns false when iteration limit hit - Live query properly shows error state if safeguard limits are hit This fixes the issue for both eager and progressive syncMode.

These tests verify the localIndexExhausted fix works correctly: 1. Does not infinite loop when WHERE filters out most data - Query wants 10 items, only 2 match - Verifies status !== 'error' (fix works, not just safeguard) 2. Resumes loading when new matching data arrives - Starts with 0 matching items - Insert new matching items - localIndexExhausted resets, loads new data 3. Handles updates that move items out of WHERE clause - Updates change values to no longer match WHERE - TopK correctly refills from remaining matching data

Adds a new test that directly reproduces the infinite loop bug by creating a collection with a custom loadSubset that synchronously injects updates matching the WHERE clause. This simulates Electric's behavior of sending continuous updates during graph execution. The test verifies: - Without the localIndexExhausted fix, loadSubset is called 100+ times (infinite loop) - With the fix, loadSubset is called < 10 times (loop terminates correctly) Also adds additional tests for edge cases around the localIndexExhausted flag. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

The removed test synchronously injected data in loadSubset, which artificially creates an infinite loop. Electric's loadSubset is async (uses await stream.fetchSnapshot), so it can't synchronously inject data during the maybeRunGraph loop. The remaining tests verify the localIndexExhausted flag's behavior correctly: - Prevents repeated load attempts when exhausted - Resets when new inserts arrive Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

The localIndexExhausted flag was incorrectly resetting when updates arrived because splitUpdates converts updates to delete+insert pairs for D2, and the hasInserts check was seeing those fake inserts. Fix: Check for ORIGINAL inserts before calling splitUpdates, and pass that information to sendChangesToPipelineWithTracking. Now the flag only resets for genuine new inserts from the sync layer. Also adds a test that verifies requestLimitedSnapshot calls are limited after the index is exhausted - with the fix, only 0-4 calls happen after initial load even when 20 updates arrive. Without the fix, it would be ~19 calls. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Documents the reasoning for external reviewers: 1. splitUpdates fake inserts don't reset the flag 2. Updates to existing rows don't add new rows to scan 3. Edge case "update makes row match WHERE" is handled by filterAndFlipChanges converting unseen key updates to inserts Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

KyleAMathews force-pushed the claude/debug-electric-infinite-loop-QJD2Z branch from f47fbd0 to 2a796ff Compare January 22, 2026 17:07

KyleAMathews force-pushed the claude/debug-electric-infinite-loop-QJD2Z branch from 453e7c7 to e698eb1 Compare January 22, 2026 17:36

KyleAMathews and others added 2 commits January 22, 2026 10:52

ci: apply automated fixes

d43a509

KyleAMathews requested review from kevin-dp and samwillis January 22, 2026 17:56

KyleAMathews changed the title ~~Add iteration limits to prevent infinite loops in graph execution~~ Fix infinite loop in ORDER BY + LIMIT queries with Electric sync Jan 22, 2026

KyleAMathews and others added 5 commits January 22, 2026 11:01

Add changeset for infinite loop fix

e6a9fbc

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Update changeset with clearer ELI5 explanation

3a5f05d

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix infinite loop in ORDER BY + LIMIT queries with Electric sync #1173

Fix infinite loop in ORDER BY + LIMIT queries with Electric sync #1173

Uh oh!

KyleAMathews commented Jan 22, 2026 •

edited

Loading

Uh oh!

changeset-bot bot commented Jan 22, 2026 •

edited

Loading

Uh oh!

pkg-pr-new bot commented Jan 22, 2026 •

edited

Loading

Uh oh!

github-actions bot commented Jan 22, 2026 •

edited

Loading

Uh oh!

github-actions bot commented Jan 22, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Fix infinite loop in ORDER BY + LIMIT queries with Electric sync #1173

Are you sure you want to change the base?

Fix infinite loop in ORDER BY + LIMIT queries with Electric sync #1173

Uh oh!

Conversation

KyleAMathews commented Jan 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Root Cause

Approach

Key Invariants

Non-goals

Verification

Files Changed

Uh oh!

changeset-bot bot commented Jan 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🦋 Changeset detected

Uh oh!

pkg-pr-new bot commented Jan 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Jan 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Jan 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

KyleAMathews commented Jan 22, 2026 •

edited

Loading

changeset-bot bot commented Jan 22, 2026 •

edited

Loading

pkg-pr-new bot commented Jan 22, 2026 •

edited

Loading

github-actions bot commented Jan 22, 2026 •

edited

Loading

github-actions bot commented Jan 22, 2026 •

edited

Loading