diff --git a/.changeset/local-query-observation.md b/.changeset/local-query-observation.md new file mode 100644 index 0000000000..f8d40e5574 --- /dev/null +++ b/.changeset/local-query-observation.md @@ -0,0 +1,6 @@ +--- +'@tanstack/db': minor +'@tanstack/react-db': minor +--- + +Add `autoLoad: false` to live-query configuration and React query hooks to observe available rows without initiating source loading. Queries remain reactive across joins and pagination, become ready independently of provider loading, and have separate identities from queries that load data automatically. diff --git a/docs/contributing/oracle-coverage.md b/docs/contributing/oracle-coverage.md index bbe651d368..afb047d6d2 100644 --- a/docs/contributing/oracle-coverage.md +++ b/docs/contributing/oracle-coverage.md @@ -70,6 +70,7 @@ that test identifiers must copy production's private data structures. | D2 Index storage | Complete for bounded prefix grammar | The contract, plain-`Map` multiset model, prefixed and unprefixed value grammar, `Index` driver, and per-addition `get`/`has` check are literate. Compaction, presence tracking, and structural payloads remain outside this owner. | | Pooled live queries | Complete for bounded eq-filter grammar | The contract, independent `eq` model, live-query Collection second formulation, sync/optimistic/mount/cleanup-restart grammar, observer driver, and per-step refinement check are literate. On-demand and persisted sources, `DbClient`, Suspense, and every clause beyond `eq` conjuncts keep the live-query Collection. | | Live-query deferred acquisition | Complete for bounded synchronous grammar | The law, independent model, history grammar over source states, shapes, and depth, real-Collection driver, and per-step refinement check are literate. Pooled views and DbClient stream preloads have their own blocks. The grammar also adds a peer consumer starting the source and a source truncate before the first subscriber; pinned blocks cover a subscriber that outlives cleanup, reads that wait for readiness, and release of two independent descriptor sources when one start fails. Asynchronous sources, several live-query consumers, and general failure histories stay open. | +| Live queries with automatic loading disabled | [local evaluation oracle](https://github.com/TanStack/db/blob/main/packages/db/tests/live-query-auto-load-oracle.test.ts), [React boundary tests](https://github.com/TanStack/db/blob/main/packages/react-db/tests/live-query-auto-load.test.tsx), [Query Collection ingestion](https://github.com/TanStack/db/blob/main/packages/query-db-collection/tests/live-query-auto-load.test.ts) | Available-row results and zero acquisition across windows, joins, includes, and lifecycle changes. Covers eager/on-demand sources and explicit index presence/absence. React checks identity isolation, mode changes, and pagination; Query Collection checks direct ingestion without fetching. Excludes remote provider protocols, arbitrary nested plans, and other frameworks. | | Flat-row change tracking | Complete for bounded flat-row grammar | Flat and proxy trackers and an independent change model run the same generated callbacks. Nested values fall back to the proxy, which its own oracles own. | | Production error messages | `packages/db/tests/production-error-messages-oracle.test.ts` | Every public error class keeps its development message for frozen sample inputs, and its coded production line shows the JSON of each shown input, even for hostile inputs. Every plain `Error`, `TypeError`, `RangeError`, or `AggregateError` site in `packages/db/src` keeps its frozen development template, has a code, and passes each showable interpolated expression or one of its sub-expressions; the TypeScript checker keeps plain objects, such as rows, out of production lines. A projection of an object, such as `Object.keys(handlers)`, is not required. Every `console` call whose argument holds library text is coded, development-only with its text frozen per guarded region, or a development-only decoration of a value. Text that reaches the console through a parameter, such as `setErrorState(message)`, is coded where it is written; the census cannot follow it. `scripts/test-production-errors.mjs` checks that a production bundle drops the full text and every development-only hint, in ESM and CommonJS. Errors built from a caller's value stay outside this owner. | | Lazy target path identity | Focused compiler boundary | A same-source union/coalesce witness keeps dotted and nested demand paths distinct during target deduplication. | diff --git a/docs/contributing/oracle-reviews/2026-10-14-local-live-query.md b/docs/contributing/oracle-reviews/2026-10-14-local-live-query.md new file mode 100644 index 0000000000..92fabc1df0 --- /dev/null +++ b/docs/contributing/oracle-reviews/2026-10-14-local-live-query.md @@ -0,0 +1,82 @@ +# Live-query automatic-loading review + +Reviewed implementation: `df2518b61963034b6873fac25fcb3c5a4a5a0194`. +Independent review found no actionable issues within the scope below. + +## Why this design + +`autoLoad: false` queries data already in collections and keeps results reactive. +Application-owned fetching is one use case: some remote API requests cannot be +derived from the query IR, so the application constructs the request and ingests +its results. The contract is defined in the architecture's +[Automatic loading](../../../packages/db/src/query/live/ARCHITECTURE.md#automatic-loading) +section. + +Suppressing `loadSubset` alone is insufficient because subscriptions can start +source sync, which can fetch independently. The implementation suppresses both +and determines readiness after evaluating available rows, without depending on +source loading. Other consumers can still load and update the shared rows. + +Filtering, joins, includes, and pagination use the ordinary query engine. +Subscriptions skip the provider-loading callbacks, and ordered windows evaluate +the available rows. This avoids a separate implementation of query evaluation. + +Loading mode is part of query identity, including explicit keys, because queries +that request data and queries that only read available rows have different +readiness behavior. The discriminator sits outside the user-supplied key array +to prevent collisions. The default remains `true`, with unchanged default hashes. + +## Evidence and limits + +The [core oracle](../../../packages/db/tests/live-query-auto-load-oracle.test.ts) +compares real Collection results with independent array calculations. +[React tests](../../../packages/react-db/tests/live-query-auto-load.test.tsx) +cover loading-mode changes, identity isolation, and pagination. The +[Query Collection integration](../../../packages/query-db-collection/tests/live-query-auto-load.test.ts) +checks application-owned ingestion without invoking the collection fetcher. +The requirement numbers below refer to the [oracle guide](../oracle-tests.md). +Additional review probes passed for callback configurations with legacy +dependencies, DbClient preload isolation under the same explicit key, and a +disabled outer query over an inner live-query Collection. + +- ORC-001–003: the proposed contract, authority, model, production path, and + limits are stated. Array filter/join/sort/slice calculations preserve selected + values, duplicates, and order independently of incremental execution. +- ORC-004–005: generated replacements and windows cover eager/on-demand sources, + windows, joins, and includes, with and without explicitly created indexes. + Index presence is asserted. Checks cover subscription, preload, truncate, + restart, readiness, and zero acquisition; pinned cases cover incremental + updates, provider failure, and coexistence with a fetching consumer. +- ORC-006: enabling acquisition in the source subscription makes the replay + below fail at the source-start assertion. Restoring `autoLoad: false` passes. +- ORC-007: all twelve mode/index/shape combinations run fixed and random + campaigns with ten cases each, scaled by the standard multiplier. Seed/path + replay is supported; command-model replay does not apply. +- ORC-008–010: the reference calculation is stateless, so additional model-state + obligations do not apply. Cleanup preserves primary failures, and the reduced + failing input retains the source-start assertion. +- ORC-011–012: no shared-fault hypothesis requiring another relational + formulation is claimed. This record identifies the reviewed implementation + and evidence limits; it does not claim general bug-class closure. +- ORC-013–014: empty/nonempty inputs and zero/nonzero windows distinguish + available-row readiness from loading. The fetching-peer and Query Collection + cases check shared ownership and direct ingestion. Arbitrary nested plans, + remote provider protocols, and other frameworks remain outside this coverage. + +## Validation + +The automatic-loading oracle passes 27 tests. Focused identity/preload checks +pass 33 core tests; the full React suite passes 410 tests with no type errors. +Scoped lint and formatting pass. A temporary merge with upstream +`3c481f6c0455fbfe82ea94d9cf457cf00b20d3e7` passes 257 core tests and 31 React +tests with no type errors. + +The full workspace run stopped on four SQLite CLI runtime timeouts. All four +cases pass in isolation, but the full suite has not completed successfully. +A separate ordered-window timeout also reproduces on unchanged upstream code. + +Run the calibrated replay from `packages/db`: + +```sh +TANSTACK_DB_ORACLE_SEED=112919 TANSTACK_DB_ORACLE_PATH=0:0:0:0:0 TANSTACK_DB_ORACLE_PROPERTY=live-query-auto-load.on-demand.false.window pnpm exec vitest run --coverage.enabled=false tests/live-query-auto-load-oracle.test.ts +``` diff --git a/docs/guides/live-queries.md b/docs/guides/live-queries.md index 8b1ea145c8..caf47c473e 100644 --- a/docs/guides/live-queries.md +++ b/docs/guides/live-queries.md @@ -158,6 +158,57 @@ Use `queryOnce` for scripts, background tasks, data export, or AI/LLM context building. `findOne()` resolves to `undefined` when no rows match. For UI bindings and reactive updates, use live queries instead. +### Controlling automatic loading + +Set `autoLoad: false` to query data already in your collections without loading +more data. Results stay reactive, including for joins, includes, and pagination. +The default is `true`. + +One use case is application-owned fetching when a remote API request cannot be +derived from the query IR. This example ingests custom API results into a Query +Collection named `tasks`, configured with `syncMode: 'on-demand'` so explicit +startup does not trigger an eager fetch: + +```tsx +function useApiTasks(projectId: string) { + const request = useQuery({ + queryKey: ['api-tasks', projectId], + queryFn: async () => { + const rows = await fetchProjectTasks(projectId) + tasks.startSyncImmediate() + tasks.utils.writeUpsert(rows) + return rows + }, + }) + const result = useLiveQuery({ + autoLoad: false, + query: (q) => + q.from({ task: tasks }).where(({ task }) => eq(task.projectId, projectId)), + }) + return { ...result, isFetching: request.isFetching, fetchError: request.error } +} +``` + +The same option works with `createLiveQueryCollection({ query, autoLoad: false })` +and `useLiveInfiniteQuery(query, { pageSize: 20, autoLoad: false })`. +For an existing live-query Collection, configure `autoLoad` when creating it. + +`isReady` means the available rows have been evaluated, even if empty. It does +not indicate whether all remote data has been loaded. Source loading errors do +not fail this query; query evaluation errors and explicit source cleanup do. +Infinite queries expand the window over available rows, and `hasNextPage` +describes those rows. Track remote pagination separately in your API fetcher. + +Other consumers and explicit source startup can still load shared rows. A source +that is itself a live-query Collection supplies its published rows under its own +loading policy. This option does not disable loading globally. + +Queries with `autoLoad: false` have separate identities from queries using +`true` or the default, even with the same explicit `queryKey`. Switching between +these modes replaces the query and resets infinite-query pages. If a callback +returns the configuration and uses a legacy dependency array, include `autoLoad` +in those dependencies. + ### Using with Frameworks In React, you can use the `useLiveQuery` hook: diff --git a/packages/db/package.json b/packages/db/package.json index c88a107617..43632720cc 100644 --- a/packages/db/package.json +++ b/packages/db/package.json @@ -23,7 +23,7 @@ "test": "vitest --run", "test:dist": "vitest run --config vitest.dist.config.ts", "test:facade-retention": "node --expose-gc --import tsx tests/facade-retention.probe.ts", - "test:oracles": "vitest --run --coverage.enabled=false tests/reverse-index-oracle.property.test.ts tests/query/compiler/in-evaluator-oracle.property.test.ts tests/cleanup-queue-oracle.property.test.ts tests/comparison-oracle.property.test.ts tests/duration-query-equality-oracle.test.ts tests/temporal-index-order-oracle.test.ts tests/cursor-oracle.property.test.ts tests/index-update-oracle.property.test.ts tests/utils-oracle.property.test.ts tests/change-event-history-oracle.test.ts tests/index-suggestion-oracle.test.ts tests/btree-map-oracle.test.ts tests/paced-mutations-oracle.test.ts tests/db-client-hydration-authority-oracle.test.ts tests/local-storage-order-oracle.test.ts tests/local-storage-peer-oracle.test.ts tests/collection-mutation-startup-oracle.test.ts tests/collection-cleanup-restart-oracle.test.ts tests/effect-disposal-oracle.test.ts tests/optimistic-transaction-oracle.property.test.ts tests/transaction-ownership-oracle.property.test.ts tests/optimistic-settlement-boundaries-oracle.test.ts tests/optimistic-history-publication-oracle.test.ts tests/optimistic-history-outcomes-oracle.test.ts tests/collection-metadata-publication-oracle.property.test.ts tests/collection-index-value-oracle.test.ts tests/collection-state-retention-oracle.property.test.ts tests/collection-subscription-lifecycle-history-oracle.property.test.ts tests/collection-subscription-lifecycle-oracle.test.ts tests/collection-subscription-reentrancy-oracle.test.ts tests/collection-subscription-lifecycle-publication-oracle.property.test.ts tests/collection-subscription-replay-oracle.property.test.ts tests/d2-source-reconciliation-oracle.property.test.ts tests/live-query-observer-history-oracle.property.test.ts tests/live-query-observer-snapshot-oracle.property.test.ts tests/query/cold-join-reconciliation-oracle.test.ts tests/query/identity-output-shape-oracle.test.ts tests/query/optimizer-semantics-oracle.test.ts tests/query/index-path-collision-oracle.test.ts tests/query/custom-index-order-selection-oracle.test.ts tests/query/mixed-index-order-oracle.test.ts tests/query/includes-collection-oracle.property.test.ts tests/query/includes-functional-projection-oracle.test.ts tests/query/includes-functional-input-boundary.test.ts tests/query/includes-context-transport-oracle.test.ts tests/query/includes-cross-formulation-oracle.property.test.ts tests/query/includes-optimistic-oracle.property.test.ts tests/query/includes-oracle.property.test.ts tests/query/includes-alias-shadowing-oracle.test.ts tests/query/includes-outer-join-route-work-oracle.test.ts tests/query/includes-publication-oracle.test.ts tests/query/includes-query-shape-oracle.test.ts tests/query/subquery-user-value-oracle.test.ts tests/query/includes-temporal-oracle.test.ts tests/query/includes-work-counter-oracle.test.ts tests/query/load-subset-oracle.property.test.ts tests/query/load-subset-replay-refinement-oracle.test.ts tests/query/load-subset-source-readiness-refinement-oracle.test.ts tests/query/load-subset-transaction-refinement-oracle.test.ts tests/query/ordered-source-loader-state-oracle.test.ts tests/query/ordered-demand-retirement.test.ts tests/query/ordered-default-work.test.ts tests/query/ordered-lifecycle-oracle.property.test.ts tests/query/ordered-work-oracle.property.test.ts tests/query/pagination-oracle.property.test.ts tests/query/includes-space-oracle.test.ts tests/query/virtual-row-fields-oracle.test.ts tests/query/where-predicate-publication-oracle.property.test.ts tests/query/join-result-key-oracle.property.test.ts tests/query/pooled-live-query-oracle.property.test.ts tests/live-query-deferred-acquisition-oracle.test.ts tests/flat-change-tracking-oracle.property.test.ts tests/truncate-readiness-oracle.test.ts tests/proxy-detachment-contract-oracle.test.ts tests/production-error-messages-oracle.test.ts tests/query/bucket-facade-rollback-oracle.property.test.ts tests/query/aggregate-select-oracle.property.test.ts", + "test:oracles": "vitest --run --coverage.enabled=false tests/reverse-index-oracle.property.test.ts tests/query/compiler/in-evaluator-oracle.property.test.ts tests/cleanup-queue-oracle.property.test.ts tests/comparison-oracle.property.test.ts tests/duration-query-equality-oracle.test.ts tests/temporal-index-order-oracle.test.ts tests/cursor-oracle.property.test.ts tests/index-update-oracle.property.test.ts tests/utils-oracle.property.test.ts tests/change-event-history-oracle.test.ts tests/index-suggestion-oracle.test.ts tests/btree-map-oracle.test.ts tests/paced-mutations-oracle.test.ts tests/db-client-hydration-authority-oracle.test.ts tests/local-storage-order-oracle.test.ts tests/local-storage-peer-oracle.test.ts tests/collection-mutation-startup-oracle.test.ts tests/collection-cleanup-restart-oracle.test.ts tests/effect-disposal-oracle.test.ts tests/optimistic-transaction-oracle.property.test.ts tests/transaction-ownership-oracle.property.test.ts tests/optimistic-settlement-boundaries-oracle.test.ts tests/optimistic-history-publication-oracle.test.ts tests/optimistic-history-outcomes-oracle.test.ts tests/collection-metadata-publication-oracle.property.test.ts tests/collection-index-value-oracle.test.ts tests/collection-state-retention-oracle.property.test.ts tests/collection-subscription-lifecycle-history-oracle.property.test.ts tests/collection-subscription-lifecycle-oracle.test.ts tests/collection-subscription-reentrancy-oracle.test.ts tests/collection-subscription-lifecycle-publication-oracle.property.test.ts tests/collection-subscription-replay-oracle.property.test.ts tests/d2-source-reconciliation-oracle.property.test.ts tests/live-query-observer-history-oracle.property.test.ts tests/live-query-observer-snapshot-oracle.property.test.ts tests/query/cold-join-reconciliation-oracle.test.ts tests/query/identity-output-shape-oracle.test.ts tests/query/optimizer-semantics-oracle.test.ts tests/query/index-path-collision-oracle.test.ts tests/query/custom-index-order-selection-oracle.test.ts tests/query/mixed-index-order-oracle.test.ts tests/query/includes-collection-oracle.property.test.ts tests/query/includes-functional-projection-oracle.test.ts tests/query/includes-functional-input-boundary.test.ts tests/query/includes-context-transport-oracle.test.ts tests/query/includes-cross-formulation-oracle.property.test.ts tests/query/includes-optimistic-oracle.property.test.ts tests/query/includes-oracle.property.test.ts tests/query/includes-alias-shadowing-oracle.test.ts tests/query/includes-outer-join-route-work-oracle.test.ts tests/query/includes-publication-oracle.test.ts tests/query/includes-query-shape-oracle.test.ts tests/query/subquery-user-value-oracle.test.ts tests/query/includes-temporal-oracle.test.ts tests/query/includes-work-counter-oracle.test.ts tests/query/load-subset-oracle.property.test.ts tests/query/load-subset-replay-refinement-oracle.test.ts tests/query/load-subset-source-readiness-refinement-oracle.test.ts tests/query/load-subset-transaction-refinement-oracle.test.ts tests/query/ordered-source-loader-state-oracle.test.ts tests/query/ordered-demand-retirement.test.ts tests/query/ordered-default-work.test.ts tests/query/ordered-lifecycle-oracle.property.test.ts tests/query/ordered-work-oracle.property.test.ts tests/query/pagination-oracle.property.test.ts tests/query/includes-space-oracle.test.ts tests/query/virtual-row-fields-oracle.test.ts tests/query/where-predicate-publication-oracle.property.test.ts tests/query/join-result-key-oracle.property.test.ts tests/query/pooled-live-query-oracle.property.test.ts tests/live-query-deferred-acquisition-oracle.test.ts tests/live-query-auto-load-oracle.test.ts tests/flat-change-tracking-oracle.property.test.ts tests/truncate-readiness-oracle.test.ts tests/proxy-detachment-contract-oracle.test.ts tests/production-error-messages-oracle.test.ts tests/query/bucket-facade-rollback-oracle.property.test.ts tests/query/aggregate-select-oracle.property.test.ts", "bench:nested-includes": "vitest bench tests/query/includes-performance.bench.ts --run", "test:indexed-db:package": "pnpm --filter @tanstack/db... build && vitest run --config vitest.indexed-db.package.config.ts", "test:indexed-db:browser": "playwright test --config playwright.indexed-db.config.ts", diff --git a/packages/db/src/collection/changes.ts b/packages/db/src/collection/changes.ts index bed65f1ae5..6ff1443603 100644 --- a/packages/db/src/collection/changes.ts +++ b/packages/db/src/collection/changes.ts @@ -321,7 +321,8 @@ export class CollectionChangesManager< // Acquire ownership only after all fallible option validation and // user-provided predicate compilation has completed. - const defersAcquisition = opts.deferAcquisition === true + const defersAcquisition = + opts.autoLoad === false || opts.deferAcquisition === true this.addSubscriber(defersAcquisition) let subscription: CollectionSubscription | undefined diff --git a/packages/db/src/collection/subscription.ts b/packages/db/src/collection/subscription.ts index 76fbbad68a..02cdaf12a3 100644 --- a/packages/db/src/collection/subscription.ts +++ b/packages/db/src/collection/subscription.ts @@ -85,6 +85,7 @@ type CollectionSubscriptionOptions = { * subscriber or a preload in its current sync run. */ deferAcquisition?: boolean + autoLoad?: boolean /** Called once when deferred acquisition resumes. */ onResumeAcquisition?: () => void } @@ -263,6 +264,7 @@ export class CollectionSubscription this.collectionRestartCleanup = this.collection.on( `status:change`, ({ status }) => { + if (this.options.autoLoad === false) return if (status !== `loading` && status !== `ready`) return const syncRunGeneration = this.collection._sync.getSyncRunGeneration() const replayState = this.truncateReplayState @@ -293,7 +295,12 @@ export class CollectionSubscription * the on-demand demand that stayed detached while acquisition was deferred. */ public resumeDeferredAcquisition(): void { - if (!this.defersAcquisition || this.unsubscribed) return + if ( + this.options.autoLoad === false || + !this.defersAcquisition || + this.unsubscribed + ) + return // Start the source first: if its start throws, this subscription still // defers rather than claiming acquisition it never began. this.options.onResumeAcquisition?.() @@ -1035,6 +1042,13 @@ export class CollectionSubscription }, acquisitionState: `starting`, } + if (this.options.autoLoad === false) { + // Retain snapshot selection for release/replay bookkeeping, but never + // establish an acquisition lease or a wait for provider readiness. + demand.acquisitionState = `detached` + this.subsetDemands.push(demand) + return { demand, result: true, started: false } + } if ( this.collection.status === `cleaned-up` || // A deferring subscription starts no acquisition attempt until it resumes. diff --git a/packages/db/src/live-query-options.ts b/packages/db/src/live-query-options.ts index c168270f1f..543c412eba 100644 --- a/packages/db/src/live-query-options.ts +++ b/packages/db/src/live-query-options.ts @@ -86,6 +86,10 @@ function createCollectionResolver( } } +/** + * Evaluates query callbacks and binds descriptors through the supplied client. + * Nullish query results stay disabled; callers can defer descriptor sync starts. + */ export function prepareLiveQueryValue( value: unknown, dbClient: DbClient | undefined, @@ -105,19 +109,16 @@ export function prepareLiveQueryValue( ) } - if ( - value && - typeof value === `object` && - !isCollection(value) && - !(value instanceof BaseQueryBuilder) && - `query` in value - ) { + if (isCollection(value)) return value + + const queryConfig = getLiveQueryConfig(value) + if (queryConfig) { const { query, queryKey: _queryKey, client: _client, ...config - } = value as PreparedLiveQueryConfigInput + } = queryConfig const preparedQuery = prepareLiveQueryValue( query, @@ -138,6 +139,16 @@ export function prepareLiveQueryValue( return value } +/** Recognizes config-shaped inputs without evaluating their query callbacks. */ +function getLiveQueryConfig( + value: unknown, +): PreparedLiveQueryConfigInput | undefined { + if (value && typeof value === `object` && `query` in value) { + return value as PreparedLiveQueryConfigInput + } + return undefined +} + /** Concrete source objects in query-position order for a prepared value. */ export function getPreparedLiveQuerySources( value: unknown, @@ -147,9 +158,7 @@ export function getPreparedLiveQuerySources( const query = value instanceof BaseQueryBuilder ? value - : value && typeof value === `object` && `query` in value - ? value.query - : value + : (getLiveQueryConfig(value)?.query ?? value) if (query instanceof BaseQueryBuilder) { return collectSourceRefs(query._getQuery()).map((source) => source.type === `descriptorRef` @@ -162,6 +171,10 @@ export function getPreparedLiveQuerySources( return isCollection(query) ? [query] : [] } +/** + * Includes result configuration and loading mode in a derived query identity. + * Omitting autoLoad and explicitly enabling it produce the same identity. + */ export function getPreparedLiveQueryIdentity(value: unknown): unknown { if (isCollection(value)) return [`collection`, value.id] if (value instanceof BaseQueryBuilder) { @@ -172,14 +185,15 @@ export function getPreparedLiveQueryIdentity(value: unknown): unknown { ? [`query`, getStableQueryBuilderHash(value)] : [`pooled`, pooled] } - if (value && typeof value === `object` && `query` in value) { - const config = value as LiveQueryCollectionConfig + const config = getLiveQueryConfig(value) + if (config) { return [ `config`, getPreparedLiveQueryIdentity(config.query), [`getKey`, config.getKey], [`schema`, config.schema], [`singleResult`, config.singleResult === true], + ...(config.autoLoad === false ? [[`autoLoad`, false]] : []), [ `defaultStringCollation`, getStringCollationIdentity(config.defaultStringCollation), @@ -190,16 +204,25 @@ export function getPreparedLiveQueryIdentity(value: unknown): unknown { return [`value`, value] } +/** + * Identifies a query for preloading, hydration, and shared observers. + * Explicit keys replace derived identity but keep loading modes separate. + */ export function getLiveQueryHash( preparedValue: unknown, queryKey?: LiveQueryKey, ): string { - const identity = queryKey?.length + const baseIdentity = queryKey?.length ? [`queryKey`, queryKey] : isCollection(preparedValue) ? [`collection`, preparedValue.id] : [`derived`, getPreparedLiveQueryIdentity(preparedValue)] + const autoLoad = getLiveQueryConfig(preparedValue)?.autoLoad !== false + const identity = [ + ...baseIdentity, + ...(queryKey?.length && !autoLoad ? [[`autoLoad`, false]] : []), + ] return getStableValueHash(identity, `queryKey`) } diff --git a/packages/db/src/query/live/ARCHITECTURE.md b/packages/db/src/query/live/ARCHITECTURE.md index fa9a104a32..5000538cc8 100644 --- a/packages/db/src/query/live/ARCHITECTURE.md +++ b/packages/db/src/query/live/ARCHITECTURE.md @@ -28,8 +28,8 @@ This architecture covers: - coherent publication to public Collections; - the boundaries with query-db ownership and physical query planning. -The applied-settlement receipt described below is its only new public boundary -contract. Optimistic transactions are another source of weighted input changes; +Public boundary contracts include applied-settlement receipts and automatic +loading control. Optimistic transactions are another source of weighted input changes; they do not have a separate routing model. ## One relational graph @@ -689,6 +689,26 @@ render, nor does it relax the stable public Collection facade contract above. ## Demand plane +### Automatic loading + +`LiveQueryCollectionConfig.autoLoad: false` evaluates the same relational graph +over available source rows without acquiring source data. Its subscriptions +retain the source Collections against GC, but do not count as acquiring +subscribers, start base-source sync, resume acquisition, or invoke +`loadSubset`/`unloadSubset`. Joins and includes receive available source rows; +ordered windows run the graph's top-K operator without provider-prefix loading. + +Readiness means the graph has evaluated its available inputs, independently of +source readiness or provider errors. Graph failures and explicit source cleanup +retain their normal error behavior. + +Other consumers, explicit source startup, or background sync may update shared +source rows. A source that is itself a live-query Collection supplies its +published rows under its own loading policy. Queries with `autoLoad: false` +have separate identities from queries using `true` or the default, including +with explicit framework keys, so their query instances and readiness remain +separate. + ### Demand grouping and ownership Demand is derived from data, but it performs asynchronous side effects outside @@ -1576,6 +1596,12 @@ create recursive Collection machinery. counts as a preload. A deferring live query keeps its sources from garbage collection but is not counted in their `subscriberCount` or `subscribers:change`, which adapters read as a request for live data. +16. **Automatic loading:** a query configured with `autoLoad: false` never + initiates source acquisition, including on subscription, preload, window + changes, truncate, and query restart. It becomes ready on the evaluated + local result independently of provider readiness. Laws about required + acquisition settlement and provider-prefix authority apply to acquiring + queries; local windows describe only available rows. ## Glossary @@ -1632,6 +1658,7 @@ keep the meanings defined there. | Reachable nested shape | `packages/query-db-collection/tests/includes-work-counter-oracle.test.ts` | | Cleanup-start invalidation, settlement, and restart admission | `packages/db/tests/collection-cleanup-restart-oracle.test.ts` | | Pooled live queries match their live-query Collection, including cleanup | `packages/db/tests/query/pooled-live-query-oracle.property.test.ts` | +| Automatic loading disabled, zero acquisition, local readiness, and coexistence with fetching consumers | `packages/db/tests/live-query-auto-load-oracle.test.ts` | | Deferred acquisition before a subscriber or preload, readiness, and the synchronous cut | `packages/db/tests/live-query-deferred-acquisition-oracle.test.ts` | Each oracle identifies the first divergent checkpoint and compares either the diff --git a/packages/db/src/query/live/collection-config-builder.ts b/packages/db/src/query/live/collection-config-builder.ts index d683d262ac..6356784c8c 100644 --- a/packages/db/src/query/live/collection-config-builder.ts +++ b/packages/db/src/query/live/collection-config-builder.ts @@ -131,6 +131,10 @@ export class CollectionConfigBuilder< return this.liveQueryCollection?._hasSubscriberOrPreload() ?? true } + get autoLoad(): boolean { + return this.config.autoLoad !== false + } + private windowFn: ((options: WindowOptions) => void) | undefined private readonly initialWindow: WindowOptions | undefined private currentWindow: WindowOptions | undefined @@ -1109,8 +1113,8 @@ export class CollectionConfigBuilder< ) { const { status } = event - // Handle error state - any source collection in error puts live query in error - if (status === `error`) { + // Local results remain usable when provider loading fails. + if (status === `error` && this.autoLoad) { this.erroredSourceIds.add(sourceId) this.setErrorState( devBuild() && process.env.NODE_ENV !== `production` @@ -1213,7 +1217,9 @@ export class CollectionConfigBuilder< this.liveQueryCollection?._lifecycle.markError(error ?? new Error(message)) } + /** Queries that do not acquire data have no source-readiness prerequisite. */ private allRequiredSourcesReady() { + if (!this.autoLoad) return true return this.collectionSources.every(({ collection }) => sourcePermitsInitialReadiness(collection), ) @@ -1287,6 +1293,7 @@ export class CollectionConfigBuilder< const subscription = collectionSubscriber.subscribe() this.subscriptions[sourceId] = subscription + if (!this.autoLoad) return () => {} if (subscription.isDeferringAcquisition()) { // No subscriber or preload yet: source reads stay local, and the // deferred provider work starts when the first one arrives. diff --git a/packages/db/src/query/live/collection-subscriber.ts b/packages/db/src/query/live/collection-subscriber.ts index b1e64861c9..b7025639d5 100644 --- a/packages/db/src/query/live/collection-subscriber.ts +++ b/packages/db/src/query/live/collection-subscriber.ts @@ -67,7 +67,19 @@ export class CollectionSubscriber< return !this.collectionConfigBuilder.hasSubscriberOrPreload() } + /** + * With loading disabled, the graph needs all available matching rows for + * joins and top-K evaluation, independent of provider coverage. + */ private subscribeToChanges(whereExpression?: BasicExpression) { + if (!this.collectionConfigBuilder.autoLoad) { + const subscription = this.collection.subscribeChanges( + (changes) => this.sendChangesToPipeline(changes), + { includeInitialState: true, whereExpression, autoLoad: false }, + ) + this.registerSubscriptionCleanup(subscription) + return subscription + } const orderByInfo = this.collectionConfigBuilder.optimizableOrderByCollections[this.sourceId] diff --git a/packages/db/src/query/live/types.ts b/packages/db/src/query/live/types.ts index 307397698e..5eef8b5e04 100644 --- a/packages/db/src/query/live/types.ts +++ b/packages/db/src/query/live/types.ts @@ -104,6 +104,14 @@ export interface LiveQueryCollectionConfig< */ startSync?: boolean + /** + * Set to false to query available rows without acquiring source data. + * Readiness describes the result over those rows. + * Other consumers may still load and update the shared source rows. + * @default true + */ + autoLoad?: boolean + /** * GC time for the collection */ diff --git a/packages/db/src/types.ts b/packages/db/src/types.ts index 39bd8aeb33..881e05065b 100644 --- a/packages/db/src/types.ts +++ b/packages/db/src/types.ts @@ -1123,6 +1123,8 @@ export interface SubscribeChangesOptions< * @internal */ deferAcquisition?: boolean + /** Set to false to observe rows without acquiring source data. @internal */ + autoLoad?: boolean /** * Listener for subscription status changes. * Registered BEFORE any snapshot is requested, ensuring no status transitions are missed. diff --git a/packages/db/tests/live-query-auto-load-oracle.test.ts b/packages/db/tests/live-query-auto-load-oracle.test.ts new file mode 100644 index 0000000000..49564caf4c --- /dev/null +++ b/packages/db/tests/live-query-auto-load-oracle.test.ts @@ -0,0 +1,331 @@ +/** + * Proposed automatic-loading contract: subscribing, preloading, moving a window and + * restarting never start source sync or acquisition. Results equal array + * evaluation over available rows, including while source readiness is pending. + * Authority: automatic loading in query/live/ARCHITECTURE.md. + * The model replaces independent arrays before each settled graph checkpoint. + * Limits: controlled adapters, numeric order, one join or inline include; + * provider behavior and framework scheduling have separate owners. + */ +import { fc } from '@fast-check/vitest' +import { describe, expect, it, vi } from 'vitest' +import { + BTreeIndex, + createCollection, + createLiveQueryCollection, + eq, + toArray, +} from '../src' +import { + oraclePropertyOptions, + oracleRuns, + readOracleRunConfig, +} from './oracle-config' +import { stripVirtualProps, withOracleCleanup } from './utils' +import type { LoadSubsetRequestResult, SyncConfig } from '../src' + +type Row = { id: number; rank: number; parentId: number } +function source(syncMode: `eager` | `on-demand`, indexed: boolean) { + let writer: Parameters[`sync`]>[0] | undefined + const load = vi.fn((): LoadSubsetRequestResult => true) + const unload = vi.fn() + const start = vi.fn((params: Parameters[`sync`]>[0]) => { + writer = params + return { loadSubset: load, unloadSubset: unload } + }) + const collection = createCollection({ + getKey: (row) => row.id, + syncMode, + autoIndex: indexed ? `eager` : `off`, + defaultIndexType: BTreeIndex, + gcTime: 0, + sync: { sync: start }, + }) + if (indexed) collection.createIndex((row) => row.rank) + return { + collection, + load, + unload, + start, + replace(rows: Array) { + collection.startSyncImmediate() + writer!.begin() + writer!.truncate() + for (const value of rows) writer!.write({ type: `insert`, value }) + writer!.commit() + }, + } +} +// Each step atomically replaces both independently keyed sources, then moves +// a window. IDs 1..6 exclude duplicate keys; ranks 0..3 include ties; parent IDs +// 1..3 admit matching and unmatched children; limits 0..4 cross closed, full and +// underfilled windows. Source mode changes acquisition semantics; explicit rank +// indexes check parity with and without an available index. Provider loading +// plans stay inactive with autoLoad disabled. One to five replacements admit empty, +// growing and shrinking inputs. Cleanup/restart follows every generated trace. +const rowSet = fc.uniqueArray( + fc.record({ + id: fc.integer({ min: 1, max: 6 }), + rank: fc.integer({ min: 0, max: 3 }), + parentId: fc.integer({ min: 1, max: 3 }), + }), + { selector: (row) => row.id, maxLength: 6 }, +) + +describe(`live queries with automatic loading disabled`, () => { + it(`keeps local subscription readiness when another owner starts the source`, async () => { + const src = source(`on-demand`, false) + const subscription = src.collection.subscribeChanges(() => {}, { + autoLoad: false, + includeInitialState: true, + }) + await withOracleCleanup(() => { + expect(subscription.status).toBe(`ready`) + src.collection.startSyncImmediate() + expect(subscription.status).toBe(`ready`) + src.replace([{ id: 1, rank: 0, parentId: 1 }]) + expect(subscription.status).toBe(`ready`) + subscription.resumeDeferredAcquisition() + expect(src.load).not.toHaveBeenCalled() + }, [() => subscription.unsubscribe(), () => src.collection.cleanup()]) + }) + it(`publishes local window changes while an eager source is loading or failed`, async () => { + const src = source(`eager`, true) + src.collection.startSyncImmediate() + const query = createLiveQueryCollection({ + autoLoad: false, + query: (q) => + q + .from({ row: src.collection }) + .orderBy(({ row }) => row.rank) + .limit(1), + }) + await withOracleCleanup(async () => { + await query.preload() + expect(src.collection.status).toBe(`loading`) + expect(query.isReady()).toBe(true) + src.replace([ + { id: 1, rank: 1, parentId: 1 }, + { id: 2, rank: 2, parentId: 1 }, + ]) + const writer = src.start.mock.calls[0]![0] + writer.begin() + writer.write({ type: `update`, value: { id: 1, rank: 3, parentId: 1 } }) + writer.commit() + expect([...query.values()].map((row) => row.id)).toEqual([2]) + writer.begin() + writer.write({ type: `delete`, value: { id: 2, rank: 2, parentId: 1 } }) + writer.commit() + expect([...query.values()].map((row) => row.id)).toEqual([1]) + writer.markError(new Error(`provider offline`)) + expect(query.isReady()).toBe(true) + expect(src.load).not.toHaveBeenCalled() + }, [() => query.cleanup(), () => src.collection.cleanup()]) + }) + for (const syncMode of [`eager`, `on-demand`] as const) { + for (const indexed of [false, true]) { + for (const shape of [`window`, `join`, `include`] as const) { + const property = `live-query-auto-load.${syncMode}.${indexed}.${shape}` + const replay = readOracleRunConfig() + for (const fixed of [true, false]) { + const selected = + replay.replayPath === undefined || + (!fixed && replay.replayProperty === property) + const options = fixed + ? { numRuns: oracleRuns(10), seed: 112919 } + : oraclePropertyOptions(10, property) + it.skipIf(!selected)( + `${syncMode}, indexed=${indexed}, ${shape}, ${fixed ? `fixed seed 112919` : `random or replayed`}: evaluates available rows without acquiring`, + async () => { + await fc.assert( + fc.asyncProperty( + fc.array( + fc.tuple(rowSet, rowSet, fc.integer({ min: 0, max: 4 })), + { minLength: 1, maxLength: 5 }, + ), + async (history) => { + const leftSource = source(syncMode, indexed) + const rightSource = source(syncMode, indexed) + const query = createLiveQueryCollection({ + autoLoad: false, + gcTime: 0, + query: (q) => { + if (shape === `join`) + return q + .from({ left: leftSource.collection }) + .innerJoin( + { right: rightSource.collection }, + ({ left, right }) => eq(left.id, right.parentId), + ) + .orderBy(({ left }) => left.rank) + .orderBy(({ left }) => left.id) + .orderBy(({ right }) => right.id) + .select(({ left, right }) => ({ + id: left.id, + child: right.id, + })) + .limit(2) + if (shape === `include`) + return q + .from({ left: leftSource.collection }) + .orderBy(({ left }) => left.rank) + .orderBy(({ left }) => left.id) + .select(({ left }) => ({ + id: left.id, + children: toArray( + q + .from({ right: rightSource.collection }) + .where(({ right }) => + eq(right.parentId, left.id), + ) + .orderBy(({ right }) => right.id) + .select(({ right }) => ({ id: right.id })), + ), + })) + .limit(2) + return q + .from({ left: leftSource.collection }) + .orderBy(({ left }) => left.rank) + .orderBy(({ left }) => left.id) + .select(({ left }) => ({ id: left.id })) + .limit(2) + }, + }) + let stop = () => {} + await withOracleCleanup(async () => { + for (const src of [leftSource, rightSource]) { + expect(src.collection.indexes.size > 0).toBe(indexed) + } + { + const sub = query.subscribeChanges(() => {}) + stop = () => sub.unsubscribe() + } + expect(query.isReady()).toBe(true) + expect([...query.values()]).toEqual([]) + expect(leftSource.start).not.toHaveBeenCalled() + expect(rightSource.start).not.toHaveBeenCalled() + await query.preload() + for (const [parents, children, limit] of history) { + leftSource.replace(parents) + rightSource.replace(children) + await query.utils.setWindow({ limit, offset: 0 }) + const ordered = [...parents].sort( + (a, b) => a.rank - b.rank || a.id - b.id, + ) + const expected = + shape === `join` + ? ordered.flatMap((parent) => + children + .filter( + (child) => child.parentId === parent.id, + ) + .sort((a, b) => a.id - b.id) + .map((child) => ({ + id: parent.id, + child: child.id, + })), + ) + : ordered.map((parent) => + shape === `include` + ? { + id: parent.id, + children: children + .filter( + (child) => + child.parentId === parent.id, + ) + .sort((a, b) => a.id - b.id) + .map((child) => ({ id: child.id })), + } + : { id: parent.id }, + ) + expect( + [...query.values()].map(stripVirtualProps), + ).toEqual(expected.slice(0, limit)) + expect(query.isReady()).toBe(true) + expect(leftSource.load).not.toHaveBeenCalled() + expect(rightSource.load).not.toHaveBeenCalled() + expect(leftSource.collection.subscriberCount).toBe(0) + } + stop() + await query.cleanup() + { + const sub = query.subscribeChanges(() => {}) + stop = () => sub.unsubscribe() + } + await query.preload() + expect(query.isReady()).toBe(true) + expect(leftSource.load).not.toHaveBeenCalled() + expect(rightSource.load).not.toHaveBeenCalled() + }, [ + () => stop(), + () => query.cleanup(), + () => leftSource.collection.cleanup(), + () => rightSource.collection.cleanup(), + ]) + expect(leftSource.unload).not.toHaveBeenCalled() + expect(rightSource.unload).not.toHaveBeenCalled() + }, + ), + options, + ) + }, + ) + } + } + } + } + it(`coexists with a fetching consumer whose request is pending`, async () => { + const src = source(`on-demand`, false) + let resolve!: () => void + src.load.mockImplementation( + () => + new Promise((r) => { + resolve = r + }), + ) + const local = createLiveQueryCollection({ + autoLoad: false, + query: (q) => q.from({ row: src.collection }), + }) + const remote = createLiveQueryCollection((q) => + q.from({ row: src.collection }), + ) + let stopLocal = () => {} + let stopRemote = () => {} + await withOracleCleanup(async () => { + { + const sub = local.subscribeChanges(() => {}) + stopLocal = () => sub.unsubscribe() + } + expect(src.start).not.toHaveBeenCalled() + { + const sub = remote.subscribeChanges(() => {}) + stopRemote = () => sub.unsubscribe() + } + expect(src.load).toHaveBeenCalledTimes(1) + expect(local.isReady()).toBe(true) + expect(remote.isReady()).toBe(false) + expect(src.collection.subscriberCount).toBe(1) + const writer = src.start.mock.calls[0]![0] + writer.begin() + writer.write({ type: `insert`, value: { id: 1, rank: 0, parentId: 1 } }) + writer.commit() + expect([...local.values()].map((row) => row.id)).toEqual([1]) + expect(local.isReady()).toBe(true) + stopLocal() + expect(src.unload).not.toHaveBeenCalled() + resolve() + await remote.preload() + stopRemote() + await remote.cleanup() + expect(src.unload).toHaveBeenCalledTimes(1) + }, [ + () => stopLocal(), + () => stopRemote(), + () => local.cleanup(), + () => remote.cleanup(), + () => src.collection.cleanup(), + ]) + }) +}) diff --git a/packages/db/tests/oracle-config.ts b/packages/db/tests/oracle-config.ts index 0bc12b623d..70d87fd328 100644 --- a/packages/db/tests/oracle-config.ts +++ b/packages/db/tests/oracle-config.ts @@ -289,8 +289,20 @@ export function validateOraclePropertyRegistry( return registry } +const autoLoadProperties: Array = [] +for (const mode of [`eager`, `on-demand`]) { + for (const indexed of [false, true]) { + for (const shape of [`window`, `join`, `include`]) { + autoLoadProperties.push( + `live-query-auto-load.${mode}.${indexed}.${shape}`, + ) + } + } +} + export const registeredOracleProperties = validateOraclePropertyRegistry([ ...staticOracleProperties, + ...autoLoadProperties, ...publicationProperties, ...refinementProperties, ...paginationMatrixProperties, diff --git a/packages/query-db-collection/tests/live-query-auto-load.test.ts b/packages/query-db-collection/tests/live-query-auto-load.test.ts new file mode 100644 index 0000000000..6c4b3cd117 --- /dev/null +++ b/packages/query-db-collection/tests/live-query-auto-load.test.ts @@ -0,0 +1,41 @@ +import { expect, it, vi } from 'vitest' +import { QueryClient } from '@tanstack/query-core' +import { createCollection, createLiveQueryCollection } from '@tanstack/db' +import { queryCollectionOptions } from '../src/query' +import { withOracleCleanup } from '../../db/tests/utils' + +it(`observes custom API ingestion without invoking the Query Collection fetcher`, async () => { + const queryClient = new QueryClient() + const queryFn = vi.fn(() => Promise.resolve([{ id: 1, title: `server` }])) + const tasks = createCollection( + queryCollectionOptions({ + queryClient, + queryKey: [`tasks`], + queryFn, + getKey: (row) => row.id, + syncMode: `on-demand`, + }), + ) + const local = createLiveQueryCollection({ + autoLoad: false, + query: (q) => + q + .from({ task: tasks }) + .orderBy(({ task }) => task.id) + .limit(1), + }) + await withOracleCleanup(async () => { + await local.preload() + expect(local.isReady()).toBe(true) + expect(tasks.status).toBe(`idle`) + expect(queryFn).not.toHaveBeenCalled() + // The application owns initialization and ingestion from its custom API. + tasks.startSyncImmediate() + tasks.utils.writeUpsert([{ id: 1, title: `custom API` }]) + expect([...local.values()].map((row) => row.title)).toEqual([`custom API`]) + tasks.utils.writeUpsert([{ id: 1, title: `updated` }]) + expect([...local.values()].map((row) => row.title)).toEqual([`updated`]) + expect(queryFn).not.toHaveBeenCalled() + expect(tasks.subscriberCount).toBe(0) + }, [() => local.cleanup(), () => tasks.cleanup(), () => queryClient.clear()]) +}) diff --git a/packages/react-db/src/useLiveInfiniteQuery.ts b/packages/react-db/src/useLiveInfiniteQuery.ts index ca4e0b8da5..7d0a2f8d81 100644 --- a/packages/react-db/src/useLiveInfiniteQuery.ts +++ b/packages/react-db/src/useLiveInfiniteQuery.ts @@ -60,6 +60,8 @@ export type UseLiveInfiniteQueryConfig<_TContext extends Context> = { queryKey?: LiveQueryKey /** Override the nearest DbProvider for this query. */ client?: DbClient + /** Set to false to paginate available rows without acquiring source data. Defaults to true. */ + autoLoad?: boolean pageSize?: number /** First result-page label, not a server cursor or remote offset. */ initialPageParam?: number @@ -113,6 +115,7 @@ type InfiniteQueryRenderState = { inputCollection: Collection | null inputQuery: unknown client: DbClient | undefined + autoLoad: boolean identityMode: `collection` | `queryKey` | `legacyDeps` | `derived` dependencies: Array | null pageSize: number @@ -128,6 +131,7 @@ type InfiniteQueryRenderState = { type RenderedWindowCollection = { client: DbClient | undefined + autoLoad: boolean dependencies: Array collection: Collection deferredCollections: Set< @@ -141,11 +145,11 @@ type RenderedWindowCollection = { */ function queryIdentityMatches( comparison: { changed: boolean; structurallyEqual: boolean }, - sameClient: boolean, + sameConfiguration: boolean, usesLegacyDeps: boolean, ): boolean { return ( - sameClient && + sameConfiguration && (usesLegacyDeps ? !comparison.changed : comparison.structurallyEqual) ) } @@ -169,7 +173,7 @@ export function useLiveInfiniteQuery< TUtils extends Record, >( liveQueryCollection: Collection & NonSingleResult, - config: UseLiveInfiniteQueryConfig, + config: Omit, `autoLoad`>, ): UseLiveInfiniteQueryReturnWithCollection // Overload for query function @@ -179,7 +183,7 @@ export function useLiveInfiniteQuery( deps?: Array, ): UseLiveInfiniteQueryReturn -// Implementation +/** Changing loading mode recreates the query and resets its accumulated pages. */ export function useLiveInfiniteQuery( queryFnOrCollection: any, config: UseLiveInfiniteQueryConfig, @@ -266,18 +270,29 @@ export function useLiveInfiniteQuery( } } + if (inputIsCollection && config.autoLoad !== undefined) { + throw new Error(`Set autoLoad when creating the live-query Collection.`) + } + const autoLoad = config.autoLoad !== false const usesLegacyDeps = !inputIsCollection && config.queryKey === undefined && deps !== undefined const dependencyComparison = compareLiveQueryWindowDependencies( committed?.dependencies, identityDeps, ) - const sameClient = committed?.client === dbClient + const sameConfiguration = + committed?.client === dbClient && committed?.autoLoad === autoLoad const dependenciesChanged = !inputIsCollection && - !queryIdentityMatches(dependencyComparison, sameClient, usesLegacyDeps) + !queryIdentityMatches( + dependencyComparison, + sameConfiguration, + usesLegacyDeps, + ) const dependenciesStructurallyEqual = - usesLegacyDeps && sameClient && dependencyComparison.structurallyEqual + usesLegacyDeps && + sameConfiguration && + dependencyComparison.structurallyEqual const needsNewCollection = committed === null || committed.inputKind !== inputKind || @@ -336,7 +351,7 @@ export function useLiveInfiniteQuery( rendered.dependencies, identityDeps, ), - rendered.client === dbClient, + rendered.client === dbClient && rendered.autoLoad === autoLoad, usesLegacyDeps, ) && rendered.collection.status !== `cleaned-up` && @@ -366,6 +381,7 @@ export function useLiveInfiniteQuery( // setWindow. collection = createLiveQueryCollection({ query: input.query.limit(requiredLimit).offset(0), + autoLoad, gcTime: DEFAULT_GC_TIME_MS, }) startInRender = true @@ -398,6 +414,7 @@ export function useLiveInfiniteQuery( if (createdCollection) { renderedCollectionRef.current = { client: dbClient, + autoLoad, dependencies: [...identityDeps], collection, deferredCollections, @@ -409,6 +426,7 @@ export function useLiveInfiniteQuery( inputCollection: inputIsCollection ? collection : null, inputQuery: inputIsCollection ? null : queryFnOrCollection, client: dbClient, + autoLoad, identityMode, dependencies: inputIsCollection ? null : [...identityDeps], pageSize, diff --git a/packages/react-db/src/useLiveQuery.ts b/packages/react-db/src/useLiveQuery.ts index d8f532ef6e..6ae27b2ec3 100644 --- a/packages/react-db/src/useLiveQuery.ts +++ b/packages/react-db/src/useLiveQuery.ts @@ -713,14 +713,17 @@ export function useLiveQueryForSuspense( return useLiveQueryImpl(configOrQueryOrCollection, deps, true) } -// What one hook instance keeps across renders. It lives in a single ref -// slot, so a render neither looks up nor allocates more. +/** + * Keeps observer and query-reuse state in one ref per hook instance. + * Loading mode must match before a render can reuse the retained collection. + */ function createHookInstance(dbClient: DbClient | undefined) { return { collection: null as Collection | null, deps: null as Array | null, config: null as unknown, client: dbClient, + autoLoad: true, legacyUnhashableIdentity: [`legacy-unhashable`] as Array, derivedIdentityProfiler: { renderCount: 0, @@ -740,6 +743,10 @@ function createHookInstance(dbClient: DbClient | undefined) { } } +/** + * Shares subscription and identity handling between regular and Suspense hooks. + * Query reuse requires matching loading policy as well as query identity. + */ function useLiveQueryImpl( configOrQueryOrCollection: any, deps: Array | undefined, @@ -764,7 +771,7 @@ function useLiveQueryImpl( let preparedQueryValue: unknown | typeof unpreparedQueryValue = unpreparedQueryValue let identityDeps: ReadonlyArray - let streamIdentity: unknown = undefined + let streamIdentity: ReadonlyArray | undefined let identityError: UnhashableQueryIRError | undefined if (queryKey) { @@ -816,10 +823,19 @@ function useLiveQueryImpl( } } + const modeConfig = + typeof configOrQueryOrCollection === `function` + ? preparedQueryValue + : configOrQueryOrCollection + const autoLoad = modeConfig?.autoLoad !== false let queryHash: string | undefined if (streamIdentity !== undefined) { try { - queryHash = getStableValueHash(streamIdentity, `queryKey`) + const identity = [ + ...streamIdentity, + ...(queryKey && !autoLoad ? [[`autoLoad`, false]] : []), + ] + queryHash = getStableValueHash(identity, `queryKey`) } catch (error) { if (error instanceof UnhashableQueryIRError) { if (queryKey !== undefined) throw error @@ -898,7 +914,10 @@ function useLiveQueryImpl( const needsNewCollection = !instance.collection || (inputIsCollection && instance.config !== configOrQueryOrCollection) || - (!inputIsCollection && (instance.client !== dbClient || identityChanged)) + (!inputIsCollection && + (instance.client !== dbClient || + instance.autoLoad !== autoLoad || + identityChanged)) if (needsNewCollection) { if (inputIsCollection) { @@ -984,6 +1003,7 @@ function useLiveQueryImpl( instance.deps = [...identityDeps] } instance.client = dbClient + instance.autoLoad = autoLoad instance.queryHash = queryHash instance.suspenseKey = suspenseKey instance.identityError = identityError diff --git a/packages/react-db/tests/live-query-auto-load.test.tsx b/packages/react-db/tests/live-query-auto-load.test.tsx new file mode 100644 index 0000000000..f0f971a194 --- /dev/null +++ b/packages/react-db/tests/live-query-auto-load.test.tsx @@ -0,0 +1,198 @@ +import { act, renderHook } from '@testing-library/react' +import { StrictMode } from 'react' +import { describe, expect, it, vi } from 'vitest' +import { + createCollection, + eq, + getLiveQueryHash, + prepareLiveQueryValue, +} from '@tanstack/db' +import { useLiveInfiniteQuery, useLiveQuery } from '../src' +import { useLiveQueryForSuspense } from '../src/useLiveQuery' +import { getLiveQueryResultInfo } from '../src/live-query-internals' +import type { InitialQueryBuilder, SyncConfig } from '@tanstack/db' + +type Row = { id: number; name: string } +function source() { + let writer!: Parameters[`sync`]>[0] + const load = vi.fn(() => true as const) + const unload = vi.fn() + const start = vi.fn((params: typeof writer) => { + writer = params + return { loadSubset: load, unloadSubset: unload } + }) + const collection = createCollection({ + getKey: (row) => row.id, + syncMode: `on-demand`, + sync: { sync: start }, + }) + return { + collection, + start, + load, + unload, + ingest(rows: Array) { + collection.startSyncImmediate() + writer.begin() + for (const value of rows) writer.write({ type: `insert`, value }) + writer.commit() + }, + } +} + +describe(`React query automatic loading`, () => { + for (const identity of [`key`, `deps`] as const) { + it(`keeps the mode separate from caller ${identity} elements`, async () => { + const src = source() + const hook = renderHook( + ({ autoLoad }) => { + const key = autoLoad ? [`rows`, [`autoLoad`, false]] : [`rows`] + return useLiveQuery( + { + query: (q) => q.from({ row: src.collection }), + autoLoad, + ...(identity === `key` ? { queryKey: key } : {}), + }, + identity === `deps` ? key : undefined, + ) + }, + { initialProps: { autoLoad: false as boolean | undefined } }, + ) + const local = hook.result.current.collection + hook.rerender({ autoLoad: true }) + const remote = hook.result.current.collection + expect(remote).not.toBe(local) + expect(src.load).toHaveBeenCalledTimes(1) + hook.unmount() + await Promise.all([local.cleanup(), remote.cleanup()]) + await src.collection.cleanup() + }) + } + it(`is ready on empty memory and reacts to independently ingested rows in StrictMode`, async () => { + const src = source() + const hook = renderHook( + ({ enabled }) => + useLiveQuery({ + autoLoad: false, + query: (q) => + enabled + ? q + .from({ row: src.collection }) + .where(({ row }) => eq(row.name, `a`)) + : undefined, + }), + { initialProps: { enabled: true }, wrapper: StrictMode }, + ) + expect(hook.result.current.data).toEqual([]) + expect(hook.result.current.isReady).toBe(true) + expect(src.start).not.toHaveBeenCalled() + act(() => + src.ingest([ + { id: 1, name: `a` }, + { id: 2, name: `b` }, + ]), + ) + expect(hook.result.current.data?.map((row) => row.id)).toEqual([1]) + const first = hook.result.current.collection + hook.rerender({ enabled: false }) + expect(hook.result.current.isEnabled).toBe(false) + hook.rerender({ enabled: true }) + expect(hook.result.current.data?.map((row) => row.id)).toEqual([1]) + expect(src.load).not.toHaveBeenCalled() + const collection = hook.result.current.collection + hook.unmount() + await first?.cleanup() + await collection?.cleanup() + expect(src.unload).not.toHaveBeenCalled() + await src.collection.cleanup() + }) + + for (const identity of [`derived`, `key`, `deps`] as const) { + it(`switches automatic loading with ${identity} identity`, async () => { + const src = source() + const hook = renderHook( + ({ autoLoad }) => + useLiveQuery( + { + query: (q) => q.from({ row: src.collection }), + autoLoad, + ...(identity === `key` ? { queryKey: [`same`] } : {}), + }, + identity === `deps` ? [] : undefined, + ), + { initialProps: { autoLoad: false as boolean | undefined } }, + ) + const local = hook.result.current.collection + expect(src.load).not.toHaveBeenCalled() + hook.rerender({ autoLoad: true }) + const remote = hook.result.current.collection + expect(remote).not.toBe(local) + expect(src.load).toHaveBeenCalledTimes(1) + hook.rerender({ autoLoad: undefined }) + expect(hook.result.current.collection).toBe(remote) + expect(src.load).toHaveBeenCalledTimes(1) + hook.rerender({ autoLoad: false }) + expect(hook.result.current.collection).not.toBe(remote) + expect(src.load).toHaveBeenCalledTimes(1) + const last = hook.result.current.collection + hook.unmount() + await Promise.all([local.cleanup(), remote.cleanup(), last.cleanup()]) + await src.collection.cleanup() + }) + } + + it(`keeps local and fetching Suspense entries separate with the same explicit key`, async () => { + const src = source() + const config = { + queryKey: [`shared`], + query: (q: InitialQueryBuilder) => q.from({ row: src.collection }), + } + const hook = renderHook(() => ({ + local: useLiveQueryForSuspense({ ...config, autoLoad: false }, undefined), + remote: useLiveQueryForSuspense(config, undefined), + })) + const { local, remote } = hook.result.current + expect(local.collection).not.toBe(remote.collection) + expect(src.load).toHaveBeenCalledTimes(1) + expect(getLiveQueryResultInfo(local).queryHash).toBe( + getLiveQueryHash( + prepareLiveQueryValue({ ...config, autoLoad: false }, undefined), + config.queryKey, + ), + ) + hook.unmount() + await Promise.all([ + local.collection?.cleanup(), + remote.collection?.cleanup(), + ]) + await src.collection.cleanup() + }) + + it(`paginates local rows and resets pages when acquisition mode changes`, async () => { + const src = source() + src.ingest(Array.from({ length: 5 }, (_, id) => ({ id, name: `row` }))) + const hook = renderHook( + ({ autoLoad }) => + useLiveInfiniteQuery( + (q) => q.from({ row: src.collection }).orderBy(({ row }) => row.id), + { autoLoad, pageSize: 2, queryKey: [`pages`] }, + ), + { initialProps: { autoLoad: false }, wrapper: StrictMode }, + ) + expect(hook.result.current.data.map((row) => row.id)).toEqual([0, 1]) + await act(() => hook.result.current.fetchNextPage()) + expect(hook.result.current.data.map((row) => row.id)).toEqual([0, 1, 2, 3]) + await act(() => hook.result.current.fetchNextPage()) + expect(hook.result.current.hasNextPage).toBe(false) + expect(src.load).not.toHaveBeenCalled() + const local = hook.result.current.collection + hook.rerender({ autoLoad: true }) + expect(hook.result.current.collection).not.toBe(local) + expect(hook.result.current.pages).toHaveLength(1) + expect(src.load).toHaveBeenCalled() + const remote = hook.result.current.collection + hook.unmount() + await Promise.all([local?.cleanup(), remote?.cleanup()]) + await src.collection.cleanup() + }) +})