Skip to content

Custom string collation: opt-in to trust the provider's order, so ordered windows page instead of loading the full source #2240

Open

Description

@daveycodez

Since #1904, a custom string collation ({ stringSort: 'custom', compare }) marks every ordered query requiresFullSource, and computeSubscriptionOrderByHints drops orderBy/limit. That is the safe default when the backend's order is unknown. But some backends order strings with exactly that collation: InstantDB answers ordered/limited queries in Postgres glibc en_US, and our comparator matches it row for row (verified on 5,519 live rows).

Today those backends have two options. They can sort lexically, which gives the wrong pages. Or they can use the custom collation, and then every ordered window, on any field, requests its whole filtered source, because the collection default applies to every clause.

Proposal: an opt-in on the custom collation:

defaultStringCollation: {
  stringSort: 'custom',
  compare: comparePostgresText,
  trustProviderOrder: true,
}

With it set:

  • requiresFullSource and the subscription hints treat the order as provider-pageable, as for lexical order.
  • canExpressCursorOrder accepts string values under it.
  • The cursor's gt/lt carry the collation, so local evaluation agrees with the provider. gt/gte/lt/lte take an optional CustomStringCollation third argument, stored on the Func (not as an argument, so query identity still hashes; its identity is canonicalized like an orderBy collation).

Index range optimization skips collated comparisons.

Without the flag nothing changes. We have this implemented on top of 0.15.2 with tests (bounded first page, collated cursor paging, untrusted collations unchanged, comparison identity), and the full @tanstack/db suite passes. Happy to open a PR.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions