Repository navigation
[fix](timestamptz) Use local dates for year arithmetic - #70485
Conversation
|
/review |
|
run buildall |
There was a problem hiding this comment.
Request changes: one P1 partition-pruning regression and one P2 FE/BE result mismatch are verified by static code-path analysis. Reviewed all six changed files at 0bfbfbe and their relevant callers/consumers. No existing inline findings require duplicate suppression; no additional user focus was supplied.
Critical checkpoint conclusions:
- Goal and proof: ordinary local leap-day year arithmetic is corrected in FE and BE, with unit/regression coverage; inherited pruning ranges and year-zero parity remain incorrect as detailed inline.
- Scope and reuse: the change is focused and reuses DATETIME year transforms and existing timezone/DST conversion helpers.
- Concurrency and locking: no new threads, shared mutable state, locking, or lock-order changes.
- Lifecycle and initialization: new temporaries are local; the existing per-query runtime timezone remains borrowed. No new ownership or static-initialization dependency.
- Configuration: no configuration item is added or changed.
- Compatibility: no protocol, storage layout, or external function ABI changes; the arithmetic behavior change is intentional. No new libc API is introduced.
- Thrift/Protobuf: no gensrc definitions change; field-ID checks are inapplicable.
- Parallel paths: traced FE folding, BE const/const, vector/const, const/vector, vector/vector, nullable handling, addition/subtraction and DATE_ADD/DATE_SUB YEAR lowering. Unchanged transforms retain their existing execution path. Both year functions share the reported pruning issue.
- Conditions and errors: new branch comments explain local leap days, repeated hours and UTC bounds. Existing exception handling remains appropriate; TZ1 exposes a missing local-calendar check in FE.
- Test coverage: inspected new FE/BE tests for leap/century targets, signs, zero, fractions, DST gaps/overlaps and bounds, plus SQL folding and column-dispatch tests. They omit the partition-pruning and year-zero parity cases reported inline; nullable dispatch was traced rather than newly tested here.
- Expected results: all added .out values agree with ordinary +08:00 arithmetic and DATE/DATETIME controls; two constant sections correspond to the two folding modes. Generation provenance was not independently established in this static review.
- Observability: existing scalar error propagation is sufficient; new logs or metrics are unnecessary for this change.
- Transactions and persistence: no EditLog, replay, failover, visible-version or delete-bitmap logic changes.
- Writes and atomicity: no data-write or transaction lifecycle changes.
- FE/BE propagation: no new transmitted variable; the session timezone already reaches RuntimeState, and the changed column paths now pass the existing context.
- Performance and memory: fixed work per row, existing tracked output allocation and local/immutable temporaries; no additional material performance or memory-safety defect established by inspection.
- Other checks: no new includes/PCH edits, explicit instantiations, hygiene-table edits or source-.cpp test inclusions. Final special-case review found no further substantiated issue.
Coverage is complete for this static review, with no unresolved candidate or risk question. No builds, tests, benchmarks or runtime/ELF validation were executed.
Review converged after 2 round(s).
|
/review |
There was a problem hiding this comment.
Static review of PR #70485 at a19aab9 found one new P2: the BE year-arithmetic branch drops the original UTC source-validity check, so a nonzero year change can hide an invalid source date. The inline comment includes a reachable SQL/parser trace and fix direction.
Both existing findings are fixed at this head: #4545651761 is addressed by disabling TIMESTAMPTZ year range/monotonicity inference, and #4545651907 by applying the year-zero leap-day clamp locally before UTC conversion. No existing P0/P1 finding remains applicable. No additional user review focus was supplied.
Critical checkpoint conclusions:
- Goal and evidence: FE and BE now calculate TIMESTAMPTZ years in the session calendar. Source inspection supports the reported leap-day fix; M1 identifies the remaining UTC-input validation issue.
- Scope and reuse: four production files and eight test/result files form a focused change that reuses DATETIME year arithmetic and existing timezone helpers.
- Concurrency and locking: no new threads, locks or shared mutable state; existing session/query timezone state is read.
- Lifecycle and static initialization: conversion values are local temporaries; no new retained resources or initialization-order dependencies.
- Configuration: none added. Existing time_zone propagation and query-result cache separation cover the new session dependency.
- Compatibility: function signatures, precision, storage layout and wire payloads remain unchanged. Calendar results intentionally change; older executors retain the previous behavior during rollout.
- Thrift/protobuf: no definitions changed; field-ID compatibility checks are not applicable.
- Parallel paths: addition, subtraction and INTERVAL YEAR aliases reach the same implementation. Vector/vector, vector/constant, constant/vector, all-constant and nullable dispatch were traced. Other date families retain their existing paths.
- Conditions and errors: local year-zero clamping, local/UTC bounds and DST offset selection have concrete comments and reachable cases. M1 covers the missing original UTC-source check.
- Test coverage: BE/FE unit and regression cases cover leap/century/negative/zero intervals, fractions, year-zero targets, local range failures, repeated hours, gaps, pruning and fold-enabled/disabled execution paths. The invalid-UTC-source case needs coverage with the fix.
- Expected results: every added .out row was checked against its SQL and the conversion logic. Result-generation provenance was not established.
- Observability: existing scalar out-of-range errors are adequate; no new distributed tracing or metrics requirement was found.
- Transactions and persistence: no EditLog, replay, visible-version, delete-bitmap or failover changes.
- Data writes and atomicity: production write paths are unchanged; no new crash-atomicity or ownership concern was found.
- FE/BE variables: none added; existing query timezone forwarding supplies the BE context.
- Performance and memory: bounded per-row calendar/timezone work with stack temporaries in BE; conservative TIMESTAMPTZ year-pruning disablement prevents unsafe row elimination. No actionable allocation or complexity regression was found.
- Further review: optimizer consumers, inverse bucket-boundary expressions, folding failure paths, session cache separation, all changed files, existing threads and risk items were checked.
Validation was static only: no builds, tests, benchmarks, binary ABI audits or source changes were performed. Installed FE/BE timezone-database equivalence was not tested. No unresolved review candidate or changed-file coverage gap remains.
Review converged after 2 round(s).
|
/review |
There was a problem hiding this comment.
Reviewed all 12 changed files at b96c489. No new actionable findings. The BE, FE, and independent timezone reviewers each returned NO_NEW_VALUABLE_FINDINGS; main-agent verification and the final sweep found no unresolved source-level candidate. Review converged in round 1.
The previously reported concerns are addressed at this head:
- Pruning ranges, 4545651761: YearsAdd and YearsSub disable TIMESTAMPTZ range inference. Both partition evaluation and monotonicity consumers respect the gate; the new tests cover local leap days and DST cases. This P1 no longer applies.
- FE year-zero clamping, 4545651907: the local target is clamped to February 28 before UTC conversion, with add/sub coverage across positive, zero, and negative offsets.
- BE UTC source validity, 4578728534: source validity is checked before conversion, and the new tests cover zero/nonzero intervals and both directions.
Critical checkpoint conclusions from static inspection:
| Checkpoint | Conclusion |
|---|---|
| Goal and proof | FE folding and BE column execution now perform year arithmetic in the session calendar. Unit and regression coverage targets the reported local leap-day failure; tests were inspected, not executed. |
| Scope and reuse | Production changes are focused and reuse the existing DATETIME year transform and timezone/DST helpers. |
| Concurrency and locks | No new threads, shared mutation, or locks; query/session context is reused. |
| Lifecycle and static initialization | No new retained resources or static initializers; FunctionContext retains its existing RuntimeState relationship. |
| Configuration | No configuration added or changed. |
| Compatibility | No wire, storage-format, or persisted-metadata change. SQL results intentionally change for local-calendar year arithmetic; mixed-version runtime behavior was not exercised. |
| Thrift/protobuf | No definitions or IDs changed. |
| Parallel paths | Add/sub, negative/zero intervals, DATE_ADD/DATE_SUB and TIMESTAMPADD YEAR forms, and constant/vector/nullable dispatch were traced. Other date types retain their existing transform behavior. |
| Conditional checks | New guards document concrete UTC/local range, year-zero, and repeated-hour cases; invalid dates continue to fail through existing error paths. |
| Test coverage | BE/FE unit and SQL regression tests cover leap and century years, year zero, boundaries, fractional precision, DST gaps/overlaps, column operands, aliases, and partition pruning. Existing null-skip and const-unwrapping behavior was traced. |
| Changed expected results | All added expected rows were checked against source semantics and session rendering; no contradictory result found. Output-generation provenance was not independently verified. |
| Observability | Existing expression errors provide operation/value context; no new distributed operation requires logging or metrics. |
| Transactions and persistence | No EditLog, replay, transaction, or failover logic changed. |
| Data writes and atomicity | Production changes compute expressions; no write protocol, visibility/version rule, or delete-bitmap behavior changed. |
| FE-BE variables | Existing session timezone propagation is reused; no new transmitted variable or send path. |
| Performance and memory | Per-row timezone conversion is required by the corrected semantics. Existing constant-size temporaries and tracked column allocation remain; no new unbounded allocation or asymptotic work. Conservative pruning can scan more partitions for TIMESTAMPTZ year predicates. No benchmark ran. |
| Other risks | UTC/local validity, preferred offsets in repeated hours, DST gap fraction preservation, historical offset representation, error propagation, and alternative pruning consumers were checked without another substantiated defect. |
Core invariants: the change does not alter committed-data visibility, partition versions, or MoW delete bitmaps. No new memory ownership was introduced, and invalid arithmetic results fail through the existing expression error boundaries.
No additional user focus was supplied. Coverage includes the full supplied diff, related callers and consumers, tests, all existing threads, and every initial risk item. Validation was static only: no builds, tests, services, benchmarks, or ABI/runtime checks were executed, and repository source was not modified. Runtime behavior and timezone-database parity across installations remain outside this review's validation evidence.
Review converged after 1 round(s).
|
run buildall |
|
/review |
|
run buildall |
|
Codex automated review failed and did not complete. Error: Review setup stopped before context preparation ran; inspect the first failed or cancelled setup step. Please inspect the workflow logs and rerun the review after the underlying issue is resolved. |
|
/review |
|
Codex automated review failed and did not complete. Error: Review setup stopped before context preparation ran; inspect the first failed or cancelled setup step. Please inspect the workflow logs and rerun the review after the underlying issue is resolved. |
|
run buildall |
Local pipeline review — ✅ PASSschema: doris-repo-review/v1
status: PASS
pr: apache/doris#70485
commit: 96a2c6a86ae03750a5de3b972f867e713b4af850
base: 9e598d38789571a7fc30652aa054f035bd4f1184
reviewed_at: 2026-11-16T22:29+08:00
reviewer: jacktengg
model: gpt-6-astra
effort: xhigh
findings: {blocker: 0, major: 0, minor: 1, nit: 0}
rounds: 1
converged: trueNotes for maintainers
Reviewed locally with the |
96a2c6a to
ccc8288
Compare
Local pipeline review — ✅ PASSschema: doris-repo-review/v1
status: PASS
pr: apache/doris#70485
commit: ccc8288372868ad611dd2e38a0c5aef260463533
base: d7c0fb9427b005eb5ab8b9e0eb6592dd89964fc1
reviewed_at: 2026-11-17T01:27+08:00
reviewer: jacktengg
model: gpt-6-astra
effort: xhigh
findings: {blocker: 0, major: 0, minor: 1, nit: 0}
rounds: 1
converged: trueNotes for maintainers
Reviewed locally with the |
|
run buildall |
|
run nonConcurrent |
|
/review |
There was a problem hiding this comment.
Static review of all 17 changed files at ccc8288 found no new actionable findings. Two full-scope reviewers and a separate calendar/optimizer risk reviewer checked the changes and related callers.
All four existing concerns are addressed at this head:
- Year-arithmetic pruning: YearsAdd/YearsSub disable TIMESTAMPTZ range inference and monotonicity through canEvaluateRange; the partition evaluator and runtime-filter classifier respect these hooks. No existing P0/P1 remains applicable.
- FE year-zero clamping: February 29 is clamped locally before conversion back to UTC, with add/sub coverage.
- BE UTC source validation: invalid UTC input is rejected before local conversion can conceal it.
- Repeated timezone lookup: local fields and the preferred offset now come from one lookup, reused by year arithmetic and date_trunc.
Critical checkpoint conclusions:
- Goal and evidence: FE folding and BE column execution now apply YEAR arithmetic to the session-local date. Inspected unit and regression cases cover the reported leap day, add/sub, negative/zero intervals, century rules, year zero, precision, range errors, DST overlaps/gaps, and partition pruning.
- Scope and reuse: the change is focused; BE reuses DATETIMEV2 year arithmetic, existing timezone conversion and error handling. Tests account for most changed code.
- Concurrency and locking: no new threads, shared mutable state, or locks; arithmetic reads existing per-query/session timezone state.
- Lifecycle and static initialization: conversions use local values and the existing execution context; no new resource ownership or static-initialization dependency.
- Configuration: no new configuration; the existing session time_zone is used.
- Compatibility: SQL signatures, data types, serialized layouts and storage formats are unchanged. Internal helper callers were checked. TIMESTAMPTZ year results intentionally change to session-calendar semantics; deployment runtime compatibility was not executed.
- Thrift/Protobuf: no changes, so cross-branch field-ID checks are inapplicable.
- Parallel paths: add/sub, YEAR aliases, FE folding, all three BE vector/constant dispatch shapes, null handling, and date_trunc offset reuse were traced. Other temporal types and interval units retain their existing paths.
- Conditional checks: source/local/result validity checks and overlap handling have concrete failure-scenario comments; invariant failures use existing exceptions/status handling.
- Test coverage: the added FE/BE unit cases and regression suites exercise the principal success and failure branches. No source-level coverage gap remains unresolved; runtime behavior and installed FE/BE timezone-database parity were not verified.
- Expected results: all changed .out values were checked against their SQL and timezone arithmetic. Output generation provenance and the author's reported test passes were not independently verified.
- Observability: existing scalar date errors provide suitable context; no additional logging or metrics appears necessary.
- Transactions and persistence: no production transaction, EditLog, replay, or persistence changes.
- Data writes and crash behavior: no production write-path or atomicity changes; new test inserts follow existing regression patterns.
- FE/BE propagation: no new transmitted fields; the existing runtime timezone reaches all modified BE dispatch paths.
- Performance: the redundant input lookup is removed. Required local conversions remain constant work per row. Disabling TIMESTAMPTZ year range inference can reduce pruning, including in UTC, as a deliberate correctness tradeoff; no benchmark was run.
- Additional recheck: local/UTC range boundaries, year-zero policy, non-hour offsets/transitions, overlap offset retention, nullable/constant storage, aliases, and downstream pruning consumers yielded no further actionable issue.
There were no additional user-provided focus points. Coverage is complete for this static review, with no unresolved candidates or risk focuses. No builds, tests, runtime checks, benchmarks, source modifications, or submodule initialization were performed.
Review converged after 2 round(s).
### Background TIMESTAMPTZ stores a UTC date and time. Calendar-year arithmetic must use the date in the session time zone. DATE and DATETIME already use February 28 when a leap-day target is not in a leap year. ### What problem does this PR solve? Issue Number: None Related PR: None PR Type: Bugfix Problem Summary: For 2024-02-29 00:00:00 in +08:00, years_add(ts, 1) and years_sub(ts, 1) returned March 1. The stored UTC date is February 28, so UTC year arithmetic does not apply the local leap-day rule. Compute the year change on the local date in both FE constant evaluation and BE column evaluation, then convert the result to UTC. ### Details - **Change map:** 313 changed lines: 86 production, 211 tests, 16 generated output, and 0 documentation. BE uses the session calendar and passes the context through the column paths (57 lines). FE applies the same calendar and DST rules during constant evaluation (29 lines). - **Frequency:** Deterministic when a year change in UTC differs from the required change on the local calendar date. - **Root cause:** Both paths changed the stored UTC year. For the reported input, the resulting February 28 at 16:00 UTC becomes March 1 at 00:00 in +08:00. - **Why existing tests missed it:** Year tests covered DATE/DATETIME and a UTC leap day. They did not cover a local leap day whose UTC date is February 28. - **Fix:** Reuse the DATETIME year transform in BE after conversion to the session zone. Apply the same conversion in FE. Keep microseconds, preserve a valid input offset in a repeated hour, and reject dates outside the supported range. - **Other paths and branches:** Cover addition, subtraction, negative intervals, zero intervals, and the DATE_ADD/DATE_SUB YEAR forms. No Thrift, Protobuf, storage-format, or configuration change is required. - **Behavior change:** TIMESTAMPTZ year arithmetic uses the session calendar date and clamps a leap day to February 28 when the target year is not a leap year. ### Testing Local validation: - ASAN BE build and FE build passed. FE Checkstyle passed. - run-be-ut.sh: 5 relevant year-arithmetic tests passed. - run-fe-ut.sh: all 16 TimestampTzLiteralTest tests passed. With the original UTC year calculation restored, the new leap-day and DST-gap tests failed; both passed after the fix. - run-regression-test.sh: the one-row test_timestamptz_year_interval suite passed. The script generated the .out file; its dates, times, fractions, and offsets were checked. A second run compared against this file and passed (1 suite, 0 failures). - clang-format 16, clang-tidy, and build hygiene passed. Source whitespace checks passed. The generated .out file keeps the regression runner's final blank separator. ### Release note Fix TIMESTAMPTZ year arithmetic for local leap days. Adding or subtracting a year now returns February 28 when the target year has no February 29. For these leap-day inputs, the time and fractional seconds are preserved. ### Check List (For Author) - Behavior changed: Yes (see Details) - Does this need documentation: No - Pre-submission status: Ready ### AI Disclosure - Tool: Codex, GPT-6 - Generated: Code, tests, and this description - Human review: Pending author review
### Background TIMESTAMPTZ stores UTC instants, while years_add and years_sub operate on the session calendar. CalendarAddSubMonotonic derives partition ranges from folded endpoints and calendar-day bounds. ### What problem does this PR solve? Issue Number: None Related PR: None PR Type: Bugfix Problem Summary: UTC calendar bounds can exclude results produced by local leap-day clamping or DST transitions, causing matching partitions to be pruned. Java also considers year zero a leap year, making FE folding disagree with BE year arithmetic. Disable unsafe TIMESTAMPTZ year range inference and apply Doris's local date policy before UTC conversion. ### Details - **Frequency:** TIMESTAMPTZ year predicates crossing local leap-day or DST boundaries, and year arithmetic reaching year zero. - **Root cause:** Range inference still assumes UTC calendar arithmetic after year calculations moved to the session calendar. LocalDateTime.plusYears also retains February 29 in year zero, which Doris rejects. - **Why existing tests missed it:** Prior tests checked scalar local-year results without partition predicates or the year-zero boundary. - **Fix:** Disable TIMESTAMPTZ year range inference and monotonicity, clamp local year-zero February 29 to February 28, and reject invalid local input dates using the existing date validator. - **Other paths and branches:** Cover both years_add and years_sub, including signed intervals. Runtime filter monotonicity uses the same gate. Other temporal types retain existing range inference. BE's existing local-year implementation supplies the reference behavior; no BE changes are needed. - **Behavior change:** Matching partitions are retained, FE/BE year-zero results agree, and invalid local input dates consistently fail. Conservative inference may scan more partitions for TIMESTAMPTZ year predicates. ### Testing - Local FE build and integrated Checkstyle passed: FE_MAVEN_THREADS=8 MVN_OPT=-Dmaven.build.cache.enabled=false ./build.sh --fe -j8. - ./run-fe-ut.sh --run 'TimestampTzLiteralTest,CalendarAddSubMonotonicTest,PartitionPrunerTest': 47 tests passed. Before the production fix, the four new tests failed for unsafe pruning/inference, the year-zero mismatch and missing local-date rejection. - ./run-regression-test.sh --run -d datatype_p0/timestamptz -s test_timestamptz_year_interval,test_timestamptz_year_partition_pruning: both suites passed after generating output with -forceGenOut, then passed again in comparison mode. The six pruning queries returned no rows before the fix and the correct matching rows afterward. FE/BE constants and column calculations agree on year zero; both folding modes reject the invalid local date. - Inspected all generated expected results and completed the code-review checklist. Tests ran in this worktree with its isolated FE/BE ports. Temporary source configurations were restored and services stopped. ### Release note Fix missing results from TIMESTAMPTZ year predicates on partitioned tables near leap-day and DST boundaries. Align year-zero constant folding with execution by treating year zero as non-leap. ### Check List (For Author) - Behavior changed: Yes (see Details) - Does this need documentation: No - Pre-submission status: Ready ### AI Disclosure - Tool: OpenAI Codex - Generated: Code, tests, and this description. - Human review: Pending.
### Background TIMESTAMPTZ stores UTC fields, while years_add and years_sub use the session calendar. The year-arithmetic branch converts the UTC input to local time, applies the existing DATETIME transform, and validates the final UTC result. ### What problem does this PR solve? Issue Number: None Related PR: None PR Type: Bugfix Problem Summary: The local-year path skipped the UTC source validity check previously performed by TimestampTzValue::date_add_interval. A parsed UTC 0000-02-29 value can become a valid local February 28 in -08:00, allowing a nonzero year adjustment to hide the invalid source. Validate the stored UTC source before conversion so both year functions reject it consistently. ### Details - **Frequency:** TIMESTAMPTZ year arithmetic whose invalid stored UTC date becomes valid in the session timezone, including the supported year-zero boundary. - **Root cause:** The new local-year branch checked the local input and final UTC output but bypassed the original UTC input validator. - **Why existing tests missed it:** Tests covered local leap days, target clamping, DST and range overflow, but did not pass an invalid UTC date that maps to a valid local date. - **Fix:** Call from.is_valid_date() before convert_utc_to_local and report invalid values with the existing throw_out_of_bound_date_int helper. - **Other paths and branches:** The shared DateTimeOp::execute branch covers both years_add and years_sub and all constant/vector argument combinations. Existing null-map skips run before this check. FE logic, the parser and other arithmetic paths are unchanged. - **Behavior change:** Invalid stored UTC source dates fail consistently for zero and nonzero year intervals. Valid year-zero values retain their existing results and precision. ### Testing - ./build.sh --be -j48 passed with ASAN, including the automatic GLIBC_2.17 compatibility check. - ./run-be-ut.sh --run --filter='VTimestampFunctionsTest.timestamptz_years_*:VTimestampFunctionsTest.years_*_v2_test' -j48: all 6 tests passed, including the new invalid-UTC-source case and valid year-zero precision controls. - ./run-regression-test.sh --run -d datatype_p0/timestamptz -s test_timestamptz_year_interval,test_timestamptz_year_partition_pruning: both suites passed. Before the fix, the added years_add(..., 1) expected-error query failed because the operation succeeded; afterward all four new add/sub expected-error queries passed. Snapshot output is unchanged. - build-support/check-build-hygiene.sh, clang-format.sh/check-format.sh with clang-format v16 on the changed C++ files, and run-clang-tidy.sh all passed. - Tests ran in the current worktree. Temporary configurations were restored and the test services stopped. ### Release note Fix years_add and years_sub accepting invalid UTC TIMESTAMPTZ inputs when timezone conversion and a year adjustment would hide the invalid source date. ### Check List (For Author) - Behavior changed: Yes (see Details) - Does this need documentation: No - Pre-submission status: Ready ### AI Disclosure - Tool: OpenAI Codex - Generated: Code, tests, and this description. - Human review: Pending.
### Background The calendar arithmetic pruning suite checks several temporal types against shared query results and EXPLAIN expectations. TIMESTAMPTZ year range inference is conservatively disabled because session-local clamping and DST can invalidate UTC calendar bounds. ### What problem does this PR solve? Issue Number: None Related PR: None PR Type: Test Problem Summary: The years_add EXPLAIN assertion still expected three of four partitions for every temporal type, causing the TIMESTAMPTZ iteration to fail on the correct four-partition plan. Select the exact expected partition list by type while retaining the original DATETIMEV2 and TIMESTAMP_NS assertions. ### Details - **Failure:** test_calendar_arithmetic_prune expected partitions=3/4 for a TIMESTAMPTZ years_add predicate, while the plan correctly retained partitions=4/4. - **Cause:** The shared test expectation was not updated when TIMESTAMPTZ year range inference was disabled, including in UTC sessions. - **Stability evidence:** The original suite reproduced the reported failure. The corrected full suite passed with the same FE/BE binaries and data setup, including all three temporal types, result comparisons and DATE pruning. Existing snapshot files are unchanged. ### Testing Locally ran ./run-regression-test.sh --run -d nereids_rules_p0/partition_prune -s test_calendar_arithmetic_prune before and after the edit. Before: the reported EXPLAIN assertion failed. After: 1 suite passed, 0 failures. git diff --check passed. ### Release note None ### Check List (For Author) - Behavior changed: No - Does this need documentation: No - Pre-submission status: Ready ### AI Disclosure - Tool: OpenAI Codex - Generated: Test adjustment and this description. - Human review: Pending.
### Background TIMESTAMPTZ values store UTC datetime fields. Calendar-year arithmetic and date_trunc convert them to the session timezone, retaining the input offset when an output local time occurs twice during a DST transition. ### What problem does this PR solve? Issue Number: None Related PR: None PR Type: Performance Problem Summary: Year arithmetic and sub-day date_trunc reconstructed the input UTC time point and looked it up in the session timezone twice to obtain the local datetime and offset separately. Return both from the existing UTC-to-local conversion and reuse its offset in both callers. SQL results, source validation, fractional seconds, and repeated-time selection remain unchanged. ### Details - **Why it was slow:** convert_utc_to_local discarded the offset from an absolute_lookup, so each caller repeated the input conversion and lookup through utc_offset. - **Optimization:** Return the offset alongside the output datetime from the existing helper, and pass it to the preferred-offset local-to-UTC conversion. - **Results:** Input session-zone absolute lookups decrease from two to one per row on the affected paths. Whole-query latency and throughput were not benchmarked locally; no whole-query speedup is claimed. - **Applicability and cost:** Applies to TIMESTAMPTZ years_add/years_sub and date_trunc at second/minute/hour granularity. Larger truncation units keep their existing conversion policy. No allocation, new branch, configuration, or representation change is introduced. ### Testing - Local ASAN BE build: ./build.sh --be -j48 passed. - ./run-be-ut.sh --run --filter='TimeStampTzValueTest.*:VTimestampFunctionsTest.timestamptz_years_*:VTimestampFunctionsTest.years_*_v2_test:VTimestampFunctionsTest.datetrunc_test' -j48: 15 tests passed, including the rerun after test-style fixes. - ./run-regression-test.sh --run -d datatype_p0/timestamptz -s test_timestamptz_year_interval,test_timestamptz_year_partition_pruning,test_timestamptz_dst_fold: all 3 suites passed against the local worktree cluster. - New coverage checks returned offsets, fixed offsets with seconds, microseconds, round trips for both repeated-hour instants, add/sub into repeated times, zero intervals, and a skipped target hour. Existing year-zero source/target and range cases remain covered. - clang-format 16, check-format, build hygiene, and git diff --check passed. Final clang-tidy diff check passed for all six files. ### Release note Reduce duplicate timezone lookups in TIMESTAMPTZ year arithmetic and sub-day date_trunc while preserving their results. ### Check List (For Author) - Behavior changed: No - Does this need documentation: No - Pre-submission status: Ready ### AI Disclosure - Tool: OpenAI Codex - Generated: Code, tests, and this description - Human review: Pending
### Background TIMESTAMPTZ year arithmetic converts the input to the session calendar and resolves the result back to UTC. The shared conversion now shifts a local time forward by the gap length and retains its fractional seconds. ### What problem does this PR solve? Issue Number: None Related PR: apache#70204 (introduced the shared DST gap policy) PR Type: Test Problem Summary: After rebasing onto the updated conversion policy, the year-arithmetic tests still expected local 02:30 in New York's one-hour gap to become 03:00. Update FE and BE expectations to 03:30 and adjust the partition-pruning equality values. Extend that partition across the end of the gap so it continues to exercise a non-monotonic range under the new policy. ### Details - **Failure:** The old tests expected UTC 07:00:00.123456 for the year result; the shared FE and BE conversion returns UTC 07:30:00.123456. The old pruning predicates would no longer select the inserted row. - **Cause:** The previous policy clamped gap inputs to the transition; the upstream fix preserves their position within the gap. - **Stability evidence:** The gap partition now runs from local 02:10:00.900000 to 03:10:00.900000 in the target calendar. Both endpoints resolve to UTC 07:10:00.900000, while the interior row resolves to 07:30:00.100000. Addition and subtraction must both retain the matching partition. Existing generated result files remain unchanged. ### Testing - ./build.sh --be --fe -j48: local ASAN BE and FE builds passed, including FE Checkstyle. - ./run-be-ut.sh --run --filter='TimeStampTzValueTest.*:VTimestampFunctionsTest.timestamptz_years_*:VTimestampFunctionsTest.years_*_v2_test:VTimestampFunctionsTest.datetrunc_test' -j48: 16 tests passed. An initial overlapping FE generation/BE compilation attempt was retried after shared code generation completed. - ./run-fe-ut.sh --run 'TimestampTzLiteralTest,CalendarAddSubMonotonicTest,PartitionPrunerTest': 50 tests passed, no failures or skips. - ./run-regression-test.sh: test_timestamptz_year_interval, test_timestamptz_year_partition_pruning, test_timestamptz_dst_fold, and test_calendar_arithmetic_prune all passed on the isolated worktree cluster. The final gap-end partition bounds were separately rerun successfully. - clang-format 16, check-format, build hygiene, changed-line clang-tidy, and git diff --check passed. ### Release note None ### Check List (For Author) - Behavior changed: No - Does this need documentation: No - Pre-submission status: Ready ### AI Disclosure - Tool: OpenAI Codex - Generated: Test adjustments and this description - Human review: Pending
ccc8288 to
8609fb6
Compare
Local pipeline review — ✅ PASSschema: doris-repo-review/v1
status: PASS
pr: apache/doris#70485
commit: 8609fb6ca24da25d67d14932dd64ff3c880b4ac4
base: e04a785e20a87ef910ec756ff6b4d269298addb6
reviewed_at: 2026-11-17T18:28+08:00
reviewer: jacktengg
model: gpt-6-astra
effort: xhigh
findings: {blocker: 0, major: 0, minor: 1, nit: 0}
rounds: 1
converged: trueNotes for maintainers
Reviewed locally with the |
|
run buildall |
|
PR approved by anyone and no changes requested. |
|
PR approved by at least one committer and no changes requested. |
Background
TIMESTAMPTZ stores a UTC date and time. Calendar-year arithmetic must use the date in the session time zone. DATE and DATETIME already use February 28 when a leap-day target is not in a leap year.
What problem does this PR solve?
Issue Number: None
Related PR: None
PR Type: Bugfix
Problem Summary:
For 2024-02-29 00:00:00 in +08:00, years_add(ts, 1) and years_sub(ts, 1) returned March 1. The stored UTC date is February 28, so UTC year arithmetic does not apply the local leap-day rule. Compute the year change on the local date in both FE constant evaluation and BE column evaluation, then convert the result to UTC.
Details
Testing
Local validation:
Release note
Fix TIMESTAMPTZ year arithmetic for local leap days. Adding or subtracting a year now returns February 28 when the target year has no February 29. For these leap-day inputs, the time and fractional seconds are preserved.
Check List (For Author)
AI Disclosure
Background
What problem does this PR solve?
Issue Number:
Related PR:
PR Type:
Problem Summary:
Details
Testing
Notes for Reviewers
Release note
Check List (For Author)
AI Disclosure