Repository navigation
fix: return the last observation from PandasQuote - #2404
Open
朱甲文(Jiawen Zhu) (andone-07) wants to merge 1 commit into
Open
朱甲文(Jiawen Zhu) (andone-07) wants to merge 1 commit into
朱甲文(Jiawen Zhu) (andone-07) wants to merge 1 commit into
Conversation
There was a problem hiding this comment.
🟢 Approval recommended
No active review issues were identified for the change to last-observation selection.
No open issues 🎉
What changed in this PR
Fixes PandasQuote.get_data(method="last") to return the final observation in the requested time window, matching NumpyQuote without changing general resampling behavior.
Changes:
- Uses positional selection for
"last", preserving a trailing NaN. - Documents the distinction between
"last"and"ts_data_last". - Adds 17 regression cases covering backend parity, input ordering, time windows, NaN values, and unchanged aggregation behavior.
| File | Description |
|---|---|
| tests/backtest/test_pandas_quote.py | Adds regression coverage for last-observation selection and existing behavior. |
| qlib/backtest/high_performance_ds.py | Implements positional "last" selection and documents NaN handling. |
Review effort: Balanced
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Handle
method="last"inPandasQuote.get_datawith positional selection after the requested time window has been sliced. Preserve a final NaN, matchingNumpyQuote, and document the distinction fromts_data_last.Motivation and Context
BaseQuote.get_datadocumentslastas a supported aggregation returning the final observation.PandasQuotecurrently passes that string toresam_ts_data, which invokesSeries.last()without its required date offset. A three-day quote containing[1.0, 2.0, 3.0]raisesTypeError: NDFrame.last() missing 1 required positional argument: 'offset', whileNumpyQuotereturns3.0for the same timestamp-bounded query.The fix is limited to the optional Pandas quote backend. It does not change general resampling behavior or the existing default Numpy backend.
lastretains the final observation even when it is NaN;ts_data_lastcontinues to select the last non-NaN observation.How Has This Been Tested?
cd tests && python -m pytest . -m "not slow" --durations=20: 708 passed, 1 skipped, 13 deselected, 495 warnings in 1139.58s (0:18:59).python -m pytest tests/test_all_pipeline.py -q: 3 passed, 42 warnings in 152.75s (0:02:32).pip check, fullmake pylint, flake8, mypy, documentation build, new-test flake8 and sequential Black checks on 350 Python files pass.Validation scope: Linux/Python 3.12.14 with pandas 2.3.3. The high-frequency test is skipped by upstream. Of the 13 deselected slow tests, three pipeline tests passed separately and the other ten were not run. The standard Black/nbqa entrypoints are blocked by the environment's multiprocessing socket restriction; source formatting was checked with the same Black API sequentially. Notebook and cross-platform CI checks are not claimed.
The 17 regression cases cover backend parity, sorted/reversed input, ordinary/interior/trailing/all NaN values, bounded/open-ended/single-point/empty windows, and unchanged mean/sum/ts_data_last/no-aggregation behavior. Inputs are deterministic synthetic quote data exercised through the real APIs, not mocks; they demonstrate API behavior, not investment performance.
Types of changes
Prepared with AI assistance.