Repository navigation
Use every time step in PAA, SAX and 1d-SAX when the length is not a multiple of n_segments - #809
Open
iammanoj807 wants to merge 2 commits into
Open
iammanoj807 wants to merge 2 commits into
iammanoj807 wants to merge 2 commits into
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #809 +/- ##
=======================================
Coverage 96.90% 96.91%
=======================================
Files 91 91
Lines 9674 9701 +27
=======================================
+ Hits 9375 9402 +27
Misses 299 299 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
iammanoj807
force-pushed
the
fix/piecewise-segments-use-every-step
branch
from
November 13, 2026 21:59
7560a11 to
bcad2e0
Compare
When the series length was not a multiple of n_segments, segments had sz // n_segments time steps and the last sz % n_segments were ignored. inverse_transform left them empty, and distance_paa could exceed the Euclidean distance it lower bounds. Segments now have boundaries (i * sz) // n_segments, and the PAA and SAX distances weight each segment by its length.
iammanoj807
force-pushed
the
fix/piecewise-segments-use-every-step
branch
from
November 16, 2026 19:34
bcad2e0 to
043ae20
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #808.
PAA, SAX and 1d-SAX cut each series into segments of
sz // n_segmentstime steps. Whenszis not a multiple ofn_segments, the lastsz % n_segmentstime steps were never used.inverse_transformfilled them with zeros (PAA, SAX) or left them uninitialized (1d-SAX), anddistance_paacould be larger than the Euclidean distance it is meant to lower bound.Each series is now split into
n_segmentschunks with boundaries(i * sz) // n_segments, so chunk lengths differ by at most one and every time step is used. The same boundaries are used in:PiecewiseAggregateApproximation._transformandOneD_SymbolicAggregateApproximation._get_slopes;inv_transform_paa,inv_transform_saxandinv_transform_1d_sax;cydist_1d_sax, which already summed over the time steps of each segment.distance_paaandcydist_saxnow weight each segment by its length,sqrt(sum_i size_i * d_i ** 2), instead of multiplying bysqrt(sz / n_segments). This keeps the lower bound, sincesize_i * (mean difference)^2 <= sum of squared differenceswithin each segment. Whenszis a multiple ofn_segments, all sizes equalsz / n_segmentsand the results are the same as before.I left the default
sigma_l = sqrt(0.03 / (sz // n_segments))of 1d-SAX unchanged. It is a scale for the slope breakpoints, and the segment length it uses is within one time step of the real ones.Checks
szis a multiple ofn_segments(100/10 and 64/8 with 2-dimensional random walks), the outputs are bit-identical tomain:transformandinverse_transformfor PAA, SAX and 1d-SAX;distance_paa,distance_saxanddistance_1d_sax, which agree to within 1e-12.distance_paaanddistance_saxare now below the Euclidean distance in every pair for the 6 settings in the issue (1,000 pairs each). Before,distance_paawas above it in 237 to 500 pairs in each setting whereszis not a multiple ofn_segments.Tests
New:
test_paa_length_not_multiple_of_n_segments: 19 time steps in 10 segments,transformandinverse_transform.test_sax_inverse_transform_length_not_multiple_of_n_segments: SAX'sinverse_transformwasn't covered by any test.test_piecewise_distances_lower_bound_euclidean: 3 settings.These fail on
mainand pass here.test_variable_length_piecewise: the length 5 series[3, 2, 1, 0, -1]with 2 segments now uses all 5 time steps. Its expected PAA changes from[2.5, 0.5]to[2.5, 0.], with the matching SAX and 1d-SAX symbols. The other series in that test have even lengths and are unchanged.pytest tests/test_piecewise.py tests/test_variablelength.py tests/test_metrics.py tests/test_neighbors.py: 143 passed.I also added a sentence about this to the
n_segmentsdocstring of PAA, and an entry to the changelog.