Skip to content

Use every time step in PAA, SAX and 1d-SAX when the length is not a multiple of n_segments - #809

Open
iammanoj807 wants to merge 2 commits into
tslearn-team:mainfrom
iammanoj807:fix/piecewise-segments-use-every-step
Open

iammanoj807 wants to merge 2 commits into
tslearn-team:mainfrom
iammanoj807:fix/piecewise-segments-use-every-step

Conversation

@iammanoj807

@iammanoj807 iammanoj807 commented Nov 12, 2026 •

Copy link
Copy Markdown

Fixes #808.

PAA, SAX and 1d-SAX cut each series into segments of sz // n_segments time steps. When sz is not a multiple of n_segments, the last sz % n_segments time steps were never used. inverse_transform filled them with zeros (PAA, SAX) or left them uninitialized (1d-SAX), and distance_paa could be larger than the Euclidean distance it is meant to lower bound.

Each series is now split into n_segments chunks with boundaries (i * sz) // n_segments, so chunk lengths differ by at most one and every time step is used. The same boundaries are used in:

  • PiecewiseAggregateApproximation._transform and OneD_SymbolicAggregateApproximation._get_slopes;
  • inv_transform_paa, inv_transform_sax and inv_transform_1d_sax;
  • cydist_1d_sax, which already summed over the time steps of each segment.

distance_paa and cydist_sax now weight each segment by its length, sqrt(sum_i size_i * d_i ** 2), instead of multiplying by sqrt(sz / n_segments). This keeps the lower bound, since size_i * (mean difference)^2 <= sum of squared differences within each segment. When sz is a multiple of n_segments, all sizes equal sz / n_segments and the results are the same as before.

I left the default sigma_l = sqrt(0.03 / (sz // n_segments)) of 1d-SAX unchanged. It is a scale for the slope breakpoints, and the segment length it uses is within one time step of the real ones.

Checks

  • When sz is a multiple of n_segments (100/10 and 64/8 with 2-dimensional random walks), the outputs are bit-identical to main:
    • transform and inverse_transform for PAA, SAX and 1d-SAX;
    • distance_paa, distance_sax and distance_1d_sax, which agree to within 1e-12.
  • distance_paa and distance_sax are now below the Euclidean distance in every pair for the 6 settings in the issue (1,000 pairs each). Before, distance_paa was above it in 237 to 500 pairs in each setting where sz is not a multiple of n_segments.

Tests

  • New:

    • test_paa_length_not_multiple_of_n_segments: 19 time steps in 10 segments, transform and inverse_transform.
    • test_sax_inverse_transform_length_not_multiple_of_n_segments: SAX's inverse_transform wasn't covered by any test.
    • test_piecewise_distances_lower_bound_euclidean: 3 settings.

    These fail on main and pass here.

  • test_variable_length_piecewise: the length 5 series [3, 2, 1, 0, -1] with 2 segments now uses all 5 time steps. Its expected PAA changes from [2.5, 0.5] to [2.5, 0.], with the matching SAX and 1d-SAX symbols. The other series in that test have even lengths and are unchanged.

  • pytest tests/test_piecewise.py tests/test_variablelength.py tests/test_metrics.py tests/test_neighbors.py: 143 passed.

I also added a sentence about this to the n_segments docstring of PAA, and an entry to the changelog.

@codecov

codecov Bot commented Nov 12, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 96.91%. Comparing base (9ee24b5) to head (043ae20).

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #809   +/-   ##
=======================================
  Coverage   96.90%   96.91%           
=======================================
  Files          91       91           
  Lines        9674     9701   +27     
=======================================
+ Hits         9375     9402   +27     
  Misses        299      299           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@iammanoj807
iammanoj807 force-pushed the fix/piecewise-segments-use-every-step branch from 7560a11 to bcad2e0 Compare November 13, 2026 21:59
When the series length was not a multiple of n_segments, segments had
sz // n_segments time steps and the last sz % n_segments were ignored.
inverse_transform left them empty, and distance_paa could exceed the
Euclidean distance it lower bounds. Segments now have boundaries
(i * sz) // n_segments, and the PAA and SAX distances weight each
segment by its length.
@iammanoj807
iammanoj807 force-pushed the fix/piecewise-segments-use-every-step branch from bcad2e0 to 043ae20 Compare November 16, 2026 19:34

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

PAA and SAX ignore the last time steps when the series length is not a multiple of n_segments

1 participant