Skip to content

Fix integer overflow when clamping large size_t options (#462) - #1371

Open
xhon-pelushi wants to merge 2 commits into
google:mainfrom
xhon-pelushi:fix-462-cliptorange-overflow
Open

xhon-pelushi wants to merge 2 commits into
google:mainfrom
xhon-pelushi:fix-462-cliptorange-overflow

Conversation

@xhon-pelushi

Copy link
Copy Markdown

Problem

SanitizeOptions() clamps size options with ClipToRange():

template <class T, class V>
static void ClipToRange(T* ptr, V minvalue, V maxvalue) {
  if (static_cast<V>(*ptr) > maxvalue) *ptr = maxvalue;
  if (static_cast<V>(*ptr) < minvalue) *ptr = minvalue;
}
...
ClipToRange(&result.write_buffer_size, 64 << 10, 1 << 30);

write_buffer_size/max_file_size/block_size are size_t, but the bound literals are int, so V is int and static_cast<V>(*ptr) truncates. A write_buffer_size of 2 GiB casts to a negative int, so it is clamped to the minimum (64 KiB) rather than the 1 GiB maximum; values >= 4 GiB wrap modulo 2^32 (#462).

Fix

Compare in the pointee's type T, casting the bounds up to T:

if (*ptr > static_cast<T>(maxvalue)) *ptr = static_cast<T>(maxvalue);
if (*ptr < static_cast<T>(minvalue)) *ptr = static_cast<T>(minvalue);

All the min/max bounds here are positive, so casting them to size_t is well-defined, and int options (max_open_files) are unchanged.

Test

DBTest.SanitizeOptionsOverflow calls SanitizeOptions() with 2 GiB and 4 GiB sizes and asserts they clamp to the documented maxima (1 GiB / 4 MiB), plus a negative max_open_files clamping to the minimum. It fails on main (2 GiB clamps to the minimum) and passes with this change.

Fixes #462

The regression test is adapted from the earlier PR #1356, which the author self-closed.

M0oo0ry and others added 2 commits April 27, 2026 19:46
ClipToRange() cast the pointee to the bound type V before comparing. The bounds
are int literals (e.g. 1 << 30), so V is deduced as int, and a size_t option of
2 GiB or more became negative under the cast: write_buffer_size and max_file_size
were then clamped to the minimum instead of the maximum, and values >= 4 GiB
wrapped modulo 2^32.

Compare in the pointee's type T instead, casting the bounds to T. Options at or
below the limits are unaffected, and int options such as max_open_files still
clamp as before.

Adds DBTest.SanitizeOptionsOverflow.

Fixes google#462
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

overflow bug in SanitizeOptions

2 participants