Skip to content

fix(encryption): check the AGS1 stream header on read - #3700

Open
xanderbailey wants to merge 2 commits into
apache:mainfrom
xanderbailey:xb/check-ags1-header
Open

xanderbailey wants to merge 2 commits into
apache:mainfrom
xanderbailey:xb/check-ags1-header

Conversation

@xanderbailey

@xanderbailey xanderbailey commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

What changes are included in this PR?

AesGcmFileRead never read the 8-byte AGS1 header, so a stream with the wrong magic or an unsupported plaintext block size decrypted as if it were valid. Java's AesGcmInputStream.validateHeader rejects both.

The header is not authenticated, so a read that covers the first block now includes the header in its single I/O call and checks it before decrypting. Reads that start past the first block are unchanged.

Are these changes tested?

Yes, a unit test for a wrong magic and an unsupported block size.

AI Disclosure

Written with Claude Code.

AesGcmFileRead skipped the 8-byte header, so a stream with the wrong
magic or an unsupported plaintext block size was decrypted as if it
were valid AGS1. The header is not authenticated, so it is now read and
checked along with the first block, matching Java's AesGcmInputStream.
@xanderbailey

Copy link
Copy Markdown
Contributor Author

@kevinjqliu this is part of getting rust bindings for encrypt-decrypt so tagging you in for review.

@mbutrovich mbutrovich left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @xanderbailey. Reading the header in the same call as block 0 keeps the check free of extra I/O, and I'm fine with reads that start past block 0 skipping it.

Comment thread crates/iceberg/src/encryption/stream.rs Outdated
Comment on lines +95 to +108
fn check_stream_header(header: &[u8]) -> Result<()> {
let (magic, plain_block_size) = header.split_at(GCM_STREAM_MAGIC.len());
if magic != GCM_STREAM_MAGIC {
return Err(invalid_data!(
"Invalid AGS1 header: magic \"{}\" does not match \"AGS1\"",
magic.escape_ascii()
));
}

let plain_block_size = u32::from_le_bytes(
plain_block_size
.try_into()
.map_err(|_| invalid_data!("Invalid AGS1 header length: {}", header.len()))?,
);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

check_stream_header takes any slice and has an error for a bad length, but split_at(4) on line 96 runs first and panics when the slice is shorter than 4 bytes. check_stream_header(b"AG") panics, and check_stream_header(b"AGS1\x00\x00\x10") returns the length error. Could we convert to a fixed-size array up front, so every bad length gets the error and the rest of the function can't panic?

Suggested change
fn check_stream_header(header: &[u8]) -> Result<()> {
let (magic, plain_block_size) = header.split_at(GCM_STREAM_MAGIC.len());
if magic != GCM_STREAM_MAGIC {
return Err(invalid_data!(
"Invalid AGS1 header: magic \"{}\" does not match \"AGS1\"",
magic.escape_ascii()
));
}
let plain_block_size = u32::from_le_bytes(
plain_block_size
.try_into()
.map_err(|_| invalid_data!("Invalid AGS1 header length: {}", header.len()))?,
);
fn check_stream_header(header: &[u8]) -> Result<()> {
let header: &[u8; GCM_STREAM_HEADER_LENGTH as usize] = header
.try_into()
.map_err(|_| invalid_data!("Invalid AGS1 header length: {}", header.len()))?;
let magic = &header[..GCM_STREAM_MAGIC.len()];
if magic != GCM_STREAM_MAGIC {
return Err(invalid_data!(
"Invalid AGS1 header: magic \"{}\" does not match \"AGS1\"",
magic.escape_ascii()
));
}
let plain_block_size = u32::from_le_bytes([header[4], header[5], header[6], header[7]]);

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Went a step further: it now takes &[u8; 8], split off with split_first_chunk, so a short header doesn't compile. Fixed in c52cf46.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even better!

Comment on lines +972 to +1002
async fn test_invalid_header_is_rejected() {
let key = b"0123456789abcdef";
let aad_prefix = b"test-aad-prefix!";
let plaintext = b"sensitive data";
let encrypted = encrypt_ags1(plaintext, &make_cipher(key), aad_prefix);

for (header, message) in [
(
*b"AGS2\x00\x00\x10\x00",
"Invalid AGS1 header: magic \"AGS2\" does not match \"AGS1\"",
),
(
*b"AGS1\x00\x02\x00\x00",
"Unsupported AGS1 block size: 512 (expected 1048576)",
),
] {
let mut tampered = encrypted.clone();
tampered[..GCM_STREAM_HEADER_LENGTH as usize].copy_from_slice(&header);
let reader = AesGcmFileRead::new(
memory_reader(tampered),
Arc::new(make_cipher(key)),
aad_prefix.as_slice().into(),
encrypted.len() as u64,
)
.unwrap();

let err = reader.read(0..plaintext.len() as u64).await.unwrap_err();
assert_eq!(err.kind(), ErrorKind::DataInvalid);
assert_eq!(err.message(), message);
}
}

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we also run this against the empty stream (header plus one empty block)? It reaches the header check only through the plain_stream_size != 0 exception in the early return at lines 299-303. Without the header check, read(0..0) on an empty stream with a tampered header returns Ok with no bytes, because GCM doesn't cover the header.

Suggested change
async fn test_invalid_header_is_rejected() {
let key = b"0123456789abcdef";
let aad_prefix = b"test-aad-prefix!";
let plaintext = b"sensitive data";
let encrypted = encrypt_ags1(plaintext, &make_cipher(key), aad_prefix);
for (header, message) in [
(
*b"AGS2\x00\x00\x10\x00",
"Invalid AGS1 header: magic \"AGS2\" does not match \"AGS1\"",
),
(
*b"AGS1\x00\x02\x00\x00",
"Unsupported AGS1 block size: 512 (expected 1048576)",
),
] {
let mut tampered = encrypted.clone();
tampered[..GCM_STREAM_HEADER_LENGTH as usize].copy_from_slice(&header);
let reader = AesGcmFileRead::new(
memory_reader(tampered),
Arc::new(make_cipher(key)),
aad_prefix.as_slice().into(),
encrypted.len() as u64,
)
.unwrap();
let err = reader.read(0..plaintext.len() as u64).await.unwrap_err();
assert_eq!(err.kind(), ErrorKind::DataInvalid);
assert_eq!(err.message(), message);
}
}
async fn test_invalid_header_is_rejected() {
let key = b"0123456789abcdef";
let aad_prefix = b"test-aad-prefix!";
for plaintext in [b"".as_slice(), b"sensitive data".as_slice()] {
let encrypted = encrypt_ags1(plaintext, &make_cipher(key), aad_prefix);
for (header, message) in [
(
*b"AGS2\x00\x00\x10\x00",
"Invalid AGS1 header: magic \"AGS2\" does not match \"AGS1\"",
),
(
*b"AGS1\x00\x02\x00\x00",
"Unsupported AGS1 block size: 512 (expected 1048576)",
),
] {
let mut tampered = encrypted.clone();
tampered[..GCM_STREAM_HEADER_LENGTH as usize].copy_from_slice(&header);
let reader = AesGcmFileRead::new(
memory_reader(tampered),
Arc::new(make_cipher(key)),
aad_prefix.as_slice().into(),
encrypted.len() as u64,
)
.unwrap();
let err = reader.read(0..plaintext.len() as u64).await.unwrap_err();
assert_eq!(err.kind(), ErrorKind::DataInvalid);
assert_eq!(err.message(), message);
}
}
}

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done, thanks. Checked it fails on the empty stream without the header check.

check_stream_header now takes &[u8; 8], split off with
split_first_chunk, so it cannot panic on a short slice. The header test
also covers the empty stream.

@mbutrovich mbutrovich left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @xanderbailey, taking the header as a fixed-size array is a nice improvement on what I suggested.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants