Skip to content

Reimplement Shamir secret sharing in verified Rust - #8614

Draft
Amaury Chamayou (achamayou) wants to merge 20 commits into
mainfrom
achamayou-upstream-shamir-draft
Draft

Amaury Chamayou (achamayou) wants to merge 20 commits into
mainfrom
achamayou-upstream-shamir-draft

Conversation

@achamayou

@achamayou Amaury Chamayou (achamayou) commented Oct 21, 2026 •

Copy link
Copy Markdown
Member

Motivation

Move recovery-share generation and reconstruction into a small Rust computational core with an extracted Lean refinement, while retaining the original C++ implementation and full bidirectional share compatibility. No issue is linked.

Implementation summary

  • Add the dependency-free ccf-shamir crate, which forbids unsafe code and implements field arithmetic, polynomial evaluation, share-coordinate assignment, and interpolation.
  • Keep entropy sampling and C++ object handling in src/crypto/sharing_rust.cpp, separate from the raw-pointer bridge in src/rust/src/shamir.rs. Recovery writes directly into the caller's output array instead of returning an additional secret-bearing array.
  • Reproduce the production core's Lean model with pinned Aeneas/Charon tooling and prove arithmetic safety, sharing and recovery refinement to CSLib, and preservation of the output on recovery errors. CI checks extraction reproducibility and the kernel/axiom dependency closure of all project declarations.
  • Generate the C ABI header from the Rust declarations with pinned cbindgen, and run the bridge tests under Miri's Stacked Borrows and Tree Borrows models in CI. Generated lint exceptions cover only the C++ modernization and enum-size checks that conflict with the C ABI.
  • Retain the original C++ backend in src/crypto/sharing.cpp. USE_RUST_SHAMIR defaults to ON; OFF selects C++. The unselected backend is linked separately into sharing_test for interoperability checks.
  • Build both Lean import roots before formatting, including the Generated root imported by ListDeclarations.lean.
  • Document the compatibility requirement, backend selector, proof boundary, and reproduction commands in the Shamir README.

Safety and compatibility

The C++ API, 44-byte serialized share layout, byte order, field, ten-limb representation, coefficient sampling, HKDF derivation, input reduction, and ignoring shares beyond the recovery threshold are preserved. Valid shares must recover in both directions with identical secrets and derived keys; independently generated sharings are not interchangeable.

The retained C++ arithmetic comes from main revision 41645c55675d246605b2e26d45f4282de3d83fba. Its only runtime changes reject zero recovery thresholds and generation counts reaching the field characteristic, matching the Rust backend. No consensus or KV logic is changed.

The FFI bridge rejects null, misaligned, overlapping, and address-wrapping buffers and reports named errors. Allocation validity, initialization, lifetime, and concurrent-access obligations remain with the caller. The Lean proof boundary excludes that unsafe bridge, the C++ adapter, OpenSSL, the real entropy source, machine-code timing and erasure, the compiler, and the extractor. The C++ backend is not covered by the Rust refinement proofs.

Validation

Local validation has passed six standalone Rust compatibility tests, four Rust FFI tests, and the complete sharing_test with each backend selected in a Debug build with BUILD_TESTS=ON and LONG_TESTS=ON. Both configurations exercise serialized-share exchange in both directions, all threshold-sized subsets of seven-share distributions, full shares, recovered bytes, and derived keys. Archive inspection confirmed that ccfcrypto contains only the selected backend's C++ translation unit in each configuration.

The generated header reproduces locally with cbindgen 0.29.4 and passes C11 and C++23 layout checks. A before/after regression with clang-tidy 18.1.8 reproduces the original included-header failures and passes with the regenerated header under the unchanged full lint configuration. The lint follow-up changes only generated comments, not ABI declarations. Targeted formatting, release-note, prose, ASCII, and workflow-routing checks have also passed.

On the latest test-only correction, cmake --build /tmp/ccf-shamir-final-build-2fb27b85 --target sharing_test -j2 and ctest --test-dir /tmp/ccf-shamir-final-build-2fb27b85 -R '^sharing_test$' --no-tests=error --output-on-failure --timeout 600 both passed with USE_RUST_SHAMIR=ON and OFF. The complete test took 26.75 seconds and 18.62 seconds respectively. These Debug builds used Clang 21, BUILD_TESTS=ON, LONG_TESTS=ON, and CLANG_TIDY=OFF; lint regressions were checked independently.

On revision 356630e72, CI passed the full Lean models/proofs job, including export and kernel/axiom checking of all 436 Shamir project declarations and their dependencies. The kernel accepted the solution using only propext, Quot.sound, and Classical.choice. Rust extraction, both Miri modes, generated-header reproducibility, VMSS Virtual B/C, ACI SNP Genoa, and the CLA check also passed. The generated-header lint and Lean import-root failures from the preceding revision were resolved.

The remaining Virtual A failure on 356630e72 was Clang 18.1.8/libstdc++ 13.2.0 deducing function-type members for std::pair in the interoperability test. A local minimal reproducer using the same Clang parser and Azure Linux standard-library headers reproduced both original compilation errors; explicit function addresses passed. Revision e9146cb8d applies that correction without changing production code, test coverage, or assertions.

On e9146cb8d, the complete Continuous Integration workflow passed: Virtual A/B/C and ACI SNP Genoa are green. Virtual A passed ./scripts/ci-checks.sh, the full Debug build with clang-tidy, C++ case-scope checks, Python package tests, ./tests.sh --output-on-failure -L unit -j16 (65/65 tests, including sharing_test), and ./tests.sh --timeout 360 --output-on-failure -L bucket_a -j2 (3/3 tests, including lts_compatibility). Rust extraction, both Miri modes, generated-header reproducibility, and CLA also passed on this revision.

The only pending check on e9146cb8d is the full Lean models/proofs rerun; it passed on 356630e72, and this revision changes only C++ test function addresses. Long, sanitizer, and benchmark jobs were skipped, not passed. Required checks must finish successfully before merge.

Local limitations remain distinct from CI results. The complete local tidy-enabled build, cmake --build /tmp/ccf-shamir-ci-build-2fb27b85 --target sharing_test -j2, encountered a clang-tidy 18.1.8 crash in unchanged vendored fmt/format.h; the complete tidy-enabled CI build now passes. Full network targets and Miri were not rerun locally. A separate sphinx-build --fail-on-warning -b html doc doc/html build was not run locally. The Windows checkout's CRLF scripts affect direct WSL checks.

The changelog includes the introducing upstream PR reference (#8614).

Replace the C++ Shamir arithmetic in src/crypto/sharing.cpp with a safe,
dependency-free Rust core (src/rust/shamir) exposed through a C ABI from
ccf-rs, and keep sampling, cleansing and the Share type in the C++ adapter.

Extract a Lean model of the Rust core with Aeneas (lean/shamir/Generated)
and prove that evaluation and recovery refine CSLib's Shamir share and
reconstruct definitions. scripts/extract-shamir.sh reproduces the
extraction; scripts/check-shamir.sh runs the kernel and axiom audit over
every project declaration and its dependency closure.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The Lean proofs show the core arithmetic cannot overflow, so the checks
cost nothing on valid executions. Keeping them in release builds turns any
violated assumption into an abort instead of a silently wrapped value.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Validate every buffer argument before it is turned into a slice: null,
misaligned, empty, oversized, address-wrapping, or mutually overlapping
buffers now return status 1 instead of violating slice preconditions.
Dangling pointers remain the caller's obligation.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Replace the int32_t status codes with a ccf_shamir_status enum shared by
the header and a repr(C) Rust enum, so each outcome is named on both
sides and the C++ adapter switches over enumerators rather than magic
numbers. Mark the entry points nodiscard so a silently ignored failure
is a compile error under -Werror. Values and the ABI are unchanged.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Drop the derived Copy and Debug implementations from Share so that
copies of secret material are visible as explicit clone calls and
shares can never be printed through formatting machinery. Only the
source-line comments in the extracted Lean model change; the model
itself and its proofs are unaffected.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Take the recovered secret as a pointer to a ten-limb array so the
length is part of the signature rather than a documented convention,
and allocate the Lagrange weights workspace inside the FFI so callers
can no longer pass a mis-sized one. Allocation failure is reported as
CCF_SHAMIR_OUT_OF_MEMORY, which the adapter maps to std::bad_alloc,
matching the previous behaviour of allocating the workspace in C++.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The per-share evaluation loop, the assignment of share coordinates
(x = i + 1), and the share-count and threshold bounds checks now live in
the safe, Aeneas-extracted ccf-shamir crate as share_limb. The FFI entry
point ccf_shamir_share_limb only marshals buffers and status codes, and
the C++ adapter no longer computes coordinates or iterates over shares,
so the sharing side of the ABI has a single source of truth for the
x coordinates and no arithmetic outside the verified core.

SharingRefinement is restated for share_limb: filling a limb assigns
coordinate k + 1 to share k, stores CSLib's share value for that
coordinate in the requested limb, and leaves every other limb
unchanged. The new share_limb_loop_spec and share_limb_spec proofs are
kernel-checked with no axioms beyond propext, Quot.sound and
Classical.choice.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
recover now takes the caller's secret array by mutable reference and
fills it only after every Lagrange weight has been computed. The FFI no
longer materialises a temporary copy of the recovered secret and the
OPENSSL_cleanse foreign call it needed is gone, so the C++ caller's
cleansed buffer is the only place the recovered secret ever lives.

The Aeneas model threads the array through recover explicitly, which
lets the "unchanged on error" contract of the C ABI be proved rather
than inspected: recovery_error_preserves_secret shows that for any
inputs, a recovery that reports an error returns the caller's array
untouched, with no preconditions. RecoveryRefinement is restated for
the in-place signature and holds for any initial array contents.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
src/crypto/sharing_ffi.h is now rendered by cbindgen 0.29.4 from the
#[repr(C)] types and extern "C" functions in src/rust/src/shamir.rs via
scripts/generate-shamir-header.sh, which also formats it with the pinned
clang-format. Signatures, status codes, the limb count and the share layout
can no longer drift between the two sides: the Lean workflow fails if the
committed header differs from the generated one.

Impact:
- CCF_SHAMIR_LIMBS is defined by the header (from a LIMBS constant in the
  binding crate, compile-time asserted equal to the verified core's) and used
  in the share layout and the recover signature, with a C++ static_assert.
- Status enumerators are now CCF_SHAMIR_STATUS_* (cbindgen's name-prefixed
  rendering) and the C++ adapter is updated accordingly.
- All three entry points carry #[must_use] on the Rust side, which is what
  emits CCF_SHAMIR_NODISCARD in the header.
- The C contract documentation lives on the Rust declarations and is copied
  into the header, so there is a single place to keep it accurate.
- Re-extracted Lean model: only Source: line comments changed; proofs and the
  declaration check are unaffected.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The only unsafe code in the Shamir path is the pointer marshalling in
src/rust/src/shamir.rs, which the Lean proofs deliberately do not cover.
Its tests, including the rejected null, misaligned, overlapping and
address-wrapping buffers and the error paths, now run under Miri in the
Lean workflow with the default Stacked Borrows model and again with Tree
Borrows and strict provenance.

Impact: undefined behaviour in the FFI layer (out-of-bounds reads of
exactly sized share buffers, aliasing violations, misaligned accesses,
integer-to-pointer casts) now fails CI instead of going unnoticed. The
current layer is clean under all three configurations; the verified core
also passes Miri but is forbid(unsafe_code), so only the FFI crate runs
in CI.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Preserve verified generation and in-place recovery alongside selectable C++ and Rust backends, bidirectional compatibility coverage, and upstream preparation.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The job now also installs cbindgen and builds a Miri sysroot before
checking the generated header and running the C ABI tests under Miri.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Integrate concurrent parent work and apply its generated-header status names and limb-count check to the separate Rust adapter without changing the retained C++ backend.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Generate narrow clang-tidy exceptions for the C/Rust ABI without changing its declarations. Build the Generated import root before formatting ListDeclarations.lean.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Use explicit function addresses so Clang 18 with libstdc++ 13 deduces pointer members rather than invalid function members in the backend pairs.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant