Skip to content

test(e2e): decide which e2e/rust tests to port and in what order #5297

Open

Description

@matthewgrossman

User Story

I'm retiring e2e/rust (about 70 test files). Not every file should move: some duplicate tests already on testkit, some are benchmarks, and some test the environment instead of the product. I need a per-file decision and a batch order so each migration PR stays small.

Description

Sort every file in e2e/rust/tests/ into keep, drop or merge, and assign kept files a destination and a batch. Judge each file with three questions:

  1. Does a ported test already assert this?
  2. Does it test product behavior, or the test environment?
  3. Would a regression here surprise us in production?

If the answer to the third is no, drop it.

Starting point, from file names and a skim of a few files (not yet from reading every test body):

  • Conformance candidates (every driver must pass): policy and network enforcement (landlock, bypass_detection, no_proxy, policy_activation, live_policy_update, proxy_egress_pipeline, forward_proxy_*, transparent_tcp, websocket_conformance), sandbox API (sandbox_labels, sandbox_templates, ssh_host_identity, port_forward, upload_create, rootfs_tar, ephemeral_cleanup, listener_lifetime), providers and credentials (provider_*, credential_gating, stable_external_placeholders, provider_token_exchange, service_bearer_passthrough, settings_management), workspace_lifecycle.
  • Feature or driver candidates: oidc_pkce, additional_ca, config_push, local_driver_token_restart, otlp_relay, ocsf_*, mcp_sessionless, default_image; the kubernetes_*, workspace_namespace_* and credential_drivers files; podman_*, user_namespaces; docker_*, gateway_start; vm_*.
  • Possible duplicates to diff before porting:
    • sandbox_lifecycle.rs: conformance lifecycle already covers restart of a stopped sandbox and delete while stopped. The canonical-main, detached-main, exec-recovery and large-output tests don't appear to be covered.
    • host_gateway_alias.rs and podman_host_gateway.rs: their first test checks the same host.openshell.internal contract as the host-gateway suite from test(tmachine): prepare runner environments for the e2e/rust migration #5277. host_gateway_alias.rs also covers EOF handling and provider bindings, which are separate.
      • podman_host_gateway.rs is not a pure duplicate. It also asserts that host.openshell.internal stays reachable after sandbox stop and sandbox start, which the host-gateway suite doesn't cover. That contract is driver-agnostic, so port it to the host-gateway suite (or conformance lifecycle) on every driver that supports stop/start. Its other assertion, the policy-DNS relay in /etc/resolv.conf (nameserver 127.0.0.53), is Podman-specific: keep it as a Podman driver test. Don't drop the file as a duplicate until both have a home.
    • user_namespaces.rs and podman_oci_identity.rs: overlap the ports from test(e2e): migrate the features and drivers suites to testkit #5137.
  • Drop or keep out of CI: live_internet_traffic_perf and internet_network_perf (opt-in benchmarks that use the live Internet); docker_preflight (checks openshell doctor output without Docker, so it needs an environment with no Docker).
  • Needs a decision on the setup: the three *_corporate_proxy.rs files (about 2,400 lines, mostly environment setup), gpu, core_dump_hardening, edge_tunnel_e2e.
  • Heavy and last: kubernetes_shared_supervisor*, kubernetes_ha_rebalancing. These are slow, and the flake census shows Kubernetes lifecycle races, so port them last or only the ones guarding recent bugs.

Batches and their dependencies are in the "Migrating e2e/rust" table in #5281: a credential_gating pilot, then Podman, Docker and the common driver-agnostic files (in parallel, since they touch different crates), then gateway reconfiguration, then Kubernetes. Record each file's batch in that table's terms.

Context

Part of #5281.

Definition of Done

  • Every file in e2e/rust/tests/ has a recorded decision (port, merge, drop, keep on host) with a reason.
  • Each kept file has a destination family and a batch.
  • Overlaps above are resolved by reading the test bodies.

Activity

  1. matthewgrossman commented on Nov 7, 2026

    @matthewgrossman
    MemberAuthor

    I read every test body in e2e/rust/tests/ and recorded a decision, destination and PR for each. No E2E tests were run for this plan.

    Result: 222 entries (tests, unit tests and two helper/module entries) across 70 files.

    Decision Count
    Port as-is or with small fixes 111
    Merge into another test (every assertion kept) 72
    Split into two or more tests or destinations 15
    Drop 12
    Drop, replaced by a crate test 2
    Keep out of CI (live-Internet benchmark) 1
    Already owned elsewhere (#5318: 6 credential_gating tests; #4606: 3 upload_create tests) 9

    Milestones. Milestone 1, these 9 PRs, deletes the e2e/rust tests and harness. Deleting the host-lane wrapper scripts (e2e/rust/e2e-*.sh, e2e/support/*) is a later, separate milestone that needs a KVM host provisioner. Every environment tmachine can host runs on tmachine; host lanes remain only for what it can't (the VM driver and GPU).

    How environments reach tests

    An environment (tmachine, or for VM and GPU the existing host-lane wrapper script) publishes OPENSHELL_TEST_<SERVICE>_* variables. A tests/support/<service> helper reads them and fails, never skips, if one is missing (tests/support/keycloak works this way today). Which tests run where is set by the rows in .github/workflows/e2e-suites.yml; until the script-deletion milestone, the VM and GPU wrapper scripts choose their own suites. Each PR adds the helpers and environment variants its tests need, and the last PR to touch a legacy file deletes it.

    Decisions needed

    • A. How do tests restart the gateway? Proposed: the environment publishes OPENSHELL_TEST_GATEWAY_STOP and OPENSHELL_TEST_GATEWAY_START, and tests/support/gateway_control runs them. The restart test goes in features/gateway-restart, not conformance, because it also checks that the container or VM survives. Blocks part of PR6.
    • B. Where should VM rows run long term? Proposed: on a KVM host provisioner (a GitHub runner with KVM, as today) that publishes the same variables, since the VM driver can't run nested in a tmachine VM. This is for the script-deletion milestone and blocks none of the 9 PRs; until then, e2e-vm.sh runs the VM suites on the existing KVM lane.
    • C. Should the OIDC environment keep mTLS? Proposed: yes, so testkit connects as it does today and tests act as Keycloak users through tests/support/keycloak. The alternative is an OIDC-only mode in testkit. Blocks PR7.
    • D. Are the drops OK? Each is marked drop with a reason in its PR section (PR1, PR5, PR6, PR7, PR8). Approve them as a batch or strike some. Merges keep every assertion and need no approval.

    PRs

    Each PR has a collapsed section below with its test-by-test table, helpers, deleted files, defaults and Definition of Done. Each section can be filed as its own issue or handed to an agent as is.

    PR Scope Tests Can start Waits on
    PR1 Sandbox lifecycle and API → conformance 37 + 1 unit now D for its drops
    PR2 Network, L7, policy update → conformance 30 after #5318 #5318
    PR3 Providers and workspaces 5 after #5318 #5318, PR1
    PR4 CLI-driven exec, attach, SSH host identity 11 now —
    PR5 Docker and Podman feature tests 13 after PR2 PR2, #5318, D
    PR6 Gateway configuration and restart 20 after PR5 PR2, PR4, PR5, #5318, A, D; #5265 for the Podman restart row
    PR7 OIDC authorization (43 → 13 tests) 43 after PR3, PR4 PR3, PR4, C, D
    PR8 Kubernetes driver 28 + 6 unit after PR1, PR3, PR4 PR6, #5318, #5295, D
    PR9 VM and GPU 12 + 5 unit after PR1, PR2, PR5 PR6, #5295

    Order: PR1 and PR4 now; PR2 and PR3 after #5318; then PR5; then PR6 and PR7; then PR8 and PR9. #5295 (per-test timeouts and test groups) matters only for PR8 and PR9. Whichever PR merges last also deletes the e2e/rust test crate (tests, harness, Cargo.toml); the wrapper scripts stay until the script-deletion milestone.

    Mapping to the batches in #5281: PR1–PR4 are "Common", PR5 is "Podman" and "Docker" combined (most of those tests check behavior both engines share), PR6 and PR7 are "Gateway reconfiguration", PR8 is "Kubernetes", PR9 is "Host-only".

    Every file, its decision and its PR
    File PR Decision Destination
    additional_ca.rs PR6 split; tamper and CA-removal checks dropped features/additional-ca, drivers/*
    bypass_detection.rs PR2 merge conformance
    config_push.rs PR6 port features/config-push
    core_dump_hardening.rs PR1 port conformance
    credential_drivers.rs PR8 port drivers/kubernetes
    credential_gating.rs #5318 owned there conformance
    default_image.rs PR1 merge, port conformance
    docker_corporate_proxy.rs PR1, PR6 config test → crate test; splicer test dropped; egress tests merged openshell-gateway crate test, features/corporate-proxy
    docker_preflight.rs PR1 merge, port; guidance test dropped openshell-cli crate test
    driver_config_volume.rs PR5 port features/driver-config
    edge_tunnel_e2e.rs PR1 status test dropped; tunnel test → crate test openshell-server crate test
    ephemeral_cleanup.rs PR1 (+ PR5, PR8, PR9) split conformance, drivers/*
    forward_proxy_graphql_l7.rs PR2 port conformance
    forward_proxy_jsonrpc_l7.rs PR2 port, merge conformance
    forward_proxy_l7_bypass.rs PR2 merge, port conformance
    gateway_start.rs PR6 merge features/gateway-restart
    gpu.rs PR9 removed with the port —
    gpu/device_selection.rs PR9 port features/gpu-cdi, tests/support/gpu
    gpu/workloads.rs PR9 port features/gpu
    host_gateway_alias.rs PR2, PR5 provider binding and EOF tests ported; reachability dropped conformance, features/host-gateway
    internet_network_perf.rs PR1 drop —
    kubernetes_corporate_proxy.rs PR8 split features/corporate-proxy, driver unit test
    kubernetes_ha_rebalancing.rs PR8 port drivers/kubernetes
    kubernetes_runtime_reconcile.rs PR8 port; mixed-version test dropped drivers/kubernetes
    kubernetes_shared_supervisor.rs PR8 split drivers/kubernetes
    kubernetes_shared_supervisor_ha.rs PR8 port drivers/kubernetes
    kubernetes_shared_supervisor_scaling.rs PR8 port drivers/kubernetes
    landlock.rs PR1 port, merge conformance
    listener_lifetime.rs PR2 port conformance
    live_internet_traffic_perf.rs PR1 keep out of CI tasks/ script
    live_policy_update.rs PR2 port, merge conformance
    local_driver_token_restart.rs PR5, PR6 split drivers/{docker,podman}, features/gateway-restart
    mcp_sessionless.rs PR2 merge, split conformance
    no_proxy.rs PR2 port conformance
    ocsf_console.rs PR5 port features/ocsf
    ocsf_jsonl_conformance.rs PR5 merge; file-sink check dropped features/ocsf
    oidc_pkce.rs PR7 43 → 13 tests; 4 duplicates dropped features/oidc-auth
    otlp_relay.rs PR6 port features/otlp-relay, conformance
    podman_corporate_proxy.rs PR6 merge, split features/corporate-proxy, drivers/podman
    podman_gateway_start.rs PR6 merge features/gateway-restart
    podman_host_gateway.rs PR5 stop/start check ported; rest dropped features/host-gateway
    podman_oci_identity.rs PR1 port drivers/podman
    podman_resource_limits.rs PR5 port features/resource-limits
    policy_activation.rs PR6 port features/policy-activation
    port_forward.rs PR2 port conformance
    provider_auto_create.rs PR3 port conformance
    provider_files.rs PR3 port conformance
    provider_readiness.rs PR6 port features/provider-readiness
    provider_refresh_handles.rs PR6 port features/provider-refresh
    provider_token_exchange.rs PR6 port features/provider-token-exchange
    proxy_egress_pipeline.rs PR2 port conformance
    rootfs_tar.rs PR9 merge drivers/vm
    sandbox_labels.rs PR1 port conformance
    sandbox_lifecycle.rs PR1, PR4 merge, port conformance, features/cli-attach
    sandbox_templates.rs PR1 port, merge conformance
    service_bearer_passthrough.rs PR3 port features/service-exposure
    settings_management.rs PR1 split conformance
    shared_supervisor/mod.rs PR8 port tests/support/kube
    ssh_host_identity.rs PR4, PR6 split conformance, features/gateway-restart
    stable_external_placeholders.rs PR6 port features/provider-readiness
    transparent_tcp.rs PR5 port; musl DNS test dropped features/transparent-tcp
    upload_backpressure.rs PR1 split conformance
    upload_create.rs #4606 owned there —
    user_namespaces.rs PR8 drop —
    vm_bind_mount.rs PR9 port drivers/vm
    vm_corporate_proxy.rs PR1, PR9 config test → crate test; egress tests merged openshell-gateway crate test, features/corporate-proxy
    vm_gateway_start.rs PR6 merge features/gateway-restart
    vm_overlay.rs PR9 port, split drivers/vm
    websocket_conformance.rs PR2 port conformance
    workspace_lifecycle.rs PR3 port conformance
    workspace_namespace_managed.rs PR8 port, merge drivers/kubernetes, conformance
    workspace_namespace_operator.rs PR8 port drivers/kubernetes

    Overlaps called out in this issue:

    • sandbox_lifecycle.rs: the two tests conformance already covers merge into it; the rest port (PR1, PR4).
    • host_gateway_alias.rs / podman_host_gateway.rs: reachability is dropped as a duplicate of features/host-gateway. The stop/start re-check moves to features/host-gateway on every row. The 127.0.0.53 resolver check is dropped, not kept as a Podman test: the policy-DNS relay is the same on every container driver, and the transparent-TCP tests prove it works.
    • user_namespaces.rs: permanently ignored; dropped (PR8). podman_oci_identity.rs: only the part oci-image doesn't check is ported (PR1).

    PR details

    PR1: test(e2e): port sandbox lifecycle and API tests from e2e/rust to conformance

    User Story

    As a maintainer retiring e2e/rust, I want the driver-agnostic sandbox lifecycle and API tests on testkit, so they run on every conformance row and their legacy files can be deleted.

    Description

    Port, merge or drop the tests below. All conformance destinations are under tests/suites/conformance/src/. Merges keep every assertion of the tests they combine.

    Legacy test Action Destination Notes
    sandbox_lifecycle::sandbox_stop_start_preserves_workspace merge persistent_workspace.rs (persistent_workspace_survives_stop_start) Already covered except "stop while a sandbox connect session is open", which moves to PR4's attach tests.
    sandbox_lifecycle::sandbox_can_be_deleted_while_stopped merge lifecycle.rs (stopped_sandbox_can_be_deleted) Only the human output strings are lost.
    sandbox_lifecycle::sandbox_exec_outlives_startup_recovery_deadline port lifecycle.rs SDK exec; about 35 s per row.
    sandbox_lifecycle::detached_canonical_main_nonzero_exit_reaches_error + detached_canonical_main_exit_zero_reaches_completed merge lifecycle.rs One parameterized SDK test: exit 0 → Completed, exit 11 → Error with exit code 11, both after the sandbox is Ready.
    sandbox_lifecycle::canonical_main_and_exec_receive_declared_environment + canonical_tty_main_uses_sandbox_environment merge lifecycle.rs (canonical_main_environment) SDK spec.environment and spec.tty. TTY case also checks HOME, USER, TERM.
    sandbox_lifecycle::detached_main_exit_during_provisioning_is_classified_as_workload_result port lifecycle.rs Next to run_to_completion.
    sandbox_lifecycle::on_failure_policy_replaces_runtime_and_preserves_workspace port lifecycle.rs Poll get() for restart count and phase; don't use wait_ready (it errors on the Error phase).
    ephemeral_cleanup::detached_ephemeral_{success,failure}_removes_sandbox_and_driver_resources split lifecycle.rs One test, two sandboxes (exit 0 and 17), checks the sandbox record is gone. The "no driver resources left" half is rewritten in PR5 under drivers/*.
    landlock::hard_requirement_accepts_enriched_device_path port filesystem_policy.rs
    landlock::{best_effort,hard_requirement}_rejects_replaced_policy_{path,ancestor} (4 tests) merge filesystem_policy.rs One table-driven test (2 compatibility modes × 2 replace modes); keep the 20 s exec timeout.
    core_dump_hardening::sandbox_processes_disable_core_dumps port process.rs Check both ulimit -c and ulimit -Hc are 0.
    default_image::sandbox_from_default_image merge lifecycle.rs (default_command_sandbox_is_listed_ready) Add an /etc/os-release check (ID, VERSION_ID).
    default_image::sandbox_from_explicit_nvidia_ubuntu_image port image.rs "Explicit image is honored", using testkit::base_image().
    sandbox_labels::sandbox_labels_are_stored_and_filterable port labels.rs SDK labels and label_selector; 3 sandboxes; label values unique per run.
    sandbox_templates::sandbox_create_from_template_uses_reusable_workload port templates.rs Also assert created_from_workload_template.name.
    sandbox_templates:: get-after-delete, duplicate name, missing template (3 tests) merge templates.rs (template_crud_errors) Match SDK NotFound / AlreadyExists, not text.
    settings_management::settings_global_override_round_trip split settings.rs (a) sandbox_setting_round_trip; (b) global_setting_overrides_sandbox_scope, which records and restores the prior global value.
    upload_backpressure::large_upload_preserves_content_and_metadata_outside_workdir split file_transfer.rs (a) Metadata checks (mode, symlinks, overwrite, destination outside workdir) join upload_download_round_trip; (b) large_upload_backpressure keeps count, checksum and no-retry.
    podman_oci_identity::podman_supervisor_runs_as_oci_identity_without_capabilities port tests/suites/drivers/podman Only the part oci-image doesn't already check.
    docker_preflight::doctor_check_fails_without_docker + doctor_check_output_shows_docker_label merge crates/openshell-cli/tests/doctor_integration.rs Hermetic: no Docker, no gateway.
    docker_preflight::doctor_check_passes_with_docker port crates/openshell-cli/tests/doctor_integration.rs Fake docker on PATH; add a "no docker on PATH" case.
    docker_preflight::doctor_check_error_includes_guidance drop — Passes whatever the product prints. Needs approval in #5297.
    docker_corporate_proxy::docker_corporate_proxy_rejects_incoherent_configuration + vm_corporate_proxy::vm_corporate_proxy_rejects_incoherent_configuration drop → crate test crates/openshell-gateway/tests/proxy_config_preflight.rs One hermetic test running openshell-gateway config preflight per driver. Needs approval in #5297.
    docker_corporate_proxy::proxy_settings_are_inserted_before_nested_driver_tables (unit test) drop — Tests a harness TOML helper, not the product. Needs approval in #5297.
    edge_tunnel_e2e::gateway_status_reports_healthy drop — testkit checks gateway health before every suite. Needs approval in #5297.
    edge_tunnel_e2e::ws_tunnel_status_through_edge_proxy drop → crate test openshell-server crate test SDK tunnel → real WebSocket tunnel router → health RPC. Runs in no lane today. Needs approval in #5297.
    internet_network_perf::benchmark_complete_internet_path drop — No assertions; superseded by the live-traffic benchmark. Needs approval in #5297.
    live_internet_traffic_perf::benchmark_live_internet_agent_traffic keep on host tasks/ script Benchmark using the live Internet; not scheduled in CI.
    Helpers this PR adds
    • testkit: CliOutput::message() (replaces the legacy create_expect_failure).
    • A guard that records and restores a global gateway setting (used by the global settings test).
    Files deleted

    core_dump_hardening.rs, default_image.rs, docker_preflight.rs, edge_tunnel_e2e.rs, ephemeral_cleanup.rs, internet_network_perf.rs, landlock.rs, live_internet_traffic_perf.rs, podman_oci_identity.rs, sandbox_labels.rs, sandbox_templates.rs, settings_management.rs, upload_backpressure.rs.

    Shared files: sandbox_lifecycle.rs is deleted by whichever of PR1 and PR4 lands last; docker_corporate_proxy.rs by whichever of PR1 and PR6 lands last. vm_corporate_proxy.rs stays until PR9.

    Defaults (change in review if needed)
    • upload_backpressure runs on every conformance row: large uploads are a public contract and the transport differs per driver. If it nears the runner's 2-minute limit, split it further.
    • The global settings test lands in conformance. It changes ocsf_json_enabled for the whole gateway, which is safe because the runner runs one test at a time. If a family later runs more threads (test(runner): apply per-family nextest config and a thread count to runner rows #5295), give it a nextest test group.

    Context

    Definition of Done

    • Every test in the table is ported, merged, moved to a crate test, or dropped as listed, with the drops approved in test(e2e): decide which e2e/rust tests to port and in what order #5297.
    • The listed files are deleted, and every reference to them in wrapper scripts, tasks/*.toml and nextest config is removed.
    • Each ported test runs on at least the drivers its legacy lane covered (the conformance rows, plus the Podman row for drivers/podman).
    • The PR body states which defaults above were applied.
    PR2: test(e2e): port network, L7 and policy-update tests from e2e/rust to conformance

    User Story

    As a maintainer retiring e2e/rust, I want the network enforcement, L7 and live policy-update tests on testkit, so every driver proves the same egress contract and the legacy files can be deleted.

    Description

    All destinations are under tests/suites/conformance/src/. Merges keep every assertion of the tests they combine.

    Legacy test Action Destination Notes
    proxy_egress_pipeline::transparent_destination_denials_fail_connect_with_eacces port network.rs Absorbs bypass_detection (next row).
    bypass_detection::bypass_attempt_is_rejected_fast merge network.rs (direct_connect_denials) One sandbox: unlisted target, metadata, control plane, outside allowed_ips; assert EACCES/EPERM within 8 s each.
    listener_lifetime::closing_or_exiting_listener_refuses_connections_and_releases_port port network.rs
    listener_lifetime::unrelated_exec_does_not_inherit_broker_preconnect_socket port network.rs Main command via SandboxBuilder::command.
    no_proxy::sandbox_reaches_localhost_without_proxy_environment port network.rs Next to the listener tests.
    forward_proxy_l7_bypass::forward_proxy_{allows,denies}_l7_*_request (2 tests) merge network_l7.rs (rest_l7_allows_get_denies_post) One fixture, one sandbox.
    forward_proxy_l7_bypass::rest_upgrade_is_refused_in_enforce_and_audit_modes port network_l7.rs One sandbox, two endpoints (enforce and audit).
    forward_proxy_graphql_l7::graphql_l7_enforces_high_level_and_raw_transparent_paths port network_l7.rs Drop "forward" from the name.
    forward_proxy_jsonrpc_l7::jsonrpc_l7_enforces_high_level_and_raw_transparent_paths port network_l7.rs Also hosts the audit case below.
    forward_proxy_jsonrpc_l7::jsonrpc_forward_proxy_hard_denies_response_frames_in_default_audit_mode merge network_l7.rs Second endpoint in the JSON-RPC test's sandbox.
    mcp_sessionless::legacy_and_multi_version_profiles_authorize_tools_through_sandbox split network_l7/mcp.rs Three tests, one per protocol version set (each needs its own policy).
    mcp_sessionless::sessionless_discovery_tools_and_subscription_use_request_metadata merge network_l7/mcp.rs Its SSE subscription step joins the newest-version test.
    proxy_egress_pipeline::policy_reload_updates_transparent_requests_and_closes_existing_http_stream port network_egress.rs
    proxy_egress_pipeline::ambiguous_policy_update_is_rejected_without_replacing_active_policy port network_egress.rs Compare revisions with policy list -o json, not text.
    proxy_egress_pipeline::explicit_allowed_ips_and_implicit_ip_literals_succeed_transparently port network_egress.rs Test-host IP on two ports instead of two containers.
    proxy_egress_pipeline:: TLS skip relay, middleware redaction, fail-closed and fail-open middleware, first-request overflow, chunked pipeline (6 tests) port network_egress.rs Port as-is.
    proxy_egress_pipeline::http_credentials_are_rewritten_in_transparent_headers_and_bodies port credential_gating.rs Joins the credential_gating module from #5318.
    websocket_conformance::websocket_text_placeholder_is_rewritten_transparently port credential_gating.rs Reuses the WebSocket probe from #5318.
    host_gateway_alias::static_provider_credentials_are_bound_to_profile_endpoints port credential_binding.rs Unique provider and profile names per run; replace host.docker.internal with an unbound host so the 403 check holds on every driver.
    port_forward::port_forward_echo port cli.rs forward start -d plus a cleanup hook that runs forward stop.
    live_policy_update::l7_append_target_scope_round_trip port policy_update.rs Already uses JSON output.
    live_policy_update::live_policy_update_round_trip + live_policy_update_from_empty_network_policies merge policy_update.rs Start from empty policies, then A, then B; read versions from policy get/list -o json; assert no revision stays Pending.
    live_policy_update::initial_sparse_policy_is_acknowledged_as_loaded port policy_update.rs Poll policy list -o json statuses; use a unique sandbox name.
    Helpers this PR adds
    • testkit: compute_driver() (refactor of expect_compute_driver), Sandbox::logs / logs_until, exec_until, sandbox_driver_config.
    • tests/support/host-server: request handler with Request and chunked bodies, TcpServer, a WebSocket text helper, free_port.
    • testkit policy: policy::set, policy::from_yaml.
    Files deleted

    bypass_detection.rs, forward_proxy_graphql_l7.rs, forward_proxy_jsonrpc_l7.rs, forward_proxy_l7_bypass.rs, listener_lifetime.rs, live_policy_update.rs, mcp_sessionless.rs, no_proxy.rs, port_forward.rs, proxy_egress_pipeline.rs, websocket_conformance.rs.

    host_gateway_alias.rs stays until PR5 ports its other two tests.

    Defaults (change in review if needed)
    • L7 fixtures become in-process Rust servers reached at host.openshell.internal, and the legacy private allowed_ips ranges go away. If a row can't reach the test host, that test keeps a container fixture on that row.
    • The policy-reload keep-alive client runs with nohup … & through Sandbox::exec; if it doesn't survive, it runs as the sandbox's main command. No new testkit API.
    Known risks
    • On Podman the host address may be link-local and blocked before allowed_ips applies. If so, the IP-literal test keeps a container fixture.
    • Moving the L7 bypass tests to host.openshell.internal assumes every conformance row can reach the test host. Not yet verified on k3s.

    Context

    • Wave 1, after test(conformance): port the credential gating e2e tests #5318 merges (it adds the policy helpers, install_provider, HttpRecorder and the credential_gating module this PR builds on).
    • PR1 also edits testkit's sandbox context; whichever lands second rebases.
    • PR5, PR6 and PR9 use the helpers this PR adds.

    Definition of Done

    • Every test in the table is ported or merged as listed.
    • The listed files are deleted, and every reference to them in wrapper scripts, tasks/*.toml (including e2e:websocket-conformance) and nextest config is removed.
    • Each ported test runs on at least the drivers its legacy lane covered.
    • The PR body states which defaults above were applied and any row that kept a container fixture.
    PR3: test(e2e): port provider and workspace tests from e2e/rust to testkit

    User Story

    As a maintainer retiring e2e/rust, I want the provider and workspace CRUD tests on testkit, so they run on every conformance row and the legacy files can be deleted.

    Description

    Conformance destinations are under tests/suites/conformance/src/.

    Legacy test Action Destination Notes
    provider_files::provider_file_open_update_and_detach port providers.rs
    provider_auto_create::auto_created_provider_credential_available_in_sandbox port providers.rs Separate CLI path (auto-created providers).
    service_bearer_passthrough::service_bearer_passthrough_preserves_authorization_header port tests/suites/features/service-exposure New feature package; other service-exposure tests can join it later.
    workspace_lifecycle::workspace_full_crud_lifecycle port workspaces.rs Unique workspace names per run (they are gateway-global). -o json for lists; check the provider's owning workspace.
    workspace_lifecycle::workspace_terminating_rejects_creates port workspaces.rs Check Terminating on the test's own workspace; clean up provider, then workspace.
    Helpers this PR adds
    • testkit: install_provider(..).config(..) (--config), create_provider for built-in provider types, and a cleanup poll for provider delete (the gateway refuses to delete a provider still attached to a sandbox).
    • A service HTTP client for exposed services.
    • The environment routes exposed services to the test host.
    Files deleted

    provider_auto_create.rs, provider_files.rs, service_bearer_passthrough.rs, workspace_lifecycle.rs.

    Context

    Definition of Done

    • Every test in the table is ported as listed.
    • The listed files are deleted, and every reference to them in wrapper scripts, tasks/*.toml and nextest config is removed.
    • Each ported test runs on at least the drivers its legacy lane covered; features/service-exposure has rows in .github/workflows/e2e-suites.yml.
    PR4: test(e2e): port CLI-driven exec, attach and SSH host identity tests from e2e/rust

    User Story

    As a maintainer retiring e2e/rust, I want the tests that drive a long-running openshell CLI process on testkit, so attached create, exec streaming, reconnect and SSH host identity stay covered after the legacy harness is gone.

    Description

    Conformance destinations are under tests/suites/conformance/src/. Merges keep every assertion of the tests they combine.

    Legacy test Action Destination Notes
    sandbox_lifecycle::sandbox_exec_large_output_is_complete port exec_cli.rs Keep the 3 sub-checks in one test on one sandbox.
    sandbox_lifecycle::piped_exec_stdin_crosses_grpc_message_limit port exec_cli.rs The stdin-from-file case is Unix-only, which is fine on Linux CI.
    sandbox_lifecycle::canonical_main_exit_zero_completes_persistent_sandbox + canonical_main_nonzero_exit_preserves_status merge cli.rs (cli_attached_create_streams_main_and_reports_exit) Two sandboxes: exit 0 → Completed and still listed; exit 7 → Error with exit code 7. Read state with get --output json.
    sandbox_lifecycle::sandbox_create_with_no_keep_cleans_up_after_tty_command + …_preserves_failure_then_cleans_up merge cli.rs (cli_no_keep_removes_sandbox_after_main_exit) Exit 0 and 17. Pass --name and track the sandbox instead of parsing "Created sandbox:". Drop the Docker-only diagnostics; testkit failure context replaces them.
    sandbox_lifecycle::canonical_main_disconnect_reconnect_replays_history_for_same_process port attach.rs Through a streaming CLI child. Also covers "stop while a connect session is open" from PR1.
    sandbox_lifecycle::canonical_main_exit_255_is_not_retried_as_transport_failure port attach.rs Grouped with reconnect.
    sandbox_lifecycle::canonical_main_connect_recovers_its_ssh_transport port tests/suites/features/cli-attach CLI client logic, the same on every driver: one Linux row.
    sandbox_lifecycle::canonical_main_connect_forwards_pid_targeted_termination_and_reaps_ssh port tests/suites/features/cli-attach As above.
    ssh_host_identity::ssh_host_identity_survives_restarts_and_changes_after_recreation split ssh.rs (ssh_host_identity_stable_across_stop_start_and_new_on_recreate) Fingerprint via the SDK, pinned ssh, recreate. Keeps the "auth.json unreadable" security check. The gateway-restart part moves to PR6.
    Helpers this PR adds
    • testkit: cli_child, a long-running CLI process with stdin, a line reader, its pid, kill/signal and wait.
    • cli_pty only if plain Cli::run can't drive attached sandbox create -- <cmd>.
    • A helper that exports the CLI environment for a plain ssh client, and host procfs access for the pid-termination test.
    Files deleted

    ssh_host_identity.rs. sandbox_lifecycle.rs is deleted by whichever of PR1 and PR4 lands last.

    Defaults (change in review if needed)
    • Attached sandbox create -- <cmd> uses plain Cli::run first; cli_pty is added only if that fails.
    • Reconnect/replay and exit 255 go to conformance (server-side attachment contract). SSH transport recovery and pid-targeted termination go to features/cli-attach on one Linux row (client-side logic).
    • ssh_host_identity requires a named gateway and fails with a clear message otherwise.

    Context

    • Wave 1: can start now. Waits on no other PR and no decision.
    • PR1 also edits testkit's sandbox context; whichever lands second rebases.
    • PR6, PR7 and PR8 use cli_child from this PR.

    Definition of Done

    • Every test in the table is ported, merged or split as listed.
    • The listed files are deleted, and every reference to them in wrapper scripts, tasks/*.toml and nextest config is removed.
    • Conformance tests run on every conformance row; features/cli-attach has one Linux row in .github/workflows/e2e-suites.yml.
    • The PR body states which defaults above were applied.
    PR5: test(e2e): port Docker and Podman driver feature tests from e2e/rust

    User Story

    As a maintainer retiring e2e/rust, I want the tests that need a container engine on testkit, running on both the Docker and Podman rows, so container-driver behavior stays covered and the legacy files can be deleted.

    Description

    Most of these tests were gated to one engine in e2e/rust, but the behavior they check is shared by Docker and Podman. Each new feature package runs on the Docker and Podman rows unless noted.

    Legacy test Action Destination Notes
    ocsf_console::network_events_are_json_on_supervisor_stderr_only port tests/suites/features/ocsf Supervisor stderr isn't reachable through the public API, so not conformance. Also validate stderr records against the OCSF schema.
    ocsf_jsonl_conformance::supervisor_jsonl_conforms_to_the_native_schema merge tests/suites/features/ocsf Collect OCSF-JSON records from supervisor stderr through engine logs. Keep the class, attribution, schema, version, opt-in and clean-stdout checks. Use an in-process host server instead of a container fixture.
    ↳ its /var/log file-sink check drop — Same formatter as stderr; the file sink is best-effort per the docs. Needs approval in #5297.
    driver_config_volume::sandbox_mounts_existing_driver_config_volume port tests/suites/features/driver-config Create the volume with the engine CLI, labeled for resource admission, so the test works with admission enabled.
    driver_config_volume::oci_workspace_preparation_skips_nested_volume_ownership port tests/suites/features/driver-config Build from testkit::base_image() without apt, so no Internet. Runs on Podman too.
    driver_config_volume::sandbox_mounts_enabled_driver_config_bind port tests/suites/features/driver-config
    local_driver_token_restart::local_driver_restart_or_stop_respects_main_process_lifecycle split tests/suites/drivers/{docker,podman} Container part only, no gateway restart. The VM part joins PR6's gateway-restart test.
    podman_resource_limits::sandbox_resource_limits_are_enforced_via_cgroups port tests/suites/features/resource-limits Not Podman-specific: Docker, Podman and Kubernetes advertise and enforce CPU and memory limits. First assert the gateway reports limit support, then read the cgroup values.
    host_gateway_alias::sandbox_receives_eof_after_closing_http_response port tests/suites/features/host-gateway (response_eof.rs) Three cases in one sandbox.
    host_gateway_alias::sandbox_reaches_host_openshell_internal_via_host_gateway_alias drop — features/host-gateway checks reachability more strictly on 5 rows. Needs approval in #5297.
    podman_host_gateway::podman_sandbox_reaches_host_openshell_internal split tests/suites/features/host-gateway (host_server_reachable_after_stop_start) Keeps the stop/start re-check on every host-gateway row. Drops the reachability half and the hard-coded nameserver 127.0.0.53 check (the policy-DNS relay is the same on every driver; the transparent-TCP tests prove it works). Needs approval in #5297.
    transparent_tcp::local_container_native_tcp_uses_policy_dns_and_fails_closed port tests/suites/features/transparent-tcp Transparent TCP isn't offered on Kubernetes or VM, so not conformance.
    transparent_tcp::rootless_podman_private_dns_pins_do_not_cross_binary_rules port tests/suites/features/transparent-tcp On Docker the fixture uses host.openshell.internal, since the supervisor can't resolve network aliases there.
    transparent_tcp::rootless_podman_musl_getaddrinfo_uses_udp_policy_dns drop — Runs in no CI lane and needs a musl probe nobody builds (#3009 closed undone). Needs approval in #5297. If kept, rename it musl_getaddrinfo_uses_udp_policy_dns, build the probe as a musl binary in tests/support, and file a successor to #3009.

    This PR also adds the "no driver resources left" half of the ephemeral cleanup test that PR1 split off: one test per driver under tests/suites/drivers/{docker,podman}. Kubernetes and VM get theirs in PR8 and PR9.

    Helpers this PR adds
    • tests/support/container-engine: running, wait_running, split stdout/stderr logs, copy_from, build_image_from_dir. Uses the existing OPENSHELL_TEST_CONTAINER_ENGINE.
    • A driver resource probe ("nothing left for this sandbox").
    • testkit: a per-test workspace, gateway info, OCSF record parsing.
    • Environment: a driver-config-enabled gateway variant, bind-mount source paths, and the container network name.
    Files deleted

    driver_config_volume.rs, host_gateway_alias.rs, local_driver_token_restart.rs, ocsf_console.rs, ocsf_jsonl_conformance.rs, podman_host_gateway.rs, podman_resource_limits.rs, transparent_tcp.rs.

    Defaults (change in review if needed)
    • features/resource-limits adds the k3s row only after a check that k3s enforces CPU and memory limits. File an issue that the VM driver ignores limits.
    • ocsf_console and ocsf_jsonl share one features/ocsf package.

    Context

    Definition of Done

    • Every test in the table is ported, merged, split or dropped as listed, with the drops approved in test(e2e): decide which e2e/rust tests to port and in what order #5297.
    • The listed files are deleted, and every reference to them (PODMAN_CI_TESTS, OPENSHELL_E2E_DOCKER_TEST, podmanE2eFollowUpBinaries, wrapper scripts, nextest groups) is removed.
    • Each new feature package has Docker and Podman rows in .github/workflows/e2e-suites.yml; tests that ran only on one engine now run on both.
    • The PR body lists coverage that is new on a driver, so a red first run isn't read as a regression.
    PR6: test(e2e): port gateway-configuration tests from e2e/rust to feature suites

    User Story

    As a maintainer retiring e2e/rust, I want the tests that need a specially configured or restartable gateway on testkit, so each one runs on an environment built for it instead of a wrapper script that rewrites gateway config.

    Description

    Each of these tests needs a gateway configured a certain way (OTLP collector, push-mode config, additional CA, corporate proxy) or a gateway the test can restart. The environment sets that up and publishes how to reach it. The test never edits gateway config itself.

    Legacy test Action Destination Notes
    otlp_relay::agent_span_reaches_collector_with_attribution port tests/suites/features/otlp-relay Runs on a row whose gateway has a collector configured.
    otlp_relay::span_emitted_before_exit_is_delivered port tests/suites/features/otlp-relay
    otlp_relay::relay_is_inert_without_collector port tests/suites/conformance/src/otlp.rs Default-config behavior.
    config_push::streamed_policy_update_reaches_the_supervisor port tests/suites/features/config-push Keeps only its push-session check; the push-mode row also runs conformance.
    policy_activation::invalid_image_provider_bundle_waits_for_repair_before_launch port tests/suites/features/policy-activation Build from testkit::python_image() so it needs no Internet or apt.
    gateway_start::docker_…, podman_gateway_start::podman_…, vm_gateway_start::vm_gateway_restart_preserves_running_and_stopped_intent (3 tests) merge tests/suites/features/gateway-restart One test on the Docker, Podman and VM rows. Also absorbs the restart part of PR4's SSH host identity test and the VM part of local_driver_token_restart (with its token-expiry check).
    additional_ca::additional_ca_trusts_matching_and_public_hosts_and_rejects_mismatch split tests/suites/features/additional-ca + tests/suites/drivers/* Trust checks go to the feature suite; CA delivery checks go to the driver suites. Drops the tamper and CA-removal checks, the only part that restarts the gateway and edits gateway host files. Needs approval in #5297.
    docker_corporate_proxy::docker_corporate_proxy_routes_approved_tls_egress + trusts_ca_bundle_for_https_proxy, podman_corporate_proxy:: same two merge tests/suites/features/corporate-proxy (approved_egress_uses_authenticated_proxy) One environment config: HTTPS proxy, auth, CA, port-qualified no_proxy. Every proxied request proves CA trust. The http:// proxy scheme is no longer tested separately.
    docker_corporate_proxy::docker_corporate_proxy_trusts_intercepted_destination_tls port tests/suites/features/corporate-proxy (intercepting_proxy_ca_is_trusted_by_supervisor_and_workload) Runs on every corporate-proxy row, not just Docker.
    ↳ Podman proxy-auth secret lifecycle (from podman_corporate_proxy) split tests/suites/drivers/podman (proxy_auth_secret_removed_with_sandbox)
    provider_refresh_handles::long_running_process_survives_rotations_and_reconfigure_revokes port tests/suites/features/provider-refresh Separate from the Keycloak provider-refresh suite.
    provider_token_exchange:: independent grants across sandboxes; A/B/A route recovery (2 tests) port tests/suites/features/provider-token-exchange One package for both.
    provider_readiness::acknowledged_provider_changes_apply_to_fresh_clients_and_revoke_retained_references port tests/suites/features/provider-readiness Keeps its wrong host/port/path denials.
    stable_external_placeholders:: persistent rotation and revocation; ordinary revoked assertion (2 tests) port tests/suites/features/provider-readiness Shares the fixture with provider_readiness; tests stay separate.

    The Kubernetes (Helm) branch of additional_ca moves with the Kubernetes environment in PR8.

    Helpers and environment this PR adds
    • tests/support/gateway_control: reads OPENSHELL_TEST_GATEWAY_STOP and OPENSHELL_TEST_GATEWAY_START (each a command split on whitespace, like OPENSHELL_TEST_CONTAINER_ENGINE) and fails if either is missing. Depends on decision A in test(e2e): decide which e2e/rust tests to port and in what order #5297.
    • tests/support/host-server TLS: tls (servers signed by the test's own CA) and tls_env_ca (servers signed by the environment's additional CA).
    • A corporate-proxy environment (proxy with auth and its own CA) and tests/support/corporate_proxy to read it.
    • Test fixtures: a rotating OAuth issuer, an OTLP collector, a SPIFFE workload API stub.
    • Environment rows: OTLP collector, push mode (replaces the DP lane), additional CA (tmachine sets gateway.config.additional_ca and publishes OPENSHELL_TEST_ADDITIONAL_CA_{CERT,KEY}; the test signs leaf certificates for servers it starts).
    Files deleted

    additional_ca.rs, config_push.rs, gateway_start.rs, otlp_relay.rs, podman_corporate_proxy.rs, podman_gateway_start.rs, policy_activation.rs, provider_readiness.rs, provider_refresh_handles.rs, provider_token_exchange.rs, stable_external_placeholders.rs, vm_gateway_start.rs. docker_corporate_proxy.rs is deleted by whichever of PR1 and PR6 lands last.

    Defaults (change in review if needed)
    • The gateway-restart test lives in features/gateway-restart, not conformance: it checks that the container or VM survives, which needs engine or host access. Once every conformance row publishes the stop/start commands, the phase-only half can move to conformance.
    • provider_readiness keeps its wrong host/port/path denials (they prove the backend was never hit). Revisit after test(conformance): port the credential gating e2e tests #5318.

    Context

    Definition of Done

    • Every test in the table is ported, merged or split as listed, with the drop approved in test(e2e): decide which e2e/rust tests to port and in what order #5297.
    • Each new environment variable has a row in the tests/support contract table.
    • The listed files are deleted, and every reference to them in wrapper scripts, tasks/*.toml and nextest config is removed.
    • Each feature package has rows in .github/workflows/e2e-suites.yml on the drivers its legacy tests covered.
    PR7: test(e2e): port OIDC authorization tests from e2e/rust to features/oidc-auth

    User Story

    As a maintainer retiring e2e/rust, I want the OIDC role and workspace authorization tests on testkit as a small set of table-driven tests, so they run on a Keycloak environment and the 43 near-identical legacy tests become 13.

    Description

    All 43 tests come from oidc_pkce.rs and move to tests/suites/features/oidc-auth. The browser (PKCE) login runs once; every other test logs in with a password grant through tests/support/keycloak. Role × action pairs become rows of table tests. Merges keep every assertion.

    Suggested new tests and the legacy tests each one covers:

    New test Legacy tests it covers
    pkce_browser_login_persists_session The PKCE login helpers (login_identity, assert_pkce_authorization_url, assert_persisted_login) as one test
    user_reports_gateway_validated_identity user_can_report_gateway_validated_identity
    platform_admin_smoke admin_can_list_sandboxes, admin_can_inspect_gateway, admin_can_list_providers, admin_can_manage_provider, admin_can_create_workspace
    non_admins_denied_platform_actions (table) user_cannot_inspect_gateway, user_cannot_create_workspace, user_cannot_delete_workspace, workspace_admin_cannot_{create,delete}_workspace, workspace_admin_cannot_inspect_gateway
    workspace_user_read_access workspace_user_can_read_workspace, user_can_list_sandboxes, user_can_list_providers, workspace_admin_can_read_workspace
    sandbox_create_delete_by_role (table) admin_can_{create,delete}_sandbox, user_can_{create,delete}_sandbox
    workspace_user_denied_management (table) user_cannot_{create,delete}_provider, workspace_user_cannot_manage_members
    workspace_admin_manages_providers_and_members workspace_admin_can_{create,delete}_provider, workspace_admin_can_{add,remove}_user_member, membership_removal_revokes_workspace_access
    workspace_admin_cannot_grant_admin same name
    cross_workspace_access_denied (table) workspace_admin_cannot_read_another_workspace, workspace_admin_cannot_manage_another_workspace_members, workspace_user_cannot_read_another_users_workspace, workspace_user_cannot_list_another_workspace_{sandboxes,providers,members}, workspace_user_cannot_create_sandbox_in_another_workspace (also checks the other workspace has no new sandbox)
    workspace_list_hides_other_workspaces workspace_list_hides_another_users_workspace (parse JSON names; also check default is hidden)
    workspace_admin_cannot_manage_another_workspace_providers same name (checks redaction and no side effect)
    user_can_resolve_provider_profile_for_sandbox same name (imports the profile as admin with cleanup, or the environment publishes it)

    Dropped (each repeats a role × action pair another test asserts; needs approval in #5297): admin_can_delete_workspace, second_workspace_user_cannot_read_first_users_workspace, workspace_admin_can_create_sandbox, workspace_admin_can_delete_sandbox.

    All names, providers and workspaces are unique per run and created in a test-owned workspace, not default.

    Helpers and environment this PR adds
    • An OIDC environment: Keycloak plus a gateway with OIDC enabled and mTLS kept, so testkit connects as it does today (decision C in test(e2e): decide which e2e/rust tests to port and in what order #5297).
    • tests/support/keycloak gains password-grant sessions per user. Keycloak publishes the extra users as OPENSHELL_TEST_KEYCLOAK_USER_* and OPENSHELL_TEST_KEYCLOAK_USER_B_*.
    • tests/support/oidc for the PKCE flow, and a browser stand-in that records the authorization URL.
    Files deleted

    oidc_pkce.rs.

    Context

    • Wave 3: after PR3 (create_provider) and PR4 (cli_child, used for the browser login).
    • Waits on decision C in test(e2e): decide which e2e/rust tests to port and in what order #5297 (the OIDC environment keeps mTLS). The four drops wait on decision D.
    • Cost of decision C: no test may assume OIDC is the only way in. An OIDC-only mode for testkit is added later only if a test needs it.

    Definition of Done

    • All 43 legacy tests are covered by the 13 tests above or dropped as listed, with the drops approved in test(e2e): decide which e2e/rust tests to port and in what order #5297.
    • Each new environment variable has a row in the tests/support contract table.
    • oidc_pkce.rs is deleted, and every reference to it in wrapper scripts, tasks/*.toml and nextest config is removed.
    • features/oidc-auth has rows in .github/workflows/e2e-suites.yml on the drivers its legacy lane covered.
    PR8: test(e2e): port Kubernetes driver tests from e2e/rust to drivers/kubernetes

    User Story

    As a maintainer retiring e2e/rust, I want the Kubernetes-only tests (workspace namespaces, credential backends, HA, shared supervisor) on testkit, so they run against tmachine k3s environments through a Rust Kubernetes client instead of kubectl in wrapper scripts.

    Description

    Destinations are under tests/suites/drivers/kubernetes/tests/ unless noted. Merges keep every assertion.

    Legacy test Action Destination Notes
    shared_supervisor/mod.rs helpers and their 7 tests (6 unit tests and 1 host-call test) port tests/support/kube (shared_supervisor module) Become plain unit tests of the support crate.
    kubernetes_runtime_reconcile::start_refreshes_stale_bootstrap_image port reconcile.rs
    kubernetes_runtime_reconcile::old_and_new_gateways_can_restart_the_same_sandbox drop — Permanently ignored; no environment can supply a mixed-version gateway. File an upgrade-compatibility follow-up. Needs approval in #5297.
    user_namespaces::sandbox_pod_spec_has_user_namespace_fields drop — Permanently ignored; driver unit tests cover the pod spec. Needs approval in #5297.
    credential_drivers::provider_credentials_are_stored_in_configured_backend port credential_backends.rs Keep the backend-object assertions only (#5318 covers the placeholder part). Select the backend from the environment, not two wrapper passes.
    workspace_namespace_managed::managed_full_lifecycle_with_multiple_sandboxes port workspace_managed.rs Absorbs the next two rows.
    workspace_namespace_managed::managed_namespace_survives_with_remaining_sandboxes, managed_workspace_delete_removes_namespace merge workspace_managed.rs Into the full lifecycle test.
    workspace_namespace_managed:: creates namespace with labels, isolates workspaces, supervisor reads client TLS from its bootstrap secret, gateway cannot read workspace secrets, rejects namespace owned by another gateway, stop waits for pod to disappear (6 tests) port workspace_managed.rs Port as-is; share a gateway_can helper with the operator tests.
    workspace_namespace_managed::managed_rejects_invalid_dns1123_sandbox_name port tests/suites/conformance/src/cli.rs Moves to conformance if the server does the validation; otherwise stays here.
    workspace_namespace_operator:: all 6 tests port workspace_operator.rs The Helm chart path comes from the environment, not CARGO_MANIFEST_DIR.
    kubernetes_ha_rebalancing::sandbox_exec_rebalances_across_gateway_scale_and_rollout port ha.rs Replace the module-level mutex with a nextest test group.
    kubernetes_ha_rebalancing::sandbox_file_sync_survives_gateway_pod_rolls port ha.rs Payload can shrink to 8 MiB to save time.
    kubernetes_shared_supervisor::one_supervisor_isolates_and_outlives_concurrent_sandbox_instances split shared_supervisor.rs Two tests: credential isolation across instances, and route lifecycle/refusal. The 742-line test is too big to debug as one.
    kubernetes_shared_supervisor_ha:: supervisor pod loss moves sandboxes; pool sessions follow their gateway (2 tests) port shared_supervisor_ha.rs Node access and port-forwarding through tests/support/kube, not docker exec.
    kubernetes_shared_supervisor_scaling::scaling_a_supervisor_pool_up_and_down_keeps_its_sandboxes_running port shared_supervisor_scaling.rs One test; its phases depend on each other.
    kubernetes_corporate_proxy::kubernetes_corporate_proxy_uses_secret_and_never_falls_back split tests/suites/features/corporate-proxy (k3s row) The common proxy modes run on a k3s corporate-proxy row. The missing/malformed Secret cases become a driver unit test that the proxy-auth Secret reference is not optional.

    This PR also adds the Kubernetes half of the ephemeral cleanup "no driver resources left" test (PR5 adds Docker and Podman), and the Helm branch of additional_ca from PR6.

    Helpers and environment this PR adds
    • tests/support/kube: a Rust kube client (already a workspace dependency) built from published variables, namespace and sandbox-resource lookups, gateway_can (RBAC check), port-forwarding, and the shared-supervisor helpers. Tests never run kubectl.
    • Kubernetes rows publish OPENSHELL_TEST_KUBE_{KUBECONFIG,CONTEXT,NAMESPACE,RELEASE,SANDBOX_NAMESPACE,CHART}.
    • HA tests read OPENSHELL_TEST_GATEWAY_REPLICAS (k3s-ha runs 3; the legacy tests hard-code 2).
    • Environment: Helm values variants (managed and operator workspaces), Postgres, Vault, ingress.
    Files deleted

    credential_drivers.rs, kubernetes_ha_rebalancing.rs, kubernetes_runtime_reconcile.rs, kubernetes_shared_supervisor.rs, kubernetes_shared_supervisor_ha.rs, kubernetes_shared_supervisor_scaling.rs, shared_supervisor/mod.rs, user_namespaces.rs, workspace_namespace_managed.rs, workspace_namespace_operator.rs, kubernetes_corporate_proxy.rs.

    Defaults (change in review if needed)
    • HA and shared-supervisor rows run nightly, not on every PR.

    Context

    Definition of Done

    PR9: test(e2e): port VM and GPU tests from e2e/rust to testkit

    User Story

    As a maintainer retiring e2e/rust, I want the VM driver and GPU tests on testkit, running on the existing KVM and GPU lanes, so those lanes stop depending on the legacy harness and milestone 1 can finish.

    Description

    Legacy test Action Destination Notes
    vm_overlay::vm_overlay port tests/suites/drivers/vm Keep the overlay-root and tls.key permission checks; make the tls.key check unconditional or assert absence explicitly. Drop the redundant /sandbox write.
    vm_overlay::vm_identity_matches_status_and_rejects_conflicts split tests/suites/drivers/vm Three tests: identity reported and stable across restart; matching selectors accepted; conflicting selectors rejected. Stays VM-only until other drivers report resolved identity.
    vm_bind_mount::vm_bind_mount port tests/suites/drivers/vm
    rootfs_tar::sandbox_from_rootfs_tar + sandbox_from_gzipped_rootfs_tar merge tests/suites/drivers/vm One test sharing one export (plain, then .tar.gz). Build the tarball by exporting testkit::base_image() and appending a marker file, instead of docker build from the Internet.
    gpu/device_selection:: default device, each discovered device, all devices, invalid device (4 tests) port tests/suites/features/gpu-cdi Compares against a plain container on the same engine. Tighten the invalid-device assertion to name the requested id. "Each device" only adds coverage with 2 or more GPUs.
    gpu/device_selection:: CDI and docker info parsing (5 unit tests) port tests/support/gpu Plain unit tests; run in cargo test without a GPU.
    gpu/workloads::gpu_workload_manifest_runs_expected_workloads port tests/suites/features/gpu Uses only the public API (--gpu, --from), so it runs on any GPU row.
    gpu.rs (module wrapper) remove — Goes away with the port.
    vm_corporate_proxy::vm_corporate_proxy_routes_approved_tls_egress + trusts_ca_bundle_for_https_proxy merge tests/suites/features/corporate-proxy (VM row) Joins PR6's corporate-proxy test. Generalize the exact-loopback check to "target is an IP address".

    This PR also adds:

    • The VM half of the ephemeral cleanup "no driver resources left" test (PR5 adds Docker and Podman).
    • A conformance test that drivers without a staging directory reject a rootfs-tar source.
    Helpers and environment this PR adds
    • e2e-vm.sh (the existing KVM lane, e2e-vm-test.yml) runs the testkit suites against the gateway it starts, as it already does for conformance: drivers/vm (including the VM ephemeral-cleanup test), the VM row of features/corporate-proxy (the script sets up the proxy and publishes its variables), and the VM row of PR6's features/gateway-restart. This is the deliverable, not a stopgap.
    • The existing GPU workflow (e2e-gpu-test.yaml, through e2e:docker:gpu) runs features/gpu and features/gpu-cdi in place of the e2e/rust GPU tests.
    • tests/support/gpu (device inventory), a rootfs tarball fixture, and a sandbox status-conditions helper.
    Files deleted

    gpu.rs, gpu/device_selection.rs, gpu/workloads.rs, rootfs_tar.rs, vm_bind_mount.rs, vm_overlay.rs, vm_corporate_proxy.rs.

    Defaults (change in review if needed)
    • The GPU lane (e2e-gpu-test.yaml) runs only Docker today (e2e:docker:gpu on linux-arm64, linux-amd64 and WSL). e2e:podman:gpu exists in tasks/test.toml, but no workflow runs it. The GPU tests port as-is and run on the existing Docker GPU rows. File a follow-up issue for a Podman GPU row.

    Context

    • Wave 4. Only the corporate-proxy test needs PR6 (its proxy environment and fixture); the rest can be written in parallel and merge after PR6.
    • Waits on PR1, PR2 and PR5 (helpers) and on test(runner): apply per-family nextest config and a thread count to runner rows #5295: the GPU tests need 10-minute per-test timeouts.
    • Doesn't wait on decision B. The follow-up for the script-deletion milestone builds a KVM host provisioner that publishes OPENSHELL_TEST_* variables, so the VM rows can move into .github/workflows/e2e-suites.yml and e2e-vm.sh can be deleted.

    Definition of Done

    • Every test in the table is ported, merged or split as listed.
    • The listed files are deleted, and every reference to them (the e2e-vm.sh list, GPU tasks, nextest groups) is removed.
    • e2e-vm.sh runs drivers/vm and the VM rows of features/corporate-proxy and features/gateway-restart on the existing KVM lane.
    • The existing GPU lane (e2e-gpu-test.yaml) runs features/gpu and features/gpu-cdi.
    • A follow-up issue exists for the KVM host provisioner (script-deletion milestone).
    • A follow-up issue exists for a Podman GPU row.
    • If this is the last migration PR to merge, it also deletes the e2e/rust test crate (tests, harness and its unit tests, Cargo.toml) and the wrapper-script steps that ran it. The wrapper scripts themselves stay.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions