Skip to content

[chore](docker)Support rootless Docker for third-party environments - #70457

Merged
hello-stephen merged 4 commits into
apache:masterfrom
zhangstar333:codex/rootless-thirdparties
Nov 13, 2026
Merged

hello-stephen merged 4 commits into
apache:masterfrom
zhangstar333:codex/rootless-thirdparties

Conversation

@zhangstar333

Copy link
Copy Markdown
Contributor

Allow non-root users to start third-party test environments with rootless Docker.
Remove host privilege escalation and adapt file ownership, networking, initialization, and health checks.
Linux now requires a rootless daemon.

@hello-stephen

Copy link
Copy Markdown
Contributor

run buildall

@hello-stephen

Copy link
Copy Markdown
Contributor

run buildall

1 similar comment
@hello-stephen

Copy link
Copy Markdown
Contributor

run buildall

@hello-stephen
hello-stephen merged commit ccfd437 into apache:master Nov 13, 2026
32 of 33 checks passed
hello-stephen pushed a commit that referenced this pull request Nov 14, 2026
…h-4.1 (#70578)

### Background

The required status checks of every protected branch come from
`protected_branches` in the `.asf.yaml` on `master`. ASF's asfyaml
service applies that section to GitHub whenever the `github:` block
changes; the copies on release branches have no effect. Since #70459,
PRs into `master`, `branch-4.2` and `branch-4.1` are checked by the new
TeamCity, which reports statuses named `TeamCity/<pipeline>`. External
is the regression lane for external catalogs and their docker fixtures.

### What problem does this PR solve?

Issue Number: None
Related PR: #70459 (introduced the behavior this PR changes), #70457
(must be picked to branch-4.2 and branch-4.1 first)
PR Type: Build/CI/Chore

Problem Summary:

When #70459 renamed the required checks, it left `TeamCity/external`
out, so a red External run no longer blocks merges into `master`,
`branch-4.2` or `branch-4.1`. This PR adds it back to all three lists,
in the slot the old `External Regression (Doris External Regression)`
check had. Merging also makes asfyaml re-apply the whole section, and
GitHub is out of sync with it today: `branch-4.2` still requires the
nine pre-migration names, which new PRs never receive, and `branch-4.1`
and `branch-2.0` have no protection at all.

### Details

- **Affected:** GitHub branch protection only: one more required context
on `master` (17 in total), `branch-4.2` (13) and `branch-4.1` (13). No
workflow, script or TeamCity change: External uses P0's file filter, so
a PR that skips P0 also gets a skipped `TeamCity/external`. Trailing
spaces are dropped from two `branch-4.0` and `branch-3.1` lines, which
changes no value.
- **Developer and release impact:** PRs into the three branches need
`TeamCity/external` green or skipped before they can merge. On
`branch-4.2` and `branch-4.1` its preflight fails today with `Native
rootless support missing; cherry-pick PR70457 before validating this
branch`, so #70457 has to be picked to both branches before this lands,
or every 4.x PR that runs External is blocked. Once protection is
re-applied, `branch-4.1` PRs again need an approval and green required
checks, and `branch-4.2` PRs can merge on the `TeamCity/*` checks they
actually receive.
- **License and third-party:** None.
- **Rollback:** Revert this commit; asfyaml applies the previous lists
again because the `github:` block changes.

### Testing

- `asfyaml-validate` from apache/infrastructure-asfyaml (`e4c2644`), run
locally: the old and the new file are valid, and copies with a
non-integer review count or a mis-indented list item are rejected.
- Both files parsed locally with asfyaml's GitHub schema: the only data
change is `TeamCity/external` in the three `contexts` lists, with the
existing order kept and no duplicates.
- Every name in the new lists is reported on live PRs: all 17 on #70366
(`master`), and all 13 on a recent `branch-4.1` PR.

### Notes for Reviewers

- asfyaml runs only on `master`, after the merge, so whether GitHub
takes the new lists can only be checked then: `gh api
repos/apache/doris/branches/<branch> --jq
.protection.required_status_checks.contexts` for each branch, and
`.protected` for `branch-4.1` and `branch-2.0`. A failed run is mailed
to private@doris.apache.org and to whoever merged.
- `TeamCity/performance` stays optional; the comment above the lists now
says only that.
hello-stephen added a commit that referenced this pull request Nov 15, 2026
…vironments #70457 (#70579)

bp #70457

### Conflict resolution for branch-4.1

#70457 does not apply cleanly here: it builds on launcher wrappers that
master got from #64304, and it touches the Fluss fixture, which
branch-4.1 does not have. What differs from the original:

- **Fluss:** branch-4.1 has no Fluss docker environment, so
`docker-compose/fluss/README.md` and `fluss.env.tpl` stay deleted,
`fluss-network.sh` and `test/fluss-network-test.sh` are left out, and
the Fluss sentence in `docker/thirdparties/README.md` and the Fluss case
in `test/launcher-stop-test.sh` are dropped.
- **`run-thirdparties-docker.sh`:** branch-4.1 had no `docker_cli` /
`host_admin_cmd` wrappers, so besides the conflicted hunks, the 21
remaining `sudo docker` calls become `docker_cli` (as on master), the
Lance jar download and the Iceberg data extraction drop `sudo`, and
LakeSoul's `sudo rm -rf data` becomes `reset_data_dirs` plus `rmdir`,
the pattern #70457 uses for Iceberg. `iceberg-rest` is added to
`ALL_COMPONENTS`. Master-only code (MongoDB, Fluss, the Iceberg jar
check) is not added. The launcher has no `sudo` left.
- **`ranger_settings.env`:** takes #70457's overridable form. This also
renames branch-4.1's `RANGER_SOLAR_PORT` to `RANGER_SOLR_PORT`, the name
`ranger.yaml.tpl` already reads (master fixed it in #66770).
- **`test/launcher-stop-test.sh`:** branch-4.1's `custom_settings.env`
sets `CONTAINER_UID` unconditionally, so the test rewrites the copied
file the way CI does.

The other 25 files carry #70457's changes unchanged.

### Testing

- `bash -n` passes on every `docker/thirdparties/*.sh`.
`test/docker-command-test.sh`, `test/snapshot-hive-baseline-test.sh` and
`test/launcher-stop-test.sh` pass locally (macOS, bash 5.3, GNU getopt
as on Linux CI); the stop test fails when the `STOP` guard on Hive data
preparation is removed.
- Not run: the launcher against a real rootless Docker daemon. The
External run on this PR is the end-to-end check; External's preflight on
branch-4.1 requires this pick, and #70578 makes `TeamCity/external`
required here.

### AI Disclosure

- Tool: Claude Code (Claude Opus 5.5)
- Generated: the conflict resolution and the adaptations above, and this
description
- Human review: not stated

---------

Co-authored-by: zhangstar333 <zhangsida@selectdb.com>
Co-authored-by: Dongyang Li <lidongyang@selectdb.com>Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: lidongyang <lidongyang@selectdb.com>
yiguolei pushed a commit that referenced this pull request Nov 16, 2026
…vironments #70457 (#70580)

bp #70457

### Conflict resolution for branch-4.2

#70457 does not apply cleanly here: it builds on launcher wrappers that
master got from #64304, and it touches the Fluss fixture, which
branch-4.2 does not have. What differs from the original:

- **Fluss:** branch-4.2 has no Fluss docker environment, so
`docker-compose/fluss/README.md` and `fluss.env.tpl` stay deleted,
`fluss-network.sh` and `test/fluss-network-test.sh` are left out, and
the Fluss sentence in `docker/thirdparties/README.md` and the Fluss case
in `test/launcher-stop-test.sh` are dropped.
- **`run-thirdparties-docker.sh`:** branch-4.2 had no `docker_cli` /
`host_admin_cmd` wrappers, so besides the conflicted hunks, the 21
remaining `sudo docker` calls become `docker_cli` (as on master), the
Lance jar download and the Iceberg data extraction drop `sudo`, and
LakeSoul's `sudo rm -rf data` becomes `reset_data_dirs` plus `rmdir`,
the pattern #70457 uses for Iceberg. `iceberg-rest` is added to
`ALL_COMPONENTS`. Master-only code (Fluss, the Iceberg jar check) is not
added; MongoDB exists on branch-4.2 and takes #70457's change. The
launcher has no `sudo` left.
- **`ranger_settings.env`:** takes #70457's overridable form. This also
renames branch-4.2's `RANGER_SOLAR_PORT` to `RANGER_SOLR_PORT`, the name
`ranger.yaml.tpl` already reads (master fixed it in #66770).
- **`test/launcher-stop-test.sh`:** branch-4.2's `custom_settings.env`
sets `CONTAINER_UID` unconditionally, so the test rewrites the copied
file the way CI does.

The other 25 files carry #70457's changes unchanged.

### Testing

- `bash -n` passes on every `docker/thirdparties/*.sh`.
`test/docker-command-test.sh`, `test/snapshot-hive-baseline-test.sh` and
`test/launcher-stop-test.sh` pass locally (macOS, bash 5.3, GNU getopt
as on Linux CI); the stop test fails when the `STOP` guard on Hive data
preparation is removed.
- Not run: the launcher against a real rootless Docker daemon. The
External run on this PR is the end-to-end check; External's preflight on
branch-4.2 requires this pick, and #70578 makes `TeamCity/external`
required here.

### AI Disclosure

- Tool: Claude Code (Claude Opus 5.5)
- Generated: the conflict resolution and the adaptations above, and this
description
- Human review: not stated

---------

Co-authored-by: zhangstar333 <zhangsida@selectdb.com>
Co-authored-by: Dongyang Li <lidongyang@selectdb.com>Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: lidongyang <lidongyang@selectdb.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants