Skip to content

client: read-only client without plugin management or connect (client v0.2.2 adds 1.3 MB to the Docker CLI) #709

Open

Description

@Benehiko

Summary

Using github.com/docker/secrets-engine/client v0.2.2 to read the Docker Hub session (client.New → HubAuth() → GetDefaultSession / GetDefaultProfile) grows the Docker CLI binary by about 3%: +1.27 MB unstripped and +0.85 MB stripped on linux/arm64. Most of that is code a read-only consumer never calls.

The Docker CLI is adding the Docker Desktop session as the first credential source for Docker Hub and dhi.io, as docker/compose does today. The CLI only calls GetSecrets (through dockerhub).

This issue proposes two changes to the client, which together cut that growth by 85%:

  1. A read-only client package (−53%), non-breaking.
  2. That package calling GetSecrets without connect (−85% in total).

Binary growth with client v0.2.2

docker built from docker/cli master, and from the same tree with the client added:

platform unstripped stripped (-s -w)
linux/arm64 40,142,559 → 41,409,437 (+1,266,878, +3.2%) 27,590,816 → 28,442,784 (+851,968, +3.1%)
linux/amd64 42,932,536 → 44,250,534 (+1,317,998, +3.1%) 29,696,160 → 30,593,184 (+897,024, +3.0%)
darwin/arm64 41,662,322 → 42,970,274 (+1,307,952, +3.1%) 28,536,418 → 29,426,738 (+890,320, +3.1%)
windows/amd64 43,581,952 → 44,918,784 (+1,336,832, +3.1%) 30,532,096 → 31,450,112 (+918,016, +3.0%)

Modules newly linked: connectrpc.com/connect v1.19.1, github.com/docker/secrets-engine/client v0.2.2, github.com/docker/secrets-engine/x v0.8.5, golang.org/x/mod v0.41.0. Protobuf and net/http were already in the binary.

Symbol sizes added, by package (linux/arm64, go tool nm -size), 291,792 bytes in all:

package bytes
connectrpc.com/connect 173,748
client, client/dockerhub, client/realms 34,112
x/api/plugins/v1, pluginsv1connect 19,712
x/api/resolver, resolver/v1, resolverv1connect 12,584
x/secrets, x/realms, x/api, x/api/health/v1, healthv1connect 13,728
golang.org/x/mod/semver 2,560
the CLI's own code using the client 5,684

The rest of the growth is Go's per-function and per-type tables for that code (.gopclntab +410 KB, .go.type +163 KB, stripped) and DWARF.

Why a read-only consumer pays for the whole client

  • client.New wires six RPCs: GetSecrets, Authorize, ListPlugins, EnablePlugin, DisablePlugin, GetVersion. Each is its own connect.NewClient[Req, Res] instantiation, with its own copy of the call path, plus the request and response types and their descriptors.
  • Every method of *client is linked, because New returns it as an interface. From -ldflags=-dumpdep:
    type:*github.com/docker/secrets-engine/client.client <UsedInIface> -> github.com/docker/secrets-engine/client.(*client).ListPlugins
    type:*github.com/docker/secrets-engine/client.client <UsedInIface> -> github.com/docker/secrets-engine/client.(*client).Version
    
    So creating the plugin, version, and authorizer clients lazily inside the methods saves nothing. I measured it: +897 bytes.
  • Importing client also imports x/api/plugins/v1 and x/api/health/v1, whose descriptors are registered at init.

Option 1: a read-only client package

Add a package (name open, for example client/hub) that connects to a socket path and returns a dockerhub.ClientAuth, built on resolver.NewResolverClient only:

  • It does not import x/api/plugins/v1 or x/api/health/v1, and does not set up the authorizer.
  • ConnectError and its classification move to a leaf package, with an alias in client, so the new package can still return *ConnectError for "not running", "permission denied", and "timeout".
  • client.New, PluginManagement, and Version stay as they are, so existing users are not affected.

Measured on linux/arm64: +593,359 unstripped / +393,216 stripped, which saves 53% / 54%.

For comparison, removing only Version and plugin management from client itself (keeping Authorize) gives +767,243 / +524,288, a 39% saving, and breaks existing PluginManagement users. Authorize alone accounts for about 170 KB unstripped.

Option 2: GetSecrets without connect

In that package, call /resolver.v1.ResolverService/GetSecrets with the Connect protocol's unary encoding over net/http. The engine keeps serving with connect. A request is a POST with Content-Type: application/proto, Connect-Protocol-Version: 1, and a protobuf body. A non-200 response carries a JSON {"code", "message"}, mapped to secrets.ErrNotFound and secrets.ErrAccessDenied as resolver.NewResolverClient does today.

The prototype is about 60 lines:

func (r resolverClient) GetSecrets(ctx context.Context, pattern secrets.Pattern) ([]secrets.Envelope, error) {
	body, err := proto.Marshal(resolverv1.GetSecretsRequest_builder{Pattern: proto.String(pattern.String())}.Build())
	if err != nil {
		return nil, err
	}
	req, err := http.NewRequestWithContext(ctx, http.MethodPost, "http://unix"+resolverv1connect.ResolverServiceGetSecretsProcedure, bytes.NewReader(body))
	if err != nil {
		return nil, err
	}
	req.Header.Set("Content-Type", "application/proto")
	req.Header.Set("Connect-Protocol-Version", "1")
	resp, err := r.httpClient.Do(req)
	if err != nil {
		return nil, err
	}
	defer resp.Body.Close()
	data, err := io.ReadAll(io.LimitReader(resp.Body, 4<<20))
	if err != nil {
		return nil, err
	}
	if resp.StatusCode != http.StatusOK {
		var e struct {
			Code    string `json:"code"`
			Message string `json:"message"`
		}
		_ = json.Unmarshal(data, &e)
		switch e.Code {
		case "not_found":
			return nil, secrets.ErrNotFound
		case "permission_denied":
			return nil, secrets.ErrAccessDenied
		}
		return nil, fmt.Errorf("secrets engine: %s: %s %s", resp.Status, e.Code, e.Message)
	}
	var out resolverv1.GetSecretsResponse
	if err := proto.Unmarshal(data, &out); err != nil {
		return nil, err
	}
	return toEnvelopes(out.GetEnvelopes()), nil
}

toEnvelopes is the same conversion resolver.NewResolverClient does today.

The prototype builds the path from a string literal, because resolverv1connect imports connect. The real package would need the procedure path as a constant outside resolverv1connect.

Measured on linux/arm64 with options 1 and 2: +195,400 unstripped / +131,072 stripped, which saves 85% / 85%. What remains is dockerhub, x/secrets, x/api, the resolver/v1 messages, and golang.org/x/mod/semver.

Trade-offs: secrets-engine maintains the unary request and error mapping for this path, and it has no connect interceptors. It needs tests against the connect server in this repository.

Verification

Both prototypes ran in the Docker CLI against a standalone engine on macOS (5 runs per case, 1 for the hung engine):

  • Docker Hub and dhi.io pulls sent the session credentials.
  • Signed out, a socket with nothing listening, and no socket all fell back to the CLI's own credentials without a warning.
  • A hung engine fell back after the CLI's 5 s lookup timeout, with a warning.

How this was measured

docker was built from the same tree with go build -overlay, -tags grpcnotrace, CGO_ENABLED=0, and Go 1.27.1; stripped means -ldflags "-s -w". Stripped sizes move in 64 KiB steps. Release builds also use cgo and -buildmode=pie, which these builds did not.

Related: server-side connect code in clients

connect links 103 server-side symbols (19,584 bytes) into a client-only program. connect.clientConfig is stored in an interface value, which makes the protocol interface's NewHandler method reachable:

type:connectrpc.com/connect.clientConfig <UsedInIface> -> type:connectrpc.com/connect.protocol <UsedInIface>
type:*connectrpc.com/connect.protocolConnect <UsedInIface> -> connectrpc.com/connect.(*protocolConnect).NewHandler

That is for connectrpc, not this repository; option 2 avoids it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions