Skip to content

Apply always-on C2J preprocessing on Smithy path - #7442

Open
Fred1155 wants to merge 1 commit into
feature/master/smithy-migrationfrom
bole/smithy_always_on_preprocessing
Open

Fred1155 wants to merge 1 commit into
feature/master/smithy-migrationfrom
bole/smithy_always_on_preprocessing

Conversation

@Fred1155

Copy link
Copy Markdown
Contributor

Motivation and Context

Before translating a C2J model, codegen runs a fixed chain of preprocessors for every service (DefaultCustomizationProcessor). Some of them change how the client is generated; others only reject models that codegen cannot handle. The Smithy path did not run them yet. This PR adds the Smithy-path equivalent of five of them:

C2J preprocessor What it does Without this PR, the Smithy path...
MetadataModifiersProcessor (protocol part) A customServiceMetadata.protocol setting in customization.config replaces the model's protocol. Only Kinesis uses it, to generate with the legacy cbor protocol, which has no Smithy trait. generates Kinesis as awsJson
SmithyRpcV2CborProtocolProcessor rpcv2Cbor operations are sent as POST /service/{service}/operation/{operation} sends every rpcv2Cbor operation to /
NewAndLegacyEventStreamProcessor Kinesis and Transcribe Streaming generate their event streams with an older scheme, so a new event stream on either service must fail the build until someone adds it to the approved list accepts any new event stream
ExplicitStringPayloadQueryProtocolProcessor Rejects query/ec2 operations whose @httpPayload is a string, which those protocols do not support accepts them
LowercaseShapeValidatorProcessor Rejects structure names that start with a lowercase letter accepts them

Modifications

  • ProtocolUtils: new resolveProtocol(ServiceIndex, ServiceShape, CustomizationConfig). With no customServiceMetadata.protocol, it returns the same protocol as before, from the service's Smithy protocol traits. With an override, it passes the override to the existing C2J resolver, so the value is checked and resolved exactly as on the C2J path.
  • SmithyIntermediateModelBuilder: resolves the protocol once, with the override, and runs the new source-model validation at the start of build().
  • AddSmithyMetadata: takes the resolved protocol as a parameter instead of resolving it again, so the metadata and the shape translation always agree on the protocol.
  • AddSmithyInputShapes: rpcv2Cbor requests use POST /service/{service shape name}/operation/{operation name}. Other RPC protocols still use POST /.
  • New SmithySourceModelValidator: the three validations above, run on the shapes the service reaches after the smithy-build transforms, with the same error messages as the C2J processors.
  • NewAndLegacyEventStreamProcessor: adds a package-private validateEventStreams(serviceId, names) so the Smithy validator reuses the same approved-stream list instead of copying it.

Differences from the C2J processors, none of which changes the result for any current service:

  • rpcv2Cbor URI: C2J builds it from metadata.targetPrefix; the Smithy path uses the service shape name. They are equal for every current rpcv2Cbor service.
  • The approved event-stream list is keyed by C2J metadata.serviceId; the Smithy path uses the sdkId of @aws.api#service. They are equal for every service.
  • The validations check only shapes the service reaches, after transforms, so a shape that is in the model file but unused does not fail the build. C2J checks every shape in service-2.json.

Two other C2J preprocessors in the chain are specific to S3 (S3RemoveBucketFromUriProcessor) and S3 Control (S3ControlRemoveAccountIdHostPrefixProcessor). They are not ported here. They will be added as service-specific transforms when those services move to the Smithy path, so shared codegen does not gain service-specific code.

Testing

  • New SmithySourceModelValidationTest, covering each validation (failing and passing case).
  • Updated AddSmithyInputShapesTest (rpcv2Cbor request URI) and AddSmithyMetadataTest (protocol passed in).
  • Compared the Smithy and C2J intermediate models for all services, using local tooling that is not part of this PR: no differences remain in rpcv2Cbor request URIs.
  • mvn clean install -pl :codegen passes.

Screenshots (if appropriate)

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)

Checklist

  • I have read the CONTRIBUTING document
  • Local run of mvn install succeeds
  • My code follows the code style of this project
  • My change requires a change to the Javadoc documentation
  • I have updated the Javadoc documentation accordingly
  • I have added tests to cover my changes
  • All new and existing tests passed
  • I have added a changelog entry. Adding a new entry must be accomplished by running the scripts/new-change script and following the instructions. Commit the new file created by the script in .changes/next-release with your changes.
  • My change is to implement 1.11 parity feature and I have updated LaunchChangelog

License

  • I confirm that this pull request can be released under the Apache 2 license

Match the C2J processors that run for every service: set the
smithy-rpc-v2-cbor operation request URI, run the event-stream,
explicit String payload, and lowercase shape-name validations, and
honor customServiceMetadata.protocol when resolving the protocol. The
protocol is resolved once and passed to metadata construction.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant