Repository navigation
fix(train): set input_data_config to None - #6154
Conversation
… provided When no input channels are provided, input_data_config must be None rather than an empty list to avoid SageMaker API validation errors.
|
@papriwal sorry to tag you directly, but I saw are active on this repo. I was hoping to get this PR looked at. It has been open for some time. Please let me know if I missed any required process. |
|
Claude finished @danb27's task in 1m 11s —— View job Review: fix(train): set input_data_config to None
The core change is correct for the reported bug: SageMaker's 🔴 Potential regression: local training mode will crash with
|
There was a problem hiding this comment.
🤖 AI code review
This PR fixes a regression where input_data_config was serialized as an empty list [] when no input channels were provided, which the SageMaker API rejects (min=1 constraint). The change sets it to None when falsy so the key is dropped by _serialize_dict (which omits None but preserves empty lists), restoring v2 behavior. I confirmed the serialization path drops None values and that TrainingJob.create accepts None for this optional argument. The change is correct, minimal, and the unit test is updated accordingly. Looks clean.
Reviewed commit 0414237. Automated review; verify before acting.
There was a problem hiding this comment.
🤖 AI code review
This PR fixes a regression where an empty input_data_config was serialized as [] (rejected by the SageMaker API's min-length constraint) instead of being omitted. For the managed job path, input_data_config is now set to None so serialization drops the key; for the local-container path, an or [] fallback preserves an iterable since _LocalContainer iterates over the list directly. The fix is correct, matches the documented v2 behavior and team guidance (Source: #6156), and is covered by parametrized tests for the omitted/None/empty-list cases on both paths. No issues found.
Reviewed commit 1f43627. Automated review; verify before acting.
test_passing_different_pipeline_variables_to_function chains a ScriptProcessor ProcessingStep with two @step function steps (3 jobs). The 600s wait budget was too low once instance provisioning is included, so func_2 was still "Executing" at the deadline and the test failed fleet-wide (same failure on main and on unrelated PRs aws#6154/aws#6261/aws#6400/ aws#5722), not just here. Raise it to 1500s, consistent with the other multi-step tests in this file (800-1000s). Formatting-only otherwise; no product code change.
…processors (#6364) * fix(processing): Fix V3 shape regressions in processors PySparkProcessor still built ProcessingInput/ProcessingOutput with the V2 source/destination keyword signature, so spark_event_logs_s3_uri and the configuration channel failed on V3. Build them with the V3 shapes (ProcessingS3Input/ProcessingS3Output) instead. Validate that submit_py_files, submit_jars and submit_files are lists and raise a clear ValueError otherwise. FrameworkProcessor.run(requirements=...) now threads the file name into the generated runproc.sh instead of hardcoding requirements.txt. Processor.run defaults kms_key to the configured output_kms_key so uploaded code is encrypted with the same key as job outputs. The requirements file is also honored on the custom entry_point (S3 source_dir) path, and a relative path such as reqs/cpu.txt is kept as-is because the source bundle preserves the source_dir layout. Integration tests (real jobs, submit-then-stop where the fix is in the request path) cover the code-upload KMS fallback, the FrameworkProcessor requirements file, and a V3 PySparkProcessor job ported from V2 that asserts the event-log output and the conf/py-files input shape; V3 had no Spark integ coverage before. Fixes #3809 Fixes #6253 Fixes #6252 Fixes #5805 Fixes #4874 --- X-AI-Prompt: Fix S-effort PySDK V3 bugs, processing-spark theme X-AI-Tool: Kiro * fix(processing): encrypt sourcedir.tar.gz with resolved kms_key FrameworkProcessor._pack_and_upload_code computed the output_kms_key fallback (evaluated_kms_key) only after calling _package_code, and passed the raw kms_key to _package_code. FrameworkProcessor.run does not apply the fallback before delegating here, so with output_kms_key set and no explicit kms_key, sourcedir.tar.gz -- the largest, most sensitive artifact -- was uploaded unencrypted, while install_requirements.py and runproc.sh were encrypted. This left #4874 unfixed for FrameworkProcessor. Resolve the key once, before packaging, and pass it into _package_code so all three artifacts share the same key. An explicitly passed kms_key still wins; behaviour when both are unset is unchanged. Add unit tests asserting _package_code and the helper-script uploads all receive the resolved key on the fallback path (regression for #4874) and that an explicit key overrides output_kms_key. Relates-to #4874 * fix(processing): reject pipeline variable for requirements argument FrameworkProcessor.run bakes the requirements path into the generated runproc.sh script, so it cannot be a pipeline variable. Passing a ParameterString previously surfaced as an opaque "TypeError: expected str, bytes or os.PathLike object, not ParameterString" from os.path.isabs deep in script generation. Add an explicit is_pipeline_variable guard at the top of FrameworkProcessor.run, raising a clear ValueError, matching the existing code and submit_app guards. Add a unit test. Relates-to #5805 * test(workflow): raise wait_duration for 3-job step-decorator test test_passing_different_pipeline_variables_to_function chains a ScriptProcessor ProcessingStep with two @step function steps (3 jobs). The 600s wait budget was too low once instance provisioning is included, so func_2 was still "Executing" at the deadline and the test failed fleet-wide (same failure on main and on unrelated PRs #6154/#6261/#6400/ #5722), not just here. Raise it to 1500s, consistent with the other multi-step tests in this file (800-1000s). Formatting-only otherwise; no product code change. * chore(codestyle): black-format test_processing.py Apply black to tests/unit/test_processing.py (added two blank lines before TestProcessorLocalModeRole), fixing the failing codestyle-doc-tests (sagemaker-core) black-check and flake8 E302. Formatting-only. --------- Co-authored-by: Amarjeet LNU <jamjee@amazon.com> Co-authored-by: Roja Reddy Sareddy <rsareddy@amazon.com>
Issue #, if available: N/A - small fix, can create if required.
Description of changes: When no input channels are provided, input_data_config must be None rather than an empty list to avoid SageMaker API validation errors. This appears to be a regression caused by this recent fix. Before,
input_data_config=[]would be dropped from the config silently, avoiding the validation issue in the API. Now, however, it is not dropped, which breaks the validation.By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.