Skip to content

fix/issue 5897 crlf sm train - #6088

Merged
mohamedzeidan2021 merged 3 commits into
aws:masterfrom
wasim-builds:fix/issue-5897-crlf-sm-train
Oct 6, 2026
Merged

mohamedzeidan2021 merged 3 commits into
aws:masterfrom
wasim-builds:fix/issue-5897-crlf-sm-train

Conversation

@wasim-builds

@wasim-builds wasim-builds commented Jul 23, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #5897

Problem

ModelTrainer writes files with platform-default line endings, causing CRLF/LF inconsistencies on Windows.

Fix

Explicitly normalize line endings to LF when writing training artifacts.

Scope

Single-line change in sagemaker-train/src/sagemaker/train/model_trainer.py.

Verification

Training artifacts now have consistent LF line endings regardless of OS.

@wasim-builds
wasim-builds force-pushed the fix/issue-5897-crlf-sm-train branch from 15e650b to fa7b7aa Compare July 25, 2026 09:01
rsareddy0329 added a commit that referenced this pull request Sep 30, 2026
FrameworkProcessor._package_code read and deleted its temporary tar.gz
while the NamedTemporaryFile handle was still open. On Windows that
raises PermissionError (WinError 32) because the file is still held by
the open handle. Close the handle first, read the archive in a with-open
context, and unlink it in a finally block so the temp file is removed
on every path.

The LF line-ending fixes for sm_train.sh and the repack launcher that
originally shared this branch are covered by open PRs #6255 / #6088
(sm_train.sh) and #6313 (repack launcher), so they are not repeated here.

Fixes #5873

---
X-AI-Prompt: Fix S-effort PySDK V3 bugs, windows theme
X-AI-Tool: Kiro

Co-authored-by: rsareddy0329 <rsareddy0329@gmail.com>
@wasim-builds

Copy link
Copy Markdown
Contributor Author

Hi everyone,

Quick follow-up on this PR concerning fix/issue 5897 crlf sm train in aws/sagemaker-python-sdk. Working on Amazon SageMaker Python SDK machine learning and model deployment pipelines has been a great experience, and I am very keen to continue contributing to aws/sagemaker-python-sdk.

I am also open to collaborating on a contract/freelance basis, discussing full-time opportunities, or joining the organization/team.

You can view my background and other open-source contributions at https://github.com/wasim-builds.

@mohamedzeidan2021

Copy link
Copy Markdown
Collaborator

Apologies for opening and re-opening, will add unit tests and look to get this merged!

Adds the unit test from aws#5907 plus an OS-independent regression guard.

The byte-level check from aws#5907 (no CRLF in the written sm_train.sh) only
fails on Windows, since POSIX text-mode open() never translates \n. CI runs
on Linux, so that test passes with or without the fix. The second test
inspects the open() call for sm_train.sh and asserts newline="\n", mirroring
the approach merged in aws#6313 for the mlops repack launcher, so the guard
holds on any host OS.
Copilot AI balanced review requested due to automatic review settings October 5, 2026 20:03

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@mohamedzeidan2021

Copy link
Copy Markdown
Collaborator

Codebuild succeeded, tests pass: https://tiny.amazon.com/kmwe9wt0/usweconsawsamazcodecode7296

@mohamedzeidan2021
mohamedzeidan2021 merged commit febee60 into aws:master Oct 6, 2026
19 of 26 checks passed
@github-actions

github-actions Bot commented Oct 7, 2026 •

Copy link
Copy Markdown

Claude finished @mohamedzeidan2021's task in 44s —— View job


Code Review — PR #6088 (CRLF → LF for sm_train.sh)

  • Read the PR diff (/tmp/pr.diff)
  • Review the source change in model_trainer.py
  • Review the added unit tests
  • Check for similar unprotected script writes nearby

Verdict: ✅ Looks good

The one-line change is correct and precisely scoped.

sagemaker-train/src/sagemaker/train/model_trainer.py:1394

with open(os.path.join(tmp_dir.name, TRAIN_SCRIPT), "w", newline="\n") as f:
    f.write(train_script)

On Windows, text-mode open(..., "w") with the default newline=None translates every \n to os.linesep (\r\n). Since sm_train.sh is a bash script executed inside a Linux training container, CRLF endings break it (the classic #!/bin/bash\r: bad interpreter failure, plus \r corrupting any line). Pinning newline="\n" guarantees LF regardless of the host OS. This is the right fix for #5897.

Observations

  • Correctness: newline="\n" disables newline translation on write without altering the written string — behavior is unchanged on POSIX and fixed on Windows. No public API or signature change, so no backward-compatibility concern for SDK consumers.
  • Other writes are fine: the two nearby writes (_write_source_code_json:1302, _write_distributed_json:1313) use json.dumps(...) without indent, producing single-line output with no embedded newlines, so they are not affected by CRLF translation. No action needed there.
  • Tests: two tests are added.
    • test_prepare_train_script_opened_with_lf_newline is the meaningful one — it intercepts the open call and asserts newline="\n" was passed, so it fails without the fix regardless of the test host OS. This is the correct way to guard an OS-dependent behavior that can't be reproduced on the Linux CI runner.
    • test_prepare_train_script_uses_lf_line_endings is a weaker byte-level check (it would pass on POSIX even without the fix, as the docstring of the sibling test honestly notes), but it's a reasonable complementary assertion.

Minor (non-blocking) nits

  • Both tests manually defeat TemporaryDirectory cleanup (tmp_dir._cleanup = False; tmp_dir.cleanup = lambda: None) and rely on a finally: shutil.rmtree(...). Mutating the private _cleanup attribute is a bit fragile across Python versions; a tmp_dir.name pointed at a pytest tmp_path fixture would be cleaner, but this is a test-only concern and not worth blocking.
  • test_prepare_train_script_uses_lf_line_endings is largely subsumed by the second test. Keeping it is harmless.

No correctness, security, or performance issues found. The PR is already merged; no changes requested.

Note: the inline-comment tool was not available in this run, so findings are consolidated here.

This branch was successfully deployed

1 active deployment
manual-approval — b413b884 Deployed Oct 5, 2026 by mohamedzeidan2021 via wait-for-approval #1634
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Windows host writes sm_train.sh with CRLF in SDK v3, causing SageMaker training job bootstrap failure ($'\r': command not found)

3 participants