Skip to content

fix(encoder): strip ANSI escape and control characters from non-terminal output formats - #2902

Open
AdamMagued wants to merge 1 commit into
mikefarah:masterfrom
AdamMagued:fix-issue-2901-strip-ansi-encoders
Open

AdamMagued wants to merge 1 commit into
mikefarah:masterfrom
AdamMagued:fix-issue-2901-strip-ansi-encoders

Conversation

@AdamMagued

Copy link
Copy Markdown

This pull request resolves #2901 by filtering non-printable ASCII control characters in non-terminal output encoders.

Direct text formats (properties, csv, tsv, shell) write scalar values to output streams without character filtering. When input documents contain terminal escape sequences or control bytes, these encoders emit raw control bytes to stdout.

Changes made:

  • Added sanitizeControlChars in pkg/yqlib/utils.go to filter ASCII control characters (codepoints < 32 and 127) while preserving tabs, newlines, and carriage returns.
  • Applied sanitizeControlChars to scalar values in pkg/yqlib/encoder_csv.go (CSV and TSV formats).
  • Applied sanitizeControlChars to property keys and values in pkg/yqlib/encoder_properties.go.
  • Applied sanitizeControlChars to scalar values in pkg/yqlib/encoder_shellvariables.go.
  • Added unit test suites covering escape sequences and control characters across all four formats in pkg/yqlib/csv_test.go, pkg/yqlib/encoder_properties_test.go, pkg/yqlib/encoder_shellvariables_test.go, and pkg/yqlib/utils_test.go.

…nal output formats

Direct string output formats (properties, csv, tsv, shell) emit scalar values directly without filtering control characters. When processing documents that contain embedded escape sequences, these formats emit raw control bytes to terminal emulators.

Add sanitizeControlChars in pkg/yqlib/utils.go to filter ASCII control characters (codepoints < 32 and 127) while preserving tabs, newlines, and carriage returns. Apply sanitization to scalar values across the properties, csv, tsv, and shell variable encoders.

Fixes mikefarah#2901
Copilot AI balanced review requested due to automatic review settings October 11, 2026 00:21

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

properties/csv/tsv/shell output encoders don't strip terminal escape sequences

2 participants