Repository navigation
Conversation
Add
binmode STDOUT, ':encoding(UTF-8)';
inside pod2html, so that output is properly utf8 encoded.
Currently, some UTF-8 characters in pod files are producing mojibake
when rendered as HTML on Github.
Example:
https://github.com/Raku/old-design-docs/blob/master/S03-operators.pod
X Junctive and & (&) ∩
but this should be
X Junctive and & (&) ∩
I also added a test -- note this uses `=begin html` because Pod::Simple 3.48 escapes
ordinary text to ASCII entities.
There was a problem hiding this comment.
🟢 Approval recommended
The review records no active issues with the UTF-8 conversion change or its regression-test fixtures.
No open issues 🎉
What changed in this PR
This pull request fixes UTF-8 encoding for .pod files converted to HTML, preventing non-ASCII characters from appearing as mojibake.
Changes:
- Enables UTF-8 encoding for
pod2htmloutput. - Adds a POD fixture containing representative Unicode characters.
- Adds the expected UTF-8 HTML rendering.
| File | Description |
|---|---|
test/markups/README.utf8.pod.html |
Adds expected UTF-8 HTML output. |
test/markups/README.utf8.pod |
Adds Unicode regression-test input. |
lib/github/commands/pod2html |
Enables UTF-8 encoding for generated HTML output. |
This pull request is too complex or risky for automated approval.
Review effort: Lite (auto)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
summary
Some UTF-8 characters in POD files are not rendering properly when rendered as HTML on GitHub.
fix
Add
inside pod2html, so that output is properly utf8 encoded.
examples
https://github.com/Raku/old-design-docs/blob/master/S03-operators.pod
The text below is rendered as
but it should be
See #1637 for another example.
Note that both of these examples start with
=encoding utf8which indicates utf-8 encoded text.testing
I added a test -- note that this test uses
=begin htmlbecause Pod::Simple 3.48 escapes ordinary text to ASCII entities.Sample local tests using that file:
before:
after: