Skip to content

Non UTF-8 json payload produce a 500 and a stack trace #427

Closed

Description

@lindycoder

Hi,

First of all thanks for thie library, while doing some testing with random payload we stumbled upon a corner case:

Actual Behavior:

If non utf-8 data is sent with header Application/Json the core's parse_json will raise a UnicodeDecodeError, which generates a 500.

Expected Behavior

A 400 BAD REQUEST

Notes

  • Python 3.6
  • webargs==5.5.1

I would handle the problem my self but as you can see in the stack trace, my code is never reached so i would need a wrapper above @use_args

Reproduce

CODE (From README example with methods= added):

from flask import Flask
from webargs import fields
from webargs.flaskparser import use_args

app = Flask(__name__)


@app.route("/", methods=['GET', 'POST'])
@use_args({"name": fields.Str(required=True)})
def index(args):
    return "Hello " + args["name"]


if __name__ == "__main__":
    app.run()
 

Curl invocation

$ curl http://localhost:5000/ -H 'Content-Type: Application/Json' -d $(python -c "import os; import sys;  os.write(sys.stdout.fileno(), b'\xfe')")

Output

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<title>500 Internal Server Error</title>
<h1>Internal Server Error</h1>
<p>The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.</p>

Full Stack trace

.../.tox/py36/bin/python3.6 .../tests/webargs_poc.py
 * Serving Flask app "webargs_poc" (lazy loading)
 * Environment: production
   WARNING: Do not use the development server in a production environment.
   Use a production WSGI server instead.
 * Debug mode: off
 * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
[2019-09-23 09:07:54,383] ERROR in app: Exception on / [POST]
Traceback (most recent call last):
  File ".../.tox/py36/lib/python3.6/site-packages/flask/app.py", line 2292, in wsgi_app
    response = self.full_dispatch_request()
  File ".../.tox/py36/lib/python3.6/site-packages/flask/app.py", line 1815, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File ".../.tox/py36/lib/python3.6/site-packages/flask/app.py", line 1718, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File ".../.tox/py36/lib/python3.6/site-packages/flask/_compat.py", line 35, in reraise
    raise value
  File ".../.tox/py36/lib/python3.6/site-packages/flask/app.py", line 1813, in full_dispatch_request
    rv = self.dispatch_request()
  File ".../.tox/py36/lib/python3.6/site-packages/flask/app.py", line 1799, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File ".../.tox/py36/lib/python3.6/site-packages/webargs/core.py", line 444, in wrapper
    error_headers=error_headers,
  File ".../.tox/py36/lib/python3.6/site-packages/webargs/core.py", line 347, in parse
    schema=schema, req=req, locations=locations or self.locations
  File ".../.tox/py36/lib/python3.6/site-packages/webargs/core.py", line 263, in _parse_request
    parsed_value = self.parse_arg(argname, field_obj, req, locations)
  File ".../.tox/py36/lib/python3.6/site-packages/webargs/core.py", line 226, in parse_arg
    value = self._get_value(name, field, req=req, location=location)
  File ".../.tox/py36/lib/python3.6/site-packages/webargs/core.py", line 202, in _get_value
    return function(req, name, argobj)
  File ".../.tox/py36/lib/python3.6/site-packages/webargs/flaskparser.py", line 69, in parse_json
    self._cache["json"] = json_data = core.parse_json(data)
  File ".../.tox/py36/lib/python3.6/site-packages/webargs/core.py", line 115, in parse_json
    s = s.decode(encoding)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xfe in position 0: invalid start byte
127.0.0.1 - - [23/Sep/2019 09:07:54] "POST / HTTP/1.1" 500 -

Activity

  1. added a commit that references this issue on Sep 23, 2019
    7b2ef63
  2. added a commit that references this issue on Oct 6, 2019
    6347b4b
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions