Unicodeencodeerror On Linux But Not On Windows

I'm getting an UnicodeEncodeError: 'ascii' codec can't encode exception when I try to print a Unicode string on Linux. On Windows I do not get the error. The code executed on Linux

Solution 1:

It's very likely that your locale and/or environment is broken, not installed, not set or set to C. Python uses the locale settings to apply the correct encoder on stdout. This allows Unicodes to be encoded to the appropriate encoding.

If you're running Python from the command line, make sure your locale is healthy. Type locale and your should see something like:

 $ locale
LANG=en_GB.UTF-8
LANGUAGE=
LC_CTYPE="en_GB.UTF-8"
LC_NUMERIC="en_GB.UTF-8"
LC_TIME="en_GB.UTF-8"
LC_COLLATE="en_GB.UTF-8"
LC_MONETARY="en_GB.UTF-8"
LC_MESSAGES="en_GB.UTF-8"
LC_PAPER="en_GB.UTF-8"
LC_NAME="en_GB.UTF-8"
LC_ADDRESS="en_GB.UTF-8"
LC_TELEPHONE="en_GB.UTF-8"
LC_MEASUREMENT="en_GB.UTF-8"
LC_IDENTIFICATION="en_GB.UTF-8"
LC_ALL=
 $

If you see error messages or if LANG=C or similar, Python will use an ASCII encoder, which rejects non-ASCII characters.

To find the locales installed on your system, type locale -a. Select the appropriate locale, ideally one ending in "UTF-8", and set LANG accordingly. E.g.

LANG=en_GB.UTF-8

The run locale again and check for errors. If you still get errors then you will need to research how to rebuild your locales for your distribution.

If you're running within an IDE or you're unable to fix your then you may have success with adding the following environment variable to your shell or IDE run configuration:

export PYTHONIOENCODING=utf-8

This tells Python to ignore the locale and apply a UTF-8 encoder to stdout.

You can validate what Python is using for the locale by using the locale module in Python. My healthy locale returns:

>>> import locale
>>> locale.getdefaultlocale()
('en_GB', 'UTF-8')
>>> locale.getpreferredencoding()
'UTF-8'

An unhealthy locale will return US-ASCII for locale.getpreferredencoding()

Solution 2:

you can try:

printu"{0}".format(str)

or

printu"{0}".format(l.decode('utf-8'))