Encoding And Then Decoding Json With Non-ascii Characters In Python 2.7

I have a python application that encodes some objects to json, passes the json string to another program, and then reads in a possibly modified version of that json string. I nee

Solution 1:

The easiest way to fix this is to go to the thing that is generating this dict and properly encode things there as utf-8. Currently, your keys are encoded as CP-1252.

print('\xe2'.decode('cp1252'))
â

If you can't fix at the source, you'll need to do some post-processing.

d = {'\xe2': None}

fixed_d = {k.decode('cp1252'):v for k,v in d.iteritems()}

json.dumps(fixed_d)
Out[24]: '{"\\u00e2": null}'

json_dict_with_unicode_keys = json.dumps(fixed_d)

json_dict_with_unicode_keys
Out[32]: '{"\\u00e2": null}'print(json.loads(json_dict_with_unicode_keys).keys()[0])
â

(Some of the content of this answer assumes you're on python 2, there are differences in unicode handling in py3)