Extract List From Dict Of Lists Then Append To Dataframe

I'm trying to extract a field from a json that contains a list then append that list to a dataframe, but I'm running in to a few different errors. I think I can write it to a csv t

Solution 1:

You can maybe write a temporary file with StringIO, like it's done here.

Then for the second part you could do

if','indata and ', ' not indata:
    data = data.replace(',', '^')

Solution 2:

You can try the following

hit_json = json.loads(hit)
forlnin hit_json.get('hits').get('hits'):
     data = ln['_source']["payload"].split(",")
     df.loc[len(df)] = pd.Series(data, index=range(len(data)))
print(df)

The benefit of the loc is that you will not create a new dataframe each time so it will be fast. You can find the post here.

I would also like to suggest an alternative that can be faster. First create a dictionary with all the data and then dump the dictionary into a dataframe.