Parallel Processing - Pool - Python

I'm trying to learn how to use multiprocessing in Python. I read about multiprocessing, and I trying to do something like this: I have the following class(partial code), which has

Solution 1:

Python's multiprocessing doesn't share memory (unless you explicitly tell it to). That means that you won't see "side effects" of any function that gets run in a worker processes. Your generate_voronoi_diagram method works by adding data to an entry value, which is a side effect. In order to see the results, you need to be passing it back as a return values from your function.

Here's one approach that handles the entry instance as an argument and return value:

defdo_voroni(entry, seeds):
    entry.generate_voronoi_diagram(seeds)
    return entry

Now, you can use this function in your worker processes:

if __name__ == "__main__":
    entries = [ImD.ImageData(width, height) for _ inrange(images)]
    seeds = numpy.random.poisson(100, images) # array of values

    pool = multiprocessing.Pool()
    for i, e inenumerate(pool.starmap_async(do_voroni, zip(entries, seeds))):
        e.generate_heat_map_image("ImagesOutput/Entries/Entry{:02d}".format(i))

The e values in the loop are not references to the values in the entries list. Rather, they're copies of those objects, which have been passed out to the worker process (which added data to them) and then passed back.

Solution 2:

I might be wrong, but I think you should use

res = p.apply_async(entry.generate_voronoi_diagram,(seeds))

res.get(timeout=1)

you may get Can't pickle type 'instancemethod'

i think the easiest way is something like

import random
from multiprocessing import Pool


class ImageData:

    def generate_voronoi_diagram(self, seeds):
        ooxx

    def generate_heat_map_image(self, path):
        ooxx

def allinone(obj, seeds, path):
    obj.generate_voronoi_diagram(seeds)
    obj.generate_heat_map_image(path)

if __name__ == "__main__":
    entries = []
    seeds = random.random()
    p = Pool()
    entry = ImageData()
    res = p.apply_async(allinone, (entry, seeds, 'tmp.txt'))
    res.get(timeout=1)