Where Does The Performance Boost Of Map Or List Comprehension Implementations Over Calling A Function Over A Loop Come From?

I understand that you could be more efficient with memory in the implementation of map than in how you might do it over a loop. However, I see that using a map function over callin

Solution 1:

So, the issue with function-calls in loops, in a dynamic language like Python, is that the interpreter has to evalute the refernces each time, and that is costly, especially for global variables. However, notice what happens when you make things local:

import time
def square(x):
    return x*x

def test_loop(x):
    res = []
    for i in x:
        res.append(square(i))
    return res


def test_map(x):
    return  list(map(square,x))

def test_map_local(x, square=square):
    return  list(map(square,x))


def test_loop_local(x, square=square):
    res = []
    for i in x:
        res.append(square(i))
    return res

def test_loop_local_local(x, square=square):
    res = []
    append = res.append
    for i in x:
        append(square(i))
    return res

def test_comprehension(x):
    return [square(i) for i in x]

def test_comprehension_local(x, square=square):
    return [square(i) for i in x]

x = range(1,10000000)

start = time.time()
test_loop(x)
stop = time.time()
print("Loop:", stop - start,"seconds")

start = time.time()
test_loop_local(x)
stop = time.time()
print("Loop-local:", stop - start, "seconds")

start = time.time()
test_loop_local_local(x)
stop = time.time()
print("Loop-local-local:", stop - start, "seconds")

start = time.time()
test_map(x)
stop = time.time()
print("Map:", stop - start, "seconds")

start = time.time()
test_map_local(x)
stop = time.time()
print("Map-local:", stop - start, "seconds")

start = time.time()
test_comprehension(x)
stop = time.time()
print("Comprehesion:", stop - start, "seconds")

start = time.time()
test_comprehension_local(x)
stop = time.time()
print("Comprehesion-local:", stop - start, "seconds")

Results:

Loop: 3.9749317169189453 secondsLoop-local: 3.686530828475952 secondsLoop-local-local: 3.006138563156128 secondsMap: 3.1068732738494873 secondsMap-local: 3.1318843364715576 secondsComprehesion: 2.973804235458374 secondsComprehesion-local: 2.7370445728302 seconds

So, making the function look-up in map local doesn't really help, as I would expect because map does a single look-up at the beginning. What really surprised me is that there seems to be a non-negligible difference between a comprehension and a comprehension with the function made local. Not sure if this is just noise, however.

Solution 2:

The difference is due to append, not map. Try this:

res = []
res = [square2(foo) for foo in foobar]