Clear Memory In Python Loop

How I can clear memory in this Python loop? import concurrent.futures as futures with futures.ThreadPoolExecutor(max_workers=100) as executor: fs = [executor.submit(get_data, u

Solution 1:

My solution would be as such:
import concurrent.futures as futures

#split the original grand list into smaller batches  

batchurlList = [grandUrlList[x:x+batchSize] for x in range(0, len(grandUrlList), batchSize)]
for tmpurlList in batchurlList:
    with futures.ThreadPoolExecutor(max_workers=100) as executor:
        myfuture = {executor.submit(myFunction, url): url for url in tmpurlList}
        for future in futures.as_completed(myfuture, timeout=60):
            originalUrl = myfuture[future]
            results.append(future.result())

Solution 2:

I think I had the same problem recently, The answer is not del but introducing a sleep func... Try;

import time
import concurrent.futures as futures
with futures.ThreadPoolExecutor(max_workers=100) as executor:
    fs = [executor.submit(get_data, url) for url in link]
    for i, f in enumerate(futures.as_completed(fs)):
           x= (f.result())
           results.append(x)
           time.sleep(n_seconds)

Or something like this (I used a while loop over a list of urls)