Why My Multi-threading Program Is Slow?

I'm trying to make my program run faster, using threads but it takes too many time. The code must compute two kinds of matrices (word_level where I compare every two words of the q

Solution 1:

I'm just going to focus on these lines of code in this answer

thread = Thread(target = process_documents(documents, index, model, alpha, outOfVocab, out_w, out_s, queries, stemming, stoplist, q))
thread.start()

From the documentation https://docs.python.org/2/library/threading.html

target is the callable object to be invoked by the run() method. Defaults to None, meaning nothing is called.

Target should be a callable. In your code you are passing in the result of a call to process_documents. What you want to do is say target=process_documents (i.e. pass in the function itself - which is a callable) and also pass in the args/kwargs as needed.

At the moment your code is running sequentially, every call to process_documents is happening the same thread. You need to give the thread the job you want it to do, not the result of the job.