Cosine Similarity Between Two Words In A List

I am defining a function which takes a list of words and returns information about the words in the list that have non-zero, cosine similarity between each other (along with the si

Solution 1:

You could define these two functions

defword2vec(word):
    from collections import Counter
    from math import sqrt

    # count the characters in word
    cw = Counter(word)
    # precomputes a set of the different characters
    sw = set(cw)
    # precomputes the "length" of the word vector
    lw = sqrt(sum(c*c for c in cw.values()))

    # return a tuplereturn cw, sw, lw

defcosdis(v1, v2):
    # which characters are common to the two words?
    common = v1[1].intersection(v2[1])
    # by definition of cosine distance we havereturnsum(v1[0][ch]*v2[0][ch] for ch in common)/v1[2]/v2[2]

and use them as in this example

>>>a = 'safasfeqefscwaeeafweeaeawaw'>>>b = 'tsafdstrdfadsdfdswdfafdwaed'>>>c = 'optykop;lvhopijresokpghwji7'>>>>>>va = word2vec(a)>>>vb = word2vec(b)>>>vc = word2vec(c)>>>>>>print cosdis(va,vb)
0.551843662321
>>>print cosdis(vb,vc)
0.113746579656
>>>print cosdis(vc,va)
0.153494378078

BTW, the word2vec that you mention in a tag is quite a different business, that requires that one of us take a great deal of time and commitment for studying it and guess what, I'm not that one...

Solution 2:

What about this?

scipy.spatial.distance.cosine(word2vec(a),word2vec(b))

You can use word2vec library for that.