Numpy Array Of Distances To List Of (row,col,distance)

I have an nd array that looks as follows: [[ 0. 1.73205081 6.40312424 7.21110255 2.44948974] [ 1.73205081 0. 5.09901951 5.91607978 1. ] [ 6.403124

Solution 1:

squareform does all this. Read the docs and experiment. It works in both directions. If you give it a matrix it returns the upper triangle values (condensed form). If you give it those values, it returns the matrix.

In [668]: M
Out[668]: 
array([[ 0. ,  0.1,  0.5,  0.2],
       [ 0.1,  0. ,  2. ,  0.3],
       [ 0.5,  2. ,  0. ,  0.2],
       [ 0.2,  0.3,  0.2,  0. ]])
In [669]: spatial.distance.squareform(M)
Out[669]: array([ 0.1,  0.5,  0.2,  2. ,  0.3,  0.2])
In [670]: v=spatial.distance.squareform(M)
In [671]: v
Out[671]: array([ 0.1,  0.5,  0.2,  2. ,  0.3,  0.2])
In [672]: spatial.distance.squareform(v)
Out[672]: 
array([[ 0. ,  0.1,  0.5,  0.2],
       [ 0.1,  0. ,  2. ,  0.3],
       [ 0.5,  2. ,  0. ,  0.2],
       [ 0.2,  0.3,  0.2,  0. ]])

You can also specify a force and checks parameter, but without those it just goes by the shape.

Indicies can come from triu

In [677]: np.triu_indices(4,1)
Out[677]: 
(array([0, 0, 0, 1, 1, 2], dtype=int32),
 array([1, 2, 3, 2, 3, 3], dtype=int32))

In [680]: np.vstack((np.triu_indices(4,1),v)).T
Out[680]: 
array([[ 0. ,  1. ,  0.1],
       [ 0. ,  2. ,  0.5],
       [ 0. ,  3. ,  0.2],
       [ 1. ,  2. ,  2. ],
       [ 1. ,  3. ,  0.3],
       [ 2. ,  3. ,  0.2]])

Just to check, we can fill in a 4x4 matrix with these values

In [686]: A=np.vstack((np.triu_indices(4,1),v)).T
In [687]: MM = np.zeros((4,4))
In [688]: MM[A[:,0].astype(int),A[:,1].astype(int)]=A[:,2]
In [689]: MM
Out[689]: 
array([[ 0. ,  0.1,  0.5,  0.2],
       [ 0. ,  0. ,  2. ,  0.3],
       [ 0. ,  0. ,  0. ,  0.2],
       [ 0. ,  0. ,  0. ,  0. ]])

Those triu indices can also fetch the values from M:

In [693]: I,J = np.triu_indices(4,1)
In [694]: M[I,J]
Out[694]: array([ 0.1,  0.5,  0.2,  2. ,  0.3,  0.2])

squareform uses compiled code in spatial.distance._distance_wrap so I expect it will be quite fast for large arrays. Only problem it just returns the condensed form values, but not the indices. But given the shape,the indices can always be calculated. They don't need to be stored with the values.

Solution 2:

If your input is x, first generate the indices:

i0,i1 = np.indices(x.shape)

Then:

np.concatenate((i1,i0,x)).reshape(3,5,5).T

That gives you the first result--for the entire matrix.

As for taking only the upper triangle, you might considering trying np.triu() but I'm not sure exactly what result you're looking for. You can probably figure out how to mask the parts you don't want now though.

Solution 3:

you can try this,

print([(x,y, value) for (x,y), value in np.ndenumerate(numpymatrixarray)])

output [(0, 0, 0.0), (0, 1, 1.7320508100000001), (0, 2, 6.4031242400000004), (0, 3, 7.2111025499999997), (0, 4, 2.4494897400000002), (1, 0, 1.7320508100000001), (1, 1, 0.0), (1, 2, 5.0990195099999998), (1, 3, 5.9160797799999996), (1, 4, 1.0), (2, 0, 6.4031242400000004), (2, 1, 5.0990195099999998), (2, 2, 0.0), (2, 3, 1.0), (2, 4, 4.3588989400000004), (3, 0, 7.2111025499999997), (3, 1, 5.9160797799999996), (3, 2, 1.0), (3, 3, 0.0), (3, 4, 5.0990195099999998), (4, 0, 2.4494897400000002), (4, 1, 1.0), (4, 2, 4.3588989400000004), (4, 3, 5.0990195099999998), (4, 4, 0.0)]

Solution 4:

Do you really want the top triangular matrix for an [nxm] matrix where n>m? That will give you (nxn-n)/2 elements and lose all the data where m⊖n.

What you probably want is the lower triangular matrix:

def tri_reduce(m):
    n=m.shape
    if n[0]>n[1]:
        i=np.tril_indices(n[0],1,n[1])
    else:
        i=np.triu_indices(n[0],1,n[1])
    return np.vstack((i,m[i])).T

Rebuilding it into a list of tuples would require a loop though I believe. list(tri_reduce(m)) would give a list of nd arrays.