problem with hierarchical clustering in Python

Posted by user248237 on Stack Overflow See other posts from Stack Overflow or by user248237
Published on 2010-05-30T22:52:41Z Indexed on 2010/05/30 23:02 UTC
Read the original article Hit count: 244

Filed under:

python

|

numpy

|

machine-learning

|

scipy

|

hcluster

I am doing a hierarchical clustering a 2 dimensional matrix by correlation distance metric (i.e. 1 - Pearson correlation). My code is the following (the data is in a variable called "data"):

from hcluster import *

Y = pdist(data, 'correlation')
cluster_type = 'average'
Z = linkage(Y, cluster_type)
dendrogram(Z)

The error I get is:

ValueError: Linkage 'Z' contains negative distances.

What causes this error? The matrix "data" that I use is simply:

[[  156.651968  2345.168618]
 [  158.089968  2032.840106]
 [  207.996413  2786.779081]
 [  151.885804  2286.70533 ]
 [  154.33665   1967.74431 ]
 [  150.060182  1931.991169]
 [  133.800787  1978.539644]
 [  112.743217  1478.903191]
 [  125.388905  1422.3247  ]]

I don't see how pdist could ever produce negative numbers when taking 1 - pearson correlation. Any ideas on this?

thank you.

© Stack Overflow or respective owner

Related posts about python

unmet dependencies in Ubuntu 12.04

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
I tried today to install a dvb-card on my Ubuntu 12.04 (Linux blauhai-linux 3.2.0-25-generic #40-Ubuntu SMP Wed May 23 20:30:51 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux ). The installation failed with an error. After that, i tried to install python (it was already installed but i got this error): linux:~$… >>> More
How can I get sikuli-ide to work?

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
I installed sikuli-ide with sudo apt-get install sikuli-ide Everything was fine until I tried to start it from the terminal. I typed sikuli-ide But the only response I got was [info] locale: en_US The application was not started, furthermore there is no desktop file and sikuli-ide does not… >>> More
Getting PATH right for python after MacPorts install

as seen on Super User - Search for 'Super User'
I can't import some python libraries (PIL, psycopg2) that I just installed with MacPorts. I looked through these forums, and tried to adjust my PATH variable in $HOME/.bash_profile in order to fix this but it did not work. I added the location of PIL and psycopg2 to PATH. I know that Terminal is… >>> More
call python with system() in R to run a python script emulating the python console

as seen on Stack Overflow - Search for 'Stack Overflow'
I want to pass a chunk of Python code to Python in R with something like system('python ...'), and I'm wondering if there is an easy way to emulate the python console in this case. For example, suppose the code is "print 'hello world'", how can I get the output like this in R? >>> print… >>> More
Python - Calling a non python program from python?

as seen on Stack Overflow - Search for 'Stack Overflow'
Hi, I am currently struggling to call a non python program from a python script. I have a ~1000 files that when passed through this C++ program will generate ~1000 outputs. Each output file must have a distinct name. The command I wish to run is of the form: program_name -input -output -o1 -o2… >>> More

Related posts about numpy

append a numpy array to a numpy array

as seen on Stack Overflow - Search for 'Stack Overflow'
I just started programming in python and am very new to numpy packages.. so still trying to get a hang of it. I have a an numpy_array so something like [ a b c] And then I want to append it into anotehr numpyarray (Just like we create a list of lists) How do we create an array of numpy arrays containing… >>> More
How do you construct an array suitable for numpy sorting?

as seen on Stack Overflow - Search for 'Stack Overflow'
I need to sort two arrays simultaneously, or rather I need to sort one of the arrays and bring the corresponding element of its associated array with it as I sort. That is if the array is [(5, 33), (4, 44), (3, 55)] and I sort by the first axis (labeled below dtype='alpha') then I want: [(3.0, 55… >>> More
Building an interleaved buffer for pyopengl and numpy

as seen on Stack Overflow - Search for 'Stack Overflow'
I'm trying to batch up a bunch of vertices and texture coords in an interleaved array before sending it to pyOpengl's glInterleavedArrays/glDrawArrays. The only problem is that I'm unable to find a suitably fast enough way to append data into a numpy array. Is there a better way to do this? I… >>> More
Helping install mrcwa and solve problems with f2py in Ubuntu 14.04 LTS

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
I am sorry if this is the wrong section but I am starting to get desperate, please someone help me... I need to install the program mrcwa-20080820 (sourceforge.net/projects/mrcwa/) because a summer project that I am involved. I need to use it together with anaconda (store.continuum.io/cshop/anaconda/)… >>> More
Compound assignment operators in Python's Numpy library

as seen on Programmers - Search for 'Programmers'
The "vectorizing" of fancy indexing by Python's numpy library sometimes gives unexpected results. For example: import numpy a = numpy.zeros((1000,4), dtype='uint32') b = numpy.zeros((1000,4), dtype='uint32') i = numpy.random.random_integers(0,999,1000) j = numpy.random.random_integers(0,3,1000) a[i… >>> More