Wednesday, January 30, 2008

scikits.ann in PyPI

After several folks had trouble getting the scikits.ann egg from the server listed in previous posts, I've uploaded the source and OS X 10.5 egg to PyPI. You can now install it via easy_install. Of course, you can still get it via the scikits SVN.

20 comments:

Patrick said...

Hi,

I have some troubles installing the wrapper for ANN ...

The python setup.py tests during the installation does not work and if I install it, when I run the k.knn() of the example, I get this error :

pkg_resources.DistributionNotFound: configobj

Any hint ?

Thx

Patrick

Barry said...

patrick,

EasyInstall should have downloaded and installed configobj for you when you installed scikits.ann. You can try running "easy_install configobj" and re-trying scikits.ann.

I'll look into why "python setup.py test" fails. Thanks for the report.

Patrick said...

Here is the error I get running the example after the installation through easy_install :

>>> import scikits.ann as ann
>>> import numpy as np
>>> k=ann.kdtree(np.array([[0.,0],[1,0],[1.5,2]]))
>>> k.knn([0,.2],1)
Traceback (most recent call last):
File "stdin", line 1, in module
File "/usr/lib64/python2.5/site-packages/PIL/__init__.py", line 113, in knn

File "build/bdist.linux-x86_64/egg/scikits/ann/ANN.py", line 62, in _knn2
NotImplementedError: Wrong number of arguments for overloaded function '_kdtree__knn2'.
Possible C/C++ prototypes are:
_knn2(_kdtree const *,double *,int,int,int,int,int *,int,int,double *,double)
_knn2(_kdtree const *,double *,int,int,int,int,int *,int,int,double *)

>>>

Barry said...

It looks like you're on Linux (64-bit), so EasyInstall compiled scikits.ann from source for you. I don't have access to a Linux box for testing and I haven't gotten any user reports yet either so we may have to do a bit of debugging. Can you tell me what version of swig is installed on your system ('swig -version')?

Patrick said...

Yes quad core 64 bits ...

SWIG Version 1.3.33 on one machine
SWIG Version 1.3.25 on the other

(same errors)

Barry said...

Patrick,

It's possible that different SWIG versions are the problem, but I may be barking up the wrong tree. Could you email me the output of

python setup.py build

(barrywark at gmail period com)?

eli said...

Hi,

The ANN wrapper has worked well for my programming, but a question has come up. I'm generating images that plot surface densities using ANN for determining the nearest neighbors.

Now, let's assume we have a 100x100 grid to iterate over with 50 random points in the grid that are being considered for NN values. I want to iterate through each pixel in the grid and compute the density in relation to the 50 points in the grid.

If I do this using a for loop it works fine, but the algorithm is somewhat sluggish due to the Python loop. There's a function in Scipy called fromfuction that should compute the same results as the for loop. I have tried this but fromfunction does not want to call up the k.knn([a,b], N_value) when I do it. If you would like to see an example of code I can post it if needed.

Thanks,

Eli

Barry said...

eli,

I'm not sure exactly what you're asking. If you want to build a single kd-tree and then query many points for the nn, you can do this directly (see the docstring for ann.kdtree.knn) . It is implemented as a loop (obviously), but the loop is handled in compiled code so it will be faster than running the loop in Python.

I'm not sure what you mean that numpy.fromfunction won't call the knn method on a kdtree instance. Perhaps you could post some code...

eli said...

Hi Barry,

Here's a snippet of what I have programmed at this point. The code is generating a nearest neighbor density map in a matrix. The neighbors are selected points within the grid. I would like to replace the for loops with the fromfunction in Scipy. I will give an example in the next posting.

aa = zeros([xl,yl])
# arrpix is a vector of pixel coordinates.
# Example: arrpix[0,0] = [20,180]
k = ann.kdtree(arrpix)

# Starting first for loop to cycle through each 'pixel' in aa
for i in range(len(aa[:,0])):
for j in range(len(aa[0,:])):
aa[i,j]= (nth-1)/(pi*k.knn([i,j],nth)[1][0][-1]) # density per i,j pixel

# Correction loop for density aberrations where nearest function neighbor considers itelf
# as the first neighbor.
for n in range(len(arrpix)):
aa[arrpix[n][0],arrpix[n][1]] = (nth-1)/(pi*k.knn([arrpix[n][0],arrpix[n][1]],nth-1)[1][0][-1])
return aa

eli said...

Here's the fromfunction implementation.

# arrpix is a vector of pixel coordinates.
# Example: arrpix[0] = [20,180]
k = ann.kdtree(arrpix)

# Starting first for loop to cycle through each 'pixel' in aa
A = fromfunction(lambda i,j: (nth-1)/(pi*k.knn([i,j],nth)[1][0][-1], (1200,1200))

# Correction loop for density aberrations where nearest function neighbor considers itelf
# as the first neighbor.
for n in range(len(arrpix)):
aa[arrpix[n][0],arrpix[n][1]] = (nth-1)/(pi*k.knn([arrpix[n][0],arrpix[n][1]],nth-1)[1][0][-1])
return aa

eli said...

Also, in the fromfunction implementation there are some slight errors. I have not run the example script given here, so ignore the small typos.

Thanks

Barry said...

eli,

thanks for the code example. i'll try to take a look at what's going on in the next day or two...

Barry said...

eli,

you should take an other look at the fromfunction doc string... there's no way to pass other positional arguments (i.e. "nth" in your code) to the function. You'll have to create a wrapper ala:
t = scikits.ann.kdtree(...)
nth =3
def knn_wrapper(i,j):
return t.knn([i,j], nth)[1][0][-1]

Rob said...

patrick,

I had the same problem. After A LOT of searching I finally found the solution. I think that it has to do with SWIG wrapping on 64-bit machines. Anyway, in the __init__.py file, change the line:
"idx = np.empty((pts.shape[0], k), dtype= np.int_)"

to be :

"idx = np.empty((pts.shape[0], k), dtype= 'i')"

After doing this, all of the tests ran correctly for me.

--Rob

Barry said...

Three cheers for Rob! I will make the change in SVN.

Rob, if you contact me by email, I will send you your choice something swig-able.

-bary

pmos said...

Hello,
Currently the ann wrapper does not install either from source or through easy_install. I would be happy to post the errors but there seems to be a compile time error for ANN_wrap.o something about a bad symbol value. I am sure the libraries are there and compiled (I can call them from c++).
Anyone have any ideas?

thanks

C said...

With regards to the __init__.py patch:

"idx = np.empty((pts.shape[0], k), dtype= np.int_)"

to be :

"idx = np.empty((pts.shape[0], k), dtype= 'i')"

Could the current maintainer update the package? The version I just got through macports needs that patch to run.

Brian said...

With regards to the __init__.py patch:

"idx = np.empty((pts.shape[0], k), dtype= np.int_)"

to be :

"idx = np.empty((pts.shape[0], k), dtype= 'i')"

Firstly, three cheers to Rob for finding the bug, and secondly, its still the same in the svn (2 and a half years later!)

Mads said...
This comment has been removed by the author.
Mads said...

Hi,

I am running Python 2.6.6 on OS 10.5.8
When typing:
"sudo python setup.py install"
in my Terminal I get a bunch of c++: command not found errors.

I already ran "easy_install configobj".

What am I missing here?

Thanks

Mads