gmane.comp.python.numeric.general

Re: .max(0) on reshaped array returnsinconsistent results.

Pauli Virtanen — 2012-05-25T17:30:46

25.05.2012 13:46, Thouis (Ray) Jones kirjoitti:

Try using "git bisect" to find the the first failing commit:

1. git bisect start master v1.6.2

2. Rebuild, and run test

3. Pick one of the following according to the result:

   git bisect good   # test OK
   git bisect bad    # test fails with this error
   git bisect skip   # doesn't build, or some other error

4. Goto 2, until Git tells you what's the first bad commit.

Abort with "git bisect reset".

If you are on unix, the following rig can help automate this:
https://github.com/pv/scipy-build-makefile

Re: Checking for views (was: Should arr.diagonal() return a copy or aview?)

Nathaniel Smith — 2012-05-25T16:59:39

Right.

I suspect it's still pretty practical to solve for many of the arrays
we care about (strides that are multiples of each other, etc.); many
NP-hard problems are easy in the typical case, and for a lot of the
cases we care about (e.g. disjoint slices of a contiguous array)
solving the Diophantine equation will show that the bounds are
irrelevant and collisions can never occur.

Oh well, fortunately nothing depends on this :-).

- N

Re: Checking for views (was: Should arr.diagonal() return a copy or aview?)

Robert Kern — 2012-05-25T15:59:52

If both positive and negative values are allowed, then there is a
polynomial-time algorithm to solve the linear Diophantine equation,
but bounding the possible values renders it NP-complete. When you go
down to {0,1} as the only allowable values, it becomes the SUBSET-SUM
problem.

Re: Checking for views (was: Should arr.diagonal() return a copy or aview?)

Robert Kern — 2012-05-25T15:55:12

I believe that's what this is telling me:

  http://permalink.gmane.org/gmane.comp.gcc.fortran/11797

Re: .max(0) on reshaped array returnsinconsistent results.

Thouis (Ray) Jones — 2012-05-25T15:39:09

Sorry, Nathaniel's branch.  My fingers went into autopilot.

Ray
_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion< at >scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: .max(0) on reshaped array returnsinconsistent results.

Thouis (Ray) Jones — 2012-05-25T15:30:22

I've bisected it down to this commit:
https://github.com/numpy/numpy/commit/aed9925a9d5fe9a407d0ca2c65cb577116c4d0f1

This exercises it consistently for me:
while True; do python -m nose.core
../numpy.bisect/numpy/lib/tests/test_function_base.py:TestHistogramdd
--pdb --pdb-failures; done

It happens at HEAD in Nathan's separate-maskna branch, as well.

Ray Jones

Re: indexes in an array where value is greater than 1?

Benjamin Root — 2012-05-25T15:21:08

np.nonzero(arrrgh > 1)

Note, it returns a list of lists, one for each dimension of the input array.

Cheers!
Ben Root
_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion< at >scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

indexes in an array where value is greater than1?

Chris Withers — 2012-05-25T15:17:27

Hi All,

I have an array:

arrrgh = numpy.zeros(100000000)

A sparse collection of elements will have values greater than zero:

arrrgh[9999] = 2
arrrgh[3453453] =42

The *wrong* way to do this is:

for i in xrange(len(arrrgh)):
     if arrrgh[i] > 1:
         print i

What's the right way?

Chris

Re: Checking for views (was: Should arr.diagonal() return a copy or aview?)

Nathaniel Smith — 2012-05-25T14:55:01

Huh, is it really? I'm pretty sure checking the existence of a
solution to a linear Diophantine equation is cheap, but I guess
figuring out whether it falls within the "shape" bounds is less
obvious...

Retrieving and flattening lower triangular components of inner axes of 3d array

Damien Moore — 2012-05-25T14:19:34

Hi List,

I have an array, x, with dimensions (K,M,M). I would like to compute a
new array, y ~ (K,M*(M+1)/2), such that

y[k] = numpy.tril(x[k]).flatten() for each k = 0,..,K-1

Is there a way to do this without looping? I was trying with
tril_indices but couldn't quite get the logic right.

Thanks,
dm

Re: Checking for views

Dag Sverre Seljebotn — 2012-05-25T14:07:48

Well, I guess N would be the number of dimensions, so that by itself 
doesn't tell us all that much. Question is if the worst case is no 
better than the trivial O(number of elements in the matrices), which 
would be bad.

Dag

Re: Checking for views (was: Should arr.diagonal() return a copy or aview?)

Robert Kern — 2012-05-25T13:17:13


Except that it's still NP-complete.

Re: .max(0) on reshaped array returnsinconsistent results.

Thouis Jones — 2012-05-25T12:17:42

By the way, the strange phrasing comes from these lines in histogramdd():
            smin = atleast_1d(array(sample.min(0), float))
            smax = atleast_1d(array(sample.max(0), float))

Which is where I encountered the bug in the numpy tests.

Ray Jones

Re: .max(0) on reshaped array returnsinconsistent results.

Thouis Jones — 2012-05-25T12:07:46


AssertionError:
Arrays are not equal
(mismatch 100.0%)
 x: array([ 4.,  5.])
 y: array([  4.31441533e+09,   4.31441402e+09])

I don't seem to be able to reproduce with just a.max(0) or
np.array(a.max(0), np.float), but since it seems to be very unstable
to other changes in the code, I'll keep trying to find out if I can
make those simpler versions crash.

Ray Jones
_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion< at >scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: .max(0) on reshaped array returnsinconsistent results.

Nathaniel Smith — 2012-05-25T11:52:07

What do you get, if not the expected value? And are the calls to
atleast_1d, np.array, etc., necessary to trigger the problem, or will
just plain a.max(0) do it?

.max(0) on reshaped array returns inconsistentresults.

Thouis (Ray) Jones — 2012-05-25T11:46:31

I'm seeing some strange behavior from .max() on a reshaped array in
the current master, and wanted to raise it here to make sure it's not
something uniquely broken in my setup.

This code fails for me, though changing the context (adding a counter
to the loop, or running under "python -i") sometimes prevents it from
failing.  This code doesn't fail under 1.6.2.

---------------
import numpy as np

b = np.array([0, 1, 2, 3, 4, 5], np.int64)
a = b.reshape(3, 2)

while True:
    np.testing.assert_array_equal(np.atleast_1d(np.array(a.max(0), np.float)),
                                  np.atleast_1d(np.array(a.max(0), np.float)))
---------------

I spent several hours with valgrind trying to track down what was
causing this, but had no luck.  Perhaps someone with more knowledge of
the numpy ufunc internals can track it down faster than me.

I went ahead and filed a bug for it:
http://projects.scipy.org/numpy/ticket/2144

Re: Fwd: Named dtype array: Difference betweena[0]['name'] and a['name'][0]?

diehose — 2012-05-24T21:07:50

thanks a lot. I updated the question on stackoverflow and opened a ticket
 
http://projects.scipy.org/numpy/ticket/2139
 
björn
 

Am Montag, 21. Mai 2012, 15:37:36 schrieb Travis Oliphant:

This is the right place to ask, it's just that it can take time to get an 
answer because people who might know the answer may not have the time to 
respond immediately.  


The short answer is that this is not really a "normal" bug, but it could be 
considered a "design" bug (although the issues may not be straightforward to 
resolve).    What that means is that it may not be changed in the short term 
--- and you should just use the first spelling. 



Structured arrays can be a confusing area of NumPy for several of reasons.   
You've constructed an example that touches on several of them.   You have a 
data-type that is a "structure" array with one member ("tuple").  That member 
contains a 2-vector of integers.  


First of all, it is important to remember that with Python, doing a['tuple']
[0] = (1,2) is equivalen

Re: Checking for views (was: Should arr.diagonal() return a copy or aview?)

2012-05-24T19:20:12

as example:

I checked pandas recently and IIRC, I needed three .base to get a True

<class 'pandas.core.series.Series'>
False
False
True
True

Josef

Re: Checking for views (was: Should arr.diagonal() return a copy or aview?)

Nathaniel Smith — 2012-05-24T17:59:38

If using the current development version of numpy, that answer is
actually wrong... if you do
  a = np.arange(10)
  b = a.view()
  c = b.view()
then in the development version, c.base is a, not b. This is the
source of some contention and confusion right now...:
  https://github.com/numpy/numpy/pull/280#issuecomment-5888154

In any case, if "b.base is a" is True, then you can be pretty certain
that b and a share memory, but if it is False, it doesn't tell you
much at all. AFAICT np.may_share_memory would be strictly more useful.

Re: Checking for views (was: Should arr.diagonal()return a copy or aview?)

Larsen, Brian A — 2012-05-24T17:07:37

This is the stack overflow discussion mentioned.

http://stackoverflow.com/questions/9164269/can-you-tell-if-an-array-is-a-view-of-another

I basically implemented the answer from SO.  I feel like the "is" gives you a good handle on things since to be true they are actually the same location in memory.

Brian






On May 24, 2012, at 8:56 AM, Jonathan T. Niehof wrote:

On 05/23/2012 05:31 PM, T J wrote:

It seems that there are a number of ways to check if an array is a view.
Do we have a preferred way in the API that is guaranteed to stay
available? Or are all of the various methods "here to stay"?

We've settled on checking array.base, which I think was the outcome of a
stackoverflow thread that I can't dig up. (I'll check with the guy who
wrote the code.)

--
Jonathan Niehof
ISR-3 Space Data Systems
Los Alamos National Laboratory
MS-D466
Los Alamos, NM 87545

Phone: 505-667-9595
email: jniehof< at >lanl.gov<mailto:jniehof< at >lanl.gov>

Correspondence /
Technical data or Software Publicly Available
______________

Re: Checking for views (was: Should arr.diagonal() return a copy or aview?)

Robert Kern — 2012-05-24T16:52:09

numpy.may_share_memory() gets closer, but it can be defeated by
certain striding patterns. At least, it is conservative and reports
false positives but not false negatives. Implementing
numpy.does_share_memory() correctly involves some number theory and
hairy edge cases.

(Hmm, now that I think about it, the edge cases are when the strides
are 0 or negative. 0-stride axes can simply be removed, and I think we
should be able to work back to a first item and flip the sign on the
negative strides. The typical positive-stride solution can be found in
an open source C++ global array code, IIRC. Double-hmmm...)