gmane.comp.python.scientific.devel

Scipy Docs permissions

Karmel Allison — 2012-05-26T04:03:55

Hi all,

After many years of my living off the work you do, I thought I'd start helping out. Can I get edit permissions for the docs to start? I just registered with the username karmel.

Thanks!
Karmel

Adding t-test with unequal variances to stats.py

Junkshops — 2012-05-23T07:38:34

Hi all,

I've issued a pull request (http://github.com/scipy/scipy/pull/227) for 
a version of scipy/stats/stats.py with the following changes:

1) Adds a method for running a t-test with unequal or unknown population 
variances. ttest_ind assumes that population variances are equal.
2) Refactored common code in the 4 t-test methods into shared methods.
3) This section of code, which has variations in multiple methods, looks 
buggy to me:

d = np.mean(a,axis) - np.mean(b,axis)
svar = ((n1-1)*v1+(n2-1)*v2) / float(df)

t = d/np.sqrt(svar*(1.0/n1 + 1.0/n2))
t = np.where((d==0)*(svar==0), 1.0, t) #define t=0/0 = 0, identical means

Surely if d=0, regardless of svar, t should be set to 0, not 1. 
Similarly, if svar = 0 then both variances are zero (assuming that each 
data set has at least 2 points - perhaps there should be a check for 
this?). In that case, if d==0 t should be zero. Otherwise, t should be 
+/-inf. Hence, (svar==0) is redundant.

Accordingly, I've changed the lines in all functions to be the equivalent of

t = np.where((d==0), 0.0, t)

This handles the case where both d and svar are 0. The respective tests 
have also been changed.

If I'm missing something here, please let me know.

Thanks, Gavin

scipy.test() runs 0 tests

Junkshops — 2012-05-22T15:39:02

Hi all,

Hopefully I'm not doing something too stupid here, but I've googled 
around quite a bit and can't seem to find any tips. Perhaps my google-fu 
is not strong enough. Anyway, I built scipy 0.11.0 in place, seemingly 
successfully, in ~/git/scipy. Ubuntu version is 12.04. I haven't added 
scipy 0.11 to the pypath yet so I'm running in the parent dir.

js< at >ubuntuVB12:~/git/scipy$ sudo aptitude install python-nose
<snip>
Setting up python-nose (1.1.2-3) ...

js< at >ubuntuVB12:~/git/scipy$ python
Python 2.7.3 (default, Apr 20 2012, 22:39:59)
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
 >>> import scipy
 >>> scipy.test()
Running unit tests for scipy
NumPy version 1.6.1
NumPy is installed in /usr/lib/python2.7/dist-packages/numpy
SciPy version 0.11.0.dev-3852ce2
SciPy is installed in scipy
Python version 2.7.3 (default, Apr 20 2012, 22:39:59) [GCC 4.6.3]
nose version 1.1.2

----------------------------------------------------------------------
Ran 0 tests in 0.161s

OK
<nose.result.TextTestResult run=0 errors=0 failures=0>
 >>> scipy.test("full")
<snip>

----------------------------------------------------------------------
Ran 0 tests in 0.139s

OK
<nose.result.TextTestResult run=0 errors=0 failures=0>

Anyone have any idea what I'm doing wrong?

Thanks very much, Gavin

Code question about signal.ltisys.py

Warren Weckesser — 2012-05-22T00:33:06

While taking a look at https://github.com/scipy/scipy/pull/225, which makes
a small changes to signal/ltisys.py,I noticed the surrounding code, which
has lines like this:

            self.__dict__['num'], self.__dict__['den'] = normalize(*args)


My inclination is to rewrite that as

            self.num, self.den = normalize(*args)


But am I missing something?


Warren
_______________________________________________
SciPy-Dev mailing list
SciPy-Dev< at >scipy.org
http://mail.scipy.org/mailman/listinfo/scipy-dev

warnings in scipy.stats.entropy

Skipper Seabold — 2012-05-21T22:39:43

Currently in scipy.stats.entropy if you are not ignoring them you will
see warnings when the function is given a probability of zero even
though the case of zero is specifically handled in the function.
Rightly or wrongly this makes me cringe. What do people think about
fixing this by using seterr explicitly in the function or masking the
zeros. Eg.,

import numpy as np
from scipy.stats import entropy

prob = np.random.uniform(0,20, size=10)
prob[5] = 0
prob = prob/prob.sum()

np.seterr(all = 'warn')
entropy(prob) # too loud

Instead we could do (within entropy)

oldstate = np.geterr()
np.seterr(divide='ignore', invalid='ignore')
entropy(prob)
np.seterr(**oldstate)

or just mask the zeros in the first place if this is too much

idx = prob > 0
-np.sum(prob[idx] * np.log(prob[idx]))

Thoughts?

Skipper

Faster Hough/Radon transform (was sum_angle()sum_polar() functions)

Charles R Harris — 2012-05-21T21:25:23

Thought I'd advertize this under a different subject heading. The sum_angle
function is really the inner loop but the timings are about 10x faster than
the Radon transform in skimage, so some here might be interested.

Chuck
_______________________________________________
SciPy-Dev mailing list
SciPy-Dev< at >scipy.org
http://mail.scipy.org/mailman/listinfo/scipy-dev

scipy.signal.normalize problems?

Josh Lawrence — 2012-05-21T18:45:55

Hey all,

I've been having some problems designing a Chebyshev filter and I think I have narrowed down the hang-up to scipy.signal.normalize. I think what's going on in my case is that line 286 of filter_design.py (the first allclose call in the normalize function) is producing a false positive. Here's the function definition:

def normalize(b, a):
    """Normalize polynomial representation of a transfer function.

    If values of b are too close to 0, they are removed. In that case, a
    BadCoefficients warning is emitted.
    """
    b, a = map(atleast_1d, (b, a))
    if len(a.shape) != 1:
        raise ValueError("Denominator polynomial must be rank-1 array.")
    if len(b.shape) > 2:
        raise ValueError("Numerator polynomial must be rank-1 or"
                         " rank-2 array.")
    if len(b.shape) == 1:
        b = asarray([b], b.dtype.char)
    while a[0] == 0.0 and len(a) > 1:
        a = a[1:]
    outb = b * (1.0) / a[0]
    outa = a * (1.0) / a[0]
    if allclose(outb[:, 0], 0, rtol=1e-14): <------------------ Line 286
        warnings.warn("Badly conditioned filter coefficients (numerator): the "
                      "results may be meaningless", BadCoefficients)
        while allclose(outb[:, 0], 0, rtol=1e-14) and (outb.shape[-1] > 1):
            outb = outb[:, 1:]
    if outb.shape[0] == 1:
        outb = outb[0]
    return outb, outa

I marked line 286. If I reproduce all the steps carried out by scipy.signal.iirdesign, I end up with a (b, a) pair which results of scipy.signal.lp2lp and looks like this:

In [106]: b_lp2
Out[106]: array([  1.55431359e-06+0.j])

In [107]: a_lp2
Out[107]: 
array([  1.00000000e+00 +0.00000000e+00j,
         3.46306104e-01 -2.01282794e-16j,
         2.42572185e-01 -6.08207573e-17j,
         5.92946943e-02 +0.00000000e+00j,
         1.82069156e-02 +5.55318531e-18j,
         2.89328123e-03 +0.00000000e+00j,
         4.36566281e-04 -2.95766719e-19j,
         3.50842810e-05 -3.19180568e-20j,   1.64641246e-06 -1.00966301e-21j])

scipy.signal.iirdesign takes b_lp2, a_lp2 (my local variable names to keep track of what's going on) and runs them through scipy.signal.bilinear (in filter_design.py bilinear is called on line 624 within iirfilter. iirdesign calls iirfilter which calls bilinear). Inside bilinear, normalize is called on line 445. I've made my own class with bilinear copied and pasted from filter_design.py to test things. In bilinear, the input to normalize is given by

b = [  1.55431359e-06   1.24345087e-05   4.35207804e-05   8.70415608e-05
   1.08801951e-04   8.70415608e-05   4.35207804e-05   1.24345087e-05
   1.55431359e-06]
a = [   72269.02590913  -562426.61430468  1918276.19173089 -3745112.83646825
  4577612.13937628 -3586970.61385926  1759651.18184723  -494097.93515708
    60799.46134722]

In normalize, right before the allclose() call, outb is defined by

outb = [[  2.15073272e-11   1.72058618e-10   6.02205162e-10   1.20441032e-09
    1.50551290e-09   1.20441032e-09   6.02205162e-10   1.72058618e-10
    2.15073272e-11]]


b =

   1.0e-08 *

  Columns 1 through 5

   0.002150733144728   0.017205865157826   0.060220528052392   0.120441056104784   0.150551320130980

  Columns 6 through 9

   0.120441056104784   0.060220528052392   0.017205865157826   0.002150733144728

which matches up rather well for several significant figures.

I apologize if this is not clearly explained, but I'm not sure what to do. I tried messing around with the arguments to allclose (switching it to be allclose(0, outb[:,0], ...), or changing the keyword from rtol to atol). I am also not sure why normalize is setup to run on rank-2 arrays. I looked through filter_design.py and none of the functions contained in it send a rank-2 array to normalize from what I can tell. Any thoughts?

Cheers,

Josh Lawrence

sum_angle() and sum_polar() functions

Robert Jördens — 2012-05-21T04:50:11

Hello,
I submitted two functions to numpy that sum 2d matrices along angled
cartesian or polar coordinates.
https://github.com/numpy/numpy/pull/230
The two functions certainly have their main application in image
processing and might be better suited for scipy of scikits-image.

sum_angle() is not much more than the old scipy.misc.pilutils.radon()
transform. But the later is deprecated and has several problems
(floats, non-conserved sum(), interpolation, speed) as discussed in
the pull request.

The new scikits-image.transform.radon() appears to be more generic but
a bit complicated and potentially even slower than the
imrotate()-based version in scipy.

Could sum_angle() and sum_polar() find a place in scipy or
scikits-image or are they simple enough to be useful for numpy?

New contribution: bode() function for LTI systems

Bjørn Forsman — 2012-05-20T20:20:36

Hi all,

I'm new on this list. I have been wanting to have a bode() function
for LTI systems in Python for some time now. Today I got tired of
waiting and wrote one myself :-) I have sent pull request on github:

https://github.com/scipy/scipy/pull/224 - ENH: ltisys: new bode() function

While at it I also fixed a small bug in the lti class init function:

https://github.com/scipy/scipy/pull/223 - ENH: ltisys: make lti zpk
initialization work with plain lists

Is this OK? Should I post the patches here too?

Best regards,
Bjørn Forsman
_______________________________________________
SciPy-Dev mailing list
SciPy-Dev< at >scipy.org
http://mail.scipy.org/mailman/listinfo/scipy-dev

scikit for virtual instruments?

alex arsenovic — 2012-05-20T13:53:45

i was considering creating a scikit to provide a library of virtual 
instruments.

scikit-vi? import skvi?

my needs are solely for  VISA/GPIB vi's  using pyvisa, but the project 
could support other interfaces as well. perhaps it would provide  
instances of VI's for specific applications (Voltmeter, Ohmeter, etc) as 
well as general instruments (Keithely2000).

although i dont have a large library of VI's, or plan to write them, i 
have seen the writing of VI's is  being re-done constantly. i think a 
centralized effort would save a lot of wasted time, as well as provide 
more robust code.

the only existing solution to this that i am aware of is pythics, which 
i have used and i think its great, but it has a larger-than-necessary 
scope. a simple project, just for python VI's on Github would be the 
easiest for people to contribute to (in my opinion) and ease of 
contribution is necessary for a  project such as this.


any thoughts?

alex

minimize(): being strict on options

Pauli Virtanen — 2012-05-19T09:08:45

Hi,

Currently, the minimize() solvers silently accept unknown (= mistyped or
inapplicable) options. It might be useful to change this so that it
raises an error if unknown options are passed to a solver.

Alternatively, it could raise a warning instead --- when trying out
different solvers an error could be a PITA.

Code changes:

    tol = options.get('ftol', 1.48e-8)
    maxiter = options.get('maxiter', 500)

change to

    tol = options.pop('ftol', 1.48e-8)
    maxiter = options.pop('maxiter', 500)
    if options:
        warnings.warn("Unknown solver options: %r"
                        % sorted(options.keys()),
                      scipy.optimize.OptimizationWarning)

and

    class OptimizationWarning(UserWarning):
        pass

minimize(): tolerance unification

Pauli Virtanen — 2012-05-19T08:57:16

Hi,

One thing which would be useful to bring to conclusion in 0.11.0 in the
new minimize() interface: tolerance specification.

First, it would be useful if there was a common `tol` keyword in
`minimize()` which would just set whatever appropriate tolerance
parameters, so that the user wouldn't need to bother about looking into
the specific solver documentation.

Second, the main bulk of the work is actually mostly finished, but some
of the remaining different optimizers still take the termination
parameters in a non-standard way:

- fmin_cobyla: rhoend

  The `rhoend` parameter seems essentially equivalent to absolute
  x-tolerance, and could be renamed to `xtol` in minimize().

- fmin_l_bfgs_b: factr, pgtol

  The `factr` parameter specifies the absolute and relative
  f-tolerances as follows: rftol = eps * factr,
  aftol = eps * factr.

  In my opinion, the minimize() interface should expose this
  as the `ftol` parameter, and compute
  `factr = ftol/np.finfo(float).eps`.

  The `pgtol` parameter is a criterion for the projected gradient.
  Could we rename `pgtol` -> `gtol`, as some other routines use
  `gtol` to specify conditions for the gradient. The documentation
  could still describe that it's actually for the projected gradient,
  but I expect most users just don't care.

- brent: ftol

  The `ftol` is actually the relative x-tolerance, and is currently
  erroneously named in the minimize() interface!

- golden: ftol

  The `ftol` is actually the relative x-tolerance, and is currently
  erroneously named in the minimize() interface!

- fmin_tnc: pgtol

  Ditto, pgtol -> gtol?

github: disappearing comments - clarification?

2012-05-17T20:30:49

A question for the github experts:

Under what scenarios do comments on github pull requests disappear?

I've never seen a clear answer just my speculation. (based on: Did it
disappear or is my search fu insufficient?)

For example, some comments disappear after a rebased force push. all
comments or just inline code comments?
If a pull request never gets merged and the originating branch is
deleted, do the comments also disappear?

any other cases of disappearing comments?

Thanks,

Josef

SciPy Docs Edit Privileges

Josh Lawrence — 2012-05-16T17:51:11

Hello,

I found a typo in one of the doc strings in scipy and figured I should probably start contributing where I can. Could you please give me edit permissions? My username is wa03.

Thanks,

Josh

relative imports

Denis Laxalde — 2012-05-16T13:47:16

Hi,

What do people think about using relative imports [1] within scipy
modules? One advantage I can see is to allow packages to be built and
tested independently. Any drawbacks?

[1]: http://docs.python.org/whatsnew/2.5.html#pep-328-absolute-and-relative-imports

Question about stats.mstats.kendalltau_seasonal

Andreas H. — 2012-05-15T09:45:40

Hi,

looking at stats.mstats.kendalltau_seasonal, i noticed the lack of
proper documentation for that function. I'd like to write something up
about it, but wanted to verify that the original author was indeed
following the approach in

   Hirsch, Robert M., and James R. Slack. “A Nonparametric Trend Test
for Seasonal Data With Serial Dependence.” Water Resources Research 20,
no. 6 (1984): P. 727.
(http://www.agu.org/pubs/crossref/1984/WR020i006p00727.shtml)

Also, it would be nice to hear what exactly the different return values are.

If you guys tell me, I'll gladly write docstring + example.

Cheers,
Andreas.
_______________________________________________
SciPy-Dev mailing list
SciPy-Dev< at >scipy.org
http://mail.scipy.org/mailman/listinfo/scipy-dev

reading a module under test

nicky van foreest — 2012-05-14T20:01:01

Hi,

I would like to run a test on some code in my local branch of scipy.
Now the problem in one of the test files is that it says

from scipy import stats

Now this reads the standard stats module, not the one I want to test,
i.e, the one on my local branch. I changed the pythonpath, but this
does not help. Is there a generic way to say something like

from scipy import path_to_my_stats_under_test/stats    ?

Specifically, I want to load this file:
/home/nicky/prog/scipy/scipy/stats/distributions.py, and I don't want
to load /usr/lib/python2.7/dist-packages/scipy/stats/distributions.py.

Thanks for any help.

Nicky

Missing docstrings in the SciPy docstring editor

Tim Cera — 2012-05-14T14:58:06

Most of the docstrings are gone from the on-line editor...
http://docs.scipy.org/scipy/docs/

Kindest regards,
Tim
_______________________________________________
SciPy-Dev mailing list
SciPy-Dev< at >scipy.org
http://mail.scipy.org/mailman/listinfo/scipy-dev

ipython notebook for docs/examples

alex arsenovic — 2012-05-13T23:42:20

i recently took a look  at the ipython notebook feature. its  awesome. 
if it is interface-able with sphinx, i could write docs way faster. 
also, it seems to me that making interactive tutorials/examples with 
this accesable through a  server would work great.

has this idea already been considered? if not , does anyone have 
thoughts on this?


alex

dead link in cookbook

2012-05-13T12:08:34

there is a dead link in cookbook to the script
http://matplotlib.sf.net/examples/anim.py

http://www.scipy.org/Cookbook/Matplotlib/Animations

Does anyone know the correct link or is this obsolete?

Thanks,

Josef
clueless

Find points in delaunay triangulation : scipy.spatialvs. scipy.interpolation

Pablo Winant — 2012-05-13T00:13:15

Hi,

I tried to use interpolation routines in scipy recently and I have found 
two slight performance issues

  - The LinearNDInterpolation object implemented in cython requires a 
list of points and a list of values to be created. But is is not 
documented how to change the values of the interpolator without doing 
the mesh again. This is useful when one is solving the values of a 
function at the vertices of the mesh : one doesn't want to do the 
triangulation again and again. Maybe there could be a simple specific 
method to set the values in this case. In that case it would consist in 
changing the value of a property but it would be consistent with more 
general interpolation schemes.

- I tried to use the delaunay object from scipy and noticed a strange 
thing: for a given set of coordinates it takes longer to get the indices 
of the triangles containing the points than it takes to perform the 
interpolation using LinearND object. This is puzzling since apparently 
the implementation of LinearND performs many calls to the qhull library 
to get this indices. Attached is a simple exampe demonstrating this anomaly.

One last thing: I have written an interpolation object on sparse grids, 
using smolyak product of chebychev polynomials. It is written in pure 
python (vectorized) and licensed under the bsd license. Currently it 
lives in another library but I guess it would make more sense to have 
something like that in a more general scientific lib. Let me know if you 
are interested. (it is available there anyway: 
https://github.com/albop/dynare-python/tree/master/dolo/src/dolo/numeric: chebychev.py 
and smolyak.py)

Best regards,

Pablo