gmane.comp.python.scientific.devel

test coverage for each individual functions ?

2008-11-19T20:50:50

I would like to make a list of functions in scipy.stats that have no or low test coverage? Figleaf and coverage seem to work only on the module level. Does anyone know if this is easily possible or does anyone have a script for this? Thanks, Josef

LAPACK 3.2

Nils Wagner — 2008-11-19T19:24:07

FWIW Nils -------- Original Message -------- Date: Wed, 19 Nov 2008 06:03:40 -0800 From: James Demmel cs.berkeley.edu> To: Kreinovich, Vladik utep.edu>, reliable computing interval.louisiana.edu> CC: James Demmel cs.berkeley.edu> Subject: Announcement for Reliable Computing I would appreciate it if you could post this on your mailing list. Thanks, Jim Demmel Subject: New LAPACK release with "guaranteed" error bounds We have just released LAPACK 3.2, which among other improvements includes iterative refinement for solving linear systems with "guaranteed" error bounds, measured both normwise and componentwise. Portable high precision arithmetic (using another package we just released called XBLAS) is used to compute residuals. What we mean by "guarantee" is that either the error bound is correctly O(machine_epsilon), or a warning is returned that the condition number is ~1/machine_epsilon or larger. In extensive testing, we have never found it

Re: are masked array statistical function hiddenintentionally?

Pierre GM — 2008-11-19T18:32:29

Pauli, Thx a lot for the info. I should be good to go. I'm currently on the move, so won't be able to post anything on the server before a couple of days. Not that we're in a rush anyway... On Nov 19, 2008, at 11:28 AM, Pauli Virtanen wrote:

Re: are masked array statistical function hiddenintentionally?

Pauli Virtanen — 2008-11-19T16:28:08

[clip] I suggest making a new RST file "arrays.ma.rst" under numpy-docs/source. [clip] I don't know. Do :meth: and :attr: work in Sphinx with the bare method names without adding prefixes, if you haven't nested your method:: and attribute:: in the class:: directive. If yes, then they'll probably just work. Ah yes, it's currently at http://svn.scipy.org/svn/numpy/numpy-docs/trunk/ It's technically a part of Numpy's SVN repo, so that if you can commit to Numpy, you can commit to the docs. I believe that we'll move the docs under the doc/ dir in the main numpy trunk in the near future, so that they're easier to find and can be tagged etc. at the same time as the code. Yes, I believe taking a look at what's there now may help. (They're also linked to from docs.scipy.org in the sidebar.) Well, I think the implementation docs should also reside in numpy/doc, at least the functions should be manually grouped to smaller categories that make sense, so that it is easier to find the correct one if y

Re: are masked array statistical function hiddenintentionally?

Pierre GM — 2008-11-19T15:35:22

Pauli, no problem. I agree that there should be at least one specific page for numpy.ma functions/methods, organized by topics. Where should I create it (them) ? Mmh, my question was more about links to other functions/methods inside the docstring, using for eample :func:, :meth:, :attr: fields... Can you remind me where I can find numpy-docs ? It's not on the numpy SVN, right ? What's the address of the repository ? Do I have write access to it ? Agreed. I guess I'll find templates on the numpy-docs site, right ? Well, there are 2 different aspects: the actual implementation (functions docstring), and some kind of tutorial. This latter may find its place in numpy/docs, actually...

Re: are masked array statistical function hiddenintentionally?

Pauli Virtanen — 2008-11-19T15:08:14

Hi Pierre, Wed, 19 Nov 2008 08:55:19 -0500, Pierre GM wrote: [clip] This is actually my fault -- I left sorting out the functions under numpy submodules last when I added most other parts of numpy, and I haven't still finished numpy.ma. Also numpy.emath, numpy.rec, numpy.numarray, numpy.oldnumeric, numpy.ctypeslib, numpy.matlib would need work (but are less important than MA). The docs pick only those docstrings listed in an auto*:: directive in one of the *.rst files. Sphinx stuff will work in the docstrings, but also `numpy.foo` should IIRC generate reference links (this comes from the numpy Sphinx extension). I don't remember if Sphinx markup was discussed when the docstring format was agreed on, but I remember people being worried about making the docstrings more difficult to read on the terminal. If the markup doesn't compromise this, at least I don't see problems with using it. I think a useful way forward could be: 1. Editing numpy-docs/source/routines.ma.rst and adding any missing

Re: are masked array statistical function hiddenintentionally?

Pierre GM — 2008-11-19T14:09:56

Indeed. I tried to follow the basic stats organization. Sounds a cool idea, but I'd prefer to rename stats.mstats.py as stats.mstats_basic and stats.mmorestats as stats.mstats_extras, and call the namespace stats.mstats (I have a lot of code that use `import stats.mstats as mstats`, and I don't really want to spend time in a few monthstrying to figure why it doesn't work anymore... Could be a possibility, but then we need to make sure that the standard and masked versions of the functions have exactly the same syntax and return the same thing if a ndarray is used as input (I suspect it's not always the case...). If we're to create a new namespace, we could introduce an extra layer for the functions that have been checked, while still leaving the possibility to access the non-verified functions. What about something like that: * stats.mstats.__init__ from stats.mstats.basic import * from stats.mstats.extras import * from stats.mstats.basic_unchecked import *

Re: are masked array statistical function hiddenintentionally?

Pierre GM — 2008-11-19T13:55:19

On Nov 19, 2008, at 6:38 AM, Scott Sinclair wrote: All, I tend to edit the docstrings as I edit the code, and most functions/ methods do have a proper docstring that follows the numpy standard. There's definitely a lack of examples and see alsos, though... I'm quite surprised to see that so many functions are not picked up during the doc build. Pauli, could you point me towards the part of the autosummary/autodoc that lists the functions in a module ? Should I edit the docstring of the module to organize the output (using ..autofunction / ..automethod directives ? Is it legit to put sphinx directives/shortcuts in the doc, or are we still trying to ensure compatibility with an extra package? Scott, thanks a lot for your suggestion, it'd be easier indeed for me to review stats.mstats functions docstrings than (re)write them.

Re: are masked array statistical function hiddenintentionally?

2008-11-19T13:49:16

Currently the MA related functions are split over two files, and users would have to know which function is in which module. Should we create a namespace stats.ma that imports all of mstats and mmorestats? Then stats.ma can be imported and included in stats,__init__.__all__. something like (or relative imports) stats.ma.py ----------------- from scipy.stats.mstats import * from scipy.stats.mmorestats import * ---------------- this would make it easier to find, both for users and for np.lookfor. Another question: When users want to use masked array version by default, is there a recipe to overwrite all corresponding functions in scipy.stats, i.e. something like stats.__dict__.update(stats.ma.__dict__ ... but only for public function. If users need this often, there could even be a function, like: def make_ma_as_default: scipy.stats.__dict__.update with stats.ma methods Josef

Re: are masked array statistical function hiddenintentionally?

Scott Sinclair — 2008-11-19T11:38:01

2008/11/19 Pauli Virtanen iki.fi>: I'm putting some effort into this area at the moment, but I'm a marathon runner not a sprinter (tm) and also trying to work it out as I go along. Pierre, if you aren't able to find the time for this, then it might be most productive (for you) to review the docstrings as I work on them. That way you're just checking that the documentation doesn't lie or miss any subtleties. Cheers, Scott

Re: are masked array statisticalfunctionhiddenintentionally?

Pauli Virtanen — 2008-11-19T09:40:28

[clip] [clip] Just to clarify: I tried to say that the part of the Sphinx-generated docs dealing with MA are not properly organized at the present: - http://docs.scipy.org/doc/numpy/reference/arrays.classes.html#module-numpy.ma Some general text exists, but eg. special methods and properties of MAs are not systematically listed. Also, MAs might also deserve a page of their own. - http://docs.scipy.org/doc/numpy/reference/routines.ma.html#routines-ma Not all MA functions or constants (eg. ma.masked) are listed. Reorganization of this is pending in my job queue, but help is appreciated.

Re: are masked array statistical functionhiddenintentionally?

Pauli Virtanen — 2008-11-19T09:07:13

[clip] The docs don't pick up functions automatically -- instead, each function is manually added to the docs, to a place that makes sense. This is the way Sphinx works, and IMHO it is the correct way -- only the developer knows what it supposed to be the API of a module and how to best describe it. The Scipy docs live in the Scipy repository, under doc/. Please feel free to add whatever you think is missing there. Ditto for the numpy docs (they are also in Numpy's SVN, but under /numpy-docs/trunk). Pierre -- the masked array documentation in Numpy Reference Guide is especially lacking as not all MA functions are listed or the functionality explained. I'm not so familiar with MA, so I would appreciate help in writing the documentation for this part of Numpy. np.lookfor is a different beast, and it IIRC picks only items listed in __all__. I am the one to blame if the logic in it is incorrect, so please tell me if you have a better idea how it should work.

Re: vonmises_cython exposes all of numpy to np.lookfor

2008-11-19T00:19:34

I think it is missing inside the cython file, and since I never seriously touched cython or pyrex files, I would prefer if someone else does it. Josef On Tue, Nov 18, 2008 at 7:12 PM, Robert Kern gmail.com> wrote:

Re: vonmises_cython exposes all of numpy to np.lookfor

Robert Kern — 2008-11-19T00:12:07

Go ahead and add it.

vonmises_cython exposes all of numpy to np.lookfor

2008-11-19T00:10:36

vonmises_cython exposes all of numpy to np.lookfor It looks like there is no __all__ in vonmises_cython, instead it exports np, scipy and __builtins__ which seems to confuse np.lookfor ['__builtins__', '__doc__', '__file__', '__name__', 'i0', 'np', 'numpy', 'scipy' , 'von_mises_cdf', 'von_mises_cdf_normalapprox'] Josef

Re: are masked array statistical function hiddenintentionally?

2008-11-19T00:04:01

I thought it more as an "advertising" question, since the docs seem to pick up only functions that are exposed in __all__, also np.lookfor does not seem to pick up any of the functions. Josef

Re: are masked array statistical function hiddenintentionally?

Pierre GM — 2008-11-18T21:52:16

On Nov 18, 2008, at 4:35 PM, josef.pktd< at >gmail.com wrote: I'd say yes. I tried to keep the names of these functions as close to their non-masked equivalent as possible. Import ting them in __init__ would likely erase the non-masked ones (or vice-versa depending the import order). It's not that difficult to access them through: from scipy.stats.mstats import thefunctionyouwant or import scipy.stats.mstats as mstats About the documentation: well, I guess I should take the blame for not having written more thorough docstrings. However, I'm not in charge of building the whole doc. Now, I wonder whether it wouldn't be worth to consolidate things a bit, by making sure a function returns a masked array if its input is a masked array, a ndarray otherwise...

are masked array statistical function hiddenintentionally?

2008-11-18T21:35:28

scipy\stats\mstats.py and scipy\stats\mmorestats.py are masked array version of many of the statistical function. They are not imported in any __init__.py and I did not find them in the new documentation for scipy at http://docs.scipy.org/doc/scipy/reference/stats.html. Is this on purpose or not? Josef

Re: symeig integration ready

Jarrod Millman — 2008-11-18T19:27:20

That's OK with me, but I don't have a strong opinion on it. That is excellent! I would just leave them as is for now. I would rather have the tests running relatively aggressively for the upcoming 0.7.0b1 release by default. Thank you very much for putting the effort into getting this done. If you could, I would appreciate it if you would add this to the release notes: http://projects.scipy.org/scipy/scipy/browser/trunk/doc/release/0.7.0-notes.rst

symeig integration ready

Tiziano Zito — 2008-11-18T18:05:44

Dear devs, I have managed to integrate symeig in scipy, but before doing a commit I would like to get your feedback: - scipy.linalg.eigh gets the following signature: eigh(a, b=None, lower=True, eigvals_only=False, overwrite_a=False, overwrite_b=False, turbo=True, eigvals=None, type=1): this is different from the current signature: eigh(a, lower=True, eigvals_only=False, overwrite_a=False): Is it OK to break backward-compatibility? Code would only break if someone was specifing keyword arguments as positional arguments... Same thing for the eigvalsh routine. - I added the pyf wrappers for the new lapack routines in generic_lapack.pyf . They look somewhat different from all other wrappers (they were written 6 years ago without scipy in mind). In the long run I think those fortran routines should get the same kind of wrappers all others have, but I can not do such a rewriting without the guidance of the guy who wrote the generic_XXX.pyf in the first place, which I assume is

Re: help needed with 0.7.0 release notes

Thouis (Ray) Jones — 2008-11-18T17:59:23

I don't have SVN write access, but here's a patch for the release notes adding a short description the matlab io changes (hopefully I've formatted it appropriately). Ray Jones On Sun, Nov 16, 2008 at 3:05 AM, Jarrod Millman berkeley.edu> wrote: _______________________________________________ Scipy-dev mailing list Scipy-dev< at >scipy.org http://projects.scipy.org/mailman/listinfo/scipy-dev