gmane.comp.python.bio.devel

Re: UniProt GOA parser

Peter Cock — 2013-05-22T13:45:00

Great - a few more details then,

The second part of Bio/File.py has some private classes
_IndexedSeqFileProxy and _IndexedSeqFileDict and
_SQLiteManySeqFilesDict which can be used for any
sequential record file format (meaning one after the other,
not just biological sequences).

These are used by the Bio.SeqIO.index() and index_db()
functions, and their sisters in Bio.SearchIO.

The idea is you write a subclass of _IndexedSeqFileProxy
for your new file format, and then this gets used by either
_IndexedSeqFileDict (in memory offset dictionary) or
_SQLiteManySeqFilesDict (SQLite offset dictionary).

Your _IndexedSeqFileProxy subclass has to define
an __iter__ method which loops over the file giving
a tuple for each record giving the identifier string
and the start offset, and ideally the length in bytes.
It must also define a get method which must seek
to the offset and then parse the record.

For the GOA files, the __iter__ loop will just spot
batches of lines for the same identifier which together
make up

[Biopython - Feature #3432] (New) Updated/Extendedmodule MeltingTemp in Bio.SeqUtils

2013-05-21T12:24:34

Issue #3432 has been reported by Markus Piotrowski.

----------------------------------------
Feature #3432: Updated/Extended module MeltingTemp in Bio.SeqUtils
https://redmine.open-bio.org/issues/3432

Author: Markus Piotrowski
Status: New
Priority: Normal
Assignee: 
Category: 
Target version: 
URL: 


Dear Biopython developers,

I updated/extended the MeltingTemp module of SeqUtils and would be happy if you would consider it for implementing. Please find the source code attached. Any feedback is appreciated.
'Old' module:
One method, Tm_staluc, which calculates the melting temperature by the nearest neighbor method, using two different thermodynamic data sets for DNA and RNA. Fixed salt correction formula.
'Updated' module:
1. Three different Tm calculations: one 'rule of thumb' (Tm_Wallace), one using approximative formulas basing on GC content (Tm_GC) and one using nearest neighbor calculations (Tm_NN).
2. The new Tm_NN allows the usage of different thermodynamic datasets (8 tables are included for Wats

Re: UniProt GOA parser

Peter Cock — 2013-05-20T13:16:45

Hi Iddo,

Over on this branch building on your work I moved things under
Bio.UniProt.GOA, and got things a bit more in line with PEP8:
https://github.com/peterjc/biopython/tree/uniprot-goa

(Drop me an email off list if you need a hand pulling those
changes into your branch)

Do you want to have a go at re-using the index code in Bio.File
(the back end for SeqIO and SearchIO's indexing)? Let me know
if the current setup is too mysterious and I can try and document
more of it and/or do this for the GOA module.

Peter

Re: UniProt GOA parser

Iddo Friedberg — 2013-05-17T21:35:41

OK. I added a few changes as suggested by Peter.

There is a parser now to group GAF files by DB_Object_ID, and a write
function to write them. Random access not implemented yet.

On Fri, May 10, 2013 at 12:32 PM, Iddo Friedberg <idoerg< at >gmail.com> wrote:

Re: Errors about installing biopython 1.61

林行众 — 2013-05-14T01:32:23

Hi Peter
I copy the test_Clustalw_tool.py from the github, now it does work. Thank
you!
Lin

2013/5/13 Peter Cock <p.j.a.cock< at >googlemail.com>


_______________________________________________
Biopython-dev mailing list
Biopython-dev< at >lists.open-bio.org
http://lists.open-bio.org/mailman/listinfo/biopython-dev

Re: Errors about installing biopython 1.61

Peter Cock — 2013-05-13T15:50:26

Hi Lin,

Thanks for checking that, but having looked in more detail
I think this is not related to the locale settings. My first guess
was wrong :(

I think I may have solved this - my test machine has both
clustalw 2.1 and clustalw 1.83, and they behave differently
for this example. The old test only worked with v2.1, fixed:
https://github.com/biopython/biopython/commit/859d07f3c5e8b789156a5ec2e98f4153ab896e00

If you want to verify this, you could update your copy of
Tests/test_Clustalw_tool.py to that from github (or just
tried installing the latest Biopython code from github?).

Note the Clustal developers intended that clustalw 1 and 2
would behave the same as each other (Version 2 was a
rewrite as a step towards version 3, no called ClustalOmega),
but there are still some minor differences.


OK, good.


Thank you, that should be fixed in the next release:
https://github.com/biopython/biopython/commit/a3bb49b56abb5cbb9a0a00accb57674115c7004d

Your feedback has been very helpful,

Thanks,

Peter

_____

Re: Errors about installing biopython 1.61

林行众 — 2013-05-13T14:34:31

2013/5/13 Peter Cock <p.j.a.cock< at >googlemail.com>

Hi Peter
I have run the locale in my serve

$ locale
LANG=en_US.UTF-8
LC_CTYPE=zh_CN.UTF-8
LC_NUMERIC=zh_CN.UTF-8
LC_TIME=zh_CN.UTF-8
LC_COLLATE="en_US.UTF-8"
LC_MONETARY=zh_CN.UTF-8
LC_MESSAGES="en_US.UTF-8"
LC_PAPER=zh_CN.UTF-8
LC_NAME=zh_CN.UTF-8
LC_ADDRESS=zh_CN.UTF-8
LC_TELEPHONE=zh_CN.UTF-8
LC_MEASUREMENT=zh_CN.UTF-8
LC_IDENTIFICATION=zh_CN.UTF-8
LC_ALL=

Is that locale you want?

I retryed the the test_Entrez_online.py, it's all right now. As you said,
it should be a connection problem.

I have put the file in the Doc/examples file, but the error still exists.
And i find there is no my_blat.psl in Doc/examples comparing with the zip
file i downloaded from github. After i put the my_blat.psi in the
Doc/examples, the error did not show up again.

Thanks
Lin

_______________________________________________
Biopython-dev mailing list
Biopython-dev< at >lists.open-bio.org
http://lists.open-bio.org/mailman/listinfo/biopython-dev

Re: Errors about installing biopython 1.61

Peter Cock — 2013-05-13T08:49:20

Hi Lin,

I also have clustalw 1.83, so this isn't simply a version
problem. It could be something subtle about the locale -
what language is your CentOS running in (that can alter
error messages etc)?


That's good - that's what it should say :)

(Sorry late/last was my typing error).


Thanks. I could reproduce the test_Tutorial.py problem with that.
This is easy to explain - I forgot to include the test file my_blast.xml
when doing the release (and you are the first person to report this
problem). I should have noticed this myself, sorry :(

I've fixed this ready for the next release - thank you for reporting this:
https://github.com/biopython/biopython/commit/c1b63b88dd5a50fa3f6f2aef840a51fe9092e0c5

If you want to, you can get the missing file from here:
http://biopython.org/SRC/Doc/examples/my_blast.xml

or:
https://github.com/biopython/biopython/raw/master/Doc/examples/my_blast.xml

If you save that in the Biopython 1.61 source under Doc/examples
then the Tutorial test should pass.

--

Did you retry

Re: Errors about installing biopython 1.61

林行众 — 2013-05-13T01:41:30

2013/5/13 Peter Cock <p.j.a.cock< at >googlemail.com>:
"/share/fg2/Linxzh/biopython-1.61/build/lib.linux-x86_64-2.6/Bio/Entrez/__init__.py",
"/share/fg2/Linxzh/biopython-1.61/build/lib.linux-x86_64-2.6/Bio/Entrez/Parser.py",
"/share/fg2/Linxzh/biopython-1.61/build/lib.linux-x86_64-2.6/Bio/Entrez/Parser.py",
February 2013 (Biopython
1.61)cfca876b7128d7#Doc/Tutorial.tex<https://github.com/biopython/biopython/commit/b84bda01bd22e93a1cf71613a5cfca876b7128d7#Doc/Tutorial.tex>

Hi，Peter！
Clustalw I am using is 1.83.

I've found the 'Late Update' in Tutorial.tex, it's ' \date{Last Update -- 5
February 2013 (Biopython 1.61)}'.

I downloaded the tar.gz from the biopython website.

Thanks
Lin

_______________________________________________
Biopython-dev mailing list
Biopython-dev< at >lists.open-bio.org
http://lists.open-bio.org/mailman/listinfo/biopython-dev

Re: samtools threaded daemon

Saket Choudhary — 2013-05-12T18:11:46

Just completed writing samtools wrapper :
https://github.com/biopython/biopython/pull/180

Unit Tests pending.

On 11 April 2013 23:51, Chris Mitchell <chris.mit7< at >gmail.com> wrote:

Re: Errors about installing biopython 1.61

林行众 — 2013-05-12T12:57:25

I am very Sorry about my mistake.
I want to install biopython 1.61 in a local server(CentOS),
    python setup.py build
    python setup.py test
and then showed some errors:


======================================================================
FAIL: Test an input file containing a single sequence.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test_Clustalw_tool.py", line 166, in test_single_sequence
    self.assertTrue(str(err) == "No records found in handle")
AssertionError

======================================================================
ERROR: Test Entrez.read from URL
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test_Entrez_online.py", line 34, in test_read_from_url
    rec = Entrez.read(einfo)
  File "/share/fg2/Linxzh/biopython-1.61/build/lib.linux-x86_64-2.6/Bio/Entrez/__init__.py",
line 362, in read
    record = handler.read(handle)
  File "/share/fg2/Linxz

Re: UniProt GOA parser

Iddo Friedberg — 2013-05-10T16:32:43


 Ok, that's good then




I was actually thinking you are suggesting that the whole file should be
read in memory, nit just buffer by DB-Object_ID.  My mistake.

Re: UniProt GOA parser

Peter Cock — 2013-05-10T16:26:13

Not at all - we'd record a dictionary mapping the record ID to an offset
in the file on disk, or record this mapping in an SQLite index file.


The examples I looked at only seemed to have a dozen or so
lines for each DB_Object_ID - but perhaps these were easy
cases? How many lines per DB_Object_ID in the worst cases?

Peter

Re: UniProt GOA parser

Iddo Friedberg — 2013-05-10T16:20:16


OK.



Would that require reading it all into memory? Uniprot_GOA files are huge,
it is impractical to read them in fully.



Perhaps only for organism specific file, which in some cases can be read
fully into memory.

Re: UniProt GOA parser

Peter Cock — 2013-05-10T10:06:19

I think for the namespace, we might be better off using Bio.UniProt.GOA,
where Iddo's parser would be in Bio/UniProt/GOA.py and any other
UniProt specific code could also go under Bio/UniProt - for example
a web API.

Some of Bio.SwissProt might also migrate here over time.


Would it make sense to want random access to the GOA files based
on the identifier (DB_Object_ID and/or DB_Object_Symbol)? That
should be fairly straight forward to do building on the indexing code
for Bio.SeqIO and SearchIO.

Note here I am picturing combining all the (consecutive) lines
for the same DB_Object_ID - currently the parser is line based,
but batching by DB_Object_ID would be a straightforward change
and may better suit some uses.


Peter

UniProt GOA parser

Iddo Friedberg — 2013-05-08T23:28:17

A new uniprot-GOA parser is available for you to poke around:

https://github.com/idoerg/biopython/tree/uniprot-goa/Bio/UniProtGOA

More on Uniprot-GOA: http://www.ebi.ac.uk/GOA

There are three file formats: GAF (gene association file) , GPA (gene
product association) and GPI (gene product information) explained here:
http://www.ebi.ac.uk/GOA/downloads

Input GAF files can be very large, due to the growth of uniprot GOA. If you
would like to test in a timely fashion, I suggest you get historical files,
which are smaller. Once you get to the > 40 version numbers, the runtime
for the example code in UniProtGOA.py goes over 2 minutes (on my i5
machine).

Old GAF files are available here:
ftp://ftp.ebi.ac.uk/pub/databases/GO/goa/old/UNIPROT/

Current GPI and GPA files are not very large.

Thanks to Peter for his help on this.

Best,

Iddo

Progress with ticket 3336

Nate Sutton — 2013-05-07T21:12:59

Hi,

Here is a progress follow up to http://lists.open-bio.org/pipermail/biopython-dev/2013-April/010548.html .  I have added a commit to the github branch that adds an option to create claude branch lines using linecollection.  The linecollection objects are stored in a tuple before adding them to the plot.  It’s in Bio/Phylo/_utils.py.  Is this what the last bullet point was requesting in https://redmine.open-bio.org/issues/3336 ?  

Thanks!

Nate

P. S.  I used a tuple to store the linecollection objects instead of a list because that was mentioned in the ticket but if that looks like it should be different let me know.  Also, I got some global variables to work with the code but I was only able to do that after declaring them as globals twice.  If there are suggestions on how to code that differently let me know.

_______________________________________________
Biopython-dev mailing list
Biopython-dev< at >lists.open-bio.org
http://lists.open-bio.org/mailman/listinfo/biopython-dev

Re: uniprot-GOA parse

Peter Cock — 2013-05-07T16:47:16

The file arrived :)

Did you have any thoughts on where in the namespace to put this?

The idea with github is you'd register an account, say iddux (since
that's your Twitter username), and then fork the repository as
https://github.com/iddux/biopython - and make a new branch
there with your changes, and ask for feedback or make a pull
request.

All that can be done without any write access to the main repository,
and is intended to lower the barrier to entry.

In your case, given you're a past project leader etc, drop me (or Brad
etc) an email once you've mastered the git basic and we can give you
direct access.

Regards,

Peter

uniprot-GOA parse

Iddo Friedberg — 2013-05-07T16:24:00

hi,

As promised, I have written a uniprot-goa parser. Very skeletal, has
iterators for reading the three uniprot-GOA file types, a write function,
and a couple of usage examples.

No github write access, so attaching.

Cheers,

Iddo

Re: Abstract for "Biopython Project Update" at BOSC2013

Peter Cock — 2013-05-06T11:23:24

The Biopython update has been accepted for a 10 minute talk slot at
BOSC (anyone else with an abstract submitted should have had an
email by now), the reviewers' feedback was short and positive:

  (A) Keep it short and show the variety of active sub-projects and
     people involved and the presentaion will will be attractive to the
     audience. The last year's talk is a good example (based on the
     shared slides).

(Last year it was Eric at BOSC 2012 in Long Beach, CA - well done)

(B) Nice to see latest news on BioPython and future directions of one
    of the most popular OpenBio project.

(C) This talk reports an update on the BioPython project (support for
     experimental codes, Python 3 compatibility, SearchIO and genomic
     variant formats). BioPython is one of the central projects of O.B.F
     and its update is worth getting some attention at BOSC.

We have until June to revise our abstract - so perhaps we should
do the next release this month in May ;)

Peter

Re: Lazy-loading parsers, was: Biopython GSoC 2013 applications via NESCent

Markus Piotrowski — 2013-05-03T06:32:43

Hi Zhigang,

Sequence read files from Next Generation Sequencing methods are several 
GB large. Don't know if they are regulary stored in GFF files, anyhow.

Best,

Markus

Am 2013-05-02 23:18, schrieb Zhigang Wu: