gmane.science.biology.galaxy.devel

very long load times for some shared libraries...

Langhorst, Brad — 2013-05-17T21:43:54

All of a sudden, some larger libraries are taking a very long time to open.
[cid:BB83AE73-EAE2-44B9-B294-918CDE8C18F4-SemfSWNwOS8< at >public.gmane.org]

This one timed out…

Anybody else seen this?

Could this be a database issue? Didn't see any really long queries in the postgres log (and i log anything longer than 1000ms)


brad


--
Brad Langhorst
Applications and Product Development Scientist
langhorst-SemfSWNwOS8< at >public.gmane.org<mailto:langhorst-SemfSWNwOS8< at >public.gmane.org>

Galaxy lists empty file for composite datatypes

Peter Cock — 2013-05-17T15:58:40

Hello all,

In replying to someone having trouble with the BLAST+ tools, I
found a new bug (and was reminded of an old bug I'd not yet reported)
in the handling of composite datatypes without a primary file (e.g.
BLAST databases).

Consider the screenshot, where the history text for my BLAST database
says "empty" (above the line saying "format: blastdbn") and the
"full path" (highlighted) shown via the "information" icon points at an
empty file, in this case:

$ ls /mnt/galaxy/galaxy-dist/database/files/009/dataset_9085.dat -l
-rw-r--r-- 1 galaxy galaxy 0 May 17 14:40
/mnt/galaxy/galaxy-dist/database/files/009/dataset_9085.dat

The real data is here:

$ ls /mnt/galaxy/galaxy-dist/database/files/009/dataset_9085_files/
blastdb.nhd  blastdb.nhi  blastdb.nhr  blastdb.nin  blastdb.nog
blastdb.nsd  blastdb.nsi  blastdb.nsq

I am guessing the code assumes a composite datatypes like HTML
with a primary file, where it makes some sense to measure the
size of the primary file (although this omits the child files like

Re: tool_dependencies inside tool_dependencies

John Chilton — 2013-05-17T15:21:53

Hey All,

There was a long conversation about this topic in IRC yesterday (among
people who don't actually use the tool shed all that frequently), I
have posted it to the new unofficial Galaxy Google+ group if anyone
would like to read and chime in.

https://plus.google.com/111860405027053012444/posts/TkCFwA2jkDN

-John


On Tue, May 14, 2013 at 3:59 PM, Nate Coraor <nate-dukY5rQBki03uPMLIKxrzw< at >public.gmane.org> wrote:

Galaxy group at Google+

Samuel Lampa — 2013-05-17T14:21:40

Hi,

I realized there was no Galaxy group on G+, although I find G+ is a 
really nice medium for some kind of posts, especially for things like 
showing off some work you have done, without disturbing others on the 
mailing lists.

Feel free to chime in:
http://gplus.to/galaxyportal

Cheers,
// Samuel

Cross Tool Shed dependencies / blast_datatypes

Peter Cock — 2013-05-17T14:16:50

Hi Greg,

Do you guys have a rough plan for when inter-ToolShed dependencies
will be supported?

I've been using the Test Tool Shed to validate tools before uploading
them to the main Tool Shed - this uncovers things like omitted test
files which I don't find locally since I'm testing in-situ.

Right now the single biggest class of test failures amongst my tools on
the Test Tool Shed is a missing dependencies on the blast_datatypes
repository: http://toolshed.g2.bx.psu.edu/view/devteam/blast_datatypes

As a short term measure, could the dev team upload blast_datatypes
to the Test Tool Shed as well please (and give me write access)? Then
thanks to the recent change in [1] I can reference the dependency
without explicitly naming the Tool Shed.

If you can do it in a way that preserves the same revision history can
change set hashes even better - then I can replace this:

<repository toolshed="http://toolshed.g2.bx.psu.edu"
name="blast_datatypes" owner="devteam"
changeset_revision="f9a7783ed7b6" />

with:

<rep

Re: Test Tool Shed uploads broken

Peter Cock — 2013-05-17T13:49:49

Great - that works for me now :)

Peter

Add a download process to a job handler in historyqueue

Zinonas Antoniou — 2013-05-17T13:16:36

Hello,

I developed a script to download file to Galaxy from other platform. If 
the size of the file is large, the process may take several minutes and 
I can't to anything else in Galaxy. Is there a way to add the download 
process in a job handler and add it to history queue?

Any help is more than welcome! Thank you in advance!

Re: Workflow: multiple input errors

Joachim Jacob | VIB | — 2013-05-17T11:54:42

It is most likely due to the cluster configuration, since on our 
non-cluster production machine doesn't have this behaviour.

Cheers,
J

Joachim Jacob
Contact details: http://www.bits.vib.be/index.php/about/80-team


On 05/17/2013 10:16 AM, Joachim Jacob | VIB | wrote:

Re: Test Tool Shed uploads broken

Greg Von Kuster — 2013-05-17T11:37:04

Hi Peter,

Thanks for letting me know and sorry about the inconvenience.  I've fixed this in 9763:9c06caa86e2a which is now on the test tool shed. 

Greg Von Kuster

On May 17, 2013, at 5:08 AM, Peter Cock wrote:

Test Tool Shed uploads broken

Peter Cock — 2013-05-17T09:08:06

Hi Greg & Dave,

I just tried to upload an update to my Blast2GO wrapper on the Test Tool Shed,
http://testtoolshed.g2.bx.psu.edu/view/peterjc/blast2go

This gave the following error, also note typo sucessfully [sic] -> successfully.

Internal Server Error
Galaxy was unable to sucessfully complete your request

URL: http://testtoolshed.g2.bx.psu.edu/upload/upload?repository_id=e4de88c47079d971
Module galaxy.web.framework.middleware.error:149 in __call__
Module paste.debug.prints:106 in __call__
Module paste.wsgilib:543 in intercept_output
Module paste.recursive:84 in __call__
Module paste.httpexceptions:633 in __call__
Module galaxy.web.framework.base:132 in __call__
Module galaxy.web.framework.base:190 in handle_request
Module galaxy.web.framework:98 in decorator
Module galaxy.webapps.tool_shed.controllers.upload:122 in upload
Module galaxy.webapps.tool_shed.controllers.upload:326 in upload_tar
Module tool_shed.util.commit_util:116 in handle_directory_changes
NameError: global name 'undesirable_dirs' is not

Re: Handling of tool_dependencies.xml errors in Tool Shed testing

Peter Cock — 2013-05-17T09:06:01

Hi again,

I've realised that my current Blast2GO install script would be a perfect
example of where this tweak to fabric_util.py would help - when I wrote
the tool_dependencies.xml I didn't appreciate that the download_by_url
action was only allowed as the first action - currently it is ignored if listed
later in the script (just as any unknown actions are currently ignored).

http://testtoolshed.g2.bx.psu.edu/view/peterjc/blast2go/6e7694a0ae00

(I'm about to try and fix this tool_dependencies.xml file)

Peter

On Mon, May 13, 2013 at 10:34 AM, Peter Cock <p.j.a.cock-gM/Ye1E23mwN+BqQ9rBEUg< at >public.gmane.org> wrote:

Workflow: multiple input errors

Joachim Jacob | VIB | — 2013-05-17T08:16:34

Hi all,

I have fired up an cloud instance of Galaxy, 2 persistent nodes, 
scalable up to 10. I have updated the code to the latest stable, after 
issues with not being able to select multiple input for workflows (more 
background: 
http://dev.list.galaxyproject.org/selecting-multiple-inputs-for-workflows-not-possible-anymore-td4659827.html) 


Running a workflow on multiple inputs (#=58) gives me - after a very 
long page loading - this error:

************************************************


  Internal Server Error


    Galaxy was unable to sucessfully complete your request

URL: 
http://ec2-54-226-11-122.compute-1.amazonaws.com/workflow/run?id=e967e0737bb63910 

Module galaxy.web.framework.middleware.error:*149* in |__call__|
|>> 
<http://ec2-54-226-11-122.compute-1.amazonaws.com/workflow/run?id=e967e0737bb63910#>app_iter 
*=* self*.*application*(*environ*,* sr_checker*)*|
Module paste.recursive:*84* in |__call__|
|>> 
<http://ec2-54-226-11-122.compute-1.amazonaws.com/workflow/run?id=e967e0737bb63910#>

Re: selecting multiple inputs for workflows not possible anymore

Joachim Jacob | VIB | — 2013-05-17T07:29:17

Allright. I just pulled in all changesets from stable to my cloud 
Galaxy, restarted, and everything works, at first glance.

Thanks for your support!

J

Joachim Jacob

Rijvisschestraat 120, 9052 Zwijnaarde
Tel: +32 9 244.66.34
Bioinformatics Training and Services (BITS)
http://www.bits.vib.be
< at >bitsatvib

On 05/16/2013 05:21 PM, Dannon Baker wrote:

Re: Bug in history when using Admin account ??

2013-05-17T07:07:02

Hans,
      You are correct. When I look at the "Save Histories" and I double click on them I can see everything.

Thanks for the help

Neil

-----Original Message-----
From: Hans-Rudolf Hotz [mailto:hrh-y9CcQ38ZHqk< at >public.gmane.org] 
Sent: Friday, 17 May 2013 4:58 PM
To: Burdett, Neil (ICT Centre, Herston - RBWH)
Cc: galaxy-dev-btkOtGxnL5087WB9TwDe4A< at >public.gmane.org
Subject: Re: [galaxy-dev] Bug in history when using Admin account ??

Hi Neil

Just double checking: Have you looked at the list of 'Saved Histories'? 
For each user, you probably have several histories called 'Unnamed history' and one of them may contain the data you uploaded previously.

Whenever you log out and log in as a different user, a new, empty history called 'Unnamed history' is created.


Regards, Hans-Rudolf



On 05/17/2013 08:04 AM, Neil.Burdett-uXK74I8fpZ0< at >public.gmane.org wrote:

Re: Bug in history when using Admin account ??

Hans-Rudolf Hotz — 2013-05-17T06:57:58

Hi Neil

Just double checking: Have you looked at the list of 'Saved Histories'? 
For each user, you probably have several histories called 'Unnamed 
history' and one of them may contain the data you uploaded previously.

Whenever you log out and log in as a different user, a new, empty 
history called 'Unnamed history' is created.


Regards, Hans-Rudolf



On 05/17/2013 08:04 AM, Neil.Burdett-uXK74I8fpZ0< at >public.gmane.org wrote:

Bug in history when using Admin account ??

2013-05-17T06:04:30

Hi,
   I believe I have found a bug? Or maybe I'm not using Galaxy correctly? These are the steps I used:


*         Downloaded the latest version of Galaxy

*         Created a user me-BUHhN+a2lJ4< at >public.gmane.org<mailto:me-BUHhN+a2lJ4< at >public.gmane.org>

*         Uploaded several files

*         Logged out

*         Create a user you-SyO+X3TzcQY< at >public.gmane.org<mailto:you-SyO+X3TzcQY< at >public.gmane.org>

*         Uploaded several files

*         Logged out and then logged in as an Admin user (specified in universe_wsgi.ini file)

*         Uploaded a file.

Now when I log into me-BUHhN+a2lJ4< at >public.gmane.org<mailto:me-BUHhN+a2lJ4< at >public.gmane.org> or you-SyO+X3TzcQY< at >public.gmane.org<mailto:you< at >you.com> the history shows only the uploaded file that I uploaded as an Admin.  Even logging in as Admin and using the "impersonate" feature doesn't allow me to view the history of me-BUHhN+a2lJ4< at >public.gmane.org<mailto:me-BUHhN+a2lJ4< at >public.gmane.org> or you-SyO+X3TzcQY< at >public.gmane.org<mailto:you-SyO+X3TzcQY< at >pu

Re: Error with fastqc

Kathryn Sun — 2013-05-16T21:59:45

Dannon,
Thanks and it worked!
Kathryn

________________________________
 From: Dannon Baker <dannon.baker-Re5JQEeQqe8AvxtiuMwx3w< at >public.gmane.org>
To: Kathryn Sun <kathryn.sun-DaQTI0RpDDMAvxtiuMwx3w< at >public.gmane.org> 
Cc: "galaxy-dev-btkOtGxnL5087WB9TwDe4A< at >public.gmane.org" <galaxy-dev< at >lists.bx.psu.edu> 
Sent: Friday, May 10, 2013 10:07 AM
Subject: Re: Error with fastqc

Hi Kathryn,

It looks like you haven't installed FastQC (or potentially installed it and just haven't made it accessible to Galaxy).  You'll need to install FastQC from http://www.bioinformatics.babraham.ac.uk/projects/fastqc/ if you haven't already, and the main fastqc perl script must be executable from tool-data/shared/jars/FastQC/fastqc.

For dependencies like this, you can usually look at http://wiki.galaxyproject.org/Admin/Tools/Tool%20Dependencies for a few pointers.

Good luck,

Dannon

On Thu, May 9, 2013 at 9:43 AM, Kathryn Sun <kathryn.sun-DaQTI0RpDDMAvxtiuMwx3w< at >public.gmane.org> wrote:

Dannon,
fastqc to check how it

Re: Recall: [External] Re: [galaxy-user] Question on galaxy data visualization

Jennifer Jackson — 2013-05-16T18:56:50

OK, Glad things are working out!

Jen
Galaxy team

On 5/16/13 11:49 AM, Sun, Wenping [USA] wrote:

Recall: [External] Re: [galaxy-user] Question on galaxy data visualization

Sun, Wenping [USA] — 2013-05-16T18:56:33

Sun, Wenping [USA] would like to recall the message, "[External]  Re: [galaxy-user] Question on galaxy data visualization".

Recall: [External] Re: [galaxy-user] Question on galaxy data visualization

Sun, Wenping [USA] — 2013-05-16T18:49:09

Sun, Wenping [USA] would like to recall the message, "[External]  Re: [galaxy-user] Question on galaxy data visualization".

Re: [External] Re: [galaxy-user] Question on galaxy data visualization

Sun, Wenping [USA] — 2013-05-16T18:47:08

Jen,
Thank you for the information.
I am using galaxy cloud to try the trickster for one of the processed chipseq analysis. However, there is no visualization for the bed file created from galaxy. Meantime,  I am having problem to get back to galaxy now. Here is the error message as below --

The proxy could not connect to the destination in time.
URL: http://ec2-107-20-10-72.compute-1.amazonaws.com/cloud

I have tried to re-launch the cloud instance, as well as directly access the DNS from aws, and keep getting the same error.

Is this the bug? Please help to fix this since I have data there.

Thanks!
Kathryn

From: Jennifer Jackson [mailto:jen-dukY5rQBki03uPMLIKxrzw< at >public.gmane.org]
Sent: Monday, May 13, 2013 6:18 PM
To: Sun, Wenping [USA]
Cc: galaxy-user-btkOtGxnL5087WB9TwDe4A< at >public.gmane.org
Subject: [External] Re: [galaxy-user] Question on galaxy data visualization

Hello Kathryn,

Trackster is the primary data visualization tool and can be accessed by clicking on the mini graph icon within a dataset: