gmane.comp.distributed.boinc.devel

Why won't Boinc schedule both GPUs?

Charles Elliott — 2013-05-20T13:08:38

I am posting this here because the S< at >H Beta site message board will not let
me create a table.

 

Below you may see a table showing how Boinc scheduled the 2 GPUs in Computer
ID 52900 (S< at >H Beta) last night.  GPU 2 was vacant for most of the night.  My
conjecture is that when Boinc schedules an AP WU on one GPU, which is now
marked as using 0.805C + 1NV instead of the usual 0.02C + 1NV, it leaves the
other GPU vacant, but his does not explain all the rows in the table.  From
a credit per hour standpoint, it might be better to fully schedule the GPU
and let the O/S priority scheme worry about scheduling the CPU; if most of
the processes in the system are at normal priority, and Boinc tasks have a
lower priority, what it does it matter if the CPUs are over-scheduled?

 

Table: GPU Tasks scheduled on Computer 52900 (S< at >H Beta) 5/20/2013 

 


GPU 1

GPU 2



Start Time

Finish Time

WU Type

Start Time

Finish Time

WU Type


GPU GRID WU

GPU GRID WU



5/19 23:29

5/20 1:58

AP

-Infinity

1:53

GPU GRID 


  1:58

2:08

MB

1:58

2:08

MB


  2:08

2:18

MB



  2:18

3:02

AP



  3:02

3:14

MB

3:02

3:14

MB


  3:14

3:25

MB



  3:25

4:00

AP



4:00

4:12

MB

4:00

4:13

MB


  4:13

4:58

AP



  4:58

5:09

MB

4:58

5:09

MB


  5:09

5:21

MB

5:09

5:21

MB


           5:21

5:32

MB

5:21

5:32

MB

 

 

This system downloaded single-digits numbers of GPU WUs virtually every time
it made a server request last night.  At 5:00 AM it had the following WUs in
inventory:

 

MB CPU 168

        GPU 1906

AP  CPU  25

       GPU 578

 

This information posted yesterday on the Beta Site message board may help
clarify the situation.  

 

"BOINC Strange Behavior "

 

"1. Since installing 2 Kepler-class GPUs, Computer ID 52900 has been unable
to download more than a day's supply of WUs, usually there has been less
than a day's supply in inventory.

2. On May 14, the server sent 22 AP CPU WUs (which take about 27 hours
apiece to complete) with only two hours to finish them; they all timed out
and errored out.

3. Last week I suspended all the S< at >H WUs via Efmer's BoincTasks to finish up
the few AP WUs in inventory. Boinc only processed 3 more AP WUs, and then
quit using the GPUs altogether. It would only start using the GPUs again
when I unsuspended the regular S< at >H WUs, which it then processed.

4. At 21:40:00 5/18/2013, with about 1 day's supply of WUs available, all
evenly distributed between AP & S< at >H and between CPU and GPU, when an AP WU
finished on GPU 2 and after a communication with the server returning that
result, Boinc refused to schedule any further work on that GPU; it just left
it idle. I did not notice it until about 13:00:00 5/19/13. I tried
everything I could think of with preferences and cc_config.xml to try to
make Boinc use the second GPU; I even reinstalled Boinc 7.0.62 (replacing
7.0.64), but nothing worked. I could not reboot because I am running a long
experiment that would be impossible to restart except at the beginning.
Finally, at 13:39:00 I resumed GPUGRID and requested some work. When a
GPUGRID WU was finally completely downloaded at 13:52:00, it preempted the
AP WU on GPU 1, and began executing there; GPU 2 was still unused. So I
requested another WU from GPUGRID and it was scheduled on GPU 2. At that
point Boinc went wild with requests for more work from S< at >H Beta. At one time
there were 971 workunits in the transfer queue; I have not seen that for
months. Jealous mistress syndrome? Now I have 306 S< at >H CPU WUs (2 days), 1169
S< at >H GPU WUs (5.4 days), 5 AP CPU WUs (0.7 days), and 423 AP GPU WUs (14.5
days). The day's estimates may be a little fuzzy because of changes in apps,
which affects the average time per WU, but the quantities are exact.

5. Boinc no longer computes the time remaining on a WU by ratio and
proportion after the WU is 50% or even 85% complete based on how long it
took to complete the first part. When an AP WU is 99.99% complete, the GUI
still indicates it has 34 hours remaining." 

 

 

Has anyone at Berkeley ever taken a look at the top-performing computer on
the primary S< at >H site?  This guy has a Core i7-3970X with 12 CPUs and 8
NVidia GTX Titan GPUs; conservatively estimated at retail prices, that is at
least a $9,000 system.  Yet in terms of workunits, he operates always at the
margin.  For example, right this minute he has only 178 workunits in
progress.  Yet he can finish a SETI< at >home V7 MB GPU task in about a minute
and an AP task in about 33 minutes.  On the Tuesday before last, when I last
looked, he was out of workunits, sitting idle almost the entire day.
Imagine his disappointment, if not heartbreak.  Yeah, I know, he's not
complaining.  But is this really what Berkeley intends?  Does this
scheduling situation have anything to do with the new Kepler-architecture
NVidia GPUs, e.g., the absence of a cuda_kepler plan class analogous to the
cuda_fermi plan class?

 

 

Charles Elliott

Re: The ways in that BOINC tells app to quit

David Anderson — 2013-05-20T06:51:43

If BOINC needs to remove a job from memory,
it will resume from its last checkpoint
(or from the beginning if it hasn't checkpointed).

Re: Make 'receive PM of action taken by moderator' an opt-in option?

David Anderson — 2013-05-20T03:39:31

If this continues to be a problem I'll think about it.

Make 'receive PM of action taken by moderator' anopt-in option?

Jorden van der Elst — 2013-05-20T01:38:42

Hi David,

Can't we make the 'action taken by moderator' PMs an opt-in option in
the preferences? I have just moved 29 messages between two people,
they'll both be looking at ~14 PMs detailing how I moved their posts
to a new thread. As if they didn't know this, they requested the move
themselves.

I also get PMs when I make a new thread and lock it, rename it,
sticky/desticky it. Normal moderator actions that don't really need
these messages. So, please, can we get a preference that says "Don't
receive PMs on moderator actions on your posts", with the default
being on, but the option to opt-out?

--

Host getting more tasks that limits allow.

Stephen Maclagan — 2013-05-19T16:42:32

There is a Boinc 7.0.64 host over at Setiathome that has at present got 
~2000 GPU tasks,
when it should be limited to 100 CPU tasks and 100 GPU tasks:

http://setiathome.berkeley.edu/forum_thread.php?id=71687

http://setiathome.berkeley.edu/results.php?hostid=6165045&offset=460&show_names=1&state=0&appid=



It's still managing to pick up fresh GPU tasks too, Any idea why?

Claggy

Re: [patch] mystery solved? Aw: Re: BOINC having too many open files - failure in opendir()

David Anderson — 2013-05-19T03:34:16

As far as I can tell, none of these changes would fix
a file descriptor leak in the client.
We need a system-call trace that shows open()s.

It's also possible that the system is running out of file descriptors
because of software other than BOINC.

FW: BOINC logos

Patrick Stratford — 2013-05-18T20:59:33

From: paddys09-SrPyuzuu+o61Qrn1Bg8BZw< at >public.gmane.org
To: boinc_dev-C9EgComYM8RUAgJt6FLh2g< at >public.gmane.org
Subject: BOINC logos
Date: Sat, 18 May 2013 21:48:30 +0100

Hello Sir/Madam,

I am contacting you to request permission to use the BOINC  logo. I am a member of a forum and have been asked to design a badge system for various contributions, one being our own BOINC team, which I have incorporated the logo into. Obviously we don't want to run into any copyright issues, so would like to make contact beforehand to make sure that we have permission to do so? Any help on this matter would be greatly appreciated.

Thanks,
              Patrick

[patch] mystery solved? Aw: Re: BOINC having too many open files - failure in opendir()

Steffen Möller — 2013-05-18T11:00:36

Dear all,

I skimmed through all invocations of (boinc_)?fopen() in api/ and lib/, seeking the respective matching fclose().
What I found missing I placed here
http://anonscm.debian.org/gitweb/?p=pkg-boinc/boinc.git;a=blob;f=debian/patches/fopen_closing.patch;hb=HEAD
as a patch. The trickiest and possibly the most important one is the omission of a close in the destructor of the MFILE class.

Cheers,

Steffen


_______________________________________________
boinc_dev mailing list
boinc_dev< at >ssl.berkeley.edu
http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
To unsubscribe, visit the above URL and
(near bottom of page) enter your email address.

Admin Pages: Result summary per app version

Jon Sonntag — 2013-05-17T23:04:08

If I check the "list unofficial App versions" box I get the error:

Notice: Trying to get property of non-object in
/home/boincadm/projects/collatz/html/ops/pass_percentage_by_platform.php on
line 121 Database Error

at the top of the page and with the listing of the apps:

Notice: Trying to get property of non-object in
/home/boincadm/projects/collatz/html/ops/pass_percentage_by_platform.php on
line 123 Notice: Trying to get property of non-object


I'll dig into it further unless it has already been fixed in a more recent
version.

Jon Sonntag

Re: BOINC having too many open files - failure inopendir()

Nicolás Alvarez — 2013-05-17T20:00:35

Get the list of open files (ls -l /proc/$(pidof boinc)/fd) when that
happens. Does the client die after that last fopen() failure? Maybe
you could write a script to log the open file list every few minutes.

Re: The ways in that BOINC tells app to quit

Robert Miles — 2013-05-16T22:10:27

I've seen that, for the specific case case of CPU workunits that had to 
be suspended due
to BOINC running out of free memory it's allowed to use, the suspended 
workunit is not
saved in a state that allows resuming from its current point when it has 
not yet written
any checkpoints at all.  It therefore kept starting from the beginning, 
over and over.

In my case, this appears to be due to BOINC under 64-bit Windows Vista 
needing a tighter
limit on 32-bit memory space it can use than on the total 64-bit memory 
it can use.

Re: possible typo in configure.ac

McLeod, John — 2013-05-16T19:34:12

Pretty much standard for command files...

-----Original Message-----
From: boinc_dev [mailto:boinc_dev-bounces< at >ssl.berkeley.edu] On Behalf Of Toralf Förster
Sent: Thursday, May 16, 2013 3:33 PM
To: Eric J Korpela
Cc: boinc_dev< at >ssl.berkeley.edu
Subject: Re: [boinc_dev] possible typo in configure.ac

On 05/16/2013 08:38 PM, Eric J Korpela wrote:
ick - of course :-)

Re: possible typo in configure.ac

Toralf Förster — 2013-05-16T19:33:18

ick - of course :-)

Re: The ways in that BOINC tells app to quit

Raistmer the Sorcerer — 2013-05-16T19:19:42

 That flag will only show what BOINC "thinks it does". 
But as I said, it set suspend flag and doesn't set quit flag. Will BOINC make difference in logs in case of bug? I think no.
Cause suspend flag set, BOINC attempted to do suspend. But it did only half of work.
App checks both flags, it was shown on very same host.
Also, this issue appears only on some hosts, it's not common issue. Our alpha testers failed to reproduce it on own hosts. It means that sometimes BOINC sets (and app reads and obeys) all required flags.




Четверг, 16 мая 2013, 11:34 -07:00 от Eric J Korpela <korpela< at >ssl.berkeley.edu>:

_______________________________________________
boinc_dev mailing list
boinc_dev< at >ssl.berkeley.edu
http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
To unsubscribe, visit the above URL and
(near bottom of page) enter your email address.

Re: possible typo in configure.ac

Rom Walton — 2013-05-16T18:46:47

Yes

Branch charlief/wxwidgets30 contains the needed changed to compile against wxWidgets 2.9/3.0, but we haven't migrated to it yet.

----- Rom

-----Original Message-----
From: boinc_dev [mailto:boinc_dev-bounces< at >ssl.berkeley.edu] On Behalf Of Toralf Förster
Sent: Thursday, May 16, 2013 1:40 PM
To: boinc_dev< at >ssl.berkeley.edu
Subject: [boinc_dev] possible typo in configure.ac

/me wonders if here's an unwanted "x" :

AM_CONDITIONAL(ENABLE_MANAGER, [ test "x${ac_cv_have_wxwidgets}" = xyes


BTW is still wxwidgets version <= 2.8.x required ?

--
MfG/Sincerely
Toralf Förster
pgp finger print: 7B1A 07F4 EC82 0F90 D4C2 8936 872A E508 7DB6 9DA3 _______________________________________________
boinc_dev mailing list
boinc_dev< at >ssl.berkeley.edu
http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
To unsubscribe, visit the above URL and
(near bottom of page) enter your email address.
_______________________________________________
boinc_dev mailing list
boinc_dev< at >ssl.berkeley.edu
http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
To unsubscribe, visit the above URL and
(near bottom of page) enter your email address.

Re: possible typo in configure.ac

Eric J Korpela — 2013-05-16T18:38:05

That x is required.  They are there in case the tested variable is empty
(in which case it's "test x = x"  rather than the syntax error "test ="

I don't know what the current version requirement for wxwidgets is.


On Thu, May 16, 2013 at 10:40 AM, Toralf Förster <toralf.foerster-Mmb7MZpHnFY< at >public.gmane.org>wrote:

Re: The ways in that BOINC tells app to quit

Eric J Korpela — 2013-05-16T18:34:56

OK, I'll try to catch another one.


On Thu, May 16, 2013 at 11:31 AM, David Anderson <davea< at >ssl.berkeley.edu>wrote:

_______________________________________________
boinc_dev mailing list
boinc_dev< at >ssl.berkeley.edu
http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
To unsubscribe, visit the above URL and
(near bottom of page) enter your email address.

Re: The ways in that BOINC tells app to quit

David Anderson — 2013-05-16T18:31:54

Eric:

If you set the <task_debug/> logging flag,
the client will show when it suspends/resumes tasks.
This will tell us whether it's a problem in the client or the app.

Re: The ways in that BOINC tells app to quit

Eric J Korpela — 2013-05-16T18:16:59

I had "keeps running when computer is in use" problem last night with an
Einstein CUDA app, so it's clear that this isn't restricted to
SETI< at >home. There has to be a BOINC issue or some common flaw with how
exiting is
handled.

I'm wondering is there's some way that writing a checkpoint (or reporting a
checkpoint) is going wrong.  I assume that even GPU apps would suspend
rather than exit if there has been no checkpoint since it was started?

Of course in this case it's not suspending either, it's continuing to run.

David, any comments?


On Sun, May 12, 2013 at 11:47 PM, Raistmer the Sorcerer <raistmer< at >mail.ru>wrote:

_______________________________________________
boinc_dev mailing list
boinc_dev< at >ssl.berkeley.edu
http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
To unsubscribe, visit the above URL and
(near bottom of page) enter your email address.

possible typo in configure.ac

Toralf Förster — 2013-05-16T17:40:02

/me wonders if here's an unwanted "x" :

AM_CONDITIONAL(ENABLE_MANAGER, [ test "x${ac_cv_have_wxwidgets}" = xyes


BTW is still wxwidgets version <= 2.8.x required ?

Re: BOINC having too many open files - failure inopendir()

David Anderson — 2013-05-16T16:29:42

Are there tools for detecting file-descriptor leaks on Linux?