gmane.comp.programming.garbage-collection.boehmgc

Re: [Gc] Possible to specify heap start address and size?

Ivan Maidanski — 2013-05-23T22:03:07

 Hi Yi,

To control start address, specify -DUSE_MMAP -DHEAP_START=<addr>
I haven't tried, though.

To set the size, specify -DGC_INITIAL_HEAP_SIZE=<size> -DGC_MAXIMUM_HEAP_SIZE=<size>

Regards,
Ivan

Thursday, 23 May 2013, 12:22 +10:00 from Yi Lin <qinsoon< at >gmail.com>:

_______________________________________________
Gc mailing list
Gc-V9/bV5choksm30D7ZfaTJw< at >public.gmane.org
http://www.hpl.hp.com/hosted/linux/mail-archives/gc/

Re: Trying to fix test_stack on powerpc.

Pavel Raiskup — 2013-05-23T11:58:15

I tried to find the reason several times but ended up with:
http://www.hpl.hp.com/hosted/linux/mail-archives/gc/2012-December/005507.html
In case you aren't aware of that workaround.

I tried to understand the lock logic in the cuddling with x_bits but I was
unable to prove _in my head_ it is really correct ...  I gave up.

Pavel

Re: Defining USE_LIBC_PRIVATES on all glibc systems

Ludovic Courtès — 2013-05-23T10:22:22

Hi Ivan,

Ivan Maidanski <ivmai-JGs/UdohzUI< at >public.gmane.org> skribis:


Great, thanks!


I don’t recall the details, but the message was mostly a request for
comments.

Ludo’.

Possible to specify heap start address and size?

Yi Lin — 2013-05-23T02:22:41

Hi all,

I am using boehm gc in my project. I want to know if it is possible to 
configure boehm gc to specify heap start address and size thus all the 
used memory will be in a fixed memory range. Thanks very much.

Regards,
Yi

Trying to fix test_stack on powerpc.

Lennart Sorensen — 2013-05-22T22:26:37

Does anyone have a hint as to where to look for the test_stack failure
on powerpc?

It seems to fail in one of two ways.

Either it fails when one thread fails to do a pop of the stack (it gets
a 0).

Or it fails when checking the stack at the end and the stack has a
circular list in it, due to the next pointers having gotten messed up.

For example for the circular list problem it can end up like this (note
I did add some extra debug printing to things):

Initial list (nthreads = 3):
6 0x1059b0e8 next=1059b0d8
5 0x1059b0d8 next=1059b0c8
4 0x1059b0c8 next=1059b028
3 0x1059b028 next=1059b018
2 0x1059b018 next=1059b008
1 0x1059b008 next=00000000
starting thread 1
starting thread 0
starting thread 2
finished thread 2: 360828 total ops
finished thread 0: 289581 total ops
finished thread 1: 349590 total ops
3 367
final list (should be reordered initial list):
(dump of original list locations)
0: 6 0x1059b0e8 next=1059b0e8
1: 5 0x1059b0d8 next=1059b0c8
2: 4 0x1059b0c8 next=1059b018
3: 3 0x1059b028 next=1059b028
4:

Re[2]: [bdwgc] Convert readme to markdown. (#24)

Ivan Maidanski — 2013-05-22T21:56:42

 Hi David,

Thank you for the explanation (for the work of rewriting README in .md style) - I've committed it (plus minor changes and fix of a typo in the title):
*  https://github.com/ivmai/bdwgc/commit/a36f601f2219402f1dfbae82f089002e1d4f7869
*  https://github.com/ivmai/bdwgc/commit/b6664ad715fb835990da5c3b8e97cf036997958e

Regards,
Ivan

Sun, 12 May 2013, 9:36 -07:00 from David Terei <notifications< at >github.com>:
_______________________________________________
Gc mailing list
Gc-V9/bV5choksm30D7ZfaTJw< at >public.gmane.org
http://www.hpl.hp.com/hosted/linux/mail-archives/gc/

Re[2]: Defining USE_LIBC_PRIVATES on all glibc systems

Ivan Maidanski — 2013-05-22T21:49:40

 Hi Ludo,

I've committed part (with modifications, plus adding a commit message) of the proposed patch -  https://github.com/ivmai/bdwgc/commit/14a5128999f6590969741e7e74a2eee4157ab0ad

 The rest is incorrect (have you tested it?) or just no-op:

1.
  STATIC ptr_t GC_linux_main_stack_base(void) ...
#endif

What's this? You define GC_linux_main_stack_base for GLIBC & !LINUX_STACKBOTTOM case but do not use it in such a case.

2.

Syntax error (missing parenthesis)

3. Ok, let correct the syntax error, and then we will get crash at runtime (if single-threaded, otherwise GC_linux_main_stack_base is not called) because libc_stack_end points to primordial thread stack thus you should call GC_INIT from that thread.

Regards,
Ivan

Wed, 22 May 2013, 0:19 +02:00 from ludo< at >gnu.org (Ludovic Courtès):

_______________________________________________
Gc mailing list
Gc-V9/bV5choksm30D7ZfaTJw< at >public.gmane.org
http://www.hpl.hp.com/hosted/linux/mail-archives/gc/

Re: Defining USE_LIBC_PRIVATES on all glibc systems

Ludovic Courtès — 2013-05-21T22:19:48

Hello,

Any comments on this patch?

TIA,
Ludo’.

ludo-mXXj517/zsQ< at >public.gmane.org (Ludovic Courtès) skribis:

Re: [bdwgc] Convert readme to markdown. (#24)

Ivan Maidanski — 2013-05-12T12:17:51

 Hi David,

I think it is good to have ability to use some mark-down features for documentation (not only main README file). But probably a make file rule should be added to generate plain (and/or html) file from the .md file.

Any suggestions, pros/cons from the community about it?

Thank you

Regards,
Ivan

Fri, 10 May 2013, 13:22 -07:00 from David Terei <notifications< at >github.com>:

_______________________________________________
Gc mailing list
Gc-V9/bV5choksm30D7ZfaTJw< at >public.gmane.org
http://www.hpl.hp.com/hosted/linux/mail-archives/gc/

Request for Comments - Manual Generations

Brian Beuning — 2013-04-17T15:30:10

I had an idea for making GC for C / C++ more generally usable
and request your input on the idea.  But before getting to the
idea I need to cover some background.

BACKGROUND

The recent versions of Doug Lea malloc includes the concept of mspaces.
An mspace is a heap, it supports malloc() and free() for objects in
the mspace.  A process can have multiple mspaces at once and an mspace
can be deleted entirely which implicitly free's all objects allocated
in the mspace.  Some papers have called an mspace a "region".

A test process allocating 2 GB memory using Boehm GC took 6 seconds
to do a GC.  For a real-time oriented process, 6 seconds is
not acceptable.  (This was a best case scenario, in a real 2 GB
process large parts might be paged out which would make the GC
much slower.)

IDEA

So what if GC supported a GCregion object that worked much like
an mspace, and we could declare relationships between GCregion
objects that would tell the GC which regions contain pointers to
which other regions.  Then we could

Re: Silently fails to allocate memory?

Bruce Hoult — 2013-04-14T20:42:52

On Sun, Apr 14, 2013 at 8:30 PM, The Devils Jester <
thedevilsjester-Re5JQEeQqe8AvxtiuMwx3w< at >public.gmane.org> wrote:


That's what GC does. It looks for pointers to your objects and if it can't
find any then it stomps on the object and gives it to you next time you ask
for a new object of that size.

If the only pointer(s) to your objects are in memory that the GC didn't
allocate (such as the internal storage in an STL vector) then the GC won't
see those pointers and will reuse and stomp on your objects.
_______________________________________________
Gc mailing list
Gc-V9/bV5choksm30D7ZfaTJw< at >public.gmane.org
http://www.hpl.hp.com/hosted/linux/mail-archives/gc/

Re[2]: [Gc]: Usage of _Jv_AttachCurrentThread, _Jv_AttachCurrentThreadAsDaemon, _Jv_DetachCurrentThread.

Ivan Maidanski — 2013-04-14T14:17:09

 Hi Dave,

About GC_thread_suspend/resume, please look into these posts:
*  http://article.gmane.org/gmane.comp.programming.garbage-collection.boehmgc/5469
* http://article.gmane.org/gmane.comp.programming.garbage-collection.boehmgc/3988
*  http://article.gmane.org/gmane.comp.programming.garbage-collection.boehmgc/2989

Regards,
Ivan

Wed, 10 Apr 2013, 9:24 +01:00 from Andrew Haley <aph< at >redhat.com>:

_______________________________________________
Gc mailing list
Gc-V9/bV5choksm30D7ZfaTJw< at >public.gmane.org
http://www.hpl.hp.com/hosted/linux/mail-archives/gc/

Re: Silently fails to allocate memory?

The Devils Jester — 2013-04-14T08:30:38

I made a few observations that allowed me to (I hope) fix the problem.  I
did a blanket "make everything use libgc, even things that have no need for
gc" change, so its hard to narrow it down, but some values were being
stomped on (or deleted) when libgc was in play.  It seems to work now and
at this point and I am considering it a win.
_______________________________________________
Gc mailing list
Gc-V9/bV5choksm30D7ZfaTJw< at >public.gmane.org
http://www.hpl.hp.com/hosted/linux/mail-archives/gc/

Re: Silently fails to allocate memory?

The Devils Jester — 2013-04-14T06:55:18



I have said that I am working on getting a more definitive definition.  At
current I am not sure what it does exactly except that I have a large loop,
that creates objects, and compares or modifies them in some fashion and at
some seemingly arbitrary point all of the compares/modifications will
simply fail.  The code will continue to loop and "complete" so long as the
loop ending condition does not depend on the values of any of the objects
created.  If for example, I created a "number" object that held a value,
and each iteration created another number object that holds the next value,
and do this ten thousand times, it will complete the ten
thousand iterations but the "last value" that it would show (assuming I
printed each value to the console as it was created) would be about 3500.
 If instead the loop is set to loop until I get an object that has a value
of say, 5000, it would never reach that point no matter how many objects I
created with iterative values, as if the objects are just "not created"
a

Re: Silently fails to allocate memory?

Brian Beuning — 2013-04-14T01:11:55

I recently used GC to allocate 2 GB of objects and it worked fine.

I think we get what you are doing and the issue you are seeing.
There are too many things your code might be doing wrong for anyone to guess at.
If you can reproduce the issue with some small test case (under 100 lines)
then someone will probably volunteer to look at it.


----- Original Message -----
From: "The Devils Jester" <thedevilsjester-Re5JQEeQqe8AvxtiuMwx3w< at >public.gmane.org>
To: "Basile Starynkevitch" <basile-VdE74OAlGqnvXjIo7pOF+l6hYfS7NtTn< at >public.gmane.org>
Cc: gc-V9/bV5choksm30D7ZfaTJw< at >public.gmane.org
Sent: Saturday, April 13, 2013 4:56:32 PM
Subject: Re: [Gc] Silently fails to allocate memory?



On Sat, Apr 13, 2013 at 3:24 PM, Basile Starynkevitch < basile-VdE74OAlGqnvXjIo7pOF+l6hYfS7NtTn< at >public.gmane.org > wrote: 





Also, if you have a working, GC-less, variant of your program (malloc based, or new+delete based), be sure 
to test with valgrind that it does not leak memory. 




I may not be explaining my situation well en

Re: Silently fails to allocate memory?

The Devils Jester — 2013-04-13T20:56:32

On Sat, Apr 13, 2013 at 3:24 PM, Basile Starynkevitch <
basile-VdE74OAlGqnvXjIo7pOF+l6hYfS7NtTn< at >public.gmane.org> wrote:

I may not be explaining my situation well enough.  I never use delete on
the objects that inherit from gc.  When I disable libgc, these objects are
never freed (I do not replace libgc with manual delete calls).  Which is
why I say its unlikely that its a out of memory issue when libgc is
enabled, because when I disable libgc, there is as much, or more,
un-deleted memory being created.  Although if I keep libgc in the mix, and
delete where I can (I can only do this in a limited controlled test, not in
actual real world situations) then it also works fine.

I use libgc because it would be very difficult to keep track of certain
objects to know when it should delete them.  As such it would be very
difficult to implement a non libgc test that satisfies valgrind.

I know that when libgc "fails", it is definitely memory (quantity) based,
since if I create more objects in the big loop, it fails

Re: Silently fails to allocate memory?

Basile Starynkevitch — 2013-04-13T20:24:32

Use everywhere the GC. So either redefine explicitly malloc as GC_malloc (and be sure your malloc is called even 
from inside the libc) or at the very least use gc_allocator to ensure that all vectors
(even those outside of the loop you are expecting to be at issue) are GC allocated.
In a previous recent email to this list I explained how to get STL vectors which use the GC_malloc
even for their internal data.


Also, if you have a working, GC-less, variant of your program (malloc based, or new+delete based), be sure 
to test with valgrind that it does not leak memory.

Cheers.

Re: Silently fails to allocate memory?

The Devils Jester — 2013-04-13T19:57:47

While that is something important to consider (and might have caused issues
later on in my application), I am fairly certain that in the current
testing process there are no vectors being used / created / resized within
the loop.  For the sake of argument though, even if there are, would not
the worse case scenario be exactly the same as just disabling libgc
entirely?  This is what I keep coming back to.  With libgc disabled,
everything works properly (other than the steadily increasing memory leak).
 It does not run out of memory,  and all objects are created as intended.

If I am doing something wrong, what could I possibly do that would result
in this situation?
_______________________________________________
Gc mailing list
Gc-V9/bV5choksm30D7ZfaTJw< at >public.gmane.org
http://www.hpl.hp.com/hosted/linux/mail-archives/gc/

Re: Silently fails to allocate memory?

Bruce Hoult — 2013-04-13T17:05:23

On Sun, Apr 14, 2013 at 4:35 AM, The Devils Jester <
thedevilsjester-Re5JQEeQqe8AvxtiuMwx3w< at >public.gmane.org> wrote:


b is allocated by the GC. The memory used internally by a to store your
myclass*'s is not.

See other messages today for how to fix that.
_______________________________________________
Gc mailing list
Gc-V9/bV5choksm30D7ZfaTJw< at >public.gmane.org
http://www.hpl.hp.com/hosted/linux/mail-archives/gc/

Re: Silently fails to allocate memory?

The Devils Jester — 2013-04-13T16:35:48


As far as I can tell, yes.  I am working on digging deeper to find exactly
what happens when it hits its magic number.  After it "fails" (with no
message of any kind) it continues on as if everything is OK, but the
creation of new objects as far as I can tell just stops.



I am not doing anything in any destructor and have no file, stream, or
other handles that need closed so I am not worried about destructors.

If my object has a vector of pointers, would not those be automatically

I might not be understanding this correctly.  Additional comments say that
memory for my objects in a vector are created using malloc, but I
explicitly use the new operator, for example:
vector<myclass*> a;
myclass* b = new myclass();
a.push_back(b);

If myclass inherits from gc, and I use the new operator (that has been
overloaded by libgc) what would be the issue here?  Am I missing something?

On Sat, Apr 13, 2013 at 7:54 AM, Brian Beuning <bbeuning-gQiW9Mwg1h1Wk0Htik3J/w< at >public.gmane.org>
 wrote:


How would this be true?

Re: Silently fails to allocate memory?

Brian Beuning — 2013-04-13T12:54:40

The simplest answer is your process is running out of memory.
Try using the ps(1) command to check your process size.

If you are using any C++ STL containers (string, map, vector, list, etc.),
I expect they would allocate memory using regular malloc and GC will not
reclaim that memory.  You need to define an STL allocator template and
pass it to the container templates to get STL to allocate memory using GC.
(Overriding the global operator new might work on Linux, it does not work
on Windows.)

----- Original Message -----
From: "The Devils Jester" <thedevilsjester-Re5JQEeQqe8AvxtiuMwx3w< at >public.gmane.org>
To: gc-V9/bV5choksm30D7ZfaTJw< at >public.gmane.org
Sent: Friday, April 12, 2013 5:51:53 PM
Subject: [Gc] Silently fails to allocate memory?




It is difficult to describe the issue I am having with libgc. 

I have a for loop that allocates thousands of (small) objects. Each iteration of the loop only allocates a few new objects, but when the loop is sufficiently large (usually around 3500 iterations), then fo