gmane.comp.programming.garbage-collection.boehmgc

Uninitialized GC_threads and dll_thread_table vars

Ivan Maidanski — 2008-10-06T20:46:58

Hi! Is it safe to place GC_threads (both on Unix and Win32) and dll_thread_table (on Win32) to "common" section (instead of "bss")? It seems to me that not... Bye.

GC minor fix for Win32

Ivan Maidanski — 2008-10-06T20:06:35

Hi! "excpt.h" file should be included even for non-Gnu compilers on MSWIN32 in mark.c (for GetExceptionCode macro to get defined). This is required at least for Watcom C (and doesn't hurt other Win32 compilers - VC++/IntelC++, Borland, DMC). The patch is attached. Bye. diff -ru bdwgc/mark.c updated/bdwgc/mark.c --- bdwgc/mark.c2008-09-26 14:01:50.000000000 +0400 +++ updated/bdwgc/mark.c2008-10-06 23:01:56.000000000 +0400 < at >< at > -19,7 +19,7 < at >< at > # include # include "private/gc_pmark.h" -#if defined(MSWIN32) && defined(__GNUC__) +#if defined(MSWIN32) # include #endif _______________________________________________ Gc mailing list Gc-V9/bV5choksm30D7ZfaTJw< at >public.gmane.org http://www.hpl.hp.com/hosted/linux/mail-archives/gc/

RE: small buglibatomic_ops-1.2/src/atomic_ops/sysdeps/gcc/x86_64.h

Boehm, Hans — 2008-10-03T17:46:08

Thanks. I fixed it in my tree and will check it in after the usual rudimentary testing. Hans

RE: gc's embedded libatomic_ops

Boehm, Hans — 2008-10-03T17:33:04

The current GNU-style build machinery tries to use a system-wide one if it's available, and falls back to the internal one if not. It's not clear to me how we could change this without making some builds a lot more inconvenient. For example, I'm not at all sure how a Windows/VC++ build would work without the embedded libatomic_ops. The other peculiarity here is that currently the libatomic_ops tree embedded in gc IS my development tree for libatomic_ops. It is in fact a full tree that should build stand-alone without the GC. This is admittedly inelegant. It was driven by the fact that I couldn't see a way to get away from the embedded tree, and I was doing a bad job at keeping two source trees in sync, mostly for lack of time. Plus atomic_ops didn't have a proper cvs/svn repository. Hans

gc's embedded libatomic_ops

Rex Dieter — 2008-10-03T16:57:24

Just noticed the embedded copy of libatomic_ops. Any reason to *not* use a system-wide/external libatomic_ops? Any objections to my preparing patches to do that? :)

small bug libatomic_ops-1.2/src/atomic_ops/sysdeps/gcc/x86_64.h

Andrew Agno — 2008-10-03T15:40:39

While browsing the source of gc-7.1, I noticed that libatomic_ops-1.2/src/atomic_ops/sysdeps/gcc/x86_64.h has unsigned short as the return from AO_int_fetch_and_add_full instead of unsigned int (on line 99). Andrew.

Re: stack problem

Romano Tenca — 2008-10-01T14:55:43

Boehm, Hans ha scritto: I have done some control on my program, and i see that GC_get_stack_min is called many times in a row by GC_register_dynamic_libraries -> GC_cond_add_roots -> GC_get_next_stack and always with the same address to check. Romano Paolo Tenca

RE: Re[2]: stack problem

Boehm, Hans — 2008-09-30T21:42:44

GC_get_stack_min() walks the stack towards lower addresses, possibly one page at a time, looking for the lowest (hottest) committed address in the stack. It calls VirtualQuery for each page, which I would guess is where the time goes. The result is used as a plausibility check on the stack pointer. If the sp fails the plausibility check (hopefully very rare), GC_get_stack_min() is used instead of the sp for stack scanning. I suspect this code was there to deal with assembly code that didn't correctly mainatin the stack pointer, though this seems like the kind of code you really want to rely on as little as possible. I can see two possible solutions, other than throwing out the sanity check: 1) Initially call VirtualQuery on the sp. If the stack base is in the same region, we know we're OK, and don't need GC_get_stack_min. Hopefully this will be true about 100% of the time. 2) Cache the last result of GC_get_stack_min() for each thread, and start the next search with the previous value. I'd be inc

Re: stack problem

Romano Tenca — 2008-09-30T13:13:37

Boehm, Hans ha scritto: I have tested the program with last CVS version. I have tested it with native thread support and with pthreadw32 support, with THREAD_LOCAL_ALLOC and without., with LARGE_CONFIG and without, with MUNMAP and without. The results are always the same. Then i have tested it with GC without thread support and the results change a lot: After 0 recursions 10 GC_gcollect() time is: 0.03125 After 5000 recursions 10 GC_gcollect() time is: 0.0625 After 10000 recursions 10 GC_gcollect() time is: 0.109375 After 15000 recursions 10 GC_gcollect() time is: 0.140625 After 20000 recursions 10 GC_gcollect() time is: 0.171875 After 25000 recursions 10 GC_gcollect() time is: 0.21875 and when i try again the recursive functions results are exactly the same: stack problem is gone. Is the issue just that you're starting out with nothing in the heap? The final times in the two cases don't seem that different? I agree that the absolute times seem way too long for the heap size, so that's probably

RE: stack problem

Boehm, Hans — 2008-09-30T01:14:41

Could you try this with 7.1? There was a significant performance bug fix shortly before that release. I'm not sure that's the issue here, but it would be good to preclude that. If that doesn't work, see below: Is the issue just that you're starting out with nothing in the heap? The final times in the two cases don't seem that different? I agree that the absolute times seem way too long for the heap size, so that's probably not it. It would be interesting to profile the two cases. GC_push_stack_for() contains some debugging code that prints out how much of the stack is actually getting pushed. It's normally not compiled in, but it would be easy to turn on for an experiment. There is fallback code that pushes the whole stack, but that shouldn't be getting used. And it should generate warnings if it is used. Hans

stack problem

Romano Tenca — 2008-09-29T12:21:13

I use GC in a program which sometime call a recursive function which consume a large C stack. After the use of this function, when the C stack is back to a normal size, GC allocations and recycles becomes a lot more slow. To give an idea, here it is the output of a test program which reports the time used by 10 GC_gcollect() calls inside a recursive function: After 0 recursions 10 GC_gcollect() time is: 0.03125 After 5000 recursions 10 GC_gcollect() time is: 0.203125 After 10000 recursions 10 GC_gcollect() time is: 0.59375 After 15000 recursions 10 GC_gcollect() time is: 1.1875 After 20000 recursions 10 GC_gcollect() time is: 2.015625 After 25000 recursions 10 GC_gcollect() time is: 3.0625 Here are some heap values after the recursive function returns: Heap: 5505024 Free: 4296704 Used: 1208320 Collections: 61 Now i call again the recursive function: After 0 recursions 10 GC_gcollect() time is: 2.984375 After 5000 recursions 10 GC_gcollect() time is: 3.015625 After 10000 recursions 10 GC_gcollect()

Re: "Leaked composite object at",but the object was never allocated

Benjamin Smedberg — 2008-09-25T18:54:30

I did this and found the allocation location! #0 setup_header (hhdr=0x807d010, block=0x808c000, byte_sz=32, kind=2, flags=0) at ../../../src/memory/boehmgc/allchblk.c:233 #1 0xf7fca5d4 in GC_allochblk_nth (sz=32, kind=2, flags=0, n=16, may_split=1) at ../../../src/memory/boehmgc/allchblk.c:791 #2 0xf7fca799 in GC_allochblk (sz=32, kind=2, flags=0) at ../../../src/memory/boehmgc/allchblk.c:628 #3 0xf7fd8790 in GC_new_hblk (gran=4, kind=2) at ../../../src/memory/boehmgc/new_hblk.c:190 #4 0xf7fcc0d0 in GC_allocobj (gran=4, kind=2) at ../../../src/memory/boehmgc/alloc.c:1055 #5 0xf7fd206b in GC_generic_malloc_inner (lb=27, k=2) at ../../../src/memory/boehmgc/malloc.c:119 #6 0xf7fd210a in GC_generic_malloc (lb=27, k=2) at ../../../src/memory/boehmgc/malloc.c:159 #7 0xf7fd2c7f in GC_malloc_uncollectable (lb=27) at ../../../src/memory/boehmgc/mallocx.c:466 #8 0xf7fd1bdd in malloc (lb=28) at ../../../src/memory/boehmgc/malloc.c:319 #9 0x00269dd3 in _dl_map_object_deps () from /lib/ld-linux.so.2

"Leaked composite object at",but the object was never allocated

Benjamin Smedberg — 2008-09-25T14:47:08

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 In my ongoing project attempting to use Boehm GC in Mozilla, I wanted to make sure that Boehm was scanning our memory correctly and that we weren't hiding any pointers unintentionally: so I hooked up Boehm in leak-detector mode. Fairly early in startup, I think during the first GC, Boehm reports a bunch of leaked objects of size 32. I tried turning on GC_DEBUG stack traces, but this object didn't have any debugging information (allocation stacktrace) recorded. So I went through and tried to find the allocation site using a debugger... Leaked composite object at 0x808c020 Leaked composite object at 0x808c040 Leaked composite object at 0x808c060 Leaked composite object at 0x808c080 ... all the way to 0x808c220 I tried breakpoing GC_allocobj, GC_generic_malloc_inner and several other functions, but none of these functions ever return an allocation at 0x808c020. In fact, the very first allocation returned is at 0x808dfe0. I'm worried that Boehm is reporting a le

Re: interior pointers to large allocations

Benjamin Smedberg — 2008-09-19T14:40:18

Please ignore this testcase... it has a bug. It seems that interior pointers are scanned correctly, and the bug must be somewhere else in my application code. --BDS

Re: Per-type displacement

Ludovic Courtès — 2008-09-19T14:34:58

Henning Makholm public.gmane.org> writes: FWIW, memory usage with all interior pointers is around 1.5 times that of the same code with explicitly registered displacements and no interior pointers. So there *is* excess memory retention here. Thanks, Ludo'.

Re: Per-type displacement

Ludovic Courtès — 2008-09-19T14:28:27

Hi, Henning Makholm public.gmane.org> writes: Hmm, I guess I had overlooked that part of the doc: Small blocks are allocated in chunks of size `HBLKSIZE'. Each chunk is dedicated to only one object size and kind. Then, understandably, displacements larger than a cell are not going to affect chunks that contain only cells, so my concern is not valid. Right. Well, I wasn't aware of the fact that chunks are allocated to only one object size (I wasn't concerned with the possibility of misidentifying tagged immediate numbers as pointers, but rather in cases like the struct one and tagged pointers). With that in mind, I'm no longer so sure. Performance-wise, is there a run time performance penalty in using `GC_all_interior_pointers'? Thanks for your help, Ludovic.

RE: Per-type displacement

Henning Makholm — 2008-09-19T13:17:41

How so? Registered displacements that are larger than the size of objects in the heap page pointed to do not count. This will be hard to implement efficitently, unless the typed-gc stuff has changed fundamentally since I looked at it last. There are separate heap pages for typed objects, byt objects with different type information can share the same page (otherwise unacceptable slack would occur for rarely-used types). The type information is stored in a hidden word at the end of the allocated block. This is fine as long as type information is used only for tracing the object itself - when that happens the page will have be brought into RAM anyway. However if one wants to use type information while tracing pointers TO the object, this means that the marker has to access not only the page header but also the page data in order to find out whether something is a pointer. That might harm cache/paging performance, unless almost everything that looks like a pointer turns out to BE a pointer to something tha

Per-type displacement

Ludovic Courtès — 2008-09-19T12:14:24

Hi, Guile's `struct' objects are such that they can have N extra words that are hidden from the application roughly as follows: void * make_struct (size_t n_extra, size_t user_size) { char *p = GC_MALLOC (n_extra + user_size); return p + n_extra; } Registering a displacement globally for each value of N_EXTRA may lead to excess memory retention, especially since most other Guile objects ("cells") are 2 or 4 word long. Ideally, I'd create a `GC_desc' and somehow specify valid displacements that apply specifically to this type, which the `gc_typed.h' API currently doesn't allow. Does that seem like a reasonable approach? Can it be approximated using the current API? Is it just that this allocation paradigm (returning a pointer in the middle of the object) is questionable? :-) Thanks, Ludo'.

Re: I/O issues

Nicolas Cannasse — 2008-09-19T10:52:07

FYI, we located and fix the problem, which was not related to GC itself. It was a close() on the stdin that was not setting the fd to -1, and at finalization time another close() would close a random fd instead. Best, Nicolas

Re: Unregistering the main thread

Ludovic Courtès — 2008-09-18T20:28:02

Hello, Hans Boehm public.gmane.org> writes: BTW, this function is not declared in the installed headers of GC 7.1, which is inconvenient. Thanks, Ludovic.

Re: interior pointers to large allocations

Benjamin Smedberg — 2008-09-18T17:06:09

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Benjamin Smedberg wrote: I wrote a testcase which shows that interior pointers to large objects are in fact not recognized... the finalizer runs before the loop finishes, and there is a crash with the following stack: #0 GC_is_marked #1 GC_finalize #2 GC_finish_collection #3 GC_try_to_collect_inner #4 GC_try_to_collect #5 GC_gcollect #6 RunTest #7 main -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFI0oqBSSwGp5sTYNkRAt6RAKDYACQik13dod77P4MhUyD++AqtmgCcCjKj yWiOzUwV+Z3Uma4Rw0v+Vlg= =sHyS -----END PGP SIGNATURE----- #include #include struct OffsetBuffer { int shared; size_t size; char buffer[12000]; }; static void OffsetBufferFinalizer(void *obj, void *client_data) { printf("Finalizing OffsetBuffer at %p, index %i\n", obj, (long long int) client_data); } static char* CreateOffsetBuffer(long long int i) { struct OffsetBuffer *ob = GC_malloc_atomic