http://blog.gmane.org/gmane.linux.kernel.crash-dump.crash-utility hourly 1 1901-01-01T00:00+00:00 http://gmane.org/img/gmane-25t.png http://gmane.org http://comments.gmane.org/gmane.linux.kernel.crash-dump.crash-utility/4558 <pre>This is the release of crash gcore command, version 1.2.1. ChangeLog: - Fix failure of coredump at accessing memory for VSYSCALL page due to wrong conversion of uvtop which wrongly treats address VSYSCALL_START as belongs to kernel direct mapping region. This fix executes uvtop in verbose mode to make it always paging and retrieves the correct physical address from its output. Without this fix, VSYSCALL page fails to be collected and core dump process is aborted; though VSYSCALL page is done in the last so allmost all corefile is already generated. (d.hatayama&lt; at &gt;jp.fujitsu.com) - Skip page-faulted pages by lseek() rather than writing zero-filled pages. By this, generated core file has holes in the corresponding positions for each page-faulted pages if filesystem supports sparse files. This is highly useful when the target process has huge virtual memory space such as qemu process that has huge physical memory of KVM guest machine. (d.hatayama&lt; at &gt;jp.fujitsu.com) - Fix the bug that filter for hugepage shared/private memory can depend on flags other than HP or HS flags. This was introduced at the introduction of VM_DONTDUMP where VM_RESERVED flag was removed. At the time, there was a check to see if VM_REESRVED flag was set after a check to see if VM_HUGETLB. But the latter check was not changed when the former check was removed. (d.hatayama&lt; at &gt;jp.fujitsu.com) MD5 CheckSum: $ md5sum crash-gcore-command-1.2.1.tar.gz e767ab8f5e00a881f5c6e64fde541f7d crash-gcore-command-1.2.1.tar.gz </pre> HATAYAMA Daisuke 2013-05-14T04:15:51 http://comments.gmane.org/gmane.linux.kernel.crash-dump.crash-utility/4541 <pre>Hi, I got an IA64 vmcore. The stack backtrace only printed registers up to R31. How do I get the contents of R32 and above? Thanks, Jay </pre> Jay Lan 2013-04-22T23:38:02 http://comments.gmane.org/gmane.linux.kernel.crash-dump.crash-utility/4537 <pre>Hello Dave, The attachment is used to fix a bug of struct command. When using command like below, struct command displays wrong data. &lt;cut&gt; crash&gt; task_struct.fs ffff88003dfb5540 fs = 140234226652928 crash&gt; task_struct ffff88003dfb5540 ... es = 0, ds = 0, fsindex = 0, gsindex = 0, *fs = 140234226652928,* gs = 0, debugreg0 = 0, ... io_bitmap_ptr = 0x0, iopl = 0, io_bitmap_max = 0 }, *fs = 0xffff880037795f00,* files = 0xffff8800371b6940, ... &lt;cut&gt; fs showed by struct command is a member of "struct thread_struct thread;", not struct task_struct. This bug is involved by adding support to anonymous member modified by me... -- -- Regards Qiao Nuohan </pre> qiaonuohan 2013-04-12T07:26:15 http://comments.gmane.org/gmane.linux.kernel.crash-dump.crash-utility/4534 <pre>Machines are getting ever bigger. I routinely look at crash dumps from systems with 2TB or more of memory. I'm finding I'm wasting too much time waiting on crash to complete a command. For example "kmem -s" took close to an hour on the dump I'm looking at now. Has anyone ever looked into mutli-threading crash? Given the kmem -s example above, a thread could be created for each cache (up to some defined limit of threads). Things like "foreach" could spawn threads. I'm sure there are lots of other opportunities. Yes, I know, it's open source, I should just go do it myself. Still, I'd like to hear pro's and con's on this idea. </pre> James Washer 2013-04-12T00:24:56 http://comments.gmane.org/gmane.linux.kernel.crash-dump.crash-utility/4532 <pre>Hi, Starting from recently Arch uses -D_FORTIFY_SOURCE=2 flag for compilation that seems affects latest glibc. This flag causes 'crash tool' compilation error on Arch: /fibheap.c:265:7: warning: implicit declaration of function ‘abort’ [-Wimplicit-function-declaration] abort (); ^ ./fibheap.c:265:7: warning: incompatible implicit declaration of built-in function ‘abort’ [enabled by default] ./fibheap.c: In function ‘fibheap_delete’: ./fibheap.c:277:5: warning: incompatible implicit declaration of built-in function ‘free’ [enabled by default] free (fibheap_extr_min_node (heap)); ^ ./fibheap.c: In function ‘fibheap_consolidate’: ./fibheap.c:368:3: warning: implicit declaration of function ‘memset’ [-Wimplicit-function-declaration] memset (a, 0, sizeof (fibnode_t) * D); ^ ./fibheap.c:368:3: warning: incompatible implicit declaration of built-in function ‘memset’ [enabled by default] make[4]: *** [fibheap.o] Error 1 make[3]: *** [all-libiberty] Error 2 make[2]: *** [all] Error 2 gdb build failed: gdb-7.3.1/gdb/libgdb.a does not exist -- Crash-utility mailing list Crash-utility&lt; at &gt;redhat.com https://www.redhat.com/mailman/listinfo/crash-utility</pre> Anatol Pomozov 2013-04-11T18:42:53 http://comments.gmane.org/gmane.linux.kernel.crash-dump.crash-utility/4517 <pre>Hi Luc (and all crash developers on Cc), since libeppic is now used at least by two projects (crash and makedumpfile), I packaged it for openSUSE, but I cannot get it included in Factory because of versioning policies for snapshot releases. If I knew the planned target version, I could package the git snapshot as a pre-release, but currently I can't, because I don't know if you want to continue with the sial versioning scheme (last published was sial-3.0), or start again from 1.0 (or even 0.1?). Unless I'm missing something, I cannot see any public (versioned) release of libeppic right now. Please, can you advise? Thanks in advance, Petr Tesarik </pre> Petr Tesarik 2013-04-02T08:37:42 http://comments.gmane.org/gmane.linux.kernel.crash-dump.crash-utility/4510 <pre>Hi Dave, Please share your thoughts on adding a -m option to kmem to display the free pages per migrate type. This is similar to /proc/pagetypeinfo. Sample output (without formatting). Patch attached. crash&gt; Free pages count per migrate type at order [0-10]: Node 0, zone Normal, type Unmovable 155 172 92 39 20 8 10 15 7 3 1 Node 0, zone HighMem, type Unmovable 1 2 0 0 0 0 0 0 0 0 0 Node 0, zone Movable, type Unmovable 0 0 0 0 0 0 0 0 0 0 0 Node 0, zone Normal, type Reclaimable 9 3 0 0 1 1 0 0 0 0 0 Node 0, zone HighMem, type Reclaimable 0 0 0 0 0 0 0 0 0 0 0 Node 0, zone Movable, type Reclaimable 0 0 0 0 0 0 0 0 0 0 0 Node 0, zone Normal, type Movable 7 68 35 253 137 38 16 4 0 0 66 Node 0, zone HighMem, type Movable 0 1 0 0 0 0 0 0 0 0 0 Node 0, zone Movable, type Movable 0 0 0 0 0 0 0 0 0 0 0 Node 0, zone Normal, type Reserve 0 0 0 0 0 0 0 0 0 0 1 Node 0, zone HighMem, type Reserve 11 7 5 1 0 0 0 0 0 0 0 Node 0, zone Movable, type Reserve 0 0 0 0 0 0 0 0 0 0 0 Node 0, zone Normal, type Isolate 0 0 0 0 0 0 0 0 0 0 0 Node 0, zone HighMem, type Isolate 0 0 0 0 0 0 0 0 0 0 0 Node 0, zone Movable, type Isolate 0 0 0 0 0 0 0 0 0 0 0 diff --git a/crash-6.0.8/help.c b/crash-6.0.8/help.c index 6d6ac5e..9ad6c00 100755 --- a/crash-6.0.8/help.c +++ b/crash-6.0.8/help.c &lt; at &gt;&lt; at &gt; -5147,7 +5147,7 &lt; at &gt;&lt; at &gt; NULL char *help_kmem[] = { "kmem", "kernel memory", -"[-f|-F|-p|-c|-C|-i|-s|-S|-v|-V|-n|-z-o] [slab] [[-P] address]\n" +"[-f|-F|-p|-c|-C|-i|-s|-S|-v|-V|-n|-m|-z-o] [slab] [[-P] address]\n" " [-g [flags]]", " This command displays information about the use of kernel memory.\n", " -f displays the contents of the system free memory headers.", &lt; at &gt;&lt; at &gt; -5174,6 +5174,8 &lt; at &gt;&lt; at &gt; char *help_kmem[] = { " values to translate them into kernel virtual addresses.", " -g displays the enumerator value of all bits in the page structure's", " \"flags\" field.", +" -m displays the number of pages per migrate type for all orders, for all", +" nodes.", " flags when used with -g, translates all bits in this hexadecimal page", " structure flags value into its enumerator values.", " slab when used with -s or -S, limits the command to only the slab cache", &lt; at &gt;&lt; at &gt; -5605,6 +5607,24 &lt; at &gt;&lt; at &gt; char *help_kmem[] = { " PG_slab 7 0000080", " PG_head 14 0004000", " %s&gt;", +"\n Display pages per migrate type for all orders, for all nodes:\n", +" %s&gt; kmem -m", +" Free pages count per migrate type at order [0-10]:", +" Node 0, zone Normal, type Unmovable 155 172 92 39 20 8 10 15 7 3 1", +" Node 0, zone HighMem, type Unmovable 1 2 0 0 0 0 0 0 0 0 0", +" Node 0, zone Movable, type Unmovable 0 0 0 0 0 0 0 0 0 0 0", +" Node 0, zone Normal, type Reclaimable 9 3 0 0 1 1 0 0 0 0 0", +" Node 0, zone HighMem, type Reclaimable 0 0 0 0 0 0 0 0 0 0 0", +" Node 0, zone Movable, type Reclaimable 0 0 0 0 0 0 0 0 0 0 0", +" Node 0, zone Normal, type Movable 7 68 35 253 137 38 16 4 0 0 66", +" Node 0, zone HighMem, type Movable 0 1 0 0 0 0 0 0 0 0 0", +" Node 0, zone Movable, type Movable 0 0 0 0 0 0 0 0 0 0 0", +" Node 0, zone Normal, type Reserve 0 0 0 0 0 0 0 0 0 0 1", +" Node 0, zone HighMem, type Reserve 11 7 5 1 0 0 0 0 0 0 0", +" Node 0, zone Movable, type Reserve 0 0 0 0 0 0 0 0 0 0 0", +" Node 0, zone Normal, type Isolate 0 0 0 0 0 0 0 0 0 0 0", +" Node 0, zone HighMem, type Isolate 0 0 0 0 0 0 0 0 0 0 0", +" Node 0, zone Movable, type Isolate 0 0 0 0 0 0 0 0 0 0 0", NULL }; diff --git a/crash-6.0.8/memory.c b/crash-6.0.8/memory.c index 02a6de1..e03db84 100755 --- a/crash-6.0.8/memory.c +++ b/crash-6.0.8/memory.c &lt; at &gt;&lt; at &gt; -264,6 +264,7 &lt; at &gt;&lt; at &gt; static int verify_pfn(ulong); static void dump_per_cpu_offsets(void); static void dump_page_flags(ulonglong); static ulong kmem_cache_nodelists(ulong); +static void dump_pgtype_info(void); /* * Memory display modes specific to this file. &lt; at &gt;&lt; at &gt; -4044,6 +4045,8 &lt; at &gt;&lt; at &gt; get_task_mem_usage(ulong task, struct task_mem_usage *tm) * -c displays the number of pages in the page_hash_table. * -C displays all entries in the page_hash_table. * -i displays informational data shown by /proc/meminfo. + * -m displays information on the number of pages per migrate type + * for all orders, for all nodes. * * -P forces address to be defined as a physical address * address when used with -f, the address can be either a page pointer &lt; at &gt;&lt; at &gt; -4090,7 +4093,7 &lt; at &gt;&lt; at &gt; cmd_kmem(void) int i; int c; int sflag, Sflag, pflag, fflag, Fflag, vflag, zflag, oflag, gflag; -int nflag, cflag, Cflag, iflag, lflag, Lflag, Pflag, Vflag; +int nflag, cflag, Cflag, iflag, lflag, Lflag, Pflag, Vflag, mflag; struct meminfo meminfo; ulonglong value[MAXARGS]; char buf[BUFSIZE]; &lt; at &gt;&lt; at &gt; -4100,12 +4103,12 &lt; at &gt;&lt; at &gt; cmd_kmem(void) spec_addr = 0; sflag =Sflag = pflag = fflag = Fflag = Pflag = zflag = oflag = 0; vflag = Cflag = cflag = iflag = nflag = lflag = Lflag = Vflag = 0; -gflag = 0; +gflag = mflag = 0; escape = FALSE; BZERO(&amp;meminfo, sizeof(struct meminfo)); BZERO(&amp;value[0], sizeof(ulonglong)*MAXARGS); - while ((c = getopt(argcnt, args, "gI:sSFfpvczCinl:L:PVo")) != EOF) { + while ((c = getopt(argcnt, args, "gI:sSFfpvczCinml:L:PVo")) != EOF) { switch(c) { case 'V': &lt; at &gt;&lt; at &gt; -4206,6 +4209,10 &lt; at &gt;&lt; at &gt; cmd_kmem(void) gflag = 1; break; +case 'm': +mflag = 1; +break; + default: argerrs++; break; &lt; at &gt;&lt; at &gt; -4215,8 +4222,8 &lt; at &gt;&lt; at &gt; cmd_kmem(void) if (argerrs) cmd_usage(pc-&gt;curcmd, SYNOPSIS); - if ((sflag + Sflag + pflag + fflag + Fflag + Vflag + oflag + - vflag + Cflag + cflag + iflag + lflag + Lflag + gflag) &gt; 1) { + if ((sflag + Sflag + pflag + fflag + Fflag + Vflag + oflag + vflag + +Cflag + cflag + iflag + lflag + Lflag + gflag + mflag) &gt; 1) { error(INFO, "only one flag allowed!\n"); cmd_usage(pc-&gt;curcmd, SYNOPSIS); } &lt; at &gt;&lt; at &gt; -4444,9 +4451,12 &lt; at &gt;&lt; at &gt; cmd_kmem(void) if (gflag == 1) dump_page_flags(0); +if (mflag == 1) +dump_pgtype_info(); + if (!(sflag + Sflag + pflag + fflag + Fflag + vflag + Vflag + zflag + oflag + cflag + Cflag + iflag + - nflag + lflag + Lflag + gflag + meminfo.calls)) + nflag + lflag + Lflag + gflag + mflag + meminfo.calls)) cmd_usage(pc-&gt;curcmd, SYNOPSIS); } &lt; at &gt;&lt; at &gt; -7051,6 +7061,163 &lt; at &gt;&lt; at &gt; bailout: return total_free; } +static void dump_pgtype_info(void) +{ +int n, m, z, o; +int list_count = 0; +int free_cnt = 0; +int mtype_sym = 0; +int mtype_len = 0; +ulong *mtypes; +ulong node_zones; +ulong temp; +ulong freelist; +ulong *free_ptr; +char *free_list_buf; +char name_buf[BUFSIZE]; +char buf[BUFSIZE]; +struct node_table *nt; +struct list_data list_data; + +if (!(vt-&gt;flags &amp; (NODES|ZONES))) +error(FATAL, +"dump_pgtype_info called without (NODES|ZONES)\n"); + +if (!VALID_STRUCT(zone)) +error(FATAL, +"zone struct not available in this kernel\n"); + +if (VALID_STRUCT(free_area)) { +if (SIZE(free_area) == (3 * sizeof(ulong))) +error(FATAL, +"free_area type not supported by command\n"); +else +list_count = MEMBER_SIZE("free_area", +"free_list")/SIZE(list_head); +} else +error(FATAL, +"free_area structure not found\n"); + +free_list_buf = GETBUF(SIZE(list_head)); + +do { +if (symbol_exists("migratetype_names") &amp;&amp; +(get_symbol_type("migratetype_names", + NULL, NULL) == TYPE_CODE_ARRAY)) { + +open_tmpfile(); +sprintf(buf, "whatis migratetype_names"); +if (!gdb_pass_through(buf, fp, GNU_RETURN_ON_ERROR)) { +close_tmpfile(); +break; +} + +rewind(pc-&gt;tmpfile); +while (fgets(buf, BUFSIZE, pc-&gt;tmpfile)) { +if (STRNEQ(buf, "type = ")) +break; +} +close_tmpfile(); + +if (!strstr(buf, "char *") || +(count_chars(buf, '[') != 1) || +(count_chars(buf, ']') != 1)) +break; + +mtype_len = get_array_length("migratetype_names", +NULL, 0); + +mtypes = (ulong *)GETBUF(mtype_len * sizeof(ulong)); + +readmem(symbol_value("migratetype_names"), +KVADDR, mtypes, +(mtype_len * sizeof(ulong)), +NULL, FAULT_ON_ERROR); + +mtype_sym = 1; +} +} while (0); + +fprintf(fp, "%-43s [%d-%d]:", +"Free pages count per migrate type at order", +0, vt-&gt;nr_free_areas - 1); + +fprintf(fp, "\n"); + +for (n = 0; n &lt; vt-&gt;numnodes; n++) { +nt = &amp;vt-&gt;node_table[n]; +node_zones = nt-&gt;pgdat + OFFSET(pglist_data_node_zones); + +for (m = 0; m &lt; list_count; m++) { + +for (z = 0; z &lt; vt-&gt;nr_zones; z++) { +readmem((node_zones + (z * SIZE(zone))) ++ OFFSET(zone_name), KVADDR, &amp;temp, +sizeof(void *), "node_zones name", +FAULT_ON_ERROR); +read_string(temp, name_buf, BUFSIZE-1); + +fprintf(fp, "Node %4d, ", nt-&gt;node_id); +fprintf(fp, "zone %8s, ", name_buf); + +if (mtype_sym) { +read_string(mtypes[m], +name_buf, BUFSIZE-1); +fprintf(fp, "type %12s ", name_buf); +} else +fprintf(fp, "type %12d ", m); + +for (o = 0; o &lt; vt-&gt;nr_free_areas; o++) { +freelist = +(node_zones + (z * SIZE(zone))) ++ (OFFSET(zone_free_area) + +(o * SIZE(free_area))) + +(m * SIZE(list_head)); + +readmem(freelist, KVADDR, free_list_buf, +SIZE(list_head), +"free_area free_list", +FAULT_ON_ERROR); + +free_ptr = (ulong *)free_list_buf; + +if (!(*free_ptr) || +(*free_ptr == freelist)) { +fprintf(fp, "%6lu ", 0); +continue; +} + +BZERO(&amp;list_data, +sizeof(struct list_data)); +list_data.flags = RETURN_ON_DUPLICATE; +list_data.start = *free_ptr; +list_data.end = freelist; +list_data.list_head_offset = +OFFSET(page_lru) + +OFFSET(list_head_next); + +free_cnt = do_list(&amp;list_data); +if (free_cnt &lt; 0) { +error(pc-&gt;curcmd_flags &amp; +IGNORE_ERRORS ? INFO : FATAL, +"corrupted free list\n"); +free_cnt = 0; +} + +fprintf(fp, "%6lu ", free_cnt); +} +fprintf(fp, "\n"); +} +} +node_zones + OFFSET(zone_free_area); +} + +FREEBUF(free_list_buf); + +if (mtype_sym) +FREEBUF(mtypes); +} + /* * dump_kmeminfo displays basic memory use information typically shown * by /proc/meminfo, and then some... </pre> vinayak menon 2013-03-29T14:13:21 http://comments.gmane.org/gmane.linux.kernel.crash-dump.crash-utility/4509 <pre>Hi Dave, The newly modification of gdb involved some bug. When using mount, crash prints the following message. &lt;cut&gt; crash&gt; mount mount: invalid structure member offset: mount_mnt_devname FILE: filesys.c LINE: 1396 FUNCTION: show_mounts() [./crash] error trace: 45dbf7 =&gt; 49dc02 =&gt; 49af31 =&gt; 502cdd MOUNT SUPERBLK TYPE DEVNAME DIRNAME 502cdd: OFFSET_verify+189 49af31: show_mounts+785 49dc02: cmd_mount+1234 45dbf7: exec_command+919 mount: invalid structure member offset: mount_mnt_devname FILE: filesys.c LINE: 1396 FUNCTION: show_mounts() crash&gt; &lt;cut&gt; Then I try to find what caused the bug. I found a wrong number is returned when using "STRUCT_SIZE_INIT(mount, "mount");". and then I referred to the changelog to see what had happened, and I guess from changelog, the bug is related to &lt;cut&gt; - Patch to the internal gdb_get_datatype() function to return the typecode and length of integer variables. (adrian.wenl&lt; at &gt;gmail.com, anderson&lt; at &gt;redhat.com) &lt;cut&gt; So I made the patch to fix it. -- -- Regards Qiao Nuohan </pre> qiaonuohan 2013-03-29T10:10:38 http://comments.gmane.org/gmane.linux.kernel.crash-dump.crash-utility/4506 <pre> Download from: http://people.redhat.com/anderson Changelog: - Fix for the ARM "irq" command. Without the patch, on 2.6.34 and later kernels configured with CONFIG_SPARSE_IRQ, the command fails with the error message "irq: cannot determine number of IRQs". (Jan.Karlsson&lt; at &gt;sonymobile.com) - Fix for a segmentation violation generated during invocation while parsing a makedumpfile-created "flat-format" vmcore-incomplete file. Without the patch, the crash session would display the error message "crash: unable to seek dump file vmcore-incomplete", followed by a segmentation violation. (anderson&lt; at &gt;redhat.com) - Fix for a segmentation violation generated by the "kmem -s" option when encountering a corrupted array_cache structure that contains a bogus "avail" count that is greater than the maximum legitimate limit value. Without the patch, the "kmem -s" command would print a warning message regarding the invalid array_cache, complete the command normally, and then generate a segmentation violation when freeing buffers used by the command. (anderson&lt; at &gt;redhat.com) - Update to the "kmem -s" function to include the errors found in slab structures to the display of total errors found when the command completes. Without the patch, invalid list_head pointers, bad inuse counters, and bad s_mem pointers were not added to the total number of errors found. (anderson&lt; at &gt;redhat.com) - Fix for "crash --osrelease &lt;dumpfile&gt;" and "crash --log &lt;dumpfile&gt;" when run on an ARM compressed kdump with a crash binary that was built with "make target=ARM" on an x86 or x86_64 host. Without the patch, if the compressed kdump header version is 4 or 5, "crash --osrelease" fails with the error message "crash: compressed kdump: cannot lseek dump vmcoreinfo" followed by "unknown", and "crash --log" fails with the error message "crash: &lt;dumpfile&gt;: no VMCOREINFO section". (anderson&lt; at &gt;redhat.com) - Enhancement to the "swap" command to display the swap_info_struct address of each configured swap device. The output has been changed to display the address in the first column, and the variable-length device name has been moved to the last column. (anderson&lt; at &gt;redhat.com) - Fix for the "kmem -[sS]" options on kernels that configured with both CONFIG_SLUB and CONFIG_NODES_SHIFT, and that are running on hardware that generates NUMA nodes that contain no memory. Without the patch, both command options fail immediately with the message "kmem: invalid kernel virtual address: 8 type: kmem_cache_node nr_partial". (anderson&lt; at &gt;redhat.com) - Increment the PPC64 NR_CPUS maximum value from 1024 to 2048. (anderson&lt; at &gt;redhat.com) - Strip the ".isra." and ".part." appendages to cloned text symbol names, which seem to have been introduced by gcc-4.6.0. To keep them intact, a "--no_strip" command line option has been added. (anderson&lt; at &gt;redhat.com) - Patch to the internal gdb_get_datatype() function to return the typecode and length of integer variables. (adrian.wenl&lt; at &gt;gmail.com, anderson&lt; at &gt;redhat.com) - Fix for the "dev -d" option on Linux 3.6 and later kernels. Without the patch the option fails with the message "dev: invalid structure member offset: request_queue_rq". (holzheu&lt; at &gt;linux.vnet.ibm.com) - Export the red/black tree utility functions rb_first(), rb_parent(), rb_right(), rb_left(), rp_next() and rb_last(). Without the patch, they are statically declared and only used by the "runq" command. (qiaonuohan&lt; at &gt;cn.fujitsu.com) - Implemented a new "timer -r" option that displays the hrtimer queues, supporting all versions from Linux 2.6.16 to the present. (qiaonuohan&lt; at &gt;cn.fujitsu.com, anderson&lt; at &gt;redhat.com) - Fix for "kmem -s" on Linux 3.8 and later kernels that are configured with CONFIG_SLAB. The kmem_cache.array[] length has been extended to store the nodelist pointers, so the original method to determine the per-cpu array limit can go out-of-range. Without the patch, during session initialization there may be a message that indicates "crash: invalid kernel virtual address: &lt;address&gt; type: array cache limit", followed by "crash: unable to initialize kmem slab cache subsystem"; if those messages do get shown, then "kmem -s" will subsequently fail during runtime with the message "kmem: kmem cache slab subsystem not available". (qiaonuohan&lt; at &gt;cn.fujitsu.com) - Two Xen hypervisor fixes: (1) Fix console buffer content length calculation: Xen changeset 26447 (x86: re-introduce map_domain_page() et al) once again altered virtual address space. The current algorithm calculating its start could not cope with that change. New version establishes this value on the base of image start address and is more generic. (2) Improve calculation of beginning of virtual address space: Function displaying console buffer always assumes its content length equal to console buffer size. This is not true and sometimes it sends garbage to the screen. This patch fixes this issue. (daniel.kiper&lt; at &gt;oracle.com) - Fix for the ARM "vtop" command when run on a module address. Without the patch, the command fails with error message "vtop: ambiguous address: &lt;module-address&gt; (requires -u or -k)". (anderson&lt; at &gt;redhat.com) - Add the "--active" command line option to the crash(8) man page and to the "crash [-h|--help]" output. (anderson&lt; at &gt;redhat.com) - Add the "--buildinfo" command line option to the crash(8) man page and to the "crash [-h|--help]" output. (anderson&lt; at &gt;redhat.com) - Remove the unadvertised and unnecessary "--data_debug" command line option, given that it is the default setting. (anderson&lt; at &gt;redhat.com) - Remove the unadvertised and obsolete "--no_namelist_gzip" command line option. (anderson&lt; at &gt;redhat.com) - Add the "-g [namelist]" command line option to the crash(8) man page and to the "crash [-h|--help]" output. (anderson&lt; at &gt;redhat.com) - Remove the unadvertised and never-implemented "--shadow_page_tables" command line option. (anderson&lt; at &gt;redhat.com) - Fix for the ARM "vtop" command when run on a user virtual address of the panic task. Prior to Linux 3.3, the panic task's pgd gets overwritten with a pgd that identity-maps the whole address space, and therefore crash loses the capability of translating any user virtual address into its original physical address. (mika.westerberg&lt; at &gt;iki.fi) - Fix to prevent the ARM linker mapping symbols "$d" and "$a" from being added to the list of symbols from kernel modules. Without the patch, the two symbols would only be rejected from the base kernel's symbol list, but would be added to the symbol list of individual kernel modules. (mika.westerberg&lt; at &gt;iki.fi) - Fix for the X86_64 "bt" command to recognize that the kernel was built with CONFIG_FRAME_POINTER on Linux 3.7 and later kernels that are configured with CONFIG_FUNCTION_TRACER. In those kernels, the special 4-byte NOP instruction that can be overwritten during runtime for dynamic ftracing has been moved to the very beginning of each function, before the function preamble. Without the patch, the test that checks the function preamble to determine whether CONFIG_FRAME_POINTER was configured would fail, which could potentially lead to less reliable backtraces. (anderson&lt; at &gt;redhat.com) </pre> Dave Anderson 2013-03-28T17:48:33 http://comments.gmane.org/gmane.linux.kernel.crash-dump.crash-utility/4497 <pre> I noticed that on newer kernels, the "dev -d" command is failing. Seems like the 'rq' field in the request_queue structure is now called 'root_rl'. The little change in the patch below seems to fix this problem, at least for me on i686 &amp; x86_64. (I was using crash 6.1.4.) crash&gt; dev -d MAJOR GENDISK NAME REQUEST QUEUE TOTAL ASYNC SYNC DRV dev: invalid structure member offset: request_queue_rq FILE: dev.c LINE: 3807 FUNCTION: get_diskio_1() [/sbin/crash] error trace: 45dd17 =&gt; 4d6432 =&gt; 4d5eba =&gt; 4ff0fd 4ff0fd: OFFSET_verify+189 4d5eba: get_diskio_1+58 4d6432: display_all_diskio+1090 45dd17: exec_command+919 dev: invalid structure member offset: request_queue_rq FILE: dev.c LINE: 3807 FUNCTION: get_diskio_1() Index: b/dev.c =================================================================== --- a/dev.c +++ b/dev.c &lt; at &gt;&lt; at &gt; -4051,6 +4051,9 &lt; at &gt;&lt; at &gt; void diskio_init(void) MEMBER_OFFSET_INIT(request_queue_in_flight, "request_queue", "in_flight"); MEMBER_OFFSET_INIT(request_queue_rq, "request_queue", "rq"); +if (INVALID_MEMBER(request_queue_rq)) +MEMBER_OFFSET_INIT(request_queue_rq, +"request_queue", "root_rl"); MEMBER_OFFSET_INIT(subsys_private_klist_devices, "subsys_private", "klist_devices"); MEMBER_OFFSET_INIT(subsystem_kset, "subsystem", "kset"); </pre> John Blackwood 2013-03-27T16:09:36 http://comments.gmane.org/gmane.linux.kernel.crash-dump.crash-utility/4492 <pre>kmem -m|-M is used for displaying information of all ksm pages or some ksm pages for specified ksm stable tree node addresses for example: crash&gt; kmem -m STABLE_NODE : ffff88083fc927e0 PAGE : ffffea000e667998 PHYSICAL ADDRESS: 41d475000 PID: 2967 MAPPING: 3 STABLE_NODE : ffff88083fc84a10 PAGE : ffffea000e3dd5d8 PHYSICAL ADDRESS: 411aad000 PID: 2967 MAPPING: 7 STABLE_NODE : ffff88041980dda8 PAGE : ffffea000e335568 PHYSICAL ADDRESS: 40eaab000 PID: 2967 MAPPING: 8 ... STABLE_NODE : ffff880841ea43f8 PAGE : ffffea000f62de38 PHYSICAL ADDRESS: 465641000 PID: 2967 MAPPING: 729 PID: 3017 MAPPING: 499 P.S. This patch is based on the patch from Qiao(qiaonuohan&lt; at &gt;cn.fujitsu.com) 0001-make-rbtree-manipulation-functions-global.patch Because this patch also uses rb_tree operations. Signed-off-by: Zhang Yanfei &lt;zhangyanfei&lt; at &gt;cn.fujitsu.com&gt; --- defs.h | 6 ++ help.c | 41 +++++++++++- memory.c | 209 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-- symbols.c | 12 ++++ 4 files changed, 259 insertions(+), 9 deletions(-) diff --git a/defs.h b/defs.h index 2993f2b..f6996be 100755 --- a/defs.h +++ b/defs.h &lt; at &gt;&lt; at &gt; -1848,6 +1848,12 &lt; at &gt;&lt; at &gt; struct offset_table { /* stash of commonly-used offsets */ long vmap_area_list; long vmap_area_flags; long vmap_area_vm; +long stable_node_node; +long stable_node_hlist; +long stable_node_kpfn; +long rmap_item_mm; +long rmap_item_address; +long rmap_item_hlist; }; struct size_table { /* stash of commonly-used sizes */ diff --git a/help.c b/help.c index c542743..38a3341 100755 --- a/help.c +++ b/help.c &lt; at &gt;&lt; at &gt; -5147,8 +5147,8 &lt; at &gt;&lt; at &gt; NULL char *help_kmem[] = { "kmem", "kernel memory", -"[-f|-F|-p|-c|-C|-i|-s|-S|-v|-V|-n|-z-o] [slab] [[-P] address]\n" -" [-g [flags]]", +"[-f|-F|-p|-c|-C|-i|-s|-S|-v|-V|-n|-z|-o|-m|-M] [slab] [[-P] address]\n" +" [-g [flags]", " This command displays information about the use of kernel memory.\n", " -f displays the contents of the system free memory headers.", " also verifies that the page count equals nr_free_pages.", &lt; at &gt;&lt; at &gt; -5172,6 +5172,9 &lt; at &gt;&lt; at &gt; char *help_kmem[] = { " -z displays per-zone memory statistics.", " -o displays each cpu's offset value that is added to per-cpu symbol", " values to translate them into kernel virtual addresses.", +" -m displays information of ksm pages.", +" -M same as -m, but also dumps virtual addresses that mapping the", +" ksm pages.", " -g displays the enumerator value of all bits in the page structure's", " \"flags\" field.", " flags when used with -g, translates all bits in this hexadecimal page", &lt; at &gt;&lt; at &gt; -5204,7 +5207,10 &lt; at &gt;&lt; at &gt; char *help_kmem[] = { " the page address is displayed if it is contained with the list.", " address when used with -v, the address can be a mapped kernel virtual", " address or physical address; the mapped region containing the", -" address is displayed.\n", +" address is displayed.", +" address when used with -m or -M, the address can be either a ksm stable", +" tree node address, a page's physical address, or a page pointer,", +" the information of the ksm page (if it is) is displayed.\n", " All address arguments above must be expressed in hexadecimal format.", "\nEXAMPLES", " Display memory usage information:\n", &lt; at &gt;&lt; at &gt; -5605,6 +5611,35 &lt; at &gt;&lt; at &gt; char *help_kmem[] = { " PG_slab 7 0000080", " PG_head 14 0004000", " %s&gt;", +"\n Display information of ksm pages:\n", +" %s&gt; kmem -m ffff88086f22eec0 ffff8803573964c0", +" STABLE_NODE : ffff88083fc927e0", +" PAGE : ffffea000e667998", +" PHYSICAL ADDRESS: 41d475000", +"", +" PID: 2967 MAPPING: 3", +"", +" STABLE_NODE : ffff88083fc84a10", +" PAGE : ffffea000e3dd5d8", +" PHYSICAL ADDRESS: 411aad000", +"", +" PID: 2967 MAPPING: 7", +"", +" STABLE_NODE : ffff88041980dda8", +" PAGE : ffffea000e335568", +" PHYSICAL ADDRESS: 40eaab000", +"", +" PID: 2967 MAPPING: 8", +"", +" ......", +"", +" STABLE_NODE : ffff880841ea43f8", +" PAGE : ffffea000f62de38", +" PHYSICAL ADDRESS: 465641000", +"", +" PID: 2967 MAPPING: 729", +" PID: 3017 MAPPING: 499", +"", NULL }; diff --git a/memory.c b/memory.c index 02a6de1..9623949 100755 --- a/memory.c +++ b/memory.c &lt; at &gt;&lt; at &gt; -264,6 +264,7 &lt; at &gt;&lt; at &gt; static int verify_pfn(ulong); static void dump_per_cpu_offsets(void); static void dump_page_flags(ulonglong); static ulong kmem_cache_nodelists(ulong); +static void dump_ksm(struct meminfo *); /* * Memory display modes specific to this file. &lt; at &gt;&lt; at &gt; -1012,6 +1013,16 &lt; at &gt;&lt; at &gt; vm_init(void) PG_reserved_flag_init(); PG_slab_flag_init(); +if (STRUCT_EXISTS("stable_node")) { +MEMBER_OFFSET_INIT(stable_node_node, "stable_node", "node"); +MEMBER_OFFSET_INIT(stable_node_hlist, "stable_node", "hlist"); +MEMBER_OFFSET_INIT(stable_node_kpfn, "stable_node", "kpfn"); +MEMBER_OFFSET_INIT(stable_node_node, "stable_node", "node"); +MEMBER_OFFSET_INIT(rmap_item_mm, "rmap_item", "mm"); +MEMBER_OFFSET_INIT(rmap_item_address, "rmap_item", "address"); +ANON_MEMBER_OFFSET_INIT(rmap_item_hlist, "rmap_item", "hlist"); +} + vt-&gt;flags |= VM_INIT; } &lt; at &gt;&lt; at &gt; -4090,22 +4101,25 &lt; at &gt;&lt; at &gt; cmd_kmem(void) int i; int c; int sflag, Sflag, pflag, fflag, Fflag, vflag, zflag, oflag, gflag; -int nflag, cflag, Cflag, iflag, lflag, Lflag, Pflag, Vflag; +int nflag, cflag, Cflag, iflag, lflag, Lflag, Pflag, Vflag, mflag; +int Mflag; struct meminfo meminfo; ulonglong value[MAXARGS]; char buf[BUFSIZE]; char *p1; int spec_addr, escape; +ulong ksm_pages_shared; spec_addr = 0; sflag =Sflag = pflag = fflag = Fflag = Pflag = zflag = oflag = 0; vflag = Cflag = cflag = iflag = nflag = lflag = Lflag = Vflag = 0; -gflag = 0; +gflag = mflag = Mflag = 0; escape = FALSE; +ksm_pages_shared = 0; BZERO(&amp;meminfo, sizeof(struct meminfo)); BZERO(&amp;value[0], sizeof(ulonglong)*MAXARGS); - while ((c = getopt(argcnt, args, "gI:sSFfpvczCinl:L:PVo")) != EOF) { + while ((c = getopt(argcnt, args, "gI:sSFfpvczCinl:L:PVomM")) != EOF) { switch(c) { case 'V': &lt; at &gt;&lt; at &gt; -4206,6 +4220,18 &lt; at &gt;&lt; at &gt; cmd_kmem(void) gflag = 1; break; +case 'm': +if (INVALID_MEMBER(stable_node_node)) +option_not_supported(c); +mflag = 1; +break; + +case 'M': +if (INVALID_MEMBER(stable_node_node)) +option_not_supported(c); +Mflag = 1; +break; + default: argerrs++; break; &lt; at &gt;&lt; at &gt; -4216,7 +4242,8 &lt; at &gt;&lt; at &gt; cmd_kmem(void) cmd_usage(pc-&gt;curcmd, SYNOPSIS); if ((sflag + Sflag + pflag + fflag + Fflag + Vflag + oflag + - vflag + Cflag + cflag + iflag + lflag + Lflag + gflag) &gt; 1) { + vflag + Cflag + cflag + iflag + lflag + Lflag + gflag + + mflag + Mflag) &gt; 1) { error(INFO, "only one flag allowed!\n"); cmd_usage(pc-&gt;curcmd, SYNOPSIS); } &lt; at &gt;&lt; at &gt; -4224,6 +4251,15 &lt; at &gt;&lt; at &gt; cmd_kmem(void) if (sflag || Sflag || !(vt-&gt;flags &amp; KMEM_CACHE_INIT)) kmem_cache_init(); +if (mflag || Mflag) { +get_symbol_data("ksm_pages_shared", sizeof(ulong), +&amp;ksm_pages_shared); +if (!ksm_pages_shared) { +fprintf(fp, "ksm may not be enabled\n"); +return; +} +} + while (args[optind]) { if (hexadecimal(args[optind], 0)) { value[spec_addr++] = &lt; at &gt;&lt; at &gt; -4347,6 +4383,18 &lt; at &gt;&lt; at &gt; cmd_kmem(void) gflag++; } +if (mflag || Mflag) { +meminfo.spec_addr = value[i]; +meminfo.flags = ADDRESS_SPECIFIED; +if (Mflag) +meminfo.flags |= VERBOSE; +dump_ksm(&amp;meminfo); +if (mflag) +mflag++; +if (Mflag) +Mflag++; +} + /* * no value arguments allowed! */ &lt; at &gt;&lt; at &gt; -4358,7 +4406,7 &lt; at &gt;&lt; at &gt; cmd_kmem(void) } if (!(sflag + Sflag + pflag + fflag + vflag + cflag + - lflag + Lflag + gflag)) { + lflag + Lflag + gflag + mflag + Mflag)) { meminfo.spec_addr = value[i]; meminfo.flags = ADDRESS_SPECIFIED; if (meminfo.calls++) &lt; at &gt;&lt; at &gt; -4444,9 +4492,18 &lt; at &gt;&lt; at &gt; cmd_kmem(void) if (gflag == 1) dump_page_flags(0); +if (mflag == 1) +dump_ksm(NULL); + +if (Mflag == 1) { +meminfo.flags |= VERBOSE; +dump_ksm(&amp;meminfo); +} + if (!(sflag + Sflag + pflag + fflag + Fflag + vflag + Vflag + zflag + oflag + cflag + Cflag + iflag + - nflag + lflag + Lflag + gflag + meminfo.calls)) + nflag + lflag + Lflag + gflag + mflag + Mflag + + meminfo.calls)) cmd_usage(pc-&gt;curcmd, SYNOPSIS); } &lt; at &gt;&lt; at &gt; -15799,6 +15856,146 &lt; at &gt;&lt; at &gt; dump_page_flags(ulonglong flags) close_tmpfile(); } +struct page_ref { +ulong mm; +ulong pid; +int ref; +}; + +/* + * dump_ksm() displays information of ksm pages. + */ +static void +dump_ksm(struct meminfo *mi) +{ +ulong root_stable_tree, stable_node, kpfn; +ulong rmap_item, mm, paddr; +struct rb_root *root; +struct rb_node *node; +ulong first, next; +struct task_context *tc; +int i, ref_size, refs, found; +struct page_ref *ref; +ulong page, address; + +if (!symbol_exists("root_stable_tree")) { +error(INFO, "cannot determine ksm stable tree address from root_stable_tree\n"); +return; +} +root_stable_tree = symbol_value("root_stable_tree"); +root = (struct rb_root *)root_stable_tree; + +refs = 0; +ref_size = sizeof(struct page_ref) * RUNNING_TASKS(); +ref = (struct page_ref *)GETBUF(ref_size); +BZERO(ref, ref_size); + +found = (mi &amp;&amp; mi-&gt;flags &amp; ADDRESS_SPECIFIED) ? 0 : -1; +for (node = rb_first(root); node; node = rb_next(node)) { +stable_node = (ulong) node - OFFSET(stable_node_node); +if (CRASHDEBUG(1)) +fprintf(fp, " stable_node = %lx\n", stable_node); + +readmem(stable_node + OFFSET(stable_node_hlist), +KVADDR, &amp;first, sizeof(ulong), +"stable_node hlist", FAULT_ON_ERROR); +readmem(stable_node + OFFSET(stable_node_kpfn), +KVADDR, &amp;kpfn, sizeof(ulong), +"stable_node kpfn", FAULT_ON_ERROR); +paddr = kpfn &lt;&lt; PAGE_SHIFT; +phys_to_page(paddr, &amp;page); + +if (found == 0) { +if ((mi-&gt;memtype == KVADDR) &amp;&amp; + (((mi-&gt;spec_addr &amp; ~0x3) == stable_node) || + (mi-&gt;spec_addr == page))) +found = 1; +if ((mi-&gt;memtype == PHYSADDR) &amp;&amp; + (mi-&gt;spec_addr == paddr)) +found = 1; +} +if (found == 0) +continue; + +fprintf(fp, "STABLE_NODE : %lx\n", stable_node); +fprintf(fp, "PAGE : %lx\n", page); +fprintf(fp, "PHYSICAL ADDRESS: %lx\n\n", paddr); + +readmem(stable_node + OFFSET(stable_node_hlist), +KVADDR, &amp;first, sizeof(ulong), +"stable_node hlist", FAULT_ON_ERROR); + +next = first; +while (next) { +rmap_item = next - OFFSET(rmap_item_hlist); +readmem(rmap_item + OFFSET(rmap_item_mm), +KVADDR, &amp;mm, sizeof(ulong), +"rmap_item mm", FAULT_ON_ERROR); + +for (i = 0; i &lt; refs; i++) { +if (ref[i].mm == mm) { +ref[i].ref += 1; +goto next; +} +} + +tc = FIRST_CONTEXT(); +for (i = 0; i &lt; RUNNING_TASKS(); i++, tc++) { +if (tc-&gt;mm_struct == mm) { +ref[refs].mm = mm; +ref[refs].pid = tc-&gt;pid; +ref[refs++].ref = 1; +break; +} +} + +next: +readmem(next + OFFSET(hlist_node_next), +KVADDR, &amp;next, sizeof(ulong), +"hlist_node next", FAULT_ON_ERROR); +}; + +for (i = 0; i &lt; refs; i++) { +fprintf(fp, " PID: %ld ", ref[i].pid); +fprintf(fp, "MAPPING: %d\n", ref[i].ref); + +if (!(mi &amp;&amp; mi-&gt;flags &amp; VERBOSE)) +continue; + +fprintf(fp, " VIRTUAL:\n"); +next = first; +while (next) { +rmap_item = next - OFFSET(rmap_item_hlist); +readmem(rmap_item + OFFSET(rmap_item_mm), +KVADDR, &amp;mm, sizeof(ulong), +"rmap_item mm", FAULT_ON_ERROR); +if (ref[i].mm == mm) { +readmem(rmap_item + OFFSET(rmap_item_address), +KVADDR, &amp;address, sizeof(ulong), +"rmap_item address", FAULT_ON_ERROR); +fprintf(fp, " %lx\n", +PAGEBASE(address)); +} +readmem(next + OFFSET(hlist_node_next), +KVADDR, &amp;next, sizeof(ulong), +"hlist_node next", FAULT_ON_ERROR); +} +fprintf(fp, "\n"); +} +if (!(mi &amp;&amp; mi-&gt;flags &amp; VERBOSE)) +fprintf(fp, "\n"); +refs = 0; + +if (found == 1) +break; +} + +if (found == 0) +fprintf(fp, "address 0x%llx cannot specify a ksm stable tree node\n", +mi-&gt;spec_addr); + +FREEBUF(ref); +} /* * Support for slub.c slab cache. diff --git a/symbols.c b/symbols.c index 4fb397c..6c730ad 100755 --- a/symbols.c +++ b/symbols.c &lt; at &gt;&lt; at &gt; -8860,6 +8860,18 &lt; at &gt;&lt; at &gt; dump_offset_table(char *spec, ulong makestruct) OFFSET(rt_rq_highest_prio)); fprintf(fp, " rt_rq_rt_nr_running: %ld\n", OFFSET(rt_rq_rt_nr_running)); +fprintf(fp, " stable_node_node: %ld\n", +OFFSET(stable_node_node)); +fprintf(fp, " stable_node_hlist: %ld\n", +OFFSET(stable_node_hlist)); +fprintf(fp, " stable_node_kpfn: %ld\n", +OFFSET(stable_node_kpfn)); +fprintf(fp, " rmap_item_mm: %ld\n", +OFFSET(rmap_item_mm)); +fprintf(fp, " rmap_item_address: %ld\n", +OFFSET(rmap_item_address)); +fprintf(fp, " rmap_item_hlist: %ld\n", +OFFSET(rmap_item_hlist)); fprintf(fp, "\n size_table:\n"); fprintf(fp, " page: %ld\n", SIZE(page)); </pre> Zhang Yanfei 2013-03-27T02:30:39 http://comments.gmane.org/gmane.linux.kernel.crash-dump.crash-utility/4489 <pre>Hello, I'm working now on task of obtaining function parameters from stack frames. Can't say that it's really possible to get every parameter of every function, but some of them are available. My idea was the following: at start we have "reliable" register RSP, after that I'm starting to parse frame: push %rbp mov %rsp,%rbp push %r13 push %r12 push %rbx hence we have rbx, r12, r13 "reliable" registers and so on. At the end of frame we have the following: mov %r13,%rcx mov %rax,%rdx mov %r12,%rsi mov %rbx,%rdi callq *%r8 Source is callee-save register, destination is parameter register. Then the next frame: mov %rbx, -0x28(%rbp) mov %r12, -0x20(%rbp) mov %r13, -0x18(%rbp) mov %r14, -0x10(%rbp) and we have 3 of 4 parameters in stack. (RDI which is RBX, RSI which is R12, RCX which is R13). I've already written this logic (and not only this - there are a lot of heuristics), but I only have one single dump to test is. So I'd like to ask you all to provide some dumps for testing (preferably RedHat kernels). It would be great to get nested IRQs, different exceptions: * double fault * stack fault that is something with stack switch (IST). This logic has not been written yet. Thanks in advance. Alexandr </pre> Alexandr Terekhov 2013-03-26T13:26:26 http://comments.gmane.org/gmane.linux.kernel.crash-dump.crash-utility/4488 <pre>Hi, list. I use crash-utility to analyse crash dump core from ARM soc. When I execute command below, I get the error "crash: read error: kernel virtual address: c0c1e040 type: "first vmap_area va_start"". I also test it by gdb. It works fine. The Linux kernel's version is v3.0.8. hfli&lt; at &gt;pc1935:~/work/crash-utility$ ./crash vmlinux Vmcore crash 6.1.4 Copyright (C) 2002-2013 Red Hat, Inc. Copyright (C) 2004, 2005, 2006, 2010 IBM Corporation Copyright (C) 1999-2006 Hewlett-Packard Co Copyright (C) 2005, 2006, 2011, 2012 Fujitsu Limited Copyright (C) 2006, 2007 VA Linux Systems Japan K.K. Copyright (C) 2005, 2011 NEC Corporation Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc. Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc. This program is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Enter "help copying" to see the conditions. This program has absolutely no warranty. Enter "help warranty" for details. GNU gdb (GDB) 7.3.1 Copyright (C) 2011 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later &lt;http://gnu.org/licenses/gpl.html&gt; This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "--host=i686-pc-linux-gnu --target=arm-elf-linux"... crash: read error: kernel virtual address: c0c1e040 type: "first vmap_area va_start" Errors like the one above typically occur when the kernel and memory source do not match. These are the files being used: KERNEL: vmlinux DUMPFILE: Vmcore -------------------------------------------------------------------------------------------- GDB works fine. hfli&lt; at &gt;pc1935:~/work/crash-utility$ ./gdb-7.5/gdb/gdb vmlinux Vmcore GNU gdb (GDB) 7.5 Copyright (C) 2012 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later &lt;http://gnu.org/licenses/gpl.html&gt; This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "--host=x86 --target=arm-linux-gnueabi". For bug reporting instructions, please see: &lt;http://www.gnu.org/software/gdb/bugs/&gt;... Reading symbols from /home/hfli/work/crash-utility/vmlinux...done. warning: exec file is newer than core file. [New LWP 332] #0 0xc0158de0 in sysrq_handle_crash (key=99) at drivers/tty/sysrq.c:134 warning: Source file is more recent than executable. 134 *killer = 1; (gdb) Any suggestion will be appreciate. Thanks. </pre> Li Haifeng 2013-03-26T11:10:20 http://comments.gmane.org/gmane.linux.kernel.crash-dump.crash-utility/4468 <pre>Hi list, I am current tracking one issue related with memory. What I want to know if a kernel address which is alloced by kmalloc, 1. whether that address is already freed or not 2. if not freed, could I know which task or struct is owning that range? I am also try to use the kmem command to get more info, but I don't know the meaning for its output... Like "CACHE"/"ALLOCATED"/"TOTAL"/"SLABS" member, what are they referring to? And according to "FREE / [ALLOCATED]" as below, does it mean that 0xe1416acc already be freed? crash&gt; kmem -S 0xe1416acc CACHE NAME OBJSIZE ALLOCATED TOTAL SLABS SSIZE e0002400 kmalloc-2048 2048 156 160 10 32k SLAB MEMORY NODE TOTAL ALLOCATED FREE c1628200 e1410000 0 12 12 0 FREE / [ALLOCATED] [e1410000] [e1410800] [e1411000] [e1411800] e1412000 (cpu 0 cache) e1412800 (cpu 0 cache) e1413000 (cpu 0 cache) e1413800 (cpu 0 cache) e1414000 (cpu 0 cache) e1414800 (cpu 0 cache) e1415000 (cpu 0 cache) [e1415800] crash&gt; Also seems current kmem only support SLAB, right? If memory is allocated with like SLUB or SLOB, could the kmem still handle it? Thanks, Lei </pre> Lei Wen 2013-03-24T11:41:31 http://comments.gmane.org/gmane.linux.kernel.crash-dump.crash-utility/4444 <pre>Hi list, I find in current 6.1.4 crash, when load module with "mod -S", only module's function symbol could be parsed out. But for module's global variant, crash cannot find related symbol... Any idea for this? Thanks, Lei </pre> Lei Wen 2013-03-19T12:33:22 http://comments.gmane.org/gmane.linux.kernel.crash-dump.crash-utility/4425 <pre>Hi Dave, et al., I have this little problem. I am trying to get a lustre file system extension working again. It used to work, but does no more. It first calls is_page_ptr(kvaddr, &amp;kpaddr) to convert a virtual address into a physical address, and then calls: to fetch the bytes. Updating the release to SLES-11 SP2 causes this to now fail. In my debugging of crash/gdb, this: is about to fail. However, this: works just fine. I've stepped through x_command until it gets to x86_64_kvtop() where I'm finding the logic a little twisty. But it pretty clearly does not rely on section_mem_map_addr() stuff. So, here's my point: this is confusing. What should I look for to determine why "is_page_ptr()" is saying 0xffffea001cdad420 is invalid while "x86_64_kvtop()" is saying that it is and its physical address is 0x87afad420? 0xffffea001cdad420: 0x0200000000000000 0xffffffff00000001 0xffffea001cdad430: 0x0000000000000000 0x0000000000000000 Help, please? Thank you! </pre> Bruce Korb 2013-03-14T23:38:13 http://comments.gmane.org/gmane.linux.kernel.crash-dump.crash-utility/4420 <pre>Hello Dave, When using crash on Linux 3.8, I found kmem -s is not working well. After some investigations, I got to know commit 3c58346525d82625e68e24f071804c2dc057b6f4 changes the usage of kmem_cache.array, nodelists is also stored in it, which caused original code failed to work. I made some fix, please refer to the attachment. </pre> qiaonuohan 2013-03-14T06:32:11 http://comments.gmane.org/gmane.linux.kernel.crash-dump.crash-utility/4413 <pre>Hello Dave, I found the option -r fails when it comes to blkio subsystem on kernel after 3.5. This patch can fix the bug. Thanks, Zhang Xiaohe </pre> Zhang Xiaohe 2013-03-12T08:54:24 http://comments.gmane.org/gmane.linux.kernel.crash-dump.crash-utility/4409 <pre>Dave, Here is the patch which add feature to whatis that would display full parameter name when it detect the symbol is a function. And this patch already include previous fix for gdb_get_datatype, please help review it. Thanks, Lei From: Lei Wen &lt;leiwen&lt; at &gt;marvell.com&gt; Date: Mon, 11 Mar 2013 10:34:15 +0800 Subject: [PATCH] whatis: display full parameter name when symbol is function Signed-off-by: Lei Wen &lt;leiwen&lt; at &gt;marvell.com&gt; --- defs.h | 1 + gdb-7.3.1.patch | 77 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ gdb_interface.c | 3 ++ symbols.c | 50 +++++++++++++++++++++++++++++++++++- 4 files changed, 130 insertions(+), 1 deletions(-) diff --git a/defs.h b/defs.h index 1f693c3..1b31d1f 100755 --- a/defs.h +++ b/defs.h &lt; at &gt;&lt; at &gt; -3685,6 +3685,7 &lt; at &gt;&lt; at &gt; struct gnu_request { #define GNU_GET_SYMBOL_TYPE (15) #define GNU_USER_PRINT_OPTION (16) #define GNU_SET_CRASH_BLOCK (17) +#define GNU_FUNCTION_NAMEARGS (18) #define GNU_DEBUG_COMMAND (100) /* * GNU flags diff --git a/gdb-7.3.1.patch b/gdb-7.3.1.patch index a12d3d4..3b0f0d1 100644 --- a/gdb-7.3.1.patch +++ b/gdb-7.3.1.patch &lt; at &gt;&lt; at &gt; -1821,3 +1821,80 &lt; at &gt;&lt; at &gt; diff -up gdb-7.3.1/gdb/psymtab.c.orig gdb-7.3.1/gdb/psymtab.c break; if (cur_sec == NULL) +--- gdb-7.3.1/gdb/symtab.c.orig ++++ gdb-7.3.1/gdb/symtab.c +&lt; at &gt;&lt; at &gt; -4848,6 +4848,7 &lt; at &gt;&lt; at &gt; static void gdb_get_symbol_type(struct gnu_request *); + static void gdb_command_exists(struct gnu_request *); + static void gdb_debug_command(struct gnu_request *); + static void gdb_function_numargs(struct gnu_request *); ++static void gdb_function_nameargs(struct gnu_request *); + static void gdb_add_symbol_file(struct gnu_request *); + static void gdb_delete_symbol_file(struct gnu_request *); + static void gdb_patch_symbol_values(struct gnu_request *); +&lt; at &gt;&lt; at &gt; -4952,6 +4953,10 &lt; at &gt;&lt; at &gt; gdb_command_funnel(struct gnu_request *req) + gdb_set_crash_block(req); + break; + ++case GNU_FUNCTION_NAMEARGS: ++gdb_function_nameargs(req); ++break; ++ + default: + req-&gt;flags |= GNU_COMMAND_FAILED; + break; +&lt; at &gt;&lt; at &gt; -5054,8 +5059,9 &lt; at &gt;&lt; at &gt; gdb_get_datatype(struct gnu_request *req) + if (gdb_CRASHDEBUG(2)) + console("expr-&gt;elts[0].opcode: OP_VAR_VALUE\n"); + type = expr-&gt;elts[2].symbol-&gt;type; ++req-&gt;typecode = TYPE_CODE(type); ++req-&gt;length = TYPE_LENGTH(type); + if (TYPE_CODE(type) == TYPE_CODE_ENUM) { +-req-&gt;typecode = TYPE_CODE(type); + req-&gt;value = SYMBOL_VALUE(expr-&gt;elts[2].symbol); + req-&gt;tagname = TYPE_TAG_NAME(type); + if (!req-&gt;tagname) { +&lt; at &gt;&lt; at &gt; -5243,6 +5249,44 &lt; at &gt;&lt; at &gt; gdb_function_numargs(struct gnu_request *req) + req-&gt;value = (ulong)TYPE_NFIELDS(sym-&gt;type); + } + ++static void ++gdb_function_nameargs(struct gnu_request *req) ++{ ++struct block *b; ++struct dict_iterator iter; ++struct symbol *sym = NULL; ++int len; ++char *buf = req-&gt;buf; ++ ++b = block_for_pc_sect(req-&gt;pc, find_pc_mapped_section(req-&gt;pc)); ++/* Get the lexical block, which is not a inline function */ ++while ((BLOCK_FUNCTION(b) == NULL || block_inlined_p(b)) ++&amp;&amp; BLOCK_SUPERBLOCK(b) != NULL) ++b = BLOCK_SUPERBLOCK(b); ++ ++ALL_BLOCK_SYMBOLS (b, iter, sym) ++{ ++if (!SYMBOL_IS_ARGUMENT (sym)) ++continue; ++ ++if (*SYMBOL_LINKAGE_NAME (sym)) ++{ ++struct symbol *nsym; ++ ++nsym = lookup_symbol (SYMBOL_LINKAGE_NAME (sym), ++b, VAR_DOMAIN, NULL); ++gdb_assert (nsym != NULL); ++if (SYMBOL_CLASS (nsym) != LOC_REGISTER ++|| SYMBOL_IS_ARGUMENT (nsym)) ++sym = nsym; ++} ++ ++len = strlen(SYMBOL_PRINT_NAME(sym)); ++sprintf(buf, "%s,", SYMBOL_PRINT_NAME(sym)); ++buf += len + 1; ++} ++} ++ + struct load_module *gdb_current_load_module = NULL; + + static void diff --git a/gdb_interface.c b/gdb_interface.c index afc197c..a460ea8 100755 --- a/gdb_interface.c +++ b/gdb_interface.c &lt; at &gt;&lt; at &gt; -590,6 +590,9 &lt; at &gt;&lt; at &gt; gdb_command_string(int cmd, char *buf, int live) case GNU_SET_CRASH_BLOCK: sprintf(buf, "GNU_SET_CRASH_BLOCK"); break; +case GNU_FUNCTION_NAMEARGS: + sprintf(buf, "GNU_FUNCTION_NAMEARGS"); +break; case 0: buf[0] = NULLCHAR; break; diff --git a/symbols.c b/symbols.c index 4fb397c..b38a2a3 100755 --- a/symbols.c +++ b/symbols.c &lt; at &gt;&lt; at &gt; -6660,18 +6660,62 &lt; at &gt;&lt; at &gt; whatis_datatype(char *st, ulong flags, FILE *ofp) } /* + * add the function argument to the function type showing + * The arg name input is seperated by comma + */ +static void +add_function_name(char *argnames, char *func) +{ +char *arg, *seperator, *tmp; + +tmp = func; +for (arg = strtok(argnames, ","); arg; ) { +seperator = strstr(tmp, ","); +if (!seperator) +seperator= strrchr(func, ')'); + +shift_string_right(seperator, strlen(arg) + 1); +BCOPY(arg, seperator + 1, strlen(arg)); +tmp = seperator + 2 + strlen(arg); +arg = strtok(NULL, ","); +} +} + +/* * Scan the symbol file for a variable declaration. */ static void whatis_variable(struct syment *sp) { +struct datatype_member datatype_member, *dm; +struct gnu_request *req; +int ret; char *p1; char buf[BUFSIZE]; +dm = &amp;datatype_member; +strcpy(buf, sp-&gt;name); +if (!arg_to_datatype(buf, dm, RETURN_ON_ERROR|DATATYPE_QUERY)) +return FALSE; + open_tmpfile(); sprintf(buf, "whatis %s", sp-&gt;name); - if (!gdb_pass_through(buf, fp, GNU_RETURN_ON_ERROR)) { + +req = (struct gnu_request *)GETBUF(sizeof(struct gnu_request)); +req-&gt;buf = GETBUF(BUFSIZE); + +ret = gdb_pass_through(buf, fp, GNU_RETURN_ON_ERROR); +if (ret &amp;&amp; dm-&gt;type == FUNCTION) { +req-&gt;command = GNU_FUNCTION_NAMEARGS; +req-&gt;flags |= GNU_RETURN_ON_ERROR; +req-&gt;pc = symbol_value(sp-&gt;name); + +gdb_interface(req); +} + if (!ret || req-&gt;flags &amp; GNU_COMMAND_FAILED) { close_tmpfile(); +FREEBUF(req-&gt;buf); +FREEBUF(req); error(FATAL, "gdb request failed: whatis %s\n", sp-&gt;name); } &lt; at &gt;&lt; at &gt; -6693,6 +6737,7 &lt; at &gt;&lt; at &gt; whatis_variable(struct syment *sp) if (index(buf, '(') == rindex(buf, '(')) { shift_string_right(p1, strlen(sp-&gt;name)); BCOPY(sp-&gt;name, p1, strlen(sp-&gt;name)); +add_function_name(req-&gt;buf, p1 + strlen(sp-&gt;name)); } else { p1 = strstr(buf, ")"); shift_string_right(p1, strlen(sp-&gt;name)); &lt; at &gt;&lt; at &gt; -6705,6 +6750,9 &lt; at &gt;&lt; at &gt; whatis_variable(struct syment *sp) fprintf(fp, "%s%s%s;\n", p1, LASTCHAR(p1) == '*' ? "":" ", sp-&gt;name); } + +FREEBUF(req-&gt;buf); +FREEBUF(req); } /* </pre> Lei Wen 2013-03-11T02:42:13 http://comments.gmane.org/gmane.linux.kernel.crash-dump.crash-utility/4403 <pre>Hi, Here is a scenario in which three structures (of type 'struct anon_vma_chain') are linked with the element 'struct list_head same_anon_vma'. Please check the following structure. crash&gt; struct anon_vma_chain -o struct anon_vma_chain { [0] struct vm_area_struct *vma; [4] struct anon_vma *anon_vma; [8] struct list_head same_vma; [16] struct list_head same_anon_vma; } SIZE: 24 The following are the three structures that are linked: crash&gt; anon_vma_chain -l anon_vma_chain.same_anon_vma 0xe4315510 struct anon_vma_chain { vma = 0xe43144f0, anon_vma = 0xe49e2d98, same_vma = { next = 0xe4314528, prev = 0xe43154f0 }, same_anon_vma = { next = 0xe4fe4f60, prev = 0xe4e833a8 } } same_anon_vma.next=0xe4fe4f60 from the previous structure, is used to extract the next structure. crash&gt; anon_vma_chain -l anon_vma_chain.same_anon_vma 0xe4fe4f60 struct anon_vma_chain { vma = 0xe4fe85f8, anon_vma = 0xe49e2d98, same_vma = { next = 0xe4fe8630, prev = 0xe4fe4f40 }, same_anon_vma = { next = 0xe45a1df8, prev = 0xe4315510 } } same_anon_vma.next=0xe45a1df8 from the previous structure, is used to extract the next structure. crash&gt; anon_vma_chain -l anon_vma_chain.same_anon_vma 0xe45a1df8 struct anon_vma_chain { vma = 0xe45a73e8, anon_vma = 0xe49e2d98, same_vma = { next = 0xe45a7420, prev = 0xe45a1dd8 }, same_anon_vma = { next = 0xe461fb40, prev = 0xe4fe4f60 } } Is there something like "anon_vma_chain -l anon_vma_chain.same_anon_vma 0xe4315510 -c 3" to get three structures that are linked using anon_vma_chain.same_anon_vma. Actually "anon_vma_chain -l anon_vma_chain.same_anon_vma 0xe4315510 -c 3" doesn't work in this case. It seems like -c option works only in case of array of structures. Could someone help me to print the above three structures with a single command? Thanks, Yogesh </pre> Yogesh Blore 2013-03-08T14:51:57 http://comments.gmane.org/gmane.linux.kernel.crash-dump.crash-utility/4401 <pre>Hello Dave, The following kernel commit changes the member "rq" to member "root_rl" in the request_queue structure: commit a051661ca6d134c18599498b185b667859d4339b Author: Tejun Heo &lt;tj&lt; at &gt;kernel.org&gt; Date: Tue Jun 26 15:05:44 2012 -0700 blkcg: implement per-blkg request allocation - struct request_list rq; + struct request_list root_rl; This breaks the "dev -d" crash command: crash&gt; dev -d MAJOR GENDISK NAME REQUEST QUEUE TOTAL ASYNC SYNC DRV dev: invalid structure member offset: request_queue_rq FILE: dev.c LINE: 3807 FUNCTION: get_diskio_1() 80116438: OFFSET_verify+96 800f55da: get_diskio_1+62 800f5c52: display_all_diskio+1318 8006ff1c: exec_command+972 dev: invalid structure member offset: request_queue_rq FILE: dev.c LINE: 3807 FUNCTION: get_diskio_1() The following fix just uses "root_rl" instead of "rq". Not completely sure, if this is sufficient. With the fix on my s390 system I get: crash&gt; dev -d MAJOR GENDISK NAME REQUEST QUEUE TOTAL ASYNC SYNC DRV 94 0x1ea144a8 dasda 0x1ebcc2b8 0 0 0 0 9 0x1e93d9c8 md0 0x1ebcb560 0 0 0 0 --- dev.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) --- a/dev.c +++ b/dev.c &lt; at &gt;&lt; at &gt; -4050,7 +4050,11 &lt; at &gt;&lt; at &gt; void diskio_init(void) MEMBER_OFFSET_INIT(request_list_count, "request_list", "count"); MEMBER_OFFSET_INIT(request_queue_in_flight, "request_queue", "in_flight"); -MEMBER_OFFSET_INIT(request_queue_rq, "request_queue", "rq"); +if (MEMBER_EXISTS("request_queue", "rq")) +MEMBER_OFFSET_INIT(request_queue_rq, "request_queue", "rq"); +else +MEMBER_OFFSET_INIT(request_queue_rq, + "request_queue", "root_rl"); MEMBER_OFFSET_INIT(subsys_private_klist_devices, "subsys_private", "klist_devices"); MEMBER_OFFSET_INIT(subsystem_kset, "subsystem", "kset"); </pre> Michael Holzheu 2013-03-08T14:20:17 Search the mailing list at Gmane query http://search.gmane.org/?group=$group=gmane.linux.kernel.crash-dump.crash-utility