gmane.linux.kernel.kexec

Re: [PATCH] makedumpfile: warn on vmlinux without dwarf

Cliff Wickman — 2013-05-20T12:17:12

Agreed.  An error code is good.

-Cliff

Re: [PATCH] makedumpfile: warn on vmlinux without dwarf

Atsushi Kumagai — 2013-05-20T02:20:44

On Wed, 15 May 2013 13:43:59 -0500
Cliff Wickman <cpw-sJ/iWh9BUns< at >public.gmane.org> wrote:


I agree to insert this message here, but don't agree to abort without
any error handling.
I think the code below seems better:

 if (dwarf_info.dwarfd == NULL) {
 ERRMSG("dwfl_module_getdwarf error.\n");
 return DWARF_CB_ABORT;
 }


Thanks
Atsushi Kumagai

Re: [PATCH] makedumpfile: message on non-mmap kernels

Atsushi Kumagai — 2013-05-20T02:20:32

On Wed, 15 May 2013 13:43:59 -0500
Cliff Wickman <cpw-sJ/iWh9BUns< at >public.gmane.org> wrote:


Where came from this original code ?
The current code in mmap branch doesn't look the return value of
update_mmap_range():

if (!is_mapped_with_mmap(offset))
update_mmap_range(offset);

Anyway, I'll merge this patch into mmap branch after I push the patch
below.


Thanks
Atsushi Kumagai

-----------------------------------------------------------------------
Author: Atsushi Kumagai <kumagai-atsushi-biTfD1RFvDe45+QrQBaojngSJqDPrsil< at >public.gmane.org>
Date:   Thu May 17 15:02:19 2013 +0900

    [PATCH] Add error handling for read_with_mmap().

    Add error handling for read_with_mmap the same as read()'s case.

    Signed-off-by: Atsushi Kumagai <kumagai-atsushi-biTfD1RFvDe45+QrQBaojngSJqDPrsil< at >public.gmane.org>

diff --git a/makedumpfile.c b/makedumpfile.c
index fce2e35..1f656e8 100644
--- a/makedumpfile.c
+++ b/makedumpfile.c
< at >< at > -291,7 +291,8 < at >< at > read_with_mmap(off_t offset, void *bufptr, unsigned long size) {
 nex

[PATCH] makedumpfile: buddy identification when noncyclic

Cliff Wickman — 2013-05-15T18:43:59

From: Cliff Wickman <cpw-sJ/iWh9BUns< at >public.gmane.org>

The 'buddy' identification of free pages should be done for non-cyclical
dumps as well as cyclical.
Remove the cyclic test from this condition:
    if (info->flag_cyclic && (info->dump_level & DL_EXCLUDE_FREE))
I find this to speed the scan of page structures for a 1TB system from
60sec to 30sec.

And only do the exclude_free_page() procedure if free pages cannot be
identified using the buddy technique.

Signed-off-by: Cliff Wickman <cpw-sJ/iWh9BUns< at >public.gmane.org>
---
 makedumpfile.c |    9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)

Index: makedumpfile.mmap/makedumpfile.c
===================================================================
--- makedumpfile.mmap.orig/makedumpfile.c
+++ makedumpfile.mmap/makedumpfile.c
< at >< at > -2979,7 +2979,9 < at >< at > out:
 if (!get_value_for_old_linux())
 return FALSE;
 
-if (info->flag_cyclic && (info->dump_level & DL_EXCLUDE_FREE))
+/* use buddy identification of free pages whether cyclic or not */
+/* (t

[PATCH] makedumpfile: message on non-mmap kernels

Cliff Wickman — 2013-05-15T18:43:59

From: Cliff Wickman <cpw-sJ/iWh9BUns< at >public.gmane.org>

If the crash kernel does not support mmap(2) of /proc/vmcore, make
makedumpfile fall back to using reads with a non-alarming message.

Signed-off-by: Cliff Wickman <cpw-sJ/iWh9BUns< at >public.gmane.org>
---
 makedumpfile.c |   37 ++++++++++++++++++++++---------------
 1 file changed, 22 insertions(+), 15 deletions(-)

Index: makedumpfile.mmap/makedumpfile.c
===================================================================
--- makedumpfile.mmap.orig/makedumpfile.c
+++ makedumpfile.mmap/makedumpfile.c
< at >< at > -239,7 +239,7 < at >< at > read_page_desc(unsigned long long paddr,
 }
 
 static int
-update_mmap_range(off_t offset) {
+update_mmap_range(off_t offset, int initial) {
 off_t start_offset;
 off_t map_size;
 off_t max_offset = get_max_file_offset();
< at >< at > -258,10 +258,11 < at >< at > update_mmap_range(off_t offset) {
      info->fd_memory, start_offset);
 
 if (info->mmap_buf == MAP_FAILED) {
-ERRMSG("Can't map [%llx-%llx] with mmap()\n %s",
-       (ulonglong)start_of

[PATCH] makedumpfile: non-cyclic default

Cliff Wickman — 2013-05-15T18:43:59

From: Cliff Wickman <cpw-sJ/iWh9BUns< at >public.gmane.org>

Make non-cyclic the default, as it is faster.

Dumping a 1TB memory is successful with a crashkernel=512M.
And allow -Y on the command line, as well as --non-cylic.

Signed-off-by: Cliff Wickman <cpw-sJ/iWh9BUns< at >public.gmane.org>
---
 makedumpfile.c |    9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)

Index: makedumpfile.mmap/makedumpfile.c
===================================================================
--- makedumpfile.mmap.orig/makedumpfile.c
+++ makedumpfile.mmap/makedumpfile.c
< at >< at > -8585,13 +8585,14 < at >< at > main(int argc, char *argv[])
 initialize_tables();
 
 /*
- * By default, makedumpfile works in constant memory space.
+ * By default, makedumpfile makes one pass through page structures.
+ * Use --cyclic-buffer to work in constant memory space.
  */
-info->flag_cyclic = TRUE;
+info->flag_cyclic = FALSE;
 
 info->block_order = DEFAULT_ORDER;
 message_level = DEFAULT_MSG_LEVEL;
-while ((opt = getopt_long(argc, argv, "b:cDd:EF

[PATCH] makedumpfile: turn off mmap for free page scan

Cliff Wickman — 2013-05-15T18:43:59

From: Cliff Wickman <cpw-sJ/iWh9BUns< at >public.gmane.org>

Because addresses of each successive page structure are often descending
it is much faster to use read()'s, so set off info->flag_usemmap
while doing exclude_free_page().

Signed-off-by: Cliff Wickman <cpw-sJ/iWh9BUns< at >public.gmane.org>
---
 makedumpfile.c |   15 ++++++++++++++-
 1 file changed, 14 insertions(+), 1 deletion(-)

Index: makedumpfile.mmap/makedumpfile.c
===================================================================
--- makedumpfile.mmap.orig/makedumpfile.c
+++ makedumpfile.mmap/makedumpfile.c
< at >< at > -3925,9 +3925,18 < at >< at > _exclude_free_page(void)
 return TRUE;
 }
 
+/*
+ * Note that this is a very lengthy process, even using mmap(2). The
+ * page structures in the free lists are read one-at-a-time, but using
+ * readpage_elf(), which reads an entire page.  And addresses of each
+ * successive page structure are often descending, which means that each
+ * read is another mmap(2) operation.
+ * It is much faster to use read()'s, so set off inf

[PATCH] makedumpfile: warn on vmlinux without dwarf

Cliff Wickman — 2013-05-15T18:43:59

From: Cliff Wickman <cpw-sJ/iWh9BUns< at >public.gmane.org>

If the vmlinux does not have dwarf information makedumpfile fails in
a rather obscure way, with a flood of redundant errors,

Make it fail with more of a hint of what is wrong.

Signed-off-by: Cliff Wickman <cpw-sJ/iWh9BUns< at >public.gmane.org>
---
 dwarf_info.c |    4 ++++
 1 file changed, 4 insertions(+)

Index: makedumpfile.mmap/dwarf_info.c
===================================================================
--- makedumpfile.mmap.orig/dwarf_info.c
+++ makedumpfile.mmap/dwarf_info.c
< at >< at > -139,6 +139,10 < at >< at > process_module (Dwfl_Module *dwflmod,
 
 /* get a debug context descriptor.*/
 dwarf_info.dwarfd = dwfl_module_getdwarf (dwflmod, &dwbias);
+if (dwarf_info.dwarfd == NULL) {
+fprintf(stderr, "makedumpfile: dwfl_module_getdwarf error\n");
+exit(1);
+}
 dwarf_info.elfd = dwarf_getelf(dwarf_info.dwarfd);
 
 mod_name = dwfl_module_info(dwflmod, NULL, NULL, NULL, NULL, NULL,

Re: Kernel panics when using kexec for rebooting

Eric W. Biederman — 2013-05-15T16:53:06


Some hardware does not respect the bus master bit.  I assume that Mellanox
has that problem.

Eric

Re: Kernel panics when using kexec for rebooting

Dave Lloyd — 2013-05-15T15:50:40

On Tue, May 14, 2013 at 6:14 PM, Eric W. Biederman
<ebiederm-aS9lmoZGLiVWk0Htik3J/w< at >public.gmane.org> wrote:


Removing the infiniband drivers in /etc/init.d/halt right before the
kexec -e definitely seems to have done the trick. Thanks!

My follow up question is, if we are clearing the bus master DMA bit,
why are the Mellanox not reset or at least stop doing DMA until the
driver is loaded again? Is this a case of the Mellanox hardware
needing an extra poke to do the right thing?

--dlloyd

Re: [PATCH] kexec: check if memory is reserved only when loading kdump kernel.

Simon Horman — 2013-05-15T00:35:33

Thanks, applied.

Re: Kernel panics when using kexec for rebooting

Dave Lloyd — 2013-05-14T22:25:26

On Tue, May 14, 2013 at 5:01 PM, Eric W. Biederman
<ebiederm-aS9lmoZGLiVWk0Htik3J/w< at >public.gmane.org> wrote:


Thank goodness it's not just me! :-)


I'm trying this now. In one panic, the pte referenced was
0x100010000000000 which sure looks a whole like someone wrote his
registers in there. It certainly doesn't look like a valid pte.

So far, unloading pata_acpi and pata_amd seem to have eliminated the
ACPI exception messages. I believe that this resets the device
properly. Unfortunately, it looks like lots of drivers don't implement
the pci_driver->shutdown call, so it would make sense that this is a
relatively widespread problem.

--dlloyd

Re: makedumpfile: question about memory hole

HATAYAMA Daisuke — 2013-05-15T01:00:41

Thanks for your investigation. I'm now very clear to what's happening there.

Re: Kernel panics when using kexec for rebooting

Eric W. Biederman — 2013-05-14T23:14:33


Oh. Yeah.  IB definitely sets up memory for ongoing dma.  So if it
doesn't have a shutdown method and IB traffic comes in during boot just
about anything cood happen.


Maybe.  Or it might be luck of timing, which memory was stomped when
incomming IB packets stomped on memory.

Eric

Re: Kernel panics when using kexec for rebooting

Dave Lloyd — 2013-05-14T22:57:26

On Tue, May 14, 2013 at 5:33 PM, Eric W. Biederman
<ebiederm-aS9lmoZGLiVWk0Htik3J/w< at >public.gmane.org> wrote:

It could certainly be buggy hardware. Other details include:

Kernel 3.0.29.0 and we are also using infiniband (which I believe I
found a reference to the Mellanox hardware potentially causing this
issue unless the driver was unloaded before reboot with kexec). The
potential issue with unloading the IB drivers doesn't bug me nearly as
much as not unloading pata_amd and pata_acpi causing the ACPI Error
messages upon reboot with kexec.

I'm inclined to chalk the ACPI Error mesages up to potentially buggy
BIOS/hardware from the vendor since pata_amd and pata_acpi are in wide
use and I would expect to see more issues reported were there truly an
issue with rebooting with kexec and not unloading pata_amd and
pata_acpi.


Interesting. Thanks for the information. I have to admit, this part of
the kernel is a bit of a mystery to me.

--dlloyd

Re: Kernel panics when using kexec for rebooting

Eric W. Biederman — 2013-05-14T22:33:29


It is a classic issue, although I suspect something is unique in your
setup because it has (to my knowledge) not been a widespread problem for
years.


Most devices don't leave dma setup if you reboot, and even more the
generic pci clears the bus master DMA bit which shuts down a lot more
dma.

So the actual lack of a shutdown method is not as much of an issue as it
might appear.

Eric

Re: Kernel panics when using kexec for rebooting

Eric W. Biederman — 2013-05-14T22:01:06


Yes this does seem to be all over the place, and memory corruption
probably caused by ongoing-dma seems like a reasonable hypothesis.

The easy first thing to try is to remove all of your kernel modules
before you reboot with kexec.  Not infrequently the module remove path
is better tested than the device shutdown path.

Eric

[PATCH] kexec: check if memory is reserved only when loading kdump kernel.

WANG Chao — 2013-05-14T10:32:50

When memory for crashkernel isn't reserved, it's confusing that kexec
spits error message for unloading kdump kernel as if we were loading it:

  # kexec -p -u
  Memory for crashkernel is not reserved
  Please reserve memory by passing "crashkernel=X< at >Y" parameter to the kernel
  Then try loading kdump kernel
  # echo $?
  1

It's more appropriate to test if memory is reserved only when
loading kdump kernel. With this patch:

  # kexec -p -u
  # echo $?
  0

It's also the same behavior with the case of trying to unload kernel
from unloaded state.

Signed-off-by: WANG Chao <chaowang-H+wXaHxf7aLQT0dZR+AlfA< at >public.gmane.org>
---
 kexec/kexec.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/kexec/kexec.c b/kexec/kexec.c
index 911c0f6..6a13f79 100644
--- a/kexec/kexec.c
+++ b/kexec/kexec.c
< at >< at > -1175,7 +1175,8 < at >< at > int main(int argc, char *argv[])
 }
 }
 
-if ((kexec_flags & KEXEC_ON_CRASH) && !is_crashkernel_mem_reserved()) {
+if (do_load && (kexec_flags & KEXEC_ON_CRASH) &&
+    !is_cras

Re: [PATCH] kexec: Replace printf() with die() to error out to stderr

Zhang Yanfei — 2013-05-14T04:35:36

于 2013年05月14日 11:56, WANG Chao 写道:

Looks good to me.

Acked-by: Zhang Yanfei <zhangyanfei< at >cn.fujitsu.com>



_______________________________________________
kexec mailing list
kexec< at >lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

Re: [PATCH] kexec: Replace printf() with die() to error out to stderr

Simon Horman — 2013-05-14T04:35:49

Thanks, applied.

[PATCH] kexec: Replace printf() with die() to error out to stderr

WANG Chao — 2013-05-14T03:56:48

Error messages are mixed stderr with stdout when we use die() along with
printf(). So use die() to keep error out consistent.

Signed-off-by: WANG Chao <chaowang-H+wXaHxf7aLQT0dZR+AlfA< at >public.gmane.org>
---
 kexec/kexec.c | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/kexec/kexec.c b/kexec/kexec.c
index 53d6edc..911c0f6 100644
--- a/kexec/kexec.c
+++ b/kexec/kexec.c
< at >< at > -1176,17 +1176,17 < at >< at > int main(int argc, char *argv[])
 }
 
 if ((kexec_flags & KEXEC_ON_CRASH) && !is_crashkernel_mem_reserved()) {
-printf("Memory for crashkernel is not reserved\n");
-printf("Please reserve memory by passing ");
-printf("\"crashkernel=X< at >Y\" parameter to the kernel\n");
-die("Then try loading kdump kernel\n");
+die("Memory for crashkernel is not reserved\n"
+    "Please reserve memory by passing"
+    "\"crashkernel=X< at >Y\" parameter to kernel\n"
+    "Then try to loading kdump kernel\n");
 }
 
 if (do_load && (kexec_flags & KEXEC_PRESERVE_CONTEXT) &&
     mem_max == ULON