gmane.comp.emulators.xen.devel

Re: [PATCH] make pygrub cope better with big files in guest

M A Young — 2012-05-16T23:44:45


I am attaching the third version of this patch. The datafile object 
doesn't have an explicit close so this patch uses del to clean it. I was 
also using unlink incorrectly which is now fixed.

I have tested it for basic functionality and also used pygrub directly to 
write to a very small file system and with this patch pygrub deletes the 
temporary files it creates if it runs out of file space.

 Michael YoungMake pygrub cope better with big files in the guest.
Only read the first megabyte of a configuration file (grub etc.)
Read the kernel and ramdisk files from the guest in one megabyte pieces
so pygrub doesn't grow too large if they are large.
If there are problems writing the temporary copies of the kernel and ramdisk
files delete them and exit.

Signed-off-by: Michael Young <m.a.young< at >durham.ac.uk>

--- xen-4.2.0/tools/pygrub/src/pygrub.orig2012-05-12 16:40:48.000000000 +0100
+++ xen-4.2.0/tools/pygrub/src/pygrub
< at >< at > -28,6 +28,7 < at >< at >
 import grub.ExtLinuxConf
 
 PYGRUB_VER = 0.6
+fs_re

[PATCH] xl: track child processes for the benefit oflibxl

Ian Jackson — 2012-05-16T20:40:05

Each time xl forks, it needs to record the pid, so that its exit
status can be preserved if it happens that libxl's event loop reaped
it.  Consequently we also have a new wrapper for waitpid, which in
that case returns the previously-reaped status.

When we get a console ready callback from libxl, check to see if we
have spawned but not reaped a previous console client, and if so reap
it now.  (This is necessary to prevent improper use of the xlchild
struct, but has the happy consequence of checking the exit status of
the first console client in the pygrub case.)

Refactor the two calls to libxl_ctx_alloc into a new function
xl_ctx_alloc which also sets the child reaped handler callback.

All of this has the effect of suppressing a message
   unknown child [nnnn] unexpected exited status zero
which would sometimes (depending on a race) appear with `xl create -c'
and pygrub.

Signed-off-by: Ian Jackson <ian.jackson< at >eu.citrix.com>
Reported-by: Ian Campbell <ian.campbell< at >citrix.com>
Cc: Roger Pau Monne <roger.p

[xen-unstable test] 12893: regressions - FAIL

xen.org — 2012-05-16T19:32:21

flight 12893 xen-unstable real [real]
http://www.chiark.greenend.org.uk/~xensrcts/logs/12893/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-xl-pcipt-intel  8 debian-fixup           fail REGR. vs. 12884
 test-amd64-amd64-xl-qemuu-win7-amd64 12 guest-localmigrate/x10 fail REGR. vs. 12876

Tests which are failing intermittently (not blocking):
 test-amd64-i386-qemuu-rhel6hvm-intel  7 redhat-install      fail pass in 12889
 test-i386-i386-xl             9 guest-start        fail in 12889 pass in 12893

Regressions which are regarded as allowable (not blocking):
 test-amd64-i386-qemuu-rhel6hvm-amd  9 guest-start.2            fail like 12884

Tests which did not succeed, but are not blocking:
 test-amd64-i386-rhel6hvm-intel 11 leak-check/check             fail never pass
 test-i386-i386-xl-qemuu-winxpsp3 13 guest-stop                 fail never pass
 test-amd64-amd64-win         16 leak-check/check             fail   never pass
 test-amd64

[xen-4.1-testing test] 12886: tolerable FAIL - PUSHED

xen.org — 2012-05-16T19:10:13

flight 12886 xen-4.1-testing real [real]
http://www.chiark.greenend.org.uk/~xensrcts/logs/12886/

Failures :-/ but no regressions.

Regressions which are regarded as allowable (not blocking):
 test-amd64-amd64-xl-sedf      5 xen-boot                     fail   like 12825

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-pcipt-intel  9 guest-start                 fail never pass
 test-amd64-i386-qemuu-rhel6hvm-amd 11 leak-check/check         fail never pass
 test-amd64-i386-rhel6hvm-amd 11 leak-check/check             fail   never pass
 test-amd64-i386-rhel6hvm-intel 11 leak-check/check             fail never pass
 test-amd64-amd64-xl-qemuu-winxpsp3 13 guest-stop               fail never pass
 test-amd64-amd64-win         16 leak-check/check             fail   never pass
 test-amd64-i386-xend-winxpsp3 16 leak-check/check             fail  never pass
 test-amd64-amd64-xl-win      13 guest-stop                   fail   never pass
 test-amd64-i386-qemuu-rhel6hvm-intel 11 leak-check/check

[xen-unstable bisection] completetest-amd64-amd64-xl-pcipt-intel

xen.org — 2012-05-16T18:49:56

branch xen-unstable
xen branch xen-unstable
job test-amd64-amd64-xl-pcipt-intel
test debian-fixup

Tree: linux git://xenbits.xen.org/linux-pvops.git
Tree: qemu git://xenbits.xen.org/staging/qemu-xen-unstable.git
Tree: qemuu git://xenbits.xen.org/staging/qemu-upstream-unstable.git
Tree: xen http://xenbits.xen.org/staging/xen-unstable.hg

*** Found and reproduced problem changeset ***

  Bug is in tree:  xen http://xenbits.xen.org/staging/xen-unstable.hg
  Bug introduced:  28dedda66ec5
  Bug not present: db6a951a4bbb


  changeset:   25346:28dedda66ec5
  user:        George Dunlap <george.dunlap< at >eu.citrix.com>
  date:        Tue May 15 16:28:15 2012 +0100
      
      libxl: Rename pci_list_assignable to pci_assignable_list
      
      ...to prepare for a consistent "pci_assignable_*" naming scheme.
      
      Also move the man page entry into the PCI PASS-THROUGH section, rather
      than the XEN HOST section.
      
      No functional changes.
      
      Signed-off-by: George Dunlap <george.dunlap< at >eu.

[xen-4.0-testing test] 12885: regressions - FAIL

xen.org — 2012-05-16T18:00:57

flight 12885 xen-4.0-testing real [real]
http://www.chiark.greenend.org.uk/~xensrcts/logs/12885/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-xl-qemuu-winxpsp3  5 xen-boot            fail REGR. vs. 12795

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl          15 guest-stop                   fail   never pass
 test-amd64-amd64-xl-pcipt-intel  8 debian-fixup                fail never pass
 test-amd64-i386-xl-multivcpu 15 guest-stop                   fail   never pass
 test-i386-i386-xl            15 guest-stop                   fail   never pass
 test-amd64-amd64-xl-sedf-pin 15 guest-stop                   fail   never pass
 test-amd64-i386-xl           15 guest-stop                   fail   never pass
 test-amd64-amd64-xl-qemuu-win7-amd64  8 guest-saverestore      fail never pass
 test-amd64-amd64-xl-sedf     15 guest-stop                   fail   never pass
 test-amd64-amd64-xl-win7-amd64  8 guest-saveresto

[PATCH 1.1 V2] xen: Fix PV-on-HVM

Anthony PERARD — 2012-05-16T17:50:10

In the context of PV-on-HVM under Xen, the emulated nics are supposed to be
unplug before the guest drivers are initialized, when the guest write to a
specific IO port.

Without this patch, the guest end up with two nics with the same MAC, the
emulated nic and the PV nic.

Signed-off-by: Anthony PERARD <anthony.perard< at >citrix.com>
---
 hw/xen_platform.c |    5 ++++-
 1 files changed, 4 insertions(+), 1 deletions(-)

diff --git a/hw/xen_platform.c b/hw/xen_platform.c
index a9c52a6..0214f37 100644
--- a/hw/xen_platform.c
+++ b/hw/xen_platform.c
< at >< at > -87,7 +87,10 < at >< at > static void unplug_nic(PCIBus *b, PCIDevice *d)
 {
     if (pci_get_word(d->config + PCI_CLASS_DEVICE) ==
             PCI_CLASS_NETWORK_ETHERNET) {
-        qdev_unplug(&(d->qdev), NULL);
+        /* Until qdev_free includes a call to object_unparent, we call it here
+         */
+        object_unparent(&d->qdev.parent_obj);
+        qdev_free(&d->qdev);
     }
 }

Re: Xen 3.3.x on recent dom0 kernels

Alex Bligh — 2012-05-16T17:45:44

Jan, Pasi,

--On 16 May 2012 14:08:39 +0100 Jan Beulich <JBeulich< at >suse.com> wrote:

...

Fantastic - thank you.

[IDEA] xendhcp proposal

George Shuklin — 2012-05-16T16:35:27

Good day.

During automated VM deploy I found some very annoying problem - network 
VM configuration. Classic DHCP is not very easy to manage in virtual 
environment (protection from stray DHCP-servers, problems with 
autoidentification).

Problems are:
* how to identificate VM?
* how to provide network configuration in native way for guest (e.g. 
support of mount order for network fs, if-up scripts and so on)
* how to reconfigure hosts 'on demand'? (even DHCP is not very 'on 
demand' beause of lease time).

Propose:

Store network configuration in domain part of xenstore, use xendhcp 
service to read those data from guest and acts like classic dhcp client.

Implementation detail:

1) Store data in DHCP-like way (option code - answer).
2) Subscribe for changes and reacts to it like we have lease expiration.
3) xendhcp should replace normal dhcp (may be even with conclict in 
packages with original dhcp client, and provide it functionality), mimic 
/sbin/dhclient functionality.

That allows to keep original n

[PATCH 10/13] libxl: add option to choose who executeshotplug scripts

Roger Pau Monne — 2012-05-16T16:11:53

Add and option to xl.conf file to decide if hotplug scripts are
executed from the toolstack (xl) or from udev as it used to be in the
past.

This option is only introduced in this patch, but it has no effect
since the code to call hotplug scripts from libxl is introduced in a
latter patch.

This choice will be saved in "libxl/disable_udev", as specified in the
DISABLE_UDEV_PATH constant.

Cc: Ian Jackson <ian.jackson< at >eu.citrix.com>
Signed-off-by: Roger Pau Monne <roger.pau< at >citrix.com>
---
 docs/man/xl.conf.pod.5       |    8 ++++++++
 tools/examples/xl.conf       |    5 +++++
 tools/libxl/libxl_create.c   |   25 ++++++++++++++++++++++++-
 tools/libxl/libxl_internal.h |    1 +
 tools/libxl/libxl_types.idl  |    1 +
 tools/libxl/xl.c             |    4 ++++
 tools/libxl/xl.h             |    1 +
 tools/libxl/xl_cmdimpl.c     |    1 +
 8 files changed, 45 insertions(+), 1 deletions(-)

diff --git a/docs/man/xl.conf.pod.5 b/docs/man/xl.conf.pod.5
index 8bd45ea..72825a0 100644
--- a/docs/man/xl.conf.pod.5
+++ b/d

[PATCH 12/13] libxl: call hotplug scripts for diskdevices from libxl

Roger Pau Monne — 2012-05-16T16:11:55

Since most of the needed work is already done in previous patches,
this patch only contains the necessary code to call hotplug scripts
for disk devices, that should be called when the device is added or
removed from a guest.

We will chain the launch of the disk hotplug scripts after the
device_backend_callback callback, or directly from
libxl__initiate_device_{add,remove} if the device is already in the
desired state.

Cc: Ian Jackson <ian.jackson< at >eu.citrix.com>
Signed-off-by: Roger Pau Monne <roger.pau< at >citrix.com>
---
 tools/hotplug/Linux/xen-backend.rules     |    6 +-
 tools/hotplug/Linux/xen-hotplug-common.sh |    6 ++
 tools/libxl/Makefile                      |    3 +-
 tools/libxl/libxl_device.c                |   23 ++++-
 tools/libxl/libxl_hotplug.c               |   84 +++++++++++++++++++
 tools/libxl/libxl_internal.h              |   17 ++++
 tools/libxl/libxl_linux.c                 |  126 +++++++++++++++++++++++++++++
 7 files changed, 256 insertions(+), 9 deletions(-)
 create mode 100644 tools

Re: [PATCH 1/4] Introduce a new hotplug state: Forceeject.

Michael S. Tsirkin — 2012-05-16T16:25:44

No sure which this patch is meant, the force eject is not a good idea.
Freeing directly from Xen is fine.

Re: [PATCH 1/4] Introduce a new hotplug state: Forceeject.

Michael S. Tsirkin — 2012-05-16T16:24:15

Okay. Can you fix the bug Amos reported this way too
to avoid confusion?  Or prefer Amos to do it?

[PATCH 07/13] libxl: convert libxl_domain_destroy to anAO op

Roger Pau Monne — 2012-05-16T16:11:50

This change introduces some new structures, and breaks the mutual
dependency that libxl_domain_destroy and libxl__destroy_device_model
had. This is done by checking if the domid passed to
libxl_domain_destroy has a stubdom, and then having the bulk of the
destroy machinery in a separate function (libxl__destroy_domid) that
doesn't check for stubdom presence, since we check for it in the upper
level function. The reason behind this change is the need to use
structures for ao operations, and it was impossible to have two
different self-referencing structs.

All uses of libxl_domain_destroy have been changed, and either
replaced by the new libxl_domain_destroy ao function or by the
internal libxl__domain_destroy that can be used inside an already
running ao.

Cc: Ian Jackson <ian.jackson< at >eu.citrix.com>
Signed-off-by: Roger Pau Monne <roger.pau< at >citrix.com>
---
 tools/libxl/libxl.c               |  180 +++++++++++-----------
 tools/libxl/libxl.h               |    3 +-
 tools/libxl/libxl_create.c        |   30 ++

[PATCH 09/13] libxl: convert libxl_device_nic_add to anasync operation

Roger Pau Monne — 2012-05-16T16:11:52

This patch converts libxl_device_nic_add to an ao operation that
waits for device backend to reach state XenbusStateInitWait and then
marks the operation as completed. This is not really useful now, but
will be used by latter patches that will launch hotplug scripts after
we reached the desired xenbus state.

Calls to libxl_device_nic_add have also been moved to occur after the
device model has been launched, so when hotplug scripts are called
from this functions the interfaces already exists.

As usual, libxl_device_nic_add callers have been modified, and the
internal function libxl__device_disk_add has been used if the call was
inside an already running ao.

Cc: Ian Jackson <ian.jackson< at >eu.citrix.com>
Signed-off-by: Roger Pau Monne <roger.pau< at >citrix.com>
---
 tools/libxl/libxl.c          |   15 +++++---
 tools/libxl/libxl.h          |    3 +-
 tools/libxl/libxl_create.c   |   73 ++++++++++++++++++++++++++++++++++++++----
 tools/libxl/libxl_device.c   |   23 ++++++++++---
 tools/libxl/libxl_dm.c       |   5

[PATCH 03/13] libxl: add libxl__xs_path_cleanup

Roger Pau Monne — 2012-05-16T16:11:46

Add a function which behaves like "xenstore-rm -t", and which will be
used to clean xenstore after unplug since we will be no longer
executing xen-hotplug-cleanup script, that used to do that for us.

Changes since v4:

 * Caller must supply a transaction.

Changes since v3:

 * Better error checking and function description.

Changes since v2:

 * Moved the function to libxl_xshelp.c and added the prototype to
   libxl_internal.h.

Cc: Ian Jackson <ian.jackson< at >eu.citrix.com>
Signed-off-by: Roger Pau Monne <roger.pau< at >citrix.com>
---
 tools/libxl/libxl_internal.h |    8 ++++++++
 tools/libxl/libxl_xshelp.c   |   42 ++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 50 insertions(+), 0 deletions(-)

diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h
index aaaf644..ae5527b 100644
--- a/tools/libxl/libxl_internal.h
+++ b/tools/libxl/libxl_internal.h
< at >< at > -494,6 +494,14 < at >< at > _hidden bool libxl__xs_mkdir(libxl__gc *gc, xs_transaction_t t,
 
 _hidden char *libxl__xs_libxl_path(libxl__gc

[PATCH 08/13] libxl: convert libxl_device_disk_add toan async operation

Roger Pau Monne — 2012-05-16T16:11:51

This patch converts libxl_device_disk_add to an ao operation that
waits for device backend to reach state XenbusStateInitWait and then
marks the operation as completed. This is not really useful now, but
will be used by latter patches that will launch hotplug scripts after
we reached the desired xenbus state.

As usual, libxl_device_disk_add callers have been modified, and the
internal function libxl__device_disk_add has been used if the call was
inside an already running ao.

Cc: Ian Jackson <ian.jackson< at >eu.citrix.com>
Signed-off-by: Roger Pau Monne <roger.pau< at >citrix.com>
---
 tools/libxl/libxl.c          |   22 +++++++++-----
 tools/libxl/libxl.h          |    4 ++-
 tools/libxl/libxl_create.c   |   51 ++++++++++++++++++++++++++++----
 tools/libxl/libxl_device.c   |   66 ++++++++++++++++++++++++++++++++++++------
 tools/libxl/libxl_dm.c       |   62 +++++++++++++++++++++++++++++++--------
 tools/libxl/libxl_internal.h |   21 ++++++++++++-
 tools/libxl/xl_cmdimpl.c     |    2 +-
 7 files changed, 187 insert

[PATCH 13/13] libxl: call hotplug scripts for nicdevices from libxl

Roger Pau Monne — 2012-05-16T16:11:56

Since most of the needed work is already done in previous patches,
this patch only contains the necessary code to call hotplug scripts
for nic devices, that should be called when the device is added or
removed from a guest.

Since the call to libxl__device_hotplug is already there, this patch
only adds the necessary code to this function to handle the vif case,
and the necessary modification in the udev rules.

Cc: Ian Jackson <ian.jackson< at >eu.citrix.com>
Signed-off-by: Roger Pau Monne <roger.pau< at >citrix.com>
---
 tools/hotplug/Linux/xen-backend.rules |    6 +-
 tools/libxl/libxl_internal.h          |    1 +
 tools/libxl/libxl_linux.c             |  130 ++++++++++++++++++++++++++++++++-
 3 files changed, 133 insertions(+), 4 deletions(-)

diff --git a/tools/hotplug/Linux/xen-backend.rules b/tools/hotplug/Linux/xen-backend.rules
index d55ff11..c591a3f 100644
--- a/tools/hotplug/Linux/xen-backend.rules
+++ b/tools/hotplug/Linux/xen-backend.rules
< at >< at > -2,8 +2,8 < at >< at > SUBSYSTEM=="xen-backend", KERNEL=="tap*", ENV{UDEV_

[PATCH 11/13] libxl: set nic type to VIF by default

Roger Pau Monne — 2012-05-16T16:11:54

Set the default value for nic interfaces to VIF, since it used to be
IOEMU, even for PV guests.

Cc: Ian Jackson <ian.jackson< at >eu.citrix.com>
Signed-off-by: Roger Pau Monne <roger.pau< at >citrix.com>
---
 tools/libxl/libxl.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/tools/libxl/libxl.c b/tools/libxl/libxl.c
index 2490138..631de15 100644
--- a/tools/libxl/libxl.c
+++ b/tools/libxl/libxl.c
< at >< at > -1637,7 +1637,7 < at >< at > int libxl__device_nic_setdefault(libxl__gc *gc, libxl_device_nic *nic)
                                   libxl__xen_script_dir_path()) < 0 )
         return ERROR_FAIL;
     if (!nic->nictype)
-        nic->nictype = LIBXL_NIC_TYPE_IOEMU;
+        nic->nictype = LIBXL_NIC_TYPE_VIF;
     return 0;
 }

[PATCH 05/13] libxl: move libxl_device_nic_add tolibxl_device

Roger Pau Monne — 2012-05-16T16:11:48

Move the code of this function to libxl_device.c so it can be made
asyncronious later on the series. The static function
libxl__device_from_nic also has to be moved to libxl_device and it is
no longer static.

The code will be fixed in a latter patch, replacing libxl__sprintf,
LIBXL_LOG* and lines > 80.

Cc: Ian Jackson <ian.jackson< at >eu.citrix.com>
Signed-off-by: Roger Pau Monne <roger.pau< at >citrix.com>
---
 tools/libxl/libxl.c          |  108 +---------------------------------------
 tools/libxl/libxl_device.c   |  113 ++++++++++++++++++++++++++++++++++++++++++
 tools/libxl/libxl_internal.h |    6 ++
 3 files changed, 121 insertions(+), 106 deletions(-)

diff --git a/tools/libxl/libxl.c b/tools/libxl/libxl.c
index 2d8abd0..d3b6a53 100644
--- a/tools/libxl/libxl.c
+++ b/tools/libxl/libxl.c
< at >< at > -1655,117 +1655,13 < at >< at > int libxl__device_nic_setdefault(libxl__gc *gc, libxl_device_nic *nic)
     return 0;
 }
 
-static int libxl__device_from_nic(libxl__gc *gc, uint32_t domid,
-                                  libxl_de

[PATCH 01/13] libxl: pass env vars to libxl__exec

Roger Pau Monne — 2012-05-16T16:11:44

Add another parameter to libxl__exec call that contains the
environment variables to use when performing the execvp call.

Changes since v4:

 * Unify error checking.

Changes since v3:

 * Use CTX instead of defining libxl_ctx *ctx.

 * Error checking on setenv.

 * Better comment on header for libxl__exec function.

 * Added some const-correctness.

Cc: Ian Jackson <ian.jackson< at >eu.citrix.com>
Signed-off-by: Roger Pau Monne <roger.pau< at >citrix.com>
---
 tools/libxl/libxl.c            |    2 +-
 tools/libxl/libxl_bootloader.c |    4 ++--
 tools/libxl/libxl_dm.c         |    2 +-
 tools/libxl/libxl_exec.c       |   14 ++++++++++++--
 tools/libxl/libxl_internal.h   |   20 ++++++++++++++++++--
 5 files changed, 34 insertions(+), 8 deletions(-)

diff --git a/tools/libxl/libxl.c b/tools/libxl/libxl.c
index 4d01cf8..2a09192 100644
--- a/tools/libxl/libxl.c
+++ b/tools/libxl/libxl.c
< at >< at > -1261,7 +1261,7 < at >< at > int libxl_vncviewer_exec(libxl_ctx *ctx, uint32_t domid, int autopass)
         args[2] = "-autopass";
     }
 
-