File libxl.set-migration-constraints-from-cmdline.patch of Package xen

Overview Repositories Revisions Requests Users Attributes Meta

File libxl.set-migration-constraints-from-cmdline.patch of Package xen (Revision 7edea4ffb8eae0b14e88a795bde87f42)

Currently displaying revision 7edea4ffb8eae0b14e88a795bde87f42 , Show latest

From 77deb80879859ed279e24a790ec08e9c5d37dd0e Mon Sep 17 00:00:00 2001
From: Olaf Hering <olaf@aepfle.de>
Date: Wed, 5 Feb 2014 14:37:53 +0100
Subject: libxl: set migration constraints from cmdline

Add new options to xl migrate to control the process of migration.
The intention is to optionally abort the migration if it takes too long
to migrate a busy guest due to the high number of new dirty pages.
Currently the guest is suspended to transfer the remaining dirty pages.
The suspend/resume cycle will cause a time jump. This transfer can take
a long time, which can confuse the guest if the time jump is too far.
The new options allow to override the built-in default values, which are
not changed by this patch.

--max_iters <number>  Number of iterations before final suspend (default: 30)

--max_factor <factor> Max amount of memory to transfer before final suspend (default: 3*RAM)

--min_remaing <pages> Number of dirty pages before stop&copy (default: 50)

--abort_if_busy       Abort migration instead of doing final suspend.

The changes to libxl change the API, handle LIBXL_API_VERSION == 0x040200.

v8:
  - merge --min_remaing changes
  - tools/libxc: print stats if migration is aborted
  - use special _suse version of lib calls to preserve ABI

v7:
  - remove short options
  - update description of --abort_if_busy in xl.1
  - extend description of --abort_if_busy in xl help
  - add comment to libxl_domain_suspend declaration, props is optional

v6:
 - update the LIBXL_API_VERSION handling for libxl_domain_suspend
   change it to an inline function if LIBXL_API_VERSION is defined to 4.2.0
 - rename libxl_save_properties to libxl_domain_suspend_properties
 - rename ->xlflags to ->flags within that struct

v5:
 - adjust libxl_domain_suspend prototype, move flags, max_iters,
   max_factor into a new, optional struct libxl_save_properties
 - rename XCFLAGS_DOMSAVE_NOSUSPEND to XCFLAGS_DOMSAVE_ABORT_IF_BUSY
 - rename LIBXL_SUSPEND_NO_FINAL_SUSPEND to LIBXL_SUSPEND_ABORT_IF_BUSY
 - rename variables no_suspend to abort_if_busy
 - rename option -N/--no_suspend to -A/--abort_if_busy
 - update xl.1, extend description of -A option

v4:
 - update default for no_suspend from None to 0 in XendCheckpoint.py:save
 - update logoutput in setMigrateConstraints
 - change xm migrate defaults from None to 0
 - add new options to xl.1
 - fix syntax error in XendDomain.py:domain_migrate_constraints_set
 - fix xm migrate -N option name to match xl migrate

v3:
 - move logic errors in libxl__domain_suspend and fixed help text in
   cmd_table to separate patches
 - fix syntax error in XendCheckpoint.py
 - really pass max_iters and max_factor in libxl__xc_domain_save
 - make libxl_domain_suspend_0x040200 declaration globally visible
 - bump libxenlight.so SONAME from 2.0 to 2.1 due to changed
   libxl_domain_suspend

v2:
 - use LIBXL_API_VERSION and define libxl_domain_suspend_0x040200
 - fix logic error in min_reached check in xc_domain_save
 - add longopts
 - update --help text
 - correct description of migrate --help text

Signed-off-by: Olaf Hering <olaf@aepfle.de>
---
 docs/man/xl.pod.1                |   20 +++++++++++++++++++
 tools/libxc/include/xenguest.h   |    6 +++++
 tools/libxc/xc_domain_save.c     |   26 ++++++++++++++++++++++--
 tools/libxc/xc_nomigrate.c       |    9 ++++++++
 tools/libxl/libxl.c              |   27 ++++++++++++++++++++++---
 tools/libxl/libxl.h              |   15 ++++++++++++++
 tools/libxl/libxl_dom.c          |    1 
 tools/libxl/libxl_internal.h     |    4 +++
 tools/libxl/libxl_save_callout.c |    4 ++-
 tools/libxl/libxl_save_helper.c  |    4 ++-
 tools/libxl/xl_cmdimpl.c         |   41 +++++++++++++++++++++++++++++++++------
 tools/libxl/xl_cmdtable.c        |   23 ++++++++++++++-------
 12 files changed, 159 insertions(+), 21 deletions(-)

Index: xen-4.5.0-testing/docs/man/xl.pod.1
===================================================================
--- xen-4.5.0-testing.orig/docs/man/xl.pod.1
+++ xen-4.5.0-testing/docs/man/xl.pod.1
@@ -428,6 +428,26 @@ Send <config> instead of config file fro
 
 Print huge (!) amount of debug during the migration process.
 
+=item B<--max_iters> I<number>
+
+Number of iterations before final suspend (default: 30)
+
+=item B<--max_factor> I<factor>
+
+Max amount of memory to transfer before final suspend (default: 3*RAM)
+
+=item B<--min_remaining>
+
+Number of remaining dirty pages. If the number of dirty pages drops that
+low the guest is suspended and the remaing pages are transfered to <host>.
+
+=item B<--abort_if_busy>
+
+Abort migration instead of doing final suspend/transfer/resume if the
+guest has still dirty pages after the number of iterations and/or the
+amount of RAM transferred. This avoids long periods of time where the
+guest is suspended.
+
 =back
 
 =item B<remus> [I<OPTIONS>] I<domain-id> I<host>
Index: xen-4.5.0-testing/tools/libxc/include/xenguest.h
===================================================================
--- xen-4.5.0-testing.orig/tools/libxc/include/xenguest.h
+++ xen-4.5.0-testing/tools/libxc/include/xenguest.h
@@ -28,6 +28,7 @@
 #define XCFLAGS_HVM       (1 << 2)
 #define XCFLAGS_STDVGA    (1 << 3)
 #define XCFLAGS_CHECKPOINT_COMPRESS    (1 << 4)
+#define XCFLAGS_DOMSAVE_ABORT_IF_BUSY  (1 << 5)
 
 #define X86_64_B_SIZE   64 
 #define X86_32_B_SIZE   32
@@ -88,6 +89,11 @@ int xc_domain_save(xc_interface *xch, in
                    uint32_t max_factor, uint32_t flags /* XCFLAGS_xxx */,
                    struct save_callbacks* callbacks, int hvm);
 
+int xc_domain_save_suse(xc_interface *xch, int io_fd, uint32_t dom, uint32_t max_iters,
+                   uint32_t max_factor, uint32_t flags /* XCFLAGS_xxx */,
+                   uint32_t min_remaining,
+                   struct save_callbacks* callbacks, int hvm);
+
 
 /* callbacks provided by xc_domain_restore */
 struct restore_callbacks {
Index: xen-4.5.0-testing/tools/libxc/xc_domain_save.c
===================================================================
--- xen-4.5.0-testing.orig/tools/libxc/xc_domain_save.c
+++ xen-4.5.0-testing/tools/libxc/xc_domain_save.c
@@ -44,6 +44,7 @@
 */
 #define DEF_MAX_ITERS   29   /* limit us to 30 times round loop   */
 #define DEF_MAX_FACTOR   3   /* never send more than 3x p2m_size  */
+#define DEF_MIN_REMAINING 50 /* low water mark of dirty pages */
 
 struct save_ctx {
     unsigned long hvirt_start; /* virtual starting address of the hypervisor */
@@ -800,8 +801,9 @@ static int save_tsc_info(xc_interface *x
     return 0;
 }
 
-int xc_domain_save(xc_interface *xch, int io_fd, uint32_t dom, uint32_t max_iters,
+int xc_domain_save_suse(xc_interface *xch, int io_fd, uint32_t dom, uint32_t max_iters,
                    uint32_t max_factor, uint32_t flags,
+                   uint32_t min_remaining,
                    struct save_callbacks* callbacks, int hvm)
 {
     xc_dominfo_t info;
@@ -810,6 +812,7 @@ int xc_domain_save(xc_interface *xch, in
     int rc, frc, i, j, last_iter = 0, iter = 0;
     int live  = (flags & XCFLAGS_LIVE);
     int debug = (flags & XCFLAGS_DEBUG);
+    int abort_if_busy = (flags & XCFLAGS_DOMSAVE_ABORT_IF_BUSY);
     int superpages = !!hvm;
     int race = 0, sent_last_iter, skip_this_iter = 0;
     unsigned int sent_this_iter = 0;
@@ -910,6 +913,7 @@ int xc_domain_save(xc_interface *xch, in
     /* If no explicit control parameters given, use defaults */
     max_iters  = max_iters  ? : DEF_MAX_ITERS;
     max_factor = max_factor ? : DEF_MAX_FACTOR;
+    min_remaining = min_remaining ? : DEF_MIN_REMAINING;
 
     if ( !get_platform_info(xch, dom,
                             &ctx->max_mfn, &ctx->hvirt_start, &ctx->pt_levels, &dinfo->guest_width) )
@@ -1536,10 +1540,21 @@ int xc_domain_save(xc_interface *xch, in
 
         if ( live )
         {
+            int min_reached = sent_this_iter + skip_this_iter < min_remaining;
             if ( (iter >= max_iters) ||
-                 (sent_this_iter+skip_this_iter < 50) ||
+                 min_reached ||
                  (total_sent > dinfo->p2m_size*max_factor) )
             {
+                if ( !min_reached && abort_if_busy )
+                {
+                    DPRINTF("Live migration aborted, as requested. (guest too busy?)");
+                    DPRINTF(" total_sent %lu iter %d, max_iters %u max_factor %u",
+                      total_sent, iter, max_iters, max_factor);
+                    print_stats(xch, dom, sent_this_iter, &time_stats, &shadow_stats, 1);
+                    rc = 1;
+                    goto out;
+                }
+
                 DPRINTF("Start last iteration\n");
                 last_iter = 1;
 
@@ -2181,6 +2196,13 @@ exit:
     return !!errno;
 }
 
+int xc_domain_save(xc_interface *xch, int io_fd, uint32_t dom, uint32_t max_iters,
+                   uint32_t max_factor, uint32_t flags,
+                   struct save_callbacks* callbacks, int hvm)
+{
+    return xc_domain_save_suse(xch, io_fd, dom, max_iters, max_factor, flags, 0, callbacks, hvm);
+}
+
 /*
  * Local variables:
  * mode: C
Index: xen-4.5.0-testing/tools/libxc/xc_nomigrate.c
===================================================================
--- xen-4.5.0-testing.orig/tools/libxc/xc_nomigrate.c
+++ xen-4.5.0-testing/tools/libxc/xc_nomigrate.c
@@ -21,6 +21,15 @@
 #include <xenctrl.h>
 #include <xenguest.h>
 
+int xc_domain_save_suse(xc_interface *xch, int io_fd, uint32_t dom, uint32_t max_iters,
+                   uint32_t max_factor, uint32_t flags,
+                   uint32_t min_remaining,
+                   struct save_callbacks* callbacks, int hvm)
+{
+    errno = ENOSYS;
+    return -1;
+}
+
 int xc_domain_save(xc_interface *xch, int io_fd, uint32_t dom, uint32_t max_iters,
                    uint32_t max_factor, uint32_t flags,
                    struct save_callbacks* callbacks, int hvm)
Index: xen-4.5.0-testing/tools/libxl/libxl.c
===================================================================
--- xen-4.5.0-testing.orig/tools/libxl/libxl.c
+++ xen-4.5.0-testing/tools/libxl/libxl.c
@@ -944,7 +944,8 @@ static void domain_suspend_cb(libxl__egc
 
 }
 
-int libxl_domain_suspend(libxl_ctx *ctx, uint32_t domid, int fd, int flags,
+static int do_libxl_domain_suspend(libxl_ctx *ctx, uint32_t domid, int fd,
+                         const libxl_domain_suspend_suse_properties *props,
                          const libxl_asyncop_how *ao_how)
 {
     AO_CREATE(ctx, domid, ao_how);
@@ -965,8 +966,14 @@ int libxl_domain_suspend(libxl_ctx *ctx,
     dss->domid = domid;
     dss->fd = fd;
     dss->type = type;
-    dss->live = flags & LIBXL_SUSPEND_LIVE;
-    dss->debug = flags & LIBXL_SUSPEND_DEBUG;
+    if (props) {
+        dss->live = props->flags & LIBXL_SUSPEND_LIVE;
+        dss->debug = props->flags & LIBXL_SUSPEND_DEBUG;
+        dss->max_iters = props->max_iters;
+        dss->max_factor = props->max_factor;
+        dss->min_remaining = props->min_remaining;
+        dss->xlflags = props->flags;
+    }
 
     libxl__domain_suspend(egc, dss);
     return AO_INPROGRESS;
@@ -975,6 +982,20 @@ int libxl_domain_suspend(libxl_ctx *ctx,
     return AO_ABORT(rc);
 }
 
+int libxl_domain_suspend_suse(libxl_ctx *ctx, uint32_t domid, int fd,
+                         const libxl_domain_suspend_suse_properties *props,
+                         const libxl_asyncop_how *ao_how)
+{
+    return do_libxl_domain_suspend(ctx, domid, fd, props, ao_how);
+}
+
+int libxl_domain_suspend(libxl_ctx *ctx, uint32_t domid, int fd, int flags,
+                         const libxl_asyncop_how *ao_how)
+{
+    libxl_domain_suspend_suse_properties props = { .flags = flags };
+    return do_libxl_domain_suspend(ctx, domid, fd, &props, ao_how);
+}
+
 int libxl_domain_pause(libxl_ctx *ctx, uint32_t domid)
 {
     int ret;
Index: xen-4.5.0-testing/tools/libxl/libxl.h
===================================================================
--- xen-4.5.0-testing.orig/tools/libxl/libxl.h
+++ xen-4.5.0-testing/tools/libxl/libxl.h
@@ -949,8 +949,23 @@ int libxl_domain_suspend(libxl_ctx *ctx,
                          int flags, /* LIBXL_SUSPEND_* */
                          const libxl_asyncop_how *ao_how)
                          LIBXL_EXTERNAL_CALLERS_ONLY;
+
+typedef struct {
+    int flags; /* LIBXL_SUSPEND_* */
+    int max_iters;
+    int max_factor;
+    int min_remaining;
+} libxl_domain_suspend_suse_properties;
+
+#define LIBXL_HAVE_DOMAIN_SUSPEND_SUSE
+int libxl_domain_suspend_suse(libxl_ctx *ctx, uint32_t domid, int fd,
+                         const libxl_domain_suspend_suse_properties *props, /* optional */
+                         const libxl_asyncop_how *ao_how)
+                         LIBXL_EXTERNAL_CALLERS_ONLY;
+
 #define LIBXL_SUSPEND_DEBUG 1
 #define LIBXL_SUSPEND_LIVE 2
+#define LIBXL_SUSPEND_ABORT_IF_BUSY 4
 
 /* @param suspend_cancel [from xenctrl.h:xc_domain_resume( @param fast )]
  *   If this parameter is true, use co-operative resume. The guest
Index: xen-4.5.0-testing/tools/libxl/libxl_dom.c
===================================================================
--- xen-4.5.0-testing.orig/tools/libxl/libxl_dom.c
+++ xen-4.5.0-testing/tools/libxl/libxl_dom.c
@@ -1808,6 +1808,7 @@ void libxl__domain_suspend(libxl__egc *e
 
     dss->xcflags = (live ? XCFLAGS_LIVE : 0)
           | (debug ? XCFLAGS_DEBUG : 0)
+          | (dss->xlflags & LIBXL_SUSPEND_ABORT_IF_BUSY ? XCFLAGS_DOMSAVE_ABORT_IF_BUSY : 0)
           | (dss->hvm ? XCFLAGS_HVM : 0);
 
     dss->guest_evtchn.port = -1;
Index: xen-4.5.0-testing/tools/libxl/libxl_internal.h
===================================================================
--- xen-4.5.0-testing.orig/tools/libxl/libxl_internal.h
+++ xen-4.5.0-testing/tools/libxl/libxl_internal.h
@@ -2803,6 +2803,10 @@ struct libxl__domain_suspend_state {
     libxl__ev_evtchn guest_evtchn;
     int guest_evtchn_lockfd;
     int hvm;
+    int max_iters;
+    int max_factor;
+    int min_remaining;
+    int xlflags;
     int xcflags;
     int guest_responded;
     libxl__xswait_state pvcontrol;
Index: xen-4.5.0-testing/tools/libxl/libxl_save_callout.c
===================================================================
--- xen-4.5.0-testing.orig/tools/libxl/libxl_save_callout.c
+++ xen-4.5.0-testing/tools/libxl/libxl_save_callout.c
@@ -110,7 +110,9 @@ void libxl__xc_domain_save(libxl__egc *e
     }
 
     const unsigned long argnums[] = {
-        dss->domid, 0, 0, dss->xcflags, dss->hvm,
+        dss->domid,
+        dss->max_iters, dss->max_factor, dss->min_remaining,
+        dss->xcflags, dss->hvm,
         toolstack_data_fd, toolstack_data_len,
         cbflags,
     };
Index: xen-4.5.0-testing/tools/libxl/libxl_save_helper.c
===================================================================
--- xen-4.5.0-testing.orig/tools/libxl/libxl_save_helper.c
+++ xen-4.5.0-testing/tools/libxl/libxl_save_helper.c
@@ -215,6 +215,7 @@ int main(int argc, char **argv)
         uint32_t dom =             strtoul(NEXTARG,0,10);
         uint32_t max_iters =       strtoul(NEXTARG,0,10);
         uint32_t max_factor =      strtoul(NEXTARG,0,10);
+        uint32_t min_remaining =   strtoul(NEXTARG,0,10);
         uint32_t flags =           strtoul(NEXTARG,0,10);
         int hvm =                  atoi(NEXTARG);
         toolstack_save_fd  =       atoi(NEXTARG);
@@ -228,7 +229,8 @@ int main(int argc, char **argv)
         helper_setcallbacks_save(&helper_save_callbacks, cbflags);
 
         startup("save");
-        r = xc_domain_save(xch, io_fd, dom, max_iters, max_factor, flags,
+        r = xc_domain_save_suse(xch, io_fd, dom, max_iters, max_factor, flags,
+                           min_remaining,
                            &helper_save_callbacks, hvm);
         complete(r);
 
Index: xen-4.5.0-testing/tools/libxl/xl_cmdimpl.c
===================================================================
--- xen-4.5.0-testing.orig/tools/libxl/xl_cmdimpl.c
+++ xen-4.5.0-testing/tools/libxl/xl_cmdimpl.c
@@ -3878,6 +3878,8 @@ static void migrate_do_preamble(int send
 }
 
 static void migrate_domain(uint32_t domid, const char *rune, int debug,
+                           int max_iters, int max_factor,
+                           int min_remaining, int abort_if_busy,
                            const char *override_config_file)
 {
     pid_t child = -1;
@@ -3886,7 +3888,13 @@ static void migrate_domain(uint32_t domi
     char *away_domname;
     char rc_buf;
     uint8_t *config_data;
-    int config_len, flags = LIBXL_SUSPEND_LIVE;
+    int config_len;
+    libxl_domain_suspend_suse_properties props = {
+        .flags = LIBXL_SUSPEND_LIVE,
+        .max_iters = max_iters,
+        .max_factor = max_factor,
+        .min_remaining = min_remaining,
+    };
 
     save_domain_core_begin(domid, override_config_file,
                            &config_data, &config_len);
@@ -3905,10 +3913,13 @@ static void migrate_domain(uint32_t domi
     xtl_stdiostream_adjust_flags(logger, XTL_STDIOSTREAM_HIDE_PROGRESS, 0);
 
     if (debug)
-        flags |= LIBXL_SUSPEND_DEBUG;
-    rc = libxl_domain_suspend(ctx, domid, send_fd, flags, NULL);
+        props.flags |= LIBXL_SUSPEND_DEBUG;
+    if (abort_if_busy)
+        props.flags |= LIBXL_SUSPEND_ABORT_IF_BUSY;
+
+    rc = libxl_domain_suspend_suse(ctx, domid, send_fd, &props, NULL);
     if (rc) {
-        fprintf(stderr, "migration sender: libxl_domain_suspend failed"
+        fprintf(stderr, "migration sender: libxl_domain_suspend_suse failed"
                 " (rc=%d)\n", rc);
         if (rc == ERROR_GUEST_TIMEDOUT)
             goto failed_suspend;
@@ -4295,13 +4306,18 @@ int main_migrate(int argc, char **argv)
     char *rune = NULL;
     char *host;
     int opt, daemonize = 1, monitor = 1, debug = 0;
+    int max_iters = 0, max_factor = 0, min_remaining = 0, abort_if_busy = 0;
     static struct option opts[] = {
         {"debug", 0, 0, 0x100},
+        {"max_iters", 1, 0, 0x101},
+        {"max_factor", 1, 0, 0x102},
+        {"min_remaining", 1, 0, 0x103},
+        {"abort_if_busy", 0, 0, 0x104},
         COMMON_LONG_OPTS,
         {0, 0, 0, 0}
     };
 
-    SWITCH_FOREACH_OPT(opt, "FC:s:e", opts, "migrate", 2) {
+    SWITCH_FOREACH_OPT(opt, "FC:s:eM:m:A", opts, "migrate", 2) {
     case 'C':
         config_filename = optarg;
         break;
@@ -4318,6 +4334,18 @@ int main_migrate(int argc, char **argv)
     case 0x100:
         debug = 1;
         break;
+    case 0x101:
+        max_iters = atoi(optarg);
+        break;
+    case 0x102:
+        max_factor = atoi(optarg);
+        break;
+    case 0x103:
+        min_remaining = atoi(optarg);
+        break;
+    case 0x104:
+        abort_if_busy = 1;
+        break;
     }
 
     domid = find_domain(argv[optind]);
@@ -4348,7 +4376,8 @@ int main_migrate(int argc, char **argv)
             return 1;
     }
 
-    migrate_domain(domid, rune, debug, config_filename);
+    migrate_domain(domid, rune, debug, max_iters, max_factor, min_remaining,
+                   abort_if_busy, config_filename);
     return 0;
 }
 #endif
Index: xen-4.5.0-testing/tools/libxl/xl_cmdtable.c
===================================================================
--- xen-4.5.0-testing.orig/tools/libxl/xl_cmdtable.c
+++ xen-4.5.0-testing/tools/libxl/xl_cmdtable.c
@@ -155,14 +155,21 @@ struct cmd_spec cmd_table[] = {
       &main_migrate, 0, 1,
       "Migrate a domain to another host",
       "[options] <Domain> <host>",
-      "-h              Print this help.\n"
-      "-C <config>     Send <config> instead of config file from creation.\n"
-      "-s <sshcommand> Use <sshcommand> instead of ssh.  String will be passed\n"
-      "                to sh. If empty, run <host> instead of ssh <host> xl\n"
-      "                migrate-receive [-d -e]\n"
-      "-e              Do not wait in the background (on <host>) for the death\n"
-      "                of the domain.\n"
-      "--debug         Print huge (!) amount of debug during the migration process."
+      "-h                      Print this help.\n"
+      "-C <config>             Send <config> instead of config file from creation.\n"
+      "-s <sshcommand>         Use <sshcommand> instead of ssh.  String will be passed\n"
+      "                        to sh. If empty, run <host> instead of ssh <host> xl\n"
+      "                        migrate-receive [-d -e]\n"
+      "-e                      Do not wait in the background (on <host>) for the death\n"
+      "                        of the domain.\n"
+      "--debug                 Print huge (!) amount of debug during the migration process.\n"
+      "\n"
+      "SUSE Linux specific options:\n"
+      "--max_iters <number>    Number of iterations before final suspend (default: 30)\n"
+      "--max_factor <factor>   Max amount of memory to transfer before final suspend (default: 3*RAM).\n"
+      "--min_remaining <pages> Number of remaining dirty pages before final suspend (default: 50).\n"
+      "--abort_if_busy         Abort migration instead of doing final suspend, if number\n"
+      "                        of iterations or amount of transfered memory is exceeded."
     },
     { "restore",
       &main_restore, 0, 1,

Places

File libxl.set-migration-constraints-from-cmdline.patch of Package xen (Revision 7edea4ffb8eae0b14e88a795bde87f42)

Places