File backport-virt-patches-from-3001-256.patch of Package salt - openSUSE Build Service

Overview Repositories Revisions Requests Users Attributes Meta

File backport-virt-patches-from-3001-256.patch of Package salt (Revision 764d440dfc50ee828b4accd23849fdab)

Currently displaying revision 764d440dfc50ee828b4accd23849fdab , Show latest

From e932261250fbdd4deff7341c1c2826ed562b102c Mon Sep 17 00:00:00 2001
From: Cedric Bosdonnat <cbosdonnat@suse.com>
Date: Mon, 7 Sep 2020 15:00:40 +0200
Subject: [PATCH] Backport virt patches from 3001+ (#256)

* Fix various spelling mistakes in master branch (#55954)

* Fix typo of additional

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

* Fix typo of against

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

* Fix typo of amount

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

* Fix typo of argument

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

* Fix typo of attempt

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

* Fix typo of bandwidth

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

* Fix typo of caught

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

* Fix typo of compatibility

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

* Fix typo of consistency

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

* Fix typo of conversions

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

* Fix typo of corresponding

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

* Fix typo of dependent

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

* Fix typo of dictionary

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

* Fix typo of disabled

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

* Fix typo of adapters

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

* Fix typo of disassociates

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

* Fix typo of changes

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

* Fix typo of command

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

* Fix typo of communicate

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

* Fix typo of community

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

* Fix typo of configuration

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

* Fix typo of default

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

* Fix typo of absence

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

* Fix typo of attribute

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

* Fix typo of container

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

* Fix typo of described

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

* Fix typo of existence

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

* Fix typo of explicit

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

* Fix typo of formatted

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

* Fix typo of guarantees

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

* Fix typo of hexadecimal

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

* Fix typo of hierarchy

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

* Fix typo of initialize

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

* Fix typo of label

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

* Fix typo of management

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

* Fix typo of mismatch

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

* Fix typo of don't

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

* Fix typo of manually

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

* Fix typo of getting

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

* Fix typo of information

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

* Fix typo of meant

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

* Fix typo of nonexistent

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

* Fix typo of occur

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

* Fix typo of omitted

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

* Fix typo of normally

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

* Fix typo of overridden

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

* Fix typo of repository

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

* Fix typo of separate

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

* Fix typo of separator

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

* Fix typo of specific

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

* Fix typo of successful

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

* Fix typo of succeeded

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

* Fix typo of support

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

* Fix typo of version

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

* Fix typo of that's

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

* Fix typo of "will be removed"

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

* Fix typo of release

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

* Fix typo of synchronize

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

* Fix typo of python

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

* Fix typo of usually

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

* Fix typo of override

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

* Fix typo of running

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

* Fix typo of whether

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

* Fix typo of package

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

* Fix typo of persist

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

* Fix typo of preferred

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

* Fix typo of present

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

* Fix typo of run

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

* Fix spelling mistake of "allows someone to..."

"Allows to" is not correct English. It must either be "allows someone
to" or "allows doing".

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

* Fix spelling mistake of "number of times"

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

* Fix spelling mistake of msgpack

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

* Fix spelling mistake of daemonized

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

* Fix spelling mistake of daemons

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

* Fix spelling mistake of extemporaneous

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

* Fix spelling mistake of instead

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

* Fix spelling mistake of returning

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>

* Fix literal comparissons

* virt: Convert cpu_baseline ElementTree to string

In commit 0f5184c (Remove minidom use in virt module) the value
of `cpu` become `xml.etree.ElementTree.Element` and no longer
has a method `toxml()`. This results in the following error:

$ salt '*' virt.cpu_baseline
host2:
    The minion function caused an exception: Traceback (most recent call last):
      File "/usr/lib/python3.7/site-packages/salt/minion.py", line 1675, in _thread_return
        return_data = minion_instance.executors[fname](opts, data, func, args, kwargs)
      File "/usr/lib/python3.7/site-packages/salt/executors/direct_call.py", line 12, in execute
        return func(*args, **kwargs)
      File "/usr/lib/python3.7/site-packages/salt/modules/virt.py", line 4410, in cpu_baseline
        return cpu.toxml()
    AttributeError: 'xml.etree.ElementTree.Element' object has no attribute 'toxml'

Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>

* PR#57374 backport

virt: pool secret should be undefined in pool_undefine not pool_delete

virt: handle build differently depending on the pool type

virt: don't fail if the pool secret has been removed

* PR #57396 backport

add firmware auto select feature

* virt: Update dependencies

Closes: #57641

Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>

* use null in sls file to map None object

add sls file example

reword doc

* Update virt module and states and their tests to python3

* PR #57545 backport

Move virt.init boot_dev parameter away from the kwargs

virt: handle boot device in virt.update()

virt: add boot_dev parameter to virt.running state

* PR #57431 backport

virt: Handle no available hypervisors

virt: Remove unused imports

* Blacken salt

* Add method to remove circular references in data objects and add test (#54930)

* Add method to remove circular references in data objects and add test

* remove trailing whitespace

* Blacken changed files

Co-authored-by: xeacott <jeacott@saltstack.com>
Co-authored-by: Frode Gundersen <fgundersen@saltstack.com>
Co-authored-by: Daniel A. Wozniak <dwozniak@saltstack.com>

* PR #58332 backport

virt: add debug log with VM XML definition

Add xmlutil.get_xml_node() helper function

Add salt.utils.data.get_value function

Add change_xml() function to xmlutil

virt.update: refactor the XML diffing code

virt.test_update: move some code to make test more readable

Co-authored-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>
Co-authored-by: Pedro Algarvio <pedro@algarvio.me>
Co-authored-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
Co-authored-by: Firefly <leevn2011@hotmail.com>
Co-authored-by: Blacken Salt <jenkins@saltstack.com>
Co-authored-by: Joe Eacott <31625359+xeacott@users.noreply.github.com>
Co-authored-by: xeacott <jeacott@saltstack.com>
Co-authored-by: Frode Gundersen <fgundersen@saltstack.com>
Co-authored-by: Daniel A. Wozniak <dwozniak@saltstack.com>
---
 changelog/56454.fixed                    |    1 +
 changelog/57544.added                    |    1 +
 changelog/58331.fixed                    |    1 +
 salt/modules/virt.py                     |  442 ++++---
 salt/states/virt.py                      |  171 ++-
 salt/templates/virt/libvirt_domain.jinja |    2 +-
 salt/utils/data.py                       |  976 +++++++++------
 salt/utils/xmlutil.py                    |  251 +++-
 tests/pytests/unit/utils/test_data.py    |   57 +
 tests/pytests/unit/utils/test_xmlutil.py |  169 +++
 tests/unit/modules/test_virt.py          |  218 ++--
 tests/unit/states/test_virt.py           |   98 +-
 tests/unit/utils/test_data.py            | 1399 ++++++++++++----------
 tests/unit/utils/test_xmlutil.py         |  164 +--
 14 files changed, 2588 insertions(+), 1362 deletions(-)
 create mode 100644 changelog/56454.fixed
 create mode 100644 changelog/57544.added
 create mode 100644 changelog/58331.fixed
 create mode 100644 tests/pytests/unit/utils/test_data.py
 create mode 100644 tests/pytests/unit/utils/test_xmlutil.py

diff --git a/changelog/56454.fixed b/changelog/56454.fixed
new file mode 100644
index 0000000000..978b4b6e03
--- /dev/null
+++ b/changelog/56454.fixed
@@ -0,0 +1 @@
+Better handle virt.pool_rebuild in virt.pool_running and virt.pool_defined states
diff --git a/changelog/57544.added b/changelog/57544.added
new file mode 100644
index 0000000000..52071cf2c7
--- /dev/null
+++ b/changelog/57544.added
@@ -0,0 +1 @@
+Allow setting VM boot devices order in virt.running and virt.defined states
diff --git a/changelog/58331.fixed b/changelog/58331.fixed
new file mode 100644
index 0000000000..4b8f78dd53
--- /dev/null
+++ b/changelog/58331.fixed
@@ -0,0 +1 @@
+Leave boot parameters untouched if boot parameter is set to None in virt.update
diff --git a/salt/modules/virt.py b/salt/modules/virt.py
index a78c21e323..cd80fbe608 100644
--- a/salt/modules/virt.py
+++ b/salt/modules/virt.py
@@ -1,8 +1,11 @@
-# -*- coding: utf-8 -*-
 """
 Work with virtual machines managed by libvirt
 
-:depends: libvirt Python module
+:depends:
+    * libvirt Python module
+    * libvirt client
+    * qemu-img
+    * grep
 
 Connection
 ==========
@@ -73,7 +76,7 @@ The calls not using the libvirt connection setup are:
 # of his in the virt func module have been used
 
 # Import python libs
-from __future__ import absolute_import, print_function, unicode_literals
+
 import base64
 import copy
 import datetime
@@ -89,23 +92,19 @@ from xml.etree import ElementTree
 from xml.sax import saxutils
 
 # Import third party libs
-import jinja2
 import jinja2.exceptions
 
 # Import salt libs
+import salt.utils.data
 import salt.utils.files
 import salt.utils.json
-import salt.utils.network
 import salt.utils.path
 import salt.utils.stringutils
 import salt.utils.templates
-import salt.utils.validate.net
-import salt.utils.versions
 import salt.utils.xmlutil as xmlutil
 import salt.utils.yaml
 from salt._compat import ipaddress
 from salt.exceptions import CommandExecutionError, SaltInvocationError
-from salt.ext import six
 from salt.ext.six.moves import range  # pylint: disable=import-error,redefined-builtin
 from salt.ext.six.moves.urllib.parse import urlparse, urlunparse
 from salt.utils.virt import check_remote, download_remote
@@ -227,8 +226,8 @@ def __get_conn(**kwargs):
         )
     except Exception:  # pylint: disable=broad-except
         raise CommandExecutionError(
-            "Sorry, {0} failed to open a connection to the hypervisor "
-            "software at {1}".format(__grains__["fqdn"], conn_str)
+            "Sorry, {} failed to open a connection to the hypervisor "
+            "software at {}".format(__grains__["fqdn"], conn_str)
         )
     return conn
 
@@ -405,7 +404,7 @@ def _get_nics(dom):
             # driver, source, and match can all have optional attributes
             if re.match("(driver|source|address)", v_node.tag):
                 temp = {}
-                for key, value in six.iteritems(v_node.attrib):
+                for key, value in v_node.attrib.items():
                     temp[key] = value
                 nic[v_node.tag] = temp
             # virtualport needs to be handled separately, to pick up the
@@ -413,7 +412,7 @@ def _get_nics(dom):
             if v_node.tag == "virtualport":
                 temp = {}
                 temp["type"] = v_node.get("type")
-                for key, value in six.iteritems(v_node.attrib):
+                for key, value in v_node.attrib.items():
                     temp[key] = value
                 nic["virtualport"] = temp
         if "mac" not in nic:
@@ -435,7 +434,7 @@ def _get_graphics(dom):
     }
     doc = ElementTree.fromstring(dom.XMLDesc(0))
     for g_node in doc.findall("devices/graphics"):
-        for key, value in six.iteritems(g_node.attrib):
+        for key, value in g_node.attrib.items():
             out[key] = value
     return out
 
@@ -448,7 +447,7 @@ def _get_loader(dom):
     doc = ElementTree.fromstring(dom.XMLDesc(0))
     for g_node in doc.findall("os/loader"):
         out["path"] = g_node.text
-        for key, value in six.iteritems(g_node.attrib):
+        for key, value in g_node.attrib.items():
             out[key] = value
     return out
 
@@ -503,7 +502,7 @@ def _get_disks(conn, dom):
                 qemu_target = source.get("protocol")
                 source_name = source.get("name")
                 if source_name:
-                    qemu_target = "{0}:{1}".format(qemu_target, source_name)
+                    qemu_target = "{}:{}".format(qemu_target, source_name)
 
                 # Reverse the magic for the rbd and gluster pools
                 if source.get("protocol") in ["rbd", "gluster"]:
@@ -633,7 +632,7 @@ def _get_target(target, ssh):
     proto = "qemu"
     if ssh:
         proto += "+ssh"
-    return " {0}://{1}/{2}".format(proto, target, "system")
+    return " {}://{}/{}".format(proto, target, "system")
 
 
 def _gen_xml(
@@ -648,6 +647,7 @@ def _gen_xml(
     arch,
     graphics=None,
     boot=None,
+    boot_dev=None,
     **kwargs
 ):
     """
@@ -657,8 +657,8 @@ def _gen_xml(
     context = {
         "hypervisor": hypervisor,
         "name": name,
-        "cpu": six.text_type(cpu),
-        "mem": six.text_type(mem),
+        "cpu": str(cpu),
+        "mem": str(mem),
     }
     if hypervisor in ["qemu", "kvm"]:
         context["controller_model"] = False
@@ -681,15 +681,17 @@ def _gen_xml(
             graphics = None
     context["graphics"] = graphics
 
-    if "boot_dev" in kwargs:
-        context["boot_dev"] = []
-        for dev in kwargs["boot_dev"].split():
-            context["boot_dev"].append(dev)
-    else:
-        context["boot_dev"] = ["hd"]
+    context["boot_dev"] = boot_dev.split() if boot_dev is not None else ["hd"]
 
     context["boot"] = boot if boot else {}
 
+    # if efi parameter is specified, prepare os_attrib
+    efi_value = context["boot"].get("efi", None) if boot else None
+    if efi_value is True:
+        context["boot"]["os_attrib"] = "firmware='efi'"
+    elif efi_value is not None and type(efi_value) != bool:
+        raise SaltInvocationError("Invalid efi value")
+
     if os_type == "xen":
         # Compute the Xen PV boot method
         if __grains__["os_family"] == "Suse":
@@ -720,7 +722,7 @@ def _gen_xml(
             "target_dev": _get_disk_target(targets, len(diskp), prefix),
             "disk_bus": disk["model"],
             "format": disk.get("format", "raw"),
-            "index": six.text_type(i),
+            "index": str(i),
         }
         targets.append(disk_context["target_dev"])
         if disk.get("source_file"):
@@ -825,8 +827,8 @@ def _gen_vol_xml(
         "name": name,
         "target": {"permissions": permissions, "nocow": nocow},
         "format": format,
-        "size": six.text_type(size),
-        "allocation": six.text_type(int(allocation) * 1024),
+        "size": str(size),
+        "allocation": str(int(allocation) * 1024),
         "backingStore": backing_store,
     }
     fn_ = "libvirt_volume.jinja"
@@ -978,31 +980,29 @@ def _zfs_image_create(
     """
     if not disk_image_name and not disk_size:
         raise CommandExecutionError(
-            "Unable to create new disk {0}, please specify"
+            "Unable to create new disk {}, please specify"
             " the disk image name or disk size argument".format(disk_name)
         )
 
     if not pool:
         raise CommandExecutionError(
-            "Unable to create new disk {0}, please specify"
+            "Unable to create new disk {}, please specify"
             " the disk pool name".format(disk_name)
         )
 
-    destination_fs = os.path.join(pool, "{0}.{1}".format(vm_name, disk_name))
+    destination_fs = os.path.join(pool, "{}.{}".format(vm_name, disk_name))
     log.debug("Image destination will be %s", destination_fs)
 
     existing_disk = __salt__["zfs.list"](name=pool)
     if "error" in existing_disk:
         raise CommandExecutionError(
-            "Unable to create new disk {0}. {1}".format(
+            "Unable to create new disk {}. {}".format(
                 destination_fs, existing_disk["error"]
             )
         )
     elif destination_fs in existing_disk:
         log.info(
-            "ZFS filesystem {0} already exists. Skipping creation".format(
-                destination_fs
-            )
+            "ZFS filesystem {} already exists. Skipping creation".format(destination_fs)
         )
         blockdevice_path = os.path.join("/dev/zvol", pool, vm_name)
         return blockdevice_path
@@ -1025,7 +1025,7 @@ def _zfs_image_create(
         )
 
     blockdevice_path = os.path.join(
-        "/dev/zvol", pool, "{0}.{1}".format(vm_name, disk_name)
+        "/dev/zvol", pool, "{}.{}".format(vm_name, disk_name)
     )
     log.debug("Image path will be %s", blockdevice_path)
     return blockdevice_path
@@ -1042,7 +1042,7 @@ def _qemu_image_create(disk, create_overlay=False, saltenv="base"):
 
     if not disk_size and not disk_image:
         raise CommandExecutionError(
-            "Unable to create new disk {0}, please specify"
+            "Unable to create new disk {}, please specify"
             " disk size and/or disk image argument".format(disk["filename"])
         )
 
@@ -1066,7 +1066,7 @@ def _qemu_image_create(disk, create_overlay=False, saltenv="base"):
             if create_overlay and qcow2:
                 log.info("Cloning qcow2 image %s using copy on write", sfn)
                 __salt__["cmd.run"](
-                    'qemu-img create -f qcow2 -o backing_file="{0}" "{1}"'.format(
+                    'qemu-img create -f qcow2 -o backing_file="{}" "{}"'.format(
                         sfn, img_dest
                     ).split()
                 )
@@ -1079,16 +1079,16 @@ def _qemu_image_create(disk, create_overlay=False, saltenv="base"):
             if disk_size and qcow2:
                 log.debug("Resize qcow2 image to %sM", disk_size)
                 __salt__["cmd.run"](
-                    'qemu-img resize "{0}" {1}M'.format(img_dest, disk_size)
+                    'qemu-img resize "{}" {}M'.format(img_dest, disk_size)
                 )
 
             log.debug("Apply umask and remove exec bit")
             mode = (0o0777 ^ mask) & 0o0666
             os.chmod(img_dest, mode)
 
-        except (IOError, OSError) as err:
+        except OSError as err:
             raise CommandExecutionError(
-                "Problem while copying image. {0} - {1}".format(disk_image, err)
+                "Problem while copying image. {} - {}".format(disk_image, err)
             )
 
     else:
@@ -1099,13 +1099,13 @@ def _qemu_image_create(disk, create_overlay=False, saltenv="base"):
             if disk_size:
                 log.debug("Create empty image with size %sM", disk_size)
                 __salt__["cmd.run"](
-                    'qemu-img create -f {0} "{1}" {2}M'.format(
+                    'qemu-img create -f {} "{}" {}M'.format(
                         disk.get("format", "qcow2"), img_dest, disk_size
                     )
                 )
             else:
                 raise CommandExecutionError(
-                    "Unable to create new disk {0},"
+                    "Unable to create new disk {},"
                     " please specify <size> argument".format(img_dest)
                 )
 
@@ -1113,9 +1113,9 @@ def _qemu_image_create(disk, create_overlay=False, saltenv="base"):
             mode = (0o0777 ^ mask) & 0o0666
             os.chmod(img_dest, mode)
 
-        except (IOError, OSError) as err:
+        except OSError as err:
             raise CommandExecutionError(
-                "Problem while creating volume {0} - {1}".format(img_dest, err)
+                "Problem while creating volume {} - {}".format(img_dest, err)
             )
 
     return img_dest
@@ -1252,7 +1252,7 @@ def _disk_profile(conn, profile, hypervisor, disks, vm_name):
             __salt__["config.get"]("virt:disk", {}).get(profile, default)
         )
 
-        # Transform the list to remove one level of dictionnary and add the name as a property
+        # Transform the list to remove one level of dictionary and add the name as a property
         disklist = [dict(d, name=name) for disk in disklist for name, d in disk.items()]
 
     # Merge with the user-provided disks definitions
@@ -1274,7 +1274,7 @@ def _disk_profile(conn, profile, hypervisor, disks, vm_name):
             disk["model"] = "ide"
 
         # Add the missing properties that have defaults
-        for key, val in six.iteritems(overlay):
+        for key, val in overlay.items():
             if key not in disk:
                 disk[key] = val
 
@@ -1296,7 +1296,7 @@ def _fill_disk_filename(conn, vm_name, disk, hypervisor, pool_caps):
     Compute the disk file name and update it in the disk value.
     """
     # Compute the filename without extension since it may not make sense for some pool types
-    disk["filename"] = "{0}_{1}".format(vm_name, disk["name"])
+    disk["filename"] = "{}_{}".format(vm_name, disk["name"])
 
     # Compute the source file path
     base_dir = disk.get("pool", None)
@@ -1311,7 +1311,7 @@ def _fill_disk_filename(conn, vm_name, disk, hypervisor, pool_caps):
             # For path-based disks, keep the qcow2 default format
             if not disk.get("format"):
                 disk["format"] = "qcow2"
-            disk["filename"] = "{0}.{1}".format(disk["filename"], disk["format"])
+            disk["filename"] = "{}.{}".format(disk["filename"], disk["format"])
             disk["source_file"] = os.path.join(base_dir, disk["filename"])
         else:
             if "pool" not in disk:
@@ -1365,7 +1365,7 @@ def _fill_disk_filename(conn, vm_name, disk, hypervisor, pool_caps):
                     disk["format"] = volume_options.get("default_format", None)
 
     elif hypervisor == "bhyve" and vm_name:
-        disk["filename"] = "{0}.{1}".format(vm_name, disk["name"])
+        disk["filename"] = "{}.{}".format(vm_name, disk["name"])
         disk["source_file"] = os.path.join(
             "/dev/zvol", base_dir or "", disk["filename"]
         )
@@ -1373,8 +1373,8 @@ def _fill_disk_filename(conn, vm_name, disk, hypervisor, pool_caps):
     elif hypervisor in ["esxi", "vmware"]:
         if not base_dir:
             base_dir = __salt__["config.get"]("virt:storagepool", "[0] ")
-        disk["filename"] = "{0}.{1}".format(disk["filename"], disk["format"])
-        disk["source_file"] = "{0}{1}".format(base_dir, disk["filename"])
+        disk["filename"] = "{}.{}".format(disk["filename"], disk["format"])
+        disk["source_file"] = "{}{}".format(base_dir, disk["filename"])
 
 
 def _complete_nics(interfaces, hypervisor):
@@ -1422,7 +1422,7 @@ def _complete_nics(interfaces, hypervisor):
         """
         Apply the default overlay to attributes
         """
-        for key, value in six.iteritems(overlays[hypervisor]):
+        for key, value in overlays[hypervisor].items():
             if key not in attributes or not attributes[key]:
                 attributes[key] = value
 
@@ -1449,7 +1449,7 @@ def _nic_profile(profile_name, hypervisor):
         """
         Append dictionary profile data to interfaces list
         """
-        for interface_name, attributes in six.iteritems(profile_dict):
+        for interface_name, attributes in profile_dict.items():
             attributes["name"] = interface_name
             interfaces.append(attributes)
 
@@ -1522,17 +1522,24 @@ def _handle_remote_boot_params(orig_boot):
     new_boot = orig_boot.copy()
     keys = orig_boot.keys()
     cases = [
+        {"efi"},
+        {"kernel", "initrd", "efi"},
+        {"kernel", "initrd", "cmdline", "efi"},
         {"loader", "nvram"},
         {"kernel", "initrd"},
         {"kernel", "initrd", "cmdline"},
-        {"loader", "nvram", "kernel", "initrd"},
-        {"loader", "nvram", "kernel", "initrd", "cmdline"},
+        {"kernel", "initrd", "loader", "nvram"},
+        {"kernel", "initrd", "cmdline", "loader", "nvram"},
     ]
 
     try:
         if keys in cases:
             for key in keys:
-                if orig_boot.get(key) is not None and check_remote(orig_boot.get(key)):
+                if key == "efi" and type(orig_boot.get(key)) == bool:
+                    new_boot[key] = orig_boot.get(key)
+                elif orig_boot.get(key) is not None and check_remote(
+                    orig_boot.get(key)
+                ):
                     if saltinst_dir is None:
                         os.makedirs(CACHE_DIR)
                         saltinst_dir = CACHE_DIR
@@ -1540,12 +1547,41 @@ def _handle_remote_boot_params(orig_boot):
             return new_boot
         else:
             raise SaltInvocationError(
-                "Invalid boot parameters, (kernel, initrd) or/and (loader, nvram) must be both present"
+                "Invalid boot parameters,It has to follow this combination: [(kernel, initrd) or/and cmdline] or/and [(loader, nvram) or efi]"
             )
     except Exception as err:  # pylint: disable=broad-except
         raise err
 
 
+def _handle_efi_param(boot, desc):
+    """
+    Checks if boot parameter contains efi boolean value, if so, handles the firmware attribute.
+    :param boot: The boot parameters passed to the init or update functions.
+    :param desc: The XML description of that domain.
+    :return: A boolean value.
+    """
+    efi_value = boot.get("efi", None) if boot else None
+    parent_tag = desc.find("os")
+    os_attrib = parent_tag.attrib
+
+    # newly defined vm without running, loader tag might not be filled yet
+    if efi_value is False and os_attrib != {}:
+        parent_tag.attrib.pop("firmware", None)
+        return True
+
+    # check the case that loader tag might be present. This happens after the vm ran
+    elif type(efi_value) == bool and os_attrib == {}:
+        if efi_value is True and parent_tag.find("loader") is None:
+            parent_tag.set("firmware", "efi")
+        if efi_value is False and parent_tag.find("loader") is not None:
+            parent_tag.remove(parent_tag.find("loader"))
+            parent_tag.remove(parent_tag.find("nvram"))
+        return True
+    elif type(efi_value) != bool:
+        raise SaltInvocationError("Invalid efi value")
+    return False
+
+
 def init(
     name,
     cpu,
@@ -1566,6 +1602,7 @@ def init(
     os_type=None,
     arch=None,
     boot=None,
+    boot_dev=None,
     **kwargs
 ):
     """
@@ -1635,7 +1672,8 @@ def init(
         This is an optional parameter, all of the keys are optional within the dictionary. The structure of
         the dictionary is documented in :ref:`init-boot-def`. If a remote path is provided to kernel or initrd,
         salt will handle the downloading of the specified remote file and modify the XML accordingly.
-        To boot VM with UEFI, specify loader and nvram path.
+        To boot VM with UEFI, specify loader and nvram path or specify 'efi': ``True`` if your libvirtd version
+        is >= 5.2.0 and QEMU >= 3.0.0.
 
         .. versionadded:: 3000
 
@@ -1649,6 +1687,12 @@ def init(
                 'nvram': '/usr/share/OVMF/OVMF_VARS.ms.fd'
             }
 
+    :param boot_dev:
+        Space separated list of devices to boot from sorted by decreasing priority.
+        Values can be ``hd``, ``fd``, ``cdrom`` or ``network``.
+
+        By default, the value will ``"hd"``.
+
     .. _init-boot-def:
 
     .. rubric:: Boot parameters definition
@@ -1674,6 +1718,11 @@ def init(
 
         .. versionadded:: sodium
 
+    efi
+       A boolean value.
+
+       .. versionadded:: sodium
+
     .. _init-nic-def:
 
     .. rubric:: Network Interfaces Definitions
@@ -1797,7 +1846,7 @@ def init(
 
     .. rubric:: Graphics Definition
 
-    The graphics dictionnary can have the following properties:
+    The graphics dictionary can have the following properties:
 
     type
         Graphics type. The possible values are ``none``, ``'spice'``, ``'vnc'`` and other values
@@ -1858,6 +1907,8 @@ def init(
                     for x in y
                 }
             )
+            if len(hypervisors) == 0:
+                raise SaltInvocationError("No supported hypervisors were found")
             virt_hypervisor = "kvm" if "kvm" in hypervisors else hypervisors[0]
 
         # esxi used to be a possible value for the hypervisor: map it to vmware since it's the same
@@ -1890,8 +1941,8 @@ def init(
                 else:
                     # assume libvirt manages disks for us
                     log.debug("Generating libvirt XML for %s", _disk)
-                    volume_name = "{0}/{1}".format(name, _disk["name"])
-                    filename = "{0}.{1}".format(volume_name, _disk["format"])
+                    volume_name = "{}/{}".format(name, _disk["name"])
+                    filename = "{}.{}".format(volume_name, _disk["format"])
                     vol_xml = _gen_vol_xml(
                         filename, _disk["size"], format=_disk["format"]
                     )
@@ -1939,7 +1990,7 @@ def init(
             else:
                 # Unknown hypervisor
                 raise SaltInvocationError(
-                    "Unsupported hypervisor when handling disk image: {0}".format(
+                    "Unsupported hypervisor when handling disk image: {}".format(
                         virt_hypervisor
                     )
                 )
@@ -1965,8 +2016,10 @@ def init(
             arch,
             graphics,
             boot,
+            boot_dev,
             **kwargs
         )
+        log.debug("New virtual machine definition: %s", vm_xml)
         conn.defineXML(vm_xml)
     except libvirt.libvirtError as err:
         conn.close()
@@ -2192,6 +2245,7 @@ def update(
     live=True,
     boot=None,
     test=False,
+    boot_dev=None,
     **kwargs
 ):
     """
@@ -2234,11 +2288,28 @@ def update(
 
         Refer to :ref:`init-boot-def` for the complete boot parameter description.
 
-        To update any boot parameters, specify the new path for each. To remove any boot parameters,
-        pass a None object, for instance: 'kernel': ``None``.
+        To update any boot parameters, specify the new path for each. To remove any boot parameters, pass ``None`` object,
+        for instance: 'kernel': ``None``. To switch back to BIOS boot, specify ('loader': ``None`` and 'nvram': ``None``)
+        or 'efi': ``False``. Please note that ``None`` is mapped to ``null`` in sls file, pass ``null`` in sls file instead.
+
+        SLS file Example:
+
+        .. code-block:: yaml
+
+            - boot:
+                loader: null
+                nvram: null
 
         .. versionadded:: 3000
 
+    :param boot_dev:
+        Space separated list of devices to boot from sorted by decreasing priority.
+        Values can be ``hd``, ``fd``, ``cdrom`` or ``network``.
+
+        By default, the value will ``"hd"``.
+
+        .. versionadded:: Magnesium
+
     :param test: run in dry-run mode if set to True
 
         .. versionadded:: sodium
@@ -2286,6 +2357,8 @@ def update(
 
     if boot is not None:
         boot = _handle_remote_boot_params(boot)
+        if boot.get("efi", None) is not None:
+            need_update = _handle_efi_param(boot, desc)
 
     new_desc = ElementTree.fromstring(
         _gen_xml(
@@ -2307,76 +2380,58 @@ def update(
     # Update the cpu
     cpu_node = desc.find("vcpu")
     if cpu and int(cpu_node.text) != cpu:
-        cpu_node.text = six.text_type(cpu)
-        cpu_node.set("current", six.text_type(cpu))
+        cpu_node.text = str(cpu)
+        cpu_node.set("current", str(cpu))
         need_update = True
 
-    # Update the kernel boot parameters
-    boot_tags = ["kernel", "initrd", "cmdline", "loader", "nvram"]
-    parent_tag = desc.find("os")
-
-    # We need to search for each possible subelement, and update it.
-    for tag in boot_tags:
-        # The Existing Tag...
-        found_tag = parent_tag.find(tag)
-
-        # The new value
-        boot_tag_value = boot.get(tag, None) if boot else None
-
-        # Existing tag is found and values don't match
-        if found_tag is not None and found_tag.text != boot_tag_value:
-
-            # If the existing tag is found, but the new value is None
-            # remove it. If the existing tag is found, and the new value
-            # doesn't match update it. In either case, mark for update.
-            if boot_tag_value is None and boot is not None and parent_tag is not None:
-                parent_tag.remove(found_tag)
-            else:
-                found_tag.text = boot_tag_value
+    def _set_loader(node, value):
+        salt.utils.xmlutil.set_node_text(node, value)
+        if value is not None:
+            node.set("readonly", "yes")
+            node.set("type", "pflash")
 
-            # If the existing tag is loader or nvram, we need to update the corresponding attribute
-            if found_tag.tag == "loader" and boot_tag_value is not None:
-                found_tag.set("readonly", "yes")
-                found_tag.set("type", "pflash")
+    def _set_nvram(node, value):
+        node.set("template", value)
 
-            if found_tag.tag == "nvram" and boot_tag_value is not None:
-                found_tag.set("template", found_tag.text)
-                found_tag.text = None
+    def _set_with_mib_unit(node, value):
+        node.text = str(value)
+        node.set("unit", "MiB")
 
-            need_update = True
-
-        # Existing tag is not found, but value is not None
-        elif found_tag is None and boot_tag_value is not None:
-
-            # Need to check for parent tag, and add it if it does not exist.
-            # Add a subelement and set the value to the new value, and then
-            # mark for update.
-            if parent_tag is not None:
-                child_tag = ElementTree.SubElement(parent_tag, tag)
-            else:
-                new_parent_tag = ElementTree.Element("os")
-                child_tag = ElementTree.SubElement(new_parent_tag, tag)
-
-            child_tag.text = boot_tag_value
-
-            # If the newly created tag is loader or nvram, we need to update the corresponding attribute
-            if child_tag.tag == "loader":
-                child_tag.set("readonly", "yes")
-                child_tag.set("type", "pflash")
-
-            if child_tag.tag == "nvram":
-                child_tag.set("template", child_tag.text)
-                child_tag.text = None
-
-            need_update = True
+    # Update the kernel boot parameters
+    params_mapping = [
+        {"path": "boot:kernel", "xpath": "os/kernel"},
+        {"path": "boot:initrd", "xpath": "os/initrd"},
+        {"path": "boot:cmdline", "xpath": "os/cmdline"},
+        {"path": "boot:loader", "xpath": "os/loader", "set": _set_loader},
+        {"path": "boot:nvram", "xpath": "os/nvram", "set": _set_nvram},
+        # Update the memory, note that libvirt outputs all memory sizes in KiB
+        {
+            "path": "mem",
+            "xpath": "memory",
+            "get": lambda n: int(n.text) / 1024,
+            "set": _set_with_mib_unit,
+        },
+        {
+            "path": "mem",
+            "xpath": "currentMemory",
+            "get": lambda n: int(n.text) / 1024,
+            "set": _set_with_mib_unit,
+        },
+        {
+            "path": "boot_dev:{dev}",
+            "xpath": "os/boot[$dev]",
+            "get": lambda n: n.get("dev"),
+            "set": lambda n, v: n.set("dev", v),
+            "del": salt.utils.xmlutil.del_attribute("dev"),
+        },
+    ]
 
-    # Update the memory, note that libvirt outputs all memory sizes in KiB
-    for mem_node_name in ["memory", "currentMemory"]:
-        mem_node = desc.find(mem_node_name)
-        if mem and int(mem_node.text) != mem * 1024:
-            mem_node.text = six.text_type(mem)
-            mem_node.set("unit", "MiB")
-            need_update = True
+    data = {k: v for k, v in locals().items() if bool(v)}
+    if boot_dev:
+        data["boot_dev"] = {i + 1: dev for i, dev in enumerate(boot_dev.split())}
+    need_update = need_update or salt.utils.xmlutil.change_xml(
+        desc, data, params_mapping
+    )
 
     # Update the XML definition with the new disks and diff changes
     devices_node = desc.find("devices")
@@ -2395,8 +2450,8 @@ def update(
             if func_locals.get(param, None) is not None
         ]:
             old = devices_node.findall(dev_type)
-            new = new_desc.findall("devices/{0}".format(dev_type))
-            changes[dev_type] = globals()["_diff_{0}_lists".format(dev_type)](old, new)
+            new = new_desc.findall("devices/{}".format(dev_type))
+            changes[dev_type] = globals()["_diff_{}_lists".format(dev_type)](old, new)
             if changes[dev_type]["deleted"] or changes[dev_type]["new"]:
                 for item in old:
                     devices_node.remove(item)
@@ -2423,9 +2478,9 @@ def update(
                         _disk_volume_create(conn, all_disks[idx])
 
             if not test:
-                conn.defineXML(
-                    salt.utils.stringutils.to_str(ElementTree.tostring(desc))
-                )
+                xml_desc = ElementTree.tostring(desc)
+                log.debug("Update virtual machine definition: %s", xml_desc)
+                conn.defineXML(salt.utils.stringutils.to_str(xml_desc))
             status["definition"] = True
         except libvirt.libvirtError as err:
             conn.close()
@@ -2554,7 +2609,7 @@ def update(
             except libvirt.libvirtError as err:
                 if "errors" not in status:
                     status["errors"] = []
-                status["errors"].append(six.text_type(err))
+                status["errors"].append(str(err))
 
     conn.close()
     return status
@@ -2768,7 +2823,7 @@ def _node_info(conn):
     info = {
         "cpucores": raw[6],
         "cpumhz": raw[3],
-        "cpumodel": six.text_type(raw[0]),
+        "cpumodel": str(raw[0]),
         "cpus": raw[2],
         "cputhreads": raw[7],
         "numanodes": raw[4],
@@ -3207,24 +3262,21 @@ def get_profiles(hypervisor=None, **kwargs):
             for x in y
         }
     )
-    default_hypervisor = "kvm" if "kvm" in hypervisors else hypervisors[0]
+    if len(hypervisors) == 0:
+        raise SaltInvocationError("No supported hypervisors were found")
 
     if not hypervisor:
-        hypervisor = default_hypervisor
+        hypervisor = "kvm" if "kvm" in hypervisors else hypervisors[0]
     virtconf = __salt__["config.get"]("virt", {})
     for typ in ["disk", "nic"]:
-        _func = getattr(sys.modules[__name__], "_{0}_profile".format(typ))
+        _func = getattr(sys.modules[__name__], "_{}_profile".format(typ))
         ret[typ] = {
-            "default": _func(
-                "default", hypervisor if hypervisor else default_hypervisor
-            )
+            "default": _func("default", hypervisor)
         }
         if typ in virtconf:
             ret.setdefault(typ, {})
             for prf in virtconf[typ]:
-                ret[typ][prf] = _func(
-                    prf, hypervisor if hypervisor else default_hypervisor
-                )
+                ret[typ][prf] = _func(prf, hypervisor)
     return ret
 
 
@@ -3506,7 +3558,7 @@ def create_xml_path(path, **kwargs):
             return create_xml_str(
                 salt.utils.stringutils.to_unicode(fp_.read()), **kwargs
             )
-    except (OSError, IOError):
+    except OSError:
         return False
 
 
@@ -3564,7 +3616,7 @@ def define_xml_path(path, **kwargs):
             return define_xml_str(
                 salt.utils.stringutils.to_unicode(fp_.read()), **kwargs
             )
-    except (OSError, IOError):
+    except OSError:
         return False
 
 
@@ -3576,7 +3628,7 @@ def _define_vol_xml_str(conn, xml, pool=None):  # pylint: disable=redefined-oute
     poolname = (
         pool if pool else __salt__["config.get"]("virt:storagepool", default_pool)
     )
-    pool = conn.storagePoolLookupByName(six.text_type(poolname))
+    pool = conn.storagePoolLookupByName(str(poolname))
     ret = pool.createXML(xml, 0) is not None
     return ret
 
@@ -3660,7 +3712,7 @@ def define_vol_xml_path(path, pool=None, **kwargs):
             return define_vol_xml_str(
                 salt.utils.stringutils.to_unicode(fp_.read()), pool=pool, **kwargs
             )
-    except (OSError, IOError):
+    except OSError:
         return False
 
 
@@ -3777,7 +3829,7 @@ def seed_non_shared_migrate(disks, force=False):
 
         salt '*' virt.seed_non_shared_migrate <disks>
     """
-    for _, data in six.iteritems(disks):
+    for _, data in disks.items():
         fn_ = data["file"]
         form = data["file format"]
         size = data["virtual size"].split()[1][1:]
@@ -3921,14 +3973,14 @@ def purge(vm_, dirs=False, removables=False, **kwargs):
             # TODO create solution for 'dataset is busy'
             time.sleep(3)
             fs_name = disks[disk]["file"][len("/dev/zvol/") :]
-            log.info("Destroying VM ZFS volume {0}".format(fs_name))
+            log.info("Destroying VM ZFS volume {}".format(fs_name))
             __salt__["zfs.destroy"](name=fs_name, force=True)
         elif os.path.exists(disks[disk]["file"]):
             os.remove(disks[disk]["file"])
             directories.add(os.path.dirname(disks[disk]["file"]))
         else:
             # We may have a volume to delete here
-            matcher = re.match("^(?P<pool>[^/]+)/(?P<volume>.*)$", disks[disk]["file"],)
+            matcher = re.match("^(?P<pool>[^/]+)/(?P<volume>.*)$", disks[disk]["file"])
             if matcher:
                 pool_name = matcher.group("pool")
                 pool = None
@@ -3975,7 +4027,7 @@ def _is_kvm_hyper():
         with salt.utils.files.fopen("/proc/modules") as fp_:
             if "kvm_" not in salt.utils.stringutils.to_unicode(fp_.read()):
                 return False
-    except IOError:
+    except OSError:
         # No /proc/modules? Are we on Windows? Or Solaris?
         return False
     return "libvirtd" in __salt__["cmd.run"](__grains__["ps"])
@@ -3995,7 +4047,7 @@ def _is_xen_hyper():
         with salt.utils.files.fopen("/proc/modules") as fp_:
             if "xen_" not in salt.utils.stringutils.to_unicode(fp_.read()):
                 return False
-    except (OSError, IOError):
+    except OSError:
         # No /proc/modules? Are we on Windows? Or Solaris?
         return False
     return "libvirtd" in __salt__["cmd.run"](__grains__["ps"])
@@ -4110,7 +4162,7 @@ def vm_cputime(vm_=None, **kwargs):
             cputime_percent = (1.0e-7 * cputime / host_cpus) / vcpus
         return {
             "cputime": int(raw[4]),
-            "cputime_percent": int("{0:.0f}".format(cputime_percent)),
+            "cputime_percent": int("{:.0f}".format(cputime_percent)),
         }
 
     info = {}
@@ -4180,7 +4232,7 @@ def vm_netstats(vm_=None, **kwargs):
             "tx_errs": 0,
             "tx_drop": 0,
         }
-        for attrs in six.itervalues(nics):
+        for attrs in nics.values():
             if "target" in attrs:
                 dev = attrs["target"]
                 stats = dom.interfaceStats(dev)
@@ -4508,7 +4560,7 @@ def revert_snapshot(name, vm_snapshot=None, cleanup=False, **kwargs):
         conn.close()
         raise CommandExecutionError(
             snapshot
-            and 'Snapshot "{0}" not found'.format(vm_snapshot)
+            and 'Snapshot "{}" not found'.format(vm_snapshot)
             or "No more previous snapshots available"
         )
     elif snap.isCurrent():
@@ -5102,10 +5154,10 @@ def cpu_baseline(full=False, migratable=False, out="libvirt", **kwargs):
             ]
 
             if not cpu_specs:
-                raise ValueError("Model {0} not found in CPU map".format(cpu_model))
+                raise ValueError("Model {} not found in CPU map".format(cpu_model))
             elif len(cpu_specs) > 1:
                 raise ValueError(
-                    "Multiple models {0} found in CPU map".format(cpu_model)
+                    "Multiple models {} found in CPU map".format(cpu_model)
                 )
 
             cpu_specs = cpu_specs[0]
@@ -5126,7 +5178,7 @@ def cpu_baseline(full=False, migratable=False, out="libvirt", **kwargs):
             "vendor": cpu.find("vendor").text,
             "features": [feature.get("name") for feature in cpu.findall("feature")],
         }
-    return cpu.toxml()
+    return ElementTree.tostring(cpu)
 
 
 def network_define(name, bridge, forward, ipv4_config=None, ipv6_config=None, **kwargs):
@@ -5250,7 +5302,7 @@ def list_networks(**kwargs):
 
 def network_info(name=None, **kwargs):
     """
-    Return informations on a virtual network provided its name.
+    Return information on a virtual network provided its name.
 
     :param name: virtual network name
     :param connection: libvirt connection URI, overriding defaults
@@ -5446,20 +5498,20 @@ def _parse_pools_caps(doc):
         for option_kind in ["pool", "vol"]:
             options = {}
             default_format_node = pool.find(
-                "{0}Options/defaultFormat".format(option_kind)
+                "{}Options/defaultFormat".format(option_kind)
             )
             if default_format_node is not None:
                 options["default_format"] = default_format_node.get("type")
             options_enums = {
                 enum.get("name"): [value.text for value in enum.findall("value")]
-                for enum in pool.findall("{0}Options/enum".format(option_kind))
+                for enum in pool.findall("{}Options/enum".format(option_kind))
             }
             if options_enums:
                 options.update(options_enums)
             if options:
                 if "options" not in pool_caps:
                     pool_caps["options"] = {}
-                kind = option_kind if option_kind is not "vol" else "volume"
+                kind = option_kind if option_kind != "vol" else "volume"
                 pool_caps["options"][kind] = options
         return pool_caps
 
@@ -5695,7 +5747,7 @@ def pool_define(
         keys. The path is the qualified name for iSCSI devices.
 
         Report to `this libvirt page <https://libvirt.org/formatstorage.html#StoragePool>`_
-        for more informations on the use of ``part_separator``
+        for more information on the use of ``part_separator``
     :param source_dir:
         Path to the source directory for pools of type ``dir``, ``netfs`` or ``gluster``.
         (Default: ``None``)
@@ -5847,15 +5899,19 @@ def _pool_set_secret(
         if secret_type:
             # Get the previously defined secret if any
             secret = None
-            if usage:
-                usage_type = (
-                    libvirt.VIR_SECRET_USAGE_TYPE_CEPH
-                    if secret_type == "ceph"
-                    else libvirt.VIR_SECRET_USAGE_TYPE_ISCSI
-                )
-                secret = conn.secretLookupByUsage(usage_type, usage)
-            elif uuid:
-                secret = conn.secretLookupByUUIDString(uuid)
+            try:
+                if usage:
+                    usage_type = (
+                        libvirt.VIR_SECRET_USAGE_TYPE_CEPH
+                        if secret_type == "ceph"
+                        else libvirt.VIR_SECRET_USAGE_TYPE_ISCSI
+                    )
+                    secret = conn.secretLookupByUsage(usage_type, usage)
+                elif uuid:
+                    secret = conn.secretLookupByUUIDString(uuid)
+            except libvirt.libvirtError as err:
+                # For some reason the secret has been removed. Don't fail since we'll recreate it
+                log.info("Secret not found: %s", err.get_error_message())
 
             # Create secret if needed
             if not secret:
@@ -5918,7 +5974,7 @@ def pool_update(
         keys. The path is the qualified name for iSCSI devices.
 
         Report to `this libvirt page <https://libvirt.org/formatstorage.html#StoragePool>`_
-        for more informations on the use of ``part_separator``
+        for more information on the use of ``part_separator``
     :param source_dir:
         Path to the source directory for pools of type ``dir``, ``netfs`` or ``gluster``.
         (Default: ``None``)
@@ -6107,7 +6163,7 @@ def list_pools(**kwargs):
 
 def pool_info(name=None, **kwargs):
     """
-    Return informations on a storage pool provided its name.
+    Return information on a storage pool provided its name.
 
     :param name: libvirt storage pool name
     :param connection: libvirt connection URI, overriding defaults
@@ -6283,6 +6339,22 @@ def pool_undefine(name, **kwargs):
     conn = __get_conn(**kwargs)
     try:
         pool = conn.storagePoolLookupByName(name)
+        desc = ElementTree.fromstring(pool.XMLDesc())
+
+        # Is there a secret that we generated and would need to be removed?
+        # Don't remove the other secrets
+        auth_node = desc.find("source/auth")
+        if auth_node is not None:
+            auth_types = {
+                "ceph": libvirt.VIR_SECRET_USAGE_TYPE_CEPH,
+                "iscsi": libvirt.VIR_SECRET_USAGE_TYPE_ISCSI,
+            }
+            secret_type = auth_types[auth_node.get("type")]
+            secret_usage = auth_node.find("secret").get("usage")
+            if secret_type and "pool_{}".format(name) == secret_usage:
+                secret = conn.secretLookupByUsage(secret_type, secret_usage)
+                secret.undefine()
+
         return not bool(pool.undefine())
     finally:
         conn.close()
@@ -6308,22 +6380,6 @@ def pool_delete(name, **kwargs):
     conn = __get_conn(**kwargs)
     try:
         pool = conn.storagePoolLookupByName(name)
-        desc = ElementTree.fromstring(pool.XMLDesc())
-
-        # Is there a secret that we generated and would need to be removed?
-        # Don't remove the other secrets
-        auth_node = desc.find("source/auth")
-        if auth_node is not None:
-            auth_types = {
-                "ceph": libvirt.VIR_SECRET_USAGE_TYPE_CEPH,
-                "iscsi": libvirt.VIR_SECRET_USAGE_TYPE_ISCSI,
-            }
-            secret_type = auth_types[auth_node.get("type")]
-            secret_usage = auth_node.find("secret").get("usage")
-            if secret_type and "pool_{}".format(name) == secret_usage:
-                secret = conn.secretLookupByUsage(secret_type, secret_usage)
-                secret.undefine()
-
         return not bool(pool.delete(libvirt.VIR_STORAGE_POOL_DELETE_NORMAL))
     finally:
         conn.close()
@@ -6768,7 +6824,7 @@ def _volume_upload(conn, pool, volume, file, offset=0, length=0, sparse=False):
                     stream.abort()
                 if ret:
                     raise CommandExecutionError(
-                        "Failed to close file: {0}".format(err.strerror)
+                        "Failed to close file: {}".format(err.strerror)
                     )
         if stream:
             try:
@@ -6776,7 +6832,7 @@ def _volume_upload(conn, pool, volume, file, offset=0, length=0, sparse=False):
             except libvirt.libvirtError as err:
                 if ret:
                     raise CommandExecutionError(
-                        "Failed to finish stream: {0}".format(err.get_error_message())
+                        "Failed to finish stream: {}".format(err.get_error_message())
                     )
     return ret
 
diff --git a/salt/states/virt.py b/salt/states/virt.py
index fdef002293..3d99fd53c8 100644
--- a/salt/states/virt.py
+++ b/salt/states/virt.py
@@ -1,4 +1,3 @@
-# -*- coding: utf-8 -*-
 """
 Manage virt
 ===========
@@ -13,9 +12,9 @@ for the generation and signing of certificates for systems running libvirt:
 """
 
 # Import Python libs
-from __future__ import absolute_import, print_function, unicode_literals
 
 import fnmatch
+import logging
 import os
 
 # Import Salt libs
@@ -25,9 +24,6 @@ import salt.utils.stringutils
 import salt.utils.versions
 from salt.exceptions import CommandExecutionError, SaltInvocationError
 
-# Import 3rd-party libs
-from salt.ext import six
-
 try:
     import libvirt  # pylint: disable=import-error
 
@@ -38,6 +34,8 @@ except ImportError:
 
 __virtualname__ = "virt"
 
+log = logging.getLogger(__name__)
+
 
 def __virtual__():
     """
@@ -99,8 +97,8 @@ def keys(name, basepath="/etc/pki", **kwargs):
     # rename them to something hopefully unique to avoid
     # overriding anything existing
     pillar_kwargs = {}
-    for key, value in six.iteritems(kwargs):
-        pillar_kwargs["ext_pillar_virt.{0}".format(key)] = value
+    for key, value in kwargs.items():
+        pillar_kwargs["ext_pillar_virt.{}".format(key)] = value
 
     pillar = __salt__["pillar.ext"]({"libvirt": "_"}, pillar_kwargs)
     paths = {
@@ -112,7 +110,7 @@ def keys(name, basepath="/etc/pki", **kwargs):
     }
 
     for key in paths:
-        p_key = "libvirt.{0}.pem".format(key)
+        p_key = "libvirt.{}.pem".format(key)
         if p_key not in pillar:
             continue
         if not os.path.exists(os.path.dirname(paths[key])):
@@ -134,7 +132,7 @@ def keys(name, basepath="/etc/pki", **kwargs):
         for key in ret["changes"]:
             with salt.utils.files.fopen(paths[key], "w+") as fp_:
                 fp_.write(
-                    salt.utils.stringutils.to_str(pillar["libvirt.{0}.pem".format(key)])
+                    salt.utils.stringutils.to_str(pillar["libvirt.{}.pem".format(key)])
                 )
 
         ret["comment"] = "Updated libvirt certs and keys"
@@ -176,7 +174,7 @@ def _virt_call(
                 domain_state = __salt__["virt.vm_state"](targeted_domain)
                 action_needed = domain_state.get(targeted_domain) != state
             if action_needed:
-                response = __salt__["virt.{0}".format(function)](
+                response = __salt__["virt.{}".format(function)](
                     targeted_domain,
                     connection=connection,
                     username=username,
@@ -189,9 +187,7 @@ def _virt_call(
             else:
                 noaction_domains.append(targeted_domain)
         except libvirt.libvirtError as err:
-            ignored_domains.append(
-                {"domain": targeted_domain, "issue": six.text_type(err)}
-            )
+            ignored_domains.append({"domain": targeted_domain, "issue": str(err)})
     if not changed_domains:
         ret["result"] = not ignored_domains and bool(targeted_domains)
         ret["comment"] = "No changes had happened"
@@ -292,6 +288,7 @@ def defined(
     arch=None,
     boot=None,
     update=True,
+    boot_dev=None,
 ):
     """
     Starts an existing guest, or defines and starts a new VM with specified arguments.
@@ -352,6 +349,14 @@ def defined(
 
         .. deprecated:: sodium
 
+    :param boot_dev:
+        Space separated list of devices to boot from sorted by decreasing priority.
+        Values can be ``hd``, ``fd``, ``cdrom`` or ``network``.
+
+        By default, the value will ``"hd"``.
+
+        .. versionadded:: Magnesium
+
     .. rubric:: Example States
 
     Make sure a virtual machine called ``domain_name`` is defined:
@@ -362,6 +367,7 @@ def defined(
           virt.defined:
             - cpu: 2
             - mem: 2048
+            - boot_dev: network hd
             - disk_profile: prod
             - disks:
               - name: system
@@ -414,17 +420,18 @@ def defined(
                     password=password,
                     boot=boot,
                     test=__opts__["test"],
+                    boot_dev=boot_dev,
                 )
             ret["changes"][name] = status
             if not status.get("definition"):
-                ret["comment"] = "Domain {0} unchanged".format(name)
+                ret["comment"] = "Domain {} unchanged".format(name)
                 ret["result"] = True
             elif status.get("errors"):
                 ret[
                     "comment"
-                ] = "Domain {0} updated with live update(s) failures".format(name)
+                ] = "Domain {} updated with live update(s) failures".format(name)
             else:
-                ret["comment"] = "Domain {0} updated".format(name)
+                ret["comment"] = "Domain {} updated".format(name)
         else:
             if not __opts__["test"]:
                 __salt__["virt.init"](
@@ -448,12 +455,13 @@ def defined(
                     password=password,
                     boot=boot,
                     start=False,
+                    boot_dev=boot_dev,
                 )
             ret["changes"][name] = {"definition": True}
-            ret["comment"] = "Domain {0} defined".format(name)
+            ret["comment"] = "Domain {} defined".format(name)
     except libvirt.libvirtError as err:
         # Something bad happened when defining / updating the VM, report it
-        ret["comment"] = six.text_type(err)
+        ret["comment"] = str(err)
         ret["result"] = False
 
     return ret
@@ -480,6 +488,7 @@ def running(
     os_type=None,
     arch=None,
     boot=None,
+    boot_dev=None,
 ):
     """
     Starts an existing guest, or defines and starts a new VM with specified arguments.
@@ -591,6 +600,14 @@ def running(
 
         .. versionadded:: 3000
 
+    :param boot_dev:
+        Space separated list of devices to boot from sorted by decreasing priority.
+        Values can be ``hd``, ``fd``, ``cdrom`` or ``network``.
+
+        By default, the value will ``"hd"``.
+
+        .. versionadded:: Magnesium
+
     .. rubric:: Example States
 
     Make sure an already-defined virtual machine called ``domain_name`` is running:
@@ -609,6 +626,7 @@ def running(
             - cpu: 2
             - mem: 2048
             - disk_profile: prod
+            - boot_dev: network hd
             - disks:
               - name: system
                 size: 8192
@@ -657,6 +675,7 @@ def running(
         arch=arch,
         boot=boot,
         update=update,
+        boot_dev=boot_dev,
         connection=connection,
         username=username,
         password=password,
@@ -681,11 +700,11 @@ def running(
                 ret["comment"] = comment
                 ret["changes"][name]["started"] = True
             elif not changed:
-                ret["comment"] = "Domain {0} exists and is running".format(name)
+                ret["comment"] = "Domain {} exists and is running".format(name)
 
         except libvirt.libvirtError as err:
             # Something bad happened when starting / updating the VM, report it
-            ret["comment"] = six.text_type(err)
+            ret["comment"] = str(err)
             ret["result"] = False
 
     return ret
@@ -830,7 +849,7 @@ def reverted(
     try:
         domains = fnmatch.filter(__salt__["virt.list_domains"](), name)
         if not domains:
-            ret["comment"] = 'No domains found for criteria "{0}"'.format(name)
+            ret["comment"] = 'No domains found for criteria "{}"'.format(name)
         else:
             ignored_domains = list()
             if len(domains) > 1:
@@ -848,9 +867,7 @@ def reverted(
                     }
                 except CommandExecutionError as err:
                     if len(domains) > 1:
-                        ignored_domains.append(
-                            {"domain": domain, "issue": six.text_type(err)}
-                        )
+                        ignored_domains.append({"domain": domain, "issue": str(err)})
                 if len(domains) > 1:
                     if result:
                         ret["changes"]["reverted"].append(result)
@@ -860,7 +877,7 @@ def reverted(
 
             ret["result"] = len(domains) != len(ignored_domains)
             if ret["result"]:
-                ret["comment"] = "Domain{0} has been reverted".format(
+                ret["comment"] = "Domain{} has been reverted".format(
                     len(domains) > 1 and "s" or ""
                 )
             if ignored_domains:
@@ -868,9 +885,9 @@ def reverted(
             if not ret["changes"]["reverted"]:
                 ret["changes"].pop("reverted")
     except libvirt.libvirtError as err:
-        ret["comment"] = six.text_type(err)
+        ret["comment"] = str(err)
     except CommandExecutionError as err:
-        ret["comment"] = six.text_type(err)
+        ret["comment"] = str(err)
 
     return ret
 
@@ -955,7 +972,7 @@ def network_defined(
             name, connection=connection, username=username, password=password
         )
         if info and info[name]:
-            ret["comment"] = "Network {0} exists".format(name)
+            ret["comment"] = "Network {} exists".format(name)
             ret["result"] = True
         else:
             if not __opts__["test"]:
@@ -974,7 +991,7 @@ def network_defined(
                     password=password,
                 )
             ret["changes"][name] = "Network defined"
-            ret["comment"] = "Network {0} defined".format(name)
+            ret["comment"] = "Network {} defined".format(name)
     except libvirt.libvirtError as err:
         ret["result"] = False
         ret["comment"] = err.get_error_message()
@@ -1108,6 +1125,10 @@ def network_running(
     return ret
 
 
+# Some of the libvirt storage drivers do not support the build action
+BUILDABLE_POOL_TYPES = {"disk", "fs", "netfs", "dir", "logical", "vstorage", "zfs"}
+
+
 def pool_defined(
     name,
     ptype=None,
@@ -1222,25 +1243,35 @@ def pool_defined(
 
                 action = ""
                 if info[name]["state"] != "running":
-                    if not __opts__["test"]:
-                        __salt__["virt.pool_build"](
-                            name,
-                            connection=connection,
-                            username=username,
-                            password=password,
-                        )
-                    action = ", built"
+                    if ptype in BUILDABLE_POOL_TYPES:
+                        if not __opts__["test"]:
+                            # Storage pools build like disk or logical will fail if the disk or LV group
+                            # was already existing. Since we can't easily figure that out, just log the
+                            # possible libvirt error.
+                            try:
+                                __salt__["virt.pool_build"](
+                                    name,
+                                    connection=connection,
+                                    username=username,
+                                    password=password,
+                                )
+                            except libvirt.libvirtError as err:
+                                log.warning(
+                                    "Failed to build libvirt storage pool: %s",
+                                    err.get_error_message(),
+                                )
+                        action = ", built"
 
                 action = (
                     "{}, autostart flag changed".format(action)
                     if needs_autostart
                     else action
                 )
-                ret["changes"][name] = "Pool updated{0}".format(action)
-                ret["comment"] = "Pool {0} updated{1}".format(name, action)
+                ret["changes"][name] = "Pool updated{}".format(action)
+                ret["comment"] = "Pool {} updated{}".format(name, action)
 
             else:
-                ret["comment"] = "Pool {0} unchanged".format(name)
+                ret["comment"] = "Pool {} unchanged".format(name)
                 ret["result"] = True
         else:
             needs_autostart = autostart
@@ -1265,15 +1296,28 @@ def pool_defined(
                     password=password,
                 )
 
-                __salt__["virt.pool_build"](
-                    name, connection=connection, username=username, password=password
-                )
+                if ptype in BUILDABLE_POOL_TYPES:
+                    # Storage pools build like disk or logical will fail if the disk or LV group
+                    # was already existing. Since we can't easily figure that out, just log the
+                    # possible libvirt error.
+                    try:
+                        __salt__["virt.pool_build"](
+                            name,
+                            connection=connection,
+                            username=username,
+                            password=password,
+                        )
+                    except libvirt.libvirtError as err:
+                        log.warning(
+                            "Failed to build libvirt storage pool: %s",
+                            err.get_error_message(),
+                        )
             if needs_autostart:
                 ret["changes"][name] = "Pool defined, marked for autostart"
-                ret["comment"] = "Pool {0} defined, marked for autostart".format(name)
+                ret["comment"] = "Pool {} defined, marked for autostart".format(name)
             else:
                 ret["changes"][name] = "Pool defined"
-                ret["comment"] = "Pool {0} defined".format(name)
+                ret["comment"] = "Pool {} defined".format(name)
 
         if needs_autostart:
             if not __opts__["test"]:
@@ -1374,7 +1418,7 @@ def pool_running(
             is_running = info.get(name, {}).get("state", "stopped") == "running"
             if is_running:
                 if updated:
-                    action = "built, restarted"
+                    action = "restarted"
                     if not __opts__["test"]:
                         __salt__["virt.pool_stop"](
                             name,
@@ -1382,13 +1426,16 @@ def pool_running(
                             username=username,
                             password=password,
                         )
-                    if not __opts__["test"]:
-                        __salt__["virt.pool_build"](
-                            name,
-                            connection=connection,
-                            username=username,
-                            password=password,
-                        )
+                    # if the disk or LV group is already existing build will fail (issue #56454)
+                    if ptype in BUILDABLE_POOL_TYPES - {"disk", "logical"}:
+                        if not __opts__["test"]:
+                            __salt__["virt.pool_build"](
+                                name,
+                                connection=connection,
+                                username=username,
+                                password=password,
+                            )
+                        action = "built, {}".format(action)
                 else:
                     action = "already running"
                     result = True
@@ -1402,16 +1449,16 @@ def pool_running(
                         password=password,
                     )
 
-            comment = "Pool {0}".format(name)
+            comment = "Pool {}".format(name)
             change = "Pool"
             if name in ret["changes"]:
-                comment = "{0},".format(ret["comment"])
-                change = "{0},".format(ret["changes"][name])
+                comment = "{},".format(ret["comment"])
+                change = "{},".format(ret["changes"][name])
 
             if action != "already running":
-                ret["changes"][name] = "{0} {1}".format(change, action)
+                ret["changes"][name] = "{} {}".format(change, action)
 
-            ret["comment"] = "{0} {1}".format(comment, action)
+            ret["comment"] = "{} {}".format(comment, action)
             ret["result"] = result
 
         except libvirt.libvirtError as err:
@@ -1539,15 +1586,13 @@ def pool_deleted(name, purge=False, connection=None, username=None, password=Non
                 ret["result"] = None
 
             if unsupported:
-                ret[
-                    "comment"
-                ] = 'Unsupported actions for pool of type "{0}": {1}'.format(
+                ret["comment"] = 'Unsupported actions for pool of type "{}": {}'.format(
                     info[name]["type"], ", ".join(unsupported)
                 )
         else:
-            ret["comment"] = "Storage pool could not be found: {0}".format(name)
+            ret["comment"] = "Storage pool could not be found: {}".format(name)
     except libvirt.libvirtError as err:
-        ret["comment"] = "Failed deleting pool: {0}".format(err.get_error_message())
+        ret["comment"] = "Failed deleting pool: {}".format(err.get_error_message())
         ret["result"] = False
 
     return ret
diff --git a/salt/templates/virt/libvirt_domain.jinja b/salt/templates/virt/libvirt_domain.jinja
index aac6283eb0..04a61ffa78 100644
--- a/salt/templates/virt/libvirt_domain.jinja
+++ b/salt/templates/virt/libvirt_domain.jinja
@@ -3,7 +3,7 @@
         <vcpu>{{ cpu }}</vcpu>
         <memory unit='KiB'>{{ mem }}</memory>
         <currentMemory unit='KiB'>{{ mem }}</currentMemory>
-        <os>
+        <os {{boot.os_attrib}}>
                 <type arch='{{ arch }}'>{{ os_type }}</type>
                 {% if boot %}
                   {% if 'kernel' in boot %}
diff --git a/salt/utils/data.py b/salt/utils/data.py
index 8f84c2ea42..1c4c22efb3 100644
--- a/salt/utils/data.py
+++ b/salt/utils/data.py
@@ -1,22 +1,16 @@
 # -*- coding: utf-8 -*-
-'''
+"""
 Functions for manipulating, inspecting, or otherwise working with data types
 and data structures.
-'''
+"""
 
-from __future__ import absolute_import, print_function, unicode_literals
 
 # Import Python libs
 import copy
 import fnmatch
+import functools
 import logging
 import re
-import functools
-
-try:
-    from collections.abc import Mapping, MutableMapping, Sequence
-except ImportError:
-    from collections import Mapping, MutableMapping, Sequence
 
 # Import Salt libs
 import salt.utils.dictupdate
@@ -24,13 +18,22 @@ import salt.utils.stringutils
 import salt.utils.yaml
 from salt.defaults import DEFAULT_TARGET_DELIM
 from salt.exceptions import SaltException
-from salt.utils.decorators.jinja import jinja_filter
-from salt.utils.odict import OrderedDict
+from salt.ext import six
 
 # Import 3rd-party libs
-from salt.ext.six.moves import zip  # pylint: disable=redefined-builtin
-from salt.ext import six
 from salt.ext.six.moves import range  # pylint: disable=redefined-builtin
+from salt.ext.six.moves import zip  # pylint: disable=redefined-builtin
+from salt.utils.decorators.jinja import jinja_filter
+from salt.utils.odict import OrderedDict
+
+try:
+    from collections.abc import Mapping, MutableMapping, Sequence
+except ImportError:
+    # pylint: disable=no-name-in-module
+    from collections import Mapping, MutableMapping, Sequence
+
+    # pylint: enable=no-name-in-module
+
 
 try:
     import jmespath
@@ -41,15 +44,16 @@ log = logging.getLogger(__name__)
 
 
 class CaseInsensitiveDict(MutableMapping):
-    '''
+    """
     Inspired by requests' case-insensitive dict implementation, but works with
     non-string keys as well.
-    '''
+    """
+
     def __init__(self, init=None, **kwargs):
-        '''
+        """
         Force internal dict to be ordered to ensure a consistent iteration
         order, irrespective of case.
-        '''
+        """
         self._data = OrderedDict()
         self.update(init or {}, **kwargs)
 
@@ -67,7 +71,7 @@ class CaseInsensitiveDict(MutableMapping):
         return self._data[to_lowercase(key)][1]
 
     def __iter__(self):
-        return (item[0] for item in six.itervalues(self._data))
+        return (item[0] for item in self._data.values())
 
     def __eq__(self, rval):
         if not isinstance(rval, Mapping):
@@ -76,28 +80,28 @@ class CaseInsensitiveDict(MutableMapping):
         return dict(self.items_lower()) == dict(CaseInsensitiveDict(rval).items_lower())
 
     def __repr__(self):
-        return repr(dict(six.iteritems(self)))
+        return repr(dict(self.items()))
 
     def items_lower(self):
-        '''
+        """
         Returns a generator iterating over keys and values, with the keys all
         being lowercase.
-        '''
-        return ((key, val[1]) for key, val in six.iteritems(self._data))
+        """
+        return ((key, val[1]) for key, val in self._data.items())
 
     def copy(self):
-        '''
+        """
         Returns a copy of the object
-        '''
-        return CaseInsensitiveDict(six.iteritems(self._data))
+        """
+        return CaseInsensitiveDict(self._data.items())
 
 
 def __change_case(data, attr, preserve_dict_class=False):
-    '''
+    """
     Calls data.attr() if data has an attribute/method called attr.
     Processes data recursively if data is a Mapping or Sequence.
     For Mapping, processes both keys and values.
-    '''
+    """
     try:
         return getattr(data, attr)()
     except AttributeError:
@@ -107,73 +111,120 @@ def __change_case(data, attr, preserve_dict_class=False):
 
     if isinstance(data, Mapping):
         return (data_type if preserve_dict_class else dict)(
-            (__change_case(key, attr, preserve_dict_class),
-             __change_case(val, attr, preserve_dict_class))
-            for key, val in six.iteritems(data)
+            (
+                __change_case(key, attr, preserve_dict_class),
+                __change_case(val, attr, preserve_dict_class),
+            )
+            for key, val in data.items()
         )
     if isinstance(data, Sequence):
         return data_type(
-            __change_case(item, attr, preserve_dict_class) for item in data)
+            __change_case(item, attr, preserve_dict_class) for item in data
+        )
     return data
 
 
 def to_lowercase(data, preserve_dict_class=False):
-    '''
+    """
     Recursively changes everything in data to lowercase.
-    '''
-    return __change_case(data, 'lower', preserve_dict_class)
+    """
+    return __change_case(data, "lower", preserve_dict_class)
 
 
 def to_uppercase(data, preserve_dict_class=False):
-    '''
+    """
     Recursively changes everything in data to uppercase.
-    '''
-    return __change_case(data, 'upper', preserve_dict_class)
+    """
+    return __change_case(data, "upper", preserve_dict_class)
 
 
-@jinja_filter('compare_dicts')
+@jinja_filter("compare_dicts")
 def compare_dicts(old=None, new=None):
-    '''
+    """
     Compare before and after results from various salt functions, returning a
     dict describing the changes that were made.
-    '''
+    """
     ret = {}
-    for key in set((new or {})).union((old or {})):
+    for key in set(new or {}).union(old or {}):
         if key not in old:
             # New key
-            ret[key] = {'old': '',
-                        'new': new[key]}
+            ret[key] = {"old": "", "new": new[key]}
         elif key not in new:
             # Key removed
-            ret[key] = {'new': '',
-                        'old': old[key]}
+            ret[key] = {"new": "", "old": old[key]}
         elif new[key] != old[key]:
             # Key modified
-            ret[key] = {'old': old[key],
-                        'new': new[key]}
+            ret[key] = {"old": old[key], "new": new[key]}
     return ret
 
 
-@jinja_filter('compare_lists')
+@jinja_filter("compare_lists")
 def compare_lists(old=None, new=None):
-    '''
+    """
     Compare before and after results from various salt functions, returning a
     dict describing the changes that were made
-    '''
+    """
     ret = {}
     for item in new:
         if item not in old:
-            ret.setdefault('new', []).append(item)
+            ret.setdefault("new", []).append(item)
     for item in old:
         if item not in new:
-            ret.setdefault('old', []).append(item)
+            ret.setdefault("old", []).append(item)
     return ret
 
 
-def decode(data, encoding=None, errors='strict', keep=False,
-           normalize=False, preserve_dict_class=False, preserve_tuples=False,
-           to_str=False):
-    '''
+def _remove_circular_refs(ob, _seen=None):
+    """
+    Generic method to remove circular references from objects.
+    This has been taken from author Martijn Pieters
+    https://stackoverflow.com/questions/44777369/
+    remove-circular-references-in-dicts-lists-tuples/44777477#44777477
+    :param ob: dict, list, typle, set, and frozenset
+        Standard python object
+    :param object _seen:
+        Object that has circular reference
+    :returns:
+        Cleaned Python object
+    :rtype:
+        type(ob)
+    """
+    if _seen is None:
+        _seen = set()
+    if id(ob) in _seen:
+        # Here we caught a circular reference.
+        # Alert user and cleanup to continue.
+        log.exception(
+            "Caught a circular reference in data structure below."
+            "Cleaning and continuing execution.\n%r\n",
+            ob,
+        )
+        return None
+    _seen.add(id(ob))
+    res = ob
+    if isinstance(ob, dict):
+        res = {
+            _remove_circular_refs(k, _seen): _remove_circular_refs(v, _seen)
+            for k, v in ob.items()
+        }
+    elif isinstance(ob, (list, tuple, set, frozenset)):
+        res = type(ob)(_remove_circular_refs(v, _seen) for v in ob)
+    # remove id again; only *nested* references count
+    _seen.remove(id(ob))
+    return res
+
+
+def decode(
+    data,
+    encoding=None,
+    errors="strict",
+    keep=False,
+    normalize=False,
+    preserve_dict_class=False,
+    preserve_tuples=False,
+    to_str=False,
+):
+    """
     Generic function which will decode whichever type is passed, if necessary.
     Optionally use to_str=True to ensure strings are str types and not unicode
     on Python 2.
@@ -199,22 +250,55 @@ def decode(data, encoding=None, errors='strict', keep=False,
     two strings above, in which "й" is represented as two code points (i.e. one
     for the base character, and one for the breve mark). Normalizing allows for
     a more reliable test case.
-    '''
-    _decode_func = salt.utils.stringutils.to_unicode \
-        if not to_str \
+
+    """
+    # Clean data object before decoding to avoid circular references
+    data = _remove_circular_refs(data)
+
+    _decode_func = (
+        salt.utils.stringutils.to_unicode
+        if not to_str
         else salt.utils.stringutils.to_str
+    )
     if isinstance(data, Mapping):
-        return decode_dict(data, encoding, errors, keep, normalize,
-                           preserve_dict_class, preserve_tuples, to_str)
+        return decode_dict(
+            data,
+            encoding,
+            errors,
+            keep,
+            normalize,
+            preserve_dict_class,
+            preserve_tuples,
+            to_str,
+        )
     if isinstance(data, list):
-        return decode_list(data, encoding, errors, keep, normalize,
-                           preserve_dict_class, preserve_tuples, to_str)
+        return decode_list(
+            data,
+            encoding,
+            errors,
+            keep,
+            normalize,
+            preserve_dict_class,
+            preserve_tuples,
+            to_str,
+        )
     if isinstance(data, tuple):
-        return decode_tuple(data, encoding, errors, keep, normalize,
-                            preserve_dict_class, to_str) \
-            if preserve_tuples \
-            else decode_list(data, encoding, errors, keep, normalize,
-                             preserve_dict_class, preserve_tuples, to_str)
+        return (
+            decode_tuple(
+                data, encoding, errors, keep, normalize, preserve_dict_class, to_str
+            )
+            if preserve_tuples
+            else decode_list(
+                data,
+                encoding,
+                errors,
+                keep,
+                normalize,
+                preserve_dict_class,
+                preserve_tuples,
+                to_str,
+            )
+        )
     try:
         data = _decode_func(data, encoding, errors, normalize)
     except TypeError:
@@ -228,25 +312,48 @@ def decode(data, encoding=None, errors='strict', keep=False,
     return data
 
 
-def decode_dict(data, encoding=None, errors='strict', keep=False,
-                normalize=False, preserve_dict_class=False,
-                preserve_tuples=False, to_str=False):
-    '''
+def decode_dict(
+    data,
+    encoding=None,
+    errors="strict",
+    keep=False,
+    normalize=False,
+    preserve_dict_class=False,
+    preserve_tuples=False,
+    to_str=False,
+):
+    """
     Decode all string values to Unicode. Optionally use to_str=True to ensure
     strings are str types and not unicode on Python 2.
-    '''
-    _decode_func = salt.utils.stringutils.to_unicode \
-        if not to_str \
+    """
+    # Clean data object before decoding to avoid circular references
+    data = _remove_circular_refs(data)
+
+    _decode_func = (
+        salt.utils.stringutils.to_unicode
+        if not to_str
         else salt.utils.stringutils.to_str
+    )
     # Make sure we preserve OrderedDicts
     ret = data.__class__() if preserve_dict_class else {}
-    for key, value in six.iteritems(data):
+    for key, value in data.items():
         if isinstance(key, tuple):
-            key = decode_tuple(key, encoding, errors, keep, normalize,
-                               preserve_dict_class, to_str) \
-                if preserve_tuples \
-                else decode_list(key, encoding, errors, keep, normalize,
-                                 preserve_dict_class, preserve_tuples, to_str)
+            key = (
+                decode_tuple(
+                    key, encoding, errors, keep, normalize, preserve_dict_class, to_str
+                )
+                if preserve_tuples
+                else decode_list(
+                    key,
+                    encoding,
+                    errors,
+                    keep,
+                    normalize,
+                    preserve_dict_class,
+                    preserve_tuples,
+                    to_str,
+                )
+            )
         else:
             try:
                 key = _decode_func(key, encoding, errors, normalize)
@@ -260,17 +367,50 @@ def decode_dict(data, encoding=None, errors='strict', keep=False,
                     raise
 
         if isinstance(value, list):
-            value = decode_list(value, encoding, errors, keep, normalize,
-                                preserve_dict_class, preserve_tuples, to_str)
+            value = decode_list(
+                value,
+                encoding,
+                errors,
+                keep,
+                normalize,
+                preserve_dict_class,
+                preserve_tuples,
+                to_str,
+            )
         elif isinstance(value, tuple):
-            value = decode_tuple(value, encoding, errors, keep, normalize,
-                                 preserve_dict_class, to_str) \
-                if preserve_tuples \
-                else decode_list(value, encoding, errors, keep, normalize,
-                                 preserve_dict_class, preserve_tuples, to_str)
+            value = (
+                decode_tuple(
+                    value,
+                    encoding,
+                    errors,
+                    keep,
+                    normalize,
+                    preserve_dict_class,
+                    to_str,
+                )
+                if preserve_tuples
+                else decode_list(
+                    value,
+                    encoding,
+                    errors,
+                    keep,
+                    normalize,
+                    preserve_dict_class,
+                    preserve_tuples,
+                    to_str,
+                )
+            )
         elif isinstance(value, Mapping):
-            value = decode_dict(value, encoding, errors, keep, normalize,
-                                preserve_dict_class, preserve_tuples, to_str)
+            value = decode_dict(
+                value,
+                encoding,
+                errors,
+                keep,
+                normalize,
+                preserve_dict_class,
+                preserve_tuples,
+                to_str,
+            )
         else:
             try:
                 value = _decode_func(value, encoding, errors, normalize)
@@ -287,30 +427,69 @@ def decode_dict(data, encoding=None, errors='strict', keep=False,
     return ret
 
 
-def decode_list(data, encoding=None, errors='strict', keep=False,
-                normalize=False, preserve_dict_class=False,
-                preserve_tuples=False, to_str=False):
-    '''
+def decode_list(
+    data,
+    encoding=None,
+    errors="strict",
+    keep=False,
+    normalize=False,
+    preserve_dict_class=False,
+    preserve_tuples=False,
+    to_str=False,
+):
+    """
     Decode all string values to Unicode. Optionally use to_str=True to ensure
     strings are str types and not unicode on Python 2.
-    '''
-    _decode_func = salt.utils.stringutils.to_unicode \
-        if not to_str \
+    """
+    # Clean data object before decoding to avoid circular references
+    data = _remove_circular_refs(data)
+
+    _decode_func = (
+        salt.utils.stringutils.to_unicode
+        if not to_str
         else salt.utils.stringutils.to_str
+    )
     ret = []
     for item in data:
         if isinstance(item, list):
-            item = decode_list(item, encoding, errors, keep, normalize,
-                               preserve_dict_class, preserve_tuples, to_str)
+            item = decode_list(
+                item,
+                encoding,
+                errors,
+                keep,
+                normalize,
+                preserve_dict_class,
+                preserve_tuples,
+                to_str,
+            )
         elif isinstance(item, tuple):
-            item = decode_tuple(item, encoding, errors, keep, normalize,
-                                preserve_dict_class, to_str) \
-                if preserve_tuples \
-                else decode_list(item, encoding, errors, keep, normalize,
-                                 preserve_dict_class, preserve_tuples, to_str)
+            item = (
+                decode_tuple(
+                    item, encoding, errors, keep, normalize, preserve_dict_class, to_str
+                )
+                if preserve_tuples
+                else decode_list(
+                    item,
+                    encoding,
+                    errors,
+                    keep,
+                    normalize,
+                    preserve_dict_class,
+                    preserve_tuples,
+                    to_str,
+                )
+            )
         elif isinstance(item, Mapping):
-            item = decode_dict(item, encoding, errors, keep, normalize,
-                               preserve_dict_class, preserve_tuples, to_str)
+            item = decode_dict(
+                item,
+                encoding,
+                errors,
+                keep,
+                normalize,
+                preserve_dict_class,
+                preserve_tuples,
+                to_str,
+            )
         else:
             try:
                 item = _decode_func(item, encoding, errors, normalize)
@@ -327,21 +506,35 @@ def decode_list(data, encoding=None, errors='strict', keep=False,
     return ret
 
 
-def decode_tuple(data, encoding=None, errors='strict', keep=False,
-                 normalize=False, preserve_dict_class=False, to_str=False):
-    '''
+def decode_tuple(
+    data,
+    encoding=None,
+    errors="strict",
+    keep=False,
+    normalize=False,
+    preserve_dict_class=False,
+    to_str=False,
+):
+    """
     Decode all string values to Unicode. Optionally use to_str=True to ensure
     strings are str types and not unicode on Python 2.
-    '''
+    """
     return tuple(
-        decode_list(data, encoding, errors, keep, normalize,
-                    preserve_dict_class, True, to_str)
+        decode_list(
+            data, encoding, errors, keep, normalize, preserve_dict_class, True, to_str
+        )
     )
 
 
-def encode(data, encoding=None, errors='strict', keep=False,
-           preserve_dict_class=False, preserve_tuples=False):
-    '''
+def encode(
+    data,
+    encoding=None,
+    errors="strict",
+    keep=False,
+    preserve_dict_class=False,
+    preserve_tuples=False,
+):
+    """
     Generic function which will encode whichever type is passed, if necessary
 
     If `strict` is True, and `keep` is False, and we fail to encode, a
@@ -349,18 +542,27 @@ def encode(data, encoding=None, errors='strict', keep=False,
     original value to silently be returned in cases where encoding fails. This
     can be useful for cases where the data passed to this function is likely to
     contain binary blobs.
-    '''
+
+    """
+    # Clean data object before encoding to avoid circular references
+    data = _remove_circular_refs(data)
+
     if isinstance(data, Mapping):
-        return encode_dict(data, encoding, errors, keep,
-                           preserve_dict_class, preserve_tuples)
+        return encode_dict(
+            data, encoding, errors, keep, preserve_dict_class, preserve_tuples
+        )
     if isinstance(data, list):
-        return encode_list(data, encoding, errors, keep,
-                           preserve_dict_class, preserve_tuples)
+        return encode_list(
+            data, encoding, errors, keep, preserve_dict_class, preserve_tuples
+        )
     if isinstance(data, tuple):
-        return encode_tuple(data, encoding, errors, keep, preserve_dict_class) \
-            if preserve_tuples \
-            else encode_list(data, encoding, errors, keep,
-                             preserve_dict_class, preserve_tuples)
+        return (
+            encode_tuple(data, encoding, errors, keep, preserve_dict_class)
+            if preserve_tuples
+            else encode_list(
+                data, encoding, errors, keep, preserve_dict_class, preserve_tuples
+            )
+        )
     try:
         return salt.utils.stringutils.to_bytes(data, encoding, errors)
     except TypeError:
@@ -374,20 +576,31 @@ def encode(data, encoding=None, errors='strict', keep=False,
     return data
 
 
-@jinja_filter('json_decode_dict')  # Remove this for Aluminium
-@jinja_filter('json_encode_dict')
-def encode_dict(data, encoding=None, errors='strict', keep=False,
-                preserve_dict_class=False, preserve_tuples=False):
-    '''
+@jinja_filter("json_decode_dict")  # Remove this for Aluminium
+@jinja_filter("json_encode_dict")
+def encode_dict(
+    data,
+    encoding=None,
+    errors="strict",
+    keep=False,
+    preserve_dict_class=False,
+    preserve_tuples=False,
+):
+    """
     Encode all string values to bytes
-    '''
+    """
+    # Clean data object before encoding to avoid circular references
+    data = _remove_circular_refs(data)
     ret = data.__class__() if preserve_dict_class else {}
-    for key, value in six.iteritems(data):
+    for key, value in data.items():
         if isinstance(key, tuple):
-            key = encode_tuple(key, encoding, errors, keep, preserve_dict_class) \
-                if preserve_tuples \
-                else encode_list(key, encoding, errors, keep,
-                                 preserve_dict_class, preserve_tuples)
+            key = (
+                encode_tuple(key, encoding, errors, keep, preserve_dict_class)
+                if preserve_tuples
+                else encode_list(
+                    key, encoding, errors, keep, preserve_dict_class, preserve_tuples
+                )
+            )
         else:
             try:
                 key = salt.utils.stringutils.to_bytes(key, encoding, errors)
@@ -401,16 +614,21 @@ def encode_dict(data, encoding=None, errors='strict', keep=False,
                     raise
 
         if isinstance(value, list):
-            value = encode_list(value, encoding, errors, keep,
-                                preserve_dict_class, preserve_tuples)
+            value = encode_list(
+                value, encoding, errors, keep, preserve_dict_class, preserve_tuples
+            )
         elif isinstance(value, tuple):
-            value = encode_tuple(value, encoding, errors, keep, preserve_dict_class) \
-                if preserve_tuples \
-                else encode_list(value, encoding, errors, keep,
-                                 preserve_dict_class, preserve_tuples)
+            value = (
+                encode_tuple(value, encoding, errors, keep, preserve_dict_class)
+                if preserve_tuples
+                else encode_list(
+                    value, encoding, errors, keep, preserve_dict_class, preserve_tuples
+                )
+            )
         elif isinstance(value, Mapping):
-            value = encode_dict(value, encoding, errors, keep,
-                                preserve_dict_class, preserve_tuples)
+            value = encode_dict(
+                value, encoding, errors, keep, preserve_dict_class, preserve_tuples
+            )
         else:
             try:
                 value = salt.utils.stringutils.to_bytes(value, encoding, errors)
@@ -427,26 +645,40 @@ def encode_dict(data, encoding=None, errors='strict', keep=False,
     return ret
 
 
-@jinja_filter('json_decode_list')  # Remove this for Aluminium
-@jinja_filter('json_encode_list')
-def encode_list(data, encoding=None, errors='strict', keep=False,
-                preserve_dict_class=False, preserve_tuples=False):
-    '''
+@jinja_filter("json_decode_list")  # Remove this for Aluminium
+@jinja_filter("json_encode_list")
+def encode_list(
+    data,
+    encoding=None,
+    errors="strict",
+    keep=False,
+    preserve_dict_class=False,
+    preserve_tuples=False,
+):
+    """
     Encode all string values to bytes
-    '''
+    """
+    # Clean data object before encoding to avoid circular references
+    data = _remove_circular_refs(data)
+
     ret = []
     for item in data:
         if isinstance(item, list):
-            item = encode_list(item, encoding, errors, keep,
-                               preserve_dict_class, preserve_tuples)
+            item = encode_list(
+                item, encoding, errors, keep, preserve_dict_class, preserve_tuples
+            )
         elif isinstance(item, tuple):
-            item = encode_tuple(item, encoding, errors, keep, preserve_dict_class) \
-                if preserve_tuples \
-                else encode_list(item, encoding, errors, keep,
-                                 preserve_dict_class, preserve_tuples)
+            item = (
+                encode_tuple(item, encoding, errors, keep, preserve_dict_class)
+                if preserve_tuples
+                else encode_list(
+                    item, encoding, errors, keep, preserve_dict_class, preserve_tuples
+                )
+            )
         elif isinstance(item, Mapping):
-            item = encode_dict(item, encoding, errors, keep,
-                               preserve_dict_class, preserve_tuples)
+            item = encode_dict(
+                item, encoding, errors, keep, preserve_dict_class, preserve_tuples
+            )
         else:
             try:
                 item = salt.utils.stringutils.to_bytes(item, encoding, errors)
@@ -463,42 +695,37 @@ def encode_list(data, encoding=None, errors='strict', keep=False,
     return ret
 
 
-def encode_tuple(data, encoding=None, errors='strict', keep=False,
-                 preserve_dict_class=False):
-    '''
+def encode_tuple(
+    data, encoding=None, errors="strict", keep=False, preserve_dict_class=False
+):
+    """
     Encode all string values to Unicode
-    '''
-    return tuple(
-        encode_list(data, encoding, errors, keep, preserve_dict_class, True))
+    """
+    return tuple(encode_list(data, encoding, errors, keep, preserve_dict_class, True))
 
 
-@jinja_filter('exactly_n_true')
+@jinja_filter("exactly_n_true")
 def exactly_n(iterable, amount=1):
-    '''
+    """
     Tests that exactly N items in an iterable are "truthy" (neither None,
     False, nor 0).
-    '''
+    """
     i = iter(iterable)
     return all(any(i) for j in range(amount)) and not any(i)
 
 
-@jinja_filter('exactly_one_true')
+@jinja_filter("exactly_one_true")
 def exactly_one(iterable):
-    '''
+    """
     Check if only one item is not None, False, or 0 in an iterable.
-    '''
+    """
     return exactly_n(iterable)
 
 
-def filter_by(lookup_dict,
-              lookup,
-              traverse,
-              merge=None,
-              default='default',
-              base=None):
-    '''
+def filter_by(lookup_dict, lookup, traverse, merge=None, default="default", base=None):
+    """
     Common code to filter data structures like grains and pillar
-    '''
+    """
     ret = None
     # Default value would be an empty list if lookup not found
     val = traverse_dict_and_list(traverse, lookup, [])
@@ -507,10 +734,8 @@ def filter_by(lookup_dict,
     # lookup_dict keys
     for each in val if isinstance(val, list) else [val]:
         for key in lookup_dict:
-            test_key = key if isinstance(key, six.string_types) \
-                else six.text_type(key)
-            test_each = each if isinstance(each, six.string_types) \
-                else six.text_type(each)
+            test_key = key if isinstance(key, str) else str(key)
+            test_each = each if isinstance(each, str) else str(each)
             if fnmatch.fnmatchcase(test_each, test_key):
                 ret = lookup_dict[key]
                 break
@@ -528,14 +753,13 @@ def filter_by(lookup_dict,
         elif isinstance(base_values, Mapping):
             if not isinstance(ret, Mapping):
                 raise SaltException(
-                    'filter_by default and look-up values must both be '
-                    'dictionaries.')
+                    "filter_by default and look-up values must both be " "dictionaries."
+                )
             ret = salt.utils.dictupdate.update(copy.deepcopy(base_values), ret)
 
     if merge:
         if not isinstance(merge, Mapping):
-            raise SaltException(
-                'filter_by merge argument must be a dictionary.')
+            raise SaltException("filter_by merge argument must be a dictionary.")
 
         if ret is None:
             ret = merge
@@ -546,12 +770,12 @@ def filter_by(lookup_dict,
 
 
 def traverse_dict(data, key, default=None, delimiter=DEFAULT_TARGET_DELIM):
-    '''
+    """
     Traverse a dict using a colon-delimited (or otherwise delimited, using the
     'delimiter' param) target string. The target 'foo:bar:baz' will return
     data['foo']['bar']['baz'] if this value exists, and will otherwise return
     the dict in the default argument.
-    '''
+    """
     ptr = data
     try:
         for each in key.split(delimiter):
@@ -562,9 +786,9 @@ def traverse_dict(data, key, default=None, delimiter=DEFAULT_TARGET_DELIM):
     return ptr
 
 
-@jinja_filter('traverse')
+@jinja_filter("traverse")
 def traverse_dict_and_list(data, key, default=None, delimiter=DEFAULT_TARGET_DELIM):
-    '''
+    """
     Traverse a dict or list using a colon-delimited (or otherwise delimited,
     using the 'delimiter' param) target string. The target 'foo:bar:0' will
     return data['foo']['bar'][0] if this value exists, and will otherwise
@@ -573,7 +797,7 @@ def traverse_dict_and_list(data, key, default=None, delimiter=DEFAULT_TARGET_DEL
     The target 'foo:bar:0' will return data['foo']['bar'][0] if data like
     {'foo':{'bar':['baz']}} , if data like {'foo':{'bar':{'0':'baz'}}}
     then return data['foo']['bar']['0']
-    '''
+    """
     ptr = data
     for each in key.split(delimiter):
         if isinstance(ptr, list):
@@ -605,18 +829,17 @@ def traverse_dict_and_list(data, key, default=None, delimiter=DEFAULT_TARGET_DEL
     return ptr
 
 
-def subdict_match(data,
-                  expr,
-                  delimiter=DEFAULT_TARGET_DELIM,
-                  regex_match=False,
-                  exact_match=False):
-    '''
+def subdict_match(
+    data, expr, delimiter=DEFAULT_TARGET_DELIM, regex_match=False, exact_match=False
+):
+    """
     Check for a match in a dictionary using a delimiter character to denote
     levels of subdicts, and also allowing the delimiter character to be
     matched. Thus, 'foo:bar:baz' will match data['foo'] == 'bar:baz' and
     data['foo']['bar'] == 'baz'. The latter would take priority over the
     former, as more deeply-nested matches are tried first.
-    '''
+    """
+
     def _match(target, pattern, regex_match=False, exact_match=False):
         # The reason for using six.text_type first and _then_ using
         # to_unicode as a fallback is because we want to eventually have
@@ -628,11 +851,11 @@ def subdict_match(data,
         # begin with is that (by design) to_unicode will raise a TypeError if a
         # non-string/bytestring/bytearray value is passed.
         try:
-            target = six.text_type(target).lower()
+            target = str(target).lower()
         except UnicodeDecodeError:
             target = salt.utils.stringutils.to_unicode(target).lower()
         try:
-            pattern = six.text_type(pattern).lower()
+            pattern = str(pattern).lower()
         except UnicodeDecodeError:
             pattern = salt.utils.stringutils.to_unicode(pattern).lower()
 
@@ -640,48 +863,54 @@ def subdict_match(data,
             try:
                 return re.match(pattern, target)
             except Exception:  # pylint: disable=broad-except
-                log.error('Invalid regex \'%s\' in match', pattern)
+                log.error("Invalid regex '%s' in match", pattern)
                 return False
         else:
-            return target == pattern if exact_match \
-                else fnmatch.fnmatch(target, pattern)
+            return (
+                target == pattern if exact_match else fnmatch.fnmatch(target, pattern)
+            )
 
     def _dict_match(target, pattern, regex_match=False, exact_match=False):
         ret = False
-        wildcard = pattern.startswith('*:')
+        wildcard = pattern.startswith("*:")
         if wildcard:
             pattern = pattern[2:]
 
-        if pattern == '*':
+        if pattern == "*":
             # We are just checking that the key exists
             ret = True
         if not ret and pattern in target:
             # We might want to search for a key
             ret = True
-        if not ret and subdict_match(target,
-                                     pattern,
-                                     regex_match=regex_match,
-                                     exact_match=exact_match):
+        if not ret and subdict_match(
+            target, pattern, regex_match=regex_match, exact_match=exact_match
+        ):
             ret = True
         if not ret and wildcard:
             for key in target:
                 if isinstance(target[key], dict):
-                    if _dict_match(target[key],
-                                   pattern,
-                                   regex_match=regex_match,
-                                   exact_match=exact_match):
+                    if _dict_match(
+                        target[key],
+                        pattern,
+                        regex_match=regex_match,
+                        exact_match=exact_match,
+                    ):
                         return True
                 elif isinstance(target[key], list):
                     for item in target[key]:
-                        if _match(item,
-                                  pattern,
-                                  regex_match=regex_match,
-                                  exact_match=exact_match):
-                            return True
-                elif _match(target[key],
+                        if _match(
+                            item,
                             pattern,
                             regex_match=regex_match,
-                            exact_match=exact_match):
+                            exact_match=exact_match,
+                        ):
+                            return True
+                elif _match(
+                    target[key],
+                    pattern,
+                    regex_match=regex_match,
+                    exact_match=exact_match,
+                ):
                     return True
         return ret
 
@@ -695,7 +924,7 @@ def subdict_match(data,
     # want to use are 3, 2, and 1, in that order.
     for idx in range(num_splits - 1, 0, -1):
         key = delimiter.join(splits[:idx])
-        if key == '*':
+        if key == "*":
             # We are matching on everything under the top level, so we need to
             # treat the match as the entire data being passed in
             matchstr = expr
@@ -703,54 +932,55 @@ def subdict_match(data,
         else:
             matchstr = delimiter.join(splits[idx:])
             match = traverse_dict_and_list(data, key, {}, delimiter=delimiter)
-        log.debug("Attempting to match '%s' in '%s' using delimiter '%s'",
-                  matchstr, key, delimiter)
+        log.debug(
+            "Attempting to match '%s' in '%s' using delimiter '%s'",
+            matchstr,
+            key,
+            delimiter,
+        )
         if match == {}:
             continue
         if isinstance(match, dict):
-            if _dict_match(match,
-                           matchstr,
-                           regex_match=regex_match,
-                           exact_match=exact_match):
+            if _dict_match(
+                match, matchstr, regex_match=regex_match, exact_match=exact_match
+            ):
                 return True
             continue
         if isinstance(match, (list, tuple)):
             # We are matching a single component to a single list member
             for member in match:
                 if isinstance(member, dict):
-                    if _dict_match(member,
-                                   matchstr,
-                                   regex_match=regex_match,
-                                   exact_match=exact_match):
+                    if _dict_match(
+                        member,
+                        matchstr,
+                        regex_match=regex_match,
+                        exact_match=exact_match,
+                    ):
                         return True
-                if _match(member,
-                          matchstr,
-                          regex_match=regex_match,
-                          exact_match=exact_match):
+                if _match(
+                    member, matchstr, regex_match=regex_match, exact_match=exact_match
+                ):
                     return True
             continue
-        if _match(match,
-                  matchstr,
-                  regex_match=regex_match,
-                  exact_match=exact_match):
+        if _match(match, matchstr, regex_match=regex_match, exact_match=exact_match):
             return True
     return False
 
 
-@jinja_filter('substring_in_list')
+@jinja_filter("substring_in_list")
 def substr_in_list(string_to_search_for, list_to_search):
-    '''
+    """
     Return a boolean value that indicates whether or not a given
     string is present in any of the strings which comprise a list
-    '''
+    """
     return any(string_to_search_for in s for s in list_to_search)
 
 
 def is_dictlist(data):
-    '''
+    """
     Returns True if data is a list of one-element dicts (as found in many SLS
     schemas), otherwise returns False
-    '''
+    """
     if isinstance(data, list):
         for element in data:
             if isinstance(element, dict):
@@ -762,16 +992,12 @@ def is_dictlist(data):
     return False
 
 
-def repack_dictlist(data,
-                    strict=False,
-                    recurse=False,
-                    key_cb=None,
-                    val_cb=None):
-    '''
+def repack_dictlist(data, strict=False, recurse=False, key_cb=None, val_cb=None):
+    """
     Takes a list of one-element dicts (as found in many SLS schemas) and
     repacks into a single dictionary.
-    '''
-    if isinstance(data, six.string_types):
+    """
+    if isinstance(data, str):
         try:
             data = salt.utils.yaml.safe_load(data)
         except salt.utils.yaml.parser.ParserError as err:
@@ -783,7 +1009,7 @@ def repack_dictlist(data,
     if val_cb is None:
         val_cb = lambda x, y: y
 
-    valid_non_dict = (six.string_types, six.integer_types, float)
+    valid_non_dict = ((str,), (int,), float)
     if isinstance(data, list):
         for element in data:
             if isinstance(element, valid_non_dict):
@@ -791,21 +1017,21 @@ def repack_dictlist(data,
             if isinstance(element, dict):
                 if len(element) != 1:
                     log.error(
-                        'Invalid input for repack_dictlist: key/value pairs '
-                        'must contain only one element (data passed: %s).',
-                        element
+                        "Invalid input for repack_dictlist: key/value pairs "
+                        "must contain only one element (data passed: %s).",
+                        element,
                     )
                     return {}
             else:
                 log.error(
-                    'Invalid input for repack_dictlist: element %s is '
-                    'not a string/dict/numeric value', element
+                    "Invalid input for repack_dictlist: element %s is "
+                    "not a string/dict/numeric value",
+                    element,
                 )
                 return {}
     else:
         log.error(
-            'Invalid input for repack_dictlist, data passed is not a list '
-            '(%s)', data
+            "Invalid input for repack_dictlist, data passed is not a list " "(%s)", data
         )
         return {}
 
@@ -821,8 +1047,8 @@ def repack_dictlist(data,
                     ret[key_cb(key)] = repack_dictlist(val, recurse=recurse)
                 elif strict:
                     log.error(
-                        'Invalid input for repack_dictlist: nested dictlist '
-                        'found, but recurse is set to False'
+                        "Invalid input for repack_dictlist: nested dictlist "
+                        "found, but recurse is set to False"
                     )
                     return {}
                 else:
@@ -832,17 +1058,17 @@ def repack_dictlist(data,
     return ret
 
 
-@jinja_filter('is_list')
+@jinja_filter("is_list")
 def is_list(value):
-    '''
+    """
     Check if a variable is a list.
-    '''
+    """
     return isinstance(value, list)
 
 
-@jinja_filter('is_iter')
-def is_iter(thing, ignore=six.string_types):
-    '''
+@jinja_filter("is_iter")
+def is_iter(thing, ignore=(str,)):
+    """
     Test if an object is iterable, but not a string type.
 
     Test if an object is an iterator or is iterable itself. By default this
@@ -853,7 +1079,7 @@ def is_iter(thing, ignore=six.string_types):
     dictionaries or named tuples.
 
     Based on https://bitbucket.org/petershinners/yter
-    '''
+    """
     if ignore and isinstance(thing, ignore):
         return False
     try:
@@ -863,9 +1089,9 @@ def is_iter(thing, ignore=six.string_types):
         return False
 
 
-@jinja_filter('sorted_ignorecase')
+@jinja_filter("sorted_ignorecase")
 def sorted_ignorecase(to_sort):
-    '''
+    """
     Sort a list of strings ignoring case.
 
     >>> L = ['foo', 'Foo', 'bar', 'Bar']
@@ -874,19 +1100,19 @@ def sorted_ignorecase(to_sort):
     >>> sorted(L, key=lambda x: x.lower())
     ['bar', 'Bar', 'foo', 'Foo']
     >>>
-    '''
+    """
     return sorted(to_sort, key=lambda x: x.lower())
 
 
 def is_true(value=None):
-    '''
+    """
     Returns a boolean value representing the "truth" of the value passed. The
     rules for what is a "True" value are:
 
         1. Integer/float values greater than 0
         2. The string values "True" and "true"
         3. Any object for which bool(obj) returns True
-    '''
+    """
     # First, try int/float conversion
     try:
         value = int(value)
@@ -898,26 +1124,26 @@ def is_true(value=None):
         pass
 
     # Now check for truthiness
-    if isinstance(value, (six.integer_types, float)):
+    if isinstance(value, ((int,), float)):
         return value > 0
-    if isinstance(value, six.string_types):
-        return six.text_type(value).lower() == 'true'
+    if isinstance(value, str):
+        return str(value).lower() == "true"
     return bool(value)
 
 
-@jinja_filter('mysql_to_dict')
+@jinja_filter("mysql_to_dict")
 def mysql_to_dict(data, key):
-    '''
+    """
     Convert MySQL-style output to a python dictionary
-    '''
+    """
     ret = {}
-    headers = ['']
+    headers = [""]
     for line in data:
         if not line:
             continue
-        if line.startswith('+'):
+        if line.startswith("+"):
             continue
-        comps = line.split('|')
+        comps = line.split("|")
         for comp in range(len(comps)):
             comps[comp] = comps[comp].strip()
         if len(headers) > 1:
@@ -934,14 +1160,14 @@ def mysql_to_dict(data, key):
 
 
 def simple_types_filter(data):
-    '''
+    """
     Convert the data list, dictionary into simple types, i.e., int, float, string,
     bool, etc.
-    '''
+    """
     if data is None:
         return data
 
-    simpletypes_keys = (six.string_types, six.text_type, six.integer_types, float, bool)
+    simpletypes_keys = ((str,), str, (int,), float, bool)
     simpletypes_values = tuple(list(simpletypes_keys) + [list, tuple])
 
     if isinstance(data, (list, tuple)):
@@ -957,7 +1183,7 @@ def simple_types_filter(data):
 
     if isinstance(data, dict):
         simpledict = {}
-        for key, value in six.iteritems(data):
+        for key, value in data.items():
             if key is not None and not isinstance(key, simpletypes_keys):
                 key = repr(key)
             if value is not None and isinstance(value, (dict, list, tuple)):
@@ -971,23 +1197,23 @@ def simple_types_filter(data):
 
 
 def stringify(data):
-    '''
+    """
     Given an iterable, returns its items as a list, with any non-string items
     converted to unicode strings.
-    '''
+    """
     ret = []
     for item in data:
         if six.PY2 and isinstance(item, str):
             item = salt.utils.stringutils.to_unicode(item)
-        elif not isinstance(item, six.string_types):
-            item = six.text_type(item)
+        elif not isinstance(item, str):
+            item = str(item)
         ret.append(item)
     return ret
 
 
-@jinja_filter('json_query')
+@jinja_filter("json_query")
 def json_query(data, expr):
-    '''
+    """
     Query data using JMESPath language (http://jmespath.org).
 
     Requires the https://github.com/jmespath/jmespath.py library.
@@ -1009,16 +1235,16 @@ def json_query(data, expr):
     .. code-block:: text
 
         [80, 25, 22]
-    '''
+    """
     if jmespath is None:
-        err = 'json_query requires jmespath module installed'
+        err = "json_query requires jmespath module installed"
         log.error(err)
         raise RuntimeError(err)
     return jmespath.search(expr, data)
 
 
 def _is_not_considered_falsey(value, ignore_types=()):
-    '''
+    """
     Helper function for filter_falsey to determine if something is not to be
     considered falsey.
 
@@ -1026,12 +1252,12 @@ def _is_not_considered_falsey(value, ignore_types=()):
     :param list ignore_types: The types to ignore when considering the value.
 
     :return bool
-    '''
+    """
     return isinstance(value, bool) or type(value) in ignore_types or value
 
 
 def filter_falsey(data, recurse_depth=None, ignore_types=()):
-    '''
+    """
     Helper function to remove items from an iterable with falsey value.
     Removes ``None``, ``{}`` and ``[]``, 0, '' (but does not remove ``False``).
     Recurses into sub-iterables if ``recurse`` is set to ``True``.
@@ -1045,37 +1271,42 @@ def filter_falsey(data, recurse_depth=None, ignore_types=()):
     :return type(data)
 
     .. versionadded:: 3000
-    '''
+    """
     filter_element = (
-        functools.partial(filter_falsey,
-                          recurse_depth=recurse_depth-1,
-                          ignore_types=ignore_types)
-        if recurse_depth else lambda x: x
+        functools.partial(
+            filter_falsey, recurse_depth=recurse_depth - 1, ignore_types=ignore_types
+        )
+        if recurse_depth
+        else lambda x: x
     )
 
     if isinstance(data, dict):
-        processed_elements = [(key, filter_element(value)) for key, value in six.iteritems(data)]
-        return type(data)([
-            (key, value)
-            for key, value in processed_elements
-            if _is_not_considered_falsey(value, ignore_types=ignore_types)
-        ])
+        processed_elements = [
+            (key, filter_element(value)) for key, value in data.items()
+        ]
+        return type(data)(
+            [
+                (key, value)
+                for key, value in processed_elements
+                if _is_not_considered_falsey(value, ignore_types=ignore_types)
+            ]
+        )
     if is_iter(data):
         processed_elements = (filter_element(value) for value in data)
-        return type(data)([
-            value for value in processed_elements
-            if _is_not_considered_falsey(value, ignore_types=ignore_types)
-        ])
+        return type(data)(
+            [
+                value
+                for value in processed_elements
+                if _is_not_considered_falsey(value, ignore_types=ignore_types)
+            ]
+        )
     return data
 
 
 def recursive_diff(
-        old,
-        new,
-        ignore_keys=None,
-        ignore_order=False,
-        ignore_missing_keys=False):
-    '''
+    old, new, ignore_keys=None, ignore_order=False, ignore_missing_keys=False
+):
+    """
     Performs a recursive diff on mappings and/or iterables and returns the result
     in a {'old': values, 'new': values}-style.
     Compares dicts and sets unordered (obviously), OrderedDicts and Lists ordered
@@ -1090,12 +1321,16 @@ def recursive_diff(
         but missing in ``new``. Only works for regular dicts.
 
     :return dict: Returns dict with keys 'old' and 'new' containing the differences.
-    '''
+    """
     ignore_keys = ignore_keys or []
     res = {}
     ret_old = copy.deepcopy(old)
     ret_new = copy.deepcopy(new)
-    if isinstance(old, OrderedDict) and isinstance(new, OrderedDict) and not ignore_order:
+    if (
+        isinstance(old, OrderedDict)
+        and isinstance(new, OrderedDict)
+        and not ignore_order
+    ):
         append_old, append_new = [], []
         if len(old) != len(new):
             min_length = min(len(old), len(new))
@@ -1114,13 +1349,14 @@ def recursive_diff(
                         new[key_new],
                         ignore_keys=ignore_keys,
                         ignore_order=ignore_order,
-                        ignore_missing_keys=ignore_missing_keys)
+                        ignore_missing_keys=ignore_missing_keys,
+                    )
                     if not res:  # Equal
                         del ret_old[key_old]
                         del ret_new[key_new]
                     else:
-                        ret_old[key_old] = res['old']
-                        ret_new[key_new] = res['new']
+                        ret_old[key_old] = res["old"]
+                        ret_new[key_new] = res["new"]
             else:
                 if key_old in ignore_keys:
                     del ret_old[key_old]
@@ -1131,7 +1367,7 @@ def recursive_diff(
             ret_old[item] = old[item]
         for item in append_new:
             ret_new[item] = new[item]
-        ret = {'old': ret_old, 'new': ret_new} if ret_old or ret_new else {}
+        ret = {"old": ret_old, "new": ret_new} if ret_old or ret_new else {}
     elif isinstance(old, Mapping) and isinstance(new, Mapping):
         # Compare unordered
         for key in set(list(old) + list(new)):
@@ -1146,16 +1382,17 @@ def recursive_diff(
                     new[key],
                     ignore_keys=ignore_keys,
                     ignore_order=ignore_order,
-                    ignore_missing_keys=ignore_missing_keys)
+                    ignore_missing_keys=ignore_missing_keys,
+                )
                 if not res:  # Equal
                     del ret_old[key]
                     del ret_new[key]
                 else:
-                    ret_old[key] = res['old']
-                    ret_new[key] = res['new']
-        ret = {'old': ret_old, 'new': ret_new} if ret_old or ret_new else {}
+                    ret_old[key] = res["old"]
+                    ret_new[key] = res["new"]
+        ret = {"old": ret_old, "new": ret_new} if ret_old or ret_new else {}
     elif isinstance(old, set) and isinstance(new, set):
-        ret = {'old': old - new, 'new': new - old} if old - new or new - old else {}
+        ret = {"old": old - new, "new": new - old} if old - new or new - old else {}
     elif is_iter(old) and is_iter(new):
         # Create a list so we can edit on an index-basis.
         list_old = list(ret_old)
@@ -1168,7 +1405,8 @@ def recursive_diff(
                         item_new,
                         ignore_keys=ignore_keys,
                         ignore_order=ignore_order,
-                        ignore_missing_keys=ignore_missing_keys)
+                        ignore_missing_keys=ignore_missing_keys,
+                    )
                     if not res:
                         list_old.remove(item_old)
                         list_new.remove(item_new)
@@ -1181,19 +1419,87 @@ def recursive_diff(
                     iter_new,
                     ignore_keys=ignore_keys,
                     ignore_order=ignore_order,
-                    ignore_missing_keys=ignore_missing_keys)
+                    ignore_missing_keys=ignore_missing_keys,
+                )
                 if not res:  # Equal
                     remove_indices.append(index)
                 else:
-                    list_old[index] = res['old']
-                    list_new[index] = res['new']
+                    list_old[index] = res["old"]
+                    list_new[index] = res["new"]
             for index in reversed(remove_indices):
                 list_old.pop(index)
                 list_new.pop(index)
         # Instantiate a new whatever-it-was using the list as iterable source.
         # This may not be the most optimized in way of speed and memory usage,
         # but it will work for all iterable types.
-        ret = {'old': type(old)(list_old), 'new': type(new)(list_new)} if list_old or list_new else {}
+        ret = (
+            {"old": type(old)(list_old), "new": type(new)(list_new)}
+            if list_old or list_new
+            else {}
+        )
     else:
-        ret = {} if old == new else {'old': ret_old, 'new': ret_new}
+        ret = {} if old == new else {"old": ret_old, "new": ret_new}
     return ret
+
+
+def get_value(obj, path, default=None):
+    """
+    Get the values for a given path.
+
+    :param path:
+        keys of the properties in the tree separated by colons.
+        One segment in the path can be replaced by an id surrounded by curly braces.
+        This will match all items in a list of dictionary.
+
+    :param default:
+        default value to return when no value is found
+
+    :return:
+        a list of dictionaries, with at least the "value" key providing the actual value.
+        If a placeholder was used, the placeholder id will be a key providing the replacement for it.
+        Note that a value that wasn't found in the tree will be an empty list.
+        This ensures we can make the difference with a None value set by the user.
+    """
+    res = [{"value": obj}]
+    if path:
+        key = path[: path.find(":")] if ":" in path else path
+        next_path = path[path.find(":") + 1 :] if ":" in path else None
+
+        if key.startswith("{") and key.endswith("}"):
+            placeholder_name = key[1:-1]
+            # There will be multiple values to get here
+            items = []
+            if obj is None:
+                return res
+            if isinstance(obj, dict):
+                items = obj.items()
+            elif isinstance(obj, list):
+                items = enumerate(obj)
+
+            def _append_placeholder(value_dict, key):
+                value_dict[placeholder_name] = key
+                return value_dict
+
+            values = [
+                [
+                    _append_placeholder(item, key)
+                    for item in get_value(val, next_path, default)
+                ]
+                for key, val in items
+            ]
+
+            # flatten the list
+            values = [y for x in values for y in x]
+            return values
+        elif isinstance(obj, dict):
+            if key not in obj.keys():
+                return [{"value": default}]
+
+            value = obj.get(key)
+            if res is not None:
+                res = get_value(value, next_path, default)
+            else:
+                res = [{"value": value}]
+        else:
+            return [{"value": default if obj is not None else obj}]
+    return res
diff --git a/salt/utils/xmlutil.py b/salt/utils/xmlutil.py
index 6d8d74fd3f..2b9c7bf43f 100644
--- a/salt/utils/xmlutil.py
+++ b/salt/utils/xmlutil.py
@@ -1,30 +1,34 @@
-# -*- coding: utf-8 -*-
-'''
+"""
 Various XML utilities
-'''
+"""
 
 # Import Python libs
-from __future__ import absolute_import, print_function, unicode_literals
+import re
+import string  # pylint: disable=deprecated-module
+from xml.etree import ElementTree
+
+# Import salt libs
+import salt.utils.data
 
 
 def _conv_name(x):
-    '''
+    """
     If this XML tree has an xmlns attribute, then etree will add it
     to the beginning of the tag, like: "{http://path}tag".
-    '''
-    if '}' in x:
-        comps = x.split('}')
+    """
+    if "}" in x:
+        comps = x.split("}")
         name = comps[1]
         return name
     return x
 
 
 def _to_dict(xmltree):
-    '''
+    """
     Converts an XML ElementTree to a dictionary that only contains items.
     This is the default behavior in version 2017.7. This will default to prevent
     unexpected parsing issues on modules dependant on this.
-    '''
+    """
     # If this object has no children, the for..loop below will return nothing
     # for it, so just return a single dict representing it.
     if len(xmltree.getchildren()) < 1:
@@ -51,9 +55,9 @@ def _to_dict(xmltree):
 
 
 def _to_full_dict(xmltree):
-    '''
+    """
     Returns the full XML dictionary including attributes.
-    '''
+    """
     xmldict = {}
 
     for attrName, attrValue in xmltree.attrib.items():
@@ -87,15 +91,234 @@ def _to_full_dict(xmltree):
 
 
 def to_dict(xmltree, attr=False):
-    '''
+    """
     Convert an XML tree into a dict. The tree that is passed in must be an
     ElementTree object.
     Args:
         xmltree: An ElementTree object.
         attr: If true, attributes will be parsed. If false, they will be ignored.
 
-    '''
+    """
     if attr:
         return _to_full_dict(xmltree)
     else:
         return _to_dict(xmltree)
+
+
+def get_xml_node(node, xpath):
+    """
+    Get an XML node using a path (super simple xpath showing complete node ancestry).
+    This also creates the missing nodes.
+
+    The supported XPath can contain elements filtering using [@attr='value'].
+
+    Args:
+        node: an Element object
+        xpath: simple XPath to look for.
+    """
+    if not xpath.startswith("./"):
+        xpath = "./{}".format(xpath)
+    res = node.find(xpath)
+    if res is None:
+        parent_xpath = xpath[: xpath.rfind("/")]
+        parent = node.find(parent_xpath)
+        if parent is None:
+            parent = get_xml_node(node, parent_xpath)
+        segment = xpath[xpath.rfind("/") + 1 :]
+        # We may have [] filter in the segment
+        matcher = re.match(
+            r"""(?P<tag>[^[]+)(?:\[@(?P<attr>\w+)=["'](?P<value>[^"']+)["']])?""",
+            segment,
+        )
+        attrib = (
+            {matcher.group("attr"): matcher.group("value")}
+            if matcher.group("attr") and matcher.group("value")
+            else {}
+        )
+        res = ElementTree.SubElement(parent, matcher.group("tag"), attrib)
+    return res
+
+
+def set_node_text(node, value):
+    """
+    Function to use in the ``set`` value in the :py:func:`change_xml` mapping items to set the text.
+    This is the default.
+
+    :param node: the node to set the text to
+    :param value: the value to set
+    """
+    node.text = str(value)
+
+
+def clean_node(parent_map, node, ignored=None):
+    """
+    Remove the node from its parent if it has no attribute but the ignored ones, no text and no child.
+    Recursively called up to the document root to ensure no empty node is left.
+
+    :param parent_map: dictionary mapping each node to its parent
+    :param node: the node to clean
+    :param ignored: a list of ignored attributes.
+    """
+    has_text = node.text is not None and node.text.strip()
+    parent = parent_map.get(node)
+    if (
+        len(node.attrib.keys() - (ignored or [])) == 0
+        and not list(node)
+        and not has_text
+    ):
+        parent.remove(node)
+    # Clean parent nodes if needed
+    if parent is not None:
+        clean_node(parent_map, parent, ignored)
+
+
+def del_text(parent_map, node):
+    """
+    Function to use as ``del`` value in the :py:func:`change_xml` mapping items to remove the text.
+    This is the default function.
+    Calls :py:func:`clean_node` before returning.
+    """
+    parent = parent_map[node]
+    parent.remove(node)
+    clean_node(parent, node)
+
+
+def del_attribute(attribute, ignored=None):
+    """
+    Helper returning a function to use as ``del`` value in the :py:func:`change_xml` mapping items to
+    remove an attribute.
+
+    The generated function calls :py:func:`clean_node` before returning.
+
+    :param attribute: the name of the attribute to remove
+    :param ignored: the list of attributes to ignore during the cleanup
+
+    :return: the function called by :py:func:`change_xml`.
+    """
+
+    def _do_delete(parent_map, node):
+        if attribute not in node.keys():
+            return
+        node.attrib.pop(attribute)
+        clean_node(parent_map, node, ignored)
+
+    return _do_delete
+
+
+def change_xml(doc, data, mapping):
+    """
+    Change an XML ElementTree document according.
+
+    :param doc: the ElementTree parsed XML document to modify
+    :param data: the dictionary of values used to modify the XML.
+    :param mapping: a list of items describing how to modify the XML document.
+        Each item is a dictionary containing the following keys:
+
+        .. glossary::
+            path
+                the path to the value to set or remove in the ``data`` parameter.
+                See :py:func:`salt.utils.data.get_value <salt.utils.data.get_value>` for the format
+                of the value.
+
+            xpath
+                Simplified XPath expression used to locate the change in the XML tree.
+                See :py:func:`get_xml_node` documentation for details on the supported XPath syntax
+
+            get
+                function gettin the value from the XML.
+                Takes a single parameter for the XML node found by the XPath expression.
+                Default returns the node text value.
+                This may be used to return an attribute or to perform value transformation.
+
+            set
+                function setting the value in the XML.
+                Takes two parameters for the XML node and the value to set.
+                Default is to set the text value.
+
+            del
+                function deleting the value in the XML.
+                Takes two parameters for the parent node and the node matched by the XPath.
+                Default is to remove the text value.
+                More cleanup may be performed, see the :py:func:`clean_node` function for details.
+
+            convert
+                function modifying the user-provided value right before comparing it with the one from the XML.
+                Takes the value as single parameter.
+                Default is to apply no conversion.
+
+    :return: ``True`` if the XML has been modified, ``False`` otherwise.
+    """
+    need_update = False
+    for param in mapping:
+        # Get the value from the function parameter using the path-like description
+        # Using an empty list as a default value will cause values not provided by the user
+        # to be left untouched, as opposed to explicit None unsetting the value
+        values = salt.utils.data.get_value(data, param["path"], [])
+        xpath = param["xpath"]
+        # Prepend the xpath with ./ to handle the root more easily
+        if not xpath.startswith("./"):
+            xpath = "./{}".format(xpath)
+
+        placeholders = [
+            s[1:-1]
+            for s in param["path"].split(":")
+            if s.startswith("{") and s.endswith("}")
+        ]
+
+        ctx = {placeholder: "$$$" for placeholder in placeholders}
+        all_nodes_xpath = string.Template(xpath).substitute(ctx)
+        all_nodes_xpath = re.sub(
+            r"""(?:=['"]\$\$\$["'])|(?:\[\$\$\$\])""", "", all_nodes_xpath
+        )
+
+        # Store the nodes that are not removed for later cleanup
+        kept_nodes = set()
+
+        for value_item in values:
+            new_value = value_item["value"]
+
+            # Only handle simple type values. Use multiple entries or a custom get for dict or lists
+            if isinstance(new_value, list) or isinstance(new_value, dict):
+                continue
+
+            if new_value is not None:
+                ctx = {
+                    placeholder: value_item.get(placeholder, "")
+                    for placeholder in placeholders
+                }
+                node_xpath = string.Template(xpath).substitute(ctx)
+                node = get_xml_node(doc, node_xpath)
+
+                kept_nodes.add(node)
+
+                get_fn = param.get("get", lambda n: n.text)
+                set_fn = param.get("set", set_node_text)
+                current_value = get_fn(node)
+
+                # Do we need to apply some conversion to the user-provided value?
+                convert_fn = param.get("convert")
+                if convert_fn:
+                    new_value = convert_fn(new_value)
+
+                if current_value != new_value:
+                    set_fn(node, new_value)
+                    need_update = True
+            else:
+                nodes = doc.findall(all_nodes_xpath)
+                del_fn = param.get("del", del_text)
+                parent_map = {c: p for p in doc.iter() for c in p}
+                for node in nodes:
+                    del_fn(parent_map, node)
+                    need_update = True
+
+        # Clean the left over XML elements if there were placeholders
+        if placeholders and values[0].get("value") != []:
+            all_nodes = set(doc.findall(all_nodes_xpath))
+            to_remove = all_nodes - kept_nodes
+            del_fn = param.get("del", del_text)
+            parent_map = {c: p for p in doc.iter() for c in p}
+            for node in to_remove:
+                del_fn(parent_map, node)
+                need_update = True
+
+    return need_update
diff --git a/tests/pytests/unit/utils/test_data.py b/tests/pytests/unit/utils/test_data.py
new file mode 100644
index 0000000000..b3f0ba04ae
--- /dev/null
+++ b/tests/pytests/unit/utils/test_data.py
@@ -0,0 +1,57 @@
+import salt.utils.data
+
+
+def test_get_value_simple_path():
+    data = {"a": {"b": {"c": "foo"}}}
+    assert [{"value": "foo"}] == salt.utils.data.get_value(data, "a:b:c")
+
+
+def test_get_value_placeholder_dict():
+    data = {"a": {"b": {"name": "foo"}, "c": {"name": "bar"}}}
+    assert [
+        {"value": "foo", "id": "b"},
+        {"value": "bar", "id": "c"},
+    ] == salt.utils.data.get_value(data, "a:{id}:name")
+
+
+def test_get_value_placeholder_list():
+    data = {"a": [{"name": "foo"}, {"name": "bar"}]}
+    assert [
+        {"value": "foo", "id": 0},
+        {"value": "bar", "id": 1},
+    ] == salt.utils.data.get_value(data, "a:{id}:name")
+
+
+def test_get_value_nested_placeholder():
+    data = {
+        "a": {
+            "b": {"b1": {"name": "foo1"}, "b2": {"name": "foo2"}},
+            "c": {"c1": {"name": "bar"}},
+        }
+    }
+    assert [
+        {"value": "foo1", "id": "b", "sub": "b1"},
+        {"value": "foo2", "id": "b", "sub": "b2"},
+        {"value": "bar", "id": "c", "sub": "c1"},
+    ] == salt.utils.data.get_value(data, "a:{id}:{sub}:name")
+
+
+def test_get_value_nested_notfound():
+    data = {"a": {"b": {"c": "foo"}}}
+    assert [{"value": []}] == salt.utils.data.get_value(data, "a:b:d", [])
+
+
+def test_get_value_not_found():
+    assert [{"value": []}] == salt.utils.data.get_value({}, "a", [])
+
+
+def test_get_value_none():
+    assert [{"value": None}] == salt.utils.data.get_value({"a": None}, "a")
+
+
+def test_get_value_simple_type_path():
+    assert [{"value": []}] == salt.utils.data.get_value({"a": 1024}, "a:b", [])
+
+
+def test_get_value_None_path():
+    assert [{"value": None}] == salt.utils.data.get_value({"a": None}, "a:b", [])
diff --git a/tests/pytests/unit/utils/test_xmlutil.py b/tests/pytests/unit/utils/test_xmlutil.py
new file mode 100644
index 0000000000..081cc64193
--- /dev/null
+++ b/tests/pytests/unit/utils/test_xmlutil.py
@@ -0,0 +1,169 @@
+import pytest
+import salt.utils.xmlutil as xml
+from salt._compat import ElementTree as ET
+
+
+@pytest.fixture
+def xml_doc():
+    return ET.fromstring(
+        """
+        <domain>
+            <name>test01</name>
+            <memory unit="MiB">1024</memory>
+            <cpu>
+                <topology sockets="1"/>
+            </cpu>
+            <vcpus>
+              <vcpu enabled="yes" id="1"/>
+            </vcpus>
+        </domain>
+    """
+    )
+
+
+def test_change_xml_text(xml_doc):
+    ret = xml.change_xml(
+        xml_doc, {"name": "test02"}, [{"path": "name", "xpath": "name"}]
+    )
+    assert ret
+    assert "test02" == xml_doc.find("name").text
+
+
+def test_change_xml_text_nochange(xml_doc):
+    ret = xml.change_xml(
+        xml_doc, {"name": "test01"}, [{"path": "name", "xpath": "name"}]
+    )
+    assert not ret
+
+
+def test_change_xml_text_notdefined(xml_doc):
+    ret = xml.change_xml(xml_doc, {}, [{"path": "name", "xpath": "name"}])
+    assert not ret
+
+
+def test_change_xml_text_removed(xml_doc):
+    ret = xml.change_xml(xml_doc, {"name": None}, [{"path": "name", "xpath": "name"}])
+    assert ret
+    assert xml_doc.find("name") is None
+
+
+def test_change_xml_text_add(xml_doc):
+    ret = xml.change_xml(
+        xml_doc,
+        {"cpu": {"vendor": "ACME"}},
+        [{"path": "cpu:vendor", "xpath": "cpu/vendor"}],
+    )
+    assert ret
+    assert "ACME" == xml_doc.find("cpu/vendor").text
+
+
+def test_change_xml_convert(xml_doc):
+    ret = xml.change_xml(
+        xml_doc,
+        {"mem": 2},
+        [{"path": "mem", "xpath": "memory", "convert": lambda v: v * 1024}],
+    )
+    assert ret
+    assert "2048" == xml_doc.find("memory").text
+
+
+def test_change_xml_attr(xml_doc):
+    ret = xml.change_xml(
+        xml_doc,
+        {"cpu": {"topology": {"cores": 4}}},
+        [
+            {
+                "path": "cpu:topology:cores",
+                "xpath": "cpu/topology",
+                "get": lambda n: int(n.get("cores")) if n.get("cores") else None,
+                "set": lambda n, v: n.set("cores", str(v)),
+                "del": xml.del_attribute("cores"),
+            }
+        ],
+    )
+    assert ret
+    assert "4" == xml_doc.find("cpu/topology").get("cores")
+
+
+def test_change_xml_attr_unchanged(xml_doc):
+    ret = xml.change_xml(
+        xml_doc,
+        {"cpu": {"topology": {"sockets": 1}}},
+        [
+            {
+                "path": "cpu:topology:sockets",
+                "xpath": "cpu/topology",
+                "get": lambda n: int(n.get("sockets")) if n.get("sockets") else None,
+                "set": lambda n, v: n.set("sockets", str(v)),
+                "del": xml.del_attribute("sockets"),
+            }
+        ],
+    )
+    assert not ret
+
+
+def test_change_xml_attr_remove(xml_doc):
+    ret = xml.change_xml(
+        xml_doc,
+        {"cpu": {"topology": {"sockets": None}}},
+        [
+            {
+                "path": "cpu:topology:sockets",
+                "xpath": "./cpu/topology",
+                "get": lambda n: int(n.get("sockets")) if n.get("sockets") else None,
+                "set": lambda n, v: n.set("sockets", str(v)),
+                "del": xml.del_attribute("sockets"),
+            }
+        ],
+    )
+    assert ret
+    assert xml_doc.find("cpu") is None
+
+
+def test_change_xml_not_simple_value(xml_doc):
+    ret = xml.change_xml(
+        xml_doc,
+        {"cpu": {"topology": {"sockets": None}}},
+        [{"path": "cpu", "xpath": "vcpu", "get": lambda n: int(n.text)}],
+    )
+    assert not ret
+
+
+def test_change_xml_template(xml_doc):
+    ret = xml.change_xml(
+        xml_doc,
+        {"cpu": {"vcpus": {2: {"enabled": True}, 4: {"enabled": False}}}},
+        [
+            {
+                "path": "cpu:vcpus:{id}:enabled",
+                "xpath": "vcpus/vcpu[@id='$id']",
+                "convert": lambda v: "yes" if v else "no",
+                "get": lambda n: n.get("enabled"),
+                "set": lambda n, v: n.set("enabled", v),
+                "del": xml.del_attribute("enabled", ["id"]),
+            },
+        ],
+    )
+    assert ret
+    assert xml_doc.find("vcpus/vcpu[@id='1']") is None
+    assert "yes" == xml_doc.find("vcpus/vcpu[@id='2']").get("enabled")
+    assert "no" == xml_doc.find("vcpus/vcpu[@id='4']").get("enabled")
+
+
+def test_change_xml_template_remove(xml_doc):
+    ret = xml.change_xml(
+        xml_doc,
+        {"cpu": {"vcpus": None}},
+        [
+            {
+                "path": "cpu:vcpus:{id}:enabled",
+                "xpath": "vcpus/vcpu[@id='$id']",
+                "convert": lambda v: "yes" if v else "no",
+                "get": lambda n: n.get("enabled"),
+                "set": lambda n, v: n.set("enabled", v),
+                "del": xml.del_attribute("enabled", ["id"]),
+            },
+        ],
+    )
+    assert ret
+    assert xml_doc.find("vcpus") is None
diff --git a/tests/unit/modules/test_virt.py b/tests/unit/modules/test_virt.py
index d3988464f6..5ec8de77e7 100644
--- a/tests/unit/modules/test_virt.py
+++ b/tests/unit/modules/test_virt.py
@@ -1,4 +1,3 @@
-# -*- coding: utf-8 -*-
 """
 virt execution module unit tests
 """
@@ -6,7 +5,6 @@ virt execution module unit tests
 # pylint: disable=3rd-party-module-not-gated
 
 # Import python libs
-from __future__ import absolute_import, print_function, unicode_literals
 
 import datetime
 import os
@@ -23,9 +21,6 @@ import salt.utils.yaml
 from salt._compat import ElementTree as ET
 from salt.exceptions import CommandExecutionError, SaltInvocationError
 
-# Import third party libs
-from salt.ext import six
-
 # pylint: disable=import-error
 from salt.ext.six.moves import range  # pylint: disable=redefined-builtin
 
@@ -136,7 +131,7 @@ class VirtTestCase(TestCase, LoaderModuleMockMixin):
                     "model": "virtio",
                     "filename": "myvm_system.qcow2",
                     "image": "/path/to/image",
-                    "source_file": "{0}{1}myvm_system.qcow2".format(root_dir, os.sep),
+                    "source_file": "{}{}myvm_system.qcow2".format(root_dir, os.sep),
                 },
                 {
                     "name": "data",
@@ -145,7 +140,7 @@ class VirtTestCase(TestCase, LoaderModuleMockMixin):
                     "format": "raw",
                     "model": "virtio",
                     "filename": "myvm_data.raw",
-                    "source_file": "{0}{1}myvm_data.raw".format(root_dir, os.sep),
+                    "source_file": "{}{}myvm_data.raw".format(root_dir, os.sep),
                 },
             ],
             disks,
@@ -582,8 +577,8 @@ class VirtTestCase(TestCase, LoaderModuleMockMixin):
         self.assertIsNone(root.get("type"))
         self.assertEqual(root.find("name").text, "vmname/system.vmdk")
         self.assertEqual(root.find("capacity").attrib["unit"], "KiB")
-        self.assertEqual(root.find("capacity").text, six.text_type(8192 * 1024))
-        self.assertEqual(root.find("allocation").text, six.text_type(0))
+        self.assertEqual(root.find("capacity").text, str(8192 * 1024))
+        self.assertEqual(root.find("allocation").text, str(0))
         self.assertEqual(root.find("target/format").get("type"), "vmdk")
         self.assertIsNone(root.find("target/permissions"))
         self.assertIsNone(root.find("target/nocow"))
@@ -615,9 +610,9 @@ class VirtTestCase(TestCase, LoaderModuleMockMixin):
         self.assertIsNone(root.find("target/path"))
         self.assertEqual(root.find("target/format").get("type"), "qcow2")
         self.assertEqual(root.find("capacity").attrib["unit"], "KiB")
-        self.assertEqual(root.find("capacity").text, six.text_type(8192 * 1024))
+        self.assertEqual(root.find("capacity").text, str(8192 * 1024))
         self.assertEqual(root.find("capacity").attrib["unit"], "KiB")
-        self.assertEqual(root.find("allocation").text, six.text_type(4096 * 1024))
+        self.assertEqual(root.find("allocation").text, str(4096 * 1024))
         self.assertEqual(root.find("target/permissions/mode").text, "0775")
         self.assertEqual(root.find("target/permissions/owner").text, "123")
         self.assertEqual(root.find("target/permissions/group").text, "456")
@@ -638,7 +633,7 @@ class VirtTestCase(TestCase, LoaderModuleMockMixin):
         root = ET.fromstring(xml_data)
         self.assertEqual(root.attrib["type"], "kvm")
         self.assertEqual(root.find("vcpu").text, "1")
-        self.assertEqual(root.find("memory").text, six.text_type(512 * 1024))
+        self.assertEqual(root.find("memory").text, str(512 * 1024))
         self.assertEqual(root.find("memory").attrib["unit"], "KiB")
 
         disks = root.findall(".//disk")
@@ -671,7 +666,7 @@ class VirtTestCase(TestCase, LoaderModuleMockMixin):
         root = ET.fromstring(xml_data)
         self.assertEqual(root.attrib["type"], "vmware")
         self.assertEqual(root.find("vcpu").text, "1")
-        self.assertEqual(root.find("memory").text, six.text_type(512 * 1024))
+        self.assertEqual(root.find("memory").text, str(512 * 1024))
         self.assertEqual(root.find("memory").attrib["unit"], "KiB")
 
         disks = root.findall(".//disk")
@@ -714,7 +709,7 @@ class VirtTestCase(TestCase, LoaderModuleMockMixin):
             root = ET.fromstring(xml_data)
             self.assertEqual(root.attrib["type"], "xen")
             self.assertEqual(root.find("vcpu").text, "1")
-            self.assertEqual(root.find("memory").text, six.text_type(512 * 1024))
+            self.assertEqual(root.find("memory").text, str(512 * 1024))
             self.assertEqual(root.find("memory").attrib["unit"], "KiB")
             self.assertEqual(
                 root.find(".//kernel").text, "/usr/lib/grub2/x86_64-xen/grub.xen"
@@ -768,7 +763,7 @@ class VirtTestCase(TestCase, LoaderModuleMockMixin):
             root = ET.fromstring(xml_data)
             self.assertEqual(root.attrib["type"], "vmware")
             self.assertEqual(root.find("vcpu").text, "1")
-            self.assertEqual(root.find("memory").text, six.text_type(512 * 1024))
+            self.assertEqual(root.find("memory").text, str(512 * 1024))
             self.assertEqual(root.find("memory").attrib["unit"], "KiB")
             self.assertTrue(len(root.findall(".//disk")) == 2)
             self.assertTrue(len(root.findall(".//interface")) == 2)
@@ -801,7 +796,7 @@ class VirtTestCase(TestCase, LoaderModuleMockMixin):
             root = ET.fromstring(xml_data)
             self.assertEqual(root.attrib["type"], "kvm")
             self.assertEqual(root.find("vcpu").text, "1")
-            self.assertEqual(root.find("memory").text, six.text_type(512 * 1024))
+            self.assertEqual(root.find("memory").text, str(512 * 1024))
             self.assertEqual(root.find("memory").attrib["unit"], "KiB")
             disks = root.findall(".//disk")
             self.assertTrue(len(disks) == 2)
@@ -1635,7 +1630,7 @@ class VirtTestCase(TestCase, LoaderModuleMockMixin):
                 self.assertIsNone(definition.find("./devices/disk[2]/source"))
                 self.assertEqual(
                     mock_run.call_args[0][0],
-                    'qemu-img create -f qcow2 "{0}" 10240M'.format(expected_disk_path),
+                    'qemu-img create -f qcow2 "{}" 10240M'.format(expected_disk_path),
                 )
                 self.assertEqual(mock_chmod.call_args[0][0], expected_disk_path)
 
@@ -1729,11 +1724,12 @@ class VirtTestCase(TestCase, LoaderModuleMockMixin):
               <vcpu placement='auto'>1</vcpu>
               <os>
                 <type arch='x86_64' machine='pc-i440fx-2.6'>hvm</type>
+                <boot dev="hd"/>
               </os>
               <devices>
                 <disk type='file' device='disk'>
                   <driver name='qemu' type='qcow2'/>
-                  <source file='{0}{1}my_vm_system.qcow2'/>
+                  <source file='{}{}my_vm_system.qcow2'/>
                   <backingStore/>
                   <target dev='vda' bus='virtio'/>
                   <alias name='virtio-disk0'/>
@@ -1850,17 +1846,36 @@ class VirtTestCase(TestCase, LoaderModuleMockMixin):
             "cmdline": "console=ttyS0 ks=http://example.com/f8-i386/os/",
         }
 
-        boot_uefi = {
-            "loader": "/usr/share/OVMF/OVMF_CODE.fd",
-            "nvram": "/usr/share/OVMF/OVMF_VARS.ms.fd",
-        }
+        # Update boot devices case
+        define_mock.reset_mock()
+        self.assertEqual(
+            {
+                "definition": True,
+                "disk": {"attached": [], "detached": [], "updated": []},
+                "interface": {"attached": [], "detached": []},
+            },
+            virt.update("my_vm", boot_dev="cdrom network hd"),
+        )
+        setxml = ET.fromstring(define_mock.call_args[0][0])
+        self.assertEqual(
+            ["cdrom", "network", "hd"],
+            [node.get("dev") for node in setxml.findall("os/boot")],
+        )
 
-        invalid_boot = {
-            "loader": "/usr/share/OVMF/OVMF_CODE.fd",
-            "initrd": "/root/f8-i386-initrd",
-        }
+        # Update unchanged boot devices case
+        define_mock.reset_mock()
+        self.assertEqual(
+            {
+                "definition": False,
+                "disk": {"attached": [], "detached": [], "updated": []},
+                "interface": {"attached": [], "detached": []},
+            },
+            virt.update("my_vm", boot_dev="hd"),
+        )
+        define_mock.assert_not_called()
 
         # Update with boot parameter case
+        define_mock.reset_mock()
         self.assertEqual(
             {
                 "definition": True,
@@ -1884,6 +1899,11 @@ class VirtTestCase(TestCase, LoaderModuleMockMixin):
             "console=ttyS0 ks=http://example.com/f8-i386/os/",
         )
 
+        boot_uefi = {
+            "loader": "/usr/share/OVMF/OVMF_CODE.fd",
+            "nvram": "/usr/share/OVMF/OVMF_VARS.ms.fd",
+        }
+
         self.assertEqual(
             {
                 "definition": True,
@@ -1903,9 +1923,28 @@ class VirtTestCase(TestCase, LoaderModuleMockMixin):
             "/usr/share/OVMF/OVMF_VARS.ms.fd",
         )
 
+        self.assertEqual(
+            {
+                "definition": True,
+                "disk": {"attached": [], "detached": [], "updated": []},
+                "interface": {"attached": [], "detached": []},
+            },
+            virt.update("my_vm", boot={"efi": True}),
+        )
+        setxml = ET.fromstring(define_mock.call_args[0][0])
+        self.assertEqual(setxml.find("os").attrib.get("firmware"), "efi")
+
+        invalid_boot = {
+            "loader": "/usr/share/OVMF/OVMF_CODE.fd",
+            "initrd": "/root/f8-i386-initrd",
+        }
+
         with self.assertRaises(SaltInvocationError):
             virt.update("my_vm", boot=invalid_boot)
 
+        with self.assertRaises(SaltInvocationError):
+            virt.update("my_vm", boot={"efi": "Not a boolean value"})
+
         # Update memory case
         setmem_mock = MagicMock(return_value=0)
         domain_mock.setMemoryFlags = setmem_mock
@@ -1955,7 +1994,7 @@ class VirtTestCase(TestCase, LoaderModuleMockMixin):
                 )  # pylint: disable=no-member
                 self.assertEqual(
                     mock_run.call_args[0][0],
-                    'qemu-img create -f qcow2 "{0}" 2048M'.format(added_disk_path),
+                    'qemu-img create -f qcow2 "{}" 2048M'.format(added_disk_path),
                 )
                 self.assertEqual(mock_chmod.call_args[0][0], added_disk_path)
                 self.assertListEqual(
@@ -2397,6 +2436,43 @@ class VirtTestCase(TestCase, LoaderModuleMockMixin):
             ],
         )
 
+    def test_update_xen_boot_params(self):
+        """
+        Test virt.update() a Xen definition no boot parameter.
+        """
+        root_dir = os.path.join(salt.syspaths.ROOT_DIR, "srv", "salt-images")
+        xml_boot = """
+            <domain type='xen' id='8'>
+              <name>vm</name>
+              <memory unit='KiB'>1048576</memory>
+              <currentMemory unit='KiB'>1048576</currentMemory>
+              <vcpu placement='auto'>1</vcpu>
+              <os>
+                <type arch='x86_64' machine='xenfv'>hvm</type>
+                <loader type='rom'>/usr/lib/xen/boot/hvmloader</loader>
+              </os>
+            </domain>
+        """
+        domain_mock_boot = self.set_mock_vm("vm", xml_boot)
+        domain_mock_boot.OSType = MagicMock(return_value="hvm")
+        define_mock_boot = MagicMock(return_value=True)
+        define_mock_boot.setVcpusFlags = MagicMock(return_value=0)
+        self.mock_conn.defineXML = define_mock_boot
+        self.assertEqual(
+            {
+                "cpu": False,
+                "definition": True,
+                "disk": {"attached": [], "detached": [], "updated": []},
+                "interface": {"attached": [], "detached": []},
+            },
+            virt.update("vm", cpu=2),
+        )
+        setxml = ET.fromstring(define_mock_boot.call_args[0][0])
+        self.assertEqual(setxml.find("os").find("loader").attrib.get("type"), "rom")
+        self.assertEqual(
+            setxml.find("os").find("loader").text, "/usr/lib/xen/boot/hvmloader"
+        )
+
     def test_update_existing_boot_params(self):
         """
         Test virt.update() with existing boot parameters.
@@ -2537,6 +2613,18 @@ class VirtTestCase(TestCase, LoaderModuleMockMixin):
         self.assertEqual(setxml.find("os").find("initrd"), None)
         self.assertEqual(setxml.find("os").find("cmdline"), None)
 
+        self.assertEqual(
+            {
+                "definition": True,
+                "disk": {"attached": [], "detached": [], "updated": []},
+                "interface": {"attached": [], "detached": []},
+            },
+            virt.update("vm_with_boot_param", boot={"efi": False}),
+        )
+        setxml = ET.fromstring(define_mock_boot.call_args[0][0])
+        self.assertEqual(setxml.find("os").find("nvram"), None)
+        self.assertEqual(setxml.find("os").find("loader"), None)
+
         self.assertEqual(
             {
                 "definition": True,
@@ -2582,7 +2670,7 @@ class VirtTestCase(TestCase, LoaderModuleMockMixin):
             salt.modules.config.__opts__, mock_config  # pylint: disable=no-member
         ):
 
-            for name in six.iterkeys(mock_config["virt"]["nic"]):
+            for name in mock_config["virt"]["nic"].keys():
                 profile = salt.modules.virt._nic_profile(name, "kvm")
                 self.assertEqual(len(profile), 2)
 
@@ -3592,8 +3680,7 @@ class VirtTestCase(TestCase, LoaderModuleMockMixin):
             "44454c4c-3400-105a-8033-b3c04f4b344a", caps["host"]["host"]["uuid"]
         )
         self.assertEqual(
-            set(["qemu", "kvm"]),
-            set([domainCaps["domain"] for domainCaps in caps["domains"]]),
+            {"qemu", "kvm"}, {domainCaps["domain"] for domainCaps in caps["domains"]},
         )
 
     def test_network_tag(self):
@@ -3694,9 +3781,9 @@ class VirtTestCase(TestCase, LoaderModuleMockMixin):
         for i in range(2):
             net_mock = MagicMock()
 
-            net_mock.name.return_value = "net{0}".format(i)
+            net_mock.name.return_value = "net{}".format(i)
             net_mock.UUIDString.return_value = "some-uuid"
-            net_mock.bridgeName.return_value = "br{0}".format(i)
+            net_mock.bridgeName.return_value = "br{}".format(i)
             net_mock.autostart.return_value = True
             net_mock.isActive.return_value = False
             net_mock.isPersistent.return_value = True
@@ -4156,8 +4243,8 @@ class VirtTestCase(TestCase, LoaderModuleMockMixin):
         pool_mocks = []
         for i in range(2):
             pool_mock = MagicMock()
-            pool_mock.name.return_value = "pool{0}".format(i)
-            pool_mock.UUIDString.return_value = "some-uuid-{0}".format(i)
+            pool_mock.name.return_value = "pool{}".format(i)
+            pool_mock.UUIDString.return_value = "some-uuid-{}".format(i)
             pool_mock.info.return_value = [0, 1234, 5678, 123]
             pool_mock.autostart.return_value = True
             pool_mock.isPersistent.return_value = True
@@ -4257,7 +4344,6 @@ class VirtTestCase(TestCase, LoaderModuleMockMixin):
         """
         mock_pool = MagicMock()
         mock_pool.delete = MagicMock(return_value=0)
-        mock_pool.XMLDesc.return_value = "<pool type='dir'/>"
         self.mock_conn.storagePoolLookupByName = MagicMock(return_value=mock_pool)
 
         res = virt.pool_delete("test-pool")
@@ -4271,12 +4357,12 @@ class VirtTestCase(TestCase, LoaderModuleMockMixin):
             self.mock_libvirt.VIR_STORAGE_POOL_DELETE_NORMAL
         )
 
-    def test_pool_delete_secret(self):
+    def test_pool_undefine_secret(self):
         """
-        Test virt.pool_delete function where the pool has a secret
+        Test virt.pool_undefine function where the pool has a secret
         """
         mock_pool = MagicMock()
-        mock_pool.delete = MagicMock(return_value=0)
+        mock_pool.undefine = MagicMock(return_value=0)
         mock_pool.XMLDesc.return_value = """
             <pool type='rbd'>
               <name>test-ses</name>
@@ -4293,16 +4379,11 @@ class VirtTestCase(TestCase, LoaderModuleMockMixin):
         mock_undefine = MagicMock(return_value=0)
         self.mock_conn.secretLookupByUsage.return_value.undefine = mock_undefine
 
-        res = virt.pool_delete("test-ses")
+        res = virt.pool_undefine("test-ses")
         self.assertTrue(res)
 
         self.mock_conn.storagePoolLookupByName.assert_called_once_with("test-ses")
-
-        # Shouldn't be called with another parameter so far since those are not implemented
-        # and thus throwing exceptions.
-        mock_pool.delete.assert_called_once_with(
-            self.mock_libvirt.VIR_STORAGE_POOL_DELETE_NORMAL
-        )
+        mock_pool.undefine.assert_called_once_with()
 
         self.mock_conn.secretLookupByUsage.assert_called_once_with(
             self.mock_libvirt.VIR_SECRET_USAGE_TYPE_CEPH, "pool_test-ses"
@@ -4571,24 +4652,6 @@ class VirtTestCase(TestCase, LoaderModuleMockMixin):
           </source>
         </pool>"""
 
-        expected_xml = (
-            '<pool type="rbd">'
-            "<name>default</name>"
-            "<uuid>20fbe05c-ab40-418a-9afa-136d512f0ede</uuid>"
-            '<capacity unit="bytes">1999421108224</capacity>'
-            '<allocation unit="bytes">713207042048</allocation>'
-            '<available unit="bytes">1286214066176</available>'
-            "<source>"
-            '<host name="ses4.tf.local" />'
-            '<host name="ses5.tf.local" />'
-            '<auth type="ceph" username="libvirt">'
-            '<secret uuid="14e9a0f1-8fbf-4097-b816-5b094c182212" />'
-            "</auth>"
-            "<name>iscsi-images</name>"
-            "</source>"
-            "</pool>"
-        )
-
         mock_secret = MagicMock()
         self.mock_conn.secretLookupByUUIDString = MagicMock(return_value=mock_secret)
 
@@ -4609,6 +4672,23 @@ class VirtTestCase(TestCase, LoaderModuleMockMixin):
         self.mock_conn.storagePoolDefineXML.assert_not_called()
         mock_secret.setValue.assert_called_once_with(b"secret")
 
+        # Case where the secret can't be found
+        self.mock_conn.secretLookupByUUIDString = MagicMock(
+            side_effect=self.mock_libvirt.libvirtError("secret not found")
+        )
+        self.assertFalse(
+            virt.pool_update(
+                "default",
+                "rbd",
+                source_name="iscsi-images",
+                source_hosts=["ses4.tf.local", "ses5.tf.local"],
+                source_auth={"username": "libvirt", "password": "c2VjcmV0"},
+            )
+        )
+        self.mock_conn.storagePoolDefineXML.assert_not_called()
+        self.mock_conn.secretDefineXML.assert_called_once()
+        mock_secret.setValue.assert_called_once_with(b"secret")
+
     def test_pool_update_password_create(self):
         """
         Test the pool_update function, where the password only is changed
@@ -4695,11 +4775,11 @@ class VirtTestCase(TestCase, LoaderModuleMockMixin):
         for idx, disk in enumerate(vms_disks):
             vm = MagicMock()
             # pylint: disable=no-member
-            vm.name.return_value = "vm{0}".format(idx)
+            vm.name.return_value = "vm{}".format(idx)
             vm.XMLDesc.return_value = """
                     <domain type='kvm' id='1'>
-                      <name>vm{0}</name>
-                      <devices>{1}</devices>
+                      <name>vm{}</name>
+                      <devices>{}</devices>
                     </domain>
                 """.format(
                 idx, disk
@@ -4760,7 +4840,7 @@ class VirtTestCase(TestCase, LoaderModuleMockMixin):
                 # pylint: disable=no-member
                 mock_volume.name.return_value = vol_data["name"]
                 mock_volume.key.return_value = vol_data["key"]
-                mock_volume.path.return_value = "/path/to/{0}.qcow2".format(
+                mock_volume.path.return_value = "/path/to/{}.qcow2".format(
                     vol_data["name"]
                 )
                 if vol_data["info"]:
@@ -4769,7 +4849,7 @@ class VirtTestCase(TestCase, LoaderModuleMockMixin):
                         """
                         <backingStore>
                           <format type="qcow2"/>
-                          <path>{0}</path>
+                          <path>{}</path>
                         </backingStore>
                     """.format(
                             vol_data["backingStore"]
@@ -5234,7 +5314,7 @@ class VirtTestCase(TestCase, LoaderModuleMockMixin):
 
         def create_mock_vm(idx):
             mock_vm = MagicMock()
-            mock_vm.name.return_value = "vm{0}".format(idx)
+            mock_vm.name.return_value = "vm{}".format(idx)
             return mock_vm
 
         mock_vms = [create_mock_vm(idx) for idx in range(3)]
diff --git a/tests/unit/states/test_virt.py b/tests/unit/states/test_virt.py
index c76f8a5fc0..f03159334b 100644
--- a/tests/unit/states/test_virt.py
+++ b/tests/unit/states/test_virt.py
@@ -1,9 +1,7 @@
-# -*- coding: utf-8 -*-
 """
     :codeauthor: Jayesh Kariya <jayeshk@saltstack.com>
 """
 # Import Python libs
-from __future__ import absolute_import, print_function, unicode_literals
 
 import shutil
 import tempfile
@@ -14,7 +12,6 @@ import salt.utils.files
 from salt.exceptions import CommandExecutionError, SaltInvocationError
 
 # Import 3rd-party libs
-from salt.ext import six
 from tests.support.mixins import LoaderModuleMockMixin
 from tests.support.mock import MagicMock, mock_open, patch
 
@@ -37,7 +34,7 @@ class LibvirtMock(MagicMock):  # pylint: disable=too-many-ancestors
             """
             Fake function return error message
             """
-            return six.text_type(self)
+            return str(self)
 
 
 class LibvirtTestCase(TestCase, LoaderModuleMockMixin):
@@ -341,6 +338,7 @@ class LibvirtTestCase(TestCase, LoaderModuleMockMixin):
                         "myvm",
                         cpu=2,
                         mem=2048,
+                        boot_dev="cdrom hd",
                         os_type="linux",
                         arch="i686",
                         vm_type="qemu",
@@ -363,6 +361,7 @@ class LibvirtTestCase(TestCase, LoaderModuleMockMixin):
                     "myvm",
                     cpu=2,
                     mem=2048,
+                    boot_dev="cdrom hd",
                     os_type="linux",
                     arch="i686",
                     disk="prod",
@@ -471,10 +470,13 @@ class LibvirtTestCase(TestCase, LoaderModuleMockMixin):
                         "comment": "Domain myvm updated with live update(s) failures",
                     }
                 )
-                self.assertDictEqual(virt.defined("myvm", cpu=2), ret)
+                self.assertDictEqual(
+                    virt.defined("myvm", cpu=2, boot_dev="cdrom hd"), ret
+                )
                 update_mock.assert_called_with(
                     "myvm",
                     cpu=2,
+                    boot_dev="cdrom hd",
                     mem=None,
                     disk_profile=None,
                     disks=None,
@@ -598,6 +600,7 @@ class LibvirtTestCase(TestCase, LoaderModuleMockMixin):
                     password=None,
                     boot=None,
                     test=True,
+                    boot_dev=None,
                 )
 
             # No changes case
@@ -632,6 +635,7 @@ class LibvirtTestCase(TestCase, LoaderModuleMockMixin):
                     password=None,
                     boot=None,
                     test=True,
+                    boot_dev=None,
                 )
 
     def test_running(self):
@@ -708,6 +712,7 @@ class LibvirtTestCase(TestCase, LoaderModuleMockMixin):
                     install=True,
                     pub_key=None,
                     priv_key=None,
+                    boot_dev=None,
                     connection=None,
                     username=None,
                     password=None,
@@ -769,6 +774,7 @@ class LibvirtTestCase(TestCase, LoaderModuleMockMixin):
                         install=False,
                         pub_key="/path/to/key.pub",
                         priv_key="/path/to/key",
+                        boot_dev="network hd",
                         connection="someconnection",
                         username="libvirtuser",
                         password="supersecret",
@@ -793,6 +799,7 @@ class LibvirtTestCase(TestCase, LoaderModuleMockMixin):
                     start=False,
                     pub_key="/path/to/key.pub",
                     priv_key="/path/to/key",
+                    boot_dev="network hd",
                     connection="someconnection",
                     username="libvirtuser",
                     password="supersecret",
@@ -937,6 +944,7 @@ class LibvirtTestCase(TestCase, LoaderModuleMockMixin):
                     password=None,
                     boot=None,
                     test=False,
+                    boot_dev=None,
                 )
 
             # Failed definition update case
@@ -1055,6 +1063,7 @@ class LibvirtTestCase(TestCase, LoaderModuleMockMixin):
                     password=None,
                     boot=None,
                     test=True,
+                    boot_dev=None,
                 )
                 start_mock.assert_not_called()
 
@@ -1091,6 +1100,7 @@ class LibvirtTestCase(TestCase, LoaderModuleMockMixin):
                     password=None,
                     boot=None,
                     test=True,
+                    boot_dev=None,
                 )
 
     def test_stopped(self):
@@ -1978,6 +1988,72 @@ class LibvirtTestCase(TestCase, LoaderModuleMockMixin):
                     password="secret",
                 )
 
+            # Define a pool that doesn't handle build
+            for mock in mocks:
+                mocks[mock].reset_mock()
+            with patch.dict(
+                virt.__salt__,
+                {  # pylint: disable=no-member
+                    "virt.pool_info": MagicMock(
+                        side_effect=[
+                            {},
+                            {"mypool": {"state": "stopped", "autostart": True}},
+                        ]
+                    ),
+                    "virt.pool_define": mocks["define"],
+                    "virt.pool_build": mocks["build"],
+                    "virt.pool_set_autostart": mocks["autostart"],
+                },
+            ):
+                ret.update(
+                    {
+                        "changes": {"mypool": "Pool defined, marked for autostart"},
+                        "comment": "Pool mypool defined, marked for autostart",
+                    }
+                )
+                self.assertDictEqual(
+                    virt.pool_defined(
+                        "mypool",
+                        ptype="rbd",
+                        source={
+                            "name": "libvirt-pool",
+                            "hosts": ["ses2.tf.local", "ses3.tf.local"],
+                            "auth": {
+                                "username": "libvirt",
+                                "password": "AQAz+PRdtquBBRAASMv7nlMZYfxIyLw3St65Xw==",
+                            },
+                        },
+                        autostart=True,
+                    ),
+                    ret,
+                )
+                mocks["define"].assert_called_with(
+                    "mypool",
+                    ptype="rbd",
+                    target=None,
+                    permissions=None,
+                    source_devices=None,
+                    source_dir=None,
+                    source_adapter=None,
+                    source_hosts=["ses2.tf.local", "ses3.tf.local"],
+                    source_auth={
+                        "username": "libvirt",
+                        "password": "AQAz+PRdtquBBRAASMv7nlMZYfxIyLw3St65Xw==",
+                    },
+                    source_name="libvirt-pool",
+                    source_format=None,
+                    source_initiator=None,
+                    start=False,
+                    transient=False,
+                    connection=None,
+                    username=None,
+                    password=None,
+                )
+                mocks["autostart"].assert_called_with(
+                    "mypool", state="on", connection=None, username=None, password=None,
+                )
+                mocks["build"].assert_not_called()
+
             mocks["update"] = MagicMock(return_value=False)
             for mock in mocks:
                 mocks[mock].reset_mock()
@@ -2027,6 +2103,9 @@ class LibvirtTestCase(TestCase, LoaderModuleMockMixin):
             for mock in mocks:
                 mocks[mock].reset_mock()
             mocks["update"] = MagicMock(return_value=True)
+            mocks["build"] = MagicMock(
+                side_effect=self.mock_libvirt.libvirtError("Existing VG")
+            )
             with patch.dict(
                 virt.__salt__,
                 {  # pylint: disable=no-member
@@ -2130,6 +2209,7 @@ class LibvirtTestCase(TestCase, LoaderModuleMockMixin):
                     ),
                     ret,
                 )
+                mocks["build"].assert_not_called()
                 mocks["update"].assert_called_with(
                     "mypool",
                     ptype="logical",
@@ -2477,8 +2557,8 @@ class LibvirtTestCase(TestCase, LoaderModuleMockMixin):
             ):
                 ret.update(
                     {
-                        "changes": {"mypool": "Pool updated, built, restarted"},
-                        "comment": "Pool mypool updated, built, restarted",
+                        "changes": {"mypool": "Pool updated, restarted"},
+                        "comment": "Pool mypool updated, restarted",
                         "result": True,
                     }
                 )
@@ -2504,9 +2584,7 @@ class LibvirtTestCase(TestCase, LoaderModuleMockMixin):
                 mocks["start"].assert_called_with(
                     "mypool", connection=None, username=None, password=None
                 )
-                mocks["build"].assert_called_with(
-                    "mypool", connection=None, username=None, password=None
-                )
+                mocks["build"].assert_not_called()
                 mocks["update"].assert_called_with(
                     "mypool",
                     ptype="logical",
diff --git a/tests/unit/utils/test_data.py b/tests/unit/utils/test_data.py
index 8fa352321c..8a6956d442 100644
--- a/tests/unit/utils/test_data.py
+++ b/tests/unit/utils/test_data.py
@@ -1,38 +1,38 @@
-# -*- coding: utf-8 -*-
-'''
+"""
 Tests for salt.utils.data
-'''
+"""
 
 # Import Python libs
-from __future__ import absolute_import, print_function, unicode_literals
+
 import logging
 
 # Import Salt libs
 import salt.utils.data
 import salt.utils.stringutils
-from salt.utils.odict import OrderedDict
-from tests.support.unit import TestCase, LOREM_IPSUM
-from tests.support.mock import patch
 
 # Import 3rd party libs
-from salt.ext.six.moves import builtins  # pylint: disable=import-error,redefined-builtin
-from salt.ext import six
+from salt.ext.six.moves import (  # pylint: disable=import-error,redefined-builtin
+    builtins,
+)
+from salt.utils.odict import OrderedDict
+from tests.support.mock import patch
+from tests.support.unit import LOREM_IPSUM, TestCase
 
 log = logging.getLogger(__name__)
-_b = lambda x: x.encode('utf-8')
+_b = lambda x: x.encode("utf-8")
 _s = lambda x: salt.utils.stringutils.to_str(x, normalize=True)
 # Some randomized data that will not decode
-BYTES = b'1\x814\x10'
+BYTES = b"1\x814\x10"
 
 # This is an example of a unicode string with й constructed using two separate
 # code points. Do not modify it.
-EGGS = '\u044f\u0438\u0306\u0446\u0430'
+EGGS = "\u044f\u0438\u0306\u0446\u0430"
 
 
 class DataTestCase(TestCase):
     test_data = [
-        'unicode_str',
-        _b('питон'),
+        "unicode_str",
+        _b("питон"),
         123,
         456.789,
         True,
@@ -40,71 +40,79 @@ class DataTestCase(TestCase):
         None,
         EGGS,
         BYTES,
-        [123, 456.789, _b('спам'), True, False, None, EGGS, BYTES],
-        (987, 654.321, _b('яйца'), EGGS, None, (True, EGGS, BYTES)),
-        {_b('str_key'): _b('str_val'),
-         None: True,
-         123: 456.789,
-         EGGS: BYTES,
-         _b('subdict'): {'unicode_key': EGGS,
-                         _b('tuple'): (123, 'hello', _b('world'), True, EGGS, BYTES),
-                         _b('list'): [456, _b('спам'), False, EGGS, BYTES]}},
-        OrderedDict([(_b('foo'), 'bar'), (123, 456), (EGGS, BYTES)])
+        [123, 456.789, _b("спам"), True, False, None, EGGS, BYTES],
+        (987, 654.321, _b("яйца"), EGGS, None, (True, EGGS, BYTES)),
+        {
+            _b("str_key"): _b("str_val"),
+            None: True,
+            123: 456.789,
+            EGGS: BYTES,
+            _b("subdict"): {
+                "unicode_key": EGGS,
+                _b("tuple"): (123, "hello", _b("world"), True, EGGS, BYTES),
+                _b("list"): [456, _b("спам"), False, EGGS, BYTES],
+            },
+        },
+        OrderedDict([(_b("foo"), "bar"), (123, 456), (EGGS, BYTES)]),
     ]
 
     def test_sorted_ignorecase(self):
-        test_list = ['foo', 'Foo', 'bar', 'Bar']
-        expected_list = ['bar', 'Bar', 'foo', 'Foo']
-        self.assertEqual(
-            salt.utils.data.sorted_ignorecase(test_list), expected_list)
+        test_list = ["foo", "Foo", "bar", "Bar"]
+        expected_list = ["bar", "Bar", "foo", "Foo"]
+        self.assertEqual(salt.utils.data.sorted_ignorecase(test_list), expected_list)
 
     def test_mysql_to_dict(self):
-        test_mysql_output = ['+----+------+-----------+------+---------+------+-------+------------------+',
-                             '| Id | User | Host      | db   | Command | Time | State | Info             |',
-                             '+----+------+-----------+------+---------+------+-------+------------------+',
-                             '|  7 | root | localhost | NULL | Query   |    0 | init  | show processlist |',
-                             '+----+------+-----------+------+---------+------+-------+------------------+']
+        test_mysql_output = [
+            "+----+------+-----------+------+---------+------+-------+------------------+",
+            "| Id | User | Host      | db   | Command | Time | State | Info             |",
+            "+----+------+-----------+------+---------+------+-------+------------------+",
+            "|  7 | root | localhost | NULL | Query   |    0 | init  | show processlist |",
+            "+----+------+-----------+------+---------+------+-------+------------------+",
+        ]
 
-        ret = salt.utils.data.mysql_to_dict(test_mysql_output, 'Info')
+        ret = salt.utils.data.mysql_to_dict(test_mysql_output, "Info")
         expected_dict = {
-            'show processlist': {'Info': 'show processlist', 'db': 'NULL', 'State': 'init', 'Host': 'localhost',
-                                 'Command': 'Query', 'User': 'root', 'Time': 0, 'Id': 7}}
+            "show processlist": {
+                "Info": "show processlist",
+                "db": "NULL",
+                "State": "init",
+                "Host": "localhost",
+                "Command": "Query",
+                "User": "root",
+                "Time": 0,
+                "Id": 7,
+            }
+        }
 
         self.assertDictEqual(ret, expected_dict)
 
     def test_subdict_match(self):
-        test_two_level_dict = {'foo': {'bar': 'baz'}}
-        test_two_level_comb_dict = {'foo': {'bar': 'baz:woz'}}
+        test_two_level_dict = {"foo": {"bar": "baz"}}
+        test_two_level_comb_dict = {"foo": {"bar": "baz:woz"}}
         test_two_level_dict_and_list = {
-            'abc': ['def', 'ghi', {'lorem': {'ipsum': [{'dolor': 'sit'}]}}],
+            "abc": ["def", "ghi", {"lorem": {"ipsum": [{"dolor": "sit"}]}}],
         }
-        test_three_level_dict = {'a': {'b': {'c': 'v'}}}
+        test_three_level_dict = {"a": {"b": {"c": "v"}}}
 
         self.assertTrue(
-            salt.utils.data.subdict_match(
-                test_two_level_dict, 'foo:bar:baz'
-            )
+            salt.utils.data.subdict_match(test_two_level_dict, "foo:bar:baz")
         )
         # In test_two_level_comb_dict, 'foo:bar' corresponds to 'baz:woz', not
         # 'baz'. This match should return False.
         self.assertFalse(
-            salt.utils.data.subdict_match(
-                test_two_level_comb_dict, 'foo:bar:baz'
-            )
+            salt.utils.data.subdict_match(test_two_level_comb_dict, "foo:bar:baz")
         )
         # This tests matching with the delimiter in the value part (in other
         # words, that the path 'foo:bar' corresponds to the string 'baz:woz').
         self.assertTrue(
-            salt.utils.data.subdict_match(
-                test_two_level_comb_dict, 'foo:bar:baz:woz'
-            )
+            salt.utils.data.subdict_match(test_two_level_comb_dict, "foo:bar:baz:woz")
         )
         # This would match if test_two_level_comb_dict['foo']['bar'] was equal
         # to 'baz:woz:wiz', or if there was more deep nesting. But it does not,
         # so this should return False.
         self.assertFalse(
             salt.utils.data.subdict_match(
-                test_two_level_comb_dict, 'foo:bar:baz:woz:wiz'
+                test_two_level_comb_dict, "foo:bar:baz:woz:wiz"
             )
         )
         # This tests for cases when a key path corresponds to a list. The
@@ -115,189 +123,171 @@ class DataTestCase(TestCase):
         # salt.utils.traverse_list_and_dict() so this particular assertion is a
         # sanity check.
         self.assertTrue(
-            salt.utils.data.subdict_match(
-                test_two_level_dict_and_list, 'abc:ghi'
-            )
+            salt.utils.data.subdict_match(test_two_level_dict_and_list, "abc:ghi")
         )
         # This tests the use case of a dict embedded in a list, embedded in a
         # list, embedded in a dict. This is a rather absurd case, but it
         # confirms that match recursion works properly.
         self.assertTrue(
             salt.utils.data.subdict_match(
-                test_two_level_dict_and_list, 'abc:lorem:ipsum:dolor:sit'
+                test_two_level_dict_and_list, "abc:lorem:ipsum:dolor:sit"
             )
         )
         # Test four level dict match for reference
-        self.assertTrue(
-            salt.utils.data.subdict_match(
-                test_three_level_dict, 'a:b:c:v'
-            )
-        )
+        self.assertTrue(salt.utils.data.subdict_match(test_three_level_dict, "a:b:c:v"))
         # Test regression in 2015.8 where 'a:c:v' would match 'a:b:c:v'
-        self.assertFalse(
-            salt.utils.data.subdict_match(
-                test_three_level_dict, 'a:c:v'
-            )
-        )
+        self.assertFalse(salt.utils.data.subdict_match(test_three_level_dict, "a:c:v"))
         # Test wildcard match
-        self.assertTrue(
-            salt.utils.data.subdict_match(
-                test_three_level_dict, 'a:*:c:v'
-            )
-        )
+        self.assertTrue(salt.utils.data.subdict_match(test_three_level_dict, "a:*:c:v"))
 
     def test_subdict_match_with_wildcards(self):
-        '''
+        """
         Tests subdict matching when wildcards are used in the expression
-        '''
-        data = {
-            'a': {
-                'b': {
-                    'ç': 'd',
-                    'é': ['eff', 'gee', '8ch'],
-                    'ĩ': {'j': 'k'}
-                }
-            }
-        }
-        assert salt.utils.data.subdict_match(data, '*:*:*:*')
-        assert salt.utils.data.subdict_match(data, 'a:*:*:*')
-        assert salt.utils.data.subdict_match(data, 'a:b:*:*')
-        assert salt.utils.data.subdict_match(data, 'a:b:ç:*')
-        assert salt.utils.data.subdict_match(data, 'a:b:*:d')
-        assert salt.utils.data.subdict_match(data, 'a:*:ç:d')
-        assert salt.utils.data.subdict_match(data, '*:b:ç:d')
-        assert salt.utils.data.subdict_match(data, '*:*:ç:d')
-        assert salt.utils.data.subdict_match(data, '*:*:*:d')
-        assert salt.utils.data.subdict_match(data, 'a:*:*:d')
-        assert salt.utils.data.subdict_match(data, 'a:b:*:ef*')
-        assert salt.utils.data.subdict_match(data, 'a:b:*:g*')
-        assert salt.utils.data.subdict_match(data, 'a:b:*:j:*')
-        assert salt.utils.data.subdict_match(data, 'a:b:*:j:k')
-        assert salt.utils.data.subdict_match(data, 'a:b:*:*:k')
-        assert salt.utils.data.subdict_match(data, 'a:b:*:*:*')
+        """
+        data = {"a": {"b": {"ç": "d", "é": ["eff", "gee", "8ch"], "ĩ": {"j": "k"}}}}
+        assert salt.utils.data.subdict_match(data, "*:*:*:*")
+        assert salt.utils.data.subdict_match(data, "a:*:*:*")
+        assert salt.utils.data.subdict_match(data, "a:b:*:*")
+        assert salt.utils.data.subdict_match(data, "a:b:ç:*")
+        assert salt.utils.data.subdict_match(data, "a:b:*:d")
+        assert salt.utils.data.subdict_match(data, "a:*:ç:d")
+        assert salt.utils.data.subdict_match(data, "*:b:ç:d")
+        assert salt.utils.data.subdict_match(data, "*:*:ç:d")
+        assert salt.utils.data.subdict_match(data, "*:*:*:d")
+        assert salt.utils.data.subdict_match(data, "a:*:*:d")
+        assert salt.utils.data.subdict_match(data, "a:b:*:ef*")
+        assert salt.utils.data.subdict_match(data, "a:b:*:g*")
+        assert salt.utils.data.subdict_match(data, "a:b:*:j:*")
+        assert salt.utils.data.subdict_match(data, "a:b:*:j:k")
+        assert salt.utils.data.subdict_match(data, "a:b:*:*:k")
+        assert salt.utils.data.subdict_match(data, "a:b:*:*:*")
 
     def test_traverse_dict(self):
-        test_two_level_dict = {'foo': {'bar': 'baz'}}
+        test_two_level_dict = {"foo": {"bar": "baz"}}
 
         self.assertDictEqual(
-            {'not_found': 'nope'},
+            {"not_found": "nope"},
             salt.utils.data.traverse_dict(
-                test_two_level_dict, 'foo:bar:baz', {'not_found': 'nope'}
-            )
+                test_two_level_dict, "foo:bar:baz", {"not_found": "nope"}
+            ),
         )
         self.assertEqual(
-            'baz',
+            "baz",
             salt.utils.data.traverse_dict(
-                test_two_level_dict, 'foo:bar', {'not_found': 'not_found'}
-            )
+                test_two_level_dict, "foo:bar", {"not_found": "not_found"}
+            ),
         )
 
     def test_traverse_dict_and_list(self):
-        test_two_level_dict = {'foo': {'bar': 'baz'}}
+        test_two_level_dict = {"foo": {"bar": "baz"}}
         test_two_level_dict_and_list = {
-            'foo': ['bar', 'baz', {'lorem': {'ipsum': [{'dolor': 'sit'}]}}]
+            "foo": ["bar", "baz", {"lorem": {"ipsum": [{"dolor": "sit"}]}}]
         }
 
         # Check traversing too far: salt.utils.data.traverse_dict_and_list() returns
         # the value corresponding to a given key path, and baz is a value
         # corresponding to the key path foo:bar.
         self.assertDictEqual(
-            {'not_found': 'nope'},
+            {"not_found": "nope"},
             salt.utils.data.traverse_dict_and_list(
-                test_two_level_dict, 'foo:bar:baz', {'not_found': 'nope'}
-            )
+                test_two_level_dict, "foo:bar:baz", {"not_found": "nope"}
+            ),
         )
         # Now check to ensure that foo:bar corresponds to baz
         self.assertEqual(
-            'baz',
+            "baz",
             salt.utils.data.traverse_dict_and_list(
-                test_two_level_dict, 'foo:bar', {'not_found': 'not_found'}
-            )
+                test_two_level_dict, "foo:bar", {"not_found": "not_found"}
+            ),
         )
         # Check traversing too far
         self.assertDictEqual(
-            {'not_found': 'nope'},
+            {"not_found": "nope"},
             salt.utils.data.traverse_dict_and_list(
-                test_two_level_dict_and_list, 'foo:bar', {'not_found': 'nope'}
-            )
+                test_two_level_dict_and_list, "foo:bar", {"not_found": "nope"}
+            ),
         )
         # Check index 1 (2nd element) of list corresponding to path 'foo'
         self.assertEqual(
-            'baz',
+            "baz",
             salt.utils.data.traverse_dict_and_list(
-                test_two_level_dict_and_list, 'foo:1', {'not_found': 'not_found'}
-            )
+                test_two_level_dict_and_list, "foo:1", {"not_found": "not_found"}
+            ),
         )
         # Traverse a couple times into dicts embedded in lists
         self.assertEqual(
-            'sit',
+            "sit",
             salt.utils.data.traverse_dict_and_list(
                 test_two_level_dict_and_list,
-                'foo:lorem:ipsum:dolor',
-                {'not_found': 'not_found'}
-            )
+                "foo:lorem:ipsum:dolor",
+                {"not_found": "not_found"},
+            ),
         )
 
     def test_compare_dicts(self):
-        ret = salt.utils.data.compare_dicts(old={'foo': 'bar'}, new={'foo': 'bar'})
+        ret = salt.utils.data.compare_dicts(old={"foo": "bar"}, new={"foo": "bar"})
         self.assertEqual(ret, {})
 
-        ret = salt.utils.data.compare_dicts(old={'foo': 'bar'}, new={'foo': 'woz'})
-        expected_ret = {'foo': {'new': 'woz', 'old': 'bar'}}
+        ret = salt.utils.data.compare_dicts(old={"foo": "bar"}, new={"foo": "woz"})
+        expected_ret = {"foo": {"new": "woz", "old": "bar"}}
         self.assertDictEqual(ret, expected_ret)
 
     def test_compare_lists_no_change(self):
-        ret = salt.utils.data.compare_lists(old=[1, 2, 3, 'a', 'b', 'c'],
-                                            new=[1, 2, 3, 'a', 'b', 'c'])
+        ret = salt.utils.data.compare_lists(
+            old=[1, 2, 3, "a", "b", "c"], new=[1, 2, 3, "a", "b", "c"]
+        )
         expected = {}
         self.assertDictEqual(ret, expected)
 
     def test_compare_lists_changes(self):
-        ret = salt.utils.data.compare_lists(old=[1, 2, 3, 'a', 'b', 'c'],
-                                            new=[1, 2, 4, 'x', 'y', 'z'])
-        expected = {'new': [4, 'x', 'y', 'z'], 'old': [3, 'a', 'b', 'c']}
+        ret = salt.utils.data.compare_lists(
+            old=[1, 2, 3, "a", "b", "c"], new=[1, 2, 4, "x", "y", "z"]
+        )
+        expected = {"new": [4, "x", "y", "z"], "old": [3, "a", "b", "c"]}
         self.assertDictEqual(ret, expected)
 
     def test_compare_lists_changes_new(self):
-        ret = salt.utils.data.compare_lists(old=[1, 2, 3],
-                                            new=[1, 2, 3, 'x', 'y', 'z'])
-        expected = {'new': ['x', 'y', 'z']}
+        ret = salt.utils.data.compare_lists(old=[1, 2, 3], new=[1, 2, 3, "x", "y", "z"])
+        expected = {"new": ["x", "y", "z"]}
         self.assertDictEqual(ret, expected)
 
     def test_compare_lists_changes_old(self):
-        ret = salt.utils.data.compare_lists(old=[1, 2, 3, 'a', 'b', 'c'],
-                                            new=[1, 2, 3])
-        expected = {'old': ['a', 'b', 'c']}
+        ret = salt.utils.data.compare_lists(old=[1, 2, 3, "a", "b", "c"], new=[1, 2, 3])
+        expected = {"old": ["a", "b", "c"]}
         self.assertDictEqual(ret, expected)
 
     def test_decode(self):
-        '''
+        """
         Companion to test_decode_to_str, they should both be kept up-to-date
         with one another.
 
         NOTE: This uses the lambda "_b" defined above in the global scope,
         which encodes a string to a bytestring, assuming utf-8.
-        '''
+        """
         expected = [
-            'unicode_str',
-            'питон',
+            "unicode_str",
+            "питон",
             123,
             456.789,
             True,
             False,
             None,
-            'яйца',
+            "яйца",
             BYTES,
-            [123, 456.789, 'спам', True, False, None, 'яйца', BYTES],
-            (987, 654.321, 'яйца', 'яйца', None, (True, 'яйца', BYTES)),
-            {'str_key': 'str_val',
-             None: True,
-             123: 456.789,
-             'яйца': BYTES,
-             'subdict': {'unicode_key': 'яйца',
-                         'tuple': (123, 'hello', 'world', True, 'яйца', BYTES),
-                         'list': [456, 'спам', False, 'яйца', BYTES]}},
-            OrderedDict([('foo', 'bar'), (123, 456), ('яйца', BYTES)])
+            [123, 456.789, "спам", True, False, None, "яйца", BYTES],
+            (987, 654.321, "яйца", "яйца", None, (True, "яйца", BYTES)),
+            {
+                "str_key": "str_val",
+                None: True,
+                123: 456.789,
+                "яйца": BYTES,
+                "subdict": {
+                    "unicode_key": "яйца",
+                    "tuple": (123, "hello", "world", True, "яйца", BYTES),
+                    "list": [456, "спам", False, "яйца", BYTES],
+                },
+            },
+            OrderedDict([("foo", "bar"), (123, 456), ("яйца", BYTES)]),
         ]
 
         ret = salt.utils.data.decode(
@@ -305,7 +295,8 @@ class DataTestCase(TestCase):
             keep=True,
             normalize=True,
             preserve_dict_class=True,
-            preserve_tuples=True)
+            preserve_tuples=True,
+        )
         self.assertEqual(ret, expected)
 
         # The binary data in the data structure should fail to decode, even
@@ -317,74 +308,100 @@ class DataTestCase(TestCase):
             keep=False,
             normalize=True,
             preserve_dict_class=True,
-            preserve_tuples=True)
+            preserve_tuples=True,
+        )
 
         # Now munge the expected data so that we get what we would expect if we
         # disable preservation of dict class and tuples
-        expected[10] = [987, 654.321, 'яйца', 'яйца', None, [True, 'яйца', BYTES]]
-        expected[11]['subdict']['tuple'] = [123, 'hello', 'world', True, 'яйца', BYTES]
-        expected[12] = {'foo': 'bar', 123: 456, 'яйца': BYTES}
+        expected[10] = [987, 654.321, "яйца", "яйца", None, [True, "яйца", BYTES]]
+        expected[11]["subdict"]["tuple"] = [123, "hello", "world", True, "яйца", BYTES]
+        expected[12] = {"foo": "bar", 123: 456, "яйца": BYTES}
 
         ret = salt.utils.data.decode(
             self.test_data,
             keep=True,
             normalize=True,
             preserve_dict_class=False,
-            preserve_tuples=False)
+            preserve_tuples=False,
+        )
         self.assertEqual(ret, expected)
 
         # Now test single non-string, non-data-structure items, these should
         # return the same value when passed to this function
         for item in (123, 4.56, True, False, None):
-            log.debug('Testing decode of %s', item)
+            log.debug("Testing decode of %s", item)
             self.assertEqual(salt.utils.data.decode(item), item)
 
         # Test single strings (not in a data structure)
-        self.assertEqual(salt.utils.data.decode('foo'), 'foo')
-        self.assertEqual(salt.utils.data.decode(_b('bar')), 'bar')
-        self.assertEqual(salt.utils.data.decode(EGGS, normalize=True), 'яйца')
+        self.assertEqual(salt.utils.data.decode("foo"), "foo")
+        self.assertEqual(salt.utils.data.decode(_b("bar")), "bar")
+        self.assertEqual(salt.utils.data.decode(EGGS, normalize=True), "яйца")
         self.assertEqual(salt.utils.data.decode(EGGS, normalize=False), EGGS)
 
         # Test binary blob
         self.assertEqual(salt.utils.data.decode(BYTES, keep=True), BYTES)
-        self.assertRaises(
-            UnicodeDecodeError,
-            salt.utils.data.decode,
-            BYTES,
-            keep=False)
+        self.assertRaises(UnicodeDecodeError, salt.utils.data.decode, BYTES, keep=False)
+
+    def test_circular_refs_dicts(self):
+        test_dict = {"key": "value", "type": "test1"}
+        test_dict["self"] = test_dict
+        ret = salt.utils.data._remove_circular_refs(ob=test_dict)
+        self.assertDictEqual(ret, {"key": "value", "type": "test1", "self": None})
+
+    def test_circular_refs_lists(self):
+        test_list = {
+            "foo": [],
+        }
+        test_list["foo"].append((test_list,))
+        ret = salt.utils.data._remove_circular_refs(ob=test_list)
+        self.assertDictEqual(ret, {"foo": [(None,)]})
+
+    def test_circular_refs_tuple(self):
+        test_dup = {"foo": "string 1", "bar": "string 1", "ham": 1, "spam": 1}
+        ret = salt.utils.data._remove_circular_refs(ob=test_dup)
+        self.assertDictEqual(
+            ret, {"foo": "string 1", "bar": "string 1", "ham": 1, "spam": 1}
+        )
 
     def test_decode_to_str(self):
-        '''
+        """
         Companion to test_decode, they should both be kept up-to-date with one
         another.
 
         NOTE: This uses the lambda "_s" defined above in the global scope,
         which converts the string/bytestring to a str type.
-        '''
+        """
         expected = [
-            _s('unicode_str'),
-            _s('питон'),
+            _s("unicode_str"),
+            _s("питон"),
             123,
             456.789,
             True,
             False,
             None,
-            _s('яйца'),
+            _s("яйца"),
             BYTES,
-            [123, 456.789, _s('спам'), True, False, None, _s('яйца'), BYTES],
-            (987, 654.321, _s('яйца'), _s('яйца'), None, (True, _s('яйца'), BYTES)),
+            [123, 456.789, _s("спам"), True, False, None, _s("яйца"), BYTES],
+            (987, 654.321, _s("яйца"), _s("яйца"), None, (True, _s("яйца"), BYTES)),
             {
-                _s('str_key'): _s('str_val'),
+                _s("str_key"): _s("str_val"),
                 None: True,
                 123: 456.789,
-                _s('яйца'): BYTES,
-                _s('subdict'): {
-                    _s('unicode_key'): _s('яйца'),
-                    _s('tuple'): (123, _s('hello'), _s('world'), True, _s('яйца'), BYTES),
-                    _s('list'): [456, _s('спам'), False, _s('яйца'), BYTES]
-                }
+                _s("яйца"): BYTES,
+                _s("subdict"): {
+                    _s("unicode_key"): _s("яйца"),
+                    _s("tuple"): (
+                        123,
+                        _s("hello"),
+                        _s("world"),
+                        True,
+                        _s("яйца"),
+                        BYTES,
+                    ),
+                    _s("list"): [456, _s("спам"), False, _s("яйца"), BYTES],
+                },
             },
-            OrderedDict([(_s('foo'), _s('bar')), (123, 456), (_s('яйца'), BYTES)])
+            OrderedDict([(_s("foo"), _s("bar")), (123, 456), (_s("яйца"), BYTES)]),
         ]
 
         ret = salt.utils.data.decode(
@@ -393,27 +410,42 @@ class DataTestCase(TestCase):
             normalize=True,
             preserve_dict_class=True,
             preserve_tuples=True,
-            to_str=True)
+            to_str=True,
+        )
         self.assertEqual(ret, expected)
 
-        if six.PY3:
-            # The binary data in the data structure should fail to decode, even
-            # using the fallback, and raise an exception.
-            self.assertRaises(
-                UnicodeDecodeError,
-                salt.utils.data.decode,
-                self.test_data,
-                keep=False,
-                normalize=True,
-                preserve_dict_class=True,
-                preserve_tuples=True,
-                to_str=True)
+        # The binary data in the data structure should fail to decode, even
+        # using the fallback, and raise an exception.
+        self.assertRaises(
+            UnicodeDecodeError,
+            salt.utils.data.decode,
+            self.test_data,
+            keep=False,
+            normalize=True,
+            preserve_dict_class=True,
+            preserve_tuples=True,
+            to_str=True,
+        )
 
         # Now munge the expected data so that we get what we would expect if we
         # disable preservation of dict class and tuples
-        expected[10] = [987, 654.321, _s('яйца'), _s('яйца'), None, [True, _s('яйца'), BYTES]]
-        expected[11][_s('subdict')][_s('tuple')] = [123, _s('hello'), _s('world'), True, _s('яйца'), BYTES]
-        expected[12] = {_s('foo'): _s('bar'), 123: 456, _s('яйца'): BYTES}
+        expected[10] = [
+            987,
+            654.321,
+            _s("яйца"),
+            _s("яйца"),
+            None,
+            [True, _s("яйца"), BYTES],
+        ]
+        expected[11][_s("subdict")][_s("tuple")] = [
+            123,
+            _s("hello"),
+            _s("world"),
+            True,
+            _s("яйца"),
+            BYTES,
+        ]
+        expected[12] = {_s("foo"): _s("bar"), 123: 456, _s("яйца"): BYTES}
 
         ret = salt.utils.data.decode(
             self.test_data,
@@ -421,47 +453,41 @@ class DataTestCase(TestCase):
             normalize=True,
             preserve_dict_class=False,
             preserve_tuples=False,
-            to_str=True)
+            to_str=True,
+        )
         self.assertEqual(ret, expected)
 
         # Now test single non-string, non-data-structure items, these should
         # return the same value when passed to this function
         for item in (123, 4.56, True, False, None):
-            log.debug('Testing decode of %s', item)
+            log.debug("Testing decode of %s", item)
             self.assertEqual(salt.utils.data.decode(item, to_str=True), item)
 
         # Test single strings (not in a data structure)
-        self.assertEqual(salt.utils.data.decode('foo', to_str=True), _s('foo'))
-        self.assertEqual(salt.utils.data.decode(_b('bar'), to_str=True), _s('bar'))
+        self.assertEqual(salt.utils.data.decode("foo", to_str=True), _s("foo"))
+        self.assertEqual(salt.utils.data.decode(_b("bar"), to_str=True), _s("bar"))
 
         # Test binary blob
-        self.assertEqual(
-            salt.utils.data.decode(BYTES, keep=True, to_str=True),
-            BYTES
+        self.assertEqual(salt.utils.data.decode(BYTES, keep=True, to_str=True), BYTES)
+        self.assertRaises(
+            UnicodeDecodeError, salt.utils.data.decode, BYTES, keep=False, to_str=True,
         )
-        if six.PY3:
-            self.assertRaises(
-                UnicodeDecodeError,
-                salt.utils.data.decode,
-                BYTES,
-                keep=False,
-                to_str=True)
 
     def test_decode_fallback(self):
-        '''
+        """
         Test fallback to utf-8
-        '''
-        with patch.object(builtins, '__salt_system_encoding__', 'ascii'):
-            self.assertEqual(salt.utils.data.decode(_b('яйца')), 'яйца')
+        """
+        with patch.object(builtins, "__salt_system_encoding__", "ascii"):
+            self.assertEqual(salt.utils.data.decode(_b("яйца")), "яйца")
 
     def test_encode(self):
-        '''
+        """
         NOTE: This uses the lambda "_b" defined above in the global scope,
         which encodes a string to a bytestring, assuming utf-8.
-        '''
+        """
         expected = [
-            _b('unicode_str'),
-            _b('питон'),
+            _b("unicode_str"),
+            _b("питон"),
             123,
             456.789,
             True,
@@ -469,67 +495,71 @@ class DataTestCase(TestCase):
             None,
             _b(EGGS),
             BYTES,
-            [123, 456.789, _b('спам'), True, False, None, _b(EGGS), BYTES],
-            (987, 654.321, _b('яйца'), _b(EGGS), None, (True, _b(EGGS), BYTES)),
+            [123, 456.789, _b("спам"), True, False, None, _b(EGGS), BYTES],
+            (987, 654.321, _b("яйца"), _b(EGGS), None, (True, _b(EGGS), BYTES)),
             {
-                _b('str_key'): _b('str_val'),
+                _b("str_key"): _b("str_val"),
                 None: True,
                 123: 456.789,
                 _b(EGGS): BYTES,
-                _b('subdict'): {
-                    _b('unicode_key'): _b(EGGS),
-                    _b('tuple'): (123, _b('hello'), _b('world'), True, _b(EGGS), BYTES),
-                    _b('list'): [456, _b('спам'), False, _b(EGGS), BYTES]
-                }
+                _b("subdict"): {
+                    _b("unicode_key"): _b(EGGS),
+                    _b("tuple"): (123, _b("hello"), _b("world"), True, _b(EGGS), BYTES),
+                    _b("list"): [456, _b("спам"), False, _b(EGGS), BYTES],
+                },
             },
-            OrderedDict([(_b('foo'), _b('bar')), (123, 456), (_b(EGGS), BYTES)])
+            OrderedDict([(_b("foo"), _b("bar")), (123, 456), (_b(EGGS), BYTES)]),
         ]
 
         # Both keep=True and keep=False should work because the BYTES data is
         # already bytes.
         ret = salt.utils.data.encode(
-            self.test_data,
-            keep=True,
-            preserve_dict_class=True,
-            preserve_tuples=True)
+            self.test_data, keep=True, preserve_dict_class=True, preserve_tuples=True
+        )
         self.assertEqual(ret, expected)
         ret = salt.utils.data.encode(
-            self.test_data,
-            keep=False,
-            preserve_dict_class=True,
-            preserve_tuples=True)
+            self.test_data, keep=False, preserve_dict_class=True, preserve_tuples=True
+        )
         self.assertEqual(ret, expected)
 
         # Now munge the expected data so that we get what we would expect if we
         # disable preservation of dict class and tuples
-        expected[10] = [987, 654.321, _b('яйца'), _b(EGGS), None, [True, _b(EGGS), BYTES]]
-        expected[11][_b('subdict')][_b('tuple')] = [
-            123, _b('hello'), _b('world'), True, _b(EGGS), BYTES
+        expected[10] = [
+            987,
+            654.321,
+            _b("яйца"),
+            _b(EGGS),
+            None,
+            [True, _b(EGGS), BYTES],
         ]
-        expected[12] = {_b('foo'): _b('bar'), 123: 456, _b(EGGS): BYTES}
+        expected[11][_b("subdict")][_b("tuple")] = [
+            123,
+            _b("hello"),
+            _b("world"),
+            True,
+            _b(EGGS),
+            BYTES,
+        ]
+        expected[12] = {_b("foo"): _b("bar"), 123: 456, _b(EGGS): BYTES}
 
         ret = salt.utils.data.encode(
-            self.test_data,
-            keep=True,
-            preserve_dict_class=False,
-            preserve_tuples=False)
+            self.test_data, keep=True, preserve_dict_class=False, preserve_tuples=False
+        )
         self.assertEqual(ret, expected)
         ret = salt.utils.data.encode(
-            self.test_data,
-            keep=False,
-            preserve_dict_class=False,
-            preserve_tuples=False)
+            self.test_data, keep=False, preserve_dict_class=False, preserve_tuples=False
+        )
         self.assertEqual(ret, expected)
 
         # Now test single non-string, non-data-structure items, these should
         # return the same value when passed to this function
         for item in (123, 4.56, True, False, None):
-            log.debug('Testing encode of %s', item)
+            log.debug("Testing encode of %s", item)
             self.assertEqual(salt.utils.data.encode(item), item)
 
         # Test single strings (not in a data structure)
-        self.assertEqual(salt.utils.data.encode('foo'), _b('foo'))
-        self.assertEqual(salt.utils.data.encode(_b('bar')), _b('bar'))
+        self.assertEqual(salt.utils.data.encode("foo"), _b("foo"))
+        self.assertEqual(salt.utils.data.encode(_b("bar")), _b("bar"))
 
         # Test binary blob, nothing should happen even when keep=False since
         # the data is already bytes
@@ -537,41 +567,43 @@ class DataTestCase(TestCase):
         self.assertEqual(salt.utils.data.encode(BYTES, keep=False), BYTES)
 
     def test_encode_keep(self):
-        '''
+        """
         Whereas we tested the keep argument in test_decode, it is much easier
         to do a more comprehensive test of keep in its own function where we
         can force the encoding.
-        '''
-        unicode_str = 'питон'
-        encoding = 'ascii'
+        """
+        unicode_str = "питон"
+        encoding = "ascii"
 
         # Test single string
         self.assertEqual(
-            salt.utils.data.encode(unicode_str, encoding, keep=True),
-            unicode_str)
+            salt.utils.data.encode(unicode_str, encoding, keep=True), unicode_str
+        )
         self.assertRaises(
             UnicodeEncodeError,
             salt.utils.data.encode,
             unicode_str,
             encoding,
-            keep=False)
+            keep=False,
+        )
 
         data = [
             unicode_str,
-            [b'foo', [unicode_str], {b'key': unicode_str}, (unicode_str,)],
-            {b'list': [b'foo', unicode_str],
-             b'dict': {b'key': unicode_str},
-             b'tuple': (b'foo', unicode_str)},
-            ([b'foo', unicode_str], {b'key': unicode_str}, (unicode_str,))
+            [b"foo", [unicode_str], {b"key": unicode_str}, (unicode_str,)],
+            {
+                b"list": [b"foo", unicode_str],
+                b"dict": {b"key": unicode_str},
+                b"tuple": (b"foo", unicode_str),
+            },
+            ([b"foo", unicode_str], {b"key": unicode_str}, (unicode_str,)),
         ]
 
         # Since everything was a bytestring aside from the bogus data, the
         # return data should be identical. We don't need to test recursive
         # decoding, that has already been tested in test_encode.
         self.assertEqual(
-            salt.utils.data.encode(data, encoding,
-                                   keep=True, preserve_tuples=True),
-            data
+            salt.utils.data.encode(data, encoding, keep=True, preserve_tuples=True),
+            data,
         )
         self.assertRaises(
             UnicodeEncodeError,
@@ -579,13 +611,15 @@ class DataTestCase(TestCase):
             data,
             encoding,
             keep=False,
-            preserve_tuples=True)
+            preserve_tuples=True,
+        )
 
         for index, _ in enumerate(data):
             self.assertEqual(
-                salt.utils.data.encode(data[index], encoding,
-                                       keep=True, preserve_tuples=True),
-                data[index]
+                salt.utils.data.encode(
+                    data[index], encoding, keep=True, preserve_tuples=True
+                ),
+                data[index],
             )
             self.assertRaises(
                 UnicodeEncodeError,
@@ -593,31 +627,36 @@ class DataTestCase(TestCase):
                 data[index],
                 encoding,
                 keep=False,
-                preserve_tuples=True)
+                preserve_tuples=True,
+            )
 
     def test_encode_fallback(self):
-        '''
+        """
         Test fallback to utf-8
-        '''
-        with patch.object(builtins, '__salt_system_encoding__', 'ascii'):
-            self.assertEqual(salt.utils.data.encode('яйца'), _b('яйца'))
-        with patch.object(builtins, '__salt_system_encoding__', 'CP1252'):
-            self.assertEqual(salt.utils.data.encode('Ψ'), _b('Ψ'))
+        """
+        with patch.object(builtins, "__salt_system_encoding__", "ascii"):
+            self.assertEqual(salt.utils.data.encode("яйца"), _b("яйца"))
+        with patch.object(builtins, "__salt_system_encoding__", "CP1252"):
+            self.assertEqual(salt.utils.data.encode("Ψ"), _b("Ψ"))
 
     def test_repack_dict(self):
-        list_of_one_element_dicts = [{'dict_key_1': 'dict_val_1'},
-                                     {'dict_key_2': 'dict_val_2'},
-                                     {'dict_key_3': 'dict_val_3'}]
-        expected_ret = {'dict_key_1': 'dict_val_1',
-                        'dict_key_2': 'dict_val_2',
-                        'dict_key_3': 'dict_val_3'}
+        list_of_one_element_dicts = [
+            {"dict_key_1": "dict_val_1"},
+            {"dict_key_2": "dict_val_2"},
+            {"dict_key_3": "dict_val_3"},
+        ]
+        expected_ret = {
+            "dict_key_1": "dict_val_1",
+            "dict_key_2": "dict_val_2",
+            "dict_key_3": "dict_val_3",
+        }
         ret = salt.utils.data.repack_dictlist(list_of_one_element_dicts)
         self.assertDictEqual(ret, expected_ret)
 
         # Try with yaml
-        yaml_key_val_pair = '- key1: val1'
+        yaml_key_val_pair = "- key1: val1"
         ret = salt.utils.data.repack_dictlist(yaml_key_val_pair)
-        self.assertDictEqual(ret, {'key1': 'val1'})
+        self.assertDictEqual(ret, {"key1": "val1"})
 
         # Make sure we handle non-yaml junk data
         ret = salt.utils.data.repack_dictlist(LOREM_IPSUM)
@@ -626,43 +665,47 @@ class DataTestCase(TestCase):
     def test_stringify(self):
         self.assertRaises(TypeError, salt.utils.data.stringify, 9)
         self.assertEqual(
-            salt.utils.data.stringify(['one', 'two', str('three'), 4, 5]),  # future lint: disable=blacklisted-function
-            ['one', 'two', 'three', '4', '5']
+            salt.utils.data.stringify(
+                ["one", "two", "three", 4, 5]
+            ),  # future lint: disable=blacklisted-function
+            ["one", "two", "three", "4", "5"],
         )
 
     def test_json_query(self):
         # Raises exception if jmespath module is not found
-        with patch('salt.utils.data.jmespath', None):
+        with patch("salt.utils.data.jmespath", None):
             self.assertRaisesRegex(
-                RuntimeError, 'requires jmespath',
-                salt.utils.data.json_query, {}, '@'
+                RuntimeError, "requires jmespath", salt.utils.data.json_query, {}, "@"
             )
 
         # Test search
         user_groups = {
-            'user1': {'groups': ['group1', 'group2', 'group3']},
-            'user2': {'groups': ['group1', 'group2']},
-            'user3': {'groups': ['group3']},
+            "user1": {"groups": ["group1", "group2", "group3"]},
+            "user2": {"groups": ["group1", "group2"]},
+            "user3": {"groups": ["group3"]},
         }
-        expression = '*.groups[0]'
-        primary_groups = ['group1', 'group1', 'group3']
+        expression = "*.groups[0]"
+        primary_groups = ["group1", "group1", "group3"]
         self.assertEqual(
-            sorted(salt.utils.data.json_query(user_groups, expression)),
-            primary_groups
+            sorted(salt.utils.data.json_query(user_groups, expression)), primary_groups
         )
 
 
 class FilterFalseyTestCase(TestCase):
-    '''
+    """
     Test suite for salt.utils.data.filter_falsey
-    '''
+    """
 
     def test_nop(self):
-        '''
+        """
         Test cases where nothing will be done.
-        '''
+        """
         # Test with dictionary without recursion
-        old_dict = {'foo': 'bar', 'bar': {'baz': {'qux': 'quux'}}, 'baz': ['qux', {'foo': 'bar'}]}
+        old_dict = {
+            "foo": "bar",
+            "bar": {"baz": {"qux": "quux"}},
+            "baz": ["qux", {"foo": "bar"}],
+        }
         new_dict = salt.utils.data.filter_falsey(old_dict)
         self.assertEqual(old_dict, new_dict)
         # Check returned type equality
@@ -671,23 +714,25 @@ class FilterFalseyTestCase(TestCase):
         new_dict = salt.utils.data.filter_falsey(old_dict, recurse_depth=3)
         self.assertEqual(old_dict, new_dict)
         # Test with list
-        old_list = ['foo', 'bar']
+        old_list = ["foo", "bar"]
         new_list = salt.utils.data.filter_falsey(old_list)
         self.assertEqual(old_list, new_list)
         # Check returned type equality
         self.assertIs(type(old_list), type(new_list))
         # Test with set
-        old_set = set(['foo', 'bar'])
+        old_set = {"foo", "bar"}
         new_set = salt.utils.data.filter_falsey(old_set)
         self.assertEqual(old_set, new_set)
         # Check returned type equality
         self.assertIs(type(old_set), type(new_set))
         # Test with OrderedDict
-        old_dict = OrderedDict([
-            ('foo', 'bar'),
-            ('bar', OrderedDict([('qux', 'quux')])),
-            ('baz', ['qux', OrderedDict([('foo', 'bar')])])
-        ])
+        old_dict = OrderedDict(
+            [
+                ("foo", "bar"),
+                ("bar", OrderedDict([("qux", "quux")])),
+                ("baz", ["qux", OrderedDict([("foo", "bar")])]),
+            ]
+        )
         new_dict = salt.utils.data.filter_falsey(old_dict)
         self.assertEqual(old_dict, new_dict)
         self.assertIs(type(old_dict), type(new_dict))
@@ -696,8 +741,8 @@ class FilterFalseyTestCase(TestCase):
         new_list = salt.utils.data.filter_falsey(old_list, ignore_types=[type(0)])
         self.assertEqual(old_list, new_list)
         # Test excluding str (or unicode) (or both)
-        old_list = ['']
-        new_list = salt.utils.data.filter_falsey(old_list, ignore_types=[type('')])
+        old_list = [""]
+        new_list = salt.utils.data.filter_falsey(old_list, ignore_types=[type("")])
         self.assertEqual(old_list, new_list)
         # Test excluding list
         old_list = [[]]
@@ -709,185 +754,264 @@ class FilterFalseyTestCase(TestCase):
         self.assertEqual(old_list, new_list)
 
     def test_filter_dict_no_recurse(self):
-        '''
+        """
         Test filtering a dictionary without recursing.
         This will only filter out key-values where the values are falsey.
-        '''
-        old_dict = {'foo': None,
-                    'bar': {'baz': {'qux': None, 'quux': '', 'foo': []}},
-                    'baz': ['qux'],
-                    'qux': {},
-                    'quux': []}
+        """
+        old_dict = {
+            "foo": None,
+            "bar": {"baz": {"qux": None, "quux": "", "foo": []}},
+            "baz": ["qux"],
+            "qux": {},
+            "quux": [],
+        }
         new_dict = salt.utils.data.filter_falsey(old_dict)
-        expect_dict = {'bar': {'baz': {'qux': None, 'quux': '', 'foo': []}}, 'baz': ['qux']}
+        expect_dict = {
+            "bar": {"baz": {"qux": None, "quux": "", "foo": []}},
+            "baz": ["qux"],
+        }
         self.assertEqual(expect_dict, new_dict)
         self.assertIs(type(expect_dict), type(new_dict))
 
     def test_filter_dict_recurse(self):
-        '''
+        """
         Test filtering a dictionary with recursing.
         This will filter out any key-values where the values are falsey or when
         the values *become* falsey after filtering their contents (in case they
         are lists or dicts).
-        '''
-        old_dict = {'foo': None,
-                    'bar': {'baz': {'qux': None, 'quux': '', 'foo': []}},
-                    'baz': ['qux'],
-                    'qux': {},
-                    'quux': []}
+        """
+        old_dict = {
+            "foo": None,
+            "bar": {"baz": {"qux": None, "quux": "", "foo": []}},
+            "baz": ["qux"],
+            "qux": {},
+            "quux": [],
+        }
         new_dict = salt.utils.data.filter_falsey(old_dict, recurse_depth=3)
-        expect_dict = {'baz': ['qux']}
+        expect_dict = {"baz": ["qux"]}
         self.assertEqual(expect_dict, new_dict)
         self.assertIs(type(expect_dict), type(new_dict))
 
     def test_filter_list_no_recurse(self):
-        '''
+        """
         Test filtering a list without recursing.
         This will only filter out items which are falsey.
-        '''
-        old_list = ['foo', None, [], {}, 0, '']
+        """
+        old_list = ["foo", None, [], {}, 0, ""]
         new_list = salt.utils.data.filter_falsey(old_list)
-        expect_list = ['foo']
+        expect_list = ["foo"]
         self.assertEqual(expect_list, new_list)
         self.assertIs(type(expect_list), type(new_list))
         # Ensure nested values are *not* filtered out.
         old_list = [
-            'foo',
-            ['foo'],
-            ['foo', None],
-            {'foo': 0},
-            {'foo': 'bar', 'baz': []},
-            [{'foo': ''}],
+            "foo",
+            ["foo"],
+            ["foo", None],
+            {"foo": 0},
+            {"foo": "bar", "baz": []},
+            [{"foo": ""}],
         ]
         new_list = salt.utils.data.filter_falsey(old_list)
         self.assertEqual(old_list, new_list)
         self.assertIs(type(old_list), type(new_list))
 
     def test_filter_list_recurse(self):
-        '''
+        """
         Test filtering a list with recursing.
         This will filter out any items which are falsey, or which become falsey
         after filtering their contents (in case they are lists or dicts).
-        '''
+        """
         old_list = [
-            'foo',
-            ['foo'],
-            ['foo', None],
-            {'foo': 0},
-            {'foo': 'bar', 'baz': []},
-            [{'foo': ''}]
+            "foo",
+            ["foo"],
+            ["foo", None],
+            {"foo": 0},
+            {"foo": "bar", "baz": []},
+            [{"foo": ""}],
         ]
         new_list = salt.utils.data.filter_falsey(old_list, recurse_depth=3)
-        expect_list = ['foo', ['foo'], ['foo'], {'foo': 'bar'}]
+        expect_list = ["foo", ["foo"], ["foo"], {"foo": "bar"}]
         self.assertEqual(expect_list, new_list)
         self.assertIs(type(expect_list), type(new_list))
 
     def test_filter_set_no_recurse(self):
-        '''
+        """
         Test filtering a set without recursing.
         Note that a set cannot contain unhashable types, so recursion is not possible.
-        '''
-        old_set = set([
-            'foo',
-            None,
-            0,
-            '',
-        ])
+        """
+        old_set = {"foo", None, 0, ""}
         new_set = salt.utils.data.filter_falsey(old_set)
-        expect_set = set(['foo'])
+        expect_set = {"foo"}
         self.assertEqual(expect_set, new_set)
         self.assertIs(type(expect_set), type(new_set))
 
     def test_filter_ordereddict_no_recurse(self):
-        '''
+        """
         Test filtering an OrderedDict without recursing.
-        '''
-        old_dict = OrderedDict([
-            ('foo', None),
-            ('bar', OrderedDict([('baz', OrderedDict([('qux', None), ('quux', ''), ('foo', [])]))])),
-            ('baz', ['qux']),
-            ('qux', {}),
-            ('quux', [])
-        ])
+        """
+        old_dict = OrderedDict(
+            [
+                ("foo", None),
+                (
+                    "bar",
+                    OrderedDict(
+                        [
+                            (
+                                "baz",
+                                OrderedDict([("qux", None), ("quux", ""), ("foo", [])]),
+                            )
+                        ]
+                    ),
+                ),
+                ("baz", ["qux"]),
+                ("qux", {}),
+                ("quux", []),
+            ]
+        )
         new_dict = salt.utils.data.filter_falsey(old_dict)
-        expect_dict = OrderedDict([
-            ('bar', OrderedDict([('baz', OrderedDict([('qux', None), ('quux', ''), ('foo', [])]))])),
-            ('baz', ['qux']),
-        ])
+        expect_dict = OrderedDict(
+            [
+                (
+                    "bar",
+                    OrderedDict(
+                        [
+                            (
+                                "baz",
+                                OrderedDict([("qux", None), ("quux", ""), ("foo", [])]),
+                            )
+                        ]
+                    ),
+                ),
+                ("baz", ["qux"]),
+            ]
+        )
         self.assertEqual(expect_dict, new_dict)
         self.assertIs(type(expect_dict), type(new_dict))
 
     def test_filter_ordereddict_recurse(self):
-        '''
+        """
         Test filtering an OrderedDict with recursing.
-        '''
-        old_dict = OrderedDict([
-            ('foo', None),
-            ('bar', OrderedDict([('baz', OrderedDict([('qux', None), ('quux', ''), ('foo', [])]))])),
-            ('baz', ['qux']),
-            ('qux', {}),
-            ('quux', [])
-        ])
+        """
+        old_dict = OrderedDict(
+            [
+                ("foo", None),
+                (
+                    "bar",
+                    OrderedDict(
+                        [
+                            (
+                                "baz",
+                                OrderedDict([("qux", None), ("quux", ""), ("foo", [])]),
+                            )
+                        ]
+                    ),
+                ),
+                ("baz", ["qux"]),
+                ("qux", {}),
+                ("quux", []),
+            ]
+        )
         new_dict = salt.utils.data.filter_falsey(old_dict, recurse_depth=3)
-        expect_dict = OrderedDict([
-            ('baz', ['qux']),
-        ])
+        expect_dict = OrderedDict([("baz", ["qux"])])
         self.assertEqual(expect_dict, new_dict)
         self.assertIs(type(expect_dict), type(new_dict))
 
     def test_filter_list_recurse_limit(self):
-        '''
+        """
         Test filtering a list with recursing, but with a limited depth.
         Note that the top-level is always processed, so a recursion depth of 2
         means that two *additional* levels are processed.
-        '''
+        """
         old_list = [None, [None, [None, [None]]]]
         new_list = salt.utils.data.filter_falsey(old_list, recurse_depth=2)
         self.assertEqual([[[[None]]]], new_list)
 
     def test_filter_dict_recurse_limit(self):
-        '''
+        """
         Test filtering a dict with recursing, but with a limited depth.
         Note that the top-level is always processed, so a recursion depth of 2
         means that two *additional* levels are processed.
-        '''
-        old_dict = {'one': None,
-                    'foo': {'two': None, 'bar': {'three': None, 'baz': {'four': None}}}}
+        """
+        old_dict = {
+            "one": None,
+            "foo": {"two": None, "bar": {"three": None, "baz": {"four": None}}},
+        }
         new_dict = salt.utils.data.filter_falsey(old_dict, recurse_depth=2)
-        self.assertEqual({'foo': {'bar': {'baz': {'four': None}}}}, new_dict)
+        self.assertEqual({"foo": {"bar": {"baz": {"four": None}}}}, new_dict)
 
     def test_filter_exclude_types(self):
-        '''
+        """
         Test filtering a list recursively, but also ignoring (i.e. not filtering)
         out certain types that can be falsey.
-        '''
+        """
         # Ignore int, unicode
-        old_list = ['foo', ['foo'], ['foo', None], {'foo': 0}, {'foo': 'bar', 'baz': []}, [{'foo': ''}]]
-        new_list = salt.utils.data.filter_falsey(old_list, recurse_depth=3, ignore_types=[type(0), type('')])
-        self.assertEqual(['foo', ['foo'], ['foo'], {'foo': 0}, {'foo': 'bar'}, [{'foo': ''}]], new_list)
+        old_list = [
+            "foo",
+            ["foo"],
+            ["foo", None],
+            {"foo": 0},
+            {"foo": "bar", "baz": []},
+            [{"foo": ""}],
+        ]
+        new_list = salt.utils.data.filter_falsey(
+            old_list, recurse_depth=3, ignore_types=[type(0), type("")]
+        )
+        self.assertEqual(
+            ["foo", ["foo"], ["foo"], {"foo": 0}, {"foo": "bar"}, [{"foo": ""}]],
+            new_list,
+        )
         # Ignore list
-        old_list = ['foo', ['foo'], ['foo', None], {'foo': 0}, {'foo': 'bar', 'baz': []}, [{'foo': ''}]]
-        new_list = salt.utils.data.filter_falsey(old_list, recurse_depth=3, ignore_types=[type([])])
-        self.assertEqual(['foo', ['foo'], ['foo'], {'foo': 'bar', 'baz': []}, []], new_list)
+        old_list = [
+            "foo",
+            ["foo"],
+            ["foo", None],
+            {"foo": 0},
+            {"foo": "bar", "baz": []},
+            [{"foo": ""}],
+        ]
+        new_list = salt.utils.data.filter_falsey(
+            old_list, recurse_depth=3, ignore_types=[type([])]
+        )
+        self.assertEqual(
+            ["foo", ["foo"], ["foo"], {"foo": "bar", "baz": []}, []], new_list
+        )
         # Ignore dict
-        old_list = ['foo', ['foo'], ['foo', None], {'foo': 0}, {'foo': 'bar', 'baz': []}, [{'foo': ''}]]
-        new_list = salt.utils.data.filter_falsey(old_list, recurse_depth=3, ignore_types=[type({})])
-        self.assertEqual(['foo', ['foo'], ['foo'], {}, {'foo': 'bar'}, [{}]], new_list)
+        old_list = [
+            "foo",
+            ["foo"],
+            ["foo", None],
+            {"foo": 0},
+            {"foo": "bar", "baz": []},
+            [{"foo": ""}],
+        ]
+        new_list = salt.utils.data.filter_falsey(
+            old_list, recurse_depth=3, ignore_types=[type({})]
+        )
+        self.assertEqual(["foo", ["foo"], ["foo"], {}, {"foo": "bar"}, [{}]], new_list)
         # Ignore NoneType
-        old_list = ['foo', ['foo'], ['foo', None], {'foo': 0}, {'foo': 'bar', 'baz': []}, [{'foo': ''}]]
-        new_list = salt.utils.data.filter_falsey(old_list, recurse_depth=3, ignore_types=[type(None)])
-        self.assertEqual(['foo', ['foo'], ['foo', None], {'foo': 'bar'}], new_list)
+        old_list = [
+            "foo",
+            ["foo"],
+            ["foo", None],
+            {"foo": 0},
+            {"foo": "bar", "baz": []},
+            [{"foo": ""}],
+        ]
+        new_list = salt.utils.data.filter_falsey(
+            old_list, recurse_depth=3, ignore_types=[type(None)]
+        )
+        self.assertEqual(["foo", ["foo"], ["foo", None], {"foo": "bar"}], new_list)
 
 
 class FilterRecursiveDiff(TestCase):
-    '''
+    """
     Test suite for salt.utils.data.recursive_diff
-    '''
+    """
 
     def test_list_equality(self):
-        '''
+        """
         Test cases where equal lists are compared.
-        '''
+        """
         test_list = [0, 1, 2]
         self.assertEqual({}, salt.utils.data.recursive_diff(test_list, test_list))
 
@@ -895,392 +1019,455 @@ class FilterRecursiveDiff(TestCase):
         self.assertEqual({}, salt.utils.data.recursive_diff(test_list, test_list))
 
     def test_dict_equality(self):
-        '''
+        """
         Test cases where equal dicts are compared.
-        '''
-        test_dict = {'foo': 'bar', 'bar': {'baz': {'qux': 'quux'}}, 'frop': 0}
+        """
+        test_dict = {"foo": "bar", "bar": {"baz": {"qux": "quux"}}, "frop": 0}
         self.assertEqual({}, salt.utils.data.recursive_diff(test_dict, test_dict))
 
     def test_ordereddict_equality(self):
-        '''
+        """
         Test cases where equal OrderedDicts are compared.
-        '''
-        test_dict = OrderedDict([
-            ('foo', 'bar'),
-            ('bar', OrderedDict([('baz', OrderedDict([('qux', 'quux')]))])),
-            ('frop', 0)])
+        """
+        test_dict = OrderedDict(
+            [
+                ("foo", "bar"),
+                ("bar", OrderedDict([("baz", OrderedDict([("qux", "quux")]))])),
+                ("frop", 0),
+            ]
+        )
         self.assertEqual({}, salt.utils.data.recursive_diff(test_dict, test_dict))
 
     def test_mixed_equality(self):
-        '''
+        """
         Test cases where mixed nested lists and dicts are compared.
-        '''
+        """
         test_data = {
-            'foo': 'bar',
-            'baz': [0, 1, 2],
-            'bar': {'baz': [{'qux': 'quux'}, {'froop', 0}]}
+            "foo": "bar",
+            "baz": [0, 1, 2],
+            "bar": {"baz": [{"qux": "quux"}, {"froop", 0}]},
         }
         self.assertEqual({}, salt.utils.data.recursive_diff(test_data, test_data))
 
     def test_set_equality(self):
-        '''
+        """
         Test cases where equal sets are compared.
-        '''
-        test_set = set([0, 1, 2, 3, 'foo'])
+        """
+        test_set = {0, 1, 2, 3, "foo"}
         self.assertEqual({}, salt.utils.data.recursive_diff(test_set, test_set))
 
         # This is a bit of an oddity, as python seems to sort the sets in memory
         # so both sets end up with the same ordering (0..3).
-        set_one = set([0, 1, 2, 3])
-        set_two = set([3, 2, 1, 0])
+        set_one = {0, 1, 2, 3}
+        set_two = {3, 2, 1, 0}
         self.assertEqual({}, salt.utils.data.recursive_diff(set_one, set_two))
 
     def test_tuple_equality(self):
-        '''
+        """
         Test cases where equal tuples are compared.
-        '''
-        test_tuple = (0, 1, 2, 3, 'foo')
+        """
+        test_tuple = (0, 1, 2, 3, "foo")
         self.assertEqual({}, salt.utils.data.recursive_diff(test_tuple, test_tuple))
 
     def test_list_inequality(self):
-        '''
+        """
         Test cases where two inequal lists are compared.
-        '''
+        """
         list_one = [0, 1, 2]
-        list_two = ['foo', 'bar', 'baz']
-        expected_result = {'old': list_one, 'new': list_two}
-        self.assertEqual(expected_result, salt.utils.data.recursive_diff(list_one, list_two))
-        expected_result = {'new': list_one, 'old': list_two}
-        self.assertEqual(expected_result, salt.utils.data.recursive_diff(list_two, list_one))
-
-        list_one = [0, 'foo', 1, 'bar']
-        list_two = [1, 'foo', 1, 'qux']
-        expected_result = {'old': [0, 'bar'], 'new': [1, 'qux']}
-        self.assertEqual(expected_result, salt.utils.data.recursive_diff(list_one, list_two))
-        expected_result = {'new': [0, 'bar'], 'old': [1, 'qux']}
-        self.assertEqual(expected_result, salt.utils.data.recursive_diff(list_two, list_one))
+        list_two = ["foo", "bar", "baz"]
+        expected_result = {"old": list_one, "new": list_two}
+        self.assertEqual(
+            expected_result, salt.utils.data.recursive_diff(list_one, list_two)
+        )
+        expected_result = {"new": list_one, "old": list_two}
+        self.assertEqual(
+            expected_result, salt.utils.data.recursive_diff(list_two, list_one)
+        )
+
+        list_one = [0, "foo", 1, "bar"]
+        list_two = [1, "foo", 1, "qux"]
+        expected_result = {"old": [0, "bar"], "new": [1, "qux"]}
+        self.assertEqual(
+            expected_result, salt.utils.data.recursive_diff(list_one, list_two)
+        )
+        expected_result = {"new": [0, "bar"], "old": [1, "qux"]}
+        self.assertEqual(
+            expected_result, salt.utils.data.recursive_diff(list_two, list_one)
+        )
 
         list_one = [0, 1, [2, 3]]
-        list_two = [0, 1, ['foo', 'bar']]
-        expected_result = {'old': [[2, 3]], 'new': [['foo', 'bar']]}
-        self.assertEqual(expected_result, salt.utils.data.recursive_diff(list_one, list_two))
-        expected_result = {'new': [[2, 3]], 'old': [['foo', 'bar']]}
-        self.assertEqual(expected_result, salt.utils.data.recursive_diff(list_two, list_one))
+        list_two = [0, 1, ["foo", "bar"]]
+        expected_result = {"old": [[2, 3]], "new": [["foo", "bar"]]}
+        self.assertEqual(
+            expected_result, salt.utils.data.recursive_diff(list_one, list_two)
+        )
+        expected_result = {"new": [[2, 3]], "old": [["foo", "bar"]]}
+        self.assertEqual(
+            expected_result, salt.utils.data.recursive_diff(list_two, list_one)
+        )
 
     def test_dict_inequality(self):
-        '''
+        """
         Test cases where two inequal dicts are compared.
-        '''
-        dict_one = {'foo': 1, 'bar': 2, 'baz': 3}
-        dict_two = {'foo': 2, 1: 'bar', 'baz': 3}
-        expected_result = {'old': {'foo': 1, 'bar': 2}, 'new': {'foo': 2, 1: 'bar'}}
-        self.assertEqual(expected_result, salt.utils.data.recursive_diff(dict_one, dict_two))
-        expected_result = {'new': {'foo': 1, 'bar': 2}, 'old': {'foo': 2, 1: 'bar'}}
-        self.assertEqual(expected_result, salt.utils.data.recursive_diff(dict_two, dict_one))
-
-        dict_one = {'foo': {'bar': {'baz': 1}}}
-        dict_two = {'foo': {'qux': {'baz': 1}}}
-        expected_result = {'old': dict_one, 'new': dict_two}
-        self.assertEqual(expected_result, salt.utils.data.recursive_diff(dict_one, dict_two))
-        expected_result = {'new': dict_one, 'old': dict_two}
-        self.assertEqual(expected_result, salt.utils.data.recursive_diff(dict_two, dict_one))
+        """
+        dict_one = {"foo": 1, "bar": 2, "baz": 3}
+        dict_two = {"foo": 2, 1: "bar", "baz": 3}
+        expected_result = {"old": {"foo": 1, "bar": 2}, "new": {"foo": 2, 1: "bar"}}
+        self.assertEqual(
+            expected_result, salt.utils.data.recursive_diff(dict_one, dict_two)
+        )
+        expected_result = {"new": {"foo": 1, "bar": 2}, "old": {"foo": 2, 1: "bar"}}
+        self.assertEqual(
+            expected_result, salt.utils.data.recursive_diff(dict_two, dict_one)
+        )
+
+        dict_one = {"foo": {"bar": {"baz": 1}}}
+        dict_two = {"foo": {"qux": {"baz": 1}}}
+        expected_result = {"old": dict_one, "new": dict_two}
+        self.assertEqual(
+            expected_result, salt.utils.data.recursive_diff(dict_one, dict_two)
+        )
+        expected_result = {"new": dict_one, "old": dict_two}
+        self.assertEqual(
+            expected_result, salt.utils.data.recursive_diff(dict_two, dict_one)
+        )
 
     def test_ordereddict_inequality(self):
-        '''
+        """
         Test cases where two inequal OrderedDicts are compared.
-        '''
-        odict_one = OrderedDict([('foo', 'bar'), ('bar', 'baz')])
-        odict_two = OrderedDict([('bar', 'baz'), ('foo', 'bar')])
-        expected_result = {'old': odict_one, 'new': odict_two}
-        self.assertEqual(expected_result, salt.utils.data.recursive_diff(odict_one, odict_two))
+        """
+        odict_one = OrderedDict([("foo", "bar"), ("bar", "baz")])
+        odict_two = OrderedDict([("bar", "baz"), ("foo", "bar")])
+        expected_result = {"old": odict_one, "new": odict_two}
+        self.assertEqual(
+            expected_result, salt.utils.data.recursive_diff(odict_one, odict_two)
+        )
 
     def test_set_inequality(self):
-        '''
+        """
         Test cases where two inequal sets are compared.
         Tricky as the sets are compared zipped, so shuffled sets of equal values
         are considered different.
-        '''
-        set_one = set([0, 1, 2, 4])
-        set_two = set([0, 1, 3, 4])
-        expected_result = {'old': set([2]), 'new': set([3])}
-        self.assertEqual(expected_result, salt.utils.data.recursive_diff(set_one, set_two))
-        expected_result = {'new': set([2]), 'old': set([3])}
-        self.assertEqual(expected_result, salt.utils.data.recursive_diff(set_two, set_one))
+        """
+        set_one = {0, 1, 2, 4}
+        set_two = {0, 1, 3, 4}
+        expected_result = {"old": {2}, "new": {3}}
+        self.assertEqual(
+            expected_result, salt.utils.data.recursive_diff(set_one, set_two)
+        )
+        expected_result = {"new": {2}, "old": {3}}
+        self.assertEqual(
+            expected_result, salt.utils.data.recursive_diff(set_two, set_one)
+        )
 
         # It is unknown how different python versions will store sets in memory.
         # Python 2.7 seems to sort it (i.e. set_one below becomes {0, 1, 'foo', 'bar'}
         # However Python 3.6.8 stores it differently each run.
         # So just test for "not equal" here.
-        set_one = set([0, 'foo', 1, 'bar'])
-        set_two = set(['foo', 1, 'bar', 2])
+        set_one = {0, "foo", 1, "bar"}
+        set_two = {"foo", 1, "bar", 2}
         expected_result = {}
-        self.assertNotEqual(expected_result, salt.utils.data.recursive_diff(set_one, set_two))
+        self.assertNotEqual(
+            expected_result, salt.utils.data.recursive_diff(set_one, set_two)
+        )
 
     def test_mixed_inequality(self):
-        '''
+        """
         Test cases where two mixed dicts/iterables that are different are compared.
-        '''
-        dict_one = {'foo': [1, 2, 3]}
-        dict_two = {'foo': [3, 2, 1]}
-        expected_result = {'old': {'foo': [1, 3]}, 'new': {'foo': [3, 1]}}
-        self.assertEqual(expected_result, salt.utils.data.recursive_diff(dict_one, dict_two))
-        expected_result = {'new': {'foo': [1, 3]}, 'old': {'foo': [3, 1]}}
-        self.assertEqual(expected_result, salt.utils.data.recursive_diff(dict_two, dict_one))
-
-        list_one = [1, 2, {'foo': ['bar', {'foo': 1, 'bar': 2}]}]
-        list_two = [3, 4, {'foo': ['qux', {'foo': 1, 'bar': 2}]}]
-        expected_result = {'old': [1, 2, {'foo': ['bar']}], 'new': [3, 4, {'foo': ['qux']}]}
-        self.assertEqual(expected_result, salt.utils.data.recursive_diff(list_one, list_two))
-        expected_result = {'new': [1, 2, {'foo': ['bar']}], 'old': [3, 4, {'foo': ['qux']}]}
-        self.assertEqual(expected_result, salt.utils.data.recursive_diff(list_two, list_one))
-
-        mixed_one = {'foo': set([0, 1, 2]), 'bar': [0, 1, 2]}
-        mixed_two = {'foo': set([1, 2, 3]), 'bar': [1, 2, 3]}
+        """
+        dict_one = {"foo": [1, 2, 3]}
+        dict_two = {"foo": [3, 2, 1]}
+        expected_result = {"old": {"foo": [1, 3]}, "new": {"foo": [3, 1]}}
+        self.assertEqual(
+            expected_result, salt.utils.data.recursive_diff(dict_one, dict_two)
+        )
+        expected_result = {"new": {"foo": [1, 3]}, "old": {"foo": [3, 1]}}
+        self.assertEqual(
+            expected_result, salt.utils.data.recursive_diff(dict_two, dict_one)
+        )
+
+        list_one = [1, 2, {"foo": ["bar", {"foo": 1, "bar": 2}]}]
+        list_two = [3, 4, {"foo": ["qux", {"foo": 1, "bar": 2}]}]
         expected_result = {
-            'old': {'foo': set([0]), 'bar': [0, 1, 2]},
-            'new': {'foo': set([3]), 'bar': [1, 2, 3]}
+            "old": [1, 2, {"foo": ["bar"]}],
+            "new": [3, 4, {"foo": ["qux"]}],
         }
-        self.assertEqual(expected_result, salt.utils.data.recursive_diff(mixed_one, mixed_two))
+        self.assertEqual(
+            expected_result, salt.utils.data.recursive_diff(list_one, list_two)
+        )
+        expected_result = {
+            "new": [1, 2, {"foo": ["bar"]}],
+            "old": [3, 4, {"foo": ["qux"]}],
+        }
+        self.assertEqual(
+            expected_result, salt.utils.data.recursive_diff(list_two, list_one)
+        )
+
+        mixed_one = {"foo": {0, 1, 2}, "bar": [0, 1, 2]}
+        mixed_two = {"foo": {1, 2, 3}, "bar": [1, 2, 3]}
         expected_result = {
-            'new': {'foo': set([0]), 'bar': [0, 1, 2]},
-            'old': {'foo': set([3]), 'bar': [1, 2, 3]}
+            "old": {"foo": {0}, "bar": [0, 1, 2]},
+            "new": {"foo": {3}, "bar": [1, 2, 3]},
         }
-        self.assertEqual(expected_result, salt.utils.data.recursive_diff(mixed_two, mixed_one))
+        self.assertEqual(
+            expected_result, salt.utils.data.recursive_diff(mixed_one, mixed_two)
+        )
+        expected_result = {
+            "new": {"foo": {0}, "bar": [0, 1, 2]},
+            "old": {"foo": {3}, "bar": [1, 2, 3]},
+        }
+        self.assertEqual(
+            expected_result, salt.utils.data.recursive_diff(mixed_two, mixed_one)
+        )
 
     def test_tuple_inequality(self):
-        '''
+        """
         Test cases where two tuples that are different are compared.
-        '''
+        """
         tuple_one = (1, 2, 3)
         tuple_two = (3, 2, 1)
-        expected_result = {'old': (1, 3), 'new': (3, 1)}
-        self.assertEqual(expected_result, salt.utils.data.recursive_diff(tuple_one, tuple_two))
+        expected_result = {"old": (1, 3), "new": (3, 1)}
+        self.assertEqual(
+            expected_result, salt.utils.data.recursive_diff(tuple_one, tuple_two)
+        )
 
     def test_list_vs_set(self):
-        '''
+        """
         Test case comparing a list with a set, will be compared unordered.
-        '''
+        """
         mixed_one = [1, 2, 3]
-        mixed_two = set([3, 2, 1])
+        mixed_two = {3, 2, 1}
         expected_result = {}
-        self.assertEqual(expected_result, salt.utils.data.recursive_diff(mixed_one, mixed_two))
-        self.assertEqual(expected_result, salt.utils.data.recursive_diff(mixed_two, mixed_one))
+        self.assertEqual(
+            expected_result, salt.utils.data.recursive_diff(mixed_one, mixed_two)
+        )
+        self.assertEqual(
+            expected_result, salt.utils.data.recursive_diff(mixed_two, mixed_one)
+        )
 
     def test_dict_vs_ordereddict(self):
-        '''
+        """
         Test case comparing a dict with an ordereddict, will be compared unordered.
-        '''
-        test_dict = {'foo': 'bar', 'bar': 'baz'}
-        test_odict = OrderedDict([('foo', 'bar'), ('bar', 'baz')])
+        """
+        test_dict = {"foo": "bar", "bar": "baz"}
+        test_odict = OrderedDict([("foo", "bar"), ("bar", "baz")])
         self.assertEqual({}, salt.utils.data.recursive_diff(test_dict, test_odict))
         self.assertEqual({}, salt.utils.data.recursive_diff(test_odict, test_dict))
 
-        test_odict2 = OrderedDict([('bar', 'baz'), ('foo', 'bar')])
+        test_odict2 = OrderedDict([("bar", "baz"), ("foo", "bar")])
         self.assertEqual({}, salt.utils.data.recursive_diff(test_dict, test_odict2))
         self.assertEqual({}, salt.utils.data.recursive_diff(test_odict2, test_dict))
 
     def test_list_ignore_ignored(self):
-        '''
+        """
         Test case comparing two lists with ignore-list supplied (which is not used
         when comparing lists).
-        '''
+        """
         list_one = [1, 2, 3]
         list_two = [3, 2, 1]
-        expected_result = {'old': [1, 3], 'new': [3, 1]}
+        expected_result = {"old": [1, 3], "new": [3, 1]}
         self.assertEqual(
             expected_result,
-            salt.utils.data.recursive_diff(list_one, list_two, ignore_keys=[1, 3])
+            salt.utils.data.recursive_diff(list_one, list_two, ignore_keys=[1, 3]),
         )
 
     def test_dict_ignore(self):
-        '''
+        """
         Test case comparing two dicts with ignore-list supplied.
-        '''
-        dict_one = {'foo': 1, 'bar': 2, 'baz': 3}
-        dict_two = {'foo': 3, 'bar': 2, 'baz': 1}
-        expected_result = {'old': {'baz': 3}, 'new': {'baz': 1}}
+        """
+        dict_one = {"foo": 1, "bar": 2, "baz": 3}
+        dict_two = {"foo": 3, "bar": 2, "baz": 1}
+        expected_result = {"old": {"baz": 3}, "new": {"baz": 1}}
         self.assertEqual(
             expected_result,
-            salt.utils.data.recursive_diff(dict_one, dict_two, ignore_keys=['foo'])
+            salt.utils.data.recursive_diff(dict_one, dict_two, ignore_keys=["foo"]),
         )
 
     def test_ordereddict_ignore(self):
-        '''
+        """
         Test case comparing two OrderedDicts with ignore-list supplied.
-        '''
-        odict_one = OrderedDict([('foo', 1), ('bar', 2), ('baz', 3)])
-        odict_two = OrderedDict([('baz', 1), ('bar', 2), ('foo', 3)])
+        """
+        odict_one = OrderedDict([("foo", 1), ("bar", 2), ("baz", 3)])
+        odict_two = OrderedDict([("baz", 1), ("bar", 2), ("foo", 3)])
         # The key 'foo' will be ignored, which means the key from the other OrderedDict
         # will always be considered "different" since OrderedDicts are compared ordered.
-        expected_result = {'old': OrderedDict([('baz', 3)]), 'new': OrderedDict([('baz', 1)])}
+        expected_result = {
+            "old": OrderedDict([("baz", 3)]),
+            "new": OrderedDict([("baz", 1)]),
+        }
         self.assertEqual(
             expected_result,
-            salt.utils.data.recursive_diff(odict_one, odict_two, ignore_keys=['foo'])
+            salt.utils.data.recursive_diff(odict_one, odict_two, ignore_keys=["foo"]),
         )
 
     def test_dict_vs_ordereddict_ignore(self):
-        '''
+        """
         Test case comparing a dict with an OrderedDict with ignore-list supplied.
-        '''
-        dict_one = {'foo': 1, 'bar': 2, 'baz': 3}
-        odict_two = OrderedDict([('foo', 3), ('bar', 2), ('baz', 1)])
-        expected_result = {'old': {'baz': 3}, 'new': OrderedDict([('baz', 1)])}
+        """
+        dict_one = {"foo": 1, "bar": 2, "baz": 3}
+        odict_two = OrderedDict([("foo", 3), ("bar", 2), ("baz", 1)])
+        expected_result = {"old": {"baz": 3}, "new": OrderedDict([("baz", 1)])}
         self.assertEqual(
             expected_result,
-            salt.utils.data.recursive_diff(dict_one, odict_two, ignore_keys=['foo'])
+            salt.utils.data.recursive_diff(dict_one, odict_two, ignore_keys=["foo"]),
         )
 
     def test_mixed_nested_ignore(self):
-        '''
+        """
         Test case comparing mixed, nested items with ignore-list supplied.
-        '''
-        dict_one = {'foo': [1], 'bar': {'foo': 1, 'bar': 2}, 'baz': 3}
-        dict_two = {'foo': [2], 'bar': {'foo': 3, 'bar': 2}, 'baz': 1}
-        expected_result = {'old': {'baz': 3}, 'new': {'baz': 1}}
+        """
+        dict_one = {"foo": [1], "bar": {"foo": 1, "bar": 2}, "baz": 3}
+        dict_two = {"foo": [2], "bar": {"foo": 3, "bar": 2}, "baz": 1}
+        expected_result = {"old": {"baz": 3}, "new": {"baz": 1}}
         self.assertEqual(
             expected_result,
-            salt.utils.data.recursive_diff(dict_one, dict_two, ignore_keys=['foo'])
+            salt.utils.data.recursive_diff(dict_one, dict_two, ignore_keys=["foo"]),
         )
 
     def test_ordered_dict_unequal_length(self):
-        '''
+        """
         Test case comparing two OrderedDicts of unequal length.
-        '''
-        odict_one = OrderedDict([('foo', 1), ('bar', 2), ('baz', 3)])
-        odict_two = OrderedDict([('foo', 1), ('bar', 2)])
-        expected_result = {'old': OrderedDict([('baz', 3)]), 'new': {}}
+        """
+        odict_one = OrderedDict([("foo", 1), ("bar", 2), ("baz", 3)])
+        odict_two = OrderedDict([("foo", 1), ("bar", 2)])
+        expected_result = {"old": OrderedDict([("baz", 3)]), "new": {}}
         self.assertEqual(
-            expected_result,
-            salt.utils.data.recursive_diff(odict_one, odict_two)
+            expected_result, salt.utils.data.recursive_diff(odict_one, odict_two)
         )
 
     def test_list_unequal_length(self):
-        '''
+        """
         Test case comparing two lists of unequal length.
-        '''
+        """
         list_one = [1, 2, 3]
         list_two = [1, 2, 3, 4]
-        expected_result = {'old': [], 'new': [4]}
+        expected_result = {"old": [], "new": [4]}
         self.assertEqual(
-            expected_result,
-            salt.utils.data.recursive_diff(list_one, list_two)
+            expected_result, salt.utils.data.recursive_diff(list_one, list_two)
         )
 
     def test_set_unequal_length(self):
-        '''
+        """
         Test case comparing two sets of unequal length.
         This does not do anything special, as it is unordered.
-        '''
-        set_one = set([1, 2, 3])
-        set_two = set([4, 3, 2, 1])
-        expected_result = {'old': set([]), 'new': set([4])}
+        """
+        set_one = {1, 2, 3}
+        set_two = {4, 3, 2, 1}
+        expected_result = {"old": set(), "new": {4}}
         self.assertEqual(
-            expected_result,
-            salt.utils.data.recursive_diff(set_one, set_two)
+            expected_result, salt.utils.data.recursive_diff(set_one, set_two)
         )
 
     def test_tuple_unequal_length(self):
-        '''
+        """
         Test case comparing two tuples of unequal length.
         This should be the same as comparing two ordered lists.
-        '''
+        """
         tuple_one = (1, 2, 3)
         tuple_two = (1, 2, 3, 4)
-        expected_result = {'old': (), 'new': (4,)}
+        expected_result = {"old": (), "new": (4,)}
         self.assertEqual(
-            expected_result,
-            salt.utils.data.recursive_diff(tuple_one, tuple_two)
+            expected_result, salt.utils.data.recursive_diff(tuple_one, tuple_two)
         )
 
     def test_list_unordered(self):
-        '''
+        """
         Test case comparing two lists unordered.
-        '''
+        """
         list_one = [1, 2, 3, 4]
         list_two = [4, 3, 2]
-        expected_result = {'old': [1], 'new': []}
+        expected_result = {"old": [1], "new": []}
         self.assertEqual(
             expected_result,
-            salt.utils.data.recursive_diff(list_one, list_two, ignore_order=True)
+            salt.utils.data.recursive_diff(list_one, list_two, ignore_order=True),
         )
 
     def test_mixed_nested_unordered(self):
-        '''
+        """
         Test case comparing nested dicts/lists unordered.
-        '''
-        dict_one = {'foo': {'bar': [1, 2, 3]}, 'bar': [{'foo': 4}, 0]}
-        dict_two = {'foo': {'bar': [3, 2, 1]}, 'bar': [0, {'foo': 4}]}
+        """
+        dict_one = {"foo": {"bar": [1, 2, 3]}, "bar": [{"foo": 4}, 0]}
+        dict_two = {"foo": {"bar": [3, 2, 1]}, "bar": [0, {"foo": 4}]}
         expected_result = {}
         self.assertEqual(
             expected_result,
-            salt.utils.data.recursive_diff(dict_one, dict_two, ignore_order=True)
+            salt.utils.data.recursive_diff(dict_one, dict_two, ignore_order=True),
         )
         expected_result = {
-            'old': {'foo': {'bar': [1, 3]}, 'bar': [{'foo': 4}, 0]},
-            'new': {'foo': {'bar': [3, 1]}, 'bar': [0, {'foo': 4}]},
+            "old": {"foo": {"bar": [1, 3]}, "bar": [{"foo": 4}, 0]},
+            "new": {"foo": {"bar": [3, 1]}, "bar": [0, {"foo": 4}]},
         }
         self.assertEqual(
-            expected_result,
-            salt.utils.data.recursive_diff(dict_one, dict_two)
+            expected_result, salt.utils.data.recursive_diff(dict_one, dict_two)
         )
 
     def test_ordered_dict_unordered(self):
-        '''
+        """
         Test case comparing OrderedDicts unordered.
-        '''
-        odict_one = OrderedDict([('foo', 1), ('bar', 2), ('baz', 3)])
-        odict_two = OrderedDict([('baz', 3), ('bar', 2), ('foo', 1)])
+        """
+        odict_one = OrderedDict([("foo", 1), ("bar", 2), ("baz", 3)])
+        odict_two = OrderedDict([("baz", 3), ("bar", 2), ("foo", 1)])
         expected_result = {}
         self.assertEqual(
             expected_result,
-            salt.utils.data.recursive_diff(odict_one, odict_two, ignore_order=True)
+            salt.utils.data.recursive_diff(odict_one, odict_two, ignore_order=True),
         )
 
     def test_ignore_missing_keys_dict(self):
-        '''
+        """
         Test case ignoring missing keys on a comparison of dicts.
-        '''
-        dict_one = {'foo': 1, 'bar': 2, 'baz': 3}
-        dict_two = {'bar': 3}
-        expected_result = {'old': {'bar': 2}, 'new': {'bar': 3}}
+        """
+        dict_one = {"foo": 1, "bar": 2, "baz": 3}
+        dict_two = {"bar": 3}
+        expected_result = {"old": {"bar": 2}, "new": {"bar": 3}}
         self.assertEqual(
             expected_result,
-            salt.utils.data.recursive_diff(dict_one, dict_two, ignore_missing_keys=True)
+            salt.utils.data.recursive_diff(
+                dict_one, dict_two, ignore_missing_keys=True
+            ),
         )
 
     def test_ignore_missing_keys_ordered_dict(self):
-        '''
+        """
         Test case not ignoring missing keys on a comparison of OrderedDicts.
-        '''
-        odict_one = OrderedDict([('foo', 1), ('bar', 2), ('baz', 3)])
-        odict_two = OrderedDict([('bar', 3)])
-        expected_result = {'old': odict_one, 'new': odict_two}
+        """
+        odict_one = OrderedDict([("foo", 1), ("bar", 2), ("baz", 3)])
+        odict_two = OrderedDict([("bar", 3)])
+        expected_result = {"old": odict_one, "new": odict_two}
         self.assertEqual(
             expected_result,
-            salt.utils.data.recursive_diff(odict_one, odict_two, ignore_missing_keys=True)
+            salt.utils.data.recursive_diff(
+                odict_one, odict_two, ignore_missing_keys=True
+            ),
         )
 
     def test_ignore_missing_keys_recursive(self):
-        '''
+        """
         Test case ignoring missing keys on a comparison of nested dicts.
-        '''
-        dict_one = {'foo': {'bar': 2, 'baz': 3}}
-        dict_two = {'foo': {'baz': 3}}
+        """
+        dict_one = {"foo": {"bar": 2, "baz": 3}}
+        dict_two = {"foo": {"baz": 3}}
         expected_result = {}
         self.assertEqual(
             expected_result,
-            salt.utils.data.recursive_diff(dict_one, dict_two, ignore_missing_keys=True)
+            salt.utils.data.recursive_diff(
+                dict_one, dict_two, ignore_missing_keys=True
+            ),
         )
         # Compare from dict-in-dict
         dict_two = {}
         self.assertEqual(
             expected_result,
-            salt.utils.data.recursive_diff(dict_one, dict_two, ignore_missing_keys=True)
+            salt.utils.data.recursive_diff(
+                dict_one, dict_two, ignore_missing_keys=True
+            ),
         )
         # Compare from dict-in-list
-        dict_one = {'foo': ['bar', {'baz': 3}]}
-        dict_two = {'foo': ['bar', {}]}
+        dict_one = {"foo": ["bar", {"baz": 3}]}
+        dict_two = {"foo": ["bar", {}]}
         self.assertEqual(
             expected_result,
-            salt.utils.data.recursive_diff(dict_one, dict_two, ignore_missing_keys=True)
+            salt.utils.data.recursive_diff(
+                dict_one, dict_two, ignore_missing_keys=True
+            ),
         )
diff --git a/tests/unit/utils/test_xmlutil.py b/tests/unit/utils/test_xmlutil.py
index c04f39498e..cbf73861e5 100644
--- a/tests/unit/utils/test_xmlutil.py
+++ b/tests/unit/utils/test_xmlutil.py
@@ -1,148 +1,170 @@
-# -*- coding: utf-8 -*-
-'''
+"""
     tests.unit.xmlutil_test
     ~~~~~~~~~~~~~~~~~~~~
-'''
-from __future__ import absolute_import, print_function, unicode_literals
-# Import Salt Testing libs
-from tests.support.unit import TestCase
+"""
+import salt.utils.xmlutil as xml
 
 # Import Salt libs
 from salt._compat import ElementTree as ET
-import salt.utils.xmlutil as xml
+
+# Import Salt Testing libs
+from tests.support.unit import TestCase
 
 
 class XMLUtilTestCase(TestCase):
-    '''
+    """
     Tests that salt.utils.xmlutil properly parses XML data and returns as a properly formatted
     dictionary. The default method of parsing will ignore attributes and return only the child
     items. The full method will include parsing attributes.
-    '''
+    """
 
     def setUp(self):
 
         # Populate our use cases for specific XML formats.
         self.cases = {
-            'a': {
-                'xml': '<parent>data</parent>',
-                'legacy': {'parent': 'data'},
-                'full': 'data'
+            "a": {
+                "xml": "<parent>data</parent>",
+                "legacy": {"parent": "data"},
+                "full": "data",
             },
-            'b': {
-                'xml': '<parent value="data">data</parent>',
-                'legacy': {'parent': 'data'},
-                'full': {'parent': 'data', 'value': 'data'}
+            "b": {
+                "xml": '<parent value="data">data</parent>',
+                "legacy": {"parent": "data"},
+                "full": {"parent": "data", "value": "data"},
             },
-            'c': {
-                'xml': '<parent><child>data</child><child value="data">data</child>'
-                       '<child value="data"/><child/></parent>',
-                'legacy': {'child': ['data', {'child': 'data'}, {'child': None}, {'child': None}]},
-                'full': {'child': ['data', {'child': 'data', 'value': 'data'}, {'value': 'data'}, None]}
+            "c": {
+                "xml": '<parent><child>data</child><child value="data">data</child>'
+                '<child value="data"/><child/></parent>',
+                "legacy": {
+                    "child": [
+                        "data",
+                        {"child": "data"},
+                        {"child": None},
+                        {"child": None},
+                    ]
+                },
+                "full": {
+                    "child": [
+                        "data",
+                        {"child": "data", "value": "data"},
+                        {"value": "data"},
+                        None,
+                    ]
+                },
             },
-            'd': {
-                'xml': '<parent value="data" another="data"><child>data</child></parent>',
-                'legacy': {'child': 'data'},
-                'full': {'child': 'data', 'another': 'data', 'value': 'data'}
+            "d": {
+                "xml": '<parent value="data" another="data"><child>data</child></parent>',
+                "legacy": {"child": "data"},
+                "full": {"child": "data", "another": "data", "value": "data"},
             },
-            'e': {
-                'xml': '<parent value="data" another="data"><child value="data">data</child></parent>',
-                'legacy': {'child': 'data'},
-                'full': {'child': {'child': 'data', 'value': 'data'}, 'another': 'data', 'value': 'data'}
+            "e": {
+                "xml": '<parent value="data" another="data"><child value="data">data</child></parent>',
+                "legacy": {"child": "data"},
+                "full": {
+                    "child": {"child": "data", "value": "data"},
+                    "another": "data",
+                    "value": "data",
+                },
             },
-            'f': {
-                'xml': '<parent><child><sub-child value="data">data</sub-child></child>'
-                       '<child>data</child></parent>',
-                'legacy': {'child': [{'sub-child': 'data'}, {'child': 'data'}]},
-                'full': {'child': [{'sub-child': {'value': 'data', 'sub-child': 'data'}}, 'data']}
+            "f": {
+                "xml": '<parent><child><sub-child value="data">data</sub-child></child>'
+                "<child>data</child></parent>",
+                "legacy": {"child": [{"sub-child": "data"}, {"child": "data"}]},
+                "full": {
+                    "child": [
+                        {"sub-child": {"value": "data", "sub-child": "data"}},
+                        "data",
+                    ]
+                },
             },
         }
 
     def test_xml_case_a(self):
-        xmldata = ET.fromstring(self.cases['a']['xml'])
+        xmldata = ET.fromstring(self.cases["a"]["xml"])
         defaultdict = xml.to_dict(xmldata)
-        self.assertEqual(defaultdict, self.cases['a']['legacy'])
+        self.assertEqual(defaultdict, self.cases["a"]["legacy"])
 
     def test_xml_case_a_legacy(self):
-        xmldata = ET.fromstring(self.cases['a']['xml'])
+        xmldata = ET.fromstring(self.cases["a"]["xml"])
         defaultdict = xml.to_dict(xmldata, False)
-        self.assertEqual(defaultdict, self.cases['a']['legacy'])
+        self.assertEqual(defaultdict, self.cases["a"]["legacy"])
 
     def test_xml_case_a_full(self):
-        xmldata = ET.fromstring(self.cases['a']['xml'])
+        xmldata = ET.fromstring(self.cases["a"]["xml"])
         defaultdict = xml.to_dict(xmldata, True)
-        self.assertEqual(defaultdict, self.cases['a']['full'])
+        self.assertEqual(defaultdict, self.cases["a"]["full"])
 
     def test_xml_case_b(self):
-        xmldata = ET.fromstring(self.cases['b']['xml'])
+        xmldata = ET.fromstring(self.cases["b"]["xml"])
         defaultdict = xml.to_dict(xmldata)
-        self.assertEqual(defaultdict, self.cases['b']['legacy'])
+        self.assertEqual(defaultdict, self.cases["b"]["legacy"])
 
     def test_xml_case_b_legacy(self):
-        xmldata = ET.fromstring(self.cases['b']['xml'])
+        xmldata = ET.fromstring(self.cases["b"]["xml"])
         defaultdict = xml.to_dict(xmldata, False)
-        self.assertEqual(defaultdict, self.cases['b']['legacy'])
+        self.assertEqual(defaultdict, self.cases["b"]["legacy"])
 
     def test_xml_case_b_full(self):
-        xmldata = ET.fromstring(self.cases['b']['xml'])
+        xmldata = ET.fromstring(self.cases["b"]["xml"])
         defaultdict = xml.to_dict(xmldata, True)
-        self.assertEqual(defaultdict, self.cases['b']['full'])
+        self.assertEqual(defaultdict, self.cases["b"]["full"])
 
     def test_xml_case_c(self):
-        xmldata = ET.fromstring(self.cases['c']['xml'])
+        xmldata = ET.fromstring(self.cases["c"]["xml"])
         defaultdict = xml.to_dict(xmldata)
-        self.assertEqual(defaultdict, self.cases['c']['legacy'])
+        self.assertEqual(defaultdict, self.cases["c"]["legacy"])
 
     def test_xml_case_c_legacy(self):
-        xmldata = ET.fromstring(self.cases['c']['xml'])
+        xmldata = ET.fromstring(self.cases["c"]["xml"])
         defaultdict = xml.to_dict(xmldata, False)
-        self.assertEqual(defaultdict, self.cases['c']['legacy'])
+        self.assertEqual(defaultdict, self.cases["c"]["legacy"])
 
     def test_xml_case_c_full(self):
-        xmldata = ET.fromstring(self.cases['c']['xml'])
+        xmldata = ET.fromstring(self.cases["c"]["xml"])
         defaultdict = xml.to_dict(xmldata, True)
-        self.assertEqual(defaultdict, self.cases['c']['full'])
+        self.assertEqual(defaultdict, self.cases["c"]["full"])
 
     def test_xml_case_d(self):
-        xmldata = ET.fromstring(self.cases['d']['xml'])
+        xmldata = ET.fromstring(self.cases["d"]["xml"])
         defaultdict = xml.to_dict(xmldata)
-        self.assertEqual(defaultdict, self.cases['d']['legacy'])
+        self.assertEqual(defaultdict, self.cases["d"]["legacy"])
 
     def test_xml_case_d_legacy(self):
-        xmldata = ET.fromstring(self.cases['d']['xml'])
+        xmldata = ET.fromstring(self.cases["d"]["xml"])
         defaultdict = xml.to_dict(xmldata, False)
-        self.assertEqual(defaultdict, self.cases['d']['legacy'])
+        self.assertEqual(defaultdict, self.cases["d"]["legacy"])
 
     def test_xml_case_d_full(self):
-        xmldata = ET.fromstring(self.cases['d']['xml'])
+        xmldata = ET.fromstring(self.cases["d"]["xml"])
         defaultdict = xml.to_dict(xmldata, True)
-        self.assertEqual(defaultdict, self.cases['d']['full'])
+        self.assertEqual(defaultdict, self.cases["d"]["full"])
 
     def test_xml_case_e(self):
-        xmldata = ET.fromstring(self.cases['e']['xml'])
+        xmldata = ET.fromstring(self.cases["e"]["xml"])
         defaultdict = xml.to_dict(xmldata)
-        self.assertEqual(defaultdict, self.cases['e']['legacy'])
+        self.assertEqual(defaultdict, self.cases["e"]["legacy"])
 
     def test_xml_case_e_legacy(self):
-        xmldata = ET.fromstring(self.cases['e']['xml'])
+        xmldata = ET.fromstring(self.cases["e"]["xml"])
         defaultdict = xml.to_dict(xmldata, False)
-        self.assertEqual(defaultdict, self.cases['e']['legacy'])
+        self.assertEqual(defaultdict, self.cases["e"]["legacy"])
 
     def test_xml_case_e_full(self):
-        xmldata = ET.fromstring(self.cases['e']['xml'])
+        xmldata = ET.fromstring(self.cases["e"]["xml"])
         defaultdict = xml.to_dict(xmldata, True)
-        self.assertEqual(defaultdict, self.cases['e']['full'])
+        self.assertEqual(defaultdict, self.cases["e"]["full"])
 
     def test_xml_case_f(self):
-        xmldata = ET.fromstring(self.cases['f']['xml'])
+        xmldata = ET.fromstring(self.cases["f"]["xml"])
         defaultdict = xml.to_dict(xmldata)
-        self.assertEqual(defaultdict, self.cases['f']['legacy'])
+        self.assertEqual(defaultdict, self.cases["f"]["legacy"])
 
     def test_xml_case_f_legacy(self):
-        xmldata = ET.fromstring(self.cases['f']['xml'])
+        xmldata = ET.fromstring(self.cases["f"]["xml"])
         defaultdict = xml.to_dict(xmldata, False)
-        self.assertEqual(defaultdict, self.cases['f']['legacy'])
+        self.assertEqual(defaultdict, self.cases["f"]["legacy"])
 
     def test_xml_case_f_full(self):
-        xmldata = ET.fromstring(self.cases['f']['xml'])
+        xmldata = ET.fromstring(self.cases["f"]["xml"])
         defaultdict = xml.to_dict(xmldata, True)
-        self.assertEqual(defaultdict, self.cases['f']['full'])
+        self.assertEqual(defaultdict, self.cases["f"]["full"])
-- 
2.28.0

openSUSE Build Service is sponsored by