File 0001-SCRD-3016-maintenance-update-workflow-help... of Package ardana-ansible

Overview Repositories Revisions Requests Users Attributes Meta

File 0001-SCRD-3016-maintenance-update-workflow-helper-playboo.patch of Package ardana-ansible (Revision e4a2bba0aa8198fec711773bceb0bea3)

Currently displaying revision e4a2bba0aa8198fec711773bceb0bea3 , Show latest

From 0d9d1ee1904b82a034a0981120aeb219dec0635b Mon Sep 17 00:00:00 2001
From: Stefan Nica <snica@suse.com>
Date: Wed, 18 Apr 2018 13:31:47 +0200
Subject: [PATCH] SCRD-3016 maintenance update workflow helper playbooks

Adds a few playbooks automating some of the maintenance
update workflow steps that are not already covered by
ardana-update.yml.

A set of ardana_notify_ special variables are introduced
and a set of playbooks that can be used to interact with
them, to cope with the fact that top-level playbooks cannot
be conditionally included in ansible<2.0.

To provide the maximum flexibility and reusability, the
various automated workflow steps are implemented by separate
playbooks:

- ardana-update-pkgs.yml - this top-level playbook automates
  the installation of package updates on a single node (or even
  multiple nodes, if the single-node restriction is overridden
  by setting the SKIP_SINGLE_HOST_CHECKS environment variable).
  The behavior can be customized by controlling the following
  variables:

* zypper_update_method: define which zypper subcommand to use for
      updating. Valid values are:

'patch' (default) - will install all patches applying to
      the system from the configured update repositories that are available.

'update' - will update packages from all configured repositories
      (not just the update repositories) that have a higher version
      number than the installed packages.

'dist-upgrade' - replaces each package installed with the version
      from the repository and deletes packages not available in the
      repositories.

* zypper_update_gpg_checks (default: true): enable GPG checks.
      If set to true, checks if packages are correctly signed.

* zypper_update_licenses_agree (default: false): automatically agree
      with licenses. If set to true, zypper automatically accepts third
      party licenses.

* zypper_update_include_reboot_patches (default: false): include patches
      that need reboots. Installs patches that require a reboot (for example
      kernel or glibc updates).

During an ardana-update-pkgs.yml run, the update ardana_notify_ variables
  are checked and updated to reflect the status of the update operation.

- ardana-reboot.yml: top-level playbook that automates the steps
    required to reboot a node.
    It involves pre-boot and post-boot phases, which can/will be extended
    to include additional checks, such as verifying the reboot readiness
    of a node (e.g. making sure the singleton agent services have been
    migrated on controller nodes, checking for runninng VMs on compute
    nodes etc.)

- roles/ardana-upgrade-tools/tasks/pkg-update.yml implements
  the actual installation of package updates. This is the equivalent
  of the updater chef recipe in crowbar [1].

- ardana-update-status.yml - this playbook can be used to check
  or reset the update related status variables maintained by the
  update playbooks. The main reason for having this mechanism is
  to allow the update status to be checked at any point during the
  update procedure. It will also be used heavily by the automation
  scripts to orchestrate installing maintenance updates on multiple
  nodes.

With these playbooks in place, the management update workflow can
be summarized as follows:

1. install package updates, e.g.:

ansible-playbook -i hosts/verb_hosts ardana-update-pkgs.yml \
        --limit <ansible-node-hostname> \
        -e \
          "zypper_update_method=patch
           zypper_update_gpg_checks=true
           zypper_update_licenses_agree=true
           zypper_update_include_reboot_patches=true"

2. check the update status at any time to figure out what is left
to be done:

ansible-playbook -i hosts/verb_hosts ardana-update-status.yml \
        --limit <ansible-node-hostname>

3. update services:

ansible-playbook -i hosts/verb_hosts ardana-update.yml \
        --limit <ansible-node-hostname>

4. reboot the node:

ansible-playbook -i hosts/verb_hosts ardana-reboot.yml \
        --limit <ansible-node-hostname>

NOTE: this changeset also includes a minor fix for the rpm_list plugin which
is required to allow the pkg-update.yml playbook to detect when
packages are updated to the staging version, e.g.:

[1] https://github.com/crowbar/crowbar-core/blob/stable/4.0/chef/cookbooks/updater/recipes/default.rb

Change-Id: If611f95926ee371734f54ce211425cf04809e603
---
 _ardana-post-reboot.yml                            |  25 +++
 _ardana-pre-reboot.yml                             |  17 ++
 _ardana-update-base.yml                            |  51 +++++
 _ardana-update-status.yml                          |  81 ++++++++
 ardana-reboot.yml                                  |  67 +++++++
 ardana-update-pkgs.yml                             |  45 +++++
 ardana-update-status.yml                           |  36 ++++
 library/rpm_list.py                                |   2 +-
 roles/ardana-upgrade-tools/defaults/main.yml       |  14 +-
 .../ardana-upgrade-tools/tasks/check-deployer.yml  |  21 ++
 .../tasks/check-pending-clm-update.yml             | 105 ++++++++++
 .../tasks/check-pending-service-update.yml         | 100 ++++++++++
 .../tasks/check-pending-system-reboot.yml          | 102 ++++++++++
 .../tasks/check-single-host.yml                    |  28 +++
 roles/ardana-upgrade-tools/tasks/pkg-update.yml    | 217 +++++++++++++++++++++
 roles/ardana-upgrade-tools/tasks/reboot.yml        |  32 +++
 16 files changed, 941 insertions(+), 2 deletions(-)
 create mode 100644 _ardana-post-reboot.yml
 create mode 100644 _ardana-pre-reboot.yml
 create mode 100644 _ardana-update-base.yml
 create mode 100644 _ardana-update-status.yml
 create mode 100644 ardana-reboot.yml
 create mode 100644 ardana-update-pkgs.yml
 create mode 100644 ardana-update-status.yml
 create mode 100644 roles/ardana-upgrade-tools/tasks/check-deployer.yml
 create mode 100644 roles/ardana-upgrade-tools/tasks/check-pending-clm-update.yml
 create mode 100644 roles/ardana-upgrade-tools/tasks/check-pending-service-update.yml
 create mode 100644 roles/ardana-upgrade-tools/tasks/check-pending-system-reboot.yml
 create mode 100644 roles/ardana-upgrade-tools/tasks/check-single-host.yml
 create mode 100644 roles/ardana-upgrade-tools/tasks/pkg-update.yml
 create mode 100644 roles/ardana-upgrade-tools/tasks/reboot.yml

diff --git a/_ardana-post-reboot.yml b/_ardana-post-reboot.yml
new file mode 100644
index 0000000..41b7161
--- /dev/null
+++ b/_ardana-post-reboot.yml
@@ -0,0 +1,25 @@
+#
+# (c) Copyright 2018 SUSE LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License"); you may
+# not use this file except in compliance with the License. You may obtain
+# a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations
+# under the License.
+#
+---
+# Node reboot was completed successfully, reset the pending reboot status
+# variable (useful on the deployer, which is rebooted manually)
+- include: _ardana-update-status.yml
+  vars:
+    update_status_var: system-reboot
+    update_status_reset: true
+
+- include: ardana-start.yml
+- include: ardana-status.yml
diff --git a/_ardana-pre-reboot.yml b/_ardana-pre-reboot.yml
new file mode 100644
index 0000000..469a242
--- /dev/null
+++ b/_ardana-pre-reboot.yml
@@ -0,0 +1,17 @@
+#
+# (c) Copyright 2018 SUSE LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License"); you may
+# not use this file except in compliance with the License. You may obtain
+# a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations
+# under the License.
+#
+---
+- include: ardana-stop.yml
diff --git a/_ardana-update-base.yml b/_ardana-update-base.yml
new file mode 100644
index 0000000..ab22a54
--- /dev/null
+++ b/_ardana-update-base.yml
@@ -0,0 +1,51 @@
+#
+# (c) Copyright 2018 SUSE LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License"); you may
+# not use this file except in compliance with the License. You may obtain
+# a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations
+# under the License.
+#
+# Use this playbook to install available system package updates
+---
+
+- hosts: resources
+  roles:
+    - ardana-upgrade-tools
+  tasks:
+
+  - include: roles/ardana-upgrade-tools/tasks/check-deployer.yml
+
+  # When running on multiple nodes (not recommended), first update packages on
+  # deployer, to account for possible CLM package updates
+  - name: _ardana-update-base | Update packages on the deployer node
+    include: roles/ardana-upgrade-tools/tasks/pkg-update.yml
+    when: ansible_os_family | lower == 'suse' and
+          install_package_updates | default(false) | bool and
+          running_on_deployer
+
+  # Bail out if CLM package updates were installed on the deployer
+  # and if other nodes are targeted by the play run
+  - include: roles/ardana-upgrade-tools/tasks/check-pending-clm-update.yml
+    when: running_on_deployer and (play_hosts | length) > 1
+
+  - name: _ardana-update-base | Fail if CLM packages were updated
+    fail:
+      msg: "{{ pending_clm_update_msg }}"
+    run_once: true
+    when: running_on_deployer and (play_hosts | length) > 1 and
+          pending_clm_update | bool
+
+  # Then install package updates on remaining nodes
+  - name: _ardana-update-base | Update packages on non deployer nodes
+    include: roles/ardana-upgrade-tools/tasks/pkg-update.yml
+    when: ansible_os_family | lower == 'suse' and
+          install_package_updates | default(false) | bool and
+          not running_on_deployer
diff --git a/_ardana-update-status.yml b/_ardana-update-status.yml
new file mode 100644
index 0000000..af5d281
--- /dev/null
+++ b/_ardana-update-status.yml
@@ -0,0 +1,81 @@
+#
+# (c) Copyright 2018 SUSE LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License"); you may
+# not use this file except in compliance with the License. You may obtain
+# a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations
+# under the License.
+#
+---
+- hosts: resources
+  vars:
+    update_status_brief: false
+    update_status_var: all
+    update_status_set: false
+    update_status_reset: false
+  tasks:
+    - name: _ardana-update-status | Initialise update status list
+      set_fact:
+        update_status: []
+
+    - name: _ardana-update-status | Set default global update status
+      set_fact:
+        _default_update_status: |
+          No pending update actions on the {{ inventory_hostname }} host
+          were collected or reset during this update run or persisted during
+          previous unsuccessful or incomplete update runs.
+
+    - include: roles/ardana-upgrade-tools/tasks/check-pending-clm-update.yml
+      when: update_status_var in ["clm-update", "all"]
+
+    - name: _ardana-update-status | Add pending CLM update status to status list
+      set_fact:
+        update_status: "{{ update_status +
+          [update_status_brief | bool | ternary('CLM update', update_status_msg)] }}"
+      when: update_status_var in ["clm-update", "all"] and
+            pending_clm_update|bool
+
+    - include: roles/ardana-upgrade-tools/tasks/check-pending-service-update.yml
+      when: update_status_var in ["service-update", "all"]
+
+    - name: _ardana-update-status | Add pending service update status to status list
+      set_fact:
+        update_status: "{{ update_status +
+          [update_status_brief | bool | ternary('service update', update_status_msg)] }}"
+      when: update_status_var in ["service-update", "all"] and
+            pending_service_update|bool
+
+    - include: roles/ardana-upgrade-tools/tasks/check-pending-system-reboot.yml
+      when: update_status_var in ["system-reboot", "all"]
+
+    - name: _ardana-update-status | Add pending system reboot status to status list
+      set_fact:
+        update_status: "{{ update_status +
+          [update_status_brief | bool | ternary('system reboot', update_status_msg)] }}"
+      when: update_status_var in ["system-reboot", "all"] and
+            pending_system_reboot|bool
+
+    - name: _ardana-update-status | Report brief update status
+      debug:
+        msg: "Pending update actions: {{ update_status|join(', ') }}"
+      when: update_status_brief|bool
+
+    - name: _ardana-update-status | Report update status
+      debug:
+        msg: |
+          =====================================================================
+          Update status for node {{ inventory_hostname }}:
+          =====================================================================
+          {% for status in update_status | ternary(update_status, [_default_update_status] ) %}
+          {{ status }}
+
+          {% endfor %}
+          =====================================================================
+      when: not update_status_brief|bool
diff --git a/ardana-reboot.yml b/ardana-reboot.yml
new file mode 100644
index 0000000..a5e1ccf
--- /dev/null
+++ b/ardana-reboot.yml
@@ -0,0 +1,67 @@
+#
+# (c) Copyright 2018 SUSE LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License"); you may
+# not use this file except in compliance with the License. You may obtain
+# a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations
+# under the License.
+#
+
+# Regather facts on all hosts in case of configuration changes that haven't
+# been reflected in the fact cache
+- include: ardana-refresh-facts.yml
+
+- include: pbstart.yml
+  vars:
+    playbook_name: "ardana-reboot.yml"
+
+# Prevents the playbook being run on multiple targets
+- hosts: resources
+  roles:
+    - ardana-upgrade-tools
+  tasks:
+  - include: roles/ardana-upgrade-tools/tasks/check-single-host.yml
+    vars:
+      playbook_name: "ardana-reboot.yml"
+
+- include: _ardana-pre-reboot.yml
+
+- hosts: resources
+  roles:
+    - ardana-upgrade-tools
+  tasks:
+    - include: roles/ardana-upgrade-tools/tasks/check-deployer.yml
+
+    # First reboot non-deployer nodes, if included
+    - include: roles/ardana-upgrade-tools/tasks/reboot.yml
+      when: not running_on_deployer
+
+    # Node reboot was completed successfully, reset the reboot status variable
+    - include: roles/ardana-upgrade-tools/tasks/check-pending-system-reboot.yml
+      vars:
+        update_status_reset: true
+      when: not running_on_deployer
+
+    # Next, reboot deployer node, if included
+    - name: ardana-reboot | Deployer node has to be rebooted manually
+      fail:
+        msg: "The deployer node needs to be rebooted manually.
+              After reboot, resume by running the post-reboot playbook:
+
+            cd ~/scratch/ansible/next/ardana/ansible
+            ansible-playbook -i hosts/verb_hosts _ardana-post-reboot.yml \
+              --limit {{ inventory_hostname }}"
+      when: running_on_deployer
+
+- include: _ardana-post-reboot.yml
+
+- include: pbfinish.yml
+  vars:
+    playbook_name: "ardana-reboot.yml"
diff --git a/ardana-update-pkgs.yml b/ardana-update-pkgs.yml
new file mode 100644
index 0000000..1d01df3
--- /dev/null
+++ b/ardana-update-pkgs.yml
@@ -0,0 +1,45 @@
+#
+# (c) Copyright 2018 SUSE LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License"); you may
+# not use this file except in compliance with the License. You may obtain
+# a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations
+# under the License.
+#
+
+# Regather facts on all hosts in case of configuration changes that haven't
+# been reflected in the fact cache
+- include: ardana-refresh-facts.yml
+
+- include: pbstart.yml
+  vars:
+    playbook_name: "ardana-update-pkgs.yml"
+
+# Prevents the playbook being run on multiple targets
+- hosts: resources
+  roles:
+    - ardana-upgrade-tools
+  tasks:
+  - include: roles/ardana-upgrade-tools/tasks/check-single-host.yml
+    vars:
+      playbook_name: "ardana-reboot.yml"
+
+- include: osconfig-status.yml
+
+- include: _ardana-update-base.yml
+  vars:
+    install_package_updates: true
+
+# Report update status
+- include: _ardana-update-status.yml
+
+- include: pbfinish.yml
+  vars:
+    playbook_name: "ardana-update-pkgs.yml"
diff --git a/ardana-update-status.yml b/ardana-update-status.yml
new file mode 100644
index 0000000..5c51cbe
--- /dev/null
+++ b/ardana-update-status.yml
@@ -0,0 +1,36 @@
+#
+# (c) Copyright 2018 SUSE LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License"); you may
+# not use this file except in compliance with the License. You may obtain
+# a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations
+# under the License.
+#
+# This playbook can be used to handle special ardana_notify_... variables
+# associated with outstanding update actions resulted from running the
+# update playbooks.
+#
+# The following parameters are available:
+#
+#   * update_status_var: used to select a particular update status variable.
+#     It can take the following values:
+#
+#       service-update - target the status of pending service updates
+#       clm-update - target the status of pending CLM updates
+#       system-reboot - target the status of pending system reboots
+#       all (default) - target all supported update status variables
+#
+#   * update_status_check (default: true): check if the status variable is set
+#   * update_status_reset (default: false): reset the status variable
+#   * update_status_set (default: false): set the status variable
+#   * update_status_brief (default: false): use a brief update status output
+#
+---
+- include: _ardana-update-status.yml
\ No newline at end of file
diff --git a/library/rpm_list.py b/library/rpm_list.py
index adc1696..4413293 100644
--- a/library/rpm_list.py
+++ b/library/rpm_list.py
@@ -37,7 +37,7 @@ import subprocess
 
 def get_installed_pkgs(module):
     rpm_query_bin = module.get_bin_path('rpm')
-    cmd = [rpm_query_bin, '-qa', "--qf", "%{NAME} %{VERSION}\n"]
+    cmd = [rpm_query_bin, '-qa', "--qf", "%{NAME} %{VERSION}-%{RELEASE}\n"]
     output = subprocess.check_output(cmd).splitlines()
     installed_pkgs = {}
 
diff --git a/roles/ardana-upgrade-tools/defaults/main.yml b/roles/ardana-upgrade-tools/defaults/main.yml
index 1638e87..9cb5fa9 100644
--- a/roles/ardana-upgrade-tools/defaults/main.yml
+++ b/roles/ardana-upgrade-tools/defaults/main.yml
@@ -1,6 +1,6 @@
 #
 # (c) Copyright 2016 Hewlett Packard Enterprise Development LP
-# (c) Copyright 2017 SUSE LLC
+# (c) Copyright 2017-2018 SUSE LLC
 #
 # Licensed under the Apache License, Version 2.0 (the "License"); you may
 # not use this file except in compliance with the License. You may obtain
@@ -17,3 +17,15 @@
 ---
 
 list_pkg_upgrades_base: []
+
+zypper_update_licenses_agree: false
+zypper_update_gpg_checks: true
+zypper_update_include_reboot_patches: false
+zypper_update_method: patch
+
+running_on_deployer: false
+
+skip_single_host_checks: false
+skip_clm_updates_checks: false
+
+ardanauser_home: "{{ ansible_env['HOME'] }}"
\ No newline at end of file
diff --git a/roles/ardana-upgrade-tools/tasks/check-deployer.yml b/roles/ardana-upgrade-tools/tasks/check-deployer.yml
new file mode 100644
index 0000000..56945b7
--- /dev/null
+++ b/roles/ardana-upgrade-tools/tasks/check-deployer.yml
@@ -0,0 +1,21 @@
+#
+# (c) Copyright 2018 SUSE LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License"); you may
+# not use this file except in compliance with the License. You may obtain
+# a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations
+# under the License.
+#
+# Simple task to check whether the target host is the deployer
+---
+- name: ardana-upgrade-tools | check-deployer | Check if running on deployer
+  set_fact:
+    running_on_deployer: true
+  when: hostvars[inventory_hostname].ansible_hostname == hostvars['localhost'].ansible_hostname
diff --git a/roles/ardana-upgrade-tools/tasks/check-pending-clm-update.yml b/roles/ardana-upgrade-tools/tasks/check-pending-clm-update.yml
new file mode 100644
index 0000000..b2fecc1
--- /dev/null
+++ b/roles/ardana-upgrade-tools/tasks/check-pending-clm-update.yml
@@ -0,0 +1,105 @@
+#
+# (c) Copyright 2018 SUSE LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License"); you may
+# not use this file except in compliance with the License. You may obtain
+# a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations
+# under the License.
+#
+# Simple task to handle the special ardana_notify_pending_clm_update variable
+# that encodes whether CLM packages have been updated during one of the
+# previous update playbook runs.
+#
+# Parameters:
+#
+#  update_status_reset (default: false): reset the status variable
+#  update_status_set (default: false): set the status variable
+#
+# Return:
+#
+#  The pending_clm_update fact is set to reflect the value of the status
+#  variable.
+#  The update_status_msg fact is set to an output message describing the
+#  pending action associated with the status variable.
+
+---
+
+- name: ardana-upgrade-tools | check-pending-clm-update |
+        Initialize pending CLM update action message
+  set_fact:
+    pending_clm_update_msg: |
+      WARNING: outstanding installed CLM package updates were detected !
+
+      Before continuing with the update process, the CLM itself needs to
+      be updated by running 'ardana-init' on the deployer node followed
+      by running the recommended CLM update playbooks, i.e.:
+
+        cd ~/openstack/ardana/ansible
+        ansible-playbook -i hosts/localhost cobbler-deploy.yml
+        ansible-playbook -i hosts/localhost config-processor-run.yml
+        ansible-playbook -i hosts/localhost ready-deployment.yml
+
+      Then the update can be resumed by running the same playbook again,
+      with the same parameters.
+
+      To override this check, set the SKIP_UPDATE_CLM_CHECKS environment
+      variable or the skip_update_clm_checks ansible variable.
+
+      To recheck the pending CLM update status at a later time, run:
+
+        cd ~/scratch/ansible/next/ardana/ansible
+        ansible-playbook -i hosts/verb_hosts ardana-update-status.yml \
+          --limit {{ inventory_hostname }} \
+          -e update_status_var=clm-update
+
+      To reset the pending CLM update status, run:
+
+        cd ~/scratch/ansible/next/ardana/ansible
+        ansible-playbook -i hosts/verb_hosts ardana-update-status.yml \
+          --limit {{ inventory_hostname }} \
+          -e update_status_var=clm-update \
+          -e update_status_reset=true
+
+- name: ardana-upgrade-tools | check-pending-clm-update |
+        Check pending CLM update status
+  set_fact:
+    pending_clm_update: "{{
+      pending_clm_update | default(true) | bool and
+      ardana_notify_pending_clm_update is defined and
+      ardana_notify_pending_clm_update.changed and
+      (not skip_update_clm_checks | default(false) | bool) and
+      (not lookup('env', 'SKIP_UPDATE_CLM_CHECKS')|default(false, true))}}"
+
+- name: ardana-upgrade-tools | check-pending-clm-update |
+        Reset pending CLM update persistent fact
+  command: /bin/true
+  changed_when: true
+  register: _pending_clm_update_reset
+  when: (update_status_reset | default(false) | bool) and
+        ardana_notify_pending_clm_update is defined and
+        ardana_notify_pending_clm_update.changed
+
+- name: ardana-upgrade-tools | check-pending-clm-update |
+        Reset pending CLM update cached fact
+  set_fact:
+    pending_clm_update: false
+  when: _pending_clm_update_reset | changed
+
+- name: ardana-upgrade-tools | check-pending-clm-update |
+        Set pending CLM update cached/persistent fact
+  set_fact:
+    pending_clm_update: true
+  changed_when: true
+  register: ardana_notify_pending_clm_update
+  when: (update_status_set | default(false) | bool)
+
+- name: ardana-upgrade-tools | check-pending-clm-update | Set update action
+  set_fact:
+    update_status_msg: "{{ pending_clm_update|bool|ternary(pending_clm_update_msg,'') }}"
diff --git a/roles/ardana-upgrade-tools/tasks/check-pending-service-update.yml b/roles/ardana-upgrade-tools/tasks/check-pending-service-update.yml
new file mode 100644
index 0000000..8d7e9c3
--- /dev/null
+++ b/roles/ardana-upgrade-tools/tasks/check-pending-service-update.yml
@@ -0,0 +1,100 @@
+#
+# (c) Copyright 2018 SUSE LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License"); you may
+# not use this file except in compliance with the License. You may obtain
+# a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations
+# under the License.
+#
+# Simple task to handle the special ardana_notify_pending_service_update
+# variable that encodes whether services need to be updated because package
+# updates have been installed during one of the previous update playbook runs.
+#
+#  update_status_reset (default: false): reset the status variable
+#  update_status_set (default: false): set the status variable
+#
+# Return:
+#
+#  The pending_service_update fact is set to reflect the value of the status
+#  variable.
+#  The update_status_msg fact is set to an output message describing the
+#  pending action associated with the status variable.
+
+---
+- name: ardana-upgrade-tools | check-pending-service-update |
+        Initialize pending service update action message
+  set_fact:
+    pending_service_update_msg: |
+      WARNING: packages updates have been installed on the
+      {{ inventory_hostname }} host during this run or there are outstanding
+      update actions persisted from previous unsuccessful or incomplete update
+      runs.
+
+      The update process process must be continued by running the
+      ardana-update.yml playbook to update all services, e.g.:
+
+        cd ~/scratch/ansible/next/ardana/ansible
+        ansible-playbook -i hosts/verb_hosts ardana-update.yml \
+          --limit {{ inventory_hostname }}
+
+      To override this check, set the SKIP_UPDATE_SERVICE_CHECKS environment
+      variable or the skip_update_service_checks ansible variable.
+
+      To recheck the pending service update status at a later time, run:
+
+        cd ~/scratch/ansible/next/ardana/ansible
+        ansible-playbook -i hosts/verb_hosts ardana-update-status.yml \
+          --limit {{ inventory_hostname }} \
+          -e update_status_var=service-update
+
+      To reset the pending service update status, run:
+
+        cd ~/scratch/ansible/next/ardana/ansible
+        ansible-playbook -i hosts/verb_hosts ardana-update-status.yml \
+          --limit {{ inventory_hostname }} \
+          -e update_status_var=service-update \
+          -e update_status_reset=true
+
+- name: ardana-upgrade-tools | check-pending-service-update |
+        Check pending service update status
+  set_fact:
+    pending_service_update: "{{
+      pending_service_update | default(true) | bool and
+      ardana_notify_pending_service_update is defined and
+      ardana_notify_pending_service_update.changed and
+      (not skip_update_service_checks | default(false) | bool) and
+      (not lookup('env', 'SKIP_UPDATE_SERVICE_CHECKS')|default(false, true))}}"
+
+- name: ardana-upgrade-tools | check-pending-service-update |
+        Reset pending service update persistent fact
+  command: /bin/true
+  changed_when: true
+  register: _pending_service_update_reset
+  when: (update_status_reset | default(false) | bool) and
+        ardana_notify_pending_service_update is defined and
+        ardana_notify_pending_service_update.changed
+
+- name: ardana-upgrade-tools | check-pending-service-update |
+        Reset pending service update cached fact
+  set_fact:
+    pending_service_update: false
+  when: _pending_service_update_reset | changed
+
+- name: ardana-upgrade-tools | check-pending-service-update |
+        Set pending service update cached/persistent fact
+  set_fact:
+    pending_service_update: true
+  changed_when: true
+  register: ardana_notify_pending_service_update
+  when: (update_status_set | default(false) | bool)
+
+- name: ardana-upgrade-tools | check-pending-service-update | Set update action
+  set_fact:
+    update_status_msg: "{{ pending_service_update|bool|ternary(pending_service_update_msg,'') }}"
diff --git a/roles/ardana-upgrade-tools/tasks/check-pending-system-reboot.yml b/roles/ardana-upgrade-tools/tasks/check-pending-system-reboot.yml
new file mode 100644
index 0000000..21578cf
--- /dev/null
+++ b/roles/ardana-upgrade-tools/tasks/check-pending-system-reboot.yml
@@ -0,0 +1,102 @@
+#
+# (c) Copyright 2018 SUSE LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License"); you may
+# not use this file except in compliance with the License. You may obtain
+# a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations
+# under the License.
+#
+# Simple task to handle the special ardana_notify_pending_system_reboot that
+# encodes whether a system reboot is required after packages have been updated
+# during one of the previous update playbook runs.
+#
+# Parameters:
+#
+#  update_status_reset (default: false): reset the status variable
+#  update_status_set (default: false): set the status variable
+#
+# Return:
+#
+#  The pending_system_reboot fact is set to reflect the value of the status
+#  variable.
+#  The update_status_msg fact is set to an output message describing the
+#  pending action associated with the status variable.
+
+---
+- name: ardana-upgrade-tools | check-pending-system-reboot |
+        Initialize pending system reboot action message
+  set_fact:
+    pending_system_reboot_msg: |
+      WARNING: Package updates installed on the {{ inventory_hostname }}
+      host during this run require a system reboot, or there are outstanding
+      reboot actions persisted from previous unsuccessful or incomplete update
+      or reboot runs.
+
+      The node should be rebooted to complete the update process by running the
+      ardana-reboot.yml playbook, e.g.:
+
+        cd ~/scratch/ansible/next/ardana/ansible
+        ansible-playbook -i hosts/verb_hosts ardana-reboot.yml \
+          --limit {{ inventory_hostname }}
+
+      To override this check, set the SKIP_UPDATE_REBOOT_CHECKS environment
+      variable or the skip_update_reboot_checks ansible variable.
+
+      To recheck the pending system reboot status at a later time, run:
+
+        cd ~/scratch/ansible/next/ardana/ansible
+        ansible-playbook -i hosts/verb_hosts ardana-update-status.yml \
+          --limit {{ inventory_hostname }} \
+          -e update_status_var=system-reboot
+
+      To reset the pending system reboot status, run:
+
+        cd ~/scratch/ansible/next/ardana/ansible
+        ansible-playbook -i hosts/verb_hosts ardana-update-status.yml \
+          --limit {{ inventory_hostname }} \
+          -e update_status_var=system-reboot \
+          -e update_status_reset=true
+
+- name: ardana-upgrade-tools | check-pending-system-reboot |
+        Check pending system reboot status
+  set_fact:
+    pending_system_reboot: "{{
+      pending_system_reboot | default(true) | bool and
+      ardana_notify_pending_system_reboot is defined and
+      ardana_notify_pending_system_reboot.changed and
+      (not skip_update_reboot_checks | default(false) | bool) and
+      (not lookup('env', 'SKIP_UPDATE_REBOOT_CHECKS')|default(false, true))}}"
+
+- name: ardana-upgrade-tools | check-pending-system-reboot |
+        Reset pending system reboot persistent fact
+  command: /bin/true
+  changed_when: true
+  register: _pending_system_reboot_reset
+  when: (update_status_reset | default(false) | bool) and
+        ardana_notify_pending_system_reboot is defined and
+        ardana_notify_pending_system_reboot.changed
+
+- name: ardana-upgrade-tools | check-pending-system-reboot |
+        Reset pending system reboot cached fact
+  set_fact:
+    pending_system_reboot: false
+  when: _pending_system_reboot_reset | changed
+
+- name: ardana-upgrade-tools | check-pending-system-reboot |
+        Set pending system reboot cached/persistent fact
+  set_fact:
+    pending_system_reboot: true
+  changed_when: true
+  register: ardana_notify_pending_system_reboot
+  when: (update_status_set | default(false) | bool)
+
+- name: ardana-upgrade-tools | check-pending-system-reboot | Set update action
+  set_fact:
+    update_status_msg: "{{ pending_system_reboot|bool|ternary(pending_system_reboot_msg,'') }}"
diff --git a/roles/ardana-upgrade-tools/tasks/check-single-host.yml b/roles/ardana-upgrade-tools/tasks/check-single-host.yml
new file mode 100644
index 0000000..25b23f6
--- /dev/null
+++ b/roles/ardana-upgrade-tools/tasks/check-single-host.yml
@@ -0,0 +1,28 @@
+#
+# (c) Copyright 2018 SUSE LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License"); you may
+# not use this file except in compliance with the License. You may obtain
+# a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations
+# under the License.
+#
+# Simple task to check whether the playbook is running on a single hosts
+---
+
+- name: ardana-upgrade-tools | check-single-host | Check if running on a single host
+  fail:
+    msg: |
+      It is not recommended to run {{ playbook_name }} on multiple hosts.
+      To override this check, set the SKIP_SINGLE_HOST_CHECKS environment variable
+      or the skip_single_host_checks ansible variable"
+  run_once: true
+  when: (play_hosts | length) > 1 and
+        not skip_single_host_checks and
+        (not lookup('env', 'SKIP_SINGLE_HOST_CHECKS')|default(false, true))
diff --git a/roles/ardana-upgrade-tools/tasks/pkg-update.yml b/roles/ardana-upgrade-tools/tasks/pkg-update.yml
new file mode 100644
index 0000000..4eda963
--- /dev/null
+++ b/roles/ardana-upgrade-tools/tasks/pkg-update.yml
@@ -0,0 +1,217 @@
+#
+# (c) Copyright 2018 SUSE LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License"); you may
+# not use this file except in compliance with the License. You may obtain
+# a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations
+# under the License.
+#
+# Install available zypper package updates
+#
+# The behavior can be customized with the following parameters:
+#
+#   * zypper_update_method: define which zypper subcommand to use for
+#     updating. Valid values are:
+#
+#     'patch' (default) - will install all patches applying to
+#     the system from the configured update repositories that are available.
+#
+#     'update' - will update packages from all configured repositories
+#     (not just the update repositories) that have a higher version
+#     number than the installed packages.
+#
+#     'dist-upgrade' - replaces each package installed with the version
+#     from the repository and deletes packages not available in the
+#     repositories.
+#
+#   * zypper_update_gpg_checks (default: true): enable GPG checks.
+#     If set to true, checks if packages are correctly signed.
+#
+#   * zypper_update_licenses_agree (default: false): automatically agree
+#     with licenses. If set to true, zypper automatically accepts third
+#     party licenses.
+#
+#   * zypper_update_include_reboot_patches (default: false): include patches
+#     that need reboots (e.g. kernel). Installs patches that require a reboot
+#     (for example kernel or glibc updates).
+#
+# Sets the following facts:
+#
+#   * ardana_upgraded_pkgs - list of upgraded packages
+#   * ardana_notify_pending_service_update (persistent) - set when at least one
+#     package has been updated, to indicate that services need to be updated
+#   * ardana_notify_pending_clm_update (persistent) - this flag is set when at
+#     least one of the packages used by the CLM itself (e.g. playbook packages)
+#     has been updated
+#   * ardana_notify_pending_venv_update (persistent) - this flag is set when at
+#     least one of the venv packages has been updated
+#   * ardana_notify_pending_system_reboot (persistent) - this flag is set if a
+#     system reboot is required after the installed package updates
+---
+
+- name: ardana-upgrade-tools | pkg-update | Validate update method
+  fail:
+    msg:
+      "{{ zypper_update_method }} is not a valid zypper update method."
+  when: zypper_update_method not in ["patch", "update", "dist-upgrade"]
+
+- name: ardana-upgrade-tools | pkg-update | Generate zypper update command args
+  set_fact:
+    _zypper_args: >
+      {%- set _args = ["--non-interactive"] -%}
+      {%- if not zypper_update_gpg_checks | bool -%}
+      {%-   set _ = _args.append("--no-gpg-checks") -%}
+      {%- endif -%}
+      {%- if zypper_update_method == "patch" and zypper_update_include_reboot_patches | bool -%}
+      {%-   set _ = _args.append("--non-interactive-include-reboot-patches") -%}
+      {%- endif -%}
+      {%- set _ = _args.append(zypper_update_method) -%}
+      {%- if zypper_update_licenses_agree | bool -%}
+      {%-   set _ = _args.append("--auto-agree-with-licenses") -%}
+      {%- endif -%}
+      {{ _args }}
+
+- name: ardana-upgrade-tools | pkg-update | Check if the PTF repository is configured
+  become: yes
+  shell: zypper lr PTF
+  register: _ptf_repo_result
+  ignore_errors: true
+
+- name: ardana-upgrade-tools | pkg-update | Refresh PTF repository
+  become: yes
+  command: zypper --non-interactive --gpg-auto-import-keys refresh -fr PTF
+  when: _ptf_repo_result | success
+
+- name: ardana-upgrade-tools | pkg-update | Get set of installed pkgs before update
+  rpm_list:
+
+- name: ardana-upgrade-tools | pkg-update  | Set _pre_update_pkgs fact
+  set_fact:
+    _pre_update_pkgs: "{{ installed_pkgs | default([]) }}"
+
+# Sometimes zypper needs to go through several iterations when installing
+# patches, e.g. when those patches include updates to the zypper packages
+# themselves. This will be indicated by a 103 return code.
+- name: ardana-upgrade-tools | pkg-update | Download and install package updates
+  become: yes
+  command: zypper {{ _zypper_args | join(' ') }} --download in-advance
+  register: zypper_update_result
+  until: zypper_update_result.rc != 103 # ZYPPER_EXIT_INF_RESTART_NEEDED
+  retries: 5
+  ignore_errors: yes
+
+- name: ardana-upgrade-tools | pkg-update | Get set of installed pkgs after update
+  rpm_list:
+
+- name: _ardana-update-base | Set _post_update_pkgs fact
+  set_fact:
+    _post_update_pkgs: "{{ installed_pkgs | default([]) }}"
+
+- name: ardana-upgrade-tools | pkg-update | Set fact with full set of updated pkgs
+  set_fact:
+    ardana_upgraded_pkgs: "{{ _pre_update_pkgs | upgraded_pkgs(_post_update_pkgs) }}"
+
+- name: ardana-upgrade-tools | pkg-update | Print updated packages
+  debug:
+    msg: "The following packages have been updated: {{ ardana_upgraded_pkgs.keys() | join(', ') }}"
+  when: ardana_upgraded_pkgs
+
+- name: ardana-upgrade-tools | pkg-update | Get set of installed CLM pkgs
+  shell: rpm -qa --qf "%{NAME}\n" | grep ardana
+  register: _ardana_clm_packages
+  when: running_on_deployer
+
+- name: ardana-upgrade-tools | pkg-update | Get set of installed venv pkgs
+  command: rpm -qf /opt/ardana_packager --qf "%{NAME}\n"
+  register: _ardana_venv_packages
+  when: running_on_deployer
+
+- name: ardana-upgrade-tools | pkg-update | Check if CLM packages were updated
+  set_fact:
+    _clm_packages_updated: true
+  when: running_on_deployer and
+        item in ardana_upgraded_pkgs | default({})
+  with_items: _ardana_clm_packages.stdout_lines | default([])
+
+- name: ardana-upgrade-tools | pkg-update | Check if venv packages were updated
+  set_fact:
+    _venv_packages_updated: true
+  when: running_on_deployer and
+        item in ardana_upgraded_pkgs | default({})
+  with_items: _ardana_venv_packages.stdout_lines | default([])
+
+- name: ardana-upgrade-tools | pkg-update | Check if system packages were updated
+  set_fact:
+    _system_packages_updated: true
+  when: item.key not in _ardana_clm_packages.stdout_lines | default([]) and
+        item.key not in _ardana_venv_packages.stdout_lines | default([])
+  with_dict: ardana_upgraded_pkgs | default({})
+
+  # Register a persistent pending CLM update action if
+  # CLM packages were updated
+- include: check-pending-clm-update.yml
+  vars:
+    update_status_set: true
+  when: _clm_packages_updated | default(false) | bool
+
+  # Register a persistent pending service update action for this node if
+  # any packages were updated
+- include: check-pending-service-update.yml
+  vars:
+    update_status_set: true
+  when: ardana_upgraded_pkgs
+
+  # Register a persistent pending service update action for all nodes if
+  # CLM or venv packages were updated
+  # TODO(snica): this isn't working quite right (hosts are iterated, but the
+  # condition is always evaluated as false)
+- name: ardana-upgrade-tools | pkg-update |
+        Register a pending service update action for all other nodes
+  command: /bin/true
+  changed_when: true
+  register: ardana_notify_pending_service_update
+  delegate_to: "{{ item }}"
+  when: running_on_deployer and
+        (_clm_packages_updated | default(false) | bool or
+        _venv_packages_updated | default(false) | bool)
+  with_items: groups.resources
+
+  # Register a persistent pending system reboot action for this node if
+  # installed package updates require it.
+  #
+  # TODO (snica) for now we also require that a node be rebooted if system packages
+  # were updated on it (i.e. anything other than CLM and venv packages).
+  # This can be replaced with a node-wide service restart when
+  # https://gerrit.suse.provo.cloud/3745/ is merged.
+- include: check-pending-system-reboot.yml
+  vars:
+    update_status_set: true
+  when: _system_packages_updated | default(false) | bool or
+        zypper_update_result.rc == 102 # ZYPPER_EXIT_INF_REBOOT_NEEDED
+
+  # We deferred from checking the zypper update command status up to this
+  # point to get a chance to record any pending update actions as persistent facts
+- name: ardana-upgrade-tools | pkg-update | Check update result
+  fail:
+    msg:
+      "zypper {{ zypper_update_method }} command returned {{ zypper_update_result.rc }}."
+  when: zypper_update_result.rc not in [0, 100, 101, 102, 103, 104, 105]
+
+- name: ardana-upgrade-tools | pkg-update | Check number of zypper restart cycles
+  fail:
+    msg:
+      "Ran zypper {{ zypper_update_method }} more than five times,
+      and it still requires more runs."
+  when: zypper_update_result.rc == 103 # ZYPPER_EXIT_INF_RESTART_NEEDED
+
+- name: ardana-upgrade-tools | pkg-update | Check if reboot is required
+  command: /bin/true
+  register: ardana_notify_pending_system_reboot
+  when: zypper_update_result.rc == 102 # ZYPPER_EXIT_INF_REBOOT_NEEDED
diff --git a/roles/ardana-upgrade-tools/tasks/reboot.yml b/roles/ardana-upgrade-tools/tasks/reboot.yml
new file mode 100644
index 0000000..56acd03
--- /dev/null
+++ b/roles/ardana-upgrade-tools/tasks/reboot.yml
@@ -0,0 +1,32 @@
+#
+# (c) Copyright 2018 SUSE LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License"); you may
+# not use this file except in compliance with the License. You may obtain
+# a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations
+# under the License.
+#
+---
+
+- name: ardana-upgrade-tools | install | Reboot node
+  become: yes
+  shell: sleep 2 && shutdown -r now
+  async: 1
+  poll: 0
+
+- name: ardana-upgrade-tools | install | Wait for node to come back up
+  delegate_to: localhost
+  wait_for:
+    host: "{{ ansible_ssh_host | default(inventory_hostname) }}"
+    port: 22
+    search_regex: OpenSSH
+    state: started
+    delay: 30
+    timeout: 600
-- 
2.16.3

Places

File 0001-SCRD-3016-maintenance-update-workflow-helper-playboo.patch of Package ardana-ansible (Revision e4a2bba0aa8198fec711773bceb0bea3)

Places