initial role features

README.md

@@ -0,0 +1,42 @@
+# manageiq\_workers
+Ansible role for configuring the workers on ManageIQ / CloudForms Management Engine (CFME) appliances.
+
+## Role Variables
+Information about the expected role parameters.
+
+See [Summary of Roles, Workers and Messages](https://access.redhat.com/documentation/en-us/reference_architectures/2017/html/deploying_cloudforms_at_scale/architecture#summary_of_roles_workers_and_messages) for details on the different workers and what they are for.
+
+The _Workers Tab_ colume describes which box on the Workers tab of the Advanced appliance configuration page the value is settable from. If not `Advanced` then the value is only settable via Advanced settings.
+
+| parameter                                                    | required | default       | choices         | appliance workers tab  | comments 
+| ------------------------------------------------------------ |:--------:|:-------------:| --------------- | ---------------------- |:-------- 
+| manageiq\_workers\_validate\_parameters                      | No       | True          | True, False     | N/A                    | True to enable role parameter validation based on what the ManageIQ / CFME UI allows to be configured. False to disable validation and allow the setting of values that the ManageIQ / CFME does not allow users to configure.
+| manageiq\_generic\_worker\_memory\_threshold                 | No       | 600.megabytes |                 | Generic Workers        |
+| manageiq\_generic\_worker\_thread\_count                     | No       | 4             |                 | Generic Workers        |
+| manageiq\_priority\_worker\_memory\_threshold                | No       | 600.megabytes |                 | Priorty Workers        |
+| manageiq\_priority\_worker\_thread\_count                    | No       | 4             |                 | Priorty Workers        |
+| manageiq\_c\_and\_u\_data\_collector\_memory\_threshold      | No       | 400.megabytes |                 | C & U Data Collectors  |
+| manageiq\_c\_and\_u\_data\_collector\_thread\_count          | No       | 2             |                 | C & U Data Collectors  |
+| manageiq\_c\_and\_u\_data\_processor\_memory\_threshold      | No       | 500.megabytes |                 | C & U Data Processors  |
+| manageiq\_c\_and\_u\_data\_processor\_thread\_count          | No       | 2             |                 | C & U Data Processors  |
+| manageiq\_event\_monitor\_memory\_threshold                  | No       | 2.gigabytes   |                 | Event Monitor          |
+| manageiq\_refresh\_worker                                    | No       | 2.gigabytes   |                 | Refresh                |
+| manageiq\_connection\_broker\_memory\_threshold              | No       | 2.gigabytes   |                 | Connection Broker      |
+| manageiq\_vm\_analysis\_collector\_memory\_threshold         | No       | 600.megabytes |                 | VM Analysis Collectors |
+| manageiq\_vm\_analysis\_collector\_thread\_count             | No       | 2             |                 | VM Analysis Collectors |
+| manageiq\_ui\_worker\_thread\_count                          | No       | 4             |                 | UI Worker              |
+| manageiq\_websocket\_worker\_thread\_count                   | No       | 1             |                 | Websocket Workers      |
+| manageiq\_reporting\_worker\_memory\_threshold               | No       | 500.megabytes |                 | Reporting Workers      |
+| manageiq\_reporting\_worker\_thread\_count                   | No       | 2             |                 | Reproting Workers      |
+| manageiq\_web\_service\_worker\_memory\_threshold            | No       | 1.gigabytes   |                 | Web Service Workers    |
+| manageiq\_web\_service\_worker\_thread\_count                | No       | 1             |                 | Web Service Workers    |
+| manageiq\_default\_memory\_threshold                         | No       | 400.megabytes |                 | Advanced               |
+| manageiq\_default\_thread\_count                             | No       | 1             |                 | Advanced               |
+| manageiq\_ansible\_memory\_threshold                         | No       | 0.megabytes   |                 | Advanced               |
+| manageiq\_ui\_worker\_memory\_threshold                      | No       | 1.gigabytes   |                 | Advanced               |
+| manageiq\_websocket\_worker\_memory\_threshold               | No       | 1.gigabytes   |                 | Advanced               |
+| manageiq\_vmdb\_storage\_bridge\_worker\_memory\_threshold   | No       | 2.gigabytes   |                 | Advanced               |
+| manageiq\_schedule\_worker\_memory\_threshold                | No       | 500.megabytes |                 | Advanced               |
+| manageiq\_netapp\_refresh\_worker\_memory\_threshold         | No       | 2.gigabytes   |                 | Advanced               |
+| manageiq\_sims\_refresh\_worker\_memory\_threshold           | No       | 1.gigabytes   |                 | Advanced               |
+| manageiq\_storage\_metrics\_collector\_worker\_thread\_count | No       | 2             |                 | Advanced               |

defaults/main.yml

@@ -0,0 +1,31 @@
+---
+manageiq_workers_validate_parameters: True
+manageiq_default_memory_threshold: "400.megabytes"
+manageiq_default_thread_count: 1
+manageiq_ansible_memory_threshold: "0.megabytes"
+manageiq_event_monitor_memory_threshold: "2.gigabytes"
+manageiq_c_and_u_data_collector_memory_threshold: "400.megabytes"
+manageiq_c_and_u_data_collector_thread_count: 2
+manageiq_c_and_u_data_processor_memory_threshold: "500.megabytes"
+manageiq_c_and_u_data_processor_thread_count: 2
+manageiq_generic_worker_memory_threshold: "600.megabytes"
+manageiq_generic_worker_thread_count: 4
+manageiq_priority_worker_memory_threshold: "600.megabytes"
+manageiq_priority_worker_thread_count: 4
+manageiq_reporting_worker_memory_threshold: "500.megabytes"
+manageiq_reporting_worker_thread_count: 2
+manageiq_ui_worker_memory_threshold: "1.gigabytes"
+manageiq_ui_worker_thread_count: 4
+manageiq_connection_broker_memory_threshold: "2.gigabytes"
+manageiq_web_service_worker_memory_threshold: "1.gigabytes"
+manageiq_web_service_worker_thread_count: 1
+manageiq_websocket_worker_memory_threshold: "1.gigabytes"
+manageiq_websocket_worker_thread_count: 1
+manageiq_vm_analysis_collector_memory_threshold: "600.megabytes"
+manageiq_vm_analysis_collector_thread_count: 2
+manageiq_vmdb_storage_bridge_worker_memory_threshold: "2.gigabytes"
+manageiq_schedule_worker_memory_threshold: "500.megabytes"
+manageiq_netapp_refresh_worker_memory_threshold: "2.gigabytes"
+manageiq_refresh_worker: "2.gigabytes"
+manageiq_sims_refresh_worker_memory_threshold: "1.gigabytes"
+manageiq_storage_metrics_collector_worker_thread_count: 2

library/manageiq_config.py

@@ -0,0 +1,229 @@
+#!/usr/bin/python
+# -*- coding: utf-8 -*-
+# (c) 2018, Ian Tewksbury <itewk@redhat.com>
+# GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
+
+from __future__ import absolute_import, division, print_function
+__metaclass__ = type
+
+ANSIBLE_METADATA = {
+    'metadata_version': '1.1',
+    'status': ['preview'],
+    'supported_by': 'community'
+}
+
+DOCUMENTATION = '''
+---
+module: manageiq_config
+
+short_description: Module for managing ManageIQ configuration.
+version_added: '2.5'
+author: Ian Tewksbury (@itewk)
+description:
+    - "Module for managing ManageIQ or CloudForms Managment Engine (CFME) configuration via the local rails runner on the Appliance."
+
+options:
+    name:
+        description:
+            - Name of the ManageIQ config element to modify.
+        required: True
+    value:
+        description: Dictionary to set as the value for the ManageIQ config option. Any values not given will be left at current value.
+        required: False
+        default: {}
+    vmdb_path:
+        description: Path to the VMDB directory.
+        required: False
+        default: /var/www/miq/vmdb
+    confirm_update_max_retries:
+        description: Number of times to attempt to confirm configuration has been updated.
+        required: False
+        default: 10
+    confirm_update_sleep_interval:
+        description: Number of seconds between retries for confirming configuration has been updated.
+        required: False
+        default: 1
+'''
+
+EXAMPLES = '''
+# set the smtp settings
+- manageiq_config:
+    name: smtp
+    value:
+      from: cfme@example.com
+      host: postfix.example.com
+      port: 25
+      domain: example.com
+
+# set generic worker memory threshold
+- manageiq_config:
+    name: workers
+    value:
+      worker_base:
+        queue_worker_base:
+          generic_worker:
+            memory_threshold: '900.megabytes'
+'''
+
+RETURN = '''
+name:
+    description: Name of the ManageIQ config option that is being modified.
+    type: str
+    returned: always
+original_value:
+    description: The origional value of the ManageIQ config option being modified before modification.
+    type: dict
+    returned: always
+value:
+    description: The value of the ManageIQ config option being modified after modification.
+    type: dict
+    returned: always
+'''
+
+from ansible.module_utils.basic import AnsibleModule
+from ansible.module_utils.basic import json
+import time
+import collections
+import copy
+
+
+# @source https://gist.github.com/angstwad/bf22d1822c38a92ec0a9
+def dict_merge(dct, merge_dct):
+    """ Recursive dict merge. Inspired by :meth:``dict.update()``, instead of
+    updating only top-level keys, dict_merge recurses down into dicts nested
+    to an arbitrary depth, updating keys. The ``merge_dct`` is merged into
+    ``dct``.
+    :param dct: dict onto which the merge is executed
+    :param merge_dct: dct merged into dct
+    :return: None
+    """
+    for k, v in merge_dct.items():
+        if (k in dct and isinstance(dct[k], dict)
+                and isinstance(merge_dct[k], collections.Mapping)):
+            dict_merge(dct[k], merge_dct[k])
+        else:
+            dct[k] = merge_dct[k]
+
+
+def get_manageiq_config_value(module, name):
+    """ Gets the current value for the given config option.
+    :param module: AnsibleModule making the call
+    :param name: ManageIQ config option name
+    :return: dict of value of the ManageIQ config option
+    """
+    returncode, out, err = module.run_command([
+        "rails",
+        "r",
+        "puts MiqServer.my_server.get_config(:%s).config.to_json" % (name)
+    ], cwd=module.params['vmdb_path'])
+    if returncode != 0:
+        raise Exception("Error getting existing value for ':%s' config: %s" % (name, err))
+
+    return json.loads(out)
+
+
+def wait_for_manageiq_config_value_to_be_expected(module, expected_value):
+    """ Waits until the given expected_value equals the current from ManageIQ.
+    :param module: AnsibleModule making the call
+    :param expected_value: The expected value from ManageIQ config
+    :return: bool, true if expected value matches current value after retires, false otherwise.
+    """
+    # setting the configuration is an async operation which must be waited for for completion
+    # NOTE: if anyone has a better idea how to do this please share
+    retries = 0
+    current_value = ''
+    while expected_value != current_value and retries < module.params['confirm_update_max_retries']:
+        try:
+            current_value = get_manageiq_config_value(module, module.params['name'])
+        except Exception as err:
+            module.fail_json(msg=str(err))
+
+        retries += 1
+        time.sleep(module.params['confirm_update_sleep_interval'])
+
+    return expected_value == current_value
+
+
+def main():
+    # define the module
+    module = AnsibleModule(
+        argument_spec=dict(
+            name=dict(type='str', required=True),
+            value=dict(type='dict', required=False, default={}),
+            vmdb_path=dict(type='str', required=False, default='/var/www/miq/vmdb'),
+            confirm_update_max_retries=dict(type='int', required=False, default=10),
+            confirm_update_sleep_interval=dict(type='int', required=False, default=1)
+        ),
+        supports_check_mode=True
+    )
+
+    # seed the result dict in the object
+    result = dict(
+        changed=False,
+        name='',
+        original_value={},
+        value={},
+        diff={}
+    )
+
+    # get the origional value for the given config option
+    try:
+        original_value = get_manageiq_config_value(module, module.params['name'])
+    except Exception as err:
+        module.fail_json(msg=str(err), **result)
+
+    # update the result
+    result['name'] = module.params['name']
+    result['original_value'] = original_value
+
+    # create updated value dictionary
+    update_value = copy.deepcopy(original_value)
+    dict_merge(update_value, module.params['value'])
+
+    # enable diff mode
+    result['diff'] = {
+        'before': {
+            module.params['name']: original_value
+        },
+        'after': {
+            module.params['name']: update_value
+        }
+    }
+
+    # if check_mode then stop here
+    if module.check_mode:
+        result['changed'] = original_value != update_value
+        module.exit_json(**result)
+
+    # update config if difference
+    # else no-op
+    if original_value != update_value:
+        (update_value_rc, update_value_out, update_value_err) = module.run_command([
+            "rails",
+            "r",
+            "MiqServer.my_server.set_config(:%s => JSON.parse('%s'))" % (module.params['name'], json.dumps(update_value))
+        ], cwd=module.params['vmdb_path'])
+        result['changed'] = True
+        if update_value_rc != 0:
+            module.fail_json(msg="Error updating value for ':%s' config: %s" % (module.params['name'], update_value_err), **result)
+    else:
+        module.exit_json(**result)
+
+    # wait for update value to equal current value
+    current_matches_expected_value = wait_for_manageiq_config_value_to_be_expected(module, update_value)
+
+    # if current ManageIQ config value equals expected value then exit with success
+    # else exit with error
+    if current_matches_expected_value:
+        module.exit_json(**result)
+    else:
+        module.fail_json(
+            msg="Timed out waiting for set config to take affect. { 'retries': %s, 'sleep_interval': %s }" % (
+                module.params['confirm_update_max_retries'], module.params['confirm_update_sleep_interval']
+            ),
+            **result
+        )
+
+
+if __name__ == '__main__':
+    main()

meta/main.yml

@@ -0,0 +1,25 @@
+galaxy_info:
+  author: Ian Tewksbury
+  description: Ansible role for configuring the workers on ManageIQ / CloudForms Management Engine (CFME) appliances.
+  company: Red Hat, Inc.
+  license: license (Apache)
+  min_ansible_version: 2.4
+  platforms:
+  - name: EL
+    versions:
+      - 7
+  - galaxy_tags:
+      - manageiq
+      - cfme
+      - cloudforms
+      - redhat
+      - rhc
+      - tools
+  - categories:
+      - manageiq
+      - cfme
+      - cloudforms
+      - redhat
+      - rhc
+      - tools
+dependencies: []