Tag: automation

  • ServiceNow CMDB REST API Tutorial

    In this tutorial we are going to use a practical example to show you how to use the ServiceNow REST API. I personally find ServiceNow has great online documentation but I like to see some examples to understand how other people are using it. This is the motivation for this tutorial and for the previous one on ServiceNow incidents with REST API. Let’s get to it!

    Introduction

    The CMDB (Configuration Management Database) in ServiceNow is a key component that underpins multiple services. Most organizations nowadays have a requirement to automate services delivery in order to achieve greater agility and efficiency. If we are going to implement automation in ServiceNow sooner or later we need to deal with the CMDB and the way to do this is to use the REST API.

    In this tutorial we will explore this by using a Postman collection you can find in this GitHub repo. If you need code in a specific language, Postman can help you generate code for any of the API calls in the collection. The collection also contains REST API calls to manage Incidents in ServiceNow. These calls were used in a previous tutorial called “Creating ServiceNow Incidents via REST API” which showed some examples on how to address one of the most common tasks organizations are willing to automate in ServiceNow.

    If you are reading this more than likely you know what a CMDB now, so I won’t provide much detail here. The main thing to know is that the CMDB is organized in a large hierarchy of tables. At the top of the hierarchy we have the “cmdb” table and from that a single child called “cmdb_ci”. This last table is the parent for everything else. (services, applications, servers, networks, databases …).

    All objects in this database are called “Configuration Items” or “CI” for short. You can show all the items by typing “cmdb_ci.list” in the “Navigator”. My developer instance has more than 2800 CI’s. The “class” parameter tells us what type of CI it is and as you will see it is a very important detail when dealing with the REST API.

    The CMDB is accessed in the ServiceNow interface with the “configuration” application. Once you type “configuration” in the “Navigator” you can scroll down to see everything it contains.

    The REST API

    When working with a new REST API, the first step is to learn how to authenticate. In that regard, the ServiceNow REST API is straight forward as you can use basic authentication, ie username and password.

    There are two main tools available to learn what API calls are available: the online product documentation and the REST API Explorer. In the online documentation you have to find the “REST API Reference” and then scroll down to “CMDB Instance API“. This is publicly accessible. In the following screenshot you can see it contains 7 REST API calls that allow you to do a range of CRUD operations.

    Notice how the API calls have the “classname” URL parameter. This corresponds to the “class” we mentioned earlier. Every piece of information one would expect to find (parameters, headers, status codes …) is provided in this documentation

    The second tool is the “REST API Explorer”. This a a great tool that allows you to build your API calls in a graphical manner, including the “body” payload for POST /PUT/PATCH calls. I showed how to use it in the previous tutorial “Creating ServiceNow Incidents via REST API“. However, given the sheer amount of attributes available for all classes of CI’s I am going to suggest a different way of doing this

    Learning with a practical example

    Let’s say we have an automation script that creates and configures a Linux virtual machine and as part of the same script now we want to add an entry for the virtual machine in the CMDB. The following API call is the one we need to use in order to create the CI:

    POST /now/cmdb/instance/cmdb_ci_linux_server

    This POST call requires a request “body” parameter which can have a large number of attributes as well as inbound/outbound relation information, ie how this CI is related to other CI’s. To help us configure the “body” we are going to:

    • create manually a sample resource of the same class (ie. a Linux server in this case) and configure it the way we need it. It is important to configure every attribute we want to use
    • use a GET call for that resource. The response will serve you as a very good reference for the “body” of the POST call we want to automate

    We type “configuration” and scroll down to “Servers” and click on “Linux”. Once in “Linux Servers”, click “New” to create a new Linux server. I have filled in those fields that are relevant to my use case.

    Once ready, you can click submit and the new CI will be created.

    At this point, you might want to add information on how this CI is related to other CI’s. This will be very useful for example to see what services/applications are impacted if there is an incident on this CI. In my example let’s say I want to show that this Linux server depends on a storage volume. Follow these steps:

    • Open the CI you have just created
    • Scroll down to “Related Items”
    • On the right click the “+” symbol next to “Search for CI” to open the “Relationship Editor”

    This opens the “Relationship Editor” as seen below. Now you can select the type of relationship and use the filter tool to search for the CI it depends on, ie a storage volume in our example. Then tick the CI and click the “+” symbol to add the relationship

    In my case I have also created an additional relationship with an application named “Alberto Inventory app”. For the application I chose the “Used by” relationship, meaning that the application is the one that “depends on” the Linux server. In the resulting “Relationships” table below you can see both relationships and the parent/child relationship:

    Back in Linux server record we can see at a glance the resulting relationships in the “Related Items” section. Notice how this is showing an additional relationship “Used by – Alberto Inventory Service”. I didn’t explicitly declared this relationship but I had an existing relationship between this service and the application. So in the end the service also depends on this Linux server

    Now it is time to use Postman to see what the payload looks like. Download the collection and create 2 environment variables “pwd” and “instance” as instructed in the GitHub repo. At this point you can open the “Get CMDB Linux servers” call and click “Send”. This will return a list of Linux servers. But for each of them only the “sys_id” and the “name” are displayed.

    Copy the the “sys_id” of your newly created Linux server CI and open the “GET CMDB Linux server details” call in Postman. In the URL you can the “sys_id” to the end of the URL (see highlighted in yellow) and click “Send”

    This is now showing a very good approximation of the “body” parameter we were after including attributes as well as inbound/outbound relations. Notice though, how it is nested under “result”. Notice also how the JSON payload is showing all the attributes, including attributes were not displayed in the form when we created the resource. Any attribute we didn’t specify during CI creation will be empty. At this point you can make a note of any extra attributes you want to include in your payload.

    Now in Postman let’s open the “select the “POST Create CMDB Linux server” call. This will create a second server called “alblinux02” with the same relationships as the server we created manually. Open the “Body” tab to see the JSON payload.

    The most significant change we need to make to the output of the GET call is to modify those attributes whose value was a “dictionary” with three keys (“display_value”, “link” and “value”). These kind of attributes are in the “attributes” section as well as in the “inbound and outbound relations” sections. What we need to do is to replace the whole dictionary value with just the “sys_id” which happens to be the “value” field in the GET call output

    Now we can click “Send” and the new Linux server CI will be created. You will get the status code “201 Created”. Now in the Linux servers app you should see both Linux servers.

    And if you want to see the whole stack end-to-end you can go to the “application” and click on “Show Dependency views”

    Supplementary API calls

    You might have noticed in the request “Body” of the POST call that I included several “sys_id”. While you can get “sys_id” in the ServiceNow GUI by using right-click, ultimately if you want to automate a process you will have to get that information programmatically. For that purpose the Postman collection includes three API calls to help you get “sys_id” you need:

    • CMDB relationship types
    • CMDB users
    • CMDB CI

    In the screenshot below you can see the three supplementary API calls highlighted in yellow. In the response to the “relationship types” call you would have to locate the item in the list with the “name” of the relationship you need and then extract its “sys_id”.

    The process is very similar if you need to extract the “sys_id” of a “user” or a “CI” with the other two API calls

    Tips and tricks

    It might happen that when you build the payload for a POST call, like the “Create CMDB Linux server” you might accidentally omit a mandatory attribute. You could go to the documentation to see what the mandatory attributes are, but one thing I found really useful is that the ServiceNow REST API let’s you know what you are missing in the error message. In the screenshot below I removed the “discovery_source” attribute and sent the API call. As you can see the error message let’s me know exactly what the problem is.

    One thing to bear in mind is that by default GET api calls return only 1000 records. If you are looking for a certain “CI” in order to extract its “sys_id” it might well happen that you have more than 1000 CI’s in the CMDB and the CI is not in the output. In that case you can use the “sysparm_query” parameters to narrow down the search or use the parameter “sysparm_limit” to retrieve more than 1000 records. You will notice in the Postman collection how the “Get CMDB CI” call is using “sysparm_limit” to retrieve 10000 records.

    That was a long post! Thanks for putting up with me for the last 15 mins 😉 I hope you found this tutorial useful.

  • Ansible URI Response JSON Data Parsing

    While most of the vendors and their platforms (including DellEMC) are having Ansible modules published, but there are instances where functionality you’re trying to use isn’t covered in the available module or simply modules aren’t available at all.

    In such situations you’ve no choice than using Ansible URI module. It allows you to interact with HTTP and HTTPS web services, in this particular example REST API endpoints to be precise. There are several benefits of using URI module with REST API, including but not limited to

    • Perform automation if there’s no Ansible module available
    • Use functionality that hasn’t been implemented in Ansible modules
    • Easier to redeploy your workflow to another automation tool

    While I was using the URI module I came across issue of data parsing for REST API response payload. In case of Dell EMC platforms response payload is in JSON and based on the API endpoint response can be 1000s of lines. It’s very difficult to make sense of this data and also process the same in ansible to extract required information. In this blog post I am trying to list down the process I’ve followed to parse the JSON data and extract the required information.

    Below is the sample playbook having uri module. In this example we’re talking about GET call for sample URI endpoint – which is getting device details of the existing server (BMaaS use case).

      tasks:
      - name: listdevices
        uri:
          url: https://api.sample.com/base/version/endpoint
          method: GET
          validate_certs: no
          headers:
            X-Auth-Token: "{{ api_key }}"
          status_code: 200
        register: output
        
      - name: printoutput
        ansible.builtin.debug: 
          var: output

    In this playbook there are multiple parameters used under uri task. More on using uri module with REST API coming in another blog post.

    In above example you can see that we’ve registering the output and then printing the same. Problem with this is there could be 100s of lines which probably doesn’t make sense. So for parsing this data we will need

    Below is the sample of the GET call JSON response.

    [
        {
            "id": "2b12858454174f03aece5a71bb382318",
            "name": "Drive_0_0_11",
            "extra_details": {
                "firmware_version": "GPJ99E5Q",
                "drive_type": "NVMe_SSD",
                "encryption_status": "Supported_Locked_Cluster_PIN",
                "fips_status": "FIPS_Compliance_Level_1",
                "size": 3840755982336
            }
        },
        {
            "id": "2dd2c43f2582415585a9ce0bbb961a4f",
            "name": "Drive_0_0_0",
            "extra_details": {
                "firmware_version": "GPJ99E5Q",
                "drive_type": "NVMe_SSD",
                "encryption_status": "Supported_Locked_Cluster_PIN",
                "fips_status": "FIPS_Compliance_Level_1",
                "size": 3840755982336
            }
        },
        {
            "id": "325e5abc34ed4be398715285cf2e2826",
            "name": "Drive_0_0_8",
            "extra_details": {
                "firmware_version": "GPJ99E5Q",
                "drive_type": "NVMe_SSD",
                "encryption_status": "Supported_Locked_Cluster_PIN",
                "fips_status": "FIPS_Compliance_None",
                "size": 3840755982336
            }
        },
        {
            "id": "3c9449f55ade4c04a801e804f3872eff",
            "name": "Drive_0_0_5",
            "extra_details": {
                "firmware_version": "GPJ99E5Q",
                "drive_type": "NVMe_SSD",
                "encryption_status": "Supported_Locked_Cluster_PIN",
                "fips_status": "FIPS_Compliance_None",
                "size": 3840755982336
            }
        },
        {
            "id": "426d706ef7d443a887a450e6ac4abed5",
            "name": "Drive_0_0_23",
            "extra_details": {
                "firmware_version": "3.0.45.6",
                "drive_type": "NVMe_NVRAM",
                "encryption_status": "Supported_Locked_Cluster_PIN",
                "fips_status": "FIPS_Compliance_None",
                "size": 8484552704
            }
        },
        {
            "id": "42f2145cc5ef4a5f86ec1dfe8bdc5ca3",
            "name": "Drive_0_0_24",
            "extra_details": {
                "firmware_version": "3.0.45.6",
                "drive_type": "NVMe_NVRAM",
                "encryption_status": "Supported_Locked_Cluster_PIN",
                "fips_status": "FIPS_Compliance_None",
                "size": 8484552704
            }
        },
        {
            "id": "4ed0b8b74eff4113ad85ded6f342ab34",
            "name": "Drive_0_0_6",
            "extra_details": {
                "firmware_version": "GPJ99E5Q",
                "drive_type": "NVMe_SSD",
                "encryption_status": "Supported_Locked_Cluster_PIN",
                "fips_status": "FIPS_Compliance_None",
                "size": 3840755982336
            }
        },
        {
            "id": "5bca4db416c342ebab6d3b795c604963",
            "name": "Drive_0_0_2",
            "extra_details": {
                "firmware_version": "GPJ99E5Q",
                "drive_type": "NVMe_SSD",
                "encryption_status": "Supported_Locked_Cluster_PIN",
                "fips_status": "FIPS_Compliance_None",
                "size": 3840755982336
            }
        },
        {
            "id": "6eb66e909f4b414193922e4de63cfa9f",
            "name": "Drive_0_0_1",
            "extra_details": {
                "firmware_version": "GPJ99E5Q",
                "drive_type": "NVMe_SSD",
                "encryption_status": "Supported_Locked_Cluster_PIN",
                "fips_status": "FIPS_Compliance_None",
                "size": 3840755982336
            }
        },
        {
            "id": "b413b5e98e6245b58134c0d5a8d001e1",
            "name": "Drive_0_0_7",
            "extra_details": {
                "firmware_version": "GPJ99E5Q",
                "drive_type": "NVMe_SSD",
                "encryption_status": "Supported_Locked_Cluster_PIN",
                "fips_status": "FIPS_Compliance_None",
                "size": 3840755982336
            }
        },
        {
            "id": "bae7ed28120e43a8b322b8d8b2fedc0d",
            "name": "Drive_0_0_10",
            "extra_details": {
                "firmware_version": "GPJ99E5Q",
                "drive_type": "NVMe_SSD",
                "encryption_status": "Supported_Locked_Cluster_PIN",
                "fips_status": "FIPS_Compliance_None",
                "size": 3840755982336
            }
        },
        {
            "id": "bea9d084e49248359e3191d8f648a37c",
            "name": "Drive_0_0_3",
            "extra_details": {
                "firmware_version": "GPJ99E5Q",
                "drive_type": "NVMe_SSD",
                "encryption_status": "Supported_Locked_Cluster_PIN",
                "fips_status": "FIPS_Compliance_None",
                "size": 3840755982336
            }
        },
        {
            "id": "f3af5ea2e84d4181a9068c6e85c4b750",
            "name": "Drive_0_0_4",
            "extra_details": {
                "firmware_version": "GPJ99E5Q",
                "drive_type": "NVMe_SSD",
                "encryption_status": "Supported_Locked_Cluster_PIN",
                "fips_status": "FIPS_Compliance_None",
                "size": 3840755982336
            }
        },
        {
            "id": "fca0678677cf48f5a51c28ea1e47b514",
            "name": "Drive_0_0_9",
            "extra_details": {
                "firmware_version": "GPJ99E5Q",
                "drive_type": "NVMe_SSD",
                "encryption_status": "Supported_Locked_Cluster_PIN",
                "fips_status": "FIPS_Compliance_None",
                "size": 3840755982336
            }
        }
    ]

    In above JSON example if I have to extract only name and id from the JSON, then we will need to parse the data in playbook as a separate task. Sample task is as mentioned below.

      - name: devicenames
        set_fact: 
          devicenames: "{{ devices | json_query(jmesquery) }}"
        vars:
          jmesquery: '*.devices[*].[name, id]'

    So, overall the playbook will look like below

    ---
    - hosts: localhost
      gather_facts: no
    
      tasks:
      - name: getdevices
        uri:
          url: https://<target_api_server>/v1/definition/endpoint
          method: GET
          #body_format: json
          validate_certs: no
          headers:
            X-Auth-Token: "{{ api_key }}"
          #body:
          #- [ name, your_username ]
          #- [ password, your_password ]
          #- [ enter, Sign in ]
          status_code: 200
        register: devices
        
      #- name: save the json as variable
      #  set_fact: 
      #    jsondata: "{{ devices.stdout | from_json }}"
      
      - name: devicenames
        set_fact: 
          devicenames: "{{ devices | json_query(jmesquery) }}"
        vars:
          jmesquery: '*.devices[*].[hostname, id]'
          
      - name: print devicenames
        debug:
          msg: "{{ item }}"
        with_items:
        - "{{ devicenames }}"

    In this example I’ve printed the captured device value. But in real life scenario it can be captured as a variable and use as input for subsequent tasks.

  • Ansible with DellEMC Storage: Part 7 – Install PowerStore Collection on AWX/Tower

    Ansible with DellEMC Storage: Part 7 – Install PowerStore Collection on AWX/Tower

    This blog is the continuation of Ansible with DellEMC storage multi-part blog.

    In the last (6th Part) of this blog series, we discussed how to prepare Ansible Tower/AWX with Dell EMC storage credentials.

    In this blog post, we will install Dell EMC PowerStore collection on Ansible AWX and go through the next steps.

    As we all know that Ansible has moved to Collections – a new ways of managing integrations and content management. Dell EMC has already started working towards this and have released several collections for multiple Dell EMC portfolio products, few of which are listed below.

    Apart from this list you can find other Dell portfolio collections (like OpenManage) on this link

    For the scope of this blog post we will focus on installing Dell EMC PowerStore Ansible collection on Ansible AWX. Technically, all the collections can be installed using similar steps.

    As a Pre-Requisite, this blog post assumes that you have –

    • Ansible AWX installed and running
    • Access to operating system / machine having Ansible AWX installed
    • Access to PowerStore storage system (with credentials)

    Additionally, if you’re getting started with Ansible AWX and/or integration with Dell EMC’s storage products then you can follow this blog series to get started from scratch.

    As part of the installation collection installation steps we need to Ansible AWX machine and then connect to the awx_task docker container.

    Login to the AWX machine. You can list the running AWX containers using below command

    [root@awx ~]# docker container list
    CONTAINER ID        IMAGE                     COMMAND                  CREATED             STATUS              PORTS                  NAMES
    6ced2eccbd7b        ansible/awx_task:11.2.0   "tini -- /bin/sh -c …"   13 months ago       Up 6 days           8052/tcp               awx_task
    42b14fbd15ad        ansible/awx_web:11.2.0    "tini -- /bin/sh -c …"   13 months ago       Up 6 days           0.0.0.0:80->8052/tcp   awx_web
    4c08c0e39128        memcached:alpine          "docker-entrypoint.s…"   13 months ago       Up 6 days           11211/tcp              awx_memcached
    42224676c21a        redis                     "docker-entrypoint.s…"   13 months ago       Up 6 days           6379/tcp               awx_redis
    37d0ca0c67bc        postgres:10               "docker-entrypoint.s…"   13 months ago       Up 6 days           5432/tcp               awx_postgres
    

    Then connect to the awx_task container using below command

    # docker exec -it awx_task bash

    Next, install the PowerStore Ansible Modules collection in awx_task container

    # ansible-galaxy collection install dellemc.powerstore
    Process install dependency map
    Starting collection install process
    Installing 'dellemc.powerstore:1.2.1' to '/home/awx/.ansible/collections/ansible_collections/dellemc/powerstore'
    

    Then logout from the container.

    bash-4.4# exit

    Now you have successfully installed the Dell EMC PowerStore Ansible Modules collection. Next step post installing collection are

    • Create the PowerStore credentials on the Ansible AWX
    • Create PowerStore Project – assuming you’ve PowerStore playbooks on content repo (like Git)
    • Configure Ansible AWX Job template / Workflow template for storage task automation

    All Dell EMC’s published collections comes with sample playbooks to test the functionality and also to get you started with integrations. When it comes to PowerStore you can see them under /home/awx/ansible-powerstore/dellemc_ansible/powerstore/samples directory

    # cd /home/awx/ansible-powerstore/dellemc_ansible/powerstore/samples
    # ls -l
    -rw-r--r-- 1 root root 1892 Jun  4  2020 capacity_volumes.yml
    -rw-r--r-- 1 root root 1042 Jun  4  2020 create_multiple_volumes_async.yml
    -rw-r--r-- 1 root root  799 Jun  4  2020 create_multiple_volumes.yml
    -rw-r--r-- 1 root root  790 Jun  4  2020 delete_multiple_volumes.yml
    -rw-r--r-- 1 root root 1710 Jun  4  2020 find_empty_volume_groups.yml
    -rw-r--r-- 1 root root 1141 Jun  4  2020 search_volumes.yml
    

    You can re-use these sample playbooks to quickly get started with storage automation tasks. Sample playbooks in the collection has multiple variables like –

    • array_ip
    • user
    • password
    • verifycert

    You can capture the storage credentials by creating Dell EMC storage credential type (screenshot below)

    Ansible AWX – Dell EMC Storage Credential Type

    Once Dell Storage credential type is created then you can add PowerStore array details and credentials using AWX credential manager.

    Ansible AWX – Dell EMC Storage Credential

    After adding PowerStore credential you can use the same in the AWX automation job template creation. Additional variables (like volume names, size, host etc.) can be captured using extra_vars

    Ansible AWX – Job Template Creation

    Additionally you can create survey to capture the required variables and also workflow visualizer to create multi-step breakdown of the automation tasks including but not limited storage automation. Below is the example of breaking down the storage provisioning workflow in the logical steps (like approval, quota management, provisioning, etc.)

    Ansible AWX – Workflow Visualizer

    Hope this helps everyone.

    Update: Please note that the latest version of AWX has moved to Kubernetes (instead of Docker). Please use the below steps to install the PowerStore modules.

    [root@awx ~]# kubectl -n awx exec -it awx-844c574f84-bc4ww -c awx-ee -- /bin/bash
    bash-4.4$ ansible-galaxy collection install dellemc.powerstore -c
    Starting galaxy collection install process
    Process install dependency map
    Starting collection install process
    Downloading https://galaxy.ansible.com/download/dellemc-powerstore-1.6.0.tar.gz to /home/runner/.ansible/tmp/ansible-local-3648wbuk4c_/tmprzm8bse6/dellemc-powerstore-1.6.0-ltamh_71
    Installing 'dellemc.powerstore:1.6.0' to '/home/runner/.ansible/collections/ansible_collections/dellemc/powerstore'
    dellemc.powerstore:1.6.0 was installed successfully
    bash-4.4$
  • Dell EMC VxRAIL – Using REST API

    Dell EMC VxRAIL – Using REST API

    There are many use cases where VxRAIL manager, VMware vCenter Console, or vSuite will not be enough for your goals in mind. So, for monitoring and management of your VxRAIL cluster, you can utilize the VxRAIL REST API for achieving your end goal in mind.

    There are multiple ways to get your hands around the VxRAIL REST API

    1. VxRAIL REST API Cookbook – PDF Guide
    2. VxRAIL SwaggerUI

    VxRAIL Swagger UI is always (default) runs on the VxRAIL cluster and can be accessed using a browser. Link for accessing the VxRAIL Swagger UI is – https://<VxRAIL_Manager_IP>/rest/vxm/api-doc.html

    VxRAIL – Swagger UI

    From the Swagger UI (top right – Select a definition drop-down) you can select the categories of API calls. By default, Swagger UI opens into the Day 1 Bring Up Configuration.

    Additionally VxRAIL Swagger UI allows you to play with the APIs on the same page. For this you’ll need to Authorize the page using VxRAIL manager credentials. This is to make sure that user is restricted to the right level of authorization based on their user type.

    For executing / trying the APIs on the VxRAIL cluster you can simply choose the definition from the drop-down. In this case I’ve selected Cluster definition.

    VxRAIL – Select Definition

    If you expand the selected API it will show you multiple sections (Cluster Information in this example)

    VxRAIL – Cluster Information
    • Parameters – Some APIs needs parameters as input for the successful exectution. If applicable they will be listed here
    • Responses – This section shows you the possible response codes for selected API with example output snippet.

    When you click on the Try it out button page gets into the run-mode. Once you enter required parameters (not required in this example) you can click on Execute. At this point request will be sent to VxRAIL manage and response (body and headers) will be shown on the same screen.

    Way Forward

    I hope this gave you the high level overview of VxRAIL APIs and how to access them. Though Swagger has built-in option to try the APIs, but that is the just a API explorer tool. Additionally you can also use the REST clients – like Postman – to interact with the API. Eventually you’ll integrate these APIs with your automation tools – those can be VMware vRA, Ansible, Terraform, or it can be your own developed tool. Technically speaking you can use any tool as far as it has option to interact with REST API.

    More on this coming in next blog posts 🙂

  • Creating ServiceNow Incidents via REST API

    Creating ServiceNow Incidents via REST API

    In this article we will explore how to create incidents in ServiceNow using the REST API. This article was a stepping stone for this video that shows how to integrate ServiceNow, Microsoft Teams and alerts from infrastructure. You might also be interested in the second post in this series: ServiceNow CMDB REST API tutorial

    The very first thing to do when working with a REST API is to get your hands on the reference guide and hopefully the getting started guide if there is one. The first thing to look for is how to authenticate with the API and whether there are any requirements for special headers or things like that. Afterwards things tend to flow faster and easier. Fortunately, ServiceNow supports basic authentication so there is no learning curve there, although it does support more secure authentication through OAuth if you need it

    In terms of documentation the online help is great. Here you can get an overview of the API. And this is the starting point for the online REST API reference. There are many branches or child API’s hanging of this root. In the screenshot below you can see how, for every call, the online help shows the URL (default or for a specific version) and parameters (path, query and request body). Further down it shows headers for the request and the response and even two coding examples for curl and python … as complete as it gets

    But the tool you will learn to love very quickly is the “REST API Explorer”. Please note that this is a tool that you can only access from within your instance. Once you log in go to “System Web Services” and then locate “REST API Explorer” as shown here

    You will then end up with a menu like this. Notice how you can select the specific child API in the top-left corner and the version of your environment.

    ServiceNow stores all data in tables. This is also true for Incidents which unsurprisingly are stored in the “incident” table. To manipulate tables we need to use the Table API. Different HTTP methods will enable us to do the various CRUD operations. For example if we want to create an incident we will have to use the POST method. Notice in the previous image how I have selected the “Table API” and within that API the “Create a record (POST)” call. Then on the right I have selected the “incident” table.

    When you scroll down you can see a dialog that allows you to build the request body for the incident creation. With the drop-down menu you can select from all the available fields. As you select new fields and assign values, the REST API Explorer builds the body for you in the text box immediately below. At the very bottom you can generate code in multiple languages

    We have grouped the API calls we are using in this article into a Postman collection. You can download the collection from the following the following GitHub repo :

    https://github.com/cermegno/postman-servicenow

    The collection uses 2 variables that you must add to an environment. If you are new to Postman environments check out this older article:

    • {{pwd}}. This is the password for the “admin” user of your ServiceNow instance. If you need to use a different user, you can change it in the collection settings
    • {{instance}}. This is your ServiceNow instance name, i.e. excluding the “.service-now.com” suffix. If you don’t have one or you cannot test this with your production instance, you can open your very own developer instance with ServiceNow

    The collection provides 4 calls and a saved example for each call:

    • Get details for all incidents. This will produce a 98 line JSON structure for each incident
    • Get details for a single incident. This requires you to pass the “sys_id” of the incident as part of the URL as shown below. You can get the “sys_id” for a specific incident from the body of the response during the creation (POST) operation
    • Create incident (POST). The JSON Body parameter can take “a lot” of fields but you can start small. For example the following Body produces the incident below:
    • Modify incident (PUT). This will allow you to make changes to incidents. In particular you can use it to resolve/close incidents by setting the state to “7” as shown in the screenshot below. This call also requires you to pass the “sys_id” of the incident you are modifying as part of the URL

    In this article we have used the REST API to interact with ServiceNow because this is the way I will do it in the upcoming video demo. But depending on what you are trying to do you might want to use Ansible. In that case you can use the official Ansible modules provided by ServiceNow themselves. The collection is available in Ansible Galaxy and provides just two modules designed to interact with ServiceNow tables. Follow the instructions in the Ansible Galaxy page to install the dependencies including the “pysnow” Python library

    As a next step you can visit the second post in this series: ServiceNow CMDB REST API tutorial

    We hope you find this article helpful. Let us know your thoughts in the comment section.

  • Ansible with DellEMC Storage: Part 6 – Prepare Ansible Tower / AWX with Storage Credential

    Ansible with DellEMC Storage: Part 6 – Prepare Ansible Tower / AWX with Storage Credential

    This blog is the continuation of Ansible with DellEMC storage multi-part blog.

    In the last (5th part) of this blog series, we discussed how to install Ansible AWX / Tower in a docker container and Dell EMC Ansible modules inside the Ansible AWX / Tower container.

    In this blog post, we will create a new credential type for Dell EMC storage systems and then add storage credentials in Ansible AWX / Tower.

    So first let’s get started with creating a Credential type for Dell EMC storage array. Login to your Ansible AWX / Tower console. In the navigation pane (on left) click on “Credential Types”. Then click on the green “+” icon on the right to add a new credential type.

    Ansible AWX / Tower – Create New Credential 1

    In the “New Credential Type” creation page enter below details and then click on Save

    • Name – Dell EMC Storage (User Friendly Name)
    • Description – Optional Description
    • Input Configuration – Copy and Paste below text
    fields:
      - id: target
        type: string
        label: Array IP address
      - id: username
        type: string
        label: Array username
      - id: password
        type: string
        label: Array password
        secret: true
    required:
      - target
      - username
      - password
    • Injector Configuration – Copy and Paste below text
    extra_vars:
      password: '{{ password }}'
      target: '{{ target }}'
      username: '{{ username }}'
    
    Ansible AWX / Tower – Create New Credential 2

    Once Credential type is created, click on the “Credentials” in AWX / Tower navigation pane. Then click green “+” button on right to create new Credential.

    Ansible AWX / Tower – Create New Credential 3

    In the “New Credentials” page enter below details.

    • Name – User Friendly Name to identify the storage array. In this example “Production PowerMax”
    • Description – Optional description
    • Organization – You can choose appropriate Organization as per your configuration. Note that Organization can help you to mask these credentials to few users. I have selected Default in my example
    • Credential Type – Choose “Dell EMC Storage”. This is the credential type we have created in earlier step. Credential type which we create earlier will be available on last few pages of the pop-up list. Alternatively you can search the same with “Dell EMC Storage”
    • Type Details
      • Array IP Address – Enter the management IP address of storage array
      • Array Username – Username (I suggest creating separate user for automation)
      • Array Password – Password for the entered username
    Ansible AWX / Tower – Create New Credential 4

    As per your environment you can create multiple credentials, each for one array.

    Ansible AWX / Tower – Create New Credential 5

    Once credentials are created you can now go ahead and create new Ansible AWX / Tower Templates using storage credentials we just created. In the playbook you can mention storage array credentials as variables (username and password). Using these variables you don’t have to mention array credentials as plain text in the playbooks.

    Additionally, you can add other infrastructure / application credentials using the same process.

    I hope this helps everyone. In next post I will take you through creating new Project and Template.

  • Ansible with DellEMC Storage: Part 5 – Get Started with Ansible Tower (Using AWX)

    Ansible with DellEMC Storage: Part 5 – Get Started with Ansible Tower (Using AWX)

    This blog is the continuation of Ansible with DellEMC storage multi-part blog. 

    In this 5th part, we will discuss about Ansible Tower (using AWX) and how to install and configure the same.

    Overview

    In my posts till now you might noticed that I have only used command line option to run any Ansible commands. There’s a reason for that. By default, when you install Ansible it only installs Ansible Engine, which only has Ansible CLI option. This is where Ansible Tower comes in picture. Note that AWX is the open source project for Ansible Tower.

    While Ansible Tower has many features, below are few features which are my personal favorites. Since data storage management operations can be data destructive, below features are the reasons I am highly recommending using Tower / AWX for automating storage tasks.

    • Web Interface – To manage Ansible using Web interface
    • REST API Support – To manage and integrate Ansible Tower in other platforms. For example – integrating Ansible Tower with Jenkins
    • Task Engine – To create scheduled job and centralized operations.
    • Role Based Access Control – To control Enterprise level access across different team member and limit their visibility to information. This will make sure only designated users are having access to critical data/tasks.

    Installing Ansible Tower (using AWX)

    Make sure that you’ve supported operating systems installed and running. In my case I am using CentOS V7 virtual machine

    Prerequisites –

    Make sure below packages are installed on your machine

    • Make sure you have set the selinux to permissive
      • sudo setenforce permissive
    • Ansible – min version 2.8+
      • yum install -y ansible
    • Docker – Recent version
      • yum install -y docker
    • Docker Python Module
      • Follow below steps for installing docker-compose
    [root@dw-test-1 installer]# sudo curl -L "https://github.com/docker/compose/releases/download/1.25.4/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
    
    [root@dw-test-1 installer]# sudo chmod +x /usr/local/bin/docker-compose
    
    [root@dw-test-1 installer]# sudo ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose
    
    [root@dw-test-1 installer]# docker-compose --version
    

    Once all the prerequisites are in place run below tasks

    • Clone the Git AWX repository using below command
    [root@dw-test-1 installer]# git clone https://github.com/ansible/awx.git

    This will create awx directory. cd into the awx/installer directory.

    [root@dw-test-1 installer]# cd awx/installer

    This folder has inventory file which has all the AWX parameters. Most important details in this file are passwords which are needed for logging into the AWX post installation. Note that AWX supports installation on Kubernetes and Openshift as well. In this example we will be using docker-compose.

    Most important parameters in inventory file are as mentioned below. In my case I created the vars.yml file with below inputs and used the same while running the installation.

    [root@dw-test-1 installer]# cat vars.yml
    admin_password: 'admin'
    pg_password: 'admin'
    secret_key: 'mysecret'

    Once you’ve checked and verified all the parameters in the inventory file, go ahead and run below command.

    [root@dw-test-1 installer]# ansible-playbook -i inventory install.yml -e @vars.yml

    Note – I faced couple of errors which running this playbook. I suggest taking a look at this link if you face the same.

    Once playbook execution is completed you can run below command to validate the installation.

    [root@dw-test-1 installer]# docker ps
    CONTAINER ID        IMAGE                     COMMAND                  CREATED             STATUS              PORTS                  NAMES
    23b236356057        ansible/awx_task:10.0.0   "/tini -- /bin/sh ..."   6 minutes ago       Up 2 minutes        8052/tcp               awx_task
    a23edc05283e        ansible/awx_web:10.0.0    "/tini -- /bin/sh ..."   6 minutes ago       Up 2 minutes        0.0.0.0:80->8052/tcp   awx_web
    385de9d395da        postgres:10               "docker-entrypoint..."   6 minutes ago       Up 2 minutes        5432/tcp               awx_postgres
    125b9e551823        redis                     "docker-entrypoint..."   2 hours ago         Up 2 minutes        6379/tcp               awx_redis
    2a8ad285e2d5        memcached:alpine          "docker-entrypoint..."   2 hours ago         Up 2 minutes        11211/tcp              awx_memcached
    

    At this point AWX is installed. It takes some time for container to start and configure the AWX. Run the below command and wait till the time you see similar output

    [root@dw-test-1 installer]# docker logs -f awx_task
    ...
    2020-04-12 12:23:18,458 DEBUG    awx.main.dispatch task 7b92a5a8-efa9-4b5e-8dc7-d4ca1d974508 starting awx.main.scheduler.tasks.run_task_manager(*[])
    2020-04-12 12:23:18,466 DEBUG    awx.main.scheduler Running Tower task manager.
    2020-04-12 12:23:18,472 DEBUG    awx.main.scheduler Starting Scheduler
    2020-04-12 12:23:28,468 DEBUG    awx.main.dispatch task ba9a6c3e-24b3-49e4-8cf1-62f90f315780 starting awx.main.tasks.awx_periodic_scheduler(*[])
    2020-04-12 12:23:28,478 DEBUG    awx.main.tasks Starting periodic scheduler
    2020-04-12 12:23:28,480 DEBUG    awx.main.tasks Last scheduler run was: 2020-04-12 12:22:58.567745+00:00
    2020-04-12 12:23:38,483 DEBUG    awx.main.dispatch task 7f4ac140-7231-4c6e-901f-34be80529707 starting awx.main.scheduler.tasks.run_task_manager(*[])
    2020-04-12 12:23:38,493 DEBUG    awx.main.scheduler Running Tower task manager.
    2020-04-12 12:23:38,502 DEBUG    awx.main.scheduler Starting Scheduler
    2020-04-12 12:23:58,522 DEBUG    awx.main.dispatch task 7ed5d956-ba67-4219-9509-1405713fa155 starting awx.main.tasks.cluster_node_heartbeat(*[])
    2020-04-12 12:23:58,632 DEBUG    awx.main.tasks Cluster node heartbeat task.
    2020-04-12 12:23:58,508 DEBUG    awx.main.dispatch task 7ca00dc2-ab9c-4c65-b505-f880afc47f65 starting awx.main.tasks.gather_analytics(*[])
    2020-04-12 12:23:58,560 DEBUG    awx.main.dispatch task 0e539914-105c-462f-89c6-20f1a50b1ec8 starting awx.main.tasks.awx_periodic_scheduler(*[])
    2020-04-12 12:23:58,693 DEBUG    awx.main.tasks Starting periodic scheduler
    2020-04-12 12:23:58,696 DEBUG    awx.main.tasks Last scheduler run was: 2020-04-12 12:23:28.480007+00:00
    2020-04-12 12:23:58,536 DEBUG    awx.main.dispatch task 5ec695f4-80f2-44a0-a18f-cdaa62215b10 starting awx.main.tasks.awx_k8s_reaper(*[])
    2020-04-12 12:23:58,622 WARNING  awx.main.dispatch scaling up worker pid:159
    2020-04-12 12:23:58,738 DEBUG    awx.main.dispatch task a643c365-145c-4bde-9536-d2f647d243f0 starting awx.main.scheduler.tasks.run_task_manager(*[])
    2020-04-12 12:23:58,747 DEBUG    awx.main.scheduler Running Tower task manager.
    2020-04-12 12:23:58,752 DEBUG    awx.main.scheduler Starting Scheduler
    RESULT 2
    OKREADY
    

    Now we will need to install prerequisites for DellEMC Ansible module – PyU4V package. Follow below steps to install DellEMC Ansible module dependencies.

    Connect to the container awx_task

    [root@dw-test-1 installer]# docker exec -it awx_task bash

    On the container prompt run below command to install PyU4V package

    bash-4.4# pip3 install PyU4V==9.1.1.0
    WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead.
    Collecting PyU4V==9.1.1.0
      Downloading https://files.pythonhosted.org/packages/38/83/34e7d4b823f84b74f6ac959b3cc5302882022f65c78e9f91593d531ebd1d/PyU4V-9.1.1.0-py3-none-any.whl (79kB)
        100% |████████████████████████████████| 81kB 2.4MB/s
    Collecting urllib3 (from PyU4V==9.1.1.0)
      Downloading https://files.pythonhosted.org/packages/e8/74/6e4f91745020f967d09332bb2b8b9b10090957334692eb88ea4afe91b77f/urllib3-1.25.8-py2.py3-none-any.whl (125kB)
        100% |████████████████████████████████| 133kB 3.4MB/s
    Collecting prettytable (from PyU4V==9.1.1.0)
      Downloading https://files.pythonhosted.org/packages/ef/30/4b0746848746ed5941f052479e7c23d2b56d174b82f4fd34a25e389831f5/prettytable-0.7.2.tar.bz2
    Requirement already satisfied: six in /usr/lib/python3.6/site-packages (from PyU4V==9.1.1.0)
    Requirement already satisfied: setuptools in /usr/lib/python3.6/site-packages (from PyU4V==9.1.1.0)
    Collecting requests (from PyU4V==9.1.1.0)
      Downloading https://files.pythonhosted.org/packages/1a/70/1935c770cb3be6e3a8b78ced23d7e0f3b187f5cbfab4749523ed65d7c9b1/requests-2.23.0-py2.py3-none-any.whl (58kB)
        100% |████████████████████████████████| 61kB 4.4MB/s
    Collecting chardet<4,>=3.0.2 (from requests->PyU4V==9.1.1.0)
      Downloading https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl (133kB)
        100% |████████████████████████████████| 143kB 3.8MB/s
    Collecting certifi>=2017.4.17 (from requests->PyU4V==9.1.1.0)
      Downloading https://files.pythonhosted.org/packages/57/2b/26e37a4b034800c960a00c4e1b3d9ca5d7014e983e6e729e33ea2f36426c/certifi-2020.4.5.1-py2.py3-none-any.whl (157kB)
        100% |████████████████████████████████| 163kB 3.8MB/s
    Requirement already satisfied: idna<3,>=2.5 in /usr/lib/python3.6/site-packages (from requests->PyU4V==9.1.1.0)
    Installing collected packages: urllib3, prettytable, chardet, certifi, requests, PyU4V
      Running setup.py install for prettytable ... done
    Successfully installed PyU4V-9.1.1.0 certifi-2020.4.5.1 chardet-3.0.4 prettytable-0.7.2 requests-2.23.0 urllib3-1.25.8
    

    Once completed you can check installed version using

    bash-4.4# pip3 list | grep PyU4V
    PyU4V (9.1.1.0)

    At this point you have AWX up and running and can be reached on http://localhost:80.

    Tower is now running on the host at port 80.  The rest of the setup is handled by the web interface .  If you did this on the system you are using you can use http://localhost.

    If everything went well then you can see login prompt similar to the below screenshot. Login credentials for AWX are as per inventory file.

    undefined

    This concludes Ansible Tower (AWX) installation process. In next posts we will discuss on configuring Ansible Tower / AWX.

  • Installation of AWX using Ansible fails with error – “Unable to load docker-compose. Try `pip install docker-compose`, ImportError: No module named zipp, ImportError: No module named configparser

    Installation of AWX using Ansible fails with error – “Unable to load docker-compose. Try `pip install docker-compose`, ImportError: No module named zipp, ImportError: No module named configparser

    Recently while I was trying to install AWX using Ansible I came across below error on Centos.

    Docker-compose error while installing AWX
    fatal: [localhost]: FAILED! => {"changed": false, "msg": "Unable to load docker-compose. Try `pip install docker-compose`. Error: Traceback (most recent call last):\n  File \"/tmp/ansible_docker_compose_payload_jNhEZ8/ansible_docker_compose_payload.zip/ansible/modules/cloud/docker/docker_compose.py\", line 483, in <module>\n  File \"/usr/lib/python2.7/site-packages/compose/cli/command.py\", line 12, in <module>\n    from .. import config\n  File \"/usr/lib/python2.7/site-packages/compose/config/__init__.py\", line 6, in <module>\n    from .config import ConfigurationError\n  File \"/usr/lib/python2.7/site-packages/compose/config/config.py\", line 51, in <module>\n    from .validation import match_named_volumes\n  File \"/usr/lib/python2.7/site-packages/compose/config/validation.py\", line 12, in <module>\n    from jsonschema import Draft4Validator\n  File \"/usr/lib/python2.7/site-packages/jsonschema/__init__.py\", line 33, in <module>\n    import importlib_metadata as metadata\n  File \"/usr/lib/python2.7/site-packages/importlib_metadata/__init__.py\", line 9, in <module>\n    import zipp\nImportError: No module named zipp\n"}

    While this error was referring to missing pip package “docker-compose”, but when I tried to install the same it gave below error stating that it’s already installed.

    [root@dw-test-1 installer]# pip install docker-compose
    Requirement already satisfied (use --upgrade to upgrade): docker-compose in /usr/lib/python2.7/site-packages
    Requirement already satisfied (use --upgrade to upgrade): texttable<2,>=0.9.0 in /usr/lib/python2.7/site-packages (from docker-compose)
    Requirement already satisfied (use --upgrade to upgrade): requests<3,>=2.20.0 in /usr/lib/python2.7/site-packages (from docker-compose)
    Requirement already satisfied (use --upgrade to upgrade): dockerpty<1,>=0.4.1 in /usr/lib/python2.7/site-packages (from docker-compose)
    Requirement already satisfied (use --upgrade to upgrade): six<2,>=1.3.0 in /usr/lib/python2.7/site-packages (from docker-compose)
    Requirement already satisfied (use --upgrade to upgrade): docopt<1,>=0.6.1 in /usr/lib/python2.7/site-packages (from docker-compose)
    Requirement already satisfied (use --upgrade to upgrade): ipaddress<2,>=1.0.16; python_version < "3.3" in /usr/lib/python2.7/site-packages (from docker-compose)
    Requirement already satisfied (use --upgrade to upgrade): subprocess32<4,>=3.5.4; python_version < "3.2" in /usr/lib64/python2.7/site-packages (from docker-compose)
    Requirement already satisfied (use --upgrade to upgrade): enum34<2,>=1.0.4; python_version < "3.4" in /usr/lib/python2.7/site-packages (from docker-compose)
    Requirement already satisfied (use --upgrade to upgrade): websocket-client<1,>=0.32.0 in /usr/lib/python2.7/site-packages (from docker-compose)
    Requirement already satisfied (use --upgrade to upgrade): jsonschema<4,>=2.5.1 in /usr/lib/python2.7/site-packages (from docker-compose)
    Requirement already satisfied (use --upgrade to upgrade): backports.shutil-get-terminal-size==1.0.0; python_version < "3.3" in /usr/lib/python2.7/site-packages (from docker-compose)
    Requirement already satisfied (use --upgrade to upgrade): backports.ssl-match-hostname<4,>=3.5; python_version < "3.5" in /usr/lib/python2.7/site-packages (from docker-compose)
    Requirement already satisfied (use --upgrade to upgrade): cached-property<2,>=1.2.0 in /usr/lib/python2.7/site-packages (from docker-compose)
    Requirement already satisfied (use --upgrade to upgrade): PyYAML<6,>=3.10 in /usr/lib64/python2.7/site-packages (from docker-compose)
    Requirement already satisfied (use --upgrade to upgrade): docker[ssh]<5,>=3.7.0 in /usr/lib/python2.7/site-packages (from docker-compose)
    Requirement already satisfied (use --upgrade to upgrade): urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/lib/python2.7/site-packages (from requests<3,>=2.20.0->docker-compose)
    Requirement already satisfied (use --upgrade to upgrade): idna<3,>=2.5 in /usr/lib/python2.7/site-packages (from requests<3,>=2.20.0->docker-compose)
    Requirement already satisfied (use --upgrade to upgrade): chardet<4,>=3.0.2 in /usr/lib/python2.7/site-packages (from requests<3,>=2.20.0->docker-compose)
    Requirement already satisfied (use --upgrade to upgrade): certifi>=2017.4.17 in /usr/lib/python2.7/site-packages (from requests<3,>=2.20.0->docker-compose)
    Requirement already satisfied (use --upgrade to upgrade): pyrsistent>=0.14.0 in /usr/lib64/python2.7/site-packages (from jsonschema<4,>=2.5.1->docker-compose)
    Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/lib/python2.7/site-packages (from jsonschema<4,>=2.5.1->docker-compose)
    Requirement already satisfied (use --upgrade to upgrade): attrs>=17.4.0 in /usr/lib/python2.7/site-packages (from jsonschema<4,>=2.5.1->docker-compose)
    Requirement already satisfied (use --upgrade to upgrade): importlib-metadata; python_version < "3.8" in /usr/lib/python2.7/site-packages (from jsonschema<4,>=2.5.1->docker-compose)
    Requirement already satisfied (use --upgrade to upgrade): functools32; python_version < "3" in /usr/lib/python2.7/site-packages (from jsonschema<4,>=2.5.1->docker-compose)
    Requirement already satisfied (use --upgrade to upgrade): paramiko>=2.4.2; extra == "ssh" in /usr/lib/python2.7/site-packages (from docker[ssh]<5,>=3.7.0->docker-compose)
    Collecting configparser>=3.5; python_version < "3" (from importlib-metadata; python_version < "3.8"->jsonschema<4,>=2.5.1->docker-compose)
      Using cached https://files.pythonhosted.org/packages/e5/7c/d4ccbcde76b4eea8cbd73b67b88c72578e8b4944d1270021596e80b13deb/configparser-5.0.0.tar.gz
      Running setup.py (path:/tmp/pip-build-c4RmaK/configparser/setup.py) egg_info for package configparser produced metadata for project name unknown. Fix your #egg=configparser fragments.
      Requirement already satisfied (use --upgrade to upgrade): unknown from https://files.pythonhosted.org/packages/e5/7c/d4ccbcde76b4eea8cbd73b67b88c72578e8b4944d1270021596e80b13deb/configparser-5.0.0.tar.gz#sha256=2ca44140ee259b5e3d8aaf47c79c36a7ab0d5e94d70bd4105c03ede7a20ea5a1 in /usr/lib/python2.7/site-packages (from importlib-metadata; python_version < "3.8"->jsonschema<4,>=2.5.1->docker-compose)
    Collecting zipp>=0.5 (from importlib-metadata; python_version < "3.8"->jsonschema<4,>=2.5.1->docker-compose)
      Using cached https://files.pythonhosted.org/packages/ce/8c/2c5f7dc1b418f659d36c04dec9446612fc7b45c8095cc7369dd772513055/zipp-3.1.0.tar.gz
      Running setup.py (path:/tmp/pip-build-c4RmaK/zipp/setup.py) egg_info for package zipp produced metadata for project name unknown. Fix your #egg=zipp fragments.
      Requirement already satisfied (use --upgrade to upgrade): unknown from https://files.pythonhosted.org/packages/ce/8c/2c5f7dc1b418f659d36c04dec9446612fc7b45c8095cc7369dd772513055/zipp-3.1.0.tar.gz#sha256=c599e4d75c98f6798c509911d08a22e6c021d074469042177c8c86fb92eefd96 in /usr/lib/python2.7/site-packages (from importlib-metadata; python_version < "3.8"->jsonschema<4,>=2.5.1->docker-compose)
    Requirement already satisfied (use --upgrade to upgrade): contextlib2; python_version < "3" in /usr/lib/python2.7/site-packages (from importlib-metadata; python_version < "3.8"->jsonschema<4,>=2.5.1->docker-compose)
    Requirement already satisfied (use --upgrade to upgrade): pathlib2; python_version < "3" in /usr/lib/python2.7/site-packages (from importlib-metadata; python_version < "3.8"->jsonschema<4,>=2.5.1->docker-compose)
    Requirement already satisfied (use --upgrade to upgrade): pynacl>=1.0.1 in /usr/lib64/python2.7/site-packages (from paramiko>=2.4.2; extra == "ssh"->docker[ssh]<5,>=3.7.0->docker-compose)
    Requirement already satisfied (use --upgrade to upgrade): cryptography>=2.5 in /usr/lib64/python2.7/site-packages (from paramiko>=2.4.2; extra == "ssh"->docker[ssh]<5,>=3.7.0->docker-compose)
    Requirement already satisfied (use --upgrade to upgrade): bcrypt>=3.1.3 in /usr/lib64/python2.7/site-packages (from paramiko>=2.4.2; extra == "ssh"->docker[ssh]<5,>=3.7.0->docker-compose)
    Requirement already satisfied (use --upgrade to upgrade): scandir; python_version < "3.5" in /usr/lib64/python2.7/site-packages (from pathlib2; python_version < "3"->importlib-metadata; python_version < "3.8"->jsonschema<4,>=2.5.1->docker-compose)
    Requirement already satisfied (use --upgrade to upgrade): cffi>=1.4.1 in /usr/lib64/python2.7/site-packages (from pynacl>=1.0.1->paramiko>=2.4.2; extra == "ssh"->docker[ssh]<5,>=3.7.0->docker-compose)
    Requirement already satisfied (use --upgrade to upgrade): pycparser in /usr/lib/python2.7/site-packages (from cffi>=1.4.1->pynacl>=1.0.1->paramiko>=2.4.2; extra == "ssh"->docker[ssh]<5,>=3.7.0->docker-compose)
    You are using pip version 8.1.2, however version 20.0.2 is available.
    You should consider upgrading via the 'pip install --upgrade pip' command.
    

    After trying multiple solutions on several forums and reinstalling everything from scratch finally I thought of upgrading pip.

    [root@dw-test-1 installer]# pip install --upgrade pip
    Collecting pip
      Downloading https://files.pythonhosted.org/packages/54/0c/d01aa759fdc501a58f431eb594a17495f15b88da142ce14b5845662c13f3/pip-20.0.2-py2.py3-none-any.whl (1.4MB)
        100% |████████████████████████████████| 1.4MB 787kB/s
    Installing collected packages: pip
      Found existing installation: pip 8.1.2
        Uninstalling pip-8.1.2:
          Successfully uninstalled pip-8.1.2
    Successfully installed pip-20.0.2

    Post upgrading pip to latest version I reinstalled docker-compose.

    [root@dw-test-1 installer]# pip install docker-compose
    DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
    Requirement already satisfied: docker-compose in /usr/lib/python2.7/site-packages (1.25.5)
    Requirement already satisfied: backports.shutil-get-terminal-size==1.0.0; python_version < "3.3" in /usr/lib/python2.7/site-packages (from docker-compose) (1.0.0)
    Requirement already satisfied: six<2,>=1.3.0 in /usr/lib/python2.7/site-packages (from docker-compose) (1.9.0)
    Requirement already satisfied: PyYAML<6,>=3.10 in /usr/lib64/python2.7/site-packages (from docker-compose) (3.10)
    Requirement already satisfied: docker[ssh]<5,>=3.7.0 in /usr/lib/python2.7/site-packages (from docker-compose) (4.2.0)
    Requirement already satisfied: dockerpty<1,>=0.4.1 in /usr/lib/python2.7/site-packages (from docker-compose) (0.4.1)
    Requirement already satisfied: jsonschema<4,>=2.5.1 in /usr/lib/python2.7/site-packages (from docker-compose) (3.2.0)
    Requirement already satisfied: requests<3,>=2.20.0 in /usr/lib/python2.7/site-packages (from docker-compose) (2.23.0)
    Requirement already satisfied: enum34<2,>=1.0.4; python_version < "3.4" in /usr/lib/python2.7/site-packages (from docker-compose) (1.0.4)
    Requirement already satisfied: websocket-client<1,>=0.32.0 in /usr/lib/python2.7/site-packages (from docker-compose) (0.57.0)
    Requirement already satisfied: cached-property<2,>=1.2.0 in /usr/lib/python2.7/site-packages (from docker-compose) (1.5.1)
    Requirement already satisfied: ipaddress<2,>=1.0.16; python_version < "3.3" in /usr/lib/python2.7/site-packages (from docker-compose) (1.0.16)
    Requirement already satisfied: docopt<1,>=0.6.1 in /usr/lib/python2.7/site-packages (from docker-compose) (0.6.2)
    Requirement already satisfied: subprocess32<4,>=3.5.4; python_version < "3.2" in /usr/lib64/python2.7/site-packages (from docker-compose) (3.5.4)
    Requirement already satisfied: texttable<2,>=0.9.0 in /usr/lib/python2.7/site-packages (from docker-compose) (1.6.2)
    Requirement already satisfied: backports.ssl-match-hostname<4,>=3.5; python_version < "3.5" in /usr/lib/python2.7/site-packages (from docker-compose) (3.5.0.1)
    Requirement already satisfied: paramiko>=2.4.2; extra == "ssh" in /usr/lib/python2.7/site-packages (from docker[ssh]<5,>=3.7.0->docker-compose) (2.7.1)
    Requirement already satisfied: setuptools in /usr/lib/python2.7/site-packages (from jsonschema<4,>=2.5.1->docker-compose) (0.9.8)
    Requirement already satisfied: pyrsistent>=0.14.0 in /usr/lib64/python2.7/site-packages (from jsonschema<4,>=2.5.1->docker-compose) (0.16.0)
    Requirement already satisfied: attrs>=17.4.0 in /usr/lib/python2.7/site-packages (from jsonschema<4,>=2.5.1->docker-compose) (19.3.0)
    Requirement already satisfied: importlib-metadata; python_version < "3.8" in /usr/lib/python2.7/site-packages (from jsonschema<4,>=2.5.1->docker-compose) (1.6.0)
    Requirement already satisfied: functools32; python_version < "3" in /usr/lib/python2.7/site-packages (from jsonschema<4,>=2.5.1->docker-compose) (3.2.3.post2)
    Requirement already satisfied: idna<3,>=2.5 in /usr/lib/python2.7/site-packages (from requests<3,>=2.20.0->docker-compose) (2.9)
    Requirement already satisfied: chardet<4,>=3.0.2 in /usr/lib/python2.7/site-packages (from requests<3,>=2.20.0->docker-compose) (3.0.4)
    Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/lib/python2.7/site-packages (from requests<3,>=2.20.0->docker-compose) (1.25.8)
    Requirement already satisfied: certifi>=2017.4.17 in /usr/lib/python2.7/site-packages (from requests<3,>=2.20.0->docker-compose) (2020.4.5.1)
    Requirement already satisfied: bcrypt>=3.1.3 in /usr/lib64/python2.7/site-packages (from paramiko>=2.4.2; extra == "ssh"->docker[ssh]<5,>=3.7.0->docker-compose) (3.1.7)
    Requirement already satisfied: pynacl>=1.0.1 in /usr/lib64/python2.7/site-packages (from paramiko>=2.4.2; extra == "ssh"->docker[ssh]<5,>=3.7.0->docker-compose) (1.3.0)
    Requirement already satisfied: cryptography>=2.5 in /usr/lib64/python2.7/site-packages (from paramiko>=2.4.2; extra == "ssh"->docker[ssh]<5,>=3.7.0->docker-compose) (2.9)
    Requirement already satisfied: pathlib2; python_version < "3" in /usr/lib/python2.7/site-packages (from importlib-metadata; python_version < "3.8"->jsonschema<4,>=2.5.1->docker-compose) (2.3.5)
    Requirement already satisfied: contextlib2; python_version < "3" in /usr/lib/python2.7/site-packages (from importlib-metadata; python_version < "3.8"->jsonschema<4,>=2.5.1->docker-compose) (0.6.0.post1)
    Collecting zipp>=0.5
      Downloading zipp-1.2.0-py2.py3-none-any.whl (4.8 kB)
    Collecting configparser>=3.5; python_version < "3"
      Downloading configparser-4.0.2-py2.py3-none-any.whl (22 kB)
    Requirement already satisfied: cffi>=1.1 in /usr/lib64/python2.7/site-packages (from bcrypt>=3.1.3->paramiko>=2.4.2; extra == "ssh"->docker[ssh]<5,>=3.7.0->docker-compose) (1.14.0)
    Requirement already satisfied: scandir; python_version < "3.5" in /usr/lib64/python2.7/site-packages (from pathlib2; python_version < "3"->importlib-metadata; python_version < "3.8"->jsonschema<4,>=2.5.1->docker-compose) (1.10.0)
    Requirement already satisfied: pycparser in /usr/lib/python2.7/site-packages (from cffi>=1.1->bcrypt>=3.1.3->paramiko>=2.4.2; extra == "ssh"->docker[ssh]<5,>=3.7.0->docker-compose) (2.14)
    Installing collected packages: zipp, configparser
    Successfully installed configparser-4.0.2 zipp-1.2.0
    

    This time zipp and configparser packages also got installed with docker-compose. This resolved my issue and I was able to go ahead with successful AWX installation.

    In summary –

    • First upgrade pip to latest version
    • Then install docker-compose. Make sure zipp and configparser are getting installed with the same.

    Cheers and happy automating!

  • Installation of docker fails on CentOS 8 with Error – package containerd.io-1.2.10-3.2.el7.x86_64 is excluded

    Installation of docker fails on CentOS 8 with Error – package containerd.io-1.2.10-3.2.el7.x86_64 is excluded

    I recently came across a strange issue while installing Docker on CentOS (version 8) machine.

    Issue –

    When I tried below options to install docker on my machine I got the same error every time.

    Installation Commands Used

    yum install -y -q docker-ce
    or
    yum install -y docker-ce.x86_64
    or
    yum install docker-ce

    Error message received

    Error:
     Problem: package docker-ce-3:19.03.8-3.el7.x86_64 requires containerd.io >= 1.2.2-3, but none of the providers can be installed
      - cannot install the best candidate for the job
      - package containerd.io-1.2.10-3.2.el7.x86_64 is excluded
      - package containerd.io-1.2.13-3.1.el7.x86_64 is excluded
      - package containerd.io-1.2.2-3.3.el7.x86_64 is excluded
      - package containerd.io-1.2.2-3.el7.x86_64 is excluded
      - package containerd.io-1.2.4-3.1.el7.x86_64 is excluded
      - package containerd.io-1.2.5-3.1.el7.x86_64 is excluded
      - package containerd.io-1.2.6-3.3.el7.x86_64 is excluded

    Resolution –

    To resolve this issue, first we need to manually install the containerd.io package.

    [root@test_centos8 /]# yum install -y https://download.docker.com/linux/centos/7/x86_64/stable/Packages/containerd.io-1.2.6-3.3.el7.x86_64.rpm
    Last metadata expiration check: 0:02:53 ago on Thu 02 Apr 2020 09:29:41 AM UTC.
    containerd.io-1.2.6-3.3.el7.x86_64.rpm                                                                                                                                 18 MB/s |  26 MB     00:01
    Dependencies resolved.
    ======================================================================================================================================================================================================
     Package                                               Architecture                    Version                                                            Repository                             Size
    ======================================================================================================================================================================================================
    Installing:
     containerd.io                                         x86_64                          1.2.6-3.3.el7                                                      @commandline                           26 M
    Installing dependencies:
     container-selinux                                     noarch                          2:2.124.0-1.module_el8.1.0+272+3e64ee36                            AppStream                              47 k
     checkpolicy                                           x86_64                          2.9-1.el8                                                          BaseOS                                348 k
     libselinux-utils                                      x86_64                          2.9-2.1.el8                                                        BaseOS                                243 k
     policycoreutils                                       x86_64                          2.9-3.el8_1.1                                                      BaseOS                                377 k
     policycoreutils-python-utils                          noarch                          2.9-3.el8_1.1                                                      BaseOS                                250 k
     python3-audit                                         x86_64                          3.0-0.13.20190507gitf58ec40.el8                                    BaseOS                                 85 k
     python3-libselinux                                    x86_64                          2.9-2.1.el8                                                        BaseOS                                283 k
     python3-libsemanage                                   x86_64                          2.9-1.el8                                                          BaseOS                                127 k
     python3-policycoreutils                               noarch                          2.9-3.el8_1.1                                                      BaseOS                                2.2 M
     python3-setools                                       x86_64                          4.2.2-1.el8                                                        BaseOS                                600 k
     rpm-plugin-selinux                                    x86_64                          4.14.2-25.el8                                                      BaseOS                                 73 k
     selinux-policy                                        noarch                          3.14.3-20.el8                                                      BaseOS                                602 k
     selinux-policy-targeted                               noarch                          3.14.3-20.el8                                                      BaseOS                                 15 M
    Enabling module streams:
     container-tools                                                                       rhel8
    
    Transaction Summary
    ======================================================================================================================================================================================================
    Install  14 Packages
    
    Total size: 46 M
    Total download size: 20 M
    Installed size: 158 M
    Downloading Packages:
    (1/13): libselinux-utils-2.9-2.1.el8.x86_64.rpm                                                                                                                       2.4 MB/s | 243 kB     00:00
    (2/13): container-selinux-2.124.0-1.module_el8.1.0+272+3e64ee36.noarch.rpm                                                                                            375 kB/s |  47 kB     00:00
    (3/13): checkpolicy-2.9-1.el8.x86_64.rpm                                                                                                                              2.5 MB/s | 348 kB     00:00
    (4/13): python3-audit-3.0-0.13.20190507gitf58ec40.el8.x86_64.rpm                                                                                                      2.5 MB/s |  85 kB     00:00
    (5/13): policycoreutils-2.9-3.el8_1.1.x86_64.rpm                                                                                                                      3.0 MB/s | 377 kB     00:00
    (6/13): python3-libsemanage-2.9-1.el8.x86_64.rpm                                                                                                                      2.2 MB/s | 127 kB     00:00
    (7/13): policycoreutils-python-utils-2.9-3.el8_1.1.noarch.rpm                                                                                                         685 kB/s | 250 kB     00:00
    (8/13): python3-libselinux-2.9-2.1.el8.x86_64.rpm                                                                                                                     810 kB/s | 283 kB     00:00
    (9/13): rpm-plugin-selinux-4.14.2-25.el8.x86_64.rpm                                                                                                                   288 kB/s |  73 kB     00:00
    (10/13): python3-setools-4.2.2-1.el8.x86_64.rpm                                                                                                                       1.1 MB/s | 600 kB     00:00
    (11/13): selinux-policy-3.14.3-20.el8.noarch.rpm                                                                                                                      936 kB/s | 602 kB     00:00
    (12/13): python3-policycoreutils-2.9-3.el8_1.1.noarch.rpm                                                                                                             1.3 MB/s | 2.2 MB     00:01
    (13/13): selinux-policy-targeted-3.14.3-20.el8.noarch.rpm                                                                                                             3.0 MB/s |  15 MB     00:05
    ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    Total                                                                                                                                                                 2.8 MB/s |  20 MB     00:07
    Running transaction check
    Transaction check succeeded.
    Running transaction test
    Transaction test succeeded.
    Running transaction
      Preparing        :                                                                                                                                                                              1/1
      Installing       : python3-libselinux-2.9-2.1.el8.x86_64                                                                                                                                       1/14
      Installing       : libselinux-utils-2.9-2.1.el8.x86_64                                                                                                                                         2/14
      Installing       : policycoreutils-2.9-3.el8_1.1.x86_64                                                                                                                                        3/14
      Running scriptlet: policycoreutils-2.9-3.el8_1.1.x86_64                                                                                                                                        3/14
      Installing       : rpm-plugin-selinux-4.14.2-25.el8.x86_64                                                                                                                                     4/14
      Installing       : selinux-policy-3.14.3-20.el8.noarch                                                                                                                                         5/14
      Running scriptlet: selinux-policy-3.14.3-20.el8.noarch                                                                                                                                         5/14
      Running scriptlet: selinux-policy-targeted-3.14.3-20.el8.noarch                                                                                                                                6/14
      Installing       : selinux-policy-targeted-3.14.3-20.el8.noarch                                                                                                                                6/14
      Running scriptlet: selinux-policy-targeted-3.14.3-20.el8.noarch                                                                                                                                6/14
      Installing       : python3-libsemanage-2.9-1.el8.x86_64                                                                                                                                        7/14
      Installing       : python3-setools-4.2.2-1.el8.x86_64                                                                                                                                          8/14
      Installing       : python3-audit-3.0-0.13.20190507gitf58ec40.el8.x86_64                                                                                                                        9/14
      Installing       : checkpolicy-2.9-1.el8.x86_64                                                                                                                                               10/14
      Installing       : python3-policycoreutils-2.9-3.el8_1.1.noarch                                                                                                                               11/14
      Installing       : policycoreutils-python-utils-2.9-3.el8_1.1.noarch                                                                                                                          12/14
      Running scriptlet: container-selinux-2:2.124.0-1.module_el8.1.0+272+3e64ee36.noarch                                                                                                           13/14
      Installing       : container-selinux-2:2.124.0-1.module_el8.1.0+272+3e64ee36.noarch                                                                                                           13/14
      Running scriptlet: container-selinux-2:2.124.0-1.module_el8.1.0+272+3e64ee36.noarch                                                                                                           13/14
      Installing       : containerd.io-1.2.6-3.3.el7.x86_64                                                                                                                                         14/14
      Running scriptlet: containerd.io-1.2.6-3.3.el7.x86_64                                                                                                                                         14/14
      Running scriptlet: container-selinux-2:2.124.0-1.module_el8.1.0+272+3e64ee36.noarch                                                                                                           14/14
      Running scriptlet: containerd.io-1.2.6-3.3.el7.x86_64                                                                                                                                         14/14
      Verifying        : container-selinux-2:2.124.0-1.module_el8.1.0+272+3e64ee36.noarch                                                                                                            1/14
      Verifying        : checkpolicy-2.9-1.el8.x86_64                                                                                                                                                2/14
      Verifying        : libselinux-utils-2.9-2.1.el8.x86_64                                                                                                                                         3/14
      Verifying        : policycoreutils-2.9-3.el8_1.1.x86_64                                                                                                                                        4/14
      Verifying        : policycoreutils-python-utils-2.9-3.el8_1.1.noarch                                                                                                                           5/14
      Verifying        : python3-audit-3.0-0.13.20190507gitf58ec40.el8.x86_64                                                                                                                        6/14
      Verifying        : python3-libselinux-2.9-2.1.el8.x86_64                                                                                                                                       7/14
      Verifying        : python3-libsemanage-2.9-1.el8.x86_64                                                                                                                                        8/14
      Verifying        : python3-policycoreutils-2.9-3.el8_1.1.noarch                                                                                                                                9/14
      Verifying        : python3-setools-4.2.2-1.el8.x86_64                                                                                                                                         10/14
      Verifying        : rpm-plugin-selinux-4.14.2-25.el8.x86_64                                                                                                                                    11/14
      Verifying        : selinux-policy-3.14.3-20.el8.noarch                                                                                                                                        12/14
      Verifying        : selinux-policy-targeted-3.14.3-20.el8.noarch                                                                                                                               13/14
      Verifying        : containerd.io-1.2.6-3.3.el7.x86_64                                                                                                                                         14/14
    
    Installed:
      containerd.io-1.2.6-3.3.el7.x86_64   container-selinux-2:2.124.0-1.module_el8.1.0+272+3e64ee36.noarch checkpolicy-2.9-1.el8.x86_64                         libselinux-utils-2.9-2.1.el8.x86_64
      policycoreutils-2.9-3.el8_1.1.x86_64 policycoreutils-python-utils-2.9-3.el8_1.1.noarch                python3-audit-3.0-0.13.20190507gitf58ec40.el8.x86_64 python3-libselinux-2.9-2.1.el8.x86_64
      python3-libsemanage-2.9-1.el8.x86_64 python3-policycoreutils-2.9-3.el8_1.1.noarch                     python3-setools-4.2.2-1.el8.x86_64                   rpm-plugin-selinux-4.14.2-25.el8.x86_64
      selinux-policy-3.14.3-20.el8.noarch  selinux-policy-targeted-3.14.3-20.el8.noarch
    
    Complete!

    Once containerd.io package is installed you can go ahead and install the docker without any error.

    [root@test_centos8 /]# yum install docker-ce
    Last metadata expiration check: 0:19:09 ago on Thu 02 Apr 2020 09:29:41 AM UTC.
    Dependencies resolved.
    ======================================================================================================================================================================================================
     Package                                                Architecture                           Version                                         Repository                                        Size
    ======================================================================================================================================================================================================
    Installing:
     docker-ce                                              x86_64                                 3:19.03.8-3.el7                                 docker-ce-stable                                  25 M
    Installing dependencies:
     iptables                                               x86_64                                 1.8.2-16.el8                                    BaseOS                                           586 k
     libcgroup                                              x86_64                                 0.41-19.el8                                     BaseOS                                            70 k
     libnetfilter_conntrack                                 x86_64                                 1.0.6-5.el8                                     BaseOS                                            65 k
     libnfnetlink                                           x86_64                                 1.0.1-13.el8                                    BaseOS                                            33 k
     libnftnl                                               x86_64                                 1.1.1-4.el8                                     BaseOS                                            83 k
     docker-ce-cli                                          x86_64                                 1:19.03.8-3.el7                                 docker-ce-stable                                  40 M
    
    Transaction Summary
    ======================================================================================================================================================================================================
    Install  7 Packages
    
    Total download size: 65 M
    Installed size: 276 M
    Is this ok [y/N]: y
    Downloading Packages:
    (1/7): libnetfilter_conntrack-1.0.6-5.el8.x86_64.rpm                                                                                                                  1.0 MB/s |  65 kB     00:00
    (2/7): libcgroup-0.41-19.el8.x86_64.rpm                                                                                                                               1.0 MB/s |  70 kB     00:00
    (3/7): libnfnetlink-1.0.1-13.el8.x86_64.rpm                                                                                                                           1.0 MB/s |  33 kB     00:00
    (4/7): libnftnl-1.1.1-4.el8.x86_64.rpm                                                                                                                                2.3 MB/s |  83 kB     00:00
    (5/7): iptables-1.8.2-16.el8.x86_64.rpm                                                                                                                               3.3 MB/s | 586 kB     00:00
    (6/7): docker-ce-19.03.8-3.el7.x86_64.rpm                                                                                                                             8.2 MB/s |  25 MB     00:02
    (7/7): docker-ce-cli-19.03.8-3.el7.x86_64.rpm                                                                                                                          10 MB/s |  40 MB     00:03
    ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    Total                                                                                                                                                                  14 MB/s |  65 MB     00:04
    warning: /var/cache/dnf/docker-ce-stable-091d8a9c23201250/packages/docker-ce-19.03.8-3.el7.x86_64.rpm: Header V4 RSA/SHA512 Signature, key ID 621e9f35: NOKEY
    Docker CE Stable - x86_64                                                                                                                                              39 kB/s | 1.6 kB     00:00
    Importing GPG key 0x621E9F35:
     Userid     : "Docker Release (CE rpm) <docker@docker.com>"
     Fingerprint: 060A 61C5 1B55 8A7F 742B 77AA C52F EB6B 621E 9F35
     From       : https://download.docker.com/linux/centos/gpg
    Is this ok [y/N]: y
    Key imported successfully
    Running transaction check
    Transaction check succeeded.
    Running transaction test
    Transaction test succeeded.
    Running transaction
      Preparing        :                                                                                                                                                                              1/1
      Installing       : libnfnetlink-1.0.1-13.el8.x86_64                                                                                                                                             1/7
      Running scriptlet: libnfnetlink-1.0.1-13.el8.x86_64                                                                                                                                             1/7
      Installing       : libnetfilter_conntrack-1.0.6-5.el8.x86_64                                                                                                                                    2/7
      Running scriptlet: libnetfilter_conntrack-1.0.6-5.el8.x86_64                                                                                                                                    2/7
      Installing       : docker-ce-cli-1:19.03.8-3.el7.x86_64                                                                                                                                         3/7
      Running scriptlet: docker-ce-cli-1:19.03.8-3.el7.x86_64                                                                                                                                         3/7
      Installing       : libnftnl-1.1.1-4.el8.x86_64                                                                                                                                                  4/7
      Running scriptlet: libnftnl-1.1.1-4.el8.x86_64                                                                                                                                                  4/7
      Running scriptlet: iptables-1.8.2-16.el8.x86_64                                                                                                                                                 5/7
      Installing       : iptables-1.8.2-16.el8.x86_64                                                                                                                                                 5/7
      Running scriptlet: iptables-1.8.2-16.el8.x86_64                                                                                                                                                 5/7
      Running scriptlet: libcgroup-0.41-19.el8.x86_64                                                                                                                                                 6/7
      Installing       : libcgroup-0.41-19.el8.x86_64                                                                                                                                                 6/7
      Running scriptlet: libcgroup-0.41-19.el8.x86_64                                                                                                                                                 6/7
      Installing       : docker-ce-3:19.03.8-3.el7.x86_64                                                                                                                                             7/7
      Running scriptlet: docker-ce-3:19.03.8-3.el7.x86_64                                                                                                                                             7/7
      Verifying        : iptables-1.8.2-16.el8.x86_64                                                                                                                                                 1/7
      Verifying        : libcgroup-0.41-19.el8.x86_64                                                                                                                                                 2/7
      Verifying        : libnetfilter_conntrack-1.0.6-5.el8.x86_64                                                                                                                                    3/7
      Verifying        : libnfnetlink-1.0.1-13.el8.x86_64                                                                                                                                             4/7
      Verifying        : libnftnl-1.1.1-4.el8.x86_64                                                                                                                                                  5/7
      Verifying        : docker-ce-3:19.03.8-3.el7.x86_64                                                                                                                                             6/7
      Verifying        : docker-ce-cli-1:19.03.8-3.el7.x86_64                                                                                                                                         7/7
    
    Installed:
      docker-ce-3:19.03.8-3.el7.x86_64     iptables-1.8.2-16.el8.x86_64             libcgroup-0.41-19.el8.x86_64     libnetfilter_conntrack-1.0.6-5.el8.x86_64     libnfnetlink-1.0.1-13.el8.x86_64
      libnftnl-1.1.1-4.el8.x86_64          docker-ce-cli-1:19.03.8-3.el7.x86_64
    
    Complete!

    Once docker is successfully installed you can run docker -v to check the installed version.

    [root@test_centos8 /]# docker -v
    Docker version 19.03.8, build afacb8b

    Hope this helps everyone (and saves productive time).

  • [WARNING]: The value 1 (type int) in a string field was converted to u’1′ (type string)

    [WARNING]: The value 1 (type int) in a string field was converted to u’1′ (type string)

    I have seen this error multiple times during my Ansible playbooks execution but always ignored it because I always got the expected result.

    Eventually, I wanted to get rid of any errors/warnings from my final version of the playbook and wanted to take care of this error as well. Hence I had to do little readings in this error/warning.

    [WARNING]: The value 1 (type int) in a string field was converted to u'1' (type string). If this does not look like what you expect, quote the entire value to ensure it does
    not change.

    As per the Ansible official documentation this is the Default behavior of starting Ansible version 2.8. Ansible will warn if a module expects a string, but a non-string value is passed and automatically converted to a string.

    For example – If your playbook inputs a value version number 1.10 (parsed as float value) would be converted to '1.1'. Such conversions can result in unexpected behavior depending on context and also will/might change playbook’s expected outcomes

    There are two solutions to address this issue.

    1. ANSIBLE_STRING_CONVERSION_ACTION Environment Variable

    This behavior can be changed to be an error or to be ignored by setting the ANSIBLE_STRING_CONVERSION_ACTION environment variable, or by setting the string_conversion_action configuration in the defaults section of ansible.cfg.

    You can refer to this link to change the string_conversion_action parameter in defaults section of Ansible.cfg. This will make sure Ansible will not change the input to String

    2. Quote the input values in playbook

    I personally prefer this method instead of editing Ansible.cfg file. To make sure you’re not getting string field conversion Warning message you can simply quote the input value. Below is the example for the same.

      - name: Sample sysctl task
        sysctl:
         name: net.bridge.bridge-nf-call-iptables
         value: '1'
         state: present

    Below are the playbook screen captures of before and after quoting the values in playbook

    The value 1 (type int) in a string field was converted to u'1' (type string) - Before Solution
    The value 1 (type int) in a string field was converted to u’1′ (type string) –
    Before making changes
    The value 1 (type int) in a string field was converted to u'1' (type string) - After Solution
    The value 1 (type int) in a string field was converted to u’1′ (type string) – After making changes

    I hope this helps everyone.