Tag: Cloud

  • Infrastructure as Code (IaC) Security Best Practices

    Infrastructure as Code (IaC) Security Best Practices

    Infrastructure as Code (IaC) has revolutionized how we provision and manage infrastructure, enabling speed, repeatability, and scalability. However, as with any code, IaC introduces new security considerations. Misconfigurations, exposed secrets, and lack of visibility can lead to significant risks in production environments.

    This guide explores actionable best practices for securing your Terraform, Ansible, and other IaC pipelines-helping you build robust, compliant, and resilient infrastructure. Sample code and how-to notes included!


    1. Treat IaC Like Application Code

    Just as you would with application code, store your IaC in version control systems (e.g., Git). This enables:

    • Change tracking: Who changed what, when, and why.
    • Peer reviews: Enforce code reviews and approvals before merging.
    • Rollback: Revert to previous known-good states if issues arise.

    Tip:
    Use branch protection rules and require pull request reviews for all changes.


    2. Secrets Management

    Never hard-code secrets or sensitive data (API keys, passwords, certificates) in your IaC files. Instead:

    • Use secret management tools like HashiCorp Vault, AWS Secrets Manager, or Azure Key Vault.
    • Integrate secrets into your pipelines at runtime, not in code.
    • Leverage environment variables or encrypted files for sensitive values.

    Sample: Fetching Secrets from Vault in Terraform

    provider "vault" {
      address = "https://vault.example.com"
    }
    
    resource "vault_generic_secret" "example" {
      path = "secret/data/myapp"
    }
    
    output "db_password" {
      value = vault_generic_secret.example.data["password"]
    }

    How-to:

    • Replace https://vault.example.com with your Vault server address.
    • The vault_generic_secret resource fetches secrets dynamically.
    • Output blocks can reference secrets securely-never hardcode them!

    Sample: Encrypting Secrets in Ansible with ansible-vault

    - hosts: all
      vars_files:
        - secrets.yml
      tasks:
        - name: Use secret password
          debug:
            msg: "The password is {{ secret_password }}"

    How-to:

    • Create and encrypt the secrets file: ansible-vault create secrets.yml
    • Reference the encrypted file in your playbook under vars_files.
    • Access secrets in tasks using Jinja2 templating, e.g., {{ secret_password }}.

    3. Automated Security Scanning

    Integrate security tools into your CI/CD pipeline to catch misconfigurations and vulnerabilities early:

    Sample: Terraform Security Scanning with tfsec in GitHub Actions

    name: Terraform Security Scan
    
    on: [push]
    
    jobs:
      tfsec:
        runs-on: ubuntu-latest
        steps:
          - uses: actions/checkout@v2
          - name: Run tfsec
            uses: aquasecurity/tfsec-action@v1
            with:
              tfsec_version: 'latest'

    How-to:

    • Add this workflow to your .github/workflows directory.
    • The scan will fail the build if critical security issues are found, helping you catch problems early.

    4. Principle of Least Privilege

    Ensure that your IaC tools, pipelines, and the resources they provision follow the principle of least privilege:

    • Limit IAM permissions for automation accounts.
    • Avoid using overly broad roles or root accounts.
    • Regularly audit permissions and remove unnecessary access.

    5. Policy as Code

    Define and enforce security and compliance policies programmatically:

    • Use tools like Open Policy Agent (OPA), Sentinel (for Terraform Cloud), or Conftest.
    • Enforce rules such as “no public S3 buckets,” “encryption enabled,” or “no hardcoded credentials.”

    Example:
    OPA policies can block deployments that violate security standards before they reach production.


    6. Continuous Monitoring and Drift Detection

    Even after deployment, infrastructure can drift from the desired state:

    • Use tools like Terraform Cloud, AWS Config, or Driftctl to detect and remediate drift.
    • Set up alerts for unauthorized changes and automate remediation where possible.

    7. Regular Reviews and Updates

    IaC modules and dependencies evolve-so should your security practices:

    • Schedule regular reviews of your IaC codebase and third-party modules.
    • Update modules to patch vulnerabilities and leverage new security features.
    • Document your security practices and train your team.

    Conclusion

    Securing Infrastructure as Code is not a one-time task, but a continuous process woven into your development and deployment lifecycle. By treating IaC like application code, managing secrets securely, automating scanning, enforcing least privilege, and using policy as code, you can drastically reduce risk and build trust in your automation.

  • Notifications in Ansible Tower, AAP and AWX

    We have published a few videos in the IaC Avengers YouTube channel about automation using Ansible and how this can be integrated into an ITSM tool like ServiceNow to provide a cloud-like experience for private infrastructure. In the last two videos we have even demonstrated how to treat ServiceNow as the single pane of glass to consume both private and public clouds. This approach provides a much needed unified governance and cost control in a multicloud environment.

    However, while showing the demos and having conversations with customers, I can see a question coming up more often. How do we cope with errors? It makes sense that this question is coming up now. We are taking the automation conversation out of the realm of the datacenter and elevating it all the way to the end-user in the ITSM world. This means “Enterprise” requirements, which in turn means less room for failure. Also, if we expose it to the end-user we are no longer talking about dozens of engineers, now we have potentially thousands of possible consumers.

    A sample architecture like shown in the videos is as follows:

    The requirements are:

    • let the user know that the workflow didn’t complete so that they are not sitting there waiting. Depending on the error they might want to retry
    • inform the engineers that a specific workflow is failing and they need to look into it

    This can and should be done both at the ServiceNow level and the Ansible level. In this post we are going to focus on the Ansible side of things. Most mature organizations use RedHat Ansible Automation Platform. I have also included the old name Ansible Tower because somehow is still stuck in people’s heads … it is certainly shorter and easier to pronounce. Of course this is also applicable to AWX, the community support edition

    From an Ansible syntax perspective you can do error handling with things like “blocks and rescue” or other techniques. However, our guiding principle here is not so much to make sure the playbook continues despite errors and ends gracefully. What we want in this case is to make sure that both the engineer and/or user gets notified. For this purpose I find the “Notifications” functionality does the job nicely. You can find “Notifications” on the left bar under the Administration menu. If you click the “Add” button you get a menu like this

    After providing a name you need to select the notification “Type”. Depending on your selection a number of relevant configuration options are shown. For example if email is selected it will ask for IP and port of the SMTP server and so on. Once you fill those details scroll to the very bottom and slide the “Customize messages” button. This will reveal the syntax of the notification messages. The tool supports sending notification on 7 different types of events including start, error, time out and even the outcome of an approval. Notice how the prepopulated messages use variables with the double curly bracket syntax.

    In my example I have created a notification to send emails to a Zimbra SMTP server we have in the lab. As you saw in the previous image is called “Zimbra email”. For testing purposes I have created a job template that runs a playbook called “wrong.yml”. This is single task playbook that uses the “uri” module to access a webpage. I have fed the task with an IP address that doesn’t exist, so the playbook will fail

    From the template we click in the “Notifications” tab. It will show you all the notifications you have configured. In my example “Zimbra email” is the only one. On the right side you will have the opportunity to enable any of the available notifications when the template starts, succeeds or fails. If you do the same thing for a “workflow template” it will show an additional slide button named “Approval”

    All is left to do is to run the template. When I run it fails as expected and I get an email in my Inbox with the following message. Notice how the body of the email maps to the syntax we saw in the “Customize messages” menu

    These messages could be sent to a group of engineers that look after the platform. I particularly like the fact that there is a “Webhook” type. This opens the possibility of sending a notification to a Teams channel which is a more popular choice than email these days. You can see in this previous post how to send notifications to a Teams channel. Additionally it would make sense to create an incident automatically in your ITSM tool. Some time ago we also published a tutorial to show you how to create incidents in ServiceNow programmatically.

  • Monitoring your VMware Environment From Anywhere Using DellEMC CloudIQ

    Monitoring your VMware Environment From Anywhere Using DellEMC CloudIQ

    In daily life of an administrator worst day is when you’ve to co-relate logs across the stack for performance troubleshooting or when you need to troubleshoot an issue which is not consistent. Usually for such issues you’ll need to enable continuous log collection and hope that issue will reappear during the log collection time. Phew!

    But gone are those days where you were reactive in troubleshooting, now with CloudIQ you can be proactive.

    CloudIQ is SaaS based monitoring application which is available to all DellEMC customers without any additional investment.

    Though CloudIQ allows customers to have centralized monitoring of DellEMC storage products, proactive health score and predictive analytics, but this blog is about VMware monitoring with CloudIQ. With latest update of CloudIQ now it supports monitoring of entire VMware landscape (along with storage and Connectrix).

    As I have already mentioned CloudIQ has integration with VMware for end-to-end monitoring for the virtualization environments. For enabling the CloudIQ to collect and report VMware data you need to download the CloudIQ collector, which can be downloaded directly from the CloudIQ portal (details below)

    Once the collector is downloaded follow the steps (step 3 in above exhibit) listed on the CloudIQ collector download page.

    Note that single instance of CloudIQ collector supports collecting data for upto 60000 virtual machines. Once collector is enabled it will take ~24 hours to populate the data on CloudIQ portal.

    There are many details available on CloudIQ and more enhancements are getting added very frequently. You can refer to this CloudIQ whitepaper which has details around CloudIQ overview. In this blog post is to highlight few CloudIQ features which I personally like.

    Storage LUN to Datastore to VM Mapping

    From the CloudIQ you can see the list of all the virtual machines which running on particular datastore. This is great asset as you no more have to login to vCenter console for this.

    Storage LUN –> VMware Datastore –> VM Mapping

    Co-relate VM Performance With Anomaly Detection

    You can view all required performance charts for virtual machine in single view. Additionally CloudIQ performance charts will highlight any performance anomalies during the last 24 hours. In below example you can see that for selected virtual machine storage response time was suddenly increased and went back to normal.

    Co-relate VM Performance With Anomaly Detection

    End to End Map

    Customers who have tried to deploy monitoring tools understand that it’s very difficult to get the application/virtual machine footprint map in the data center. Usually this is very useful if you want to understand all the dependent infrastructure components which you application touches in the virtualization stack. CloudIQ gives you out of box functionality to provide virtual machine end-to-end map — from Virtual machine to storage volumes/LUNs

    These are the top 3 VMware monitoring features which I personally like. Apart from this there are many features including but not limited to

    • Tracking configuration changes per VM level
    • Storage path details for each datastore

    Also I would like to mention that CloudIQ can be accessed using

    • Web view using any device having connectivity to CloudIQ
    • Mobile application – Supported on Apple and Android devices