There are many blogs which cover what is Ansible and why we should care. But this multi-part blog series will cover how DellEMC and Ansible can help you automate many storage tasks.
Ansible is being used in many organizations nowadays to manage a vast range of infrastructure, ranging from traditional configuration management to cloud resources in public clouds to physical infrastructure such as network and storage devices. There are multiple reasons why you might want to use Ansible, including – but not limited to – below examples.
You’re trying to have efficient DevOps environment
Automate Day 1/2 tasks
Need centralized configuration management
Getting Started with Ansible
Even if you’ve not installed Ansible before, it’s very easy to get started. There are many different ways to install Ansible and many dependencies will be needed based on your environment and systems you’ll be managing.
Once your Linux machine is up and running (CentOS in my case) you can follow next steps
Apart from Ansible itself there will be many dependencies which needs to be installed. In the case of DellEMC PowerMax storage we will need
In this blog we will talk about how DellEMC can help you to store you databases smartly and efficiently.
But firstly let’s talks about CDM and why it’s important. In the world of DBAs everyone’s always talking about performance, million IOPS and double/triple digit bandwidth with lowest possible latency. But the biggest question is whether performance is the only solution to all problems?
For example in one of the survey members of Independent Oracle User Group (IOUG) were asked what DB activities are taking up most of the time each week?
You can also see that performance troubleshooting is rated as second most time consuming activity, but the question is how to solve it. One of the ways people try to solve this is by upgrading their database stack. In below AWR example snippets you can see that how one of the customer has solved their performance issue by moving to DellEMC solution. Earlier they were having 92% User I/O wait time, which also means 92% of the time DB is waiting for database subsystem to respond.
But as I have mentioned earlier upgrading the stack will always give the benefits but it does not solve all the problems. Which also means most customer apply their older process and best practices to new stack which often is the culprit of the new problems. (another blog on this later)
But the real problem is what do you do with those databases and data? Most businesses today are trying to keep more data online. This allows them to understand the patterns and anomalies using AI/ML, which helps them taking right decisions or course correction. All this is driven using copy data, which means you need actual source production database instances to analyze, test, develop etc. Below picture is the great depiction of multiple use cases which needs source database.
Now think of all those use cases and start thinking of the approach DBAs will take to manage the show. Usually there’s no standard way of managing this. Processes to achieve the end goal depends on the experience of the DBA and also availability of the resources.
IDC has done some work on this problem area to understand the gravity of the situation and uncover some facts. IDC polled over 700 customers for this survey. Some of the interesting facts are –
Tools used for producing secondary data copies
As per the IDC survey most popular way is to rely on backup software but interestingly 53% of the respondents said that they use custom-written scripts to manage the CDM show.
How long does it take to refresh copies?
To which 62% of organizations, copies take ½ day or more to create (up to days). And this is very real as I have personally worked with one of my customer who used to take 2 ½ days to create single secondary copy starting weekend, on which 60 member developer team used to work.
At which frequency are copies of dev/test or analytics refreshed?
Copies are refreshed every few days by 32% of organizations, and weekly by 42%.
One of the most striking detail in this IDC document is – How many Database instances do you currently have in production in your data center environment? 77% customers said that they have more that 200 database instances and 82% said that they have more than 10 copies per database.
In simplified words this problem looks like the iceberg. People only count their production/primary database instances but secondary/re-purpose copies are usually taken for granted. If you do the quick math on IDC numbers you’ll quickly realize that this is one of the biggest operational issue in database environments.
For Example –
1 Production DB x 250GB DB size x 346 DB Instances = 87TB of Production data
14 Copies of each Production DB x 250GB DB size x 346 DB Instances = 1.2PB of Copy Sprawl
Now that we understand what copy data management issue is. Now let’s understand how DellEMC can help in simplifying the CDM problem
Integrated Copy Data Management (iCDM)
As per my personal experience below snippet is the best way to understand how iCDM functions. Providing self service to application/database teams with application integration. iCDM leverages advanced copy/snapshot function of the DellEMC storage system, which also means you need DellEMC storage in order to use iCDM 🙂
To simplify things even further – in most environments you follow 3 steps process for database copy creation.
Identify source database
Prepare database
Prepare instance
Details for these 3 steps are as below but on high level it lacks automation, it’s (very) time consuming and most importantly it lacks automation.
But with iCDM you can change this legacy process and adapt to something which is more modern and fits with today’s cloud native world. You can simply
Containers are everywhere and they are here to stay. They are great level-playing ground to break the infrastructure dependency and allow developers to release their code to any environment.
Containers also help customers to operate at greater scales with ability to quickly scaling up and down, patching with disruptions, withstand infrastructure component failures, moving easily from on-premises to public clouds, etc.
As per the survey from sysdig lifespan of containers and container images is also very short.
At this stage of the popularity of containers there are two thought processes in the container fan club – Persistent or Non-Persistent Containers
If you explore docker hub top downloads you’ll notice that 7/10 top downloads require data persistence (snippet below)
Let’s understand persistent containers in more details.
Prior to Container Storage Integration CSI, Kubernetes provided in-tree (ie as part of the core code) plugins to support volumes but that posed a problem in that storage vendors had to align to the Kubernetes release process to fix a bug or to release new features among other problems. This also means every storage vendor had their own process to present volumes to Kubernetes.
This heterogeneous non-standard integrations were one of the biggest reasons why CSI was created. CSI was developed as a standard for exposing block and file storage storage systems to containerized workloads on Container Orchestration Systems (COs) like Kubernetes. With the adoption of the Container Storage Interface (CSI), the Kubernetes volume layer becomes truly extensible. Using CSI, third-party storage providers like DellEMC can write and deploy plugins exposing new storage systems in Kubernetes without ever having to touch the core Kubernetes code. This gives Kubernetes users more options for storage and makes the system more secure and reliable. Also this approach makes sure that every vendor has standard way of interacting with Kubernetes.
With CSI Kubernetes supports Persistent Volumes (PV). PVs life-cycle independent of any Kubernetes POD. Kubernetes supports 2 ways to provision PVs
Static – Admin Pre-provisions / creates a number of PVs
Static PV provisioning
Dynamic – Cluster “automatically” provisions a volume
Dynamic PV provisioning
No matter which is the method of PV provisioning it can support varying properties such as performance, QOS, backup policies, etc. These properties are defined by StorageClass
There are 3 access modes which are supported on PV. Storage volume cannot be mounted simultaneously in more than one access mode.
ReadWriteOnce (RWO) – Volume can be mounted as ready-write by a single node
ReadOnlyMany (ROX) – Volume can be mounted read-only by many nodes
ReadWriteMany (RWX) – Volume can be mounted as read-write by many nodes
Below is the summary of persistence
DellEMC CSI Support
DellEMC understands that that Enterprise applications require persistent storage. As of now (Nov 2019) DellEMC supports CSI plugins for below storage arrays
In daily life of an administrator worst day is when you’ve to co-relate logs across the stack for performance troubleshooting or when you need to troubleshoot an issue which is not consistent. Usually for such issues you’ll need to enable continuous log collection and hope that issue will reappear during the log collection time. Phew!
But gone are those days where you were reactive in troubleshooting, now with CloudIQ you can be proactive.
CloudIQ is SaaS based monitoring application which is available to all DellEMC customers without any additional investment.
Though CloudIQ allows customers to have centralized monitoring of DellEMC storage products, proactive health score and predictive analytics, but this blog is about VMware monitoring with CloudIQ. With latest update of CloudIQ now it supports monitoring of entire VMware landscape (along with storage and Connectrix).
As I have already mentioned CloudIQ has integration with VMware for end-to-end monitoring for the virtualization environments. For enabling the CloudIQ to collect and report VMware data you need to download the CloudIQ collector, which can be downloaded directly from the CloudIQ portal (details below)
Once the collector is downloaded follow the steps (step 3 in above exhibit) listed on the CloudIQ collector download page.
Note that single instance of CloudIQ collector supports collecting data for upto 60000 virtual machines. Once collector is enabled it will take ~24 hours to populate the data on CloudIQ portal.
There are many details available on CloudIQ and more enhancements are getting added very frequently. You can refer to this CloudIQ whitepaper which has details around CloudIQ overview. In this blog post is to highlight few CloudIQ features which I personally like.
Storage LUN to Datastore to VM Mapping
From the CloudIQ you can see the list of all the virtual machines which running on particular datastore. This is great asset as you no more have to login to vCenter console for this.
Storage LUN –> VMware Datastore –> VM Mapping
Co-relate VM Performance With Anomaly Detection
You can view all required performance charts for virtual machine in single view. Additionally CloudIQ performance charts will highlight any performance anomalies during the last 24 hours. In below example you can see that for selected virtual machine storage response time was suddenly increased and went back to normal.
Co-relate VM Performance With Anomaly Detection
End to End Map
Customers who have tried to deploy monitoring tools understand that it’s very difficult to get the application/virtual machine footprint map in the data center. Usually this is very useful if you want to understand all the dependent infrastructure components which you application touches in the virtualization stack. CloudIQ gives you out of box functionality to provide virtual machine end-to-end map — from Virtual machine to storage volumes/LUNs
These are the top 3 VMware monitoring features which I personally like. Apart from this there are many features including but not limited to
Tracking configuration changes per VM level
Storage path details for each datastore
Also I would like to mention that CloudIQ can be accessed using
Web view using any device having connectivity to CloudIQ
Mobile application – Supported on Apple and Android devices
