A vendor datasheet promises a million IOPS. You buy the array, point your workload at it, and it’s slow. The number wasn’t a lie — it was just irrelevant to you. That gap, between an impressive benchmark and a disappointing production system, is where a lot of infrastructure money quietly goes to die.
I’ve spent a large part of my career with fio, vdbench, and HammerDB open in front of me. The tools are easy. Getting an honest number out of them is not. Here’s the field guide I wish more teams had before they trusted a benchmark — their own or a vendor’s.
The number isn’t wrong. It’s answering a different question.
Peak IOPS on a datasheet is a real measurement — of a workload that looks nothing like yours. Tiny block size, unlimited queue depth, a working set small enough to live entirely in cache, reads only, measured for ten seconds. Your production is a 70/30 read-write mix, 16K blocks, a working set far larger than cache, and it has to stay fast at 2 a.m. on day 400. Same tool, different universe. Before you argue about whose array is faster, make sure you’re both describing the same planet.
The traps that inflate a benchmark
Most misleading numbers come from a short list of mistakes:
The working set fits in cache. You benchmarked DRAM, not the media. Size the dataset several times larger than the controller cache or you’re measuring the wrong component.
No steady state. Flash gets slower once garbage collection kicks in. A 60-second run flatters an SSD that looks very different an hour later. Precondition the device, then measure.
Queue-depth theater. Cranking queue depth to 256 maximizes IOPS and obliterates latency. It produces a big number and an unusable response time.
Wrong block size or mix. The datasheet uses 4K reads; your database does 8K–16K with real writes. Model the actual mix or the result is fiction.
A single sample. One run is an anecdote. The variance across five runs is the actual story.
An honest fio job to start from
[global]
ioengine=libaio
direct=1
runtime=600
time_based=1
ramp_time=60 # reach steady state before recording
group_reporting=1
[db-like]
rw=randrw
rwmixread=70 # 70/30 read-write, like an OLTP database
bs=16k # your real block size, not 4k
iodepth=32 # realistic, not a vanity QD of 256
numjobs=4
size=200g # larger than the array's cache
percentile_list=99:99.9 # report the tail, not just the average
Every knob here is a decision about honesty. direct=1 bypasses the page cache so you measure storage, not memory. ramp_time throws away the artificially fast warm-up. size forces cache misses. percentile_list is the one most people skip — and it’s the one that matters most.
Read the right metrics
Peak IOPS is the vanity metric. What actually predicts whether production will be happy:
Latency at your target throughput — not throughput at unlimited latency. Those are different questions with very different answers.
Tail latency (p99, p99.9). Your users feel the worst 1% of requests, not the average. A great mean with an ugly tail is a bad system wearing a good costume.
Consistency over time. Does it hold at steady state, or degrade as the device fills and ages?
One reframing kills most bad purchases: fix a latency budget — say, 1 ms at p99 — and ask “how many IOPS can it sustain at or under that?” Suddenly the million-IOPS array and the “slower” one often trade places.
Why this is a leadership discipline, not a lab chore
Benchmarking is how you replace opinion with evidence. In a room full of vendor claims and strong personalities, the person holding a reproducible number wins the decision — and deserves to. That’s also why a benchmark is a conviction with a review date: you state a performance expectation strongly enough to plan around it, and you re-run it when the firmware, the workload, or the scale changes. A claim without a benchmark is just an opinion. A benchmark you can’t reproduce is just a different opinion with a chart attached.
The takeaway
Don’t ask “how fast is it.” Ask “how fast is it, running my workload, at my latency budget, at steady state, averaged over five runs.” The tools will answer honestly if you ask honestly. Everything else is marketing with a monospaced font.
The full fio job file is on GitHub: github.com/waghmaredb/vexpose-labs. If you benchmark enterprise storage or databases for a living, I’d like to compare methodologies — reach me on LinkedIn or X.
Enterprise storage arrays from vendors like Dell EMC, NetApp, and Pure Storage are the foundation of mission-critical IT environments. Ensuring these systems deliver consistent, high performance under real-world workloads is essential for application reliability and business continuity. In this post, we’ll explore how to benchmark enterprise storage using two industry-leading tools- Fio and Vdbench– with practical configuration examples and best practices.
Why Benchmarking Enterprise Storage is Unique
Enterprise arrays are not just fast disks-they’re complex systems with:
Benchmarking these systems is different from testing a single SSD or HDD. You must simulate production-like workloads, test at scale, and observe performance under both normal and failure conditions.
Best Practices for Enterprise Storage Benchmarking
Simulate real-world workloads: Use realistic mixes of random/sequential I/O, read/write ratios, and block sizes.
Test at scale: Ensure test data sets exceed cache sizes and run with sufficient concurrency.
Run long-duration tests: Observe steady-state performance, not just short-term cache hits.
Monitor the full stack: Track storage, network, and server metrics.
Document everything: Record hardware/software versions, configurations, and test parameters.
Include failure scenarios: Simulate controller or network failures to test resilience.
Fio: Flexible I/O Tester
Fio is a versatile, scriptable tool ideal for generating a wide range of I/O workloads against enterprise storage.
Example Fio Job File for Enterprise Storage
text[global] ioengine=libaio direct=1 rw=randrw rwmixread=70 bs=8k iodepth=64 numjobs=8 size=100G runtime=3600 time_based group_reporting filename=/dev/sdx # Replace with your LUN or device
[verify] verify=crc32
What this does:
Simulates a 70% read/30% write random workload with 8KB blocks.
Runs 8 parallel jobs with a queue depth of 64 for 1 hour.
Uses direct I/O to bypass the OS cache.
Verifies data integrity with CRC32.
Run it with:
sudo fio enterprise_test.fio
Tips:
Use multiple devices or files to simulate multi-volume workloads.
Adjust concurrency (numjobs, iodepth) to match your environment.
Use --output-format=json for detailed reporting.
Vdbench: Enterprise Workload Generator
Vdbench is designed for complex, multi-host enterprise storage validation and offers granular workload definition and data validation.
Example Vdbench Configuration
text# Storage Definitions for multiple LUNs sd=sd1,lun=/dev/sdb,size=100g,openflags=o_direct sd=sd2,lun=/dev/sdc,size=100g,openflags=o_direct sd=sd3,lun=/dev/sdd,size=100g,openflags=o_direct sd=sd4,lun=/dev/sde,size=100g,openflags=o_direct
# Workload Definition: 67% read, 33% write, 8KB random wd=wd1,sd=(sd1-sd4),xfersize=8k,rdpct=67,seekpct=100
# Run Definition: max I/O, 24 hours, 1s reporting rd=rd1,wd=wd1,iorate=max,elapsed=86400,interval=1
Run it with:
vdbench -f enterprise_vdbench.conf
Advanced tips:
Use the hd section to define multiple hosts for distributed testing.
Simulate failures (e.g., disconnect a path or controller) during the run to observe failover behavior.
Use fsd and fwd for NAS/file workloads.
Key Steps for Success
Profile your workload: Know your application’s I/O patterns. (capture the workload IO pattern)
Prepare your environment: Use dedicated test LUNs/volumes.
Configure your tools: Use Fio or Vdbench job files that match your workload.
Run and monitor: Capture storage, host, and network metrics.
Analyze results: Look for steady-state performance, latency spikes, and the impact of failures.
Document and repeat: Ensure tests are reproducible and results are transparent.
Conclusion
Benchmarking enterprise-class storage is about more than just peak numbers-it’s about understanding how your array performs under pressure, during failures, and with your real workloads. Tools like Fio and Vdbench provide the flexibility, power, and validation features needed for accurate, actionable results. By following best practices and using realistic configurations, you can ensure your storage infrastructure is ready for the demands of the modern enterprise.
In the world of software development, Murphy’s law holds an unassailable truth: Anything that can go wrong, will go wrong. As a proud member of this masochistic club, you might be looking for innovative ways to stall your Kubernetes enterprise rollout. Maybe you want to add a little chaos to your routine CI/CD workflow, or perhaps you’re just a thrill-seeker who loves the high stakes game of orchestration roulette. Either way, you’ve come to the right place. Sit back, relax, and let us guide you through the delightful maze of missteps and detours that will ensure your Kubernetes enterprise rollout is anything but a walk in the park.
Ah, Kubernetes! The open-source platform that’s become the equivalent of a Hollywood blockbuster in the tech world. It’s like the Iron Man of container orchestration, bringing together an array of superpowers including automation, scaling, and management of container deployment. Enterprises are lining up to get their tickets, excited by the promises of streamlined application deployment. But before you go head over heels for Kubernetes, remember, even Iron Man had his quirks. Navigating the CI/CD waterfall can sometimes feel more like a rollercoaster ride without a seatbelt. So before you charge headfirst into your enterprise rollout, take a moment to consider Murphy’s Law – anything that can go wrong, will go wrong. So buckle up, my friends, it’s going to be a wild ride.
Indeed, Kubernetes is a boon for developers, cloud-native architects, and business owners alike. Its versatility and flexibility can make you feel like a superhero orchestrating seamless deployments. But when it comes to deploying and scaling in Enterprise data centers, Kubernetes might just swap its Iron Man suit for a Godzilla costume, spawning fresh challenges born out of its cloud-native architecture. This transition can lead to excessive mental gymnastics as you grapple with these new beasts of burden. So, if you’re feeling like a deer caught in the headlights, staring down the Kubernetes-python poised to gobble up your Enterprise applications, fear not! This blog is your sanctuary, your guide, your ‘how-to-tame-your-dragon’ manual. Stay with us, as we venture into the labyrinth of Kubernetes deployment and come out the other side grinning. 🙂
Let’s dive into the 10 ways you might unintentionally stall your enterprise Kubernetes rollout, and inadvertently send your organization spiraling back to the ‘golden age’ of monolithic architecture:
1.Do Not Plan Your Deployment: Some may argue that the beauty of Kubernetes lies in its simplicity, and indeed, the internet is abundant with blogs and videos promoting the notion that the deployment process is a walk in the park. Following such advice without investing time in understanding your unique use case could be a significant pitfall. Kubernetes deployments require thoughtful planning, taking into account the intricacies of workload requirements, resource allocation, and network architecture. The idea of “Kubernetes is the easy button for everything” is a perilous assumption that can easily derail your enterprise rollout. Always remember, while Kubernetes does a spectacular job in many aspects, it’s not a one-size-fits-all solution for every enterprise problem.
2. Avoid Using Certified Kubernetes Distributions: Now this one’s a head-scratcher, isn’t it? Here’s the thing, though: Kubernetes is powerful, flexible, and can be customized to a dizzying degree. However, this does not mean you should do everything from scratch. Consider this – why would you build your car when you can buy a perfectly good one off the lot? Certified Kubernetes distributions, like Red Hat OpenShift or VMware Tanzu, come with the assurance of being properly configured, tested, and meeting industry standards. They’re like your ready-to-drive cars, offering robust features and world-class support. By choosing to bypass these options, you’re essentially signing up for unnecessary headaches that could easily stall your enterprise rollout. Remember, Kubernetes is a tool, not an ideology. There’s no virtue in unnecessary complexities.
3. Do NOT Implement Security Best Practices: This one’s a classic misstep in the tech world. Yes, Kubernetes is inherently secure, but that doesn’t mean it’s invincible. If you’re looking to stall your enterprise rollout, then by all means, ignore security best practices. However, if you’re keen on a smoothly functioning system, pay close attention to security measures like access control, network segmentation, and encryption. It’s akin to leaving your car unlocked in a crowded parking lot — sure, it might have an immobilizer and alarm system, but why invite trouble? Access control ensures only authorized personnel can interact with your Kubernetes clusters, network segmentation limits the blast radius of potential breaches, and encryption keeps your sensitive data safe in transit and at rest. Failing to implement these measures is like leaving the keys in your car with the engine running – a surefire way to invite mischief. Remember, in the world of Kubernetes deployments, security is not an afterthought, it’s a primary driver of successful CI/CD pipelines and enterprise rollouts.
4. Forget about Using CI/CD Pipelines: There’s a certain masochistic charm in choosing not to use CI/CD pipelines in your enterprise rollout. After all, who needs automation when you can manually deploy your applications, right? CI/CD pipelines, or Continuous Integration/Continuous Deployment pipelines, are like that studious classmate who always double-checks their work before submitting it – they automate the deployment process and ensure that all changes are thoroughly examined and validated before entering the production environment. If you’re a fan of chaos and unpredictability (and potentially stalling your Kubernetes enterprise rollout), then by all means, go ahead and give CI/CD pipelines a pass. However, if you value efficiency, reliability, and sanity, incorporating CI/CD pipelines into your operations could be a game-changer. They ensure a streamlined, error-free process that keeps your applications updated and secure, allowing your team to focus on what truly matters – building and improving your products. But hey, if you’re in the market for a bit of pandemonium, feel free to ignore this advice!
5.Visibility/Observability, what’s that?: There’s something rather intriguing about stumbling in the dark, isn’t there? For those of you who enjoy a good surprise, why not apply this approach to your Kubernetes deployment? Think about it, with no comprehensive monitoring solution in place, every day is like a thrilling game of hide-and-seek with your application’s performance, capacity, and availability. However, if you (like most sane people) prefer to know what’s happening under the hood, it’s time to incorporate a robust monitoring solution into your Kubernetes enterprise rollout. Think of it as a reliable co-pilot that keeps an eye on the road while you’re busy steering the ship. It helps you identify potential roadblocks or speed bumps, ensuring your journey toward a successful enterprise rollout is as smooth as possible. So go ahead, embrace the unknown, or better yet, ensure your unknowns are known with a comprehensive monitoring solution. But remember, no pressure; after all, it’s only your Kubernetes deployment we’re talking about here!
6.Don’t care about Config Management Tools: If you’re fond of unpredictability and enjoy the thrill of variance, throwing caution to the wind when it comes to config management could be your next adrenaline spike! Who needs tools like Ansible or Puppet that automate the configuration of your Kubernetes deployment and ensure consistent settings across your environment? Why make life easier and your enterprise rollout smoother when you can indulge in the chaotic symphony of inconsistency? Sure, these tools can simplify the management of your Kubernetes configuration, reduce errors, and ensure uniformity across your deployment, but where’s the fun in that? So sit back, relax, and let the inconsistencies rollick through your deployment, because who wouldn’t love a good configuration surprise?
7.Forget about Disaster Recovery: If you’re the type who loves to live on the edge, why not take a leap of faith with your Kubernetes rollout too? After all, implementing disaster recovery measures like backup and recovery procedures is like carrying an umbrella all the time just because it might rain. Sure, these measures could prevent your enterprise from figuratively getting drenched in the event of an unexpected outage or data loss, but what’s a little water, right? Having a disaster recovery plan could mean the difference between a minor hiccup and a full-fledged organizational crisis during a catastrophe, but let’s face it, who doesn’t love a little game of Russian Roulette with their Kubernetes deployment? So, go ahead and roll the dice. After all, disaster recovery is just for those who aren’t fans of suspense, right?
8.Train Your Staff (or Don’t): Now, here’s a real knee-slapper: education. Nothing quite like seeing your team scramble around like a bunch of cats on a hot tin roof because they don’t know their Pods from their Nodes. Who needs well-trained staff, conversant with Kubernetes best practices, when you can bask in the glorious pandemonium of mismanaged deployments instead? Sure, giving your employees the necessary skills to effectively manage and operate your Kubernetes deployment might lead to fewer issues, greater efficiency, and a more successful enterprise rollout. But let’s be real, why stifle the potential theatre of the absurd that could result from untrained staff wrestling a mammoth like Kubernetes? Life is a stage, after all, and in your Kubernetes drama, training is just too mainstream a script. So, sit back, grab some popcorn, and enjoy the show!
9.Live in the Past: Here’s a revolutionary idea – rolling with the times. You could, if you’re feeling particularly adventurous, actually stay up-to-date with the latest Kubernetes releases and security patches. That, of course, would imply that you’re interested in ensuring your deployment operates with the latest features and security updates. But hey, who doesn’t love a little nostalgia? Sure, you could prioritize keeping your enterprise rollout in line with the newest, slickest versions of Kubernetes, ensuring that your CI/CD pipelines are as cutting-edge as they come, but isn’t there a certain charm in running your enterprise on an antiquated version that’s as outdated as a floppy disk in an AI lab? After all, cybersecurity threats, outdated functionalities, and inefficiencies are just minor speed bumps on the road of enterprise rollouts. So, why not kick back, ignore those pesky update notifications, and let your Kubernetes deployment bask in the warm glow of obsolescence? Just remember – living in the past is only fun until the ghosts of security vulnerabilities and outdated features come knocking on your door.
10.Embrace Impermanence (non-persistence): In the grand scheme of things, isn’t Kubernetes is supposed to be ephemeral? Why should your data be any different? Go ahead, live dangerously. Don’t bother with planning for data persistence in your Kubernetes rollout. Imagine the thrill of living on the edge, knowing that you could lose all your data the moment a pod goes down or the system crashes. Sure, you could use the Kubernetes Persistent Volume (PV) and Persistent Volume Claim (PVC) architecture to ensure your data survives even when your pods don’t, but where’s the fun in that? Data persistence is so pedestrian. Remember, the goal here is to stall your Kubernetes enterprise rollout, not to make it robust, resilient, and reliable. So, go ahead, and throw caution (and your data) to the wind. It’s only important business information after all, right?
But hey, here’s a novel idea – what if you actually wanted to succeed in your cloud-native strategy? I know, I know, it sounds a bit radical given our prior conversation. But bear with me. For those of you who enjoy sailing smoothly on the seas of enterprise IT, without the thrill of hitting every possible iceberg, Dell has created a glorious solution. A tool, that’s as much a life preserver as it is a nautical chart, guiding you safely through the treacherous waters of Kubernetes enterprise rollouts. This magic wand is called the Container Storage Modules (CSM).
This isn’t just any tool – it’s your co-pilot on the journey to a seamless Kubernetes implementation. It’s like having a Swiss army knife for enterprise data management. The CSM ensures that your data persistence strategies are as solid as a rock, ensuring that no pod crash or system failure can sweep your data into the abyss. With CSM, you can laugh in the face of data loss, secure in the knowledge that your enterprise information is safe and sound. So, for the daredevils who actually like to succeed in their endeavors, the Dell Technologies CSM is the perfect tool to ensure your Kubernetes enterprise rollout is as smooth and trouble-free as a hot knife through butter.
I hope this post proves helpful, regardless of which direction you choose for your enterprise cloud journey. If you’re inclined to thrill and enjoy the odd game of Russian roulette with your data, you now have some innovative strategies to stall your Kubernetes rollout. However, if your preference is smooth as a jazz tune and your data as secure as Dell’s Project Fort Zero, then Dell’s Container Storage Modules (CSM) is the tool you need. The CSM is your beacon in the foggy world of Kubernetes enterprise rollout, ensuring that no data loss or system failure can derail your cloud-native strategy. It’s your data’s best friend, your enterprise’s lifeline, and your ticket to a successful Kubernetes implementation.
Enjoy the journey, and remember – with the right tools and strategies, Murphy’s Law doesn’t stand a chance!
I recently came across a strange issue while installing Docker on CentOS (version 8) machine.
Issue –
When I tried below options to install docker on my machine I got the same error every time.
Installation Commands Used
yum install -y -q docker-ce
or
yum install -y docker-ce.x86_64
or
yum install docker-ce
Error message received
Error:
Problem: package docker-ce-3:19.03.8-3.el7.x86_64 requires containerd.io >= 1.2.2-3, but none of the providers can be installed
- cannot install the best candidate for the job
- package containerd.io-1.2.10-3.2.el7.x86_64 is excluded
- package containerd.io-1.2.13-3.1.el7.x86_64 is excluded
- package containerd.io-1.2.2-3.3.el7.x86_64 is excluded
- package containerd.io-1.2.2-3.el7.x86_64 is excluded
- package containerd.io-1.2.4-3.1.el7.x86_64 is excluded
- package containerd.io-1.2.5-3.1.el7.x86_64 is excluded
- package containerd.io-1.2.6-3.3.el7.x86_64 is excluded
Resolution –
To resolve this issue, first we need to manually install the containerd.io package.
I have been using HammerDB for database performance bench-marking . This is very useful tool if you’re getting into any proof of concepts (POC) or testing of new database infrastructure stack before production roll-out.
In this blog post we will discuss around using HammerDB to generate OLTP (TPC-C workload) on Microsoft SQL database. In my example I had Microsoft SQL and HammerDB server created on AWSEC2 instances. But from HammerDB perspective it doesn’t matter as far as it can communicate to database instance.
We will follow 5 simple steps to run synthetic workload on MS SQL database. So let’s get started.
Step 1 – Installing the HammerDB Application
Before you get started make sure that –
HammerDB and MS SQL server are on same network/VLAN
MS SQL server is installed and running
Login to MS SQL console and create new SQL database. Make sure that new database is created on the target storage disks. If you create database in C drive, then you might not get expected performance due to disk bottleneck.
Microsoft SQL database console
Download HammerDB installer (Click image below)
HammerDB download link
Once HammerDB installer is downloaded, go ahead and install the same. Below are the links for installation steps.
MS SQL server ready with test database created on desired target storage disks
HammerDB installed and running
Step 2 – Configuration of Schema Build
We will need to created OLTP workload schema as per TPC-C specifications. Follow below steps for building OLTP schema on target database
Open the HammerDB console
HammerDB – Console
Under Benchmark navigation page, double-click on SQL Server. In Benchmark Options pop-up window select SQL Server and TPC-C options. Then click Ok. Click Ok once again to confirm the selection.
HammerDB – Selecting SQL database and TPC-C workload
Under Benchmark navigation now you can see SQL Server selected with TPC-C options.
HammerDB – SQL server and TPC-C benchmark
Expand TPC-C under SQL Server and then expand Schema Build
HammerDB – TPC-C Schema Build
Double-click on Options under Schema Build. Pop-up window of TPC-C Build Options will open. In this window enter below details, and then click Ok
SQL Server – IP address or hostname of SQL server. Keep (local) if HammerDB is installed on the SQL server.
Authentication – Use Windows Authentication if you want logged-in user credentials to be used for SQL server, else select SQL authentication and enter credentials.
SQL Server Database – Name of the database which we had created in Step 1
Number of Warehouses – Enter the value to number of warehouses you have chosen for your testing. Preferably select number of warehouses equal to number of cores on the server
Virtual Users to Build Schema – Set this value equal to number of warehouses.
HammerDB – TPC-C Build Options
In the Benchmark pane double-click on Build
HammerDB – TPC-C Schema Build
Click Yes to confirm creating schema in the target SQL database.
HammerDB will now start creating virtual user threads and create schema in the target database. This process takes some time. You can monitor the status in top-right corner of HammerDB console. (TPC-C creation in top-right)
HammerDB – TPC-C Schema Creation in process
Once completed you can see that the status is changed to Complete
HammerDB – TPC-C Schema Creation Completed
Step 3 – Configure DriverScript
Follow below steps to configure HammerDB driver script
Expand the Driver Script from Benchmark navigation pane and double-click on Options
HammerDB – Driver Script
In the TPC-C Driver Options pop-up window enter below details, and then click Ok
SQL Server – Keep it same as Step 2
Authentication – Keep it same as Step 2
Total Transactions Per User – Keep default value. This value will set the number of transactions each virtual user will process before logging off
TPC-C Driver Script – Keep this option as Timed Driver Script. This will run the workload for finite time as specified in Minutes for Test Duration
Minutes of Rampup Time – The rampup time defines the time in minutes for the monitoring virtual user to wait for the virtual users running the workload to connect to the database. 2 minutes in my case. You can increase this number if you’ve higher number of virtual users.
Minutes for Test Duration – The Minutes for Test Duration is shown as duration in the Driver Script. This does not include rampup time.
Use All Warehouses – Keep this option checked.
HammerDB – Driver Script Options
In the Benchmark navigation double-click on Load (under Driver Script). You need to Load the Driver Script every time you make changes to Driver Script Options.
HammerDB – Driver Script Load
Step 4 – Create Virtual User
Once HammerDB driver script is loaded, follow below steps to create virtual user.
Expand Virtual User from Benchmark pane
HammerDB – Virtual User
Double-click on Options. In the pop-up window enter below details, and then click Ok.
Virtual Users – Keep number of users same as Step 2
Keep Other inputs as default
HammerDB – Virtual User Options
In the Benchmark navigation pane double-click on Create under Virtual User. This will create virtual users and keep them idle.
HammerDB – Virtual User Create
At this point we are ready to start OLTP workload on target database.
Step 5 – Run HammerDB OLTP (TPC-C) Workload
Follow below steps to run the OLTP workload and monitor the TPMs.
In Benchmark pane double-click on Run (Under Virtual User).
HammerDB – Run OLTP (TPC-C) workload
Now Virtual Users will start logging into the target database and begin running their workload. You can monitor the status under Virtual User 1-MONITOR
HammerDB – Starting OLTP (TPC-C) Workload – In Progress
While workload is running you can monitor the real-time TPM (Transactions Per Minute) by clicking on Transaction Counter
HammerDB – Transaction Counter
Once workload is completed you can see the status in top-right corner as well as under Virtual User 1-MONITOR
Once the workload is completed note down TEST RESULT under Virtual User 1-MONITOR
HammerDB – OLTP (TPC-C) TEST RESULT
You can run multiple tests and take average across them for realistic performance numbers.
Apart from MS SQL (used in this blog post) HammerDB supports running OLTP (TPC-C) and OLAP (TPC-H) on Oracle, IBM DB2, MySQL, PostgreSQL, MariaDB and Redis.
This blogs lists the steps around creating RAID5 volume on Linux using local disks.
We will start by installing software RAID on Linux system
# yum install mdadm
Loaded plugins: ulninfo
Resolving Dependencies
--> Running transaction check
---> Package mdadm.x86_64 0:4.1-1.0.1.el7 will be installed
--> Processing Dependency: libreport-filesystem for package: mdadm-4.1-1.0.1.el7.x86_64
--> Running transaction check
---> Package libreport-filesystem.x86_64 0:2.1.11-43.0.1.el7 will be installed
--> Finished Dependency Resolution
Dependencies Resolved
================================================================================
Package Arch Version Repository Size
================================================================================
Installing:
mdadm x86_64 4.1-1.0.1.el7 ol7_latest 435 k
Installing for dependencies:
libreport-filesystem x86_64 2.1.11-43.0.1.el7 ol7_latest 40 k
Transaction Summary
================================================================================
Install 1 Package (+1 Dependent package)
Total download size: 475 k
Installed size: 1.0 M
Is this ok [y/d/N]: y
Downloading packages:
(1/2): libreport-filesystem-2.1.11-43.0.1.el7.x86_64.rpm | 40 kB 00:00
(2/2): mdadm-4.1-1.0.1.el7.x86_64.rpm | 435 kB 00:00
--------------------------------------------------------------------------------
Total 1.8 MB/s | 475 kB 00:00
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : libreport-filesystem-2.1.11-43.0.1.el7.x86_64 1/2
Installing : mdadm-4.1-1.0.1.el7.x86_64 2/2
Verifying : mdadm-4.1-1.0.1.el7.x86_64 1/2
Verifying : libreport-filesystem-2.1.11-43.0.1.el7.x86_64 2/2
Installed:
mdadm.x86_64 0:4.1-1.0.1.el7
Dependency Installed:
libreport-filesystem.x86_64 0:2.1.11-43.0.1.el7
Complete!
Now let’s identify disks which we will use for creating RAID5 volume. These disks can be local or storage LUNs. You can list the disks using running below command. In this example we will use 5 local disks – /dev/xvd(b-f)
# fdisk -l | grep /dev/
Disk /dev/xvda: 53.7 GB, 53687091200 bytes, 104857600 sectors
/dev/xvda1 * 2048 4098047 2048000 83 Linux
/dev/xvda2 4098048 86018047 40960000 83 Linux
/dev/xvda3 86018048 104857599 9419776 82 Linux swap / Solaris
Disk /dev/xvdd: 53.7 GB, 53687091200 bytes, 104857600 sectors
Disk /dev/xvde: 53.7 GB, 53687091200 bytes, 104857600 sectors
Disk /dev/xvdb: 53.7 GB, 53687091200 bytes, 104857600 sectors
Disk /dev/xvdc: 53.7 GB, 53687091200 bytes, 104857600 sectors
Disk /dev/xvdf: 53.7 GB, 53687091200 bytes, 104857600 sectors
Before you begin make sure that selected drives do not have existing RAID configured.
# mdadm -E /dev/xvd[b-f]
mdadm: No md superblock detected on /dev/xvdb.
mdadm: No md superblock detected on /dev/xvdc.
mdadm: No md superblock detected on /dev/xvdd.
mdadm: No md superblock detected on /dev/xvde.
mdadm: No md superblock detected on /dev/xvdf.
Next step is to create partition on selected disks using fdisk. Follow below steps for each selected disks – /dev/xvd(b-f) in this example
# fdisk /dev/xvdc
Welcome to fdisk (util-linux 2.23.2).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.
Device does not contain a recognized partition table
Building a new DOS disklabel with disk identifier 0x6845b853.
Command (m for help): n
Partition type:
p primary (0 primary, 0 extended, 4 free)
e extended
Select (default p): p
Partition number (1-4, default 1): 1
First sector (2048-104857599, default 2048):
Using default value 2048
Last sector, +sectors or +size{K,M,G} (2048-104857599, default 104857599):
Using default value 104857599
Partition 1 of type Linux and of size 50 GiB is set
Command (m for help): t
Selected partition 1
Hex code (type L to list all codes): fd
Changed type of partition 'Linux' to 'Linux raid autodetect'
Command (m for help): p
Disk /dev/xvdc: 53.7 GB, 53687091200 bytes, 104857600 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x6845b853
Device Boot Start End Blocks Id System
/dev/xvdc1 2048 104857599 52427776 fd Linux raid autodetect
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.
Once you’ve successfully created the partitions for all disks, you can verify the changes using below command
Note that it takes some time to complete the RAID creation. You can monitor the progress using below command. Below command will show you current % completed.
But you prefer to watch screen till 100% completion then run below command. Result of this command will be persistent on the screen and will show you real time progress.
Once RAID creation is 100% you can run below command to see the summary of the RAID device configuration. RAID device is /dev/md1 in below example with 5 devices – /dev/xvd(b-f)1
# mdadm --detail /dev/md1
/dev/md1:
Version : 1.2
Creation Time : Mon Dec 2 09:39:05 2019
Raid Level : raid5
Array Size : 209575936 (199.87 GiB 214.61 GB)
Used Dev Size : 52393984 (49.97 GiB 53.65 GB)
Raid Devices : 5
Total Devices : 5
Persistence : Superblock is persistent
Update Time : Mon Dec 2 09:53:20 2019
State : clean
Active Devices : 5
Working Devices : 5
Failed Devices : 0
Spare Devices : 0
Layout : left-symmetric
Chunk Size : 512K
Consistency Policy : resync
Name : ip-10-10-10-207:1 (local to host ip-10-10-10-207)
UUID : 07f28a7b:3c70d798:1deecf9a:292ddd2c
Events : 18
Number Major Minor RaidDevice State
0 202 17 0 active sync /dev/xvdb1
1 202 33 1 active sync /dev/xvdc1
2 202 49 2 active sync /dev/xvdd1
3 202 65 3 active sync /dev/xvde1
5 202 81 4 active sync /dev/xvdf1
At this point we have RAID5 device created (/dev/md1). Now we need to create file system. In below example we are creating EXT4 file system
# mkfs.ext4 /dev/md1
mke2fs 1.42.9 (28-Dec-2013)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=128 blocks, Stripe width=512 blocks
13099008 inodes, 52393984 blocks
2619699 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=2199912448
1599 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
4096000, 7962624, 11239424, 20480000, 23887872
Allocating group tables: done
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done
Once the file system is created on RAID5 device, create the mount point and mount the RAID5 device on the same.
This blog is not to explain what is iSCSI but rather a reference to steps which an administrator should follow for configuring iSCSI on their Linux host (CentOS in my case)
Prerequisites
Make sure Linux host has IP connectivity to storage appliance which is acting as iSCSI target
Storage appliance has iSCSI configured and service is running
Steps
Install iSCSI initiator on Linux host. Below is the example
# yum install iscsi-initiator-utils
Loaded plugins: ulninfo
Resolving Dependencies
--> Running transaction check
---> Package iscsi-initiator-utils.x86_64 0:6.2.0.874-11.0.1.el7 will be installed
--> Processing Dependency: iscsi-initiator-utils-iscsiuio >= 6.2.0.874-11.0.1.el7 for package: iscsi-initiator-utils-6.2.0.874-11.0.1.el7.x86_64
--> Running transaction check
---> Package iscsi-initiator-utils-iscsiuio.x86_64 0:6.2.0.874-11.0.1.el7 will be installed
--> Finished Dependency Resolution
Dependencies Resolved
=======================================================================================================================================================
Package Arch Version Repository Size
=======================================================================================================================================================
Installing:
iscsi-initiator-utils x86_64 6.2.0.874-11.0.1.el7 ol7_latest 428 k
Installing for dependencies:
iscsi-initiator-utils-iscsiuio x86_64 6.2.0.874-11.0.1.el7 ol7_latest 93 k
Transaction Summary
=======================================================================================================================================================
Install 1 Package (+1 Dependent package)
Total download size: 521 k
Installed size: 2.6 M
Is this ok [y/d/N]: y
Downloading packages:
(1/2): iscsi-initiator-utils-6.2.0.874-11.0.1.el7.x86_64.rpm | 428 kB 00:00:00
(2/2): iscsi-initiator-utils-iscsiuio-6.2.0.874-11.0.1.el7.x86_64.rpm | 93 kB 00:00:00
-------------------------------------------------------------------------------------------------------------------------------------------------------
Total 1.1 MB/s | 521 kB 00:00:00
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : iscsi-initiator-utils-iscsiuio-6.2.0.874-11.0.1.el7.x86_64 1/2
Installing : iscsi-initiator-utils-6.2.0.874-11.0.1.el7.x86_64 2/2
Verifying : iscsi-initiator-utils-6.2.0.874-11.0.1.el7.x86_64 1/2
Verifying : iscsi-initiator-utils-iscsiuio-6.2.0.874-11.0.1.el7.x86_64 2/2
Installed:
iscsi-initiator-utils.x86_64 0:6.2.0.874-11.0.1.el7
Dependency Installed:
iscsi-initiator-utils-iscsiuio.x86_64 0:6.2.0.874-11.0.1.el7
Complete!
Once iSCSI package is installed then edit the iSCSI configuration file and enable node.startup = automatic. You will need to find node.startup in the config file and remove the # from start of the line. This will make sure that iSCSI is logging to target automatically on startup
vi /etc/iscsi/iscsid.conf
node.startup = automatic
Now configure the iSCSI service to start automatically. Run below command
# systemctl enable iscsid
Created symlink from /etc/systemd/system/multi-user.target.wants/iscsid.service to /usr/lib/systemd/system/iscsid.service.
Start iSCSI service
# systemctl start iscsid
Now at this point you’ve iSCSI enabled on your Linux server. If you want to view iSCSI initiator name of your server then run below command. Note that in most cases iSCSI initiator name will reflect into logged in initiators on your storage appliance in some cases you’ll need to provide the initiator name.
You can run below commands to list discovered iSCSI nodes
# iscsiadm -m node
172.16.1.11:3260,2 iqn.1992-04.com.emc:cx.asfnsdbgfkdjfgdk.a2
172.16.1.12:3260,1 iqn.1992-04.com.emc:cx.asfnsdbgfkdjfgdk.b2
or
# ls /var/lib/iscsi/send_targets
172.16.1.11,3260
# ls /var/lib/iscsi/nodes
iqn.1992-04.com.emc:cx.asfnsdbgfkdjfgdk.a2
iqn.1992-04.com.emc:cx.asfnsdbgfkdjfgdk.b2
Once iSCSI nodes are discovered you can run below command to login into iSCSI target
# iscsiadm -m node --targetname "iqn.1992-04.com.emc:cx.asfnsdbgfkdjfgdk.a2" --portal "172.16.1.11:3260" --login
Logging in to [iface: default, target: iqn.1992-04.com.emc:cx.asfnsdbgfkdjfgdk.a2, portal: 172.16.1.11,3260] (multiple)
Login to [iface: default, target: iqn.1992-04.com.emc:cx.asfnsdbgfkdjfgdk.a2, portal: 172.16.1.11,3260] successful.
# iscsiadm -m node --targetname "iqn.1992-04.com.emc:cx.asfnsdbgfkdjfgdk.b2" --portal "172.16.1.12:3260" --login
Logging in to [iface: default, target: iqn.1992-04.com.emc:cx.asfnsdbgfkdjfgdk.b2, portal: 172.16.1.12,3260] (multiple)
Login to [iface: default, target: iqn.1992-04.com.emc:cx.asfnsdbgfkdjfgdk.b2, portal: 172.16.1.12,3260] successful.
At this stage you’ve Linux host configured as iSCSI initiator and logged in successfully to target storage. Now you can ask storage admin to create LUNs and assign to Linux host.
Once LUNs are assigned to Linux host you can run below command to scan new devices
This section seems pretty straight-forward but is the must for all the Ansible deployments.
Follow below steps to get through the password maze in Ansible environments
Firstly we will need to generate the SSH authentication keys. This needs to be done on the server from which you want to login to other hosts. In my case it’s Ansible master server.
Generate your key – Run the ssh-keygen command (as below). By default system will create and store the SSH key into /root/.ssh/id_rsa, which you can change. Passphrase is optional and you can leave it blank.
Note that if you already have generated the SSH key you have option to overwrite the same, but make sure that earlier key is not used anywhere for authentication. Below example shows option to replace existing SSH keys
[root@ansible-master ~]# ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa):
/root/.ssh/id_rsa already exists.
Overwrite (y/n)? y
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:D52Rvc2F7JMlbikpj7G96Ys3bSLRUz4tNmsfNP4E2Vg root@ansible-master
The key's randomart image is:
+---[RSA 2048]----+
| |
| o . . |
| o . + E|
| . o B.@ |
| S =.+o&+.|
| o.BooOoo|
| +.o+ B.|
| ..+o=.o|
| .+=* .o|
+----[SHA256]-----+
If you want to use existing keys then you can simply read the existing key which is saved in the file. Below is the example of the reading existing public key
Once you have generated the SSH key next step is to add the key into target system.
For copying the SSH keys into target system simply follow below process.
[root@ansible-master ~]# ssh-copy-id root@192.168.1.101
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/root/.ssh/id_rsa.pub"
The authenticity of host '192.168.1.101 (192.168.1.101)' can't be established.
ECDSA key fingerprint is SHA256:qnOPcE09JqKALwLLaL+SqNMy2oubvhF/EevGCcK9S9A.
ECDSA key fingerprint is MD5:4d:65:b7:e0:e8:09:41:a8:f9:4c:2d:b6:47:55:b2:09.
Are you sure you want to continue connecting (yes/no)? yes
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
root@192.168.1.101's password:
Number of key(s) added: 1
Now try logging into the machine, with: "ssh 'root@192.168.1.101'"
and check to make sure that only the key(s) you wanted were added.
You’ll notice in the command output that this process has added the key into the target system. Note that 192.168.1.101 in above example is target system in which I wanted Ansible master to login without username and password.
Once the SSH keys are added into the target system now you can try SSH into the target system and it won’t ask for password anymore.
[root@ansible-master ~]# ssh root@192.168.1.101
Last login: Tue Nov 26 09:01:16 2019 from gateway
[root@target-system ~]#
