Enterprise Storage Part 1 – What are IOPS, and other basics

If you’re specifying storage for enterprise class applications, or more recently, looking at specs of Solid State Disks – even ones for your PC, you’ll see a lot of mentions of IOPS. What exactly are IOPS?

IOPS stands for Input/Output operations Per Second, or IOs Per Second, and is a way of measuring the amount of data you can get into or out of a disk or storage system.

So how big is an IO? The simple answer is that it depends. By IOs we really mean “IO instruction”, ignoring the amount of data. Thus you can get different sized IOs, as on the whole an IO instruction with no associated data is not going to be much use when you’re trying to get data into or out of a storage device.

The chunks of data dealt with by individual IOs tend to be relatively small, ranging between about 512 bytes to 64K bytes. The performance figures for disks will, if you look hard enough, give you detailed figures for the IOPS for a range of different sizes. Consumer SSDs tend to at least mention 4KB IOs.

In the case of individual disks, in physical terms it will clearly take longer to write a larger amount of data to a disk then to write a smaller amount. Think of a spinning disk, if you write 512 bytes to the magnetic surface the disk platter will only have to rotate a small amount between where it starts to write and when it has finished, whereas to write 64KB will take somewhat longer (roughly 64,000 / 512 = 125 times longer). This is why enterprise-grade disks tend to spin faster than consumer disks. A standard consumer disk will spin at 7200rpm whereas an enterprise SAS (Serial Attached SCSI) or FC (Fibre Channel) will spin at 15,000rpm.

But that’s not the only factor that determines how many IOs a device can handle. In the case of spinning magnetic disks, you also have to take into account the “seek time”. This is how long it takes for the disk controller to move the read/write head to the correct place on the surface of the spinning disk, ready to start reading or writing data. For consumer disks this tends to be around 11ms, for decent enterprise grade disks it’ll be more like 4ms. 4ms might not sound like a lot, but it really is a very long time, especially if you are needing to read and write hundreds of times a second at random places over the surface of the disk. This seek time delay is what solid state drives effectively remove, and is one of the main reasons (when compared to magnetic disks) why their IOPS figures tend to be so high, especially for random read operations.

There are a few other factors that can affect IOPS figures for a particular disk, and one of the more interesting ones is the number of IOs that a disk can queue up. This is called Native Command Queuing (NCQ) or Tagged Command Queuing (TCQ). Enterprise disks can usually queue 32 instructions, whereas historically consumer grade disks tended not to queue more than about 2 (this changed with the introduction of the SATA interface). The reason this is important is because if the disk knows what it’s going to be asked to do in the future, it gives it the possibility of re-jigging the order of executing those instructions to take advantage of the physical characteristics of the storage medium. As a very basic example, imagine instructions as follows:

  1. Read 4k from position 0
  2. Write 8K to position 10000
  3. Read 16k from position 2
  4. Read 4k from position 10050

Executed in order, the heads on the disk will be moving rapidly from position 0 to position 10000 to position 2 to position 10050. If the device was able to see the entire queue it might instead choose to execute in the following order:

  1. Read 4k from position 0
  2. Read 16K from position 2
  3. Write 8K to position 10000
  4. Read 4K from position 10050

This has turned out to be a much longer article than I originally envisaged, so I’m going to leave it there. At some point I’ll write part 2 and cover things like RAID levels and cache.

This entry was posted in Storage and tagged , , , , , , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s