IOPS, capacity, bandwidth -...

Hello Joerg,

Nice post and you nailed it with there are more attributes for storage than capacity, particular IOPS and bandwidth which are starting to get more awareness and focus. All too often, those who do not know better look at storage on a cost per GByte/TByte basis without factoring in activity (e.g. performance) or availability.

Thus bringing IOPS (activity) and bandwidth (ability to move data) is part of expanding the discussion and hopefully awareness. However also what needs to come into the discussion particular for databases is latency or response time. With SSDs, the good news is that many people are now starting to talk about and toss IOPS around, yet one of the key attributes of SSDs is ability to reduce response time or latency. Let us also revisit that as IOPS increase which is usually due to a smaller IO size, bandwidth will also increase however not at the same rate as if a larger IO size. In other words, many small IOPS will usually max out an interface, adapter, port, device before its full bandwidth (spec speed) is achieved.

Likewise, fewer yet much larger IOPs can max out an interface, adapter, port, device bandwidth capabilities before its full IOP capabilities (spec speed) is achieved. As an example, look at some of the out of this world SSD benchmarks some vendors are touting where not tens of thousands of IOPS, rather hundreds of thousands of IOPS or more are touted. In some of those, that IOP size can be as low as 64 bytes or a fraction of a 512 byte (1/2 Kbyte ) page. Those kind of IOPs might impress however if focused on bandwidth, they would also disappoint, not to mention a 64 byte IO vs. a database IOP of 4K to 8K (e.g. 4,096 bytes to 8192 bytes) is not relative to most environments.

Likewise, availability or accessibility is also important because without it, you do not have performance, or without performance, you do not have availability.

I like your comment about purchasing or management or other informed focused on cost and capacity suggest a USB 1TB device. The reason I like that comment is a fun response would be to tell the bean counter to get rid of their computer and go back to a hand punched calculator adding machine as that too would allow them to get their job done. Granted the hand punched adding machine would not be as effective or productive, however it would sure cost a lot less.

Now back to the USB SSD, I’m assuming that is being used as an example, as other than some USB thumb drives, most if not all USB SSDs are actually SATA with a USB bridge. How do I know this? Simple, I have some of those and for fun (ok, geek fun) I have attached SATA SSDs using my bridge/connector cable to a USB 2 and USB 3 port and yes it limits IOPS as well as bandwidth and latency. Plug the same SSD into an eSATA (external SATA) port and all of the sudden the SSD starts to stretch its performance legs until the port/controller/server/system can no longer support it. Need even more performance, get a faster 6Gb/s SSD and attach it to a 6Gb/s wide SAS port (requires SAS adapter), or to a RAID card/adapter or controller that also supports caching.


However availability comes back to the discussion in that with that much data on a device for something important, regardless of if HDD, HHDD or SSD, there should be at least two in a RAID 1 (mirror) or other RAID level for protection unless being replicated in real-time to another system.

Need more performance than a single device or adapter or wide port can deliver (keep in mind wide sas is actually 6Gb/s x number of lanes, send/receive assuming the port and drive supports it)? Then get a faster device or use a PCIe cache or target card.

However back to your points, look beyond the cost per GByte/TByte; look at productivity that means IOPS (including size, reads/writes), bandwidth (transfer), and latency as well as availability.

Thus you nailed it with bringing IOPS and bandwidth into the discussion, now expand the discussion with latency, availability or HA. Also expand the IOP discussion to IO size, read/write, random and sequential along with how IOPSs and latency and bandwidth are all related.

Cheers
gs

Greg Schulz - Sr. Advisory consultant StorageIO
Author "Cloud and Virtual Data Storage Networking" (CRC Press)

Hello Kevin, Im not surprised with what you are seeing or have seen when attaching a SSD to a USB2.0. USB3.0 helps improve, if you can go to eSATA even better, however for best performance, attach the SSD to an internal SATA port or if available SAS port.

As you mentioned, IOPS should be much higher with USB3.0 vs. USB2.0 while latency is lower. Btw, what size IOs on average are you doing or seeing, along with are they reads or writes, random or sequential.

As an FWIW/FYI, I attached one of my HHDDs that has 8GB slc nand flash/SSD built into the drive, attached it to a laptop via eSATA and then copied around 25GB in about 10 minutes. Not blazing fast and IOPS were bad as was latency during that time to those devices however bandwidth was impressive for eSATA/USB3.0 category. In other words an example of what happens to IOPS when bandwidth increases and vise versa.

hope that helps
gs