But write-back caches and array intelligence have all but eliminated this “write penalty” for modern enterprise systems. ...
I'd say this is mostly true for light enough workloads/large enough write caches. I've seen perfectly acceptable response times from RAID-5 and in some cases RAID-6, the trouble is that after you've gone past a certain point or your workload changes, the response time latency can increase rapidly. As a result, almost all spindle bound benchmarks use RAID-10 (with some notable exceptions)
The old "rules of thumb" about 1:2 overheads for mirrored writes and 1:4 overhead for single parity writes and 1:6 for dual parity writes are all worst case but are indicative of the relative penalties for each kind of RAID in spindle bound random write workloads. Once you add in any level of sequentiality (present in most workloads), things start looking a lot better for parity based RAID. In the real world thanks to lots of dilligent engineering, most vendors will do better than these worst cases. For example approaches like WAFL and ZFS address this neatly with small write caches, on traditional algorithmically mapped array technologies your best bet is to use the biggest write cache you can get your hands on, or for those with bigger wallets, you can implement solid state and forget all about your per spindle IOPS worries.
Unfortunately as memory get cheaper disks also get bigger which means more data on fewer spindles, which tends to kill random read performance, so most storage admins are forced to to tune their cache and make the hard choice between good random write performance or good random read performance.
<shameless plug>
The nice thing about working for NetApp is that we can get outstanding random write performance with relatively small write caches, and can dedicate up to 2TB of intelligently managed dedup aware cache to accelerate random reads. This means our customers can get the best of both worlds.
</shameless plug>
There are other concerns with RAID-5 that happen as a result of exponentially increasing spindle sizes (increasing scrub times, longer reconstructs etc), but that's an entirely different kettle of fish.
Justin Warren
· 1 month ago
I look forward to reading the focused articles.
The Dumb Disk Fallacy is always a challenge to explain to business users. Analogies with things they're more familiar with than enterprise storage arrays seem to work sometimes. Lots of outboard powered tinnies tied together != the QE2.
DGentry
· 1 month ago
When you write about the Dumb Disk Fallacy, could you talk about why some large consumers of storage space (Google, Facebook, others) don't use RAID arrays? They use a bunch of disks, with replication and redundancy handled in higher layers of the software. Is there some capacity threshold above which an enterprise could look seriously at eschewing RAID and doing it themselves, or do you believe the engineers at those internet companies just don't know when to stop writing code and buy something off the shelf?
Telling an enterprise buyer that the "Dumb Disk Fallacy" is just not true is somewhat unconvincing. Telling the enterprise buyer that you _can_ build a storage system whose price approaches the raw drives, but the development and engineering costs only make sense at a scale several orders of magnitude above their needs, would seem to be more convincing.
All Comments
I'd say this is mostly true for light enough workloads/large enough write caches. I've seen perfectly acceptable response times from RAID-5 and in some cases RAID-6, the trouble is that after you've gone past a certain point or your workload changes, the response time latency can increase rapidly. As a result, almost all spindle bound benchmarks use RAID-10 (with some notable exceptions)
The old "rules of thumb" about 1:2 overheads for mirrored writes and 1:4 overhead for single parity writes and 1:6 for dual parity writes are all worst case but are indicative of the relative penalties for each kind of RAID in spindle bound random write workloads. Once you add in any level of sequentiality (present in most workloads), things start looking a lot better for parity based RAID. In the real world thanks to lots of dilligent engineering, most vendors will do better than these worst cases. For example approaches like WAFL and ZFS address this neatly with small write caches, on traditional algorithmically mapped array technologies your best bet is to use the biggest write cache you can get your hands on, or for those with bigger wallets, you can implement solid state and forget all about your per spindle IOPS worries.
Unfortunately as memory get cheaper disks also get bigger which means more data on fewer spindles, which tends to kill random read performance, so most storage admins are forced to to tune their cache and make the hard choice between good random write performance or good random read performance.
<shameless plug>
The nice thing about working for NetApp is that we can get outstanding random write performance with relatively small write caches, and can dedicate up to 2TB of intelligently managed dedup aware cache to accelerate random reads. This means our customers can get the best of both worlds.
</shameless plug>
There are other concerns with RAID-5 that happen as a result of exponentially increasing spindle sizes (increasing scrub times, longer reconstructs etc), but that's an entirely different kettle of fish.
The Dumb Disk Fallacy is always a challenge to explain to business users. Analogies with things they're more familiar with than enterprise storage arrays seem to work sometimes. Lots of outboard powered tinnies tied together != the QE2.
Telling an enterprise buyer that the "Dumb Disk Fallacy" is just not true is somewhat unconvincing. Telling the enterprise buyer that you _can_ build a storage system whose price approaches the raw drives, but the development and engineering costs only make sense at a scale several orders of magnitude above their needs, would seem to be more convincing.