I had three hard-drives that weren't in use and decided to make my backup a bit more resilient. Mind you that I'm doing backups in a 3-2-1 fashion, so the loss of the local disk would be just an inconvenience.
diskwrite perfread perfbenchmark2 x TOSHIBA-HDWD240190 MB/s187 MB/suserbenchmark1 x WD WD40EZRZ-00GXCB0169 MB/s172 MB/suserbenchmarkI wanted to put the drives as a RAID5 [data 4TB + data 4TB + parity 4TB], that way, if one of them failed, everything would keep working. I already had some experience with RAID; both on linux and on Intel hardware.
My issue with hardware RAID is that if the RAID controller fails, you're done. Software RAID evidently has a performance penalty.
Storage Spaces comes with Windows which is very welcome. I also like that you can connect the drives to another system and things will just work. Nothing to configure. What I wasn't expecting, though, was writes of 40 MB/s, sometimes lower!
R.A.I.D. - Redundant Array of Inexpensive Disks
Creating a "Storage Space" is as simple as searching for "Manage Storage Spaces", adding empty unformatted drives to a Storage Pool and from there creating a Storage Space.
If all goes well, you'll have a drive letter ready in no time. Couldn't be simpler. Unfortunately, it's not. If you're reading this page, you probably already experienced some crawling.
Columns, Interleave and Allocation
Where do columns and interleave sit within the Storage Space?
Column Count
Column Count is the number of physical disks that Storage Spaces stripes data across. On my scenario, it's pretty easy to understand that this number will be 3. I've got 3 disks, one column per disk.
Interleave Size
Interleave Size is the actual size of data that will be stored on each disk per stripe.
PS C:\Windows\system32> Get-StoragePool -FriendlyName "Storage Pool" | Get-ResiliencySetting
Name NumberOfDataCopies FaultDomainRedundancy NumberOfColumns Interleave NumberOfGroups
---- ------------------ --------------------- --------------- ---------- --------------
Simple 1 0 Auto 262144 1
Mirror 2 1 Auto 262144 1
Parity 1 1 Auto 262144 Auto
Let's get some info on the storage pool settings created by the GUI.We got "auto" for the number of columns and 256 KB for the interleave.
Per the image above, we may see that when we want to write 512 KB of information we'll write 256KB on HDD1 [interleave], 256KB on HDD2 and another 256KB on HDD3 which is the parity disk [that information will have to be calculated on the fly].
Allocation Unit Size
Allocation Unit Size is the minimum size of each write into your NTFS partition. Regardless of data being written, it will be done in chunks matching Allocation Unit Size. Let us check what Windows did to the NTFS partition with the default settings:
PS C:\Users\frankie> fsutil fsinfo ntfsinfo f:
NTFS Volume Serial Number : 0x762ca9782ca93451
NTFS Version : 3.1
LFS Version : 2.0
Total Sectors : 1 951 497 978 [7,3 TB]
Total Clusters : 1 951 497 978 [7,3 TB]
Free Clusters : 1 946 169 997 [7,3 TB]
Total Reserved Clusters : 1 024 [4,0 MB]
Reserved For Storage Reserve : 0 [0,0 KB]
Bytes Per Sector : 4096
Bytes Per Physical Sector : 4096
Bytes Per Cluster : 4096
Bytes Per FileRecord Segment : 1024
Clusters Per FileRecord Segment : 0
Mft Valid Data Length : 256,00 KB
[...]
Allocation Unit Size for the created partition is 4KB!Defaults for NTFS partitions are:
- 4KB for drives bellow 16TB
- 8KB for drives equal or above 16TB.
Slow writes?
Having interleave size as 256KB and Allocation Unit Size as 4KB is the bottleneck. Each write request has to allocate space on the partition in chunks of 4KB and then that has to be translated to the 256KB interlave chunks bellow!
How to fix this?
On this three disks scenario we know that disk 1 will get half data, disk 2 will get the other half and disk 3 will have data created on the fly. What if the interleave default size [256 KB] was in sync with the NTFS partition? If we were to format the disk with an Allocation Size Unit [AUS] of 512 KB:
NTFS [allocation 512 KB]user writes 512 KBDISK 1
interleave 256 KB
half of write requestDISK 2
interleave 256 KB
half of write requestDISK 3
interleave 256 KB
generated data
A setup like this would massively simplify the calculations needed for writes. We can easily format the drive with that AUS.
Why are disks formatted with Allocation Unit Sizes [AUS] of 4 KB instead of 512 KB? That has mainly to do with a trade-off between speed and storage efficiency.
Remember that AUS becomes the minimum possible file size on disk. Say you have 1000 files of 1 KB, you'd expect them to occupy 1000 KB [1 MB] on disk. However, if you have an AUS of 512 they will take over 512 MB of disk!
I choose files of 1000 KB because tiny files will be saved as "resident".
If the file is very small, to save disk space, NTFS [...] stores their contents right in the file record, so no cluster has to be allocated for it. Therefore, the size on disk is zero because there's nothing beyond the file record.
//superuser.com/a/1030802/14529
Can we control interleave size?
We most definitely can! The only hiccup is that you'll have to create the Storage Space in PowerShell. Create the Storage Pool as you normally would have:
Open up PowerShell and check your pool name:
PS C:\Windows\system32> Get-StoragePool
FriendlyName OperationalStatus HealthStatus IsPrimordial IsReadOnly Size AllocatedSize
------------ ----------------- ------------ ------------ ---------- ---- -------------
Storage pool OK Healthy False False 10.91 TB 768 MB
Primordial OK Healthy True False 23.65 TB 10.92 TB
Primordial pool represents all your drives.So my pool name is "Storage pool". We'll use that to create the "Storage Space" which in PowerShell goes by Virtual-Disk.
PS C:\Windows\system32> New-VirtualDisk -StoragePoolFriendlyName "Storage pool" -FriendlyName "WasteServerExample" -NumberOfColumns 3 -Interleave 64KB -ResiliencySettingName Parity -UseMaximumSize
FriendlyName ResiliencySettingName FaultDomainRedundancy OperationalStatus HealthStatus Size FootprintOnPool StorageEfficiency
------------ --------------------- --------------------- ----------------- ------------ ---- --------------- -----------------
WasteServerExample Parity 1 OK Healthy 7.27 TB 10.91 TB 66,65%
Create "Storage Pool" with Interleave of 64 KBPS C:\Users\frankie> Get-VirtualDisk -friendlyname "WasteServerExample" | fl
[...]
FriendlyName : WasteServerExample
HealthStatus : Healthy
Interleave : 65536
IsDeduplicationEnabled : False
IsEnclosureAware : False
[...]
Is interleave actually 64 KB? Yes it yes!Open "Computer Management", go to "Disk Management", initialize the new Virtual Disk and format it with 128 KB as the Allocation Unit Size.
What should I set my values to?
Depends. The larger the AUS the more space you'll waste with small files.
If you'll only use the disk for media files like photographs and videos, NTFS allocation unit size will impact you much less than if you have a multitude of small files.
The one thing to remember is to keep the symmetry between columns, interleave space and allocation unit size.