When I last posted about upgrading the storage in my HP Microserver for Proxmox I had hoped that the limiting factor would be the Microserver’s limit of 16GB of RAM rather than the IOPs of my disks.

I very quickly found, somewhat unsurprisingly, that a mirror of 7200RPM disks was still not sufficient for good performance, especially as I began to add more VM’s and LXC containers.

During day to day use this wasn’t too much of a problem. However during backups, VM’s would become non-responsive and when running upgrades on certain VM’s, other VM’s and containers would suffer performance issues.

As a temporary fix I trialled the addition of a single SSD to the ZFS pool as both SLOG and L2ARC.

I bought a Corsair Force Series™ LS 60GB, a slimline ODD to 2.5” HDD bay convertor and a floppy to SATA power adaptor.

Corsair LS60

I won’t go into much detail regarding ZIL, SLOG and L2ARC as there are far better explanations written by people who know a lot more about the subject than I do.

There are also a lot of misconceptions regarding ZIL/SLOG and L2ARC which are cleared up quite nicely in the FreeNAS blog article on the subject.

With the SSD connected to the Microserver’s ODD SATA port I created two partitions and added these as L2ARC and SLOG as follows:

[root@hpv-01] ~# zpool add vmDiskPool cache ata-Corsair_Force_LS_SSD_1234567890987654321-part1
[root@hpv-01] ~# zpool add vmDiskPool log ata-Corsair_Force_LS_SSD_1234567890987654321-part2
[root@hpv-01] ~# zpool status vmDiskPool
pool: vmDiskPool
state: ONLINE
config:
        NAME                                                    STATE     READ WRITE CKSUM
        vmDiskPool                                              ONLINE       0     0     0
          mirror-0                                              ONLINE       0     0     0
            ata-TOSHIBA_HDWD110_66OABCDEF                       ONLINE       0     0     0
            ata-TOSHIBA_HDWD110_66OFEDCBA                       ONLINE       0     0     0
        cache
          ata-Corsair_Force_LS_SSD_1234567890987654321-part1    ONLINE       0     0     0
        log
          ata-Corsair_Force_LS_SSD_1234567890987654321-part2    ONLINE       0     0     0

This did improve performance slightly. However due to my workload the improvements were only marginal.

This had also introduced a single point of failure into my pool and I learned very quickly that ZFS does not like when a single, non-redundant SLOG device becomes unavailable unexpectedly.

I should mention that this was not the fault of the SSD but turned out to be due to the cheap ODD bay converter.

Solution

The solution was quite obvious and that was to upgrade to SSD storage for my VM’s. I did briefly consider switching to a pool of mirrors, the ZFS equivalent of RAID 10. However as the Microserver only has 4 3.5” bays this would not have been overly practical.

As the majority of my data is stored on my NAS I only needed approximately 250GB of storage space for VM’s.

After checking PcPartPicker I settled on a pair of Crucial MX300 275GB drives which at the time offered the best capacity, reliability and performance for the price.

I also bought two bay converters that are designed to work on hotswap trays as with the majority of converters, the SATA ports do not line up.

Crucial MX300 Bay Converter

Migration

As it stood, the microserver had a single 250GB boot drive, a 500GB “scratch” drive and the two 1TB 7200RPM drives for VM storage.

To migrate I first removed one of the 7200RPM drives, degrading my existing mirror. While far from ideal, I did have good backups in the event that I had to restore.

I then inserted one of the new SSD’s and built a single disk pool SSD pool as follows:

[root@hpv-01] ~# zpool create vmFlashPool ata-Crucial_CT275MX300SSD1_1423561345

I added this new pool as ZFS storage in Proxmox and migrated my virtual machines and containers.

Once everything had been moved off the old pool I removed it from Proxmox, destroyed the pool, removed the drive and replaced it with the second SSD. I was then able to attach the second ssd to the first as a mirror:

[root@hpv-01] ~# zpool destroy vmDiskPool -f
[root@hpv-01] ~# zpool attach vmFlashPool ata-Crucial_CT275MX300SSD1_1423561345 ata-Crucial_CT275MX300SSD1_9876987734 -f
root@hpv-01:~# zpool status vmFlashPool
  pool: vmFlashPool
 state: ONLINE
config:

        NAME                                         STATE     READ WRITE CKSUM
        vmFlashPool                                  ONLINE       0     0     0
          mirror-0                                   ONLINE       0     0     0
            ata-Crucial_CT275MX300SSD1_1423561345    ONLINE       0     0     0
            ata-Crucial_CT275MX300SSD1_9876987734    ONLINE       0     0     0

After waiting a short while the pool resilvered, resulting in a pair of mirrored SSDs holding all of my VM’s.

Summary

Since migrating to SSD storage I don’t think I can ever go back.

The performance is far better, I no longer experience any issues during backups and VM’s boot almost instantly.

I also replaced my 500GB scratch drive with one of the 7200RPM 1TB disks and was also able to repurpose the 60GB Corsair SSD as L2ARC for this pool.

My current ZFS configuration as of today is as follows:

root@hpv-01:~# zpool status
  pool: vmFlashPool
 state: ONLINE
  scan: scrub repaired 0 in 0h22m with 0 errors on Sun Jun 11 00:46:18 2017
config:

        NAME                                         STATE     READ WRITE CKSUM
        vmFlashPool                                  ONLINE       0     0     0
          mirror-0                                   ONLINE       0     0     0
            ata-Crucial_CT275MX300SSD1_1423561345    ONLINE       0     0     0
            ata-Crucial_CT275MX300SSD1_9876987734    ONLINE       0     0     0

errors: No known data errors

  pool: vmScratchPool
 state: ONLINE
  scan: scrub repaired 0 in 3h35m with 0 errors on Sun Jun 11 03:59:09 2017
config:

        NAME                                             STATE     READ WRITE CKSUM
        vmScratchPool                                    ONLINE       0     0     0
          ata-TOSHIBA_HDWD110_66OABCDEF                  ONLINE       0     0     0
        cache
          ata-Corsair_Force_LS_SSD_1234567890987654321   ONLINE       0     0     0

errors: No known data errors