SSD cache to keep HDD powered down
My problem is this: I have a 1TB SSD and a 16TB HDD for home server running various file serving services (Syncthing, Copyparty, Jellyfin, Navidrome), among other things. Most of the regularly accessed files should fit on the SSD and I want to keep the HDD…
My problem is this: I have a 1TB SSD and a 16TB HDD for home server running various file serving services (Syncthing, Copyparty, Jellyfin, Navidrome), among other things. Most of the regularly accessed files should fit on the SSD and I want to keep the HDD spun down most of the time to keep it quiet. So my idea would be this: point the services at a virtual file system showing all the files on the HDD. The actual file index, all necessary metadata and full copies of the last n accessed files permanently live on the SSD. Writes always hit the SSD first and updates to the HDD are batched. So ideally the HDD would be powered on strictly only on scheduled writes or when a file not in the permanent cache is requested. This must be a fairly common problem, but I can't find a good dedicated solution, and chatbots keep going in circles of recommending solutions (mergefs+fscache, ZFS with persistent L2ARC, bcachefs,...) then admitting that they don't work as required. It seems that even unRAID has the problem that most reads will hit the HDD because files get moved out of the cache (also, I'd prefer a solution I can configure in nixOS). I want my cache to be permanently filled close to full capacity, so that the HDD is rarely needed. It also seems tricky to tell all parts of Linux to not wake up the HDD at random times. The thing is, if you have such a big HDD with 7200rpm, you either keep it powered on permanently like it's meant to be, or you really shouldn't be spinning it up often. Would appreciate any pointers/suggestions.
已收录讨论
Expand the replies to this comment to learn how AI was used in this post/project.
Wonder if optane like "tiered storage" tech will return with the insane jump in pricing lately. As the tech itself wasn't bad, but the cost of SSDs dropped quickly enough to make it quickly obsolete.
I think in the default settings it would still wake up the HDD quite often. That could maybe be disabled, but the bigger problem I see if you use a block-level caching system the way described in the post is that there's a risk of corrupting the HDD file system if your system goes out of sync.
Corruption is easy enough to solve, you just use parity data. Maybe snapRAID? I remember looking at it for redundancy purposes rather than cache/replication.
Yea it's kinda weird how bad most Linux systems are at keeping HDDs spinned down, especially where talking about dedicated NAS/self hosting systems
Great to know, thank you! If only there was a more flexible way of using it. I don't even have that big of a problem with paying, even though $250 feels a bit steep, but more with the fact that I don't want to have to use a specific, quite locked down OS.
Yeah, it's a problem that's probably strictly confined to home servers, and most of Linux is aimed at enterprise. But things like unRAID exist, showing that there is a market for products focused on the home server niche.
Thank you! policyfs sounds like someone who had the exact same problem decided to open source their solution. I've also been thinking about just writing my own mover script. Out of curiosity, could you describe your stack? What file system (ext4, btrfs,...) do you use? Do you use something like snapRAID for parity? How did you set up your mover script?
Honestly my needs are pretty simple since my ssd is large. I have a 4TB SSD that everything gets downloaded to. I have a nightly sync job that copies everything from the SSD to the HDD pool. Every 6 months or so when the SSD starts to fill up by just manually go in and prune things. I access files on the HDD rarely enough that I just don't bother to promote things to my SSD and live with accessing an old file spinning up the hard drives. I don't quite have the needs you do. I might write a mover script one day but my system has been working well enough.
Unraid does exactly what you're looking for. The mover tuning plugin lets you configure the SSD cache to fill up as high as you want, and then offload files on a schedule (I have mine set to check every 2 hours) to the HDDs based on oldest file to newest. I have mine set to fill up my 4TB cache to 70%, then offload to the HDDs until my cache drive is at 50%. My cache drives stay between 2-3TB and all my newest media plays instantly and keeps my 12 HDDs off the majority of the time.
Does bcache not work for this? https://wiki.archlinux.org/title/Bcache Edit: Evidently newer versions of LVM also support this out of the box
https://github.com/hieutdo/policyfs or https://github.com/DudeCmonMan/fscache/tree/main or just manage it yourself with mergerfs (what I do)
So you want to reinvent Intel Optane, a now defunct product because that basically was almost its whole goal to pair flash storage with a normal spinning drive for performance sake.
Not worth it - keep them spinning all the time
I think this is achievable. There isn't a dedicated solution because most folks probably don't mind leaving their disk spinning to avoid a long pause. Wouldn't MergeFS across an SSD volume and ZFS pool ( small portion of SSD as special vdev + HDD as data vdev ) and some scripting do exactly what you want? You would need to configure MergeFS for writes to hit the large SSD portion first, and set a schedule for your script shuffling less accessed files to the HDD when the SSD is over xx% capacity etc. The caveat here is that it would make data on this storage fragile - if the SSD were to be corrupted or die the data is gone, but you can mitigate this by mirroring the special vdev. I do agree - you'll need to do a lot of monitoring to figure out what is accessing your disk if it doesn't sleep as expected. Ignoring the potential reliability issues BCacheFS seems to nearly check all the boxes except that it manages the housekeeping for you on it's own schedule.
I played a bit with this using LVM cache. But found that old `bcache` was much closer to this (not bcachefs, this one https://docs.kernel.org/admin-guide/bcache.html ) . You can even use it with writeback mode to 'cache' writes if you trust your cache SSD or don't care about data... But i've not tried bcachefs.. Also note that you'll always fight with some random things like SMART monitoring, background jobs, etc.. So `sudo blktrace -d /dev/sda -o - | sudo blkparse -i -` will be your best friend...
这条评论已被删除。
这条评论已被删除。