23
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
this post was submitted on 18 Jun 2024
23 points (89.7% liked)
Selfhosted
40438 readers
247 users here now
A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.
Rules:
-
Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.
-
No spam posting.
-
Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.
-
Don't duplicate the full text of your blog or github here. Just post the link for folks to click.
-
Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).
-
No trolling.
Resources:
- selfh.st Newsletter and index of selfhosted software and apps
- awesome-selfhosted software
- awesome-sysadmin resources
- Self-Hosted Podcast from Jupiter Broadcasting
Any issues on the community? Report it using the report flag.
Questions? DM the mods!
founded 2 years ago
MODERATORS
Seconding this. For starters, when tempted to go for Raid5, go for Raid6 instead. I've had drives fail in Raid5, and in turn have a second failure during the increased I/O associated with replacing a failed drive.
And yes, setting up RAID wipes the drives. Is the data private? If not, a friendly datahoarder might help you out with temporary storage.
I run RAID5 on one device.... BUT only because it replicates data that's on 2 other local devices AND that data is backed up to a cloud storage.
And I still want it to be RAID 6.
Story time!
In this one production cluster at work (1.2PB across four machines, 36 drives per machine) everything was Raid6, except ONE single volume on one of the machines that was incorrectly set up as Raid5. It wasn't that worrysome, as the data was also stored with redundancy across the machines in the storage cluster itself (a nice functionality of beegfs), but it annoyed the fuck out of me for the longest time.
There was some other minor deferred maintenance as well which necessitated a complete wipe, but there was no real opportunity to do this and rebuild that particular RAID volume properly until last spring before the system was shipped off to Singapore to be mobilized for a survey. I planned on getting it done before the system was shipped, so I backed up what little remained after almost clearing it all out, nuked the cluster, disassembled the raid5, and then started setting up everything from scratch. Piece of cake, right?
shit
That's when I learned how much time it actually takes to rebuild a volume of 12 disks, 10TB each. I let it run as long as I could before it had to be packed up. After half a year of slow shipping it finally arrived on the other side of the planet, so I booked my plane ticket and showed up a week before anyone else just so I could connect power and continue the reraiding before the rest of the crew showed up. Basically, pushing a few buttons, followed by a week of sitting at various cafes drinking beer. Once the reraid was done, reclustering was done in less than an hour, and restoring the folder structure backup was a few hours on top of that. Not the worst work trip I've had, except from some unexpected and unrelated hardware failures, but that's a story for another day.
Fun fact: While preparing the system for shipment here in Europe, I lost one of my Jabra bluetooth buds. I searched fucking everywhere for hours, but gave up on finding it. I found it half a year later in Singapore, on top of the server rack, surprised it hadn't even rolled down. It really speaks to how little these huge container ships roll.
Fun story but I’m most impressed with the earbud part of the story. WOW. Absolutely amazing and unexpected.
Haha, everything about that story is awesome, right down to the lost and found Jabra ear bud (does Jabra exist any more? At one time their ear pieces were the best).
Yes, re-silvering takes fucking forever. Even with my little setups (a few TB), it can take a day or two to rebuild one drive in an array. One.
I can only imagine how long a PB array would take.
Jabra still exists yes. I'm still using Jabra, although I'm using a pair that I bought after I thought that one earbud was gone forever. I still use the older ones, which was Jabra Elite 4, but only with my PC, as its battery took a hit after those 6 months at sea. I currently main Jabra Active 7 or something like that, and I quite like them. I noticed that the cover doesn't stay very attached after a few proper cleans, but nothing a drop of glue doesn't fix. What I really like about the ones I currently use is that they're supposedly built to withstand sweat while training. I don't work out, but it would seem that those who do sweat A LOT, as I can wear mine while showering without any issues.
As for resilvering, the RAIDs are only a small fraction each of the complete storage cluster. I don't remember their exact sizes, but each raid volume is 12 drives of 10TB each. Each machine has three of these volumes. Four machines total contributes all of its raid volumes to the storage cluster for 1.2PB of redundant storage (although I'm tempted to drop the beegfs redundancy, as we could use the extra space, and it's usually fairly hassle free to swap in a new server and move the drives over).
EDIT: I just realized that I have this Jabra confference call speaker attached to the laptop on which I'm currently typing. I mostly use it for discord while playing project zomboid with my friends, though. I run audio output elsewhere, as the jabra is mono only.
If i goes to raid5 i lost one disk of space, to go to raid6 i have to lost 2 disks.
Its a pesonal proyect, and the motherboard has only 6satas, one of them used by the SO disk, and i want to be able of upgrade it in a future...
Wut...
I think you're missing the point of RAID here, possibly. Where's the reliability in this?
Not to speak for the person above you. But I believe they are saying they have 1 computer with a raid5 array, that backs up to two different local servers, and then at least 1 of those 3 servers backs up to a cloud provider.
If that is true then they are doing it correctly. It is highly recommended to follow a 3-2-1 storage solution, where you back up to a local backup and a cloud backup for redundancy.
Ahhh, makes sense. That kind of wrecked my brain for a moment.
Lol, sorry, I really tried to make it clear what I was doing, honest, I did! 😄
Yes, I have 3 local devices that replicate to each other, one is RAID5, (well, 2 are, but...not for long). And one of them also does backup to a cloud storage.
Not ideal, because 3 devices are colocated, but it's what I can do right now. I'm working on a backup solution to include friends and family locations (looking to replicate what Crashplan used to provide in their "backup to friends" solution).
It's possible to convert drives to RAID in-place... but strongly discouraged.
Since OP will have a blank drive, they could play musical chairs by setting up a new RAID on the new empty drive, copy data from one drive, wipe that drive, grow the array, copy data from the third drive, wipe, grow... But that's going to take a long time, and you'll have to keep notes about where you are in the process, lest you forget which drive is which over the multiple days this will take.