Solved What BYO multi-terabyte storage solution do you recommend

dacrone · Nov 12, 2023

Yeah I have one but only use it if necessary. I just use aomei weekly to ~~backup~~ clone (edited to use correct terminology

) my SSDs with OSs. Any hdd with data are running in raid (NAS, media server, etc).

johnlgalt · Nov 13, 2023

To further this, I would not be using Windows for long term storage either. Much better to use a dedicated system (and one preferably not running Windows - Like the Synology boxes, or a home-built True NAS box.

Baeolophus · Nov 13, 2023

dacrone said:
Build a nas with hdd. Synology with WD red pro 14tb in mirror raid. Easy for everyone to operate too.

Berton said:
I use a Buffalo LinkStation 2TB NAS, has 2 x 2TB Seagate HDDs running RAID 1 [mirror] for 2TB total storage. It is connected to my Wireless Router via Ethernet cable and any computer connected to the Router Wired or Wireless/Wi-Fi can have the main folder on it Mapped so as to store and have access to files in the subfolders.

Are those higher-capacity (> 2 TB) HDDs reliable? Even at 512 GB and 1 TB I had found in the past that some of did not last even three years. That’s why I am looking at RAID-5. It seems to me that there is an inverse relationship between capacity and reliability.

Baeolophus · Nov 13, 2023

barreleye said:
I'd skip RAID in all its forms and apply the savings to two sets of backup drives, one stored off-site.

My secondary, off-site backup is to my OneDrive using Arq.

dacrone · Nov 13, 2023

i've never had one fail personally. but again, thats what raid is for... and the thousands of days of warranty WD gives you (be sure you register your hdds' warranties!). i've ran 4, 8, and 14tb drives for years with daily read/write and never had issues.

johnlgalt · Nov 13, 2023

I think he might have been specifically referring to Seagate high capacity drives.

Berton · Nov 13, 2023

antspants said:
I believe that is exactly what happened. The only thing I remember clearly, is the grief. For some reason, SCSI springs to mind, also.

SCSI was going out as I got an after-retirement job in a computer store building computers and IDE becoming the prevalent drive type for PCs. IDE was the Parallel connection [PATA] and could have a Master and Slave drive on the same cable, now pretty much replaced by SATA, one drive per cable/connection.

Baeolophus said:
Are those higher-capacity (> 2 TB) HDDs reliable? Even at 512 GB and 1 TB I had found in the past that some of did not last even three years. That’s why I am looking at RAID-5. It seems to me that there is an inverse relationship between capacity and reliability.

Mine has been okay for a couple years now, the old WD drives in my previous MyCloud NAS drives are working in 2 desktops as secondary data drives, one Win10 and the other Linux Mint.

barreleye · Nov 13, 2023

Baeolophus said:
My secondary, off-site backup is to my OneDrive using Arq.

That's good, but "multiple terabytes" as per thread title?

Never heard of it, but Arq looks interesting. I have a small amount of more frequently changing data, and I backup to a couple of Bitlockered VHDX files, one daily, the other larger and weekly, that are copied automatically to OneDrive. This is in addition to my multiple terabytes I back up to two sets of bare hard drives. I use SyncBackPro for all these file-based backups, though I do prefer letting OneDrive do the uploading of the VHDX files, as it's faster and requires no additional effort and maintenance (expiring credentials) to set up and use over time.

Dru2 · Nov 13, 2023

Baeolophus said:
I am looking to put together a storage solution with something like 12–16 TB of storage, so that I have plenty available for our family’s common files and local backups. Should I look to

build a NAS with a hard disk RAID,

build a NAS with an SSD RAID,

build a RAID enclosure with hard disks, attached to one of my computers, or

build a RAID enclosure with SDDs, attached via USB-3 or -4?

If using SSDs, are their RAID solutions for NVMe drives? Can you recommend specific ones? If not SSDs, should I use NAS hard drive? I am leery of using hard drives, because of the diminishing reliability as their storage capacity increases.

I don't see the viability of an SSD NAS as storage is going to be extremely limited, unless you are willing to shell out a ton of money for high-capacity drives.

Anyway, I bought a Western Digital 4-Bay NAS enclosure (PR4100) and slapped 4 x 6TB WD Red Pro HDD drives. I'm running it in RAID 10. The cost of RAID 10 is you only get half the total capacity of installed drives - 4 x 6TB = 24TB but halved by RAID 10 = 12 (10.8 useable).

I did worry the cost of running RAID 10 was too high (losing half your storage space) but after 4 years running like this, I've found I've only used 3.5TB of space thus far, and this is with my 1.5TB of music files that I stream, as well as system backups. The positive is fast read/write speeds near SSD levels, and redundancy.

Mine is cloud based so I can access it anywhere on the net, even from my Android device. It also supports Plex natively, so if you want to do streaming, it's ready for that as well, and also why I got the model I did.

It's been running 24/7/365 for 4-years now without issue, and WD keeps both NAS OS and unit updated via automatic updates. The only thing I did when I got the unit was upgrade it from the base 4gig of RAM to 16gig of RAM.

dacrone · Nov 13, 2023

johnlgalt said:
I think he might have been specifically referring to Seagate high capacity drives.

i refuse to use seagate. had to do data recovery for at least 4 people (that i can remember) who had seagate drives fail on them. SSDs - i stick with samsung evo and storage HDD - i stick with WD Red Pro.

johnlgalt · Nov 13, 2023

I've always been the opposite - I've had to do data recovery from numerous WD drives over the years, while only one Seagate, and the two I lost were a result of direct frying from a lightning strike outside my home.

To each their own.

Dru2 said:
I don't see the viability of an SSD NAS as storage is going to be extremely limited, unless you are willing to shell out a ton of money for high-capacity drives.

This. That's a ton of money, even with lower costs of SSDs these days.

hdmi · Nov 14, 2023

Baeolophus said:
Are those higher-capacity (> 2 TB) HDDs reliable? Even at 512 GB and 1 TB I had found in the past that some of did not last even three years. That’s why I am looking at RAID-5. It seems to me that there is an inverse relationship between capacity and reliability.

Backblaze Drive Stats for Q2 2023

Read the full Q2 2023 Drive Stats report is here, and we've added some data points and new drives to the data.

www.backblaze.com

hdmi · Nov 14, 2023

johnlgalt said:
I think he might have been specifically referring to Seagate high capacity drives.

A lot of people on here are Seagate haters, but the harsh reality is that every brand makes both good models and bad models, and, you also need to consider the cost per GB. Because, if you choose to buy two Western Digital drives at $150 each, then if you can buy three Seagate drives of equal capacity at only $100 each, you'll still end up paying a total of $300 in both of these two different example scenarios. That is, despite that these three Seagate drives will give you 50 percent more storage capacity in total (when compared to choosing to go for the two Western Digital drives instead, i.e. as a result of falling victim to persistent myths of "significantly more reliable"). But the difference in failures rate is still very typically waa-aaaa-aaaaaaay less than 50 percent. Which is so obvious that it hurts each time when I laugh about it of course, and I laugh about it often.

Seagate Performed Best in Backblaze's 2020 Hard Drive Failure Report

All drives saw notable improvements in failure rates.

petapixel.com

johnlgalt · Nov 14, 2023

Yeah. But trying to change people's minds is like ....

:ffs:

Dru2 · Nov 14, 2023

Everyone has their favorites; I say stick to what's worked for "you". For "me" it's WD HDDs, and Samsung SSDs.

pparks1 · Nov 14, 2023

Just re-iterating that RAID is not a backup. It's a redundant set of disks. It should never be considered a backup. It's protection against a disk failing, but that's not a backup.

For example, a RAID 1 mirror in it's simplest sense is 2 physical hard drives, combined to show as 1 drive. Each drive will have a mirror of the data on it. So, drive 0 will look exactly like drive 1. If drive 0 were to fail, drive 1 would continue to operate and present the data. RAID 0 is good for an OS server drive where you need to ensure that if the OS drive failed, it would have another drive to continue to run and boot from.

However, if you delete, destroy or negatively edit a file on a RAID 1 mirror, it's going to be deleted, destroyed or negatively edited immediately on that second disk as well. You don't have a "backup" copy of the file that you can revert back to in the event that you needed to go back. So, if you get hit with ransomware, it's hit both of your drives at once and you don't have a way to recover, unless of course you have a backup copy of the data somewhere else.

i think using something like TrueNas or a physical NAS box like a synology is a great place to use RAID. It takes the complexity out of the RAID from the user and just lets the person easily replace a drive which has failed.

But with any solution, if you are going to have 14TB of data, be sure that you have at least 14TB of backup space...ideally offsite in another location in case your house burns down.

Dru2 · Nov 14, 2023

I don't think anybody here believes RAID an (adequate) backup solution. "Adequate" because with RAID setups where "Redundancy" is included, the other disk contains mirrored info, so technically your data is backed up in that should one drive die, the data is still on the other drive. Redundancy.

But yes, RAID is a singular unit, so if your RAID goes, so does the info, and why you should backup any info stored on a RAID setup.

Edit: "I don't think anybody here believes RAID an (adequate) backup solution." I take that back...

dacrone said:
I would strongly suggest raid for real time backup… but a secondary backup is always good.

Baeolophus · Nov 14, 2023

pparks1 said:
Just re-iterating that RAID is not a backup. It's a redundant set of disks. It should never be considered a backup. It's protection against a disk failing, but that's not a backup.

Dru2 said:
I don't think anybody here believes RAID an (adequate) backup solution. "Adequate" because with RAID setups where "Redundancy" is included, the other disk contains mirrored info, so technically your data is backed up in that should one drive die, the data is still on the other drive. Redundancy.

For me the RAID set would be used as an external storage volume on a spare Mac that would make it act as a Time-Machine-specific NAS. I am leaning toward Synology, however. Maybe a four-bay unit with four 4- or 6-TB drives in it.

Dru2 · Nov 14, 2023

Baeolophus said:
For me the RAID set would be used as an external storage volume on a spare Mac that would make it act as a Time-Machine-specific NAS.

The point is you understand RAID to be a single volume drive, so critical data on it should be backed up as well.

Baeolophus said:
I am leaning toward Synology, however.

It doesn't matter to me, just make sure it fits your needs.

Baeolophus said:
Maybe a four-bay unit with four 4- or 6-TB drives in it.

It all comes down to money as to what to get. My setup - WD NAS 4-bay enclosure, 4 x 6TB drive, and extra RAM ran me $1300 bucks. I went with a driverless NAS cause wanted to control what drives and size of drives I put in it.

Of course, you can get an NAS setup with drives included, but that still can get costly depending on size and number of drives.

There is the option to build a NAS, which I certainly considered since I build my own PCs; but that requires having the hardware, the know-how in building (including BIOS / RAID configuration) and installing and configuring the NAS OS. Additionally, if you want to run your NAS 24/7/365 as I do, you're going to want drives designed for NAS use - example Western Digital RED Pro drives.

In the end, though I have the know-how (in building), I found it simpler and easier to buy a NAS setup; especially since no matter the route, I needed to buy drives anyway. And the net difference between buying and building was less than a hundred. That's me.

Marie SWE · Nov 14, 2023

Ghot said:
Skip RAID... it's a pain in the butt.

Get two of these and use Karen's Replicator (free), to make one a copy of the other... (5yr warranty)
They seem to be available for a great price, too.

Seagate EXOS Enterprise 16 TB 3.5" 7200 RPM Internal Hard Drive (ST16000NM001G) - PCPartPicker

Seagate EXOS Enterprise 16 TB 3.5" 7200 RPM Internal Hard Drive (ST16000NM001G)

pcpartpicker.com

Karen's Replicator the Backup Utility for Windows v3.7.6 Released May 25, 2020 | Karen's Power Tools | Karen's Software

Automatically backup files, directories, even entire drives! Karen's Replicator copies selected files from one drive/folder to another. Source and Destination folders can reside anywhere on your network.

www.karenware.com

Hi Ghot

I'm actually a bit surprised to see you say RAID is a pain in the butt, i thought you was a RAIDer :wink:

-------------

and for the Thread

For pure home backup once or twice a week on 2/5devices.. I Say use a shared drive and FreeFileSync and then you sync that drive to an external drive(or two) 1=0... 2=1... 3= is a backup (one of those three for offsite storage...read housefire)

If you are a nerd that like fun stuff.. :wink:

Go NAS or a server..... with real hardware RAID6 on an 6 or more disks pool...
NOTE...... (RAID is not backup.. RAID is redundancy)
Everything you have plugged in can die on you within the next few seconds.... (NVMe/SSD/HDD/SD... Raid or non-raid)... Gone.. all it takes is One Power-spike from a faulty PSU or Lightning strike or Ransomware and it all is just gone.... bye-bye in the wind.
You always use offline backups..

This is my infrastructure.
I have 3 laptops and 4 desktops one server for primary usage.
All my laptops and desktops backup/sync against my server that has a hardware RAID solution.. 2disk RAID1 for server-OS.. then a 4disk RAID5 setup for data storage.
On my clients I use PXE boot for system image backups and restore.....Then i use Filesync for personal data backup against my server..
Then i backup my server to external disks.... But i do not backup everything from server to external disks as some is what i call junk-files.. nice-to-have-files, that is okay to lose if a lightning strikes or a ransomware hit.. like ISO images files, install programs etc that is easy to download again but takes a lot of disk space and so on so you dont want to have multiple backups on.. "Junk-files"

Then i have the important data...... The ones you never can get back..... Family photos, contracts or other important-important files..
This one i have a multi-multi backups on.... Not only on separate disks... but also on mirror folders on the same disk.
Why... It is called Bit-Rot.. yeah it is a thing.. and it hits all media NVMe/SSD/HDD etc.
I use three duplicate folders on all backup disks..... If you get Bit-Rot, 98% of the time, the file size changes with one or more bytes on that file. So if you have three folders and 1000files in them and you have total folder size of 75283635bytes and then you see one of the other folder is on 75283634bytes.. Then something is wrong and you need to find the file missing 1byte... Then you have the evil 2%.. the size is not changed, but when trying to open the photo you get a corrupted gray image or a flat line or half the pucture or in rainbow colors.... only they eye and a hash/checksum can detect that.... I use Hash/checksums as I'm to lazy to look thru 10 thousand photos just to spot a bad file. *LOL*

So Junk-files("nice to have files") i let the server RAID keep them.
kind of important data... Server + 2 Disk backup
important data... Server+ 3disk backup (one of them off-site)
important-important data... 3 duplicate folder backups + server + 3disk backup (one of them off-site)
One of those three disks is for offsite storage(cloud if you want) or in a bank safety deposit box or at a family member/friends place.. as a house fire eats disks for dinner... and a burglar won't leave one backup disk behind for you out of kindness.. :wink: