Media Server

[colemj]

So it would take 40 $25 optical disks to store 2TB? How much does the burner cost -$500?

$1,500 will buy me 20 2TB hard drives (that's today - next year it will buy 30 of them). Surely ONE of those would last?

JRM's main argument, though, is that we have too much media. A strange judgement, indeed.

Even if I just kept only my one favorite movie and song and book, optical media makes no sense - a thumb drive would be better (and is way cheaper).

Mark

[hellosailor]

dockhead-
The basic promise of RAID is that it keeps on running even if a drive fails. Great, right, sure. Except, the actual implementation can mean that you need to have an identical spare drive available for the hot swap, or you may need extra administrative skills, because all RAID systems don't just cut over and rebuild and let you swap out drives like toast from a toaster. Ever burnt toast, by the way?

The simplest way to build a reliable media library is to simply buy *3* drives. Build your library on one, then copy it to the two spares and put them away in case #1 fails. Which is eventually will, we all know. As our Mr. Gates said in the 80's, "Hardware is cheap". The system is as simple as it gets.

Now, if you are using hard drives ON A BOAT, bear in mind that you are already officially abusing them. They are not supposed to be IN MOTION while being used. Even a laptop drive is designed to be transported--not used in motion. The new drives have accelerometers built in and in theory the heads lift and park before there is enough motion to cause damages. In theory. But if you examine the specs on any drive it still will show limited g forces both while in use and while in storage.

So yes, you can and should expect drive failures on a boat.

The alternative is to use SSDs, solid state hard drives, which are even more pricey and also have a limited life, limited more by the number or write cycles than read cycles. for a media library where data is pretty much just "written once read many" that shouldn't be a problem--but I'd still make and put away a conventional hard drive as a backup.

And of course for music files, you can buy 128GB or 256GB USB drives fairly cheap these days, robust and easily backed up.

Bigger problem that I see is for movies. If you copy iso images of disks, a lot of players won't pay them. If you copy files in another format...same thing, good luck getting a player to recognize them and then you need to spend even more time converting formats, not just ripping them. Still a bit labor intensive and I keep thinking there's got to be a better way to come.

[goboatingnow]

The simplest way to build a reliable media library is to simply buy *3* drives. Build your library on one, then copy it to the two spares and put them away in case #1 fails. Which is eventually will, we all know. As our Mr. Gates said in the 80's, "Hardware is cheap". The system is as simple as it gets.

I would suggest that all mirrors are online and running, firstly this keeps teh drives at a constant temperature, secondly it allows for incremental and out of hours backup processes to run and all drives fail on startup. Just accept there is a life cycle to drives, but with 3 units as you suggest , just let one fail anyway.

The alternative is to use SSDs, solid state hard drives, which are even more pricey and also have a limited life, limited more by the number or write cycles than read cycles

SSDs are limited really only by write cycles, which is good news for media servers, furthermore advanced techniques like over provisioning, bad block management and wear levelling, mean most SSDs will now outlast the hardware driving them, and 3-year warranties are common enough. Even with NAND cells getting smaller ( and technically less reliable) modern algorithms , ECC etc are proving up to the challenge.

Were close to the end of the spining disk era for aggressive random read write applications

[avb3]

As many I have gone through numerous hardware failures. In addition I have found that backup systems are not necessarily compatible with the new computer that one obtains. Having dealt with this frustration a number times, I have now gone on to an online backup system.

I have to use Dropbox for my music, and Google automatic backup system for photograph I take with my smartphone.

It works for me. YMMV

[maytrix]

I apologize in advance as I haven't read all the replies, but here's what I'd look into:

1. Get crashplan online backup. Cheap for the amount of data you'd be backing up - always on as well, so changes are backed up instantly - obviously this won't be that useful on the boat, but even if nothing changes you'll have a copy in the cloud.

2A. Build a PC with RAID in it and store everything on that. You could easily get 3 1TB drives and configure in RAID 5

or

2B. You could get a Drobo. This is a standalone device that can fit a variety of number of hard drives and gives you the same redundancy. There's a number of configuration options for this as well.

[colemj]

Originally Posted by avb3

As many I have gone through numerous hardware failures. In addition I have found that backup systems are not necessarily compatible with the new computer that one obtains. Having dealt with this frustration a number times, I have now gone on to an online backup system.

I have to use Dropbox for my music, and Google automatic backup system for photograph I take with my smartphone.

It works for me. YMMV

That works if one has reliable, inexpensive and fast internet connections. We haven't seen one in several years. I can't imagine what accessing 2TB of on-line data would mean to us in terms of cost and time here. Well, the cost is pretty straight forward - it would cost $5,000.

I don't see how hard drives become incompatible with new computers. Even if they did (serial? Firewire? SCSI?), you can buy a new one for almost nothing.

Mark

[gjorgensen]

A little late to the party, but I can highly recommend the NAS I have at home for all our media: Synology DS212j 2 disk NAS. Amazon has them for $200 right now. Stuff a couple big NAS-specific drives in there (I put in two Western Digital 2TB "red" drives) and you have a very low power solution. The default is to do RAID 1 (mirroring) which is what I did for data protection. You can also hook up an external drive to do a backup (which we also did, and keep offsite). This is basically an ARM-powered Linux box you admin via a web interface. It'll sleep most of the time, with wake-on-access. Nice robust little box.

[avb3]

Originally Posted by colemj

That works if one has reliable, inexpensive and fast internet connections. We haven't seen one in several years. I can't imagine what accessing 2TB of on-line data would mean to us in terms of cost and time here. Well, the cost is pretty straight forward - it would cost $5,000.

I don't see how hard drives become incompatible with new computers. Even if they did (serial? Firewire? SCSI?), you can buy a new one for almost nothing.

Mark

The problem I ran into in one particular case, was I had mirrored drives, and then required to have the same hard drive. When the one hard drive crashed the manufacturer longer made that drive.

It took me forever to backup the remaining drive, as for some reason when the first one failed, the second one somehow got corrupted.

[goboatingnow]

Ive been involved in RAID system for several years on and off.

I would not recommend a RAID based solution for a media server. RAID 5 in particular often is very finicky with the drives and means you cant move the drive array from one RAID hardware to the next , or the replacement drives have to be the same. Its a outdated technology from the days of low MTBF drives.

Mirroring at a operating software level is much better, in particular you want a industry standard disk structure , so that the mirror can be recovered in different hardware.

With disks being so large a 1-3TB software mirrored array is cheap.

Cloud based backup is also available , Amazon Glacier works out around $10 a month for 1TB ( plus data out charges) its not suitable for regular streaming though , but putting data in costs nothing extra.

Ive grown to hate "custom" NAS like Buffalo etc, in my view its not the disk that fail first, and then you can be left with a PITA. a little linux box makes a nice NAS with standard SATA disks.

Offsite backups are really better handled now with cloud solutions, so you never have to worry about forgetting to bring in the backup drive and take it away again, ( so 1990's)

Dave

[JRM]

Originally Posted by colemj

So it would take 40 $25 optical disks to store 2TB? How much does the burner cost -$500?

Mine came with my NAS box, but I think the burners are around $75 if you bought one standalone. And discs aren't $25. You can buy the 50GB discs from Verbatim in a 50 pack for $150 shipped from Amazon. So $3 each. Still not ideal for the amount of data you're talking, but not exorbitant. Everything I care to backup fits on 2, so I spend a grand total of about $15 a month (two sets of offsite backups and I don't buy that many at once).

Originally Posted by colemj

$1,500 will buy me 20 2TB hard drives (that's today - next year it will buy 30 of them). Surely ONE of those would last?

JRM's main argument, though, is that we have too much media. A strange judgement, indeed.

I'm not judging your media volume. I just find it odd that someone with tens of thousands of dollars worth of entertainment balks about $500. If I'd invested that much in amassing such a collection, I'd probably be very interested in ensuring it's integrity myself.

Originally Posted by colemj

Even if I just kept only my one favorite movie and song and book, optical media makes no sense - a thumb drive would be better (and is way cheaper).

Mark

I'm not standing over you with a baseball bat screaming "YOU MUST USE OPTICAL MEDIA!" I'm merely pointing out that it's not insane. For me it's downright practical. Feel free to do something else. For me, even if I had that large volume of media I wouldn't bother backing it up, because it wouldn't bother me if I lost it. Other people see differently.

Oh, and on the discussion of the Raspberry Pi as a media server... I spent last night working on getting the Beaglebone black to run OpenCPN as a proof of concept for another project, and buried in the documentation I found that it's HDMI is limited to 24fps @ 1080p (only framebuffer drivers from TI atm). So it's out for that purpose. Definitely go with the Pi if you go that route.

JRM

[goboatingnow]

I'm not standing over you with a baseball bat screaming "YOU MUST USE OPTICAL MEDIA!" I'm merely pointing out that it's not insane. For me it's downright practical. Feel free to do something else

youve not seen this then, remember its the internet, resistance is futile, you will be assimilated.....

[maytrix]

I wouldn't rely on optical media, its prone to scratches.

Having the data on a hard drive and keeping a copy online is easy and reliable. Keep that data on two hard drives and the reliability goes up since you have a physical backup you keep with you, plus online. Odds of both being inaccessible are highly unlikely.

[colemj]

On-line backup only works if you have fast, inexpensive internet connections. For us out here, the cost to move data is $50/20GB. And each 20GB will take a day or two (or three) to move.

I don't ever see that as a reasonable solution for a cruising boat unless prices and performance change dramatically.

Mark

[maytrix]

I'm not at all suggesting you use online as a regular backup - but if you have data you want to protect, you could update online when available. Before we start cruising, all my data will be online, so the only challenge will be dealing with new data and my simply plan would be to try to get it backed up when decent internet was available.

So the only other method would be multiple backups. I'd keep a copy on my computer and then also back up to a drive or possible two and then store it in a water tight container.

[colemj]

Yes, each of our computers have two backups stored in water-tight containers. Our books and music media are included in these. Our movie media only has two copies.

Hard drives are incredibly inexpensive, so no problem having multiple ones around for backups.

We get to cheap internet land every 18-24 months.

Mark