Archiving

[DSE 3]

Curious who is doing what for archiving, if any at all?

[theonlyski 3]

I have a 750GB drive in my editing computer, all of the raw footage, vegas template and pictures are put (via a program I wrote to do so) into a folder on that drive.

End of the weekend, I copy all of the media from the weekend to an external drive and copy them to my 2TB NAS at home.

Been thinking about upping that 750GB to a 2TB, but it's not close enough to full enough to warrant it yet.

The program is pretty handy and it keeps the hierarchy of the folders consistent, which makes it easier to find things I'm looking for as well as keeping it tidy.

I put both cards in the computer (SD and Pro Duo)

Hit Alt+Home (keys I randomly came up with), it asks for the students name...

Then creates folders (MM-DD-YYYY\Studentname\ (pictures) or (video)) the program looks for the drives (as the letters may change), moves all the footage and pictures from the cards to the 750, copies my vegas template to the folder (and renames it to studentname.veg), opens the pictures and video folders, (so it's a quick check on pics, then drag and drop to burn) and also automatically opens the vegas template for that student, so I can start cranking on the video.

It saves a bunch of time making folders and whatnot. Another video guy at the DZ is using it (after some modifications to make it work right with his old ass XT) We both see a great improvement in the speed of it. Also don't have to format the memory cards after each jump, it moves the files, so I can pop the card out afterwords and it's ready for the next jump.



ETA: I also have the option of copying the media to my NAS with MS Synctoy via my VPN, but the DZ uses DSL which is SLOW trying to copy HD footage.

"I may be a dirty pirate hooker...but I'm not about to go stand on the corner." iluvtofly
DPH -7, TDS 578, Muff 5153, SCR 14890
I'm an asshole, and I approve this message

[goobersnuftda 0]

I took the issue of backing up from a completely different perspective. The key backing up is the case you house your PC in .... seriously. This is what you want.

The very unique thing to this case is that little small 3.5 bay slot at the bottom of the drive bays is a hot swappable bay for any SATA hard drive. As easy as people plug in a USB memory stick into a USB port, just slam in any SATA hard drive into that bay and away you go. No rails to install just the actual hard drive gets plugged in, the power supply is there as well as the connection to the motherboard all right as the computer is hot and running.

For $80 you can pick up a 2 TB hard drive and you have tons of removable storage. Nothing is faster than a hard wired data connection to move that data fast (especially HD formats).

The best way get the data from the SD or CF card to the HD is here. Just make sure your motherboard supports 3.0 otherwise it defaults to the slower 2.0. You will seriously appreciate this when you have those 10+ Gig files to move over. 2.0 seems so slooooooooow..... Write a simple batch file that takes the computer date and makes the folder names and away you go. If you make things simple and cheap, they will get done :) So much easier to take a backup of the tens of thousands of family pictures, videos, skydiving medias and with the push of a button, the HD ejects and you take it away for safe storage.

The fire department is great and all for coming and putting out the fire but as an added bonus, they fill up your place with water for no additional charge. Good thing the HD's are elsewhere.

[PharmerPhil 0]

I have what I feel is a great system for my personal Mac. But before PC people stop reading, I would really like to find something similar for our PC at the DZ and hope I get suggestions; particularly for automated back-up/cloning software.

My MacPro has two primary drives, a system/apps drive, and an assets drive (we have a similar set-up in our PC). Connected via Firewire 800 is an eternal drive dock for 3-1/2" SATA drives. I have two 2TB back-up drives that I rotate in this dock (on- and off-site back-ups*). Each has two partitions, one partition for each primary drive. I keep one drive plugged in and one in a safe-deposit box, and every couple of weeks I rotate them out. BTW, I had an SATA/USB 2.0 version of this dock before, but the Firewire is not only fast, but allows for hot swaps so I don't have to reboot my computer to swap out my backups. They also now make these with USB 3.0 which should be fast and hot-swappable too.

For software, I use SuperDuper! from Shirt Pocket. Unlike traditional back-up software, this completely clones my hard drives. If I ever get a hard drive crash, I don't have to reload or restore anything. I simply swap out the drives and I'm up and running right back where I was at my last back-up. The software does scheduled back-ups without prompting, and they are "smart" back-ups (i.e., it only updates what has changed since the last time). Currently, I have mine set to back up each drive twice a week late at night (I never shut my computer off), but you could set it to back up every day or more.

I would love to find something like this in the PC world. I'm sure it exists, but I don't know where. But if we have a hard drive crash, I really don't want to reload any software or re-build anything. I just want to be able to swap the drive(s) and go. Any suggestions?

* Any good backup strategy should include off-site back-ups. If your editing trailer/room/house burns down, you don't want it taking your one and only backup too.

[theonlyski 3]

Quote

If I ever get a hard drive crash, I don't have to reload or restore anything. I simply swap out the drives and I'm up and running right back where I was at my last back-up.

I meant to get on it about creating a backup like that, but never did get around to it. I should get on it... maybe I'll pull the tower from the DZ next weekend and give me the week to work on it.

I do have an extra hard drive with everything loaded that's mounted IN the chassis, so it would be a swap between the two drives SATA connections, but those are both in the same case.

I don't know of any software for PC's that creates an actual full backup of the OS while you're using it, but I could be missing something.

ETA: Looks like Microsoft has a 'disk mirror' option in Pro versions of vista and 7... May go try that out this weekend. Going to probably swap some new hardware into the tower and upgrade my OS drive to SSD.

"I may be a dirty pirate hooker...but I'm not about to go stand on the corner." iluvtofly
DPH -7, TDS 578, Muff 5153, SCR 14890
I'm an asshole, and I approve this message

[Zlew 0]

Currently I keep all of the raw footage, and edited footage on a large HD (750GB) on my local machine. I also have a home server that backs up all of that data daily to a raid 10 array with plenty of room (4TB total right now).

As the local disk gets full, I will move all the raw and edited footage to a folder on the server. I think the plan right now is to move everything older than either 6 months or 1 year permanently to the server.

The main piece I am missing right now is offsite backup. I hope to get another external HD and at least backup my pics and videos to it, and keep it at a friends house.

[Laszloimage 0]

For most of my stuff multiple HDs.
For my Tandem videos I use DVDs. Simultaneous recording on two separate recorders. One disc goes to the costumer, while I keep recording on to the other one (each video has its own menu window) as a back until it fills up. It cost 10c to back-up about 20 tandem videos without any extra time and effort.

[avgjoe 0]

Why archive past 6 months? Honestly how often have you had a customer request a new copy after 6 months. We know storage is cheap but is it worth the time and effort. The one thing you should do is have the discipline to have a greatest hits file categorized by exits, opening shots, backgrounds or whatever else cool about it. This will allow for a quick search for you greatest hits instead of having to go through a years worth of footage or stills looking for the one magic moment. Are you archiving for your own use or the customers?

Avgjoe
Hook it for safety

[DSE 3]

Quote

Why archive past 6 months? Honestly how often have you had a customer request a new copy after 6 months. We know storage is cheap but is it worth the time and effort. The one thing you should do is have the discipline to have a greatest hits file categorized by exits, opening shots, backgrounds or whatever else cool about it. This will allow for a quick search for you greatest hits instead of having to go through a years worth of footage or stills looking for the one magic moment. Are you archiving for your own use or the customers?

Avgjoe
Hook it for safety

We do get a lot of requests a year out.
Additionally, archiving is so cheap...it's more expensive, IMO, to selectively archive.
We use off-board SATA drives, backed up weekly (mondays) one drive for stills, one for video, until they're filled. Then the drive is locked, labeled with white gaffer tape labeled with the archival dates.
BTW, with a media manager...we can look for those sorts of metatags when they're inserted (they occasionally are) but you're right...takes a lot of discipline and isn't valuable for tandem-only footage.
We do archive the AFF footage with metatags tho.

But...that doesn't mean there isn't a better way.
Great discussion, keep those ideas flowing.

[CharlieAt 0]

my mac workflow/archiveing:

At the begin of the week the empty card in my video camera gets a volume name in the format: YYYYKWXX (2011KW40 for calendar week 40 in 2011). This way any projects that get worked on on the weekend reference the correct volume name.
At the end of the weekend I create a diskimage (DMG) of the entire card with the same volume id. This dmg is put on my local hd for and mounted for further work (is then recognised as though it is a camera).
It is sync'd to my external fw800 video drive.
At home it gets sync'd to a drobo that only runs while doing backups.
If the jumps were something spectacular then the dmg will be sync'd to amazon s3.
FinalCutExpress at least stores the volume id of where the media came from so reconnecting to this media at a later date a bit easier.

Basically if my flat burns down, not being able to give someone a copy of their tandem jump that they have lost is rather on the end of my list of problems ;-)

Photos are similar with all images getting ingested (aperture).
All images are then sync'd to the drobo.
Images that do not quite make the grade are then moved to an external usb disk so that should they still be needed that they can be accessed, but are not blocking up my primary drive.
Images that make the grade then get sync'd to amazon s3 for off-site storage.

For most of my file work and all of my sync'ing I use Forklift which supports s3 out of the box.

[sdls 0]

Hi,

I have used for a long time Drobo products (not on video archiving specially, but just general file storage).

Few pros about the product:
- Possibility to use normal hard disks (user changeable)
- Possibility to use different sized disks
- Hotswapping of drives
- Fast as h***

For example, I could have Drobo FS with 4x1TB HDs and 1x2TB HD, and when it gets full, I simply change two of 1TB HDs to two 3TB HDs.

If one of the disks fail, I just take that one off and put new (same size or larger) in and here we go again!

You could also easily make rsync synchronization between two Drobos (offsite -backup) (http://www.drobo.com/droboapps/apps-for-drobofs.php, Rsync).

[theonlyski 3]

Quote

I would love to find something like this in the PC world. I'm sure it exists, but I don't know where. But if we have a hard drive crash, I really don't want to reload any software or re-build anything. I just want to be able to swap the drive(s) and go. Any suggestions?

Yup, just tested it, works pretty well. http://www.howtogeek.com/howto/17926/use-drive-mirroring-for-instant-backup-in-windows-7/

You can mirror your drive across several disks and it's pretty easy to swap to the other one. Don't even need to remove it! Just a different boot option.

I'll be doing this on my machine this weekend. One drive to mirror the OS.

"I may be a dirty pirate hooker...but I'm not about to go stand on the corner." iluvtofly
DPH -7, TDS 578, Muff 5153, SCR 14890
I'm an asshole, and I approve this message

[bigorangemd 0]

I archive all to external hard drives and just keep them sitting in the locker. Had one guy who sat on his DVD about a year out and a 90 year old that no one could open the DVD with her stills when she got home. Both people called the DZO. Gave him copies within 10 min. for each. With how cheap storage is these days I'll just keep 'em piling up.