Archiving AVCHD Footage...

[Superfletch 1]

OK... so I have this CX100 now. My plan is to start using it for next season if I can get a smooth workflow going. In preparation for that I like to know some of your thoughts on how I should plan on archiving it.

Right now I'm using a mac but I may end up using Vegas. In any case I'd like to archive it so that I could pull it up on ANY computer system I deemed necessary.

I've been hearing that with the mac you have to keep the file structure in tact and with PC's all you need to do is keep the .mts files or something like that. In going on that I assume basically that for the mac I will need to copy the entire AVCHD folder over to the hard drive. Is this true? I suppose after I "bring it in" to the mac it is then a .mov file and I could just archive that, however I gather that the files are MUCH larger at this point and I'd be better off keeping the original AVCHD stuff. Any thoughts?

Also, what is a good way to store the files so that they are reasonably easy to "find" later. For instance: I could create a folder called "2009-1217-T-JoeBlow" or "2009-1217-FF-2way" Is there a better way?

I'm assuming to archive... after creating the folder I just drop my AVCHD folder in that, or can I change the name of the AVCHD folder withouth messing up the file structure.

Outside of all that... I typically use about 25 DV tapes a year, what sized hard drive would I need to store that much time's worth of footage. I assume that getting a hard drive about the size I'd need each year would probably be the best course, then at the end of each year I could just label that particular drive with a date and know that's all the footage I shot that year. Or would it be more wise to just get a HUGE hard drive and keep dropping footage on their till it filled up.

So many questions... so little time.... :P

Gary "Superfletch" Fletcher
D-26145; USPA Coach, IAD/I, AFF/I
Videographer/Photographer

[PhreeZone 15]

The better option in my opinion is to get a program that allows you to do searches and tagging of the video files so that you can label all the clips in the program and be able to search at any time later for a given keyword. I already owned Lightroom and now there is a plugin for it that lets me import my video files into the program and assign keyword tags to each file so for example I am now going through and tagging my videos to be like: "Tandem", "Joe Jumper", "Front float Exit" "Idiot Instructor" and "Dropzone name" all assigned to a given freefall clip. For another clip I have "Taxing", "Plane type", "take off", "Dropzone Name", "Joe Jumper" assigned to a clip that is the plane taxing out and taking off with the camera going to the jumpers face after take off. Now all I have to do in the future is search for "Joe jumper" and I can pull up every clip he was in and rebuild a video.

It would have saved me a ton of time at Year end time since I could have filtered out all clips with "take off", "Interview", "climbing"," landing", etc and only had to start sorting though the clips that contained the freefall or other shots I wanted. It also lets me dial in and say I only wanted "freeflying" clips of "Super Swooper" as long as I had tagged the videos with the correct tags in the database then I can locate those clips rapidly also.

There are multiple programs out there that do this type of cataloging so I'd look at several to see what fits your needs best.

25 DV tapes but how much blank space is on each? If we assume none so 60 minutes per tape and at about 10 gigs per hour in DV terms you would need a 650 gig (minus the space lost to the formatting of it) drive now to hold the tapes on a drive. AVCHD can be written larger or smaller depending on how you want to do it so figure roughly the same if not a 750 gig external drive per year at that much recording time to be safe.

At 25 hours of footage you can probably find once you move to a computer you can skip recording a lot of stuff like the out the plane shots and stuff and just use canned footage. Once you start doing that then you can get smaller drives if needed but I'd consider backing all your photos up too so you have a full archive if you ever need them.

PC's don't care one way or the other for folder formats, its great since PC's just see the file and you start editing with out any transcoding needed unlike Mac's.

Yesterday is history
And tomorrow is a mystery

Parachutemanuals.com

[PharmerPhil 0]

Just a couple of thoughts:

I would archive the original AVCHD clips. The AIC or Pro-Res files that Apple transcodes to are huge by comparison, and if you are just keeping the files for archive anyway...

As far as hard drives go, think real hard about how important these archives are. If you can't stand to lose them, keep multiple hard drives (and preferably at least one set off site). I have personally had many more hard drive failures than tape failures. This is one argument against one huge hard drive for back-up unless it is backed-up itself somewhere.

One product I really like for data back-up is the NewerTech Voyager hard drive dock from OWC. They are available with USB and SATA either with or without Firewire. The nice thing is when you need to use a new, bigger or different hard drive, you can just buy a raw drive and plug it in without having to purchase a new case and power supply. I keep a dock connected to my computer. I keep a set of raw hard drives off site, and a set at my editing station with regularly scheduled backup (SuperDuper by Shirt Pocket), and rotate them out. Also, the raw hard drives take up very little space, and can even fit in a small safe deposit box.

If you want the files accessible by any computer, will need to format your hard drives such that Windows machines can reads them too. I believe that means FAT32 instead of HFS+ (and no files larger than 4gB), but I could be wrong about that these days. Haven't used a Windows machine in a while (thankfully).

[Superfletch 1]

Quote

One product I really like for data back-up is the NewerTech Voyager hard drive dock from OWC.

Excellent suggestion and one I hadn't even considered! Why pay the added expense of the case and power supply for drives that will just be stored somewhere. I'm definitely going to look into this.

Also, thank you Phree for pointing that out as well. I had thought about something like that but wasn't sure of a program that was made for that task. I'll keep Lightroom in mind and also do some searching and see if there is something out there that fits my needs better. Is anyone out there archiving your footage on a mac and using a catalog program to track it? If so, what are you using?

Anyone archiving digital footage like to share their HD Hierarchy?

For instance: Macintosh HD\2009 Dec\4-6\Tandems\JoeBlow\AVCHD...

Is there a better way so it won't be so damn confusing?

Gary "Superfletch" Fletcher
D-26145; USPA Coach, IAD/I, AFF/I
Videographer/Photographer

[The111 0]

I have a similar dock made by Thermaltake, a brand I know and trust.

http://www.newegg.com/Product/Product.aspx?Item=N82E16817153071

I got mine on sale for about $20. If you have e-SATA on your mobo it will be as fast as an internal SATA drive.

www.WingsuitPhotos.com

[dragon2 0]

I just create a folder every month, have a "tandem" folder in that and make a new folder for each jump with date and passenger name (and include the photos in there as well). If I think a video file is worth setting apart, I copy it into a "best shots" folder.

When a drive is getting full I label it with start and end month.

It's not that much of a system but it doesn't take much time and so far it works for me: I can quickly reproduce tandem videos, team photos and the like, if and when needed
I did try lightroom for a bit but the tagging etc takes up too much time for me, so I'm doing without. Might change my mind in the future, but for now, this works absolutely fine for me.

ciel bleu,
Saskia

[antonija 0]

If you really really really need to archive anything (and I can't stress the "really really really" part enough) you can look into RAID arrays. They have become quite cheap lately, and so have the drives needed to run them.

RAID 1 if you need something stored safely, RAID 5 if you really really need something stored safely.

These things are called NAS (Network Attachable Storage) and come in many shapes and sizes (and price ranges).

When one of your archive HDDs die (provided you really really need data on it) you will be glad you have RAID.

I understand the need for conformity. Without a concise set of rules to follow we would probably all have to resort to common sense. -David Thorne

[bwilling 0]

Quote

If you have e-SATA on your mobo it will be as fast as an internal SATA drive.

If you don't have e-SATA support on the motherboard already, you can add it with a separate card. You'll be a lot happier with e-SATA transfers, USB is just too slow to move much data with, and nothing produces tons of data like AVCHD video!

"If all you ever do is all you ever did, then all you'll ever get is all you ever got."

[Icon134 0]

Quote

If you really really really need to archive anything (and I can't stress the "really really really" part enough) you can look into RAID arrays. They have become quite cheap lately, and so have the drives needed to run them.

RAID 1 if you need something stored safely, RAID 5 if you really really need something stored safely.

These things are called NAS (Network Attachable Storage) and come in many shapes and sizes (and price ranges).

When one of your archive HDDs die (provided you really really need data on it) you will be glad you have RAID.

YEp... with as much data I have racked up in the last year a NAS with a RAID setup is something I'm glad I have... I keep the originals on a drive within the computer... and that data is backed up at least 3 nights a week...

I have a Synology NAS and really like the backup interface/setup... it has a Gigabit LAN connection, USB, and can act as a web server.

Scott

Livin' on the Edge... sleeping with my rigger's wife...

[Superfletch 1]

Quote

If you have e-SATA on your mobo it will be as fast as an internal SATA drive.

I have a Unibody Macbook Pro 17". It doesn't support eSATA however I beleive I can get an eSATA express/34 card that will. What I'm not sure about however is if that would create a bottleneck for eSATA speeds. The expresscard/34 port may step the speed down to USB2. I'd have to check into that. I know my Express/34 CF card reader is at USB2 speeds. When I purchased it I was hoping that it would be much faster and it isn't. I haven't researched it all that much however...

Gary "Superfletch" Fletcher
D-26145; USPA Coach, IAD/I, AFF/I
Videographer/Photographer

[PharmerPhil 0]

Quote

I have a Unibody Macbook Pro 17". It doesn't support eSATA...

You do have Firewire 800 on that which is pretty damn fast.

[DSE 3]

Quote

The better option in my opinion is to get a program that allows you to do searches and tagging of the video files so that you can label all the clips in the program and be able to search at any time later for a given keyword. I already owned Lightroom and now there is a plugin for it that lets me import my video files into the program and assign keyword tags to each file so for example I am now going through and tagging my videos to be like: "Tandem", "Joe Jumper", "Front float Exit" "Idiot Instructor" and "Dropzone name" all assigned to a given freefall clip. For another clip I have "Taxing", "Plane type", "take off", "Dropzone Name", "Joe Jumper" assigned to a clip that is the plane taxing out and taking off with the camera going to the jumpers face after take off. Now all I have to do in the future is search for "Joe jumper" and I can pull up every clip he was in and rebuild a video.

FWIW, the Media Manager in Sony Vegas allows exactly this. Metatags can be inserted into a video clip, with up to 1000 fields per clip. It's cross referenced, indexed by a wide variety of objects. Metatags can include thumbnails, searchable by multiple keywords, etc.
Unlike other apps, it does not make copies of clips, so you only use addresses to existing pieces of data.

[Superfletch 1]

Quote

You do have Firewire 800 on that which is pretty damn fast.

Yes I have several FW800 devices I use. It is pretty quick.

Gary "Superfletch" Fletcher
D-26145; USPA Coach, IAD/I, AFF/I
Videographer/Photographer

[PhreeZone 15]

I believe Adobe Bridge does this too but I'm yet to venture into that program

Yesterday is history
And tomorrow is a mystery

Parachutemanuals.com

[pilotdave 0]

Thanks for the tip on media manager. Took forever to get it working (windows 7/64-bit is a major pain in the ass!), but it looks like it'll be helpful once I start filling my computer with CX100 files.

Dave

[DSE 3]

Quote

Thanks for the tip on media manager. Took forever to get it working (windows 7/64-bit is a major pain in the ass!), but it looks like it'll be helpful once I start filling my computer with CX100 files.

Dave

Nicely understated.
FWIW, there is a training DVD on using Media Manager, hosted by one of IBM's database engineers.

[pilotdave 0]

Oh trust me, I was searching for answers to my problems* and came across at least 2 or 3 forum posts about the training DVD... and only one by you!

* (old (32-bit) version showed up blank in vegas, downloaded 64-bit version wouldn't install, after finally installing the media library failed to open and couldn't create a new one, etc.)

If anybody else has trouble, the 64-bit version can be downloaded from sonycreativesoftware.com under updates, vegas 8.1. I had to restart to get it to stop giving me an error message. Then the media library refused to load which seemed to be fixed when I unchecked "compress contents to save disk space" on the media libraries folder advanced properties. I know from searching that I'm not the only person to have this problem, but I never came across a solution until I figured it out myself. At least I think that's what fixed it.

Dave

[parachutist 1]

At skydive utah we set up an extermal hard drive that's connected via USB to the main editing system, That drive is shared on the network. At end of each day we move all raw footage frpm all computers to that drive for archiving. Folder structure is pretty simple and it works well. Example:

\2009\September\14\Mike_Smith\