Effective Storage Handling with Final Cut Pro X

[Wit]

I'm testing the most optimum storage handling method for extended shoots. Being on the road with minimum back up means. Here's what I found so far - and what questions are left. Would help if you gave your thoughts on this ...

 

 

Set Up:

 

- Macbook pro SSD 1TB

- 2 x LaCie rugged mini 1 TB

 

Storage method:

 

- Edit on Macbook

- Store project file on Macbook drive under Films - Final Cut Projects 

- Shoot more compressed AVCHD files

- Store full copy of SD card in xtra folder in the dedicated Event folder in Final Cut Events

- Run one Time machine back up copy on LaCie 1 disk

- Make copies of the Time machine back up on LaCie 2 disk 

 

 

My idea is to store and edit until my notebook will be full. As soon as my notebook is full, I will empty the Original Media folder to safe space. I will restore the data from the back up disk to the Original Media folder when needed as soon as I'm in the situation to do. Tested this method and it works. Of course an accurate administration system will help. 

 

The other method will be to import the data again from the copy of the SD card. I didn't test this method yet but I figure it will be time consuming as I somehow think the reconnecting has to be done file by file.

 

 

My questions are:

 

1. Can I run an automatic back up via Time Machine on back up disk 2 too or will this confuse the system? 

    Therefore better refrain to making manual copies whenever there has been changes.

2. I guess the data on the backup disk will be compressed. Won't this harm the quality of my footage? 

3. What reconnecting methods are best? Via Time machine or reconnecting via SD card?

4. General thoughts on my storage handling as described above are welcome!

 

 

Thanxs!

[Axel]

With FCP X, there is almost no 'wrong' way. You seem to be quite conscious about what you do (because one can lose data!), so I just share a few thoughts:

• How much redundancy do you need? How many instances of backups are sane, and when exactly begins paranoia?

• Do you want the render files to be included in the Timemachine backup? 

 

A Timemachine backup implies that your system's harddrive, your SSD, may fail. Whereas this is possible, a whole different scenario is more likely to happen: Bug infection. Not death, but a condition where you beg for death. You have, for example, some version of FCP X. You are in the midst of editing some projects (which were sequences in older NLEs), and you install a plugin. All of a sudden, FCP X displays the beachball, renders random green or purple frames. You didn't make a Timemachine backup, because you felt secure with an expensive Raid :( . You trash preferences (old first aid remedy), no success, you uninstall the plugin, no success. Hell, the way to restore your project is full of pain. You can better go back to start.

 

Timemachine is your rescue not only with third-party plugins (also, beware, Quicktime-plugins, like Perian), I also don't trust a new version of FCP X. Keep your old versions as backups!

 

I make a Timemachine backup at the end of the day. 

Additionally, I have a freshly installed system-backup on another drive.

And the camera archives on another (the media are on three drives that way!)

 

Use the import-tools of FCPX to filter your usable clips, it helps a lot to save space!

[Wit]

I'm not paranoia - it's what other traveling filmers/photographers advised me - 2 back ups. 

Divided among two travel bags of two separate persons. They also recommend sending material home but I'm not sure about that. I just wanna understand what I'm doing so I don't loose material because of ignorance.

 

I don't know if I want the rendered files on my back up - do I are they taking up that much space? If not how can I excluded them?

Yeah for sure I won't update the system while using this method.

 

 

I make a Timemachine backup at the end of the day. 

Additionally, I have a freshly installed system-backup on another drive.

And the camera archives on another (the media are on three drives that way!)

 

What do you mean with the second phrase? Unplug the first harddrive and plug in the second to make again a Time Machine back up. Wo'nt this mess your back up system?  

[Axel]

I somehow skipped the part where you wrote that you were travelling. Then of course your care is appropriate.

 

I don't know if I want the rendered files on my back up - do I are they taking up that much space? If not how can I excluded them?

 

With very old versions of the classic FC (and all older NLEs), when harddisc space was expensive and real time performance didn't deserve the name, you often judged an effect by how it looked on the current frame only (which was always rendered in full quality). You had to see things like an animation, and then you had to render, but you triggered it manually by selecting the clip(s) in the sequence and hitting ⌘+R. Render files can consume more space than footage, and none of them are actually used for final export, they are for preview purposes only. You can put off background rendering in FCP X's preferences (here), then you can render a selected clip by hitting CTRL+R. 

 

Assumed your project consists of a hundred animations with a dozen more filters on each. Then again, you wouldn't want to keep the render files. It would be wiser to export an entire self-contained movie as a reference and delete all render files. This will delete not only all rendered clips from the current state of your projects, but also those of all former states, because, as you know, FCP X works as it's own Timemachine, and since Lion, every tiny change on every file gets auto-saved. Imagine all the automatically rendered clips!

 

Render files not only eat disc space (which is cheap nowadays), they eat RAM. This is because every file that's used to play back a sequence needs to be kept in short time memory together with all files it refers to. With this fact in mind, think of the size of your events and projects. Even on a 32 GB RAM machine, ONE giant event pool with thousands of clips will eventually slow down the system, if you use clips from this giant event in multiple projects. The same, if you had a hundred events, each only with 20 - 50 clips, but one giant project of two hours, using clips from all events and additionally cross-referential compound clips! Don't get me wrong: You can do this, and you should do it, but not all the time and not with, say, 4 GB RAM. Use Event Manager X ($4,99), until a future version of FCP X allows to uncheck unused events and projects (the main advantages of the program are, it makes you aware of your media organisation and it cleans up your GUI!). It also 'ejects' unused volumes ...

 

Then there is GPU acceleration. If your graphic card allows it, you can see all effects in real time without rendering. You can - and should - always choose >preferences >playback >better performance over >high quality, when there is a problem with playing back unrendered clips. If you pause, the current frame will always be 'high quality'. 

 

I make a Timemachine backup at the end of the day. 

Additionally, I have a freshly installed system-backup on another drive.

And the camera archives on another (the media are on three drives that way!)

 

What do you mean with the second phrase? Unplug the first harddrive and plug in the second to make again a Time Machine back up. Wo'nt this mess your back up system?  

 

First line: Backup for the work.

Second line: Backup to reset the system, to format the HD.

Third line: If, for some reason I have to relink to original media. I wipe and re-use the SD-cards.

[Wit]

Thanxs! Deleting rendered files sounds as a wise thing to do.

 

For other readers I'll include a video how to. But What do I do with the rendered files from projects which aren't anymore on my FCP X page? How do I delete them - which map or sub files should I remove?

 

 

 

 

 

 

It sounds like you're not advising me to store everything on the SSD of the notebook are you?

If you want - I can read German too.

[Axel]

But What do I do with the rendered files from projects which aren't anymore on my FCP X page? How do I delete them - which map or sub files should I remove?

 

Why are the projects no longer on the surface? Did you move them? Moving (='hiding', as FCP X calls it in the manual) is a method to get them out of the way without deleting them. Move them back, restart FCP X, they will appear. Select them, delete files. I suppose you could always use the finder to get to the render files and put the whole folders into the trash. But I never tried this myself.

 

It sounds like you're not advising me to store everything on the SSD of the notebook are you?

 

Rather that than reading/writing anything over a USB connection. I suppose you didn't experience big problems with having the OS and program on the same volume as the clips?

 

Note, that with AVCHD you always work with a copy, even if you checked 'original media'. The copying process is so fast, you don't realize it, but in the 'original media' folder live H.264 copies of your '.mts' - clips (EDIT: In case you chose 'optimized media' - the default for AVCHD -, the files will be big, but edit-friendly ProRes-files, check this).

 

There are two advantages in that:

 

1. You can connect your card reader and a LaCie drive to the MacBook, open FCP X, hit 'import' and there 'create archive'. See here for details (This was obviously before the unified import of 10.0.6, the rest stays valid). In the opening dialogue, you choose the LaCie as volume where your SD-card should be archived. Now you wait, until the backup is complete. Then eject the card reader.

 

2. After that, you create a new event with your SDD as volume, hit import and choose the camera archive on the LaCie.

 

3. Here comes a very interesting feature only available with original SD-cards or camera archives:

(here is your German  ;) )

 

The blue icon selected here is the 'clip view' of the import window. It allows for you to skim through all your clips at once, qualifying them very quickly (granted, were the archive not connected only via USB, you could skim waay faster!). Moreover, you can select multiple regions of one clip before you hit import.

 

4. That way, you only copy the portions of the original clips you actually plan to use in your project, saving space on your system volume!

[Wit]

Why are the projects no longer on the surface? Did you move them? Moving (='hiding', as FCP X calls it in the manual) is a method to get them out of the way without deleting them. Move them back, restart FCP X, they will appear. Select them, delete files. I suppose you could always use the finder to get to the render files and put the whole folders into the trash. But I never tried this myself.

 

Because I threw them away. And I didn't make a back up as it where all trials. But I figure the rendering files are still existing?

Can I remove the all the folders from the map Films/Final Cut Events/Camera render files and

 the  from the Peaks data from the Render Files map with no harm?  Nothing important yet on this machine.

 

 

 

 

h. But I never tried this myself.

 

Rather that than reading/writing anything over a USB connection. I suppose you didn't experience big problems with having the OS and program on the same volume as the clips?

 

Note, that with AVCHD you always work with a copy, even if you checked 'original media'. The copying process is so fast, you don't realize it, but in the 'original media' folder live H.264 copies of your '.mts' - clips (EDIT: In case you chose 'optimized media' - the default for AVCHD -, the files will be big, but edit-friendly ProRes-files, check this).

 

There are two advantages in that:

 

1. You can connect your card reader and a LaCie drive to the MacBook, open FCP X, hit 'import' and there 'create archive'. See here for details (This was obviously before the unified import of 10.0.6, the rest stays valid). In the opening dialogue, you choose the LaCie as volume where your SD-card should be archived. Now you wait, until the backup is complete. Then eject the card reader.

 

2. After that, you create a new event with your SDD as volume, hit import and choose the camera archive on the LaCie.

 

3. Here comes a very interesting feature only available with original SD-cards or camera archives:

(here is your German  ;) )

 

The blue icon selected here is the 'clip view' of the import window. It allows for you to skim through all your clips at once, qualifying them very quickly (granted, were the archive not connected only via USB, you could skim waay faster!). Moreover, you can select multiple regions of one clip before you hit import.

 

4. That way, you only copy the portions of the original clips you actually plan to use in your project, saving space on your system volume!

 

 

MMM it's too late now to dig into this but hey thanks I'll give it a go tomorrow. Actually I didn't do much editing on the macbook pro yet. So I don't know if I will encounter speed problems yet.

[tombaja]

Hello Blanche.

 

Are you sure about your Macbook Pro with a 1 TB SSD?  I ask because until now Apple sells Macbooks  with only 256/512 or 768GB SSD.

 

I see in your scenario some problems:

In order that Time Machine can keep backups over several weeks/month your Time machine disk should have the capacity of 2x, at least 1.5x of your Macbook hard disk.

When your notebook is full, your Time Machine disk will also be almost full and Time Machine will begin to delete the first backups you made, that means your are going to lose data.

 

Because you are very concerned about the security of the data on your LaCie disks, you are planning to have two of them for backup. But what are you going to do when your Macbook drive fails? Are you prepared?

 

OK, what would I do in your situation?

I would store all my new events and projects on my Macbook and everyday I would run Time Machine (on the first LaCie) in order to backup those data.

All finished projects (including events) I would move to an external drive (second LaCie) in order to make space. At the end of the day I would backup this external drive to another external drive (third LaCie). And I would buy another 512GB SSD for the case that my Macbook SSD fails.

 

Let’s check this scenario:

When your Macbook SSD fails, you have a copy on the Time Machine. You can install the SSD in reserve and re-install everything from the Time Machine copy.

When your Time Machine drive fails, you have still the original data on the Macbook.

When one of your external drives fails, you still have a copy on the other external drive.

 

It seems, in this scenario your data are pretty safe. What do you need? 3 x LaCie and a reserve SSD for your Macbook.

 

BTW, Time Machine does not compress the data.

 

Greetings.

[Wit]

Thankx! I was so wrong only have 250,14 GB on SSD  :o - really thought it was more. MMM this asks for another workflow maybe.

Yes if my macbook got robbed or is broken I have a problem but I can't fix all uncertancies in live.

 

Thank you so much for your answers - will dig into this tonight!

[justin0909]

Hello friend. I just got a industrial grade SSD card from RENICE. The performance is amazing. And they're now presenting some free CF, SD cards and SSD to Twitter followers http://goo.gl/o9we1A

[Quirky]

4. General thoughts on my storage handling as described above are welcome!

 

Well, one thought that came to my mind was about this:

 

 

- Edit on Macbook

- Store project file on Macbook drive under Films - Final Cut Projects 

- Shoot more compressed AVCHD files

- Store full copy of SD card in xtra folder in the dedicated Event folder in Final Cut Events

- Run one Time machine back up copy on LaCie 1 disk

- Make copies of the Time machine back up on LaCie 2 disk 

 

 

Maybe it's just me, but I would never edit on the MacHD. I'd let FCPX make the default folder(s) it wants to do in the main HD of the Mac, but never use them for anything.

I tell FCPX to make a new Projects and Events folders on a separate, relatively big and relatively fast external hard drive, and then import (copy) the sources and edit from there. I always keep the original source materials elsewhere on yet another hard drive, or import them into the work Projects folder directly from the SD(D) drives. When I'm done editing and finish the project, I consolidate the library and then get rid of the unnecessary files to save space.

 

There are also some FCPX library managers available. I've got a free/cheapo app called Final Cut Library Manager. It might be handy at some point.

 

I've never bothered with Time Machine when it comes to the render files and finished projects. It feels like an unnecessary step, as FCPX is making copies of your imported files anyway, if you let it do so, and when the render files are missing, FCPX will make new ones on the fly. So far I've done my backups manually onto a separate backup disk.

 

Just my general thoughts. Effective or not, that's how things are in the Quirky HQ at the moment. I might come up with a new workflow later next year, if I manage to gobble up  enough dough to upgrade my editing hardware.