The Big Edit - Software Conundrum - PPro 2018???

[User]

Hi folks, I've been extremely isolated in my filmmaking process, so I very much appreciate any guidance on the road ahead... I'll try to keep this short and to the point.

- My process has been to continually return to a city over the last 10 years, find new characters, and then add their files to the main project.

- The project began on Premiere Pro CS6 in 2013 and was then imported into CC 2014, its been there ever since. I'm on a 2013 Macbook Pro (16GB RAM SSD) running Mountain Lion. The media is housed in a TB 5 bay JBOD (4tb 7200 spinning discs that are all within 20-100gb of being full). The Media Cache/ Media Cache Database and Preview files are located on one of these discs containing media, some say that this is not exactly ideal but it works.

- The project file now has around 18,000 files (LongGop, H.264, ProRes etc.) associated to it and there are a great deal of sequences that I've been building up over the years. When I open the project, it takes about 20 minutes to open the file, load the media and index the files. When it auto saves in can take 2 minutes to save the 215mb file. Ugh.

- In going forward, for the first time ever, I open a new project and imported one sequence from the big main project into the new project. Needless to say, it opens and loads the media very quickly. Problem is that if I import another sequence into this new project it start duplicates some of the media and this starts to get confusing.

- Recently I've heard that Premiere Pro 2018 has the ability to open multiple projects at once. So with this, I've been imagining breaking the main big project in (6-7) smaller projects that just include one character as a way to continue. And that PPro 2018 can have multiple projects open at the same time, it seems like this could be a decent way forward but it could also be limiting and confusing in other ways. If possible it would be ideal to keep everything in one project and work from there as I have been if it were not so slow to load and save.

Yes Avid seems to handle large projects better, to a  point. But can anyone here comment on an approach that they are using that is working well in editing a massive long form project on Premiere?

 

[Axel]

The following comes from a guy using FCP X, but don't worry, I won't try to baptize you ?

First thing you have to do, of course, is to manage your media. JBOD is very risky. I have a raid5 TB, and recently one of four disks died. Easy enough to replace this, but what if the whole controller died? Go with JBOD, but provide redundancy, in short: a complete backup, elsewhere. Never ever run out of disk space!

3 hours ago, User said:

The project file now has around 18,000 files (LongGop, H.264, ProRes etc.) associated to it and there are a great deal of sequences that I've been building up over the years.

Sounds wrong, is wrong. You should better split the project. Has nothing to do with performance or launch time. There is a threshold beyond which you simply invite disaster. A project of this size can't be managed. And with it going over several versions of the NLE ...

3 hours ago, User said:

Recently I've heard that Premiere Pro 2018 has the ability to open multiple projects at once. So with this, I've been imagining breaking the main big project in (6-7) smaller projects that just include one character as a way to continue.

Good idea. However, Premiere may be less dependent on a specific OS, but I doubt very much that Premiere 2018 will run on prehistoric Mountain Lion. If it can be installed in the first place, it will probably creep.

That said, there almost certainly is a way to split your project with your old OSX and PP version. Like, saving as, deleting all edits not used for the part you want to create and consolidating it with copying to an external disk. Or so.

There may also be third party software to manage the project. 

3 hours ago, User said:

But can anyone here comment on an approach that they are using that is working well in editing a massive long form project on Premiere?

It's all about instant and continuous access to databases. Have your media in compact bundles, separated from caches, render files and project files. Limit the number of files Premiere has to access all at once to a few hundred. Remember, Premiere only has to open the folder(s) that contain(s) material for the current sequence. Close the timelines not in use. Split episodes to scenes (=sequences, rarely longer than 10 minutes). But even if for a short timeline Premiere has to open 8 folders containing 4000 clips, you are asking for trouble.

[Andrew Reid]

Have you tried Resolve 15 Public Beta? It is slowly becoming more and more like Premiere in terms of the editing interface and features, but without the performance issues.

I am highly impressed.

3 hours ago, User said:

- The project file now has around 18,000 files (LongGop, H.264, ProRes etc.) associated to it and there are a great deal of sequences that I've been building up over the years. When I open the project, it takes about 20 minutes to open the file, load the media and index the files. When it auto saves in can take 2 minutes to save the 215mb file. Ugh.

The purpose of editing is to trim the fat.

Why not start doing some editing and remove those files from the project that you no longer need.

[kye]

To add to Andrews comment about Resolve, if you're considering it there are a few articles about editing with Resolve that might be useful.  They're from editors, not all-in-one people who would colour grade as well, but they're still potentially useful.  My take on them was that Resolve is promising, if you configure it with the hotkeys the same as your current NLE then it's straight-forward to learn, but everyone says it's interesting but they're not going to use it solely for editing yet.  They don't normally give any reason, I suspect they're just being careful and not wanting to stand out as being too 'radical' as it may be too shocking for people.

I use Resolve as an all-in-one but I work on multiple projects that are tiny so my experiences with it really won't translate to your situation.

[User]

Axel.
Danke. Yes I have backups of the media on other drives. Yes I want to split up the project, that's why I searching for solutions. Of course I'll leave Mountain Lion when a decent way forward presents itself. Agreed that it's better to keep the the Media Cache/ Preview files on a separate ssd drive. Yes the smaller the project the better, but tell this to a feature documentary editor and you will come to understand that many doc editors are all facing the same issue. There is no easy solution. Premiere Pro 2018 ability to have multiple projects open might be the solution if it does not duplicate media when moving sequences between multiple project... this is the thing that is currently holding me back for upgrading OSX and Premiere. Adobe seem to be aware of this 'duplicating media' problem, not sure if there is a fix, many like me are watching/ waiting.

Andrew.
Danke. No Resolve on this side, the current project is too far along to switch over. But I will keep it in mind on the next one. Yes, editing is about editing. I have started culling files (and sequences) that might not make the big cut.

Kai, appreciate the insights. Yes Resolve sounds promising, though I think we know that it's not the tool for the long form doc... but I'd be happy if it was.

[User]

So nobody here is using PPro 2018... moving sequences between different projects? No editors here? Tisk tisk.

[hmcindie]

Yeah, I move sequences between projects on occasion. But I haven't done that with a massive project. Mostly just commercial work, ads etc which are considerably less massive than what you've get there. 

And even if I had, I wouldn't presume to know your project and suggest something that might end up catastrophically. Just finish it with what you've got if you can.

Or what I would do is -> backups of everything -> download trial of cc 2018 -> try it out.

[User]

15 hours ago, hmcindie said:

Or what I would do is -> backups of everything -> download trial of cc 2018 -> try it out.

Thanks hmcindie... I figured there must be someone here who could weigh in in this.

It seems as though there may be little other way forward... so I'll most probably give it a go.
Tell me, is PPro 2018 duplicating media as you move it from project to project?

Is PPro 2018 working well for you?

[hmcindie]

8 hours ago, User said:

It seems as though there may be little other way forward... so I'll most probably give it a go.
Tell me, is PPro 2018 duplicating media as you move it from project to project?

Is PPro 2018 working well for you?

When you import a sequence there is a button that asks what happens to duplicates (if I remember correctly). 

2018 is working okayish, im having some issues with the audio solo bug (you need to click that sucker several times) which has made me output masters with wrong audio files. A pretty serious bug actually.

[User]

On 6/11/2018 at 7:22 AM, hmcindie said:

When you import a sequence there is a button that asks what happens to duplicates (if I remember correctly). 

2018 is working okayish, im having some issues with the audio solo bug (you need to click that sucker several times) which has made me output masters with wrong audio files. A pretty serious bug actually.

Thanks again hmcindie.
Somehow there is always something with Premiere... which is why I just sort of hung on with CC2014.

[User]

A small update, if anyone cares.
So, I've come into this century and am on OSX Mojave and PPro 2019. I've separated the big main project into smaller projects. The craziest thing is that I picked up a ssd and dedicated it to Adobe's cache files and this has single handedly made a HUGE DIFFERENCE in how fast projects load! For example, the big main project used to take 25min to open and load, now it takes 1 min. Incredible! The downside is that I've experienced a huge headache with Adobe and Apple dropping support for some types .mov files so I've been batch renaming these files to .mp4 and relinking like a motherlicker... problem is that some of these files eventually lose their link. The work around is to put these same files into a separate folder inside their existing folder and relink them. There seems to be no rhyme of reason as to which files will stay linked and which will drop off. Yes I should have probably just converted them to ProRes already.
As far as how well things are going on PPro 2019, it seems to be a good thing to be able to have multiple project open but is going to take discipline to keep organized within this and not get confused as to which project and timeline I'm working in. I have to say that the new Lumetri Color is a big step up as is the quick Audio clean up feature. Yes I've traded one set of bugs for another as there has been some crashing, I'm trying to understand what contributes to this. Let's see if this move actually helps me move the project forward in a more 'enlightened' way... it's a bit crazy how beholden we are to the technology that allows us to tell stories in this way.

[webrunner5]

Yeah all these platforms are just one hack away from being a disaster. Crazy world we live in now.

[User]

Hack this.

[webrunner5]

2 hours ago, User said:

Hack this.

Well I seen that a few days ago. I was surprised some dumbass hasn't spayed Graffiti all over it by now. Then that would be a hack.