Organizing Projects

[HockeyFan12]

Broad question, but how do people organize their projects? Paid and personal? By client? By date? 

Anyone have any advice here or links to good conventions? Any post sups out there? 

[KnightsFan]

Organization is my specialty! I am actually currently building a couple of tools to help organize projects, so maybe we can bounce ideas around and I may incorporate them.

 

There are several levels of organization I use.

At the top tier, each project is in its own folder. Generally, the folders are all in the same place, on different drives as they fill up.

I keep personal projects and work projects separate at the top level. I generally organize paid work by client, and then within that, a folder for each project that is sorted by date. I keep a spreadsheet with details about each job, which I can refer to if I forget where anything is.

When organizing by something, I'll have sortable numbers at the beginning and then a descriptive title. For example, "004 Smoke simulation", or use yymmdd date format.

Depending on the size of the project, I might jump straight into a media folder within that project. The last couple things I've shot have been episode based, so I have a folder per episode, and then within that, a folder for videos, one for photos, one for audio, one for VFX. One my latest project, each episode is longer (50 min), so I have it broken down by scene after episode. So Ep1\Scene006\VFX\[some vfx shot]\[composite files, textures]... etc

I created a mirrored folder structure for proxies. That way, I can literally do a find/replace on an XML to replace proxies, or use the Reconform from Bins option in Resolve to switch between proxies and online media, assuming my Resolve bins mirror my folder structure (which it does, since I simply import the project folder all at once).

I generally keep storyboards, scripts, and other pre-production notes in a separate folder.

For actual files, I have filenames with:

- scene, angle, and take information

- The date it was shot (that way, we don't have to remember which angle we ended on if a scene is spread over multiple days)

- project name (not strictly necessary, but I do it anyway!)

File scene, angle, and take are named automatically based on timecode using the tools I mentioned earlier. Audio and video files that go together are named the same thing, so they are easy to find.

When finished, I ensure that all renders have the date in their filename, and go in a Renders folder.

In general, I try to keep no more than 10 folders in any given folder, and no more than 100 files in a given folder--except for image sequences, which go in their own folder. It's much easier to find things when you keep the number of items per folder to a small, manageable number. I keep a lot of .txt files describing what is in each folder as well.

 

I'm kind of rambling and being confusing here, maybe I'll write up a more cohesive post about it after my current project is done.

[HockeyFan12]

Thanks for the detailed advice.

Do you also organize by year? I’m trying to go through all my stuff, going way back, trying to organize even past work, so that might be a good idea for me…

And then one folder for personal work and one for paid after year?

And then projects in each?

Then: preproduction (scripts, storyboards, reference), production (footage), post production (sound, vfx, color, etc.)... and so on... 

Do you organize by client or just include that in the name of the folder?

What do you do for stock footage, plug ins, etc.? A separate directory entirely for that stuff? I'll sometimes clean install everything and it's nice to have that ready to go.

Do you store other stuff like photos, music, etc. (I take photos and am starting to work more in Ableton) in the same drives as your film work? All my paid work is video atm... but my personal work is a hot mess.

What do you do for version numbers and/or date for vfx or for design? For vfx do you organize project files by date, by shot, etc. Or just one project file per project? How do you version up?

One issue is assets and workflows can be shared across vfx shots, or one client might refer to assets from a previous project for editorial, but do I copy those assets into both folders? What about stock footage and stock sound assets and fonts? What naming conventions do you use for vfx?

What does one of the spreadsheets look like that you mentioned? Do you track hours, days worked, etc. and whether it’s been invoiced/paid/etc.? Where do you store invoices? Locally or Google Docs? I assume the project name in the spreadsheet correlates with the parent directory for the project?

What do you do for dropbbox integration? I’m often rendering stuff out to deliver to clients and rendering straight to dropbox. But do I also save those files locally, too? The issue is then I’m required to have two copies of them on my SSD since my dropbox syncs to my SSD. I guess that’s not a big deal, I can selective sync at the end of the day.

Hmm...

I have this terrible habit where I work off my SSD (only 256GB!) then back everything up to a larger drive piecemeal. It’s… terrible. But essentially to me. So that makes this a bit harder. Or maybe easier in the future, but harder now, as projects span drives.

For footage I just use the camera original file name. Do you change the name of footage or just project files, etc? I also don’t use proxy workflows often. I’ve been tempted, but usually just edit at a fractional resolution off camera original.

Overall that seems like really good advice. I’m never going to be as organized as it sounds like you are, but that makes a lot of sense.

The other issue is I have tons of unclassifiable junk: notes in notepad (scripts, to-do lists); screen grabs of stuff, cat videos, cat photos, random test shoots that don't even quality as personal work really (maybe they should, but that takes the fun out of a random test shoot).... knowing what to do with that is another challenge.

The other question is do I just throw some stuff I shoot out? For test shoots and stuff, do you keep everything you shoot or delete some of it? Maybe I should devote an entirely separate drive to assorted photos and videos that I just shoot for fun with no plans to editor or publish.

[kye]

Probably less useful to others, but I organise very differently to most.

Essentially I am documenting the history of my family, I don't really have 'projects' or 'clients'.  I will do separate videos from a trip or a day out, but I may also do a 'year in review' project, or a '21st birthday mashup of embarrassing moments' type thing.

So I organise by date, with folders in a YYYY/YYYY-MM-DD XXXXXXXX format, where XXXXX is the description of what happened that day (if it was significant).  My challenge is that in this format a project sits across multiple folders, but it works ok.

I don't think any organisation system is perfect, you're always trading off one aspect against another.

[KnightsFan]

5 hours ago, HockeyFan12 said:

Do you also organize by year? I’m trying to go through all my stuff, going way back, trying to organize even past work, so that might be a good idea for me…

No, not really. I probably have fewer projects than you, though. Right now, my projects are generally grouped by "era" on different hard drives--school projects, work during college, 2 large personal projects since then, and paid work since college.

5 hours ago, HockeyFan12 said:

And then one folder for personal work and one for paid after year? 

And then projects in each?

Then: preproduction (scripts, storyboards, reference), production (footage), post production (sound, vfx, color, etc.)... and so on... 

Pretty much, yeah. I keep personal and paid separate, and have an individual project folder and then individual elements, which vary based on the project. Like if it's a small VFX job I don't need a pre-production folder, etc.

5 hours ago, HockeyFan12 said:

What do you do for stock footage, plug ins, etc.? A separate directory entirely for that stuff? I'll sometimes clean install everything and it's nice to have that ready to go.

I don't have many plugins, but I do keep the install files in an installers folder. I do have a large library of assets: 3D models, sounds, and textures, which I keep in an "Assets" folder. They are usually very small, so I make copies if they are used in a project.

6 hours ago, HockeyFan12 said:

Do you store other stuff like photos, music, etc. (I take photos and am starting to work more in Ableton) in the same drives as your film work? All my paid work is video atm... but my personal work is a hot mess.

I have projects all stored on the same drives if they are "in use." When I archive stuff I group things on a drive which I fill as full as possible, and then never look at again haha.

6 hours ago, HockeyFan12 said:

What do you do for version numbers and/or date for vfx or for design? For vfx do you organize project files by date, by shot, etc. Or just one project file per project? How do you version up?

I just use the date. Version up just means another date. I've never needed two concurrent versions from the say day before. Most of my VFX shots are small enough that all the project files go in one folder, titled by the shot. Like I may have a Blender file and a Fusion composition and a few intermediate files all in the same folder. Then there's often a folder for textures, and a folder for renders. If I rendered to an image sequence, that is for sure in its own folder.

6 hours ago, HockeyFan12 said:

One issue is assets and workflows can be shared across vfx shots, or one client might refer to assets from a previous project for editorial, but do I copy those assets into both folders? What about stock footage and stock sound assets and fonts? What naming conventions do you use for vfx?

Client work all goes in the same folder, so I'm fine referencing textures or assets within a client's folder without making a copy. My volume is pretty low, though, so I imagine you might need a different structure if you have a decent amount of work. It's pretty rare for me to use stock anything. For VFX I really just name things descriptively. Often the project files are just "project.ae" because it's in a descriptively named folder.

6 hours ago, HockeyFan12 said:

What does one of the spreadsheets look like that you mentioned? Do you track hours, days worked, etc. and whether it’s been invoiced/paid/etc.? Where do you store invoices? Locally or Google Docs? I assume the project name in the spreadsheet correlates with the parent directory for the project?

Yeah, I keep track of hours and the amount paid. I keep it locally only. Yeah, the names correlate. I'm not a high level professional: most of my work is personal. Paid work is usually for people I have a good relationship with and work with fairly informally, so I don't have formal "invoices." But I do keep track of what I have been paid for each project, when, and how.

6 hours ago, HockeyFan12 said:

What do you do for dropbbox integration? I’m often rendering stuff out to deliver to clients and rendering straight to dropbox. But do I also save those files locally, too?

Never done that before. I use google drive to send files usually, and always have a local copy. My internet kinda sucks so I could never render straight to the internet--it would be too unreliable.

6 hours ago, HockeyFan12 said:

For footage I just use the camera original file name. Do you change the name of footage or just project files, etc?

I change the names on big projects. That's the function of my organization tools, automagically rename all the files from a card, copy them into the correct directory, and generate proxies. It makes it easier when editing long term. But it does depend on the project. I just got back from a music video shoot where I won't be renaming files--there's only ~100 clips, and no audio. It will be easy to keep straight. Another project I'm in post on has been shooting for almost two years and has thousands of files, so organization is much more important. I don't know how I'd manage if the files weren't all renamed to differentiate between different scenes and angles, some of which could have been shot months apart, or even years later in the case of some of the Foley files.

6 hours ago, HockeyFan12 said:

The other question is do I just throw some stuff I shoot out? For test shoots and stuff, do you keep everything you shoot or delete some of it?

I delete tests after a while, usually about five minutes after looking at them.

[kye]

7 hours ago, HockeyFan12 said:

The other question is do I just throw some stuff I shoot out? For test shoots and stuff, do you keep everything you shoot or delete some of it? Maybe I should devote an entirely separate drive to assorted photos and videos that I just shoot for fun with no plans to editor or publish.

There are 3 approaches that you might consider:

• Keep everything - storage is cheap
• If you're never going to look at it again then delete it, or keep it for a while and if after that time it's not still relevant then delete
• Upload it to YT in the highest quality you can, set it to be a private video, then delete it locally, and if you ever need it again it's "backed up"

I kind of do the last one for some things.  For example, with my various tests and experiments one of the reasons I document them online is that it means my observations and conclusions are essentially backed up for me, so if I need to refer back (which I do sometimes) then I know where to go to find them.

[HockeyFan12]

@kye, no that's totally applicable to me. "CAT VIDEO MM/DD/YYYY" describes a lot of my "personal work." I think that might be the better naming convention for a lot of my stuff.

Uploading to YouTube like that is a smart idea. The only caveats I can imagine are that you can never truly delete anything that way, or there's that story about Myspace losing 12 years of music. You can't really choose if it gets deleted, I guess.

Still, I do that with a lot of stuff on dropbox, always keeping essential files there, but of course have to pay monthly to keep that data online.

@KnightsFan, tons of great advice, thank you.

Are your tools OS X or Windows? One thing I didn't realize would be helpful but think would be, is including relevant information (client, project name, etc.) in the "comments" section of all the files correlated with a certain project. That way, it's searchable even if it's not in the file name, I believe. Batch modifying comments might be interesting. I'd rather not change file names in some instances, but I wouldn't mind adding comments.

The flip side of me (perhaps) having more projects to organize than you is that most of them are a couple weeks of work from start to finish at most. So I can't relate to most of those concerns about years of footage existing together in one scene; even for longer-term projects, it's a few days of shooting and then pick ups. I did something long-term like that many years ago and it was awesome, though. Despite some continuity errors and changing hairstyles...

What's the two-year project you're working on? If it involves Blender and Fusion it sounds cool...

2 hours ago, KnightsFan said:

I delete tests after a while, usually about five minutes after looking at them.

That might be the best advice of all. I try to archive footage and photos I shoot, but have no idea why I keep most of it. Been shooting a lot of raw video lately and I can fill up a few 256 GB cards in a day. Backing those up is not cheap.

[KnightsFan]

38 minutes ago, HockeyFan12 said:

Are your tools OS X or Windows? One thing I didn't realize would be helpful but think would be, is including relevant information (client, project name, etc.) in the "comments" section of all the files correlated with a certain project. That way, it's searchable even if it's not in the file name, I believe. Batch modifying comments might be interesting. I'd rather not change file names in some instances, but I wouldn't mind adding comments.

Windows. I am finding that various metadata sections can be very useful for exactly that reason. I usually just include the project name in the file name, but there are all sorts of possibilities if you can automate metadata into the comment field, even if it's just an identifier to associate a video file with an audio file. Since I'm scared of accidentally messing up and losing valuable information, I am storing the original file name in a metadata field so that I can test my organization system on a real project, but relatively risk-free.

42 minutes ago, HockeyFan12 said:

The flip side of me (perhaps) having more projects to organize than you is that most of them are a couple weeks of work from start to finish at most.

Yeah, I think in that case just keeping projects separated is the main thing, and maybe keeping a notepad in the project folder with whatever useful info you can think of. I usually keep a list of shooting dates and a rough description of what we did. Like, "2019-03-23: Scene 47 (ambient sound in ZOOM_003)" and stuff like that.

44 minutes ago, HockeyFan12 said:

What's the two-year project you're working on? If it involves Blender and Fusion it sounds cool...

Just a comedy series with some friends. I've done some motion graphics for it using Blender, Fusion, and After Effects, but nothing major.

[kye]

53 minutes ago, HockeyFan12 said:

@kye, no that's totally applicable to me. "CAT VIDEO MM/DD/YYYY" describes a lot of my "personal work." I think that might be the better naming convention for a lot of my stuff.

Uploading to YouTube like that is a smart idea. The only caveats I can imagine are that you can never truly delete anything that way, or there's that story about Myspace losing 12 years of music. You can't really choose if it gets deleted, I guess.

Still, I do that with a lot of stuff on dropbox, always keeping essential files there, but am at 850GB/1TB right now and of course have to pay monthly to keep that data online.

@KnightsFan, tons of great advice, thank you.

Are your tools OS X or Windows? One thing I didn't realize would be helpful but think would be, is including relevant information (client, project name, etc.) in the "comments" section of all the files correlated with a certain project. That way, it's searchable even if it's not in the file name, I believe. Batch modifying comments might be interesting. I'd rather not change file names in some instances, but I wouldn't mind adding comments.

The flip side of me (perhaps) having more projects to organize than you is that most of them are a couple weeks of work from start to finish at most. So I can't relate to most of those concerns about years of footage existing together in one scene; even for longer-term projects, it's a few days of shooting and then pick ups. I did something long-term like that many years ago and it was awesome, though. Despite some continuity errors and changing hairstyles...

What's the two-year project you're working on? If it involves Blender and Fusion it sounds cool...

That might be the best advice of all. I try to archive footage and photos I shoot, but have no idea why I keep most of it. Been shooting a lot of raw video lately and I can fill up a few 256 GB cards in a day. Backing those up is not cheap.

lol at "CAT VIDEO MM/DD/YYYY"

It's probably obvious, but the reason I do YMD at the start of the filename is because I can sort alphabetically and it sorts chronologically, even if you move/copy files between drives and the file system stuffs up the created/modified dates on the files.

I'm kind of lucky in comparison to people with clients because I only have one brief, so I only need to be able to have the equipment to create one look (lenses, codecs, lighting, etc) but the downside is that I have the need to categorise clips by several criteria and nothing really supports that.  Things like LightRoom are great for that, being able to recall images from certain dates, or certain subjects, or certain locations, or just those rated with 5 stars.  I was using LR to do this for video clips, which was clumsy but it worked, until I moved to a 'real' video camera and LR doesn't understand the file formats and won't import them, so game over.
In a sense video is about structuring things like CLIENT/PROJECT/DAILIES and if that's what you do then great, and if not then too bad.  Also, the tools for managing footage are much more manual and tedious which is fine if you're running a big production and have time to pay attention to it, or you're not shooting most days, but when you shoot a few clips most days and it's not your day job then you really want something like the LR import tool that will sort and catalog the footage automatically on import.  I wrote a unix script (I'm on Mac) to copy files and sort them into folders based upon the file creation date on the card, so I just run it and it copies all files from a card into the YMD folder structure.  Of course, it's a PITA if I then rename the folder but don't delete the files from the card because then when I run the script again it doesn't realise I have a folder for that footage and so re-copies it.  But I really wanted something that I could use when I was travelling and you get back to the hotel at night, and you can just put everything on charge, throw the SD card and USB HDD into the computer and hit GO and not worry about losing any footage.  

I'm happy to share my script if anyone is interested.