One big complaint that I’ve received from our operators is what appears to be the constant “Updating Still” message. This usually pops up when hitting R to go to the next clip. It shows up for a quite a while when batch applying color or changes to clips. For some projects where we have hundreds of clips and we apply color to all clips, I’ve seen the “Updating Stills” message sit there for a minute or two.
To quote from our colorist:
I would say I easily lost an hour and a half this evening waiting for stills to update. I timed it, and it took about 15-17 seconds to update a single still. So, if I make a change and save it to a single shot, fifteen seconds before I can move again. If I do that four times per set up, that’s a minute gone. Which I know doesn’t sound like a lot, but if you repeat working 30 seconds, followed by staring at a progress bar 15 seconds a hundred times, I would say it not only significantly decreases speed and efficiency but it’s probably safe to say it decreases the quality of the work significantly as well.
So the main question: is there a better way to handle the “Updating Stills”? The 15-17 seconds for a single still sounds pretty high and I need to look into it. But I know the stills are saved in the destination folder (which in our case is a NAS - so there might be network issues that need addressed). Is it possible to change the location of the stills so they are on a faster, user-defined location instead?
Here’s another thought: is it possible to make the “Updating Stills” process become a background process? The stills get updated in the background, as needed and as resources allow. The user can keep working without a dialog popping up and stopping their work. If a change gets made, just delete the current still and flag it for needing an update. A background process takes care of the stills update. The user doesn’t get stalled by “Updating Stills”.
The footage is from F55 RAW if that makes any difference.
That is high. Its definitely worth getting to the bottom of this.
The stills are saved to a location defined by your project:
Certainly worth testing on your end to see if the storage is a factor there.
This could be part of it.
Two things to note here. One optimization we made recently is that when its just the color or a LUT that is updated, we don’t actually read the frame from the source file again.
But if you change any of the following, then we have to open the file and get the frame again:
Decoding / debayering settings
Framing
And if your playback is set to decode a higher quality image, than the decode will take longer…
We also recently made some changes to better optimize the behavior when you are coloring and syncing the same job simultaneously:
Hopefully this information will help you further troubleshoot what the specific factors are causing the slowness on your system.
The message box only comes up when the operation takes longer than 1 second, and the goal is to have it be fast enough that you rarely see it.
Let us know what you find out and we can dig into it deeper with you.
OK, I’ve changed the setting so Stills and LUTs are now saved to the local SSD. Initial tests show it to be a little faster. I’ll solicit feedback from our operators.
In a related note - in the “stills” folder there are individual folders for each still. Within these folders are three files: a JPG, a thumbnail and a .buf file. The .buf file I’m assuming is some sort of raw debayered image? For 4K RAW, the .buf file is around 100MB. Seems excessive; a .buf file from ProRes 4444 HD resolution is around 16MB. 100MB is not a whole lot, but it would take about 1 second to read over a network or a spinning disk (close to the time threshold you mentioned for the “Update Stills” dialog to show up). Plus, over the course of several days, hundreds of stills will be made, getting into several GB.
So I guess the questions I’m asking are these: are these file sizes excessive? Is the file size of the .buf file part of the slowness issue? Would it be better just to save a single RAW MXF frame (like a trim) and debayer it as needed?
If the decoding and framing settings are not changed after that, then we won’t generate a new .buf file, and only the thumbnail is re-rendered for the view.
But if they are modifying the exposure index often, then it will have to regenerate the .buf image.
The still-grabber uses the current playback settings to determine how to decode the raw frame, which it then stores on disk as that .buf file.
The MTI GPU debayer always does a full-res decode, and it is the default. So that will make the 100 MB file you’re seeing. To make smaller files, you can change your playback settings to software / 2K.
That should improve the performance when changing extraction settings.
The good news is switching the debayer from MTI GPU to 2K (CPU) definitely sped things up. Playback was a bit smoother. And the .buf files were around 32MB. This pretty much made the “Updating Stills” dialog disappear. I think this debayer mode solves the problem.
The bad news is Cortex was not going to let go of the MTI debayer easily. Here’s some of this issues I experienced:
Switching to CPU debayer does not “stick”. If I switched clips to CPU debayer then closed the project, when the project was reopened it was back to using the MTI GPU for debayer again.
Media gets imported using the MTI GPU debayer (which generates 100MB .buf files). Even if I imported a single file, changed it to CPU debayer, then imported the rest of the media, the clips started using the MTI GPU debayer again. (Using the MTI GPU debayer slowed media import down to about 2.5 seconds per clip. This is after I switched the still store location to an SSD.)
At one point going from clip to clip, the debayer setting automatically switched back to MTI GPU debayer. Not sure what triggered the change, but I didn’t manually change it.
As I mentioned, using the CPU debayer seems to have solved the “Updating Stills” problem. It also helps with playback. Now if there was a way to force Cortex to use CPU debayering, I’d be set. Is there some setting that I’ve overlooked that can change the default debayer to CPU?
Yes, the decoding setting is staying from session to session. The “Updating Stills” lockout is still a bit annoying though (especially if importing several hours of media at once).
When importing files, it takes a long time to grab the timeline stills. For Sony RAW, we improved this by making the playback resolution sticky, so they can import at half res and the stills are 1/4 the size. For 4K XAVC, there is no such option.
We had this as a background task before, but it consumes resources, so the system just seems sluggish while its working in the background. Putting it in the background in a way that isn’t disruptive would be difficult.
I’m open to the possibility of doing any of the following, which we will investigate.
A: a project setting that would determine a maximum still size or
B: a project setting that would allow timeline stills not to be grabbed (basically, just get the thumbnails, or set the max res to the thumbnail size).
C: add a progress or X / Y label to the updating stills dialog.
