The Files

When you record a lot of material, the file storage quickly turns into a major problem. Consider that ScreenFlow, when recording from the computer screen, an iPad, and from the webcam¹, produces some 20-30 GB per hour of recording. The video camera additionally produces 20-25 GB per hour of recording at 1080p, 30 fps.

Then when editing, the camera files get copied into the ScreenFlow files, so that means another 20-25 GB per hour of duplicated files². Finally, ScreenFlow automatically creates some proxy files, but that only impacts the editing machine³, and then you have the output video files, which are much more efficient, but can be between 4 and 10 GB per hour, depending on content⁴.

On top of all this, you need backups. So if you back up and keep two versions, which is really bare bones, you have to triple the above numbers. The final tally comes to around 100 GB of storage per hour of material for the editing and production pipeline, and another 200 GB per hour of material for backups. In total, for a 50 hour course, that is around 5 TB in production and another 10 TB for backups⁵.

Between machines

As I already mentioned, I transport the ScreenFlow files from the laptop to the desktop using a 1TB external M.2 SSD. The camera files come in from the 128 GB SDXC cards from the camera. The iMac has a built-in reader, so that works fine. Slowly, at around 80 MB/s⁶, but fine.

While editing

Initially, I placed the camera files on the desktop, and edited from the external SSD, and that was pretty fast. My desktop iMac, however, doesn’t have that much space, so I switched to uploading the camera files to one of my Synologys, then adding them from there.

After finalizing the edits, I copy the ScreenFlow project from the external SSD to the Synology.

Storage

I have a Synology 1517+ since a while. It has five 8TB drives and I use it mainly for Retrospect backups. When I started uploading ScreenFlow projects to it as well, it quickly became a problem.

One problem is the 1Gb/s network. That is, for this kind of use, really slow. Who’d have thunk… ⁷ Also, the net capacity of that NAS is “only” around 28TB, since it’s set up as a RAID with a one-disk redundancy. And the way the PR department labels the size of the drives when you buy them, versus what the NAS box actually thinks about that size.

But even worse, it’s clearly a single point of failure. You really shouldn’t have both your source material and the backups in the same place. That’s begging for problems and total loss sooner or later.

One more thing you should consider when setting up backups: take care not to mount the backup storage as a drive on any computer, since any ransomware will be able to encrypt all your originals and your backups at the same time. That will be a bad day, indeed.

So, I have Retrospect log in to its own volume with its own user, which is not logged in anywhere else. I also take occasional offline copies of the ScreenFlow files to 4 TB drives through a desktop USB3 dock⁸, then keep them in a drawer.

So I got another Synology, this time an 1817, which is a model with built-in 10 Gb/s ethernet. I added four 12 TB helium drives to it⁹.

Now I’m using the new Synology for production files and the old for Retrospect backups. I am planning on upgrading the old Synology to 10 Gb/s, which can be done with an extension card. I’ll probably go the SFP+ route there, since there is already a fiber running to the switch right next to it.

Turns out that it’s possible to do production with the files at the other end of a 10 Gb/s network, but it’s still anything but pleasant. The solution to this is to move the production files to the desktop while working on them, and only after that moving them to the NAS. This requires enough space in the desktop, though. And that’s one of the reasons I’m getting a new iMac.

The network

To be able to enjoy the 10 Gbit/s connectivity to the Synology/Synologys, you need a 10 Gbit/s network. The only switch I could find that does 10 Gbit and that doesn’t contain a fan (I hate noise) was the Mikrotik CRS309. This switch has a single 1 Gbit/s connector and 8 SFP+ sockets, so I got 3 SPF+ modules for 10 Gbit/s copper. My network cables are already good enough (Cat 6a, I think), so it was just a question of plugging them in and away you go. I’m planning on putting in another SFP+ for LC connectors and multimode (850 nm) to connect to the older Synology later.

My backup server, a 2018 Mac Mini, already has 10 Gbit/s network, so once both Synologys are 10 Gbit/s, Retrospect backups ought to speed up quite a bit.

The new iMac will have 10 Gbit/s, but meanwhile I’m using a Thunderbolt 2¹⁰ 10GBit/s interface from Sonnet and I’m getting 300-400 MB/s to the new Synology, which I think is around the max speed you can get from four SATA drives.

1. I always include the webcam from the computer as a check on synchronization and to see when Hania shifts her attention to the iPad. It’s absolutely essential, but none of it is included in the final footage.
2. I could delete the camera files at this point, but I’m not that brave.
3. I’ve seen it use around 100 GB, but I think it depends on how much free space there is.
4. Talking head and vigorous iPad use makes for larger files. Keynote presentations for not much at all.
5. There’s another twist here, namely that in my case, with Retrospect, the deletion of old backups happen in a separate script, so even if you configure to keep two backups, if you do mass updates then temporarily you may well need another 5-10 TB for a 50 hour course.
6. The cards are speced at 170 MB/s, we’ll see if that is actually true when I get the new iMac.
7. I’m from an era were we almost wet ourselves seeing a 1 Mbit/s Arcnet for the first time…
8. The one I have maxes out at 4 TB, so I need something better. That will also be a later post.
9. Oh, by the way, I discovered there’s this “4k native” versus “512 emulated” sector size thing going on. You need to take care when ordering drives not to mix them. Life is full of surprises.
10. Since my 2014 iMac has Thunderbolt 2.