It seems like Artificial Intelligence has been inserted into everything these days, and now that includes this website.
I’ve been interested for some time in whether or not some of the recent advances in A.I. could be used to enhance some of these old movies. Often when it comes to enhancing old media it comes with a price. Well, I purchased myself a copy of Topaz Video A.I. to find out. And I found out some stuff.
First of all, I can say yes, in some cases you can upscale a movie from 480p to 1080p and get a vastly improved picture, but it isn’t magic, and it isn’t perfect. Topaz Video A.I. offers a dizzying array of options to enhance a video, and sometimes gives you options to adjust those options. Adjusting these settings without consideration can easily give the output a strange, uncanny quality, like something is off but you can’t immediately tell what it is. But if you are somewhat conservative and use only what the source video needs you can get some good results. Most importantly, and somewhat counterintuitively for a program that’s supposed to improve the quality of a video, the quality of the source input makes a huge different.
Here is an example of a video I’m upscaling as I write this.
Original is on the left and the A.I. upscaled version is on the right.
You can likely already notice that it’s done a pretty good job sharpening the text in this still from the Charlie Chan movie Shanghai Chest. For a better comparison let’s look at it in the 1080p upscaled version’s native resolution.
The difference should be clear. I think the upscaled version looks very nice.
I think the upscale here is a big improvement. And generally, when the settings are dialed in appropriately for the video file that’s being upscaled, the results are very good. Shanghai Chest happens to be a very good standard definition print. There is very little image jitter, the picture is clear, and it was either de-interlaced properly or the video was always a progressive scan. Because of this, I don’t have to enable a bunch of settings to get good results from Topaz Video A.I. For Shanghai Chest I only upscaled the image and left all the stabilizing, frame interpolation, de-interlacing, and other settings off. Results begin to vary and require more experimentation when the source video has larger issues.
Here is an example where I’m not so sure applying A.I. to a video enhances the overall watching experience. Take the opening of this Dragnet episode where I’ve upscaled a 416x320px .avi file found at archive.org and applied some stabilization for reduce the jitter present throughout the episode and upscaled the video to 720p.
This is after I’ve experimented with the settings a good bit to get as good a result as I can, and as you can see the video is sharper and amazingly stable considering its source. But I think you can start to see the limitations of what the software can do when the video source is of a sub optimal quality and so low resolution. It’s not the fault of Topaz Video A.I., and I’m eternally grateful for the people who make these old shows available at any quality, but we can start to see that the software can’t perform miracles.
As the software sharpens the picture, any problems with the video, like the edge ghosting you see in the Dragnet title, will be sharpened right along with it. And as there is not enough information in the shadow behind the badge, we see a checkerboard pattern emerging that was surely not visible in the original broadcast. So, the output is flawed, but the stabilization alone could be enough for some to prefer the upscaled version. Well, there are other things.
I’ve found in many video files where a jump cut might have a single transitionary frame you get what I find to be a very distracting fade between shots. This is especially distracting in this scene where the cuts happen in rapid succession. Some people that do work with video might assume that I have the frame interpolation setting on and turning it off would fix this, but it’s not on. I am guessing the software uses adjacent frames to get more information about how it should enhance it’s current frame, creating this effect. I’m not sure if that’s the case, but the effect makes the episode unwatchable for me.
This is just one example of some of the adverse effects upscaling a sub-optimal video can give. If the source video was de-interlaced wrong, too noisy, or even further low resolution other strange things happen to the video. And of course, it doesn’t do anything for the audio.
And then we get to the main limitation of upscaling these videos, time. I don’t have a super high-end machine to do these conversions on. Aside from the time experimenting with settings to get good results, the fastest Topaz Video A.I. runs when just doing an upscale is around 4 frames a second. Shanghai Chest’s runtime is around an hour long and took more than six hours to upscale. The resulting file was around 9GB and had to be transcoded again to make a web-friendly version of the file. Depending on how much stabilization, frame interpolating, and other settings are needed I’ve had video conversions running at less than 1 frame a second, meaning a whole movie can take more than a day to finish. And as in this episode of Dragnet’s case, sometimes the resulting file has worse issues than the ones it fixes.
So, in conclusion, tossing A.I. at these old movies and tv shows is definitely not a fix-all or going to net you Blu-Ray quality results, and getting an improved picture will take some experimentation and learning the settings. But for some already decent files it can have very good results. It’s definitely something I’ll keep working at. Topaz Video A.I. is always coming out with updates, as well so as the software improves so will the results. But I don’t think it’s ever going to be able to fix the issues in the lowest quality video files that are still the only options available at archive.org.
Leave a Reply