Motion blur and depth of field in video games, and display judder/jitter

Let me start by saying that this article started as a forum post in a discussion about motion blur, which seems to be, apart from chromatic aberration, the single most hated graphics feature people can shit on without any consideration of how well or badly it is implemented in video games – at least that’s what I’m taking away from one-liner responses, sarcastic remarks or rhetoric questions. I can’t even blame these people for the general sentiment! with so many badly executed motion blur or depth of field effects. I, too, used to have an antipathy against motion blur; but that was before I realised that there is actually well-implemented motion blur that can look great; that was before I learned how to spot it, and that it has its advantages.

This post isn’t a in-depth and overly technical article diving into the depths of a dark and gloomy matter you could probably write an entire book about. The aim of this article is to provide the interested reader a starting point—some technical terms, practical examples and screenshots, and the difference between good and bad motion blur and depth of field. I will also briefly explain what judder or jitter is, which is usualy something happening on the TV/monitor/display end of the frame rendering pipeline, and how motion blur can help with those.

Let’s start off with some high quality and accurate motion blur in Sunset Overdrive:

You may want to enlarge these screenshots by right clicking them > ‘view image’.

The motion blur applied in the first screenshot blurs everything but the player character in the centre of the screen. In this scene the character is in an upward motion from bouncing off a car (which is one of several traversal options in the game). Since the camera sticks to the character, everything else relative to the camera and the character is in motion. The farther away an object is from the camera the slower is its relative speed and thus less pronounced is the motion blur effect. Look at the alleyway in the upper right part of the image, or the outdoor furniture on the roof in the upper left part of the screenshot, and see how it is less blurred than those yellow power supply lines or the street sign on the left. If you zoom in or use 100% scaling respectively, you see some grain and stepping at many edges (=aliasing) that results from the lack of proper anti-aliasing options in the game and is more pronounced when motion blur is applied, but is not a marker of low-quality blur; scaled to fit a 1920×1080 display it hides those flaws.

The motion blur in the second screenshot is subtle and less pronounced but still quite perceivable. Again, the character is in the centre of the camera and relative to the camera only the legs of the character are in motion and are thus blurred while the upper body is relatively stationary. If you look closely, both legs and feet, and the ground the character is walking on are blurred and everything just blends neatly into each other without any clear-cut line between different degrees (e.g. light, medium, heavy) of motion blur.

Now look at an extreme example:

You may want to enlarge the screenshot by right clicking it > ‘view image’.

Because the character swings swiftly on the line, her speed relative to the camera is pretty high, causing the heavy distortion at her legs and feet, but less distortion, again, on her upper body and almost no distortion on her forearms. Even particle effects gradually become blurred: from the centre of their origins (see the bright flash at the line) that is less blurred than to the stretched edges of the particles.

All three screenshots show no artefacts (besides the aliasing) and keeps the transition from off-focus to sharp on-focus smooth as silk (see the ground on the right, in front of the stairs, that becomes gradually sharper).

However, I think good motion blur implementation isn’t common and even static blur like depth of field is often low quality in many games. When it comes to depth of field, I want to mention Mass Effect: Andromeda here. It does an excellent job at rendering depth of field and shows remarkable quality at the border of blurred and non-blurred objects. It really nails that cinematic look:

You may want to enlarge these screenshots by right clicking them > ‘view image’.

In combination with Mass Effect’s temporal anti-aliasing solution it works wonders and produces one of the best applications of depth of field I’ve seen yet in few games. In the first screenshot, look at the shiny metal object in the centre of the screen that is still within the camera’s focus and thus sharp. Everything farther away from the camera, behind that object is blurred. In such an environment, opposed to motion blur, you don’t want any transition and you see a clear cut between on and off-focus. However, even if you zoom in, there are no artefacts (grain, distortion, ghosting/repeated frames) whatsoever. Look at the helmet in the second screenshot: there is blending of foreground and background and the line is clear-cut, as in a photograph taken from a high-resolution camera with a perfectly adjusted focal length. Also look at the background on the left under the character’s arm that is blurred: the actual focus is very narrow, and only a small radius or length is drawn sharply.

To produce such clean depth of field (and motion blur in Sunset Overdrive) seems not easy and many games fail mostly in terms accuracy. Here is an example of a less well-implemented depth of field to give you an idea (it’s from Quantum Break—whose art direction and visuals I adore—which fails in many instances when it comes to blur, though):

Look at the character’s ear on the left side of the image and at his shoulders:

You may want to enlarge the screenshot by right clicking it > ‘view image’.

You see artefacts, disrupting the image on edges (the little kink of the reflection behind his ear), and ghosting (look along the line of his shoulder on the left) and the non-blurred line (right side, shoulder) between his shoulder and the background that is blurred: this is what I call accuracy. The detection of the lines where to apply blur and where to not seems off a but.

Here is another example of ghosting (an offset of the same image showed twice or even more at the same time) on ears and shoulders, making it look like a badly-used photoshop motion blur filter:

You may want to enlarge the screenshot by right clicking it > ‘view image’.

If you look at this screenshot, you’ll find the Mass Effect examples much better in accuracy and detection of where to apply blur (and how) and where not. Transition from in-focus to out of focus (the near and far-blur curve* is working perfectly together) is clear-cut without any signs of artefacts or other distortions without blending parts of the image from near to far focus (like in Quantum Break).

All this applies to motion blur as well, only that smooth transitions are even more important when fast objects move in front of stationary objects, and large objects that are farther away receive less blur due to their relatively slower speed compared to anything near the camera.

There are two reasons you might consider using motion blur:

Firstly, not every game can maintain high frame rates like 120, 144fps or even higher. Many games can’t even maintain 60fps given your PC hardware or, particularly, on consoles. The majority of console games render at 30fps (which means that every 33.33ms one frame is rendered). Good motion blur can actually compensate (to a degree) limitations of monitors and TVs, which can produce image doubling in 30fps games. What happens is that the current [B]and[/B] previous frame are shown simultaneously. This happens because a 60hz displays refreshes exactly two times (every 16.6ms) while one frame is being rendered (every 33.33ms at 30fps), and it looks like this:

source: https://forums.pubg.com/topic/291688-september-18th-hotfix-feedback-topic/page/4/?tab=comments#comment-1052671

It looks horrible and is definitely not something you would want deliberately, and it has nothing to with motion blur that is used on purpose (some refer to this issue with the term motion blur as well, which might lead to confusion). Many TV displays offer options (e.g. TrueMotion, Motion Plus, Motion Smoothing, Perfect Motion, etc. depending on your brand and model) of frame interpolation to compensate for this effect and it does works in many instances. However, it not only creates the so called “soap opera effect” in many movies and makes action and movement look somewhat ridiculous, it also creates artefacts around affected objects and edges depending on the setting and can also introduce jitter or judder, which is a timing error between a rendered frame and the refresh rate.

Sidenote: That’s also the reason why most discussion evolve around 30 and 60fps, and not 40, 50, or 55fps. As mentioned earlier, a 60Hz display refreshes exactly two times each frame is rendered after 33.33ms. Now imagine playing a game at 40, which means each frame is rendered at 25ms. Your 60Hz monitor still refreshes every 16.66ms, so technically every 25ms your monitor would have refreshed 1.5 times. But there’s no such thing as a “half refresh” – your monitor either builds/draw the frame or it doesn’t. (It’s like light absorption when one electron jumps from one state to another with no gradual transition, or think of an switch that has only on and off as options with nothing in between.) Rendering time and refresh time is out of sync and it needs roughly three refresh cycles (3*16.66ms) to show two rendered frames (2*25ms). Think of it like that: the first time your display refreshes there’s no new frame (because the frame needs 25ms), so your display shows the previous frame. The second time it refreshes, it draws a new frame that, by now, is already 8.33ms old. So either the rendered frame arrives too late or the monitor refreshes too late – it is ‘out of sync’ and this results uneven frame pacing. This is also referred to by microstutter.

But why do we talk about jitter/judder when a game achieves frames rendered at perfectly timed 33.33ms? Frame or motion interpolation is a technique to add frames between the default frames that are being shown on the display in order to artificially boost the refresh rate (that’s why some brands advertise their panels with something like “800Hz” which is just the interpolated rate). Imagine the image as shown above with an added frame that shows the steel girders a medium position between the old and new position – like drawing a flip-book where you first draw a picture every two pages but then you add an intermediate picture between these pictures. Those added frames, though, can screw up the, let’s call it “synchronicity” of refresh rate and frame times (the time need for rendering frames) and can introduce microstutter.

Jitter or microstutter causes uneven/stuttered movement and is perceived as uneven frame times (think of camera movement that should be smooth but the final image shows little jumps here and there). The jitter introduced by motion interpolation though makes a game feel less smooth even when rendered frame times (at the beginning of the render pipeline) are completely equal and smooth. Motion blur can actually compensate the judder effect to a degree where it actually looks more like one frame being somewhat distorted instead of showing two frames simultaneously, and since this only applies on motion objects where perception is reduced anyways depending on motion speed, it usually creates a smoother and more uniform perception of a frame and movement.

The left screenshot shows The Witcher 3 at 30fps on my 60hz TN panel with motion blur disabled in-game. The right screenshot shows the game with enabled motion blur. You can still see image doubling because it’s still something that happens due to technical limitations but it is masked by motion blur. The negative side-effect is you lose image clarity and details in the process and I see why people might want to disable motion blur. Since it only happens in motion where detail perception of our eyes is already lowered to a degree, and the game maintains its clarity in less moving instances, I prefer motion blur in such instances.

Of course, it’s possible to prefer the image doubling of 30fps on 60Hz displays over the use of motion blur, and this is why I’m always for more options. And it’s not that of an issue you play at 60fps or even more.

However, there’s a second reason I hinted a few paragraphs ago as to why you would want to consider using motion blur or any kind of blur, like depth of field: It is artistic choice. Photography and cinematography use diverse blur methods for a really long time now to achieve a specific effect on the audience (making them want to puke is usually not deliberate). The same way authors of poems, prose, speeches and so on, use word order manipulations, uncommon sentence structures, weird or no punctuation at all; these choices are deliberate and are made for a specific effect.

There is the argument that in a game you have the control over player character and camera and you usually want to choose the focus of objects or characters by yourself, since you already control the character into the direction where you want to, and in the end that’s what games are all about—giving control and interaction. However, in many instances this control is, of course not seen uncritically by some, taken away from the player, for examples in cutscenes. In these scenes control is totally taken away from the player (although recent trends added the controversial quick time events that still requires player engagement) and given to the developer: Not only do they choose what characters do but also how this is presented. It is easier to build up suspense and excitement when you (the developer) control any involved element. But it’s not only about making it easier, the control being taken away from the player can add to a certain effect of passiveness, helplessness that could underline the narrative involving characters and their actions. In a sequence where the hero is a mere bystander, unable to do anything to prevent the death of innocents, taking away control from the player can emphasise the powerlessness. Another way could be to not take away control from the player, let him try to do something and then realise gradually that he can’t do a damn thing. Both options achieve the same effect of powerlessness but with a different approach that subtly alters the perception and the process of realising this effect. No option is superior or inherently better than the other. From a technical point of view cutscenes also allow better animations, graphical details and overall better graphical fidelity in many instances: Geometry can be improved by increased polygons because you show only the half of what the camera would usually see; lighting can be altered in a way to pronounce certain areas, characters, and/or objects while the global lighting of a level in non-cutscene gameplay has to function on a global level; cutscenes are nowadays usually motion captured from the beginning to the end, while animations on a gameplay level, although also motion captured, still have to blend into another, follow-up animation (you see the subtle differences in many games when characters move very life-like and detailed in cutscenes, but use identical animations in gameplay over and over again, even if they have a different set of animations for the same motion, it gets repetitive eventually).

A classic use of changing focus, for example, would be a static camera in a living room, pointing towards the hallway and its staircase. In the foreground is a telephone masked by blur, a person comes down the stairs, is in focus and maybe wants to leave the hallway and the house through the front door – that’s where the audience should focus on. Then the phone rings and the focus switches from the person to the telephone. In this instance you would expect a very urgent, plot-twisting or at least somewhat important phone call, because your whole attention is now drawn to the telephone, which becomes the most important thing in that frame, and even the person becomes secondary. Depth of field is about attention and focus and emphasis, while motion blur provides a sense of speed (and also attention while moving) in moving images and still images: A photo taken with ultra-high shutter speed makes a racing scene looks like the cars are standing still, while reduced shutter speed and thus the motion blur provides a sense of (high speed) motion even there’s technically none since it’s a still image. Just look again at the Witcher screenshots above. While, just seeing at the screenshot, you couldn’t say what’s going on in the first one because there’s little sense of movement unless you mistake the image issue for motion blur (but seeing this alleged ‘motion blur’ only in the upper part of the image and not on the ground and grass, you’ll more likely feel that something’s wrong with the image). With the second one, it is clearer that there is motion, although a combat pose with a drawn sword would make it even more clear that the camera is moving, e.g. to increase action and drama.

So after all this what sounds relatively positive in regards of motion blur, you might wonder what then is badly implemented motion blur? From a technical point of view I already showed an example of not-so-well executed depth of field in Quantum Break. When it comes to motion blur the Uncharted series is notorious for less accurate motion blur:

You may want to enlarge the screenshot by right clicking it > ‘view image’.

Look at the character’s arm. It looks like he was pasted into this picture because there’s no transition from the edge of his arm to the background. The edge itself is unblurred and only the pixels left and right are blurred. Also notice the artefact at his hair. This screenshot was taken from Uncharted 4, although the earlier titles that received the motion blur option on PS4 suffer from the same issue.

Another example is Metal Gear Solid V:

You may want to enlarge the screenshot by right clicking it > ‘view image’.

It’s not as bad as in Uncharted, but the object blur is not good either. Look at the horse’s legs; it’s like a line was drawn across where the blur suddenly stops, showing almost no transition beginning from its tail, going across its hind legs and forelegs. Some transition is visible at its head.

While the examples above is more about quality, it’s also about quantity. Another negative side of motion blur is its excessive use even at low speed movements in many games. Also, camera motion blur is often not seperated from object motion blur—while some enjoy high quality object motion blur, they don’t like motion blur introduced by camera movements. Some people claim to get eye strain or even headaches, which I can confirm: I remember having heavy eye strain while playing Fable 2 back then on the Xbox 360 because the low resolution was paired with a heavy blur-effect. Playing Fallout 3 on the Xbox 360, which was also rendered at 720p but didn’t use excessive blur, was almost like going from strechted to fit 720p on a 1080p display to native 1080p; it was much more pleasent to the eyes. And given that the camera is something you alter almost every second depending on the genre, I can imagine how this gets bothersome while object motion blur is usually reserved for high speed objects coming across the camera that happens mostly in racing games, but is less common in other genres. Even in fast-paced shooter the effect should be subtle, given the relative slow speed of running characters.

In the best case motion blur is used as an real life extension and it usually requires high speed for any object to appear blurry to our eyes. Games should subtly expand on that idea and should only add slight blur to enable action-rich and dramaticly fast moments to be captured for screenshots and to invoke that sense of speed even if you look away from the center to remind you that you probably shouldn’t look away. To achieve that it’s not necessary, though, to make every detail distorted to an unrecognizable state.

Depth of field should be reserved for cinematic cutscenes and shouldn’t intervene with normal gameplay except for, again, subtle use like blurring distant mountains in the backgrounds (or to mask far away low polygon assets) or slightly blur the gun when aiming down the sights to support your sight and reduce possible distractions (e.g. reflection shaders on the weapon).

Kommentar verfassen