Per-Object Motion Blur (bevyengine#9924)

https://github.com/bevyengine/bevy/assets/2632925/e046205e-3317-47c3-9959-fc94c529f7e0

# Objective

- Adds per-object motion blur to the core 3d pipeline. This is a common
effect used in games and other simulations.
- Partially resolves bevyengine#4710

## Solution

- This is a post-process effect that uses the depth and motion vector
buffers to estimate per-object motion blur. The implementation is
combined from knowledge from multiple papers and articles. The approach
itself, and the shader are quite simple. Most of the effort was in
wiring up the bevy rendering plumbing, and properly specializing for HDR
and MSAA.
- To work with MSAA, the MULTISAMPLED_SHADING wgpu capability is
required. I've extracted this code from bevyengine#9000. This is because the
prepass buffers are multisampled, and require accessing with
`textureLoad` as opposed to the widely compatible `textureSample`.
- Added an example to demonstrate the effect of motion blur parameters.

## Future Improvements

- While this approach does have limitations, it's one of the most
commonly used, and is much better than camera motion blur, which does
not consider object velocity. For example, this implementation allows a
dolly to track an object, and that object will remain unblurred while
the background is blurred. The biggest issue with this implementation is
that blur is constrained to the boundaries of objects which results in
hard edges. There are solutions to this by either dilating the object or
the motion vector buffer, or by taking a different approach such as
https://casual-effects.com/research/McGuire2012Blur/index.html
- I'm using a noise PRNG function to jitter samples. This could be
replaced with a blue noise texture lookup or similar, however after
playing with the parameters, it gives quite nice results with 4 samples,
and is significantly better than the artifacts generated when not
jittering.

---

## Changelog

- Added: per-object motion blur. This can be enabled and configured by
adding the `MotionBlurBundle` to a camera entity.

---------

Co-authored-by: Torstein Grindvik <52322338+torsteingrindvik@users.noreply.github.com>

Cargo.toml

@@ -766,6 +766,17 @@ description = "Loads and renders a glTF file as a scene"
 category = "3D Rendering"
 wasm = true
 
+[[example]]
+name = "motion_blur"
+path = "examples/3d/motion_blur.rs"
+doc-scrape-examples = true
+
+[package.metadata.example.motion_blur]
+name = "Motion Blur"
+description = "Demonstrates per-pixel motion blur"
+category = "3D Rendering"
+wasm = false
+
 [[example]]
 name = "tonemapping"
 path = "examples/3d/tonemapping.rs"

crates/bevy_core_pipeline/src/core_3d/mod.rs

@@ -26,6 +26,7 @@ pub mod graph {
  MainTransparentPass,
  EndMainPass,
  Taa,
+ MotionBlur,
  Bloom,
  Tonemapping,
  Fxaa,

crates/bevy_core_pipeline/src/lib.rs

@@ -15,6 +15,7 @@ pub mod core_3d;
 pub mod deferred;
 pub mod fullscreen_vertex_shader;
 pub mod fxaa;
+pub mod motion_blur;
 pub mod msaa_writeback;
 pub mod prepass;
 mod skybox;
@@ -53,6 +54,7 @@ use crate::{
  deferred::copy_lighting_id::CopyDeferredLightingIdPlugin,
  fullscreen_vertex_shader::FULLSCREEN_SHADER_HANDLE,
  fxaa::FxaaPlugin,
+ motion_blur::MotionBlurPlugin,
  msaa_writeback::MsaaWritebackPlugin,
  prepass::{DeferredPrepass, DepthPrepass, MotionVectorPrepass, NormalPrepass},
  tonemapping::TonemappingPlugin,
@@ -89,6 +91,7 @@ impl Plugin for CorePipelinePlugin {
  BloomPlugin,
  FxaaPlugin,
  CASPlugin,
+ MotionBlurPlugin,
  ));
  }
 }

crates/bevy_core_pipeline/src/motion_blur/mod.rs

@@ -0,0 +1,168 @@
+//! Per-object, per-pixel motion blur.
+//!
+//! Add the [`MotionBlurBundle`] to a camera to enable motion blur. See [`MotionBlur`] for more
+//! documentation.
+
+use crate::{
+ core_3d::graph::{Core3d, Node3d},
+ prepass::{DepthPrepass, MotionVectorPrepass},
+};
+use bevy_app::{App, Plugin};
+use bevy_asset::{load_internal_asset, Handle};
+use bevy_ecs::{
+ bundle::Bundle, component::Component, query::With, reflect::ReflectComponent,
+ schedule::IntoSystemConfigs,
+};
+use bevy_reflect::{std_traits::ReflectDefault, Reflect};
+use bevy_render::{
+ camera::Camera,
+ extract_component::{ExtractComponent, ExtractComponentPlugin, UniformComponentPlugin},
+ render_graph::{RenderGraphApp, ViewNodeRunner},
+ render_resource::{Shader, ShaderType, SpecializedRenderPipelines},
+ Render, RenderApp, RenderSet,
+};
+
+pub mod node;
+pub mod pipeline;
+
+/// Adds [`MotionBlur`] and the required depth and motion vector prepasses to a camera entity.
+#[derive(Bundle, Default)]
+pub struct MotionBlurBundle {
+ pub motion_blur: MotionBlur,
+ pub depth_prepass: DepthPrepass,
+ pub motion_vector_prepass: MotionVectorPrepass,
+}
+
+/// A component that enables and configures motion blur when added to a camera.
+///
+/// Motion blur is an effect that simulates how moving objects blur as they change position during
+/// the exposure of film, a sensor, or an eyeball.
+///
+/// Because rendering simulates discrete steps in time, we use per-pixel motion vectors to estimate
+/// the path of objects between frames. This kind of implementation has some artifacts:
+/// - Fast moving objects in front of a stationary object or when in front of empty space, will not
+/// have their edges blurred.
+/// - Transparent objects do not write to depth or motion vectors, so they cannot be blurred.
+///
+/// Other approaches, such as *A Reconstruction Filter for Plausible Motion Blur* produce more
+/// correct results, but are more expensive and complex, and have other kinds of artifacts. This
+/// implementation is relatively inexpensive and effective.
+///
+/// # Usage
+///
+/// Add the [`MotionBlur`] component to a camera to enable and configure motion blur for that
+/// camera. Motion blur also requires the depth and motion vector prepass, which can be added more
+/// easily to the camera with the [`MotionBlurBundle`].
+///
+/// ```
+/// # use bevy_core_pipeline::{core_3d::Camera3dBundle, motion_blur::MotionBlurBundle};
+/// # use bevy_ecs::prelude::*;
+/// # fn test(mut commands: Commands) {
+/// commands.spawn((
+/// Camera3dBundle::default(),
+/// MotionBlurBundle::default(),
+/// ));
+/// # }
+/// ````
+#[derive(Reflect, Component, Clone, ExtractComponent, ShaderType)]
+#[reflect(Component, Default)]
+#[extract_component_filter(With<Camera>)]
+pub struct MotionBlur {
+ /// The strength of motion blur from `0.0` to `1.0`.
+ ///
+ /// The shutter angle describes the fraction of a frame that a camera's shutter is open and
+ /// exposing the film/sensor. For 24fps cinematic film, a shutter angle of 0.5 (180 degrees) is
+ /// common. This means that the shutter was open for half of the frame, or 1/48th of a second.
+ /// The lower the shutter angle, the less exposure time and thus less blur.
+ ///
+ /// A value greater than one is non-physical and results in an object's blur stretching further
+ /// than it traveled in that frame. This might be a desirable effect for artistic reasons, but
+ /// consider allowing users to opt out of this.
+ ///
+ /// This value is intentionally tied to framerate to avoid the aforementioned non-physical
+ /// over-blurring. If you want to emulate a cinematic look, your options are:
+ /// - Framelimit your app to 24fps, and set the shutter angle to 0.5 (180 deg). Note that
+ /// depending on artistic intent or the action of a scene, it is common to set the shutter
+ /// angle between 0.125 (45 deg) and 0.5 (180 deg). This is the most faithful way to
+ /// reproduce the look of film.
+ /// - Set the shutter angle greater than one. For example, to emulate the blur strength of
+ /// film while rendering at 60fps, you would set the shutter angle to `60/24 * 0.5 = 1.25`.
+ /// Note that this will result in artifacts where the motion of objects will stretch further
+ /// than they moved between frames; users may find this distracting.
+ pub shutter_angle: f32,
+ /// The quality of motion blur, corresponding to the number of per-pixel samples taken in each
+ /// direction during blur.
+ ///
+ /// Setting this to `1` results in each pixel being sampled once in the leading direction, once
+ /// in the trailing direction, and once in the middle, for a total of 3 samples (`1 * 2 + 1`).
+ /// Setting this to `3` will result in `3 * 2 + 1 = 7` samples. Setting this to `0` is
+ /// equivalent to disabling motion blur.
+ pub samples: u32,
+ #[cfg(all(feature = "webgl", target_arch = "wasm32"))]
+ // WebGL2 structs must be 16 byte aligned.
+ pub _webgl2_padding: bevy_math::Vec3,
+}
+
+impl Default for MotionBlur {
+ fn default() -> Self {
+ Self {
+ shutter_angle: 0.5,
+ samples: 1,
+ #[cfg(all(feature = "webgl", target_arch = "wasm32"))]
+ _webgl2_padding: bevy_math::Vec3::default(),
+ }
+ }
+}
+
+pub const MOTION_BLUR_SHADER_HANDLE: Handle<Shader> =
+ Handle::weak_from_u128(987457899187986082347921);
+
+/// Adds support for per-object motion blur to the app. See [`MotionBlur`] for details.
+pub struct MotionBlurPlugin;
+impl Plugin for MotionBlurPlugin {
+ fn build(&self, app: &mut App) {
+ load_internal_asset!(
+ app,
+ MOTION_BLUR_SHADER_HANDLE,
+ "motion_blur.wgsl",
+ Shader::from_wgsl
+ );
+ app.add_plugins((
+ ExtractComponentPlugin::<MotionBlur>::default(),
+ UniformComponentPlugin::<MotionBlur>::default(),
+ ));
+
+ let Some(render_app) = app.get_sub_app_mut(RenderApp) else {
+ return;
+ };
+
+ render_app
+ .init_resource::<SpecializedRenderPipelines<pipeline::MotionBlurPipeline>>()
+ .add_systems(
+ Render,
+ pipeline::prepare_motion_blur_pipelines.in_set(RenderSet::Prepare),
+ );
+
+ render_app
+ .add_render_graph_node::<ViewNodeRunner<node::MotionBlurNode>>(
+ Core3d,
+ Node3d::MotionBlur,
+ )
+ .add_render_graph_edges(
+ Core3d,
+ (
+ Node3d::EndMainPass,
+ Node3d::MotionBlur,
+ Node3d::Bloom, // we want blurred areas to bloom and tonemap properly.
+ ),
+ );
+ }
+
+ fn finish(&self, app: &mut App) {
+ let Some(render_app) = app.get_sub_app_mut(RenderApp) else {
+ return;
+ };
+
+ render_app.init_resource::<pipeline::MotionBlurPipeline>();
+ }
+}

crates/bevy_core_pipeline/src/motion_blur/motion_blur.wgsl

@@ -0,0 +1,149 @@
+#import bevy_pbr::prepass_utils
+#import bevy_pbr::utils
+#import bevy_core_pipeline::fullscreen_vertex_shader::FullscreenVertexOutput
+#import bevy_render::globals::Globals
+
+#ifdef MULTISAMPLED
+@group(0) @binding(0) var screen_texture: texture_2d<f32>;
+@group(0) @binding(1) var motion_vectors: texture_multisampled_2d<f32>;
+@group(0) @binding(2) var depth: texture_depth_multisampled_2d;
+#else
+@group(0) @binding(0) var screen_texture: texture_2d<f32>;
+@group(0) @binding(1) var motion_vectors: texture_2d<f32>;
+@group(0) @binding(2) var depth: texture_depth_2d;
+#endif
+@group(0) @binding(3) var texture_sampler: sampler;
+struct MotionBlur {
+ shutter_angle: f32,
+ samples: u32,
+#ifdef SIXTEEN_BYTE_ALIGNMENT
+ // WebGL2 structs must be 16 byte aligned.
+ _webgl2_padding: vec3<f32>
+#endif
+}
+@group(0) @binding(4) var<uniform> settings: MotionBlur;
+@group(0) @binding(5) var<uniform> globals: Globals;
+
+@fragment
+fn fragment(
+ #ifdef MULTISAMPLED
+ @builtin(sample_index) sample_index: u32,
+ #endif
+ in: FullscreenVertexOutput
+) -> @location(0) vec4<f32> { 
+ let texture_size = vec2<f32>(textureDimensions(screen_texture));
+ let frag_coords = vec2<i32>(in.uv * texture_size);
+
+#ifdef MULTISAMPLED
+ let base_color = textureLoad(screen_texture, frag_coords, i32(sample_index));
+#else
+ let base_color = textureSample(screen_texture, texture_sampler, in.uv);
+#endif
+
+ let shutter_angle = settings.shutter_angle;
+
+#ifdef MULTISAMPLED
+ let this_motion_vector = textureLoad(motion_vectors, frag_coords, i32(sample_index)).rg;
+#else
+ let this_motion_vector = textureSample(motion_vectors, texture_sampler, in.uv).rg;
+#endif
+
+#ifdef NO_DEPTH_TEXTURE_SUPPORT
+ let this_depth = 0.0;
+ let depth_supported = false;
+#else
+ let depth_supported = true;
+#ifdef MULTISAMPLED
+ let this_depth = textureLoad(depth, frag_coords, i32(sample_index));
+#else
+ let this_depth = textureSample(depth, texture_sampler, in.uv);
+#endif
+#endif
+
+ // The exposure vector is the distance that this fragment moved while the camera shutter was
+ // open. This is the motion vector (total distance traveled) multiplied by the shutter angle (a
+ // fraction). In film, the shutter angle is commonly 0.5 or "180 degrees" (out of 360 total).
+ // This means that for a frame time of 20ms, the shutter is only open for 10ms.
+ //
+ // Using a shutter angle larger than 1.0 is non-physical, objects would need to move further
+ // than they physically travelled during a frame, which is not possible. Note: we allow values
+ // larger than 1.0 because it may be desired for artistic reasons.
+ let exposure_vector = shutter_angle * this_motion_vector;
+
+ var accumulator: vec4<f32>;
+ var weight_total = 0.0;
+ let n_samples = i32(settings.samples);
+ let noise = utils::interleaved_gradient_noise(vec2<f32>(frag_coords), globals.frame_count); // 0 to 1
+
+ for (var i = -n_samples; i < n_samples; i++) {
+ // The current sample step vector, from in.uv
+ let step_vector = 0.5 * exposure_vector * (f32(i) + noise) / f32(n_samples);
+ var sample_uv = in.uv + step_vector;
+ let sample_coords = vec2<i32>(sample_uv * texture_size);
+
+ #ifdef MULTISAMPLED
+ let sample_color = textureLoad(screen_texture, sample_coords, i32(sample_index));
+ #else
+ let sample_color = textureSample(screen_texture, texture_sampler, sample_uv);
+ #endif
+ #ifdef MULTISAMPLED
+ let sample_motion = textureLoad(motion_vectors, sample_coords, i32(sample_index)).rg;
+ #else
+ let sample_motion = textureSample(motion_vectors, texture_sampler, sample_uv).rg;
+ #endif
+ #ifdef NO_DEPTH_TEXTURE_SUPPORT
+ let sample_depth = 0.0;
+ #else
+ #ifdef MULTISAMPLED
+ let sample_depth = textureLoad(depth, sample_coords, i32(sample_index));
+ #else
+ let sample_depth = textureSample(depth, texture_sampler, sample_uv);
+ #endif
+ #endif
+
+ var weight = 1.0;
+ let is_sample_in_fg = !(depth_supported && sample_depth < this_depth && sample_depth > 0.0);
+ if is_sample_in_fg {
+ // The following weight calculation is used to eliminate ghosting artifacts that are
+ // common in motion-vector-based motion blur implementations. While some resources
+ // recommend using depth, I've found that sampling the velocity results in significantly
+ // better results. Unlike a depth heuristic, this is not scale dependent.
+ //
+ // The most distracting artifacts occur when a stationary foreground object is
+ // incorrectly sampled while blurring a moving background object, causing the stationary
+ // object to blur when it should be sharp ("background bleeding"). This is most obvious
+ // when the camera is tracking a fast moving object. The tracked object should be sharp,
+ // and should not bleed into the motion blurred background.
+ //
+ // To attenuate these incorrect samples, we compare the motion of the fragment being
+ // blurred to the UV being sampled, to answer the question "is it possible that this
+ // sample was occluding the fragment?"
+ //
+ // Note to future maintainers: proceed with caution when making any changes here, and
+ // ensure you check all occlusion/disocclusion scenarios and fullscreen camera rotation
+ // blur for regressions.
+ let frag_speed = length(step_vector);
+ let sample_speed = length(sample_motion) / 2.0; // Halved because the sample is centered
+ let cos_angle = dot(step_vector, sample_motion) / (frag_speed * sample_speed * 2.0);
+ let motion_similarity = clamp(abs(cos_angle), 0.0, 1.0);
+ if sample_speed * motion_similarity < frag_speed {
+ // Project the sample's motion onto the frag's motion vector. If the sample did not
+ // cover enough distance to reach the original frag, there is no way it could have
+ // influenced this frag at all, and should be discarded.
+ weight = 0.0;
+ }
+ }
+ weight_total += weight;
+ accumulator += weight * sample_color;
+ }
+
+ let has_moved_less_than_a_pixel = 
+ dot(this_motion_vector * texture_size, this_motion_vector * texture_size) < 1.0;
+ // In case no samples were accepted, fall back to base color.
+ // We also fall back if motion is small, to not break antialiasing.
+ if weight_total <= 0.0 || has_moved_less_than_a_pixel {
+ accumulator = base_color;
+ weight_total = 1.0;
+ }
+ return accumulator / weight_total;
+}