Denoise in RTRT | Haolan Xu

Setup

Operating & compiling environment:

Visual studio 2022 (Windows 11)

Scripts:

Use build.bat/build.sh to build the project
image2video used to transfer the output from pictures to videos (Need ffmpeg)

Code

Github repository

Joint Bilateral Filter

For denoising, we can filter out the noisy signal by various filters.

The simplest one is Gaussian filter. For the final color of a target pixel, Gaussian filter will consider the color contribution from all adjacent pixels, and the weight of the contribution decreases with increasing distance from the target pixel, which ultimately achieves the purpose of removing the high frequency signal. However, this kind of filtering will also blur the boundary of the object, and we usually want to remove only the noise and maintain the object boundary sharp.

Bilateral filter is a good solution for this problem. Since there are often drastic changes about color in two sides of the object boundary, the bilateral filter takes into account the difference (distance) between colors, i.e., if the larger the difference between the color of the two pixels, the smaller the contribution in order to retain the edge information.

Joint bilateral filter, on the other hand, takes into account more reference information, such as depth, normal, and world coordinates to better guide the filtering operation.

In the following picture, A and B need considering depth; B and C need considering normal; D and E need considering color.

Filter kernel:

In the above equations, \(\widetilde{C}\) is the noisy input image, \(D_{normal}\) is the angle between the normals of two points (for normal information), \(D_{plane}\) provides a better metric than just simply calculating the difference between two depths (for depth information).

Temporal Accumulation:

Motion vector

The projection equation:

This is the advantage of graphics: mastering the matrices of the entire pipeline for calculating conveniently.

Accumulation:

For clamping, it is first necessary to compute the mean \(\mu\) and variance \(\sigma\) of \(\bar{C}^{(i)}\) in a 7x7 neighborhood, and then to clamp the color of the previous frame \(C^{i-1}\) into the range \((\mu-k\sigma,\mu+k\sigma)\).