SmoothD2

Abstract
Author	jconklin
Version	1.0.a3
Download	[x86]: SmoothD2-a3.zip [x64]: SmoothD2-a3_x64.zip
Category	Deblocker
License	GPLv2
Discussion	Doom9 Thread

Description

SmoothD2 is a deblocker. The purpose of all deblockers is to remove block encoding artifacts. The primary ones are blockiness and ringing caused by high frequency transients, “large brightness change”, at block boundaries. The trick of course is to only to remove the transients that are introduced by the encoding process, and not a line in the original image that happens to have the misfortune of being on a block boundary.

SmoothD2 reduces the high frequency transients at the block boundaries by doing the following. For each 8x8 block in an image SmoothD2 samples various 8x8 blocks shifted but still overlapping the current block. This is done by shifting the whole image. Then for each shifted image every 8x8 block is run through a fdct-quantize-dequantize-idct process that removes high frequency transients. Then the shifted image is shifted back to its original position and averaged with all of the other shifted, processed, back shifted, images to produce the deblocked image. Ideally this process reduces the high frequency transients at the block boundaries to stay within the maximum high frequency of the original image. Please see “Enhancement of JPEG-Compressed Images by Re-application of JPEG.pdf” for more information. Quant controls the overall strength of the high frequency transients removal. Matrix provides some control over the thickness and direction (horizontal vertical or diagonal lines) of the transient. See official documentation for full details.

Requirements

• [x86]: AviSynth+ or AviSynth 2.5.8 or greater
• [x64]: AviSynth+
• Progressive input only
• Supported color formats: YV12

Syntax and Parameters

Getting Started: First all parameters default to reasonable values so SmoothD2() should work, though it will provide only very weak deblocking. The most important parameter is quant, which is the primary control of deblocking strength. If you can get the amount of deblocking needed then try using the Zero Weight parameters to recover some detail. If you could not get as much deblocking as desired then try using a different Matrix. If you still can't get enough deblocking then try a different Qtype.

SmoothD2 (clip clip, clip "ZWmask", int "quant", int "num_shift", int "Matrix", int "Qtype", int "ZW", int "ZWce", int "ZWlmDark", int "ZWlmBright", int "ncpu")

clip   =

Input clip.

clip  ZWmask = Null

Zero Weight mask. This clip specifies the zero weight value to be applied to each pixel in the source clip. Any pixel values greater than 16 will be clipped to 16.

int  quant = 3

This parameter specifies the quantizer value used (1 to 31). Larger values mean stronger filtering.

int  num_shift = 3

Specifies the number of shifted versions used for filtering providing a speed vs. quality trade-off. A larger value is more accurate and time consuming.

• 1 : Uses 4 shifted versions.
• 2 : Uses 8 shifted versions.
• 3 : Uses 16 shifted versions. Best speed vs. Quality
• 4 : Uses 64 shifted versions

If using num_shift = 1 there are not enough samples to eliminate "beat" artifacts when using Qtypes > 1. "Beat" artifacts may be visible for Qtype=2 with quant>16 or Qtype=4 with quant>9. The following table gives the quant values that may produce "beat" artifacts for Qtype=3 for each matrix.

Matrix	Quant
2	22
3	16
5	22
6	24
7	10
8	10

int  Matrix = 3

Selects a prebuilt matrix to use (1 to 110). 1 thru 9 are either "standard" matrices that I found on the net or slightly modified versions of them. They are ordered according to their deblocking strength from weak to strong. They are the only ones with calibrated strengths which allows the ZW arguments to produce consistent results. Note Matrix is not used by used by Qtype’s 2 and 4 which use the built in h263 matrices.

For more information look at the file doMatrix.c which defines all of the matricies. Currently there are 110 matrices defined. They are almost all useless but are fun to play with. I built them to see how matrices worked. For an excellent description of what a matrix does see https://forum.doom9.org/showthread.php?t=54147

int  Qtype = 1

Quant Type. Selects the type of quant-dequant processing that is used in the filtering. They are ordered by their deblocking strength from least strong to strongest. h263 intra and h263 inter, Qtype’s 2 and 4, ignore the matrix argument and use their own built in matrix.

• 1 : mpeg intra
• 2 : h263 intra
• 3 : mpeg inter
• 4 : h263 inter

The intra in Qtypes 1 and 2 referes to the encoding done for I-Frames. The inter in Qtypes 3 and 4 refers to the encoding done for P-frames and B-Frames. Different matrices are usually used when encoding I-Frames then when encoding P-Frames and B-Frames. All the information about how a particular frame was originally encoded, I-Frame, P-Frame, B-Frame, and Matrix value, has been lost by the time SmoothD2 is passed a frame to process, SmoothD2 does not do automatic intra-inter Qtype or matrix switching.

Zero Weight Arguments:

The Zero Weight arguments can be used to control the strength of filtering. The strength can be specified for the frame as a whole, by the distance from a macroblock edge, and at the individual pixel level. The strength can also be specified based upon the brightness of the pixel.

The ZW arguments control the filtering strength by changing the number of original unfiltered images added to the shifted and filtered images before they are averaged to produce the output image. Obviously the more unfiltered images that are added the less filtering will occur. Internally the amount of filtering is normalized so that the lowest value for an argument produces full filtering strength and the highest value produces virtually no filtering of the original image. For ZW values 1-16 this attempts to be independent of the quant, num_shift, Matrix (for Matrix values less then 10), and Qtype values. Internally, at each pixel location, each argument will produce a normalized value. The 4 normalized values are added and the result is clipped to the value that will produce almost zero filtering.

int  ZW = 1

Zero Weight (0 to 16). Specifies the relative weight of the non-shifted original image to the weight of the shifted and processed images. A higher value counteracts filtering but keeps more detail. This operates on the frame as a whole. Note that ZW values 1-16 are normalized so that the perceived effect of the reduction of filtering due to ZWremains constant when changes are made to quant, num_shift, Matrix (for Matrix values less then 10), and Qtype values. ZW=0 does not include the non-shifted original image in the output image and will produce the strongest filtering.

int  ZWce = 1

Zero Weight Center to Edge (1 to 20). This argument allows the center of all 8x8 macroblocks to have less filtering then the edge of the macroblock. The edge of the macroblock always receives the amount of filtering determined by the other arguments. This allows for less detail loss while maintaining strong deblocking. ZWce specifies the entry in a table which is used to determine the the processing strength from the center of a macroblock to its edge.

There are 4 possible values that VP1, VP2 and VP3 can have.

• 0 - Full filtering strength
• 1 - Medium filtering strength
• 2 - Low filtering strength
• 3 - Almost zero filtering

The values ZWce can take start from 1 which sets values VP1, VP2 and VP3 to 0 which means there is full filtering strength everywhere in the macroblock. As the values get larger the filtering gets stronger from the center outward. When ZWce equals 5 VP1, VP2 are 0 and VP3 is 4 which means the the center 4 pixels have no filtering applied (the source pixel values are copied to the output frame) and the rest of the pixels in the block are filtered at full strength. This will usually produce obvious artifacts.

ZWce	VP1	VP2	VP3
1	0	0	0
2	0	0	1
3	0	0	2
4	0	0	3
5	0	1	1
6	0	1	2
7	0	1	3
8	0	2	2
9	0	2	3
10	0	3	3
11	1	1	1
12	1	1	2
13	1	1	3
14	1	2	2
15	1	2	3
16	1	3	3
17	2	2	2
18	2	2	3
19	2	3	3
20	3	3	3

int  ZWlmDark = 0

Zero Weight Luma Dark Value (0 to 255). The filter strength is reduced for any pixel value starting at ZWlmDark and reaching 0 filtering strength as the pixel value approaches 0. The filtering strength is reduced less at block boundaries. Values between 32-64 are a good place to start. This argument allows more detail to be retained in dark areas which might otherwise appear over smoothed. Obviously when setting this value it is important that the monitor you are working on displays dark grey to black with reasonable fidelity.

int  ZWlmBright = 255

Zero Weight Luma Bright Value (0 to 255). Any source pixel value that is ≥ ZWlmBright will be filtered at the value specified by the ZW argument. This can be used to reduce bright spots that ZWce might otherwise let through. Values > 192 are a good place to start.

int  ncpu = 1

Number of cpu cores to use (1 to 4). The multithreading code path is completely bypassed if only 1 cpu is used.

Examples

SmoothD2 with default settings:

MPEG2Source("Blocky.dv2")
SmoothD2(quant3=, num_shift=3, Matrix=3, Qtype=1, ZW=1, ZWce=1, ZWlmDark=0, ZWlmBright=255, ncpu=1)

More examples here: https://sites.google.com/site/jconklin754smoothd2/examples

Archived Downloads

Version	Download	Mirror
v1.0.a3	[x86]: SmoothD2-a3.zip [x64]: SmoothD2-a3_x64.zip	[x86]: SmoothD2-a3.zip [x64]: SmoothD2-a3_x64.zip

• x64 version compiled with Intel Parallel Studio XE 2015 Composer Edition for C++

External Links

• Google Sites - Official SmoothD2 homepage.
• Doom9 Forum - SmoothD2 discussion.

---

Back to External Filters ←

---