RemoveDirt

From Avisynth wiki
(Difference between revisions)
Jump to: navigation, search
(RestoreMotionBlocks)
m (Syntax and Parameters)
Line 82: Line 82:
 
The third phase, called the postprocessing phase, starts with restoring the phase 2 motion blocks by copying them from the clip "restore" to the clip "filtered". All phase 2 motion blocks become also phase 3 motion blocks. Then the edges between motion and non-motion blocks are inspected. To this end the SAD of the two adjacent border line segments is calculated twice (these line segments are either horizontal or vertical and are 8 pixels long). It is calculated first in the clip "restore" and then in the clip "filtered". In the clip "filtered" the two blocks are from two different sources, one block, the motion block was restored from the clip "restore" and the non-motion block is from the original clip "filtered". Since the frames of the clip "restore" are not changed at all, both blocks are from the same source and should therefore fit together. If the edge SAD in the clip "filtered" > (edge SAD in the clip restore) + pthreshold, then the block is marked as a new (additional) phase 3 motion block and the block is restored by copying it from "restore" to "filtered", because the two blocks in "filtered" don't fit together well enough compared with the two blocks in "restore". In other words,  in this phase, it is checked whether a restored block fits to the yet unrestored blocks. If it doesn't, the not yet restored blocks, which do not fit well, are marked as phase 3 motion blocks and are restored as well. This procedure is repeated until there are no more blocks, which can be tested. If the value of the grey variable is false, then the same is done for the luma and the chroma (for the chroma the variable cthreshold is used instead of pthreshold). If grey= true, then postprocessing is only done for the luma.
 
The third phase, called the postprocessing phase, starts with restoring the phase 2 motion blocks by copying them from the clip "restore" to the clip "filtered". All phase 2 motion blocks become also phase 3 motion blocks. Then the edges between motion and non-motion blocks are inspected. To this end the SAD of the two adjacent border line segments is calculated twice (these line segments are either horizontal or vertical and are 8 pixels long). It is calculated first in the clip "restore" and then in the clip "filtered". In the clip "filtered" the two blocks are from two different sources, one block, the motion block was restored from the clip "restore" and the non-motion block is from the original clip "filtered". Since the frames of the clip "restore" are not changed at all, both blocks are from the same source and should therefore fit together. If the edge SAD in the clip "filtered" > (edge SAD in the clip restore) + pthreshold, then the block is marked as a new (additional) phase 3 motion block and the block is restored by copying it from "restore" to "filtered", because the two blocks in "filtered" don't fit together well enough compared with the two blocks in "restore". In other words,  in this phase, it is checked whether a restored block fits to the yet unrestored blocks. If it doesn't, the not yet restored blocks, which do not fit well, are marked as phase 3 motion blocks and are restored as well. This procedure is repeated until there are no more blocks, which can be tested. If the value of the grey variable is false, then the same is done for the luma and the chroma (for the chroma the variable cthreshold is used instead of pthreshold). If grey= true, then postprocessing is only done for the luma.
 
Finally, if the percentage of all phase 3 motion blocks with respect to all blocks exceeds the value of gmthreshold, then the "filtered" frame is discarded and replaced by the corresponding frame in "alternative".  In this way, we can give special treatments for sharp scene switches and scenes with a moving or zooming camera.
 
Finally, if the percentage of all phase 3 motion blocks with respect to all blocks exceeds the value of gmthreshold, then the "filtered" frame is discarded and replaced by the corresponding frame in "alternative".  In this way, we can give special treatments for sharp scene switches and scenes with a moving or zooming camera.
<br>
 
<br>
 
====Black&White Clips====
 
If grey=true the chroma of the clip "filtered" is not touched by RestoreMotionBlocks. Also for postprocessing only the luma is used. This is slightly faster than grey=false. If you use grey=false for b&w clips, then it not only takes longer but also the quality may degrade, because chroma noise may trigger false postprocessing. Thus "grey=true" is highly recommended for b&w clips.
 
 
<br>
 
<br>
 
<br>
 
<br>
Line 166: Line 162:
 
:::Undocumented parameters.
 
:::Undocumented parameters.
 
<br>
 
<br>
 
  
 
== Archived Downloads ==
 
== Archived Downloads ==

Revision as of 15:08, 4 April 2019

Abstract
Author Kassandro, pinterf
Version v0.9.1
Download RemoveDirt-0.9.1.7z
Category Luma Equalization
License GPLv2
Discussion Doom9 Thread, Doom9 Forum (original thread)

Contents

Description

RemoveDirt is a temporal cleaner for AviSynth 2.6 and AviSynth+. It has now become a script function which involves RestoreMotionBlocks and various filters from the RemoveGrain package. The plugin contains 2 filters:

  • RestoreMotionBlocks which is the main function used in the RemoveDirt script.
  • SCSelect is a scence change detention filter meant to be use with RestoreMotionBlocks.



Requirements



Syntax and Parameters

RestoreMotionBlocks

RestoreMotionBlocks (clip filtered, clip restore, clip "neighbour", clip "neighbour2", clip "alternative", bool "planar", bool "show", bool "debug", int "gmthreshold", int "mthreshold", int "noise", int "noisy", int "dist", int "tolerance", int "dmode", int "pthreshold", int "cthreshold", bool "grey")


clip   =
clip   =
clip  neighbour =
clip  neighbour2 =
clip  alternative =
The first five variables are clip variables. All clips must be of the same type (same width, height and color space). The number of frames is the minimum of the length of all these five clips. The first two variables are mandatory and are unnamed. "filtered" is usually an aggressively filtered clip, from which motion artifacts have to removed. If RestoreMotionBlocks identfies an 8x8 block as a motion block, it copies this block from the clip "restore" to the clip "filtered". This is the basic operation of RestoreMotionBlocks. To identify motion blocks RestoreMotionBlocks uses the clip "neighbour". The default value for neighbour is the "restore" clip. However, in the RemoveDirt script "neighbour" is different from "restore".
  • The "neighbour2" is for using RemoveDirt in combination with motion compensation filters like MVTools (see RemoveDirtMC below). Finally, if the number of motion blocks exceeds the percentage specified in the "gmthreshold" variable, then RestoreMotionBlocks simply takes the frame from the clip "alternative". In this way, scene switches or global motion can be handled specifically. The clip "restore" is the default value for "alternative".


bool  planar = false
If you use planar YUY2 then you have to set "planar=true" (false is the default value).


bool  show = false
bool  debug = false
The boolean variables "show" and "debug" are used for debugging (see section Debugging).


int  gmthreshold = 80
The default value for gmthreshold is 80, i.e. if 80% of the blocks are motion blocks, then the frame is taken from "alternative" clip.


int  mthreshold = 160
"mthreshold" is similar as in the old RemoveDirt. However, because we now use the ordinary SAD for block comparison, the values should be somewhat higher, especially if the value of noise is low. The default value for "mthreshold" is 160


int  noise = 0
int  noisy = -1
With the variable "noise" one can specify a noise level, which should be ignored by the motion detection. The default value of "noise" is 0.
The variable "noisy" is used to specify the number of noisy pixels of an 8x8 block, which must be exceeded for a motion block. The default value of "noisy" is -1.
If noisy >= 0 and noise > 0, then the value of "mthreshold" is ignored.


int  dist = 1
int  tolerance = 12
dist and tolerance control basic motion detection. dist=1 and tolerance=12 are the default values. A block B is considered a neighbor of a block A by RestoreMotionBlocks, if both horizontally and vertically both blocks are only dist blocks apart. For dist=0 a block has only one neighbor, the block itself. If dist=1, then a block has 9 neighbors, if it is not located at the boundary. If dist=2, then each inner block has 25 neighbors and if dist= 3 then it has 49 neighbors etc.. Now for a given block RestoreMotionBlocks counts all the neighbor blocks which are marked as motion blocks. If the percentage of motion blocks among all neighbor blocks exceeds the value of tolerance, then the block is not cleaned. Thus in the default case of 9 neighbor blocks and tolerance=12 one motion block is allowed and cleaning will nevertheless be allowed. In particular, a motion block is cleaned, if it has no other motion blocks as neighbors. This is reasonable, because motion rarely occurs on one tiny block alone. On the other, if motion blocks have a certain density then also the neighbors should not be cleaned. This is the idea behind the variables dist and tolerance. A higher value of dist results in less cleaning. The higher the value of tolerance, the more cleaning. If tolerance >=100 then all blocks are cleaned.


int  dmode = 0
  • If dmode= 0, then all the motion neighbour blocks become phase 2 motion blocks. Thus if dmode=0 the number of motion blocks is increased quite a bit.
  • If dmode= 2, then quite the opposite happens: a phase 1 motion block only becomes a phase 2 motion block, if it is also a motion neighbour block. In particular, there are less phase 2 motion blocks than phase1 motion blocks. For instance, if dist=1, tolerance= 2, dmode= 2, then a single phase 1 motion block is discarded if there exists no further phase 1 motion block with a distance less than 1.
  • Dmode=1 is just in the middle between dmode=0 and dmode= 2: the motion neighbour blocks become the phase 2 motion blocks. Thus, if dmode=1, the phase 1 motion blocks are only relevant for detecting motion neighbour blocks. After this task is completed the phase 1 information is discarded.


int  pthreshold = 10
int  cthreshold = pthreshold
For postprocessing RestoreMotionBlocks uses the variable pthreshold and cthreshold. If the total luma difference of the two adjacent border line increases by more than pthreshold, then cleaning of the block is undone by RestoreMotionBlocks. Similarily cleaning is undone, if the chroma difference exceedes cthreshold. The maximal difference of 8 pixels is 8*255. If cthreshold is larger than this value, then chroma postprocessing is disabled. If both, pthreshold and cthreshold, are larger than the maximal value, then postprocessing is completely disabled. pthreshold=10 is the default value and if nothing else is specified cthreshold has the same value as pthreshold. Clearly luma postprocessing is much more important than chroma postprocessing. If cthreshold and especially pthreshold, then rather unpleasant blocky artifacts become visible. These are much more likely in areas with very flat contrast. Of course, there must be motion as well to get such artifacts. If you see these typical blocky artifacts, you should lower the thresholds. Postprocessing should only be disabled if all the cleaned frames are checked for artifacts. By the very nature of the algorithm no postprocessing will occur if all blocks were cleaned. There must be at least one block, which has not been cleaned to trigger postprocessing. The postprocessing algorithm loops through all the blocks as long as it can find blocks to be restored, nevertheless it is quite efficient. It is the basic philosophy of RestoreMotionBlocks, that motion detection needs only detect at least one but not all blocks of a moving object. The rest is then taken care by postprocessing.
Negative values are allowed for pthreshold and cthreshold, but are not very reasonable.


bool  grey = false
If grey=true the chroma of the clip "filtered" is not touched by RestoreMotionBlocks. Also for postprocessing only the luma is used. This is slightly faster than grey=false. If you use grey=false for b&w clips, then it not only takes longer but also the quality may degrade, because chroma noise may trigger false postprocessing. Thus "grey=true" is highly recommended for b&w clips.


How RestoreMotionBlocks works

To use the above variables properly, one has to understand how RestoreMotionBlocks works. It consists of three phases. For the first phase only the clip "neighbour" is used. Each frame is divided into a grid of 8x8 blocks. If n is the number of the current frame, then for each block of this grid RestoreMotionBlocks looks at the luma of this block in neighbour(n-1) and neighbour(n+1). Note that we don't use the frame neighbour(n). There are three comparison methods (the old RemoveDirt has only one).

  1. If noise= 0, then simply the SAD of each block in neighbour(n-1) and neighbour(n+1) is computed. If it is >= mthreshold, the block is identified as a motion block of frame n. This is the fasted method and a similar method was used in the old RemoveDirt. Its key disadvantage is that it may easily misled by noise.
  2. If noise >=0, then instead of SUM(|y-x|) RestoerMotionBlocks calculates SUM(| |y-x|-noise |). In particular, differences with absolute value <= noise are ignored. If it is >= mthreshold, then this block is identified as a motion block. We call this the noise adjusted SAD. From the way how the noise adjusted SAD is calculated, it is clear, that "mthreshold" should be decreased if "noise" is increased.
  3. If noise >= 0 and noisy >= 0, then RestoreMotionblocks counts the number of pixels of a block, for which the absolute difference between neighbour(n-1) and neighbour(n+1) is >= noise. If this number is >= value of "noisy", then the block is identified as a motion block. We call this the NPC (= noisy pixel counting) method. The value of mthreshold is ignored, if NPC is selected.

Note that a block has 64 pixels. Thus, if noisy > 64, then there can't be any motion blocks. In my view NPC is clearly the best method. It has likely about half the speed of SAD and about the same speed as NSAD. Noise=-1 and noisy=-1 are the default values. Thus SAD is the default method for the first phase. I ran most of my RemoveDirt tests with noise=8 or 10 and noisy= 12.

In the sequel the motion blocks found in the first phase are called phase 1 motion blocks. In the second phase, for each block all the motion blocks found in the first phase which have a distance <= dist are counted. If the result is >= (tolerance /100) * (the number of all first phase blocks with distance <= dist) , then this block is called a motion neighbour block. For instance, if dist = 1 and tolerance= 12 (the default values), then there are 9 blocks with a distance <= 1. Since 1 < (12/100)*9 < 2, there must be at least 2 phase 1 motion blocks among the 9 neighbour blocks such that the block is marked as a motion neighbour block. If dmode= 0, then all the motion neighbour blocks become phase 2 motion blocks. Thus if dmode=0 the number of motion blocks is increased quite a bit. If dmode= 2, then quite the opposite happens: a phase 1 motion block only becomes a phase 2 motion block, if it is also a motion neighbour block. In particular, there are less phase 2 motion blocks than phase1 motion blocks. For instance, if dist=1, tolerance= 2, dmode= 2, then a single phase 1 motion block is dicarded if there exists no further phase 1 motion block with a distance less than 1. Dmode=1 is just in the middle between dmode=0 and dmode= 2: the motion neighbour blocks become the phase 2 motion blocks. Thus, if dmode=1, the phase 1 motion blocks are only relevant for detecting motion neighbour blocks. After this task is completed the phase 1 information is discarded. If dist=0 or dmode=2 gmthreshold should be lowered to 60 or even 50.

The third phase, called the postprocessing phase, starts with restoring the phase 2 motion blocks by copying them from the clip "restore" to the clip "filtered". All phase 2 motion blocks become also phase 3 motion blocks. Then the edges between motion and non-motion blocks are inspected. To this end the SAD of the two adjacent border line segments is calculated twice (these line segments are either horizontal or vertical and are 8 pixels long). It is calculated first in the clip "restore" and then in the clip "filtered". In the clip "filtered" the two blocks are from two different sources, one block, the motion block was restored from the clip "restore" and the non-motion block is from the original clip "filtered". Since the frames of the clip "restore" are not changed at all, both blocks are from the same source and should therefore fit together. If the edge SAD in the clip "filtered" > (edge SAD in the clip restore) + pthreshold, then the block is marked as a new (additional) phase 3 motion block and the block is restored by copying it from "restore" to "filtered", because the two blocks in "filtered" don't fit together well enough compared with the two blocks in "restore". In other words, in this phase, it is checked whether a restored block fits to the yet unrestored blocks. If it doesn't, the not yet restored blocks, which do not fit well, are marked as phase 3 motion blocks and are restored as well. This procedure is repeated until there are no more blocks, which can be tested. If the value of the grey variable is false, then the same is done for the luma and the chroma (for the chroma the variable cthreshold is used instead of pthreshold). If grey= true, then postprocessing is only done for the luma. Finally, if the percentage of all phase 3 motion blocks with respect to all blocks exceeds the value of gmthreshold, then the "filtered" frame is discarded and replaced by the corresponding frame in "alternative". In this way, we can give special treatments for sharp scene switches and scenes with a moving or zooming camera.

Debugging

The boolean variable debug and show are used for debugging. If show=true, then the blocks, which are marked as motion blocks in the first phase are colored red, those found in the second phase are colored green and finally the motion blocks marked by postprocessing are colored blue. In this way, one can easily check whether the above variables were selected appropriately. if debug=true, then RestoreMotionBlocks sends output of the following kind to the debugview utility:

[348] [21495] RemoveDirt: motion blocks =  942(14%), 1652(25%),  635( 9%), loops = 31
[348] [21496] RemoveDirt: motion blocks = 1745(26%), 2330(35%),   64( 0%), loops = 3
[348] [21497] RemoveDirt: motion blocks = 1480(22%), 1973(30%),   45( 0%), loops = 4
[348] [21498] RemoveDirt: motion blocks = 1081(16%), 1915(29%),   65( 1%), loops = 2
[348] [21499] RemoveDirt: motion blocks = 1403(21%), 2380(36%),  235( 3%), loops = 10
[348] [21500] RemoveDirt: motion blocks = 2618(40%), 2204(34%),   59( 0%), loops = 5
[348] [21501] RemoveDirt: motion blocks =  986(15%), 2065(31%),   75( 1%), loops = 3
[348] [21502] RemoveDirt: motion blocks = 1214(18%), 2291(35%),   78( 1%), loops = 3
[348] [21503] RemoveDirt: motion blocks = 1348(20%), 2179(33%),   57( 0%), loops = 4
[348] [21504] RemoveDirt: motion blocks =  961(14%), 1957(30%),   71( 1%), loops = 3
[348] [21505] RemoveDirt: motion blocks = 1833(28%), 2201(33%),   38( 0%), loops = 3
[348] [21506] RemoveDirt: motion blocks = 1644(25%), 2183(33%),   53( 0%), loops = 5
[348] [21507] RemoveDirt: motion blocks = 1420(21%), 2541(39%),  132( 2%), loops = 5
[348] [21508] RemoveDirt: motion blocks = 2238(34%), 2229(34%),  104( 1%), loops = 4
[348] [21509] RemoveDirt: motion blocks = 1351(20%), 2294(35%),  181( 2%), loops = 6
[348] [21510] RemoveDirt: motion blocks =  931(14%), 1800(27%),  229( 3%), loops = 5

The first number in brackets on the left hand side is the id of the process, which runs the script, the second number in brackets is the frame number. The first number (with percentages in brackets) after "motion blocks =" is the number of phase 1 motion blocks, the second is the difference between phase 2 and phase 1 motion blocks (always >=0 if dmode=0, always <= 0 if dmode= 2) and the third is the difference between phase 3 and phase 2 motion blocks (always >= 0). Finally the number after "loops =" is the number of postprocessing loops used for this frame. Debug=true can be used to monitor RestoreMotionBlocks in an encoding process. Of course, show=true can only be used before an encoding process to find the right values for the various variables.


SCSelect

SCSelect is a special filter, which distinguishes between scene begins, scene ends and global motion. The output of SCClense is used as an "alternative" clip for RestoreMotionBlocks. It can hardly used for other purposes, because it can only make proper decisions if there are a lot of motion blocks. Only if the percentage of motion blocks is > gmthreshold, then RestoreMotionBlocks chooses a frame from the clip specified with the alternative variable and then there are always a lot of motion blocks, if gmthreshold is not too small (gmthreshold >= 30 should be sufficiently large). SCSelect yields nonsense results if there are only few motion blocks. SCSelect is used as follows:

SCSelect (clip input, clip scene_begin, clip scene_end, clip global_motion, float "dfactor", bool "debug", bool "planar", int "cache", int "gcache")


clip   =
clip   =
clip   =
clip   =
The first four clip variables are mandatory and have no name. All four clips must have the same color space, width and height. The first clip is the clip, on which SCSelect bases its decision. Usually it should be the same clip, which was specified with the "neighbour" variable in RestoreMotionBlocks. If SCSelect realises a scene begin, it selects its output frame from the clip scene_begin. If SCSelect realises a scene end, it selects its output frame from the clip scene_end. If SCSelect realises a global motion, it selects its output frame from the clip global motion. Thus SCSelect doesn't produce any new frames. It only makes a selection from three different sources.


float  dfactor = 4.0
Dfactor is the key variable for scene switch sensitivity. The higher dfactor the less scene begins and scene ends and the more global motion frames are detected. Dfactor=4.0 is the default value.


bool  debug = false
If debug=true, then SCSelect sends output of the following type to the DebugView utility:
[3416] [67865] SCSelect: global motion
[3416] [67866] SCSelect: global motion
[3416] [67870] SCSelect: global motion
[3416] [67871] SCSelect: global motion
[3416] [67873] SCSelect: global motion
[3416] [67874] SCSelect: global motion
[3416] [67877] SCSelect: global motion
[3416] [68318] SCSelect: global motion
[3416] [68319] SCSelect: global motion
[3416] [68557] SCSelect: scene end
[3416] [68558] SCSelect: scene begin
[3416] [69481] SCSelect: scene end
[3416] [69482] SCSelect: scene begin
[3416] [70240] SCSelect: scene end
[3416] [70241] SCSelect: scene begin
[3416] [70406] SCSelect: global motion
[3416] [70407] SCSelect: global motion
[3416] [70408] SCSelect: global motion
[3416] [70409] SCSelect: global motion
[3416] [70410] SCSelect: global motion
[3416] [72032] SCSelect: global motion
[3416] [72164] SCSelect: global motion
[3416] [72165] SCSelect: global motion		
To describe the basic idea behind SCSelect let SAD(n) be the SAD difference between the frames input(n) and input(n+1). Now, if SAD(n) > dfactor * SAD(n-1), then SCSelect recognizes a scene end and pulls the frame from the clip scene_end. If SAD(n-1) > dfactor * SAD(n), then SCSelect recognizes a scene begin and pulls the frame from the clip scene_begin. If both SAD(n) <= dfactor * SAD(n-1) and SAD(n-1) <= dfactor * SAD(n), then SCSelect recognizes a global motion and pulls the frame from the clip global_motion. From this description it is clear that dfactor must be > 1 for getting reasonable results. The above algorithm is optimized such that often only one and not two SADs are calculated for one requested frame. However, there are certain shortcomings. If a scene ends with global motion, then SCSelect often can't detect the scene end. If a scene begins with global motion, then SCSelect often can't detect the scene begin. These two effects are usually responsible if lonely scene begins and scene ends are detected by SCSelected, otherwise each scene begin should be preceded by a scene end. By refining the above algorithm we could avoid lonely scene begins and scene ends, but there is one situation, where even such a refinement fails. Namely if the scene ends with global motion and the new scene starts with global motion. Then a sharp scene switch can only be detected reliably with a good motion analysis, which would result in an extreme slow down of the filter.


bool  planar = false
If you use planar YUY2 then you have to set "planar=true" (false is the default value).


int  cache = 2
int  gcache = 0
Undocumented parameters.


Archived Downloads

Version Download Mirror Comments Source code
v0.9 RemoveDirt_0.9.zip RemoveDirt_0.9.zip includes statically and dynamically linked SSE2 binaries compiled with MSVC2010. Dynamically linked binaries requires the Microsoft Visual C++ 2010 Redistributable Package (x86) to be installed. Included with the binaries.
v0.9 ReduceDirt.zip ReduceDirt.zip includes 3 binaries: one statically linked (RemoveDirtS.dll) and two dynamically linked (RemoveDirt.dll, RemoveDirtSSE2.dll).

SSE2 version is recommended but unfortunately it requires the Msvcr71.dll runtime component from the very ancient Microsoft Visual C++ .NET 2003. For this reason this package is considered deprecated!

ReduceDirt-src.zip
v0.6.1 RemoveDirt.zip Old archived documentation: www.removedirt.de.tf


External Links




Back to External Filters

Personal tools