View source for SVPFlow

{{FilterCat5|External_filters|Plugins|Plugins_x64|Adjustment_filters|Frame_Rate_Conversion}}
{{Filter3
|SVP Team and [[MVTools]] authors
|4.3.0.168
|[https://www.svp-team.com/files/gpl/svpflow-4.3.0.168.zip svpflow-4.3.0.168.zip]
|Frame Rate Converters
|
*svpflow1: [https://www.gnu.org/licenses/gpl-2.0.txt GPLv2]
*svpflow2: closed source
|6=[https://forum.doom9.org/showthread.php?t=164554 Doom9 Forum]}}

== Intro ==
'''SVP 3.1''' and later is based on SVPflow Avisynth plugins. 
Additionally, Vapoursynth plugins was developed for the multi-OS '''SVP 4'''.

Motion vectors search plugin (svpflow1) is a deeply refactored and modified version of '''MVTools2''' Avisynth plugin and we're grateful to all of MVTools authors: '''Manao, Fizick, Tsp, TSchniede and SEt''' for developing such a great library. 

'''SVP wouldn't be ever possible without it'''.

We're also providing [[Plugins: MVTools2|modified MVTools 2.5.11]] with GPU-accelerated '''MSmoothFps''' function.

Parts of description were taken from [http://avisynth.org.ru/mvtools/mvtools2.html MVTools2] page so this documentation is available under [http://creativecommons.org/licenses/by-sa/3.0/ CreativeCommons BY-SA 3.0 license].


== The interface ==
=== svpflow1 ===
This is a motion vectors search plugin which is a refactored version of '''MVTools2 2.5''' so this plugin is distributed under terms of the [http://www.gnu.org/licenses/gpl-2.0.html GNU GPL license].

==== SVSuper(source, params_string) ====
Get source clip and prepare special "super" clip with multilevel (hierarchical scaled) frames data. The super clip is used by both '''SVAnalyse''' and '''SVSmoothFps''' functions. For storing and transferring its parameters we use audio properties of super clip (specifically, num_audio_samples) as a trick. So, audio is killed in super clip.

* '''source''' - source video clip
* '''params_string''' - parameters list in [http://www.json.org/ JSON] format, names quotation is optional. All parameters are optional too.

{|border="1" class="wikitable" width="100%"
|-
|width="300px" background-color="#0FF"|'''{'''
|
|-
|
:'''pel''': <font color="red">2</font>,
|The accuracy of the motion estimation. Value can only be 1, 2 or 4. 1 means a precision to the pixel, 2 means a precision to half a pixel, 4 - to quarter pixel (not recommended to use).
|-
|
:'''gpu''': <font color="red">0</font>,
|GPU usage mode: 0 - none, 1 - for frame rendering. Note that with "gpu:1" scaling up mode is always set to 0 cause subpixel planes are not actually used for frame rendering.
|-
|
:'''full''': <font color="red">true</font>,
|Turns on reduced super clip size when full=false, valid only with pel=1. It saves some memory and can be useful for processing extra large frames (like UHD (4K)).
|-
|
:'''scale''': '''{'''
|Scaling modes:
|-
|valign="top"|
::'''up''': <font color="red">2</font>,
|Subpixel interpolation method for pel=2,4.
* 0 for soft interpolation (bilinear), 
* 1 for bicubic interpolation (4 tap Catmull-Rom), 
* 2 for sharper Wiener interpolation (6 tap, similar to Lanczos).
|-
|valign="top"|
::'''down''': <font color="red">4</font>
|Hierarchical levels smoothing and reducing (halving) filter.
* 0 is simple 4 pixels averaging like unfiltered SimpleResize (old method);
* 1 is triangle (shifted) filter like ReduceBy2 for more smoothing (decrease aliasing);
* 2 is triangle filter like BilinearResize for even more smoothing;
* 3 is quadratic filter for even more smoothing;
* 4 is cubic filter like BicubicResize(b=1,c=0) for even more smoothing.
|-
|
:'''}'''
|
|-
|
:'''rc''': <font color="red">0</font>,
|Used by the SVP Manager only. You don't need to set this in your own scripts.
|-
|
'''}'''
|
|}


==== SVAnalyse(super, params_string, [src]: clip) ====
Get prepared multilevel super clip, estimate motion by block-matching method and produce special output clip with motion vectors data used by '''SVSmoothFps''' function.
Some hierarchical multi-level search methods are implemented (from coarse image scale to finest).

* '''super''' - multilevel super clip prepared by SVSuper function. You can replace this clip with '''[http://avisynth.org.ru/mvtools/mvtools2.html#functions MSuper]''' clip from original '''MVTools 2.5''' in which case you should define "gpu" parameter here.
* '''params_string''' - parameters list in JSON format.
* '''src''' - source clip, must be defined when using reduced super clip ("super.full"=false).

{|border="1" class="wikitable" width="100%"
|-
|width="300px" background-color="#0FF"|'''{'''
|
|-
|valign="top"|
:'''gpu''': <font color="red">0</font>,
|GPU usage mode: 0 - none, 1 - for frame rendering. Should be used instead of "super.gpu" if and only if '''SVSuper''' is replaced with '''MSuper'''! 
|-
|valign="top"|
:'''vectors''': <font color="red">3</font>,
|Direction of motion vectors to search for. 
* 1 - forward only, from current frame to the following one (not useful at all), 
* 2 - backward only, from following frame to the current one (useful only with "smoothfps.algo: 1"),
* 3 - search both directions.
|-
|
:'''block''': '''{'''
|Defines vectors grid step and block sizes for block matching algorithm.
|-
|valign="top"|
::'''w''': <font color="red">16</font>,
|Size of a block (horizontal). It's either 8, 16 or 32. Larger blocks are less sensitive to noise, are faster, but also less accurate, smaller blocks produce more wavy picture.
|-
|valign="top"|
::'''h''': <font color="red">16</font>,
|Vertical size of a block. Default is equal to horizontal size. Additional options are: 4 for "block.w:8", 8 for "block.w:16", 16 for "block.w:32".
|-
|valign="top"|
::'''overlap''': <font color="red">2</font>
|Block overlap value. 0 - none, 1 - 1/8 of block size in each direction, 2 - 1/4 of block size, 3 - 1/2 of block size. The greater overlap, the more blocks number, and the lesser the processing speed.

Resulting overlap value in pixels should be even with CPU rendering.
|-
|
:'''}''',
|
|-
|
:'''main''': '''{'''
|Defines main search settings.
|-
|valign="top"|
::'''levels''': <font color="red">0</font>,
|Positive value is the number of levels used in the hierarchical analysis made while searching for motion vectors. Negative or zero value is the number of coarse levels NOT used in the hierarchical analysis made while searching for motion vectors.
|-
|
::'''search''': '''{'''
|
|-
|valign="top"|
:::'''type''': <font color="red">4</font>,
|The type of search on finest level:
* 2 - Hexagon search, similar to x264,
* 3 - Uneven Multi Hexagon (UMH) search, similar to x264,
* 4 - Exhaustive search, slowest but it gives the best results.
|-
|valign="top"|
:::'''distance''': <font color="red">-2*pel</font>,
|Search range on finest level:
* 0 - don't search on finest level at all, greatly increase search speed but may still looks good with GPU rendering. This option is opposite to "super.pel".
* >0 - classic fixed range in pixels.
* <0 - "adaptive" range based on block local contrast. Range is small or zero for low contrast blocks (black/gray for example) but is big for blocks that has many visible details. Effective average range in common scenes is about 1/3 of this value.
|-
|valign="top"|
:::'''sort''': <font color="red">true</font>,
|Sort vectors from previous level by SAD values to define the order of blocks scanning so the search begins with better predictors. This option is always ON on coarse levels but may be time consuming on finest one.
|-
|valign="top"|
:::'''satd''': <font color="red">false</font>,
|Use SATD function instead of SAD on '''finest''' level. Extremely slow, do not use it!
|-
|
:::'''coarse''': '''{'''
|The same parameters for coarse levels.
|-
|
::::'''width''': <font color="red">1050</font>,
|Maximum width of a level to be processed with 'coarse' parameters. Can be useful to save CPU power when processing extra large frames (like UHD (4K)).
|-
|
::::'''type''': <font color="red">4</font>,
|Same as "main.search.type".
|-
|
::::'''distance''': <font color="red">0</font>,
|Same as "main.search.type" except zero means "-10".
|-
|valign="top"|
::::'''satd''': <font color="red">true</font>,
|Use SATD function instead of SAD on every coarse level, improves motion vector estimation at luma flicker and fades.
|-
|
::::'''trymany''': <font color="red">false</font>,
|Try to start searches around many predictors.
|-
|
::::'''bad''': '''{'''
|Wide second search for bad vectors.
|-
|
:::::'''sad''': <font color="red">1000</font>,
|SAD threshold to define "bad" vectors. Value is scaled to block size 8x8. 
|-
|valign="top"|
:::::'''range''': <font color="red">-24</font>
|The range of wide search for bad blocks. Use positive value for UMH search and negative for Exhaustive search.
|-
|
::::'''}'''
|
|-
|
:::'''}'''
|
|-
|
::'''}''',
|
|-
|
::'''penalty''': '''{'''
|Main search penalties for motion coherence.
|-
|valign="top"|
:::'''lambda''': <font color="red">10.0</font>,
|Set the coherence of the field of vectors. The higher, the more coherent. However, if set too high, some best motion vectors can be missed.

This value is different from MVTools, see [[#plevel|remark]] for explanations.
|-
|valign="top"|
:::'''plevel''': <font color="red">1.5</font>,
|penalty.lambda scaling mode between levels. 1.0 means no scaling, 2.0 - linear, 4.0 - quadratic dependence from hierarchical level number.

This value is different from MVTools, see [[#plevel|remark]] for explanations.
|-
|valign="top"|
:::'''lsad''': <font color="red">8000</font>,
|SAD limit for lambda using. Local lambda is smoothly decreased if SAD value of vector predictor is greater than the limit. It prevents bad predictors using but decreases the motion coherence. Value is scaled to block size 8x8. 
|-
|valign="top"|
:::'''pnew''': <font color="red">50</font>,
|Relative penalty (scaled to 256) to SAD cost for new candidate vector. New candidate vector must be better will be accepted as new vector only if its SAD with penalty (SAD + SAD*pnew/256) is lower then predictor cost (old SAD). It prevent replacing of quite good predictors by new vector with a little better SAD but different length and direction.
|-
|valign="top"|
:::'''pglobal''': <font color="red">50</font>,
|Relative penalty (scaled to 256) to SAD cost for global predictor vector (lambda is not used for global vector).
|-
|valign="top"|
:::'''pzero''': <font color="red">100</font>,
|Relative penalty (scaled to 256) to SAD cost for zero vector. It prevent replacing of quite good predictor by zero vector with a little better SAD (lambda is not used for zero vector).
|-
|
:::'''pnbour''': <font color="red">50</font>,
|Relative penalty (scaled to 256) to SAD cost for up to 8 neighbours vectors.
|-
|valign="top"|
:::'''prev''': <font color="red">0</font>,
|Relative penalty (scaled to 256) to SAD cost for "reverse" vector (already found vector from reverse search direction), works only with "analyse.vectors: 3".
|-
|
::'''}'''
|
|-
|
:'''}''',
|
|-
|valign="top"|
:'''refine''': '''[ {'''
|Array of structures, each element defines additional level of recalculation, on each level blocks are divided by 4. Replaces MRecalculate function.

Interpolated vectors of old blocks are used as predictors for new vectors, with recalculation of SAD.
|-
|valign="top"|
::'''thsad''': <font color="red">200</font>,
|Only bad quality new vectors with SAD above this threshold will be re-estimated by search. Good vectors are not changed, but its SAD will be updated (re-calculated). Value is scaled to block size 8x8.

Zero means "do not refine, just divide"
|-
|
::'''search''': '''{'''
|Search parameters.
|-
|
:::'''type''': <font color="red">4</font>,
|Same as main.search.type by default.
|-
|
:::'''distance''': <font color="red">pel</font>,
|Same as super.pel value by default.
|-
|
:::'''satd''': <font color="red">false</font>
|Same as main.search.satd by default.
|-
|
::'''}''',
|
|-
|
::'''penalty''': '''{'''
|
|-
|
:::'''pnew''': <font color="red">50</font>
|Same as main.penalty.pnew by default.
|-
|
::'''}'''
|
|-
|
:'''} ]''',
|
|-
|
:'''special''': '''{'''
|Some special parameters not used in SVP
|-
|
::'''delta''': <font color="red">1</font>
|Interval between analysed frames.
|-
|
:'''}'''
|
|-
|
'''}'''
|
|}

<div id="plevel"></div>


==== SVConvert(vectors, isb: bool) ====
Convert SVAnalyse output to the format of MAnalyse compatible with "client" MVTools 2.5 functions.

* '''vectors''' - motion vectors data produced by SVAnalyse function,
* '''isb''' - which vectors to extracts: forward if '''isb=false''', backward if '''isb=true'''.

    super = SVSuper(super_params)
    vectors = SVAnalyse(super, analyse_params)
    
    forward_mv = SVConvert(vectors, false)
    backward_mv = SVConvert(vectors, true)
    
    super_mv = MSuper(pel=1, hpad=0, vpad=0) '''#padding should be zero here!'''
    MFlowFps(super_mv, backward_mv, forward_mv, num=60, den=1)

==== Remark: lambda/plevel values meaning. ====

Let the finest level number is 0 and we've got N levels total => the smallest level number is N-1.<br/>
'''MVTools approach:'''
    <local lambda value at level K> = lambda * (2^(K*plevel)), where lambda is integer and plevel is integer in [0;2]
'''SVPflow approach:'''
    <local lambda value at level K> = lambda * 1000 / (plevel^(N-1-K)), where both lambda and plevel are floats.
Benefits - lambda penalty is now invariant to video frame size.


=== svpflow2 ===
A closed-source frame rendering plugin.

==== SVSmoothFps(source, super, vectors, params_string, [sar]: float, [mt]: integer) ====
Will change the framerate (fps) and number of frames of the source clip. The function can be use for framerate conversion, slow-motion effect, etc. It uses motion vectors found with SVAnalyse function to create interpolated pictures at some intermediate time moments between frames. Some internal masks (cover/uncover, vectors quality) can be used to produce the output image with minimal artifacts.

* '''source''' - source video clip.
* '''super''' - multilevel super clip prepared by SVSuper function.
* '''vectors''' - motion vectors data produced by SVAnalyse function.
* '''params_string''' - parameters list in JSON format.
* '''sar''' - optional, can be used instead of "light.sar" inside ffdshow: 
    SVSmoothFps(..., sar=float(ffdshow_sar_x)/ffdshow_sar_y, ...)
* '''mt''' - optional, workaround for current Avisynth 2.6 MT build ver. 2.6.0.3, should be equal to number of threads defined in SetMTMode():
    threads = 10
    SetMTMode(3,'''threads''')
    #....
    SVSmoothFps(..., mt='''threads''', ...)

{|border="1" class="wikitable" width="100%"
|-
|width="300px" background-color="#0FF"|'''{'''
|
|-
|
:'''rate''': '''{'''
|Target frame rate.
|-
|
::'''num''': <font color="red">2</font>,
|Numerator of multiplier for frame rate change.
|-
|
::'''den''': <font color="red">1</font>,
|Denominator of multiplier for frame rate change.
|-
|
::'''abs''': <font color="red">false</font>
|If '''true''' then num/den define absolute frame rate value instead if multiplier for source frame rate.
|-
|
:'''}''',
|
|-
|valign="top"|
:'''algo''': <font color="red">13</font>,
|Rendering algorithm or "SVP shader", available values are:
* 1 - sharp picture without any blending, moves pixels by motion vectors from next frame to current. Requires only backward motion vectors ("analyse.vectors: 2") so it's the fastest possible method.
* 2 - like 1st but moves pixels from the nearest (in terms of time) frame so it uses both backward and forward vectors. Recommended for 2D animations.
* 11 - time weighted blend of forward and backward partial motion compensations.
* 13 - same as 11th but with dynamic median added. Produces minimum artifacts but with noticeable halos around moving objects.
* 21 - 11th plus additional cover/uncover masking to minimize halos and improve frame edges.
* 23 - 21th plus extra vectors from adjacent frames for further decreasing of halos, can be less smooth than 21th.
|-
|valign="top"|
:'''block''': <font color="red">false</font>,
|Use block-based motion compensation instead of pixel-based. Always OFF with GPU rendering enabled.
|-
|valign="top"|
:'''cubic''': <font color="red">1</font>,
|Only works with GPU rendering enabled:
* 0 - use bilinear interpolation for motion vectors and all masks,
* 1 - use bicubic interpolation
|-
|valign="top"|
:'''gpuid''': <font color="red">0</font>,
|Defines which video card should be used for rendering, only works with GPU rendering enabled:
* 0 - default (use 1st available GPU),
* 11 - use '''1st''' GPU device on '''1st''' OpenCL platfrom,
* 12 - use '''2nd''' GPU device on '''1st''' OpenCL platfrom,
* 21 - use '''1st''' GPU device on '''2nd''' OpenCL platfrom an so on.
|-
|valign="top"|
:'''linear''': <font color="red">true</font>,
|Only works with GPU rendering enabled. When set to "true" frame rendering is done in linear light.
|-
|
:'''mask''': '''{'''
|Masks properties.
|-
|valign="top"|
::'''cover''': <font color="red">100</font>,
|Cover/uncover mask strength, more means "more strong mask". Recommended values 50-100.
|-
|valign="top"|
::'''area''': <font color="red">0</font>,
|Bad areas (identified by vector's SAD values) mask, more means "more strong mask". Recommended value is 100, but it can dramatically reduce smoothness effect.
|-
|
::'''area_sharp''': <font color="red">1.0</font>
|Defines the exponent of relation between SAD and area mask values.
|-
|
:'''}''',
|
|-
|
:'''scene''': '''{'''
|Extended "scene change" controls.
|-
|valign="top"|
::'''mode''': <font color="red">3</font>,
|Frames interpolation mode: 
* 0 - uniform interpolation for maximum smoothness. For example for 24->60 conversion output will be: "1mmmm1mmmm...", where "1" stands for original frame and "m" for interpolated one.
* 1 - "1m" mode that gives "1mm1m1mm1m..." output in the above example => less artifacts at the cost of less smoothness.
* 2 - "2m" mode: "1m11m11m11..." => much less artifacts and much less smoothness.
* 3 - adaptive mode that switches between modes 0,1,2 based on overall vector field quality.
|-
|valign="top"|
::'''blend''': <font color="red">false</font>,
|Blend frames at scene change like ConvertFps if true, or repeat last frame like ChangeFps if false.
|-
|valign="top"|
::'''limits''': '''{'''
|Limits for vector field quality / scene change detection.

For example scene change will be detected if number of blocks with "adjusted SAD" > "limits.scene" will be more than "limits.blocks" percents of all blocks, that has "adjusted SAD" value > "limits.zero", where "adjusted SAD" is "block SAD"/"block average luma".
|-
|
:::'''m1''': <font color="red">1600</font>,
|Limit for changing uniform mode to "1m".
|-
|
:::'''m2''': <font color="red">2800</font>,
|Limit for changing "1m" mode to "2m".
|-
|
:::'''scene''': <font color="red">4000</font>,
|Limit for scene change detection.
|-
|valign="top"|
:::'''zero''': <font color="red">200</font>,
|Vectors with "adjusted SAD" less than this value are excluded from consideration.
|-
|
:::'''blocks''': <font color="red">20</font>
|Threshold which sets how many blocks in percents have to change.
|-
|
::'''}''',
|
|-
|
::'''luma''': <font color="red">1.5</font>
|Additional correction parameter for "average luma" value.
|-
|
:'''}''',
|
|-
|
:'''light''': '''{'''
|"Ambilight"-like black fields lighting.
|-
|valign="top"|
::'''aspect''': <font color="red">0.0</font>,
|Screen aspect ratio defines black fields height (or width) and output video frame size.
|-
|
::'''sar''': <font color="red">1.0</font>,
|Source video pixel aspect ratio.
|-
|
::'''zoom''': <font color="red">0.0</font>,
|"Glow" (or "zoom out") effect size, in percents of original frame size.
|-
|
::'''lights''': <font color="red">16</font>,
|Lights count.
|-
|
::'''length''': <font color="red">100</font>,
|Flare length in percents.
|-
|
::'''cell''': <font color="red">1.0</font>,
|Width of every light.
|-
|
::'''border''': <font color="red">12</font>
|Height of averaging frame border.
|-
|
:'''}'''
|
|-
|
'''}'''
|
|}


==== SVSmoothFps_NVOF(source, params_string, ...) ====
<font color="red">TODO!</font>

The same as SVSmoothFps but the motion vectors are acquired in-place via [https://developer.nvidia.com/opticalflow-sdk NVidia Optical Flow API] so there's no need for SVSuper/SVAnalyse at all.


=== Samples ===
==== Basic Avisynth script ====
    SetMemoryMax(1024)
    LoadPlugin("svpflow1.dll")
    LoadPlugin("svpflow2.dll")
    
    threads=9
    SetMTMode(3,threads)
    # Some input here
    SetMTMode(2)
    ConvertToYV12()
    
    # All parameters set to defaults which means high quality frame doubling
    super=SVSuper("{gpu:1}")
    vectors=SVAnalyse(super, "{}")
    SVSmoothFps(super, vectors, "{}", mt=threads)

==== Basic Vapoursynth script ====
    import vapoursynth as vs
    core = vs.get_core(threads=9)
    
    core.std.LoadPlugin("svpflow1_vs.dll")
    core.std.LoadPlugin("svpflow2_vs.dll")
    
    clip = # need some input here
    clip = clip.resize.Bicubic(format=vs.YUV420P8) #convert to YV12
    
    super  = core.svp1.Super(clip,"{gpu:1}")
    vectors= core.svp1.Analyse(super["clip"],super["data"],clip,"{}")
    smooth = core.svp2.SmoothFps(clip,super["clip"],super["data"],vectors["clip"],vectors["data"],"{}")
    smooth = core.std.AssumeFPS(smooth,fpsnum=smooth.fps_num,fpsden=smooth.fps_den)
    
    smooth.set_output()

All other scripts are the same except for different JSON strings. For example:

==== Maximum smoothness for animation ====
    # Header is exactly same as in previous example
    
    super=SVSuper("'''{gpu:1}'''")
    # Small 8x8 blocks with additional refine to 4x4
    vectors=SVAnalyse(super, "'''{ block:{w:8}, refine:[{thsad:1000}] }'''")
    # Conversion to 5/2 of source frame rate with 2nd SVP-shader.
    SVSmoothFps(super, vectors, "'''{ num:5, den:2, algo:2 }'''", mt=threads)

==== More JSON magic ====
Multi-line JSON string with comments inside it:
    analyse_params="""{block:{w:8,overlap:1},
    	main:
    		{search:
    			{type:2,distance:-2,satd:true,
    			coarse:{trymany:true}
    			}
    		// the following line is commented out in C++-style 
    		//,penalty:{plevel:1.0,lambda:1.0}
    		}
    }"""
    # ...
    vectors=SVAnalyse(super, analyse_params)
    # ...


== Downloads ==
The latest version is: '''4.3.0.168''' (2019-07-28).

The distribution includes:
* svpflow1.dll, svpflow2.dll 32-/64-bit plugins for Avisynth and Vapoursynth on Windows
* svpflow1.so, svpflow2.so 64-bit plugins for Vapoursynth on Linux
* svpflow1.dylib, svpflow2.dylib 64-bit plugins for Vapoursynth on MacOS
* sample scripts
* readme and licenses files

<font color="red">Please note that '''SVPflow libs require a SVP Manager running''' (Windows and macOS only), otherwise you'll see a red rectangle around the video frame.</font>

You're free to disable SVP completely ("Temporarily disable SVP" + ui.disable_on_launch = true in All settings), however it still '''must be running in background'''.

'''Download link is here:''' https://www.svp-team.com/files/gpl/svpflow-4.3.0.168.zip
<br>
<br>
-----------------------------------------------
'''Back to [[External_filters#Frame_Rate_Conversion|External Filters]] &larr;'''