SetFilterMTMode - Revision history

Pinterf: Add avs+ readthedocs link

2024-01-16T14:54:26Z

Add avs+ readthedocs link

Pinterf: /* OnCUDA */ Add content

2021-02-10T15:04:24Z

‎OnCUDA: Add content

Pinterf: /* OnCPU */ Add content

2021-02-10T15:00:12Z

‎OnCPU: Add content

Pinterf: SetFilterMTMode: Add OnCPU (OnCUDA) section

2021-02-10T14:49:34Z

SetFilterMTMode: Add OnCPU (OnCUDA) section

Pinterf: /* Prefetch */ Add text and examples from Neo wiki

2021-02-10T14:33:41Z

‎Prefetch: Add text and examples from Neo wiki

Reel.Deal: AviSynth+: document SetFilterMTMode and Prefetch (WIP, mostly a copy paste from the avs+ page)

2020-06-15T18:45:59Z

AviSynth+: document SetFilterMTMode and Prefetch (WIP, mostly a copy paste from the avs+ page)

New page

<div style="max-width:82em; min-width:42em;" >
{{AvsPlusFilter}}
__TOC__
</div>

<div style="max-width:62em; min-width:42em;" >
By default, AviSynth+ is single-threaded. This page documents how to enable multi-threading.
</div>

=== Syntax and Parameters ===
<div style="max-width:62em; min-width:42em;" >
==== SetFilterMTMode ====
:{{FuncDef|SetFilterMTMode (string filter, int mode, bool "force")}}<br>

::{{Par2||string|}}
:::<code>filter</code>: name of the filter you want to set an MT Mode for. You cannot set the MT mode on script function calls, only on binary filters.
:::<code>"DEFAULT_MT_MODE"</code>, sets the default MT mode for all filters that do not have an MT mode explicitly set. Does not affect for source filters and filters that self-register their own MT mode.
<br>
::{{Par2||int|}}
:::Sets MT Mode, there are three basic MT modes (1,2,3) and an experimental workaround mode (4). Instead of the numbers, you can also use symbolic names for MT modes:
:::*1 : MT_NICE_FILTER
:::*2 : MT_MULTI_INSTANCE
:::*3 : MT_SERIALIZED
:::*4 : MT_SPECIAL_MT
<br>
::{{Par2|force|bool|false}}
:::Force MT mode. Default is false
<br>
==== Prefetch ====
:{{FuncDef|Prefetch (clip, int “threads”, int “frames”)}}
<br>
:{{Par2||clip|}}
::Input clip.
<br>
::{{Par2|threads|int|}}
:::Number of threads to use. If it is 0, it passes without doing anything.
<br>
::{{Par2|frames|int|}}
:::Number of frames to prefetch. Again, if it is 0, it passes without doing anything.
</div>
<br>
=== Usage ===
<div style="max-width:62em; min-width:42em;">
*Source: [http://forum.doom9.org/showthread.php?p=1666371#post1666371 Doom9 Forum]<br>
So, how to use MT in AviSynth+?<br>
By default, your script will run in single-threaded mode, just like with SEt's build. Also, just like in SEt's build, you'll have to make sure that filters use the correct MT mode, or else they might wreak havoc. There are three basic MT modes (1,2,3) and an experimental workaround mode (4) since r2440, and modes 1-3 are the same modes as in (yeah you guessed correctly) SEt's build. Which means you can use the same modes that you have used with AviSynth-MT.
There are some things though that are different and/or new in AviSynth+. The first difference is *how* you set the MT mode. In AviSynth-MT, you had to use SetMTMode(X), which caused all filters following that line to use mode X (until the next call to SetMTMode()). This meant if you needed to use multiple MT modes, you had to insert all those calls in the middle of your script, littered over many places.
</div>
<br>
<div style="max-width:62em; min-width:42em;" >
=== Setting MT Modes ===
AviSynth+ does it differently. In AviSynth+, you specify the MT-mode for only specific filters, and those filters will then automatically use their own mode, even if there were other MT-modes in between. This means you can specify all the MT modes at the beginning without polluting your script. You can even make a SetMTMode.avsi if you wish and let it autoload for all of your scripts, or import() it from their top. This is much cleaner, and it allows you to maintain all your MT-modes centrally at a single place. To make this distinction clear from AviSynth+, SetMTMode() is called SetFilterMTMode() in AviSynth+.
</div>
<br>
<div style="max-width:62em; min-width:42em;" >
=== Enabling MT ===
The other difference is how you actually enable multithreading. Calling SetFilterMTMode() is not enough, it sets the MT mode, but the MT mode only has an effect if MT is enabled at all. Note this means you can safely include/import/autoload your SetFilterMTMode() calls in even single-threaded scripts, and they will not be messed up. Uhm, onto the point: You enable MT by placing a single call to Prefetch(X) at the '''end''' of your script, where X is the number of threads to use. If there is a return statement in your script it must be placed '''after''' Prefetch().
</div>
<br>
<div style="max-width:76em; min-width:42em;" >
==== Example ====
<pre>
# This line causes all filters that don't have an MT mode explicitly use mode 2 by default.
# Mode 2 is a relatively safe choice until you don't know most of your calls to be either mode 1 or 3.
# Compared with mode 1, mode 2 trades memory for MT-safety, but only a select few filters will work with mode 1.
SetFilterMTMode("DEFAULT_MT_MODE", 2)
or
SetFilterMTMode("DEFAULT_MT_MODE", MT_MULTI_INSTANCE)

# FFVideoSource(), like most of all source filters, needs MT mode 3.
# Note: starting with AviSynth+ r2069, it will now automatically recognize source filters.
# If it sees a source filter which has no MT-mode specified at all, it will automatically use
# mode 3 instead of the default MT mode.
SetFilterMTMode("FFVideoSource", 3)
or
SetFilterMTMode("FFVideoSource", MT_SERIALIZED)

# Now comes your script as usual
FFVideoSource(...)
Trim(...)
QTGMC(...)
...

# Enable MT!
Prefetch(4)
</pre>
</div>
<br>
<div style="max-width:62em; min-width:42em;" >
=== Help filling MT modes ===
The following script contains MT modes for various plugins, save it as <tt>mtmodes.avsi</tt> and place in your auto-load folder. The script is a work-in-progess, there's still lots of plugins that need to be tested and validated. When the script is finalized, the only thing the user will have to write in his script is the <code>Prefetch</code> call, all <code>SetFilterMtMode</code> calls will be hidden in a single <tt>.avsi</tt> script.
*You can find the latest revision here: [http://publishwith.me/ep/pad/view/ro.rDkwcdWn4k9/latest AviSynth+ MT modes], if you like to contribute please do so [http://publishwith.me/ooiV92hupl here]. Report any issues [https://forum.doom9.org/showthread.php?t=181351 here].

Note for filter writers: filters can report their mt modes in their SetCacheHint, in this case the manual setting is not necessary.
</div>
<br>
<div style="max-width:62em; min-width:42em;" >
=== Choosing the correct MT mode ===
Please do check if the actual output is correct. Fast but corrupted output is useless. Easy way of checking would be using something like <code>[[ColorBars]](1920, 1080, "YV12").[[AddGrainC]](10000, 10000, seed=1)</code> as a source filter. It doesn't always work right but will do for most stuff.[http://forum.doom9.org/showthread.php?p=1667434#post1667434]

Source: http://forum.doom9.org/showthread.php?p=1667439#post1667439

*<code>MT_NICE_FILTER</code>: Some filters (like nnedi3) use some buffers to do their dirty work and with mode 1 you get multiple threads writing data from different frames to the same buffer. This causes corruption when later someone tries to read from this buffer and gets not what was expected. Most of the "more complicated" filters use some kind of temporary storage thus won't work well with this mode. Simple filters might.

*<code>MT_MULTI_INSTANCE</code>: Mode 2 doesn't have this issue because multiple threads will get their own buffers and no data will be shared. Hence mode 2 is the "default" mode which should work with ''most'' filters, but it wastes memory like crazy (take SangNom2 for example - for 1080p YV12 frame, size of temporary buffers is about 10MB, so with 4 threads you get 40MBs on single filter invocation. Now add some usual supersampling to this and multiple invocations in most aa scripts and... you get the idea).

*<code>MT_SERIALIZED</code>: If the filter requires sequential access or uses some global storage, then mode 3 is the only way to go. Source filter (filters without clip parameter) are autodetected, they do not need an explicit MT mode setting, they will automatically use MT_SERIALIZED.

*<code>MT_SPECIAL_MT</code>: Experimental. Now use only for MP_Pipeline, the filter is like a source filter (no input clip parameter), internally multithreaded, and suffer heavy performance degradation from any of the three regular mt modes. Really, this is a workaround. Available from AviSynth+ version r2440. Avisynth+ 3.6 has serious mt fixes, maybe this mode is not needed anymore.
</div>
<br>
<div style="max-width:62em; min-width:42em;" >
=== Closing notes (don't skip!) ===
*Remember that MT is only stable as long as you have specified a correct MT mode for all filters.
*Instead of the numbers 1-2-3-4, you can also use symbolic names for MT modes: MT_NICE_FILTER (1), MT_MULTI_INSTANCE (2), MT_SERIALIZED (3), MT_SPECIAL_MT (4)
*Mode 3 (MT_SERIALIZED) is evil. It is necessary for some filters, and it is usually no problem for source filters, but it can literally completely negate all advantages of MT, if such a filter is placed near the end of your script. Let us know if you meet a non-source mode 3 filter, we might be able to do something about it, but in general, avoid such calls if you want performance. (And of course, insert what you have found into here.)
*The new caches will save you a lot of memory in single-threaded scripts, but due to the way they work, they will also use more memory than before with MT enabled. The memory usage will scale much closer with the number of threads you have. Just something to keep in mind.
*MT-enabled AviSynth+ triggers a latent bug in AvsPmod v2.5.1. [https://github.com/gispos/AvsPmod/releases Use AvsPmod v2.6.x].
*Using too many threads can easily hurt performance a lot, because there are other bottlenecks too in your PC than just the CPU. For example, if you have a quad-core machine with 8 logical cores, less than 8 threads will often work much better than 8 or more.
</div>
<br>
=== Changes ===
<div style="max-width:62em; min-width:42em;" >
{|border=1 cellspacing=1 cellpadding=4
|-
| xxxxx
| xxxxx
|}
</div>

[[Category:Internal_filters]]
[[Category:Multithreading]]
[[Category:Avisynthplus]]

← Older revision		Revision as of 14:54, 16 January 2024
Line 1:		Line 1:
		+	<div {{BlueBox2\|40\|0\|3px solid purple}} >
		+	{{AvsPlusFullname}}<br>
		+	Up-to-date documentation on multithreading: [https://avisynthplus.readthedocs.io/en/latest/avisynthdoc/syntax/syntax_internal_functions_multithreading_new.html https://avisynthplus.readthedocs.io]
		+	</div>
		+
	<div style="max-width:82em; min-width:42em;" >		<div style="max-width:82em; min-width:42em;" >
	{{AvsPlusFilter}}		{{AvsPlusFilter}}

← Older revision		Revision as of 15:04, 10 February 2021
Line 103:		Line 103:

	==== OnCUDA ====		==== OnCUDA ====
		+	:{{FuncDef\|OnCUDA(clip, int “num_prefetch”, int “device_index”)}}
		+	<br>
		+	::{{Par2\|\|clip\|}}
		+	::This clip is processed by CUDA. In other words, the processing before this is processed by CUDA. A filter that does not support CUDA processing will result in an error. Currently, internal filters are rarely supported, so you can only use external filters that are specially made.
		+	<br>
		+	::{{Par2\|num_prefetch\|int\|}}
		+	:::Same as OnCPU prefetch. Here you specify the number of frames to prefetch. About 2 will give you enough performance. Unlike Prefetch, it has only one thread because it is a prefetch for parallelizing processing on the GPU and CPU.
		+	:::default: 0
		+	<br>
		+	::{{Par2\|device_index\|int\|}}
		+	:::Specifies the GPU to run. If you have only one GPU, you can only use 0. If you have two GPUs, you can specify 0 or 1. There is no limit on the number.
		+	:::default: 0
		+	<br>
		+
		+	Of course, valid only on Avisynth+ built with CUDA option and works if the system has proper device and driver combination.

	=== Usage ===		=== Usage ===

← Older revision		Revision as of 15:00, 10 February 2021
Line 70:		Line 70:

	==== OnCPU ====		==== OnCPU ====
		+	OnCPU/OnCUDA (collectively called OnDevice)
		+
		+	Since 3.6 AvisynthNeo features were backported to Avisynth+.
		+	Such as supporting "devices", like CPU and CUDA (CUDA support needs special build, still experimental after v3.7).
		+	And the data transfer between them.
		+
		+	If all are valid, the chain will be as follows.
		+
		+	Upstream → Upstream cache → Thread → Transfer → Downstream cache → Downstream
		+	→ is the flow of frame data (reverse of GetFrame call direction)
		+
		+	Number of prefetch frames
		+
		+	* 0: Synchronous call without all cache
		+	* 1: Synchronous call, but only transfer is read ahead and executed asynchronously. Downstream cache is enabled.
		+	* 2 or more: Pre-read upstream processing using threads. Both upstream and downstream caches are valid.
		+
		+	The number of upstream threads is fixed at 1 thread when prefetch = 2 or more, and the number of prefetches is fixed at 2.
		+	The downstream look-ahead number is set to the specified prefetch sheet.
		+
		+
		+	:{{FuncDef\|OnCPU(clip, int “num_prefetch”)}}
		+	<br>
		+	::{{Par2\|\|clip\|}}
		+	::This clip is processed by the CPU. In other words, the processing before this is processed by the CPU.
		+	<br>
		+	::{{Par2\|num_prefetch\|int\|}}
		+	:::Here you specify the number of frames to prefetch. About 2 will give you enough performance. Unlike Prefetch, it has only one thread because it is a prefetch for parallelizing processing on the GPU and CPU.
		+	:::default: 0
		+	<br>
		+	If 0 is specified, it will be a synchronous call without using threads.

	==== OnCUDA ====		==== OnCUDA ====

@@ Line 68: / Line 68: @@
    Filtering C
    Prefetch(1,4)
 === Usage ===

@@ Line 37: / Line 37: @@
 ::{{Par2|threads|int|}}
 :::Number of threads to use. If it is 0, it passes without doing anything.
 <br>
 ::{{Par2|frames|int|}}
 :::Number of frames to prefetch. Again, if it is 0, it passes without doing anything.
-</div>
+:::default: threads * 2
 <br>
 === Usage ===
 <div style="max-width:62em; min-width:42em;">