en-US/PSParallelPipeline-help.xml
|
<?xml version="1.0" encoding="utf-8"?>
<helpItems schema="maml" xmlns="http://msh"> <command:command xmlns:maml="http://schemas.microsoft.com/maml/2004/10" xmlns:command="http://schemas.microsoft.com/maml/dev/command/2004/10" xmlns:dev="http://schemas.microsoft.com/maml/dev/2004/10" xmlns:MSHelp="http://msdn.microsoft.com/mshelp"> <command:details> <command:name>Invoke-Parallel</command:name> <command:verb>Invoke</command:verb> <command:noun>Parallel</command:noun> <maml:description> <maml:para>Executes parallel processing of pipeline input objects using multithreading.</maml:para> </maml:description> </command:details> <maml:description> <maml:para>The `Invoke-Parallel` cmdlet enables parallel processing of input objects in PowerShell, including Windows PowerShell 5.1 and PowerShell 7+, offering functionality similar to `ForEach-Object -Parallel` introduced in PowerShell 7.0. It processes pipeline input across multiple threads, improving performance for tasks that benefit from parallel execution.</maml:para> </maml:description> <command:syntax> <command:syntaxItem> <maml:name>Invoke-Parallel</maml:name> <command:parameter required="true" variableLength="true" globbing="false" pipelineInput="False" position="0" aliases="none"> <maml:name>ScriptBlock</maml:name> <maml:description> <maml:para>Defines the script block executed for each input object in parallel.</maml:para> </maml:description> <command:parameterValue required="true" variableLength="false">ScriptBlock</command:parameterValue> <dev:type> <maml:name>ScriptBlock</maml:name> <maml:uri /> </dev:type> <dev:defaultValue>None</dev:defaultValue> </command:parameter> <command:parameter required="false" variableLength="true" globbing="false" pipelineInput="False" position="named" aliases="funcs"> <maml:name>Functions</maml:name> <maml:description> <maml:para>Specifies an array of function names from the local session to include in the runspaces’ Initial Session State (https://learn.microsoft.com/en-us/dotnet/api/system.management.automation.runspaces.initialsessionstate). This enables their use within the parallel script block.</maml:para> <maml:para>> [!TIP] > > This parameter is the recommended way to make local functions available in the parallel scope. Alternatively, you can retrieve the function definition as a string (e.g., `$def = ${function:Greet}.ToString()`) and use `$using:` to pass it into the script block, defining it there (e.g., `${function:Greet} = $using:def`).</maml:para> </maml:description> <command:parameterValue required="true" variableLength="false">String[]</command:parameterValue> <dev:type> <maml:name>String[]</maml:name> <maml:uri /> </dev:type> <dev:defaultValue>None</dev:defaultValue> </command:parameter> <command:parameter required="true" variableLength="true" globbing="false" pipelineInput="True (ByValue)" position="named" aliases="none"> <maml:name>InputObject</maml:name> <maml:description> <maml:para>Specifies the objects to process in the script block. This parameter accepts pipeline input.</maml:para> </maml:description> <command:parameterValue required="true" variableLength="false">Object</command:parameterValue> <dev:type> <maml:name>Object</maml:name> <maml:uri /> </dev:type> <dev:defaultValue>None</dev:defaultValue> </command:parameter> <command:parameter required="false" variableLength="true" globbing="false" pipelineInput="False" position="named" aliases="tl"> <maml:name>ThrottleLimit</maml:name> <maml:description> <maml:para>Sets the maximum number of script blocks executed in parallel across multiple threads. Additional input objects wait until the number of running script blocks falls below this limit. The default value is `5`.</maml:para> </maml:description> <command:parameterValue required="true" variableLength="false">Int32</command:parameterValue> <dev:type> <maml:name>Int32</maml:name> <maml:uri /> </dev:type> <dev:defaultValue>5</dev:defaultValue> </command:parameter> <command:parameter required="false" variableLength="true" globbing="false" pipelineInput="False" position="named" aliases="to"> <maml:name>TimeoutSeconds</maml:name> <maml:description> <maml:para>Specifies the maximum time (in seconds) to process all input objects. When the timeout is reached, running script blocks are terminated, and remaining input is discarded.</maml:para> <maml:para>> [!NOTE] > > A value of `0` (default) disables the timeout, allowing processing to continue until completion.</maml:para> </maml:description> <command:parameterValue required="true" variableLength="false">Int32</command:parameterValue> <dev:type> <maml:name>Int32</maml:name> <maml:uri /> </dev:type> <dev:defaultValue>0</dev:defaultValue> </command:parameter> <command:parameter required="false" variableLength="true" globbing="false" pipelineInput="False" position="named" aliases="unr"> <maml:name>UseNewRunspace</maml:name> <maml:description> <maml:para>Uses a new runspace for each parallel invocation instead of reusing existing runspaces in the runspace pool.</maml:para> </maml:description> <dev:type> <maml:name>SwitchParameter</maml:name> <maml:uri /> </dev:type> <dev:defaultValue>False</dev:defaultValue> </command:parameter> <command:parameter required="false" variableLength="true" globbing="false" pipelineInput="False" position="named" aliases="vars"> <maml:name>Variables</maml:name> <maml:description> <maml:para>Provides a hashtable of variables to make available in the parallel scope. Keys define the variable names within the script block.</maml:para> <maml:para>> [!TIP] > > Use this parameter as an alternative to the `$using:` scope modifier (https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_scopes#scope-modifiers).</maml:para> </maml:description> <command:parameterValue required="true" variableLength="false">Hashtable</command:parameterValue> <dev:type> <maml:name>Hashtable</maml:name> <maml:uri /> </dev:type> <dev:defaultValue>None</dev:defaultValue> </command:parameter> <command:parameter required="false" variableLength="true" globbing="false" pipelineInput="False" position="named" aliases="mn"> <maml:name>ModuleNames</maml:name> <maml:description> <maml:para>Specifies an array of module names to import into the runspaces’ Initial Session State (https://learn.microsoft.com/en-us/dotnet/api/system.management.automation.runspaces.initialsessionstate). This allows the script block to use cmdlets and functions from the specified modules.</maml:para> <maml:para>> [!TIP] > > Use this parameter to ensure required modules are available in the parallel scope. Module names must be discoverable via the `$env:PSModulePath` (https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_psmodulepath)environment variable, which lists installed module locations.</maml:para> </maml:description> <command:parameterValue required="true" variableLength="false">String[]</command:parameterValue> <dev:type> <maml:name>String[]</maml:name> <maml:uri /> </dev:type> <dev:defaultValue>None</dev:defaultValue> </command:parameter> <command:parameter required="false" variableLength="true" globbing="false" pipelineInput="False" position="named" aliases="mp"> <maml:name>ModulePaths</maml:name> <maml:description> <maml:para>Specifies an array of file paths to directories containing PowerShell modules (e.g., `.psm1` or `.psd1` files) to import into the runspaces’ Initial Session State (https://learn.microsoft.com/en-us/dotnet/api/system.management.automation.runspaces.initialsessionstate). This enables the script block to use cmdlets and functions from custom or local modules.</maml:para> <maml:para>> [!NOTE] > > Paths must be absolute or relative to the current working directory and must point to valid directories containing PowerShell modules. If an invalid path (e.g., a file or non-existent directory) is provided, a terminating error is thrown.</maml:para> </maml:description> <command:parameterValue required="true" variableLength="false">String[]</command:parameterValue> <dev:type> <maml:name>String[]</maml:name> <maml:uri /> </dev:type> <dev:defaultValue>None</dev:defaultValue> </command:parameter> </command:syntaxItem> </command:syntax> <command:parameters> <command:parameter required="false" variableLength="true" globbing="false" pipelineInput="False" position="named" aliases="funcs"> <maml:name>Functions</maml:name> <maml:description> <maml:para>Specifies an array of function names from the local session to include in the runspaces’ Initial Session State (https://learn.microsoft.com/en-us/dotnet/api/system.management.automation.runspaces.initialsessionstate). This enables their use within the parallel script block.</maml:para> <maml:para>> [!TIP] > > This parameter is the recommended way to make local functions available in the parallel scope. Alternatively, you can retrieve the function definition as a string (e.g., `$def = ${function:Greet}.ToString()`) and use `$using:` to pass it into the script block, defining it there (e.g., `${function:Greet} = $using:def`).</maml:para> </maml:description> <command:parameterValue required="true" variableLength="false">String[]</command:parameterValue> <dev:type> <maml:name>String[]</maml:name> <maml:uri /> </dev:type> <dev:defaultValue>None</dev:defaultValue> </command:parameter> <command:parameter required="true" variableLength="true" globbing="false" pipelineInput="True (ByValue)" position="named" aliases="none"> <maml:name>InputObject</maml:name> <maml:description> <maml:para>Specifies the objects to process in the script block. This parameter accepts pipeline input.</maml:para> </maml:description> <command:parameterValue required="true" variableLength="false">Object</command:parameterValue> <dev:type> <maml:name>Object</maml:name> <maml:uri /> </dev:type> <dev:defaultValue>None</dev:defaultValue> </command:parameter> <command:parameter required="true" variableLength="true" globbing="false" pipelineInput="False" position="0" aliases="none"> <maml:name>ScriptBlock</maml:name> <maml:description> <maml:para>Defines the script block executed for each input object in parallel.</maml:para> </maml:description> <command:parameterValue required="true" variableLength="false">ScriptBlock</command:parameterValue> <dev:type> <maml:name>ScriptBlock</maml:name> <maml:uri /> </dev:type> <dev:defaultValue>None</dev:defaultValue> </command:parameter> <command:parameter required="false" variableLength="true" globbing="false" pipelineInput="False" position="named" aliases="tl"> <maml:name>ThrottleLimit</maml:name> <maml:description> <maml:para>Sets the maximum number of script blocks executed in parallel across multiple threads. Additional input objects wait until the number of running script blocks falls below this limit. The default value is `5`.</maml:para> </maml:description> <command:parameterValue required="true" variableLength="false">Int32</command:parameterValue> <dev:type> <maml:name>Int32</maml:name> <maml:uri /> </dev:type> <dev:defaultValue>5</dev:defaultValue> </command:parameter> <command:parameter required="false" variableLength="true" globbing="false" pipelineInput="False" position="named" aliases="to"> <maml:name>TimeoutSeconds</maml:name> <maml:description> <maml:para>Specifies the maximum time (in seconds) to process all input objects. When the timeout is reached, running script blocks are terminated, and remaining input is discarded.</maml:para> <maml:para>> [!NOTE] > > A value of `0` (default) disables the timeout, allowing processing to continue until completion.</maml:para> </maml:description> <command:parameterValue required="true" variableLength="false">Int32</command:parameterValue> <dev:type> <maml:name>Int32</maml:name> <maml:uri /> </dev:type> <dev:defaultValue>0</dev:defaultValue> </command:parameter> <command:parameter required="false" variableLength="true" globbing="false" pipelineInput="False" position="named" aliases="unr"> <maml:name>UseNewRunspace</maml:name> <maml:description> <maml:para>Uses a new runspace for each parallel invocation instead of reusing existing runspaces in the runspace pool.</maml:para> </maml:description> <command:parameterValue required="false" variableLength="false">SwitchParameter</command:parameterValue> <dev:type> <maml:name>SwitchParameter</maml:name> <maml:uri /> </dev:type> <dev:defaultValue>False</dev:defaultValue> </command:parameter> <command:parameter required="false" variableLength="true" globbing="false" pipelineInput="False" position="named" aliases="vars"> <maml:name>Variables</maml:name> <maml:description> <maml:para>Provides a hashtable of variables to make available in the parallel scope. Keys define the variable names within the script block.</maml:para> <maml:para>> [!TIP] > > Use this parameter as an alternative to the `$using:` scope modifier (https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_scopes#scope-modifiers).</maml:para> </maml:description> <command:parameterValue required="true" variableLength="false">Hashtable</command:parameterValue> <dev:type> <maml:name>Hashtable</maml:name> <maml:uri /> </dev:type> <dev:defaultValue>None</dev:defaultValue> </command:parameter> <command:parameter required="false" variableLength="true" globbing="false" pipelineInput="False" position="named" aliases="mn"> <maml:name>ModuleNames</maml:name> <maml:description> <maml:para>Specifies an array of module names to import into the runspaces’ Initial Session State (https://learn.microsoft.com/en-us/dotnet/api/system.management.automation.runspaces.initialsessionstate). This allows the script block to use cmdlets and functions from the specified modules.</maml:para> <maml:para>> [!TIP] > > Use this parameter to ensure required modules are available in the parallel scope. Module names must be discoverable via the `$env:PSModulePath` (https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_psmodulepath)environment variable, which lists installed module locations.</maml:para> </maml:description> <command:parameterValue required="true" variableLength="false">String[]</command:parameterValue> <dev:type> <maml:name>String[]</maml:name> <maml:uri /> </dev:type> <dev:defaultValue>None</dev:defaultValue> </command:parameter> <command:parameter required="false" variableLength="true" globbing="false" pipelineInput="False" position="named" aliases="mp"> <maml:name>ModulePaths</maml:name> <maml:description> <maml:para>Specifies an array of file paths to directories containing PowerShell modules (e.g., `.psm1` or `.psd1` files) to import into the runspaces’ Initial Session State (https://learn.microsoft.com/en-us/dotnet/api/system.management.automation.runspaces.initialsessionstate). This enables the script block to use cmdlets and functions from custom or local modules.</maml:para> <maml:para>> [!NOTE] > > Paths must be absolute or relative to the current working directory and must point to valid directories containing PowerShell modules. If an invalid path (e.g., a file or non-existent directory) is provided, a terminating error is thrown.</maml:para> </maml:description> <command:parameterValue required="true" variableLength="false">String[]</command:parameterValue> <dev:type> <maml:name>String[]</maml:name> <maml:uri /> </dev:type> <dev:defaultValue>None</dev:defaultValue> </command:parameter> </command:parameters> <command:inputTypes> <command:inputType> <dev:type> <maml:name>System.Object</maml:name> </dev:type> <maml:description> <maml:para>You can pipe any object to this cmdlet.</maml:para> </maml:description> </command:inputType> </command:inputTypes> <command:returnValues> <command:returnValue> <dev:type> <maml:name>System.Object</maml:name> </dev:type> <maml:description> <maml:para>Returns objects produced by the script block.</maml:para> </maml:description> </command:returnValue> </command:returnValues> <maml:alertSet> <maml:alert> <maml:para>- `Invoke-Parallel` uses multithreading, which may introduce overhead. For small datasets, sequential processing might be faster.</maml:para> <maml:para>- Ensure variables or collections passed to the parallel scope are thread-safe (e.g., use `ConcurrentDictionary<TKey, TValue>` (https://learn.microsoft.com/en-us/dotnet/api/system.collections.concurrent.concurrentdictionary-2) or similar), as shown in __Example #3__ (#example-3-adding-to-a-thread-safe-collection-with-using)and __Example #4__ (#example-4-adding-to-a-thread-safe-collection-with--variables), to avoid race conditions. - By default, runspaces are reused from a pool to optimize resource usage. Using `-UseNewRunspace` (#-usenewrunspace)increases memory and startup time but ensures isolation.</maml:para> </maml:alert> </maml:alertSet> <command:examples> <command:example> <maml:title>------- Example 1: Run a slow script in parallel batches -------</maml:title> <dev:code>$message = 'Hello world from ' 0..10 | Invoke-Parallel { $using:message + [runspace]::DefaultRunspace.InstanceId Start-Sleep 3 }</dev:code> <dev:remarks> <maml:para>This example demonstrates parallel execution of a script block with a 3-second delay, appending a unique runspace ID to a message. The `$using:` scope modifier (https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_scopes#the-using-scope-modifier)is used to pass the local variable `$message` into the parallel scope, a supported method for accessing external variables in `Invoke-Parallel`.</maml:para> </dev:remarks> </command:example> <command:example> <maml:title>-------- Example 2: Demonstrates `-Variables` Parameter --------</maml:title> <dev:code>$message = 'Hello world from ' 0..10 | Invoke-Parallel { $message + [runspace]::DefaultRunspace.InstanceId Start-Sleep 3 } -Variables @{ message = $message }</dev:code> <dev:remarks> <maml:para>This example demonstrates the `-Variables` parameter (#-variables), which passes the local variable `$message` into the parallel scope using a hashtable. The key `message` in the hashtable defines the variable name available within the script block, serving as an alternative to the `$using:` scope modifier.</maml:para> </dev:remarks> </command:example> <command:example> <maml:title>- Example 3: Adding to a thread-safe collection with `$using:` -</maml:title> <dev:code>$dict = [System.Collections.Concurrent.ConcurrentDictionary[int, object]]::new() Get-Process | Invoke-Parallel { ($using:dict)[$_.Id] = $_ } $dict[$PID]</dev:code> <dev:remarks> <maml:para>This example uses a thread-safe dictionary to store process objects by ID, leveraging the `$using:` modifier for variable access.</maml:para> </dev:remarks> </command:example> <command:example> <maml:title>Example 4: Adding to a thread-safe collection with `-Variables`</maml:title> <dev:code>$dict = [System.Collections.Concurrent.ConcurrentDictionary[int, object]]::new() Get-Process | Invoke-Parallel { $dict[$_.Id] = $_ } -Variables @{ dict = $dict } $dict[$PID]</dev:code> <dev:remarks> <maml:para>Similar to Example 3, this demonstrates the same functionality using `-Variables` instead of `$using:`.</maml:para> </dev:remarks> </command:example> <command:example> <maml:title>--------- Example 5: Using the `-Functions` parameter ---------</maml:title> <dev:code>function Greet { param($s) "$s hey there!" } 0..10 | Invoke-Parallel { Greet $_ } -Functions Greet</dev:code> <dev:remarks> <maml:para>This example imports a local function `Greet` into the parallel scope using `-Functions` parameter (#-functions), allowing its use within the script block.</maml:para> </dev:remarks> </command:example> <command:example> <maml:title>----- Example 6: Setting a timeout with `-TimeoutSeconds` -----</maml:title> <dev:code>0..10 | Invoke-Parallel { Start-Sleep 1 } -TimeoutSeconds 3</dev:code> <dev:remarks> <maml:para>This example limits execution to 3 seconds, stopping all running script blocks and ignoring unprocessed input once the timeout is reached.</maml:para> </dev:remarks> </command:example> <command:example> <maml:title>--- Example 7: Creating new runspaces with `-UseNewRunspace` ---</maml:title> <dev:code>0..3 | Invoke-Parallel { [runspace]::DefaultRunspace.InstanceId } -ThrottleLimit 2 # Guid # ---- # c945ae1f-4e66-4312-b23c-f3994965308e # 1c6af45c-8727-4488-937a-4dfc1d259e9e # c945ae1f-4e66-4312-b23c-f3994965308e # 1c6af45c-8727-4488-937a-4dfc1d259e9e 0..3 | Invoke-Parallel { [runspace]::DefaultRunspace.InstanceId } -ThrottleLimit 2 -UseNewRunspace # Guid # ---- # 7a1c3871-6ce2-4b7f-ae90-fb1e92cd9678 # 2488be9e-15fe-4be2-882d-7d98b068c913 # d3dd7b5d-e7e3-457f-b6fb-def35fe837d7 # 9af7c222-061d-4c89-b073-375ee925e538</dev:code> <dev:remarks> <maml:para>This example contrasts default runspace reuse with the `-UseNewRunspace` switch, showing unique runspace IDs for each invocation in the latter case.</maml:para> </dev:remarks> </command:example> <command:example> <maml:title>-------- Example 8: Using the `-ModuleNames` parameter --------</maml:title> <dev:code>Import-Csv users.csv | Invoke-Parallel { Get-ADUser $_.UserPrincipalName } -ModuleNames ActiveDirectory</dev:code> <dev:remarks> <maml:para>This example imports the `ActiveDirectory` module into the parallel scope using `-ModuleNames`, enabling the `Get-ADUser` cmdlet within the script block.</maml:para> </dev:remarks> </command:example> <command:example> <maml:title>-------- Example 9: Using the `-ModulePaths` parameter --------</maml:title> <dev:code>$moduleDir = Join-Path $PSScriptRoot "CustomModule" 0..10 | Invoke-Parallel { Get-CustomCmdlet } -ModulePaths $moduleDir</dev:code> <dev:remarks> <maml:para>This example imports a custom module from the specified directory using `-ModulePaths`, allowing the `Get-CustomCmdlet` function to be used in the parallel script block.</maml:para> <maml:para>> [!NOTE] > > The path must point to a directory containing a valid PowerShell module.</maml:para> </dev:remarks> </command:example> </command:examples> <command:relatedLinks> <maml:navigationLink> <maml:linkText>Online Version:</maml:linkText> <maml:uri>https://github.com/santisq/PSParallelPipeline</maml:uri> </maml:navigationLink> <maml:navigationLink> <maml:linkText>__ForEach-Object -Parallel__</maml:linkText> <maml:uri>https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/foreach-object</maml:uri> </maml:navigationLink> <maml:navigationLink> <maml:linkText>__Runspaces Overview__</maml:linkText> <maml:uri>https://learn.microsoft.com/en-us/dotnet/api/system.management.automation.runspaces.runspace</maml:uri> </maml:navigationLink> <maml:navigationLink> <maml:linkText>__Managed threading best practices__</maml:linkText> <maml:uri>https://learn.microsoft.com/en-us/dotnet/standard/threading/managed-threading-best-practices</maml:uri> </maml:navigationLink> <maml:navigationLink> <maml:linkText>__Thread-safe collections__</maml:linkText> <maml:uri>https://learn.microsoft.com/en-us/dotnet/standard/collections/thread-safe/</maml:uri> </maml:navigationLink> </command:relatedLinks> </command:command> </helpItems> |