Background Jobs (PowerShell) - Revision history

Sstrutt: /* Overview */ Minor rewording

2013-04-10T09:56:48Z

Overview: Minor rewording

← Older revision		Revision as of 09:56, 10 April 2013
Line 15:		Line 15:

	$job \| Format-List * # Displays created $job object		$job \| Format-List * # Displays created $job object
			</source>

	Wait-Job -job $job # Wait on completion of job		=== Basic Commands ===
	Receive-Job -job $job # Gets result of $job (ie ~~result of Get-Process~~)		<source lang="powershell">
			Wait-Job -job $job # Wait on completion of job - pause script processing until complete
			Receive-Job -job $job # Gets result of $job (ie what would have been presented to the console had the job run in an interactive PowerShell session)
	Get-Job # Shows list of jobs (current and completed)		Get-Job # Shows list of jobs (current and completed)
			Get-Job \| Remove-Job # Clears list of jobs
	</source>		</source>

	Arguments have to be passed through to the job through the <code> -InputObject </code> parameter, which isn't particularly pretty. For further info see http://robertrobelo.wordpress.com/2010/03/14/background-jobs-input-gotcha/ for a ~~decent~~ explanation, ~~though I do kind of cover this below~~.		=== Passing Arguments / Parameters to Jobs ===
			Arguments have to be passed through to the job through the <code> -InputObject </code> parameter, which isn't particularly pretty (see below for how this works in practice). Background jobs run in a completely separate session, so objects/variables etc exists only in the scope of that job.

			For further info see http://robertrobelo.wordpress.com/2010/03/14/background-jobs-input-gotcha/ for a fuller explanation.

			=== Job Priority ===
			Background jobs normally run with Below Normal priority (most processes, including PowerShell normally run as Normal priority).

	~~Background jobs normally run with Below Normal priority (most processes, including PowerShell normally run as Normal priority.~~ For more info on checking or changing see [[Getting_Started_(PowerShell)#Process_Priority\|PowerShell Process Priority]].		For more info on checking or changing see [[Getting_Started_(PowerShell)#Process_Priority\|PowerShell Process Priority]].

	=== Script Block or Script File ? ===		=== Script Block or Script File ? ===

Sstrutt: Minor intro rewording, and Meta

2012-08-16T09:19:11Z

Minor intro rewording, and Meta

← Older revision		Revision as of 09:19, 16 August 2012
Line 1:		Line 1:
	== Overview ==		== Overview ==
	This feature allows Cmdlets to run as background jobs. Its fairly easy to work out how to make individual Cmdlet's or PowerShell scripts to run as jobs (see [http://technet.microsoft.com/en-us/library/dd315273.aspx about_Jobs], but running ~~functions are~~ a bit more of a pain as the job runs in a new scope (so any functions, variables etc that are defined in your scripts scope, have no meaning in the background job's scope). Therefore functions, etc, have to explicitly included as a script block in the background job...		This feature allows Cmdlets, script blocks, or entire scripts to run as background jobs. Its fairly easy to work out how to make individual Cmdlet's or PowerShell scripts to run as jobs (see [http://technet.microsoft.com/en-us/library/dd315273.aspx about_Jobs], but running script blocks can be a bit more of a pain as the job runs in a new scope (so any functions, variables etc that are defined in your scripts scope, have no meaning in the background job's scope - which is a bit counter-intuitive). Therefore functions, etc, have to explicitly included as a script block in the background job...

	<source lang="powershell">		<source lang="powershell">
Line 7:		Line 7:
	function TestJob {		function TestJob {
	$processList = Get-Process		$processList = Get-Process
	Return $processList # Yes could be achieved in one line, but wouldn't be much of a function!		Return $processList # Yes, this could be achieved in one line, but wouldn't be much of a function!
	}		}
	}		}

Sstrutt: /* Overview */ Minor rewording

2012-05-31T09:24:38Z

Overview: Minor rewording

← Older revision		Revision as of 09:24, 31 May 2012
Line 23:		Line 23:
	Arguments have to be passed through to the job through the <code> -InputObject </code> parameter, which isn't particularly pretty. For further info see http://robertrobelo.wordpress.com/2010/03/14/background-jobs-input-gotcha/ for a decent explanation, though I do kind of cover this below.		Arguments have to be passed through to the job through the <code> -InputObject </code> parameter, which isn't particularly pretty. For further info see http://robertrobelo.wordpress.com/2010/03/14/background-jobs-input-gotcha/ for a decent explanation, though I do kind of cover this below.

	~~'''~~Script Block~~'''~~ or ~~'''~~Script File~~'''...~~?		Background jobs normally run with Below Normal priority (most processes, including PowerShell normally run as Normal priority. For more info on checking or changing see [[Getting_Started_(PowerShell)#Process_Priority\|PowerShell Process Priority]].
	Your background task can either take the form of a script block, or a script file. Personally I prefer to keep everything in one script as it makes organisation easier, up to an extent. ~~There~~ is a limit to size of a script block, no idea what it is~~, as the~~ script I as trying to mangle into running as ~~jobs~~ was large (20 KB, nearly 1000 lines) and I didn't have a convenient way to test. But your jobs will fail if they're too big.
			=== Script Block or Script File ? ===
			Your background task can either take the form of a script block, or a script file. Personally I prefer to keep everything in one script as it makes organisation and maintenance easier, up to an extent. However, there is a limit to size of a script block, though I've no idea what it is (it may relate to maximum script line length that PowerShell can handle).

			A script I was trying to mangle into running as a background job was large (20 KB, nearly 1000 lines), and it wouldn't run. Unfortunately I was short on time and didn't have a convenient way to test where the breaking point was. But your jobs will fail if they're too big.

	== Job Control ==		== Job Control ==

Sstrutt: /* Gotchas */ Rewording of "VMware PowerCLI 64 bit"

2012-05-28T14:02:07Z

Gotchas: Rewording of "VMware PowerCLI 64 bit"

← Older revision		Revision as of 14:02, 28 May 2012
Line 105:		Line 105:
	** The background job script runs in a new context, therefore it runs in the default path. If you tend to run your scripts from a non-default path, and need to read/write files, get the current directory using <code>Get-Location</code> and pass it to your job as a parameter, then use <code>Set-Location</code> in your job.		** The background job script runs in a new context, therefore it runs in the default path. If you tend to run your scripts from a non-default path, and need to read/write files, get the current directory using <code>Get-Location</code> and pass it to your job as a parameter, then use <code>Set-Location</code> in your job.
	* '''VMware PowerCLI 64 bit'''		* '''VMware PowerCLI 64 bit'''
	** Due to a bug you can't launch background jobs ~~against VMware vSphere (VI4) hosts~~ in a 64-bit environment, PowerShell will crash. ~~Its probably~~ a ~~bug~~ in PowerCLI, ~~and will hopefully be fixed in a future release (bug exists in~~ v4.1u1 and ~~possibly other~~ versions). ~~You can~~ run scripts in 32-bit (see [[Getting_Started_(PowerShell)#Useful_One-Liners\|Useful One-Liners]] ~~on how to detect in your script)~~.		** Due to a bug you can't launch [[:Category:PowerCLI\|VMware PowerCLI]] background jobs when running in a 64-bit environment. Doing so cause launched PowerShell instances to exhibit high CPU and memory usage and eventually PowerShell will crash. Whether its a problem in PowerShell or in PowerCLI is unclear (though I suspect PowerShell), the problem definitely effects PowerCLI v4.1u1 and v5, and probably affects all versions.
			** Therefore launch from 32-bit PowerCLI shell. To run scripts in 32-bit see [[Getting_Started_(PowerCLI)#Force_32-bit\|Force 32-Bit]], to detect in a script see [[Getting_Started_(PowerShell)#Useful_One-Liners\|Useful One-Liners]].

	[[Category:PowerShell]]		[[Category:PowerShell]]

Sstrutt: Undo revision 1853 by Sstrutt (talk) - Overwrote page with incorrect content

2012-05-28T13:39:09Z

Undo revision 1853 by Sstrutt (talk) - Overwrote page with incorrect content

Show changes

Sstrutt: Initial creation - content from depreciated PowerShell page

2012-04-20T14:21:50Z

Initial creation - content from depreciated PowerShell page

Show changes

Sstrutt: Initial creation - content from depreciated PowerShell page

2012-04-20T14:21:40Z

Initial creation - content from depreciated PowerShell page

New page

== Overview ==
This feature allows Cmdlets to run as background jobs. Its fairly easy to work out how to make individual Cmdlet's or PowerShell scripts to run as jobs (see [http://technet.microsoft.com/en-us/library/dd315273.aspx about_Jobs], but running functions are a bit more of a pain as the job runs in a new scope (so any functions, variables etc that are defined in your scripts scope, have no meaning in the background job's scope). Therefore functions, etc, have to explicitly included as a script block in the background job...

<source lang="powershell">
# First define the script block and function
$funky = {
function TestJob {
$processList = Get-Process
Return $processList # Yes could be achieved in one line, but wouldn't be much of a function!
}
}

# Start job
$job = Start-Job -ScriptBlock {TestJob} -InitializationScript $funky

$job | Format-List * # Displays created $job object

Wait-Job -job $job # Wait on completion of job
Receive-Job -job $job # Gets result of $job (ie result of Get-Process)
Get-Job # Shows list of jobs (current and completed)
</source>

Arguments have to be passed through to the job through the <code> -InputObject </code> parameter, which isn't particularly pretty. For further info see http://robertrobelo.wordpress.com/2010/03/14/background-jobs-input-gotcha/ for a decent explanation, though I do kind of cover this below.

'''Script Block''' or '''Script File'''...?
Your background task can either take the form of a script block, or a script file. Personally I prefer to keep everything in one script as it makes organisation easier, up to an extent. There is a limit to size of a script block, no idea what it is, as the script I as trying to mangle into running as jobs was large (20 KB, nearly 1000 lines) and I didn't have a convenient way to test. But your jobs will fail if they're too big.

== Job Control ==
Below is a fuller example of using background jobs to manage multiple work streams

Further reading...
* http://ryan.witschger.net/?p=22 - Multi-Threading in PowerShell V2

<source lang="powershell">
$funky = {
function TestJob {
$var = $Input.‘<>4__this’.Read()
Write-Host "This is job " $var[0]
Start-Sleep $var[1]
}
}

$jobs = @()
$job = "" | Select Name, Vars, State, Obj
$job.Name = "Job1"
$job.Vars = ($job.Name, 10)
$jobs += $job
$job = "" | Select Name, Vars, State, Obj
$job.Name = "Job2"
$job.Vars = ($job.Name, 5)
$jobs += $job

foreach ($job in $jobs) {
Write-Host ("Starting " + $job.Name)
$job.Obj = Start-Job -ScriptBlock {TestJob} -InitializationScript $funky -Name $job.Name -InputObject $job.Vars
$job.State = "Running"
}

# Idle loop
While (1) {
$JobsRunning = 0
foreach ($job in $jobs) {
if ($job.State -ne $job.Obj.JobStateInfo.state) {
Write-Host ($job.Name + " state now " + $job.Obj.JobStateInfo.state)
$job.State = $job.Obj.JobStateInfo.state
}
if ($job.State -eq "Running") {
$JobsRunning += 1
}
}
Write-Host ("$JobsRunning of " + $jobs.count + " jobs still running")
if ($JobsRunning -eq 0) {
Break
}
Start-Sleep 1
}

Write-Host "All finished...!"

# To see output from jobs
# Get-Jobs - shows list of jobs
# Receive-Job -Id x - shows the data returned from the job
</source>

== Transcripts / Logging ==
I haven't been able to get transcripts working properly with background jobs, at least, not to my liking. It is possible to capture the console output from a background job into the transcript of the parent or master script. But if you're running a large background job script and want to capture the transcript/logging from job separately you have to faff around - you can't just start and stop transcribing from within the (child) background job script, it won't write anything to disk.

Similarly, you can't redirect the output from <code> Receive-Job </code> to a file, you'll lose some of the output (I think this may only capture StdErr and/or explicitly returned objects, standard <code> Write-Host </code> output is dropped).

One way around this is to stop the transcript for your master/parent script, then start a temporary trasncript to capture then return from your child job once its finished, so so something like...
<source lang="powershell">
if ($job.State.ToString() -eq "Completed") {
Write-Host ($job.Name + " writing log to job-" + $job.Name + ".log")
# Nasty logging handling (Receive-Job StdOut to console only, can't redirect to file, can only catch StdErr to file)
Stop-Transcript
Start-Transcript -Path ("job-" + $job.Name + ".log")
Receive-Job -Id $job.job.Id
Stop-Transcript
Start-Transcript -Path $Logfile -Append
}
</source>

== Gotchas ==
* '''Working Directory'''
** The background job script runs in a new context, therefore it runs in the default path. If you tend to run your scripts from a non-default path, and need to read/write files, get the current directory using <code>Get-Location</code> and pass it to your job as a parameter, then use <code>Set-Location</code> in your job.
* '''VMware PowerCLI 64 bit'''
** Due to a bug you can't launch background jobs against VMware vSphere (VI4) hosts in a 64-bit environment, PowerShell will crash. Its probably a bug in PowerCLI, and will hopefully be fixed in a future release (bug exists in v4.1u1 and possibly other versions). You can run scripts in 32-bit (see [[Getting_Started_(PowerShell)#Useful_One-Liners|Useful One-Liners]] on how to detect in your script).

[[Category:PowerShell]]