Background Jobs (PowerShell)

Overview

This feature allows Cmdlets to run as background jobs. Its fairly easy to work out how to make individual Cmdlet's or PowerShell scripts to run as jobs (see about_Jobs, but running functions are a bit more of a pain as the job runs in a new scope (so any functions, variables etc that are defined in your scripts scope, have no meaning in the background job's scope). Therefore functions, etc, have to explicitly included as a script block in the background job...

# First define the script block and function
$funky = {
    function TestJob {
        $processList = Get-Process
        Return $processList                          # Yes could be achieved in one line, but wouldn't be much of a function!
    }
}

# Start job
$job = Start-Job -ScriptBlock {TestJob} -InitializationScript $funky

$job | Format-List *                                 # Displays created $job object

Wait-Job -job $job                                   # Wait on completion of job
Receive-Job -job $job                                # Gets result of $job (ie result of Get-Process)
Get-Job                                              # Shows list of jobs (current and completed)

Arguments have to be passed through to the job through the -InputObject parameter, which isn't particularly pretty. For further info see http://robertrobelo.wordpress.com/2010/03/14/background-jobs-input-gotcha/ for a decent explanation, though I do kind of cover this below.

Script Block or Script File...? Your background task can either take the form of a script block, or a script file. Personally I prefer to keep everything in one script as it makes organisation easier, up to an extent. There is a limit to size of a script block, no idea what it is, as the script I as trying to mangle into running as jobs was large (20 KB, nearly 1000 lines) and I didn't have a convenient way to test. But your jobs will fail if they're too big.

Job Control

Below is a fuller example of using background jobs to manage multiple work streams

Further reading...

• http://ryan.witschger.net/?p=22 - Multi-Threading in PowerShell V2

$funky = {
    function TestJob {
        $var = $Input.‘<>4__this’.Read()
        Write-Host "This is job " $var[0]
        Start-Sleep $var[1]
    }
}

$jobs = @()
$job = "" | Select Name, Vars, State, Obj
$job.Name = "Job1"
$job.Vars = ($job.Name, 10)
$jobs += $job
$job = "" | Select Name, Vars, State, Obj
$job.Name = "Job2"
$job.Vars = ($job.Name, 5)
$jobs += $job

foreach ($job in $jobs) {
    Write-Host ("Starting " + $job.Name)
    $job.Obj = Start-Job -ScriptBlock {TestJob} -InitializationScript $funky -Name $job.Name -InputObject $job.Vars
    $job.State = "Running"
}

# Idle loop
While (1) {
    $JobsRunning = 0
    foreach ($job in $jobs) {
        if ($job.State -ne $job.Obj.JobStateInfo.state) {
            Write-Host ($job.Name + " state now " + $job.Obj.JobStateInfo.state)
            $job.State = $job.Obj.JobStateInfo.state
        }
        if ($job.State -eq "Running") {
            $JobsRunning += 1
        }
   }
   Write-Host ("$JobsRunning of " + $jobs.count + " jobs still running")
   if ($JobsRunning -eq 0) {
       Break
   }
   Start-Sleep 1
}
    
Write-Host "All finished...!"

# To see output from jobs
# Get-Jobs              - shows list of jobs
# Receive-Job -Id x     - shows the data returned from the job

Transcripts / Logging

I haven't been able to get transcripts working properly with background jobs, at least, not to my liking. It is possible to capture the console output from a background job into the transcript of the parent or master script. But if you're running a large background job script and want to capture the transcript/logging from job separately you have to faff around - you can't just start and stop transcribing from within the (child) background job script, it won't write anything to disk.

Similarly, you can't redirect the output from Receive-Job to a file, you'll lose some of the output (I think this may only capture StdErr and/or explicitly returned objects, standard Write-Host output is dropped).

One way around this is to stop the transcript for your master/parent script, then start a temporary trasncript to capture then return from your child job once its finished, so so something like...

if ($job.State.ToString() -eq "Completed") {
    Write-Host ($job.Name + " writing log to job-" + $job.Name + ".log")
    # Nasty logging handling (Receive-Job StdOut to console only, can't redirect to file, can only catch StdErr to file)
    Stop-Transcript
    Start-Transcript -Path ("job-" + $job.Name + ".log")
    Receive-Job -Id $job.job.Id
    Stop-Transcript
    Start-Transcript -Path $Logfile -Append
}

Gotchas

• Working Directory
  • The background job script runs in a new context, therefore it runs in the default path. If you tend to run your scripts from a non-default path, and need to read/write files, get the current directory using Get-Location and pass it to your job as a parameter, then use Set-Location in your job.
• VMware PowerCLI 64 bit
  • Due to a bug you can't launch background jobs against VMware vSphere (VI4) hosts in a 64-bit environment, PowerShell will crash. Its probably a bug in PowerCLI, and will hopefully be fixed in a future release (bug exists in v4.1u1 and possibly other versions). You can run scripts in 32-bit (see Useful One-Liners on how to detect in your script).