Thursday, June 03, 2010

Splatting Hell

Recently both at work and at home I was faced with the same problem: a PowerShell ‘control’ script that needed to pass parameters down to an arbitrary series of child scripts (i.e. enumerating over scripts in a directory, and executing them in turn).

I needed a way of binding the parameters passed to the child scripts to what was passed to the parent script, and I thought that splatting would be a great fit here. Splatting, if you aren’t aware of it, is a way of binding a hashtable or array to a command’s parameters:

# ie replace this:
dir -Path:C:\temp -Filter:*

# with this:
$dirArgs = @{Filter="*"; Path="C:\temp"}
dir @dirArgs

Note the @ sign on the last line. That’s the splatting operator (yes, its also the hashtable operator as @{}, and the array operator as @(). It’s a busy symbol). It binds $dirArgs to the parameters, rather than attempting to pass $dirArgs as the first positional argument.

So I thought I could just use this to pass any-and-all arguments passed to my ‘master’ script, and get them bound to the child scripts. By name, mind, not by position. That would be bad, because each of the child scripts has different parameters. I want PowerShell to do the heavy lifting of binding the appropriate parameters to the child scripts.

Gotcha #1

I first attempted to splat $args, but I’d forgotten that $args is only the ‘left over’ arguments after all the positional arguments had been taken out. These go into $PSBoundParameters

Gotcha #2

…but only the ones that actually match parameters in the current script/function. Even if you pass an argument to a script in ‘named parameter’ style, like this:

SomeScript.ps1 –someName:someValue

…if there’s no parameter ‘someName’ on that script, this goes into $args as two different items, one being ‘-someName:’ and the next being ‘someValue’. This was surprising. Worse, once the arguments are split up in $args they get splatted positionally, even if they would otherwise match parameters on what’s being called. This seems like a design mistake to me (update: there is a Connect issue for this).

Basically what this meant was that, unless I started parsing $args myself, all the parameters on all the child scripts had to be declared on the parent (or at least all the ones I wanted to splat).

Gotcha #3

Oh, and $PSBoundParameters only contains the named parameters assigned by the caller. Those left unset, i.e. using default values, aren’t in there. So if you want those defaults to propagate, you’ll have to add them back in yourself:

function SomeFunction(
    $someValue = 'my default'
){
    $PSBoundParameters['someValue'] = $someValue

Very tiresome.

Gotcha #4

$PSBoundParameters gets reset after you dotsource another script, so you need to capture a reference to it before that :-(

Gotcha #5

Just when you thought you were finished, if you’re using [CmdLetBinding] then you’ll probably get an error when splatting, because you’re trying to splat more arguments than the script you’re calling actually has parameters.

To avoid the error you’ll have to revert to a ‘vanilla’ from an ‘advanced’ function, but since [CmdLetBinding] is implied by any of the [Parameter] attributes, you’ll have to remove those too :-( So back to $myParam = $(throw ‘MyParam is required’) style validation, unfortunately.

(Also, if you are using CmdLetBinding, remember to remove any [switch]$verbose parameters (or any others that match the ‘common’ cmdlet parameters), or you’ll get another error about duplicate properties when splatting, since your script now has a –Verbose switch automatically. The duplication only becomes an issue when you splat)

What Did We Learn?

Either: Don’t try this at home.

Or: Capture PSBoundParameters, put the defaults back in, splat it to child scripts not using CmdLetBinding or being ‘advanced functions’

Type your parameters, and put your guard throws back, just in case you end up splatting positionally

Have a lie down

7 comments:

Anonymous said...

It sounds like you're trying to do the equivalent of a plugin / extension system. You drop a plugin (implemented as a PowerShell script) into some directory, and then the controller script runs it.

The problem that you're running into is that this model doesn't enforce any strictness in the system. You're trying to support an arbitrary definition of the “control” script, and arbitrary definitions of the child scripts. Ultimately, something has to have structure. Since there is no structure defined, there really are no features to come to the rescue :)

To be successful, this is normally done with the child scripts following a pre-defined pattern:

foreach($script in (Get-ChildItem *.ps1))
{
& $script –Argument1 -Argument2 -Argument3
}

Or, via splatting:

foreach($script in (Get-ChildItem *.ps1))
{
$arguments = @{ Argument1 = value; Argument2 = value; Argument3 = value }
& $script @arguments
}

Lee Holmes [MSFT]
Windows PowerShell Development
Microsoft Corporation

piers7 said...

Sure, and that's pretty much where I ended up, but it was exactly that kind of pre-defined pattern I was attempting to avoid.

Actually neither scenario were plugins. One was a powershell based test runner running a tree structure of tests, the other was a build script executing child scripts which represent build steps.

In both case the child scripts could have parameters that the parent need not know about (they will default) but it would be useful to have a generic 'pass though' mechanism in case the caller of the parent needed to set them.

I'm still surprised that splatting an array doesn't appear to *ever* do any named argument matching.

Anonymous said...

PowerShell is simply broken regarding arg passing and parsing for any practical purposes. Probably beyond repair (if no one wanted to fix that by 2.0 ...).

One core issue is that scripts are made to act like "functions", and that means expecting pre-parsed args as "objects" and no facility to parse them on the spot.

Untill someone makes explicit, declarative distinction between actual function (well defined params and return value, straight invocation, no dumping into "pipe"), procedure (well defined params but undefined output - dumps into "pipe" at will) and script (loosely defined params, always parses on invocation as if it's a command line and a raw string of chars, undefined output) this is always going to be in limbo.

If you have several levels of scripts you'd have to capture original $psBoundParameters and $args separately, carry them around as special args to lower level scripts, plus do splatting.

Then you'd have to merge local and original $psBoundParameters at some point (so that you don't have to replicate huge top level signature everywhere) and then either parse the rest of $args manually (if your lower level script needs a lot of extras) or recondition merged $psBoundParameters to pass them passing them to a non-ps1 executable, even a cmd with sole purpose of breaking the vicious cycle and forcing next .ps1 script to actually parse command line. Beware of .cmd rules though (got to protect your args with quotes and do escapes for special chars).

Now, if some intermediary scripts are used as stand alone as well, they have to go checking if they received arg captures or not and process their args differently based on that.

Perhaps it's high time that someone just publishes a function that does normal arg parsing. It would be a long function but at least then we can simply keep and pass everything as a cmd string, forget the whole mess and live happily ever after - more or less :-)

Come to think of it, injecting a .cmd before every further .ps1 script invocation might still be less frustrating at the end of the day than dealing with the whole mess - haven't tried though.

Anonymous said...

OK. here's the script that can be used as a pass-through, and brings in %~dp0 for extra points :-) Fell free to add more :-))

Run it from cmd.exe like : pass.cmd foo.ps1

and then inside foo.ps1 call the next one like:

& "${Here}pass.cmd ${Here}next.ps1 ${NextArgs}"

prepare ${NextArgs} like:

$ParsedArgs = $myInvocation.BoundParameters # survies . sourcing btw

$ParsedArgs.Remove("MyName") > $Null
$NextArgs = @();
$ParsedArgs.Keys |% { $NextArgs += '-'+$_; $NextArgs += $ParsedArgs[$_]; }
$NextArgs += $args;

Again, you'll probably be able to do a bit better, but this provides basic mechanism and opens path to run interference from normal .cmd for anything else you may need.

-------- pass.cmd ----------
@echo off
setlocal

set pshPath="%SystemRoot%\system32\WindowsPowerShell\v1.0\powershell.exe"

set here=%~dp0
set script=%~dp0%1

%pshPath% %script% -Here %here% -MyName %*

endlocal

Anonymous said...

Hello,

I faced a similar issue in porting an execution control and logging routine to PowerShell.

Usage:
Exec echo "Hello world"
Exec Configure-MySystem -Verbose

The solution was to let Invoke-
Expression itself decide if a -Xxxx argument is a named parameter, or a "normal" string.

Function Exec([string]$cmd) {
$txtLine = "$cmd"
$cmdLine = "$cmd"
$a = $args # Make a copy (Necessary because Invoke-Expression overwrites $args)
$n = 0
foreach ($arg in $args) { # Parse all optional arguments following the command name.
$txtLine += " $(Quote $arg)"
if (($arg -ne $null) -and ($arg.GetType().FullName -eq "System.String") -and ($arg -match '^-\w+:?$')) {
$cmdLine += " $arg" # Let Invoke-Expression decide whether it's a param name or an actual string.
} else {
$cmdLine += " `$a[$n]" # Preserve the type through Invoke-Expression.
}
$n += 1
}
if ($script:Verbose -or $script:NoExec) {
Put-Line $txtLine
}
if (!$script:NoExec) {
if (!$script:Verbose) {Put-Line $txtLine}
Invoke-Expression $cmdLine
}
}

Jean-Francois Larvoire

Anonymous said...

I think Gotcha #5 can be overcome by adding the following as a final parameter to any function (sure, not ideal, but it works):

#Required for Splatting non-positionally and/or capturing remaining arguments...
[Parameter(ValueFromRemainingArguments = $true)]$remainingArgs

as found at http://stackoverflow.com/questions/2795582/any-way-to-specify-a-non-positional-parameter-in-a-powershell-script

This will mop up any additional parameters in the splatted hashtable which otherwise would be complained about as the error message you refer to:
"A parameter cannot be found that matches parameter name ..."

( Interestingly, this error/complaint only comes about as soon as you add the
[Parameter( ... attribute to any one parameter. With just a plain param definition, the splatting works without error, even with a splat hashtable containing extra parameters not in the function being called. )

Hope that helps someone out there...

Regards
Vernon

Deikensentsu said...

I found this which looks pretty awesome to make PowerShell handle extra arguments not required by the cmdlet you happen to be calling.

http://pelebyte.net/blog/2011/07/14/better-powershell-splatting/

Popular Posts