The new starter experience

I normally avoid 'link-posts', but again Hacknot is right on the money with 'If They Come, How Will they Build It?', an eminently familiar analysis of the plight of a new developer on a project with an oral documentation culture.

In fact I'd go slightly further than Hacknot, and state that the initial experience of a new developer on the project is one of the most important things to get right. First impressions do matter, and if your first impression of a project is the frustration of:

• Not having a login
• Not having internet access
• Not being able to get latest
• Not being able to build
• Not being able to locate any documentation
• Not having clear lines of escalation
  
• Not having clear rules of engagement
• Not knowing what's expected of you
• Not having a mentor
  

...you're going to be hard pushed not to be prejudicing your opinions of the professionalism of the rest of the project. You'll start disheartened, but on the other end of this unfortunate indoctrination you're going to be just like them. You won't regard the absence of the list above as anything other than normal. You'll accept that that's just not how things are done around here. You will love big brother.

[ahem. got carried away there]

I regard the absence of guides and documentation as more than a major time-waster: it's a self-perpetuating morale hole for all future team members to climb into and die.

New staff play a vital part in ensuring a project's approach doesn't atrophy. If you waste their 'fresh' time frustrating them with missing documentation and runaround, you won't get the benefit of seeing things from their eyes. They'll have clammed up and learnt to live with how it is, and by the time you ask them they'll have forgotten that they used to care.

FixBrokenWindows - they're not all in your code

Great Powershell One-liners

Or, 'highly useful things you can do in Powershell':

Watching the last 20 lines of the last (log?) file in a directory:

(gci)[-1] | gc | select-object -last 20

Finding where .Net is installed:

(get-itemproperty HKLM:\software\microsoft\.netframework).InstallRoot

Pinging a URL [1]:

$temp = (new-object net.webclient).DownloadString($url)
if ($?) { write-host Passed -foregroundcolor "green" }

Getting the DNS suffix for a machine:

(Get-WmiObject -class "Win32_ComputerSystem").Domain

[1] Ok, that one's not a one-liner.

Why Powershell Rocks

I try and remain reasonably sanguine / sceptical / cynical about most new technologies, but there's two about at the moment that I just can't find anything to fault with: Vista's Media Center, and Windows Powershell.

So (apart from the fawning) what's so good about Powershell?

The key point is that Powershell is a shell that thinks it's a scripting language. Or is it the other way round? Well it's both anyway. So things you can do in a BAT file, you can do in Powershell:

xcopy /I /F /Y Somefile.abc ..\SomeFolder

[In many cases, like the above, you can just cut and paste the same line into Powershell and it will work]

Ever tried doing that in VBScript/JScript? Either you wrote lines of Scripting.FileSystemObject code, or you shelled out (in which case you've got the line above plus the 'shelling out' code).

But BAT files can only take you so far. Further if you're stubborn or clever, but even then there's a wall. Creating files with todays date in the name is a real swine. Setting IIS properties involves an external VBScript. HTTP-pinging a URL and checking it's up... no chance.

For these you have to write a VBScript or an (.Net?) EXE, where you can take advantage a host of supporting libraries and benefit from 'real' programmer concepts like variable scope, functions and conditionals. But integrating them with your BAT files (ie retting return values back) is quite a challenge. And you end up with your foot in both camps

Powershell does both.

I'll give you an example of the kind of hybrid approach this engenders: setting ACLs for the anonymous user on a website. Now you can assume that it's IUSR_%computername%, but in some scenarios[1] (renamed for security, ghosted image / machine renamed) it's not. So you've got to look up the user first, which is pretty tricky in a BAT file (even with adsutil.vbs), but then set some file permissions, which is nigh-on impossible in VBScript. In PowerShell this is easy:

$iisobj = [wmi]"root/MicrosoftIIsV2:IIsWebVirtualDirSetting.Name='W3SVC/1/Root/MyWebSite'"
$userName=$iisobj.AnonymousUserName
$path=$iisobj.Path

CACLS $path /E /G $userName:R

Note how I did something very 'script' - dealing with objects and properties, and followed it up with something very 'batch' - just calling a shell command (CACLS). And it just worked.

Now in my real script I didn't use CACLS, I used the .Net System.Security.AccessControl classes, because I'm a .Net developer and that's what I thought of first. I wrapped them up into a neat reusable function, so the code above actually didn't look so different. But I ended up writing 20+ lines of code, where CACLS would have done the job just as well. I'm learning too.

There's lots of other great things about Powershell too:

• Some great syntax improvements, like range operators, and reverse-indexing arrays
• Fantastic support for script parameters, and default values
• Here-strings
  

...but they're all secondary to this one, which is that Powershell is all you need[2]. That's pretty compelling to me.


[1] I.E. My scenario.
[2] Ok, unless you're doing something really wacky/Win32 [3] where the .Net BCL doesn't have the support yet.
[3] [4] Is there a difference?
[4] Recursing footnotes? Can I get away with that?

IE Advanced Security Configuration affects Powershell execution policy

Like a few other MS apps, but hardly obvious to us mortals, Powershell determines whether something is 'local' based on the Internet Explorer Security Zones. This means that (by default) scripts run from a UNC are considered part of the Local Intranet zone, and so can run unsigned under the RemoteSigned execution policy:



However, if your Win 2003 server is running the Internet Explorer Enhanced Security Configuration, UNC paths are not included in the zone unless explicitly added:



So your scripts will refuse to run unless you reduce the execution policy down to 'unrestricted', leaving you with lots of nasty messages:

Run only scripts that you trust. While scripts from the Internet can be useful, this script can potentially harm your computer. Do you want to run etc... [D] Do not run [R] Run once [S] Suspend [?] Help (default is "D"):

Yuck (and very frustrating till you work it out).

If IE Enhanced Security Configuration is mandated, the only way round this is to explicitly add the UNCs where your scripts are located into the Local Intranet zone (nb: don't add a http: prefix!)

You can do this manually, or you can just import a set of registry keys if you need to do a whole load in one go (or the same across multiple servers). If you do it through the UI the settings are user-scoped, if you do it via the registry you can add settings either for the current user, or at a machine-wide level. See Adding Sites to the Enhanced Security Configuration Zones in MSDN and Description of Internet Explorer security zones registry entries in the knowlege base for more details.

Log4net in Asp.Net redux: Implement IFixingRequired on your Active Property Values

A while back I noted that the built-in contexts in log4net were broken in the face of ASP.Net's thread agility (and pointed this out to the log4net community).

The workaround I suggested, which was also suggested on the log4net developers list, was to leverage log4net's support for deferred evaluation of logging properties. These are known as Active Property Values, but that's just a fancy/short way of saying 'any object you like, who's ToString() method results in the value that you actually want logged'. This was a pretty neat workaround, eg:

      log4net.GlobalContext.Properties["requestUrl"] = new HttpContextRequestUrlProvider();

    private class HttpContextRequestUrlProvider
    {
        public override string ToString()
        {
            HttpContext context = HttpContext.Current;
            if (context == null) return null;
            return context.Request.RawUrl;
        }
    }

However in one of my logging databases, I recently noticed logging entries attributed to me that had clearly come from someone else, which made me worry for a few hours whether this pattern was broken, or broken when using an appender that buffers (like the AdoNetAppender).

Digging through the source code showed that BufferingAppenderSkeleton does indeed attempt to 'fix' logging entries when they go into the buffer, to guard against exactly these kind of multithreading logging mishaps:

 // Because we are caching the LoggingEvent beyond the
// lifetime of the Append() method we must fix any
// volatile data in the event.
loggingEvent.Fix = this.Fix;

This causes all kinds of intrinsic log4net values (like thread ID, UserName), and the message itself to be 'fixed': i.e. fully evaluated (and rendered via the layout) now, rather than later. Otherwise all the logging events in the buffer would end up being written out with the values in-play at the time the appender flushed (ie potentially another thread/users's context).

This fixing also 'fixes' properties from the log4net contexts (ThreadContext, LogicalThreadContext and GlobalContext) if they implement IFixingRequired. And that's what I missed - one obscure interface:

 // Fix any IFixingRequired objects
IFixingRequired fixingRequired = val as IFixingRequired;
if (fixingRequired != null)
{
 val = fixingRequired.GetFixedObject();
}

But I've missed that for the last couple of years. So… actually I'd rather not think about the implications of that. Meanwhile, if anyone who does any log4net dev would actually like to update the Active Property Value doco, that'd be good, thanks.

Better still, 18 months later, can we have our Adaptive Context now please?

Production Debugging for Hung ASP.Net 2 applications – a crash course

If your app is down right now skip the intro – I’ll understand.

Introduction

First up: debugging – even in the development environment – should never be a substitute to detailed diagnostic logging and appropriate – meaningful – error handling. Any time I fix an exception, if the problem is initially non-obvious, I first fix the logging code. Then and only then do I fix the exception.

Baking in a support mentality early in the development process is the only way to avoid an application that’s hard (and by that I mean expensive) to support. The customer might be paying for the support time, but if you can’t turnaround support issues quickly – and practically any issue in a production business application needs turning around quickly – they start questioning your competence.

But sometimes logs aren’t enough, and even with the best will in the world, sometimes they’re not sufficiently detailed (they won’t - for example – be much help tracking down that memory leak). Sometimes you need to know what’s thrashing right now; to see the smoking gun.

Enter the world of crash dump debugging.

I’d been following Tess’s blog for a while now, but still had a big hole in my understanding. Sure I now knew I could use the !clrstack command to start looking at managed thread stacks, but where the hell was I typing that? This then is my introduction to Production Debugging for .Net applications (or ‘how to understand what Tess is talking about’).

I’m going to be talking about debugging hung .Net 2 ASP.Net applications, but much of this is relevant to debugging any .Net application in a production or testing environment.

Getting the Dump

Assuming you don’t have time to read a more detailed article on this (and the Microsoft guide is 170+ pages!), just do this:

WARNING: Installing the debugging tools on your production server isn’t recommended. That’s not to say you have any choice in the matter, but you should be aware there are security and availability implications that – if your app is hung – you probably don’t care about right now.

Install the Microsoft Debugging Tools for Windows and IISState on the machine where the hang occurs. You’ll need the Debugging Tools on your workstation too, but you can do this later if it’s happening right now.

If your app is thrashing CPU, use TaskManager to get the process ID (PID) of the W3WP process that’s using the most CPU (there may be more than one). NB: PIDs aren’t displayed by default: use View\Select Columns to turn them on.

Otherwise, at the command prompt, enter cscript iisapp.vbs to determine the process ID that matches your app pool.

At the command prompt CD to wherever you installed IISState (C:\IISState is the default). If there is existing content in the output folder, delete or rename the folder. Then execute the following:

IISState –p  <PID> -d

(…replacing <PID> with the PID you determined above)

WARNING: At this point your app will become unresponsive until the dump completes

IISState will put a dump file (.dmp) and a text summary (.log) of the process’s threads etc… in the output directory.

Rename the output folder to something that reflects the date/time, so future dumps don’t overwrite it.

Since we’re doing a ‘hang’, ideally at this point you’d wait a few mins, and then take another dump. If you don’t, you may regret it later and / or have to wait for the issue to re-arise before you can definitively pin it down. However, if the world and his dog want the site back up right now then skip it – it’s your call.

Kill that process. IIS will recycle it, and things should go back to normal for now. Panic over.

Analyzing the Dump

Copy the renamed output directory down onto your machine. For
the process ID you dumped there will also be an IISState summary file, eg IISState-5090.log. This log is a dump of the stack trace of all the threads in the process, and attempts to perform some additional diagnostics for you. Unfortunately, it doesn’t (at present) support .Net 2, so most likely it’s not going to tell you what the problem is. You should at least be able to see a few threads that look something like this:

Thread ID: 13
System Thread ID: d10
Kernel Time: 0:0:4.186
User Time: 0:20:25.702
Thread Type: Managed Thread. Possible ASP.Net page or other .Net worker
No valid SOS data table found.

Begin System Thread Information

# ChildEBP RetAddr  
WARNING: Frame IP not in any known module. Following frames may be wrong.
00 01a9e76c 06ee91cd 0x6eea301
01 01a9eb6c 06cc6e91 0x6ee91cd
02 01a9ee44 06cc6278 0x6cc6e91
03 01a9ee80 06cc3ddd 0x6cc6278
04 01a9eeb8 06cc381c 0x6cc3ddd
05 01a9ef3c 0736fa0f 0x6cc381c
06 01a9ef84 0736f65f 0x736fa0f
07 01a9efec 0725853d 0x736f65f
08 01a9f050 79e9272b 0x725853d
09 01a9f060 79e926ad mscorwks!DllRegisterServerInternal+0x1d6b
0a 01a9f0e0 79e9450b mscorwks!DllRegisterServerInternal+0x1ced
0b 01a9f220 79e943f3 mscorwks!DllRegisterServerInternal+0x3b4b
0c 01a9f37c 07258408 mscorwks!DllRegisterServerInternal+0x3a33
0d 05b30b78 00000008 0x7258408

What does this mean? Well IIState doesn’t understand what’s going on with the stack below that mscorwks dll, which screams .Net 2 stack to you (as does the comment against Thread Type). But the likelihood is there’s not much else we’re going to get out of that file[1]. So it’s time to look at the mini dump.

Open a Visual Studio 2005 Command Prompt window, and from there launch C:\Program Files\Debugging Tools for Windows\windbg.exe (this ensures the .Net 2 framework is in the PATH for the WinDBG process, and makes life simpler –if you don’t have VS2005 installed do it manually). Do File \ Open Crash Dump and brose to the dmp file you created with IISState. If it asks you about saving workspace information just hit no.

It’ll open a text window with some summary information about the dump. This window has a little command bar at the bottom. This is where you do stuff.

The first thing it probably says is “Symbol search path is: *** Invalid ***”. So we type .symfix into the command bar to set it up to dynamically load symbols from Microsoft as required. You can do something to your %path% so you don’t have to do this each time, but I’ve not got round to that yet.

It probably also says

This dump file has an exception of interest stored in it.
The stored exception information can be accessed via .ecxr.
(13cc.84): Wake debugger - code 80000007 (first/second chance not available)

…but this is exactly what it says it is – it’s the console debugger attaching (how IISState got the dump), and not a ‘real’ exception per-se.

You can type ~ to get a list of the threads, or ~ *kb to get them with a bit of stack trace too, but we’re still only looking at unmanaged threads here, so pretty meaningless to me.

Type .load SOS to load the managed code debugging extension SOS.dll, then type !threads to see all the managed threads. Now that’s more like it.

NB: If you’re debugging on a different platform to the original crash (eg server was Win2003, you’re on XP), then you’re going to get an error at this point “Failed to load data access DLL, 0x80004005”. The error message contains the fix: execute .cordll -ve -u -l and the correct version of mscordacwks.dll for the dump will automatically be loaded, eg:

0:000> .cordll -ve -u -l
CLRDLL: C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\mscordacwks.dll:2.0.50727.42
f:0 doesn't match desired version 2.0.50727.91 f:0
CLRDLL: Loaded DLL C:\Program Files\Debugging Tools for Windows\sym\mscordacwks_x86_x86_2.0.50727.91.dll\442337B7562000
  \mscordacwks_x86_x86_2.0.50727.91.dll
CLR DLL status: Loaded DLL C:\Program Files\Debugging Tools for 
  Windows\sym\mscordacwks_x86_x86_2.0.50727.91.dll\442337B7562000\mscordacwks_x86_x86_2.0.50727.91.dll

Your !threads output might look like this:

ThreadCount: 9
UnstartedThread: 0
BackgroundThread: 9
PendingThread: 0
DeadThread: 0
Hosted Runtime: no
      ID OSID Thread   OBJ    State   GC       Context           Domain   Count APT Exception
      12    1  d10 000dce98   1808220 Disabled 00000000:00000000 000d4008     2 Ukn (Threadpool Worker)
      14    2  9f4 000ede10      b220 Enabled  00000000:00000000 000daa20     0 MTA (Finalizer)
      15    3  6d0 00103ad8    80a220 Enabled  00000000:00000000 000daa20     0 MTA (Threadpool Completion Port)
      16    4 14b0 001074d0      1220 Enabled  00000000:00000000 000daa20     0 Ukn
      10    5  4f8 0012e408   880a220 Enabled  00000000:00000000 000daa20     0 MTA (Threadpool Completion Port)
      18    9 1184 06a32e20    800220 Enabled  00000000:00000000 000daa20     0 Ukn (Threadpool Completion Port)
      26    6 15a0 06ada200   180b220 Enabled  00000000:00000000 000daa20     0 MTA (Threadpool Worker)
      28    b   e0 06a5bd40   880b220 Enabled  00000000:00000000 000daa20     0 MTA (Threadpool Completion Port)
      29    c  fd0 06a3c368   180b220 Enabled  00000000:00000000 000daa20     0 MTA (Threadpool Worker)

Notice how the last column tells you what type of thread it is. You can get the managed stack trace for each one of these threads by selecting which thread you’re interested in from the Processes and Threads window (up on the toolbar) and typing !clrstack, but I find it more useful to just look at all of them straight off: ~*e !clrstack

You’ll see a lot of this:

OS Thread Id: 0x84 (0)
Unable to walk the managed stack. The current thread is likely not a managed thread. 
You can run !threads to get a list of managed threads in the process

...because what you're doing is looking at the managed stack trace for all threads (and many of them won't be .Net threads remember), but eventually you’ll see some managed stack traces pop up.

In my case it was a bit of an open-and-shut case, because there was only one .Net thread actually doing anything in the process – kinda a giveaway:

OS Thread Id: 0xd10 (12)
ESP       EIP     
[… trimmed 3rd party vendor stack trace…]
01a9eec4 06cc381c Aspose.Pdf.Pdf.Save(System.IO.Stream)
01a9eed4 069d4a05 MyApp.Server.Utils.WordToPdf.AsposeImpl.Generate(System.IO.MemoryStream)
01a9ef44 0736fa0f MyApp.Server.Document.Implementations.AbstractWordToPdfDocumentGenerationImpl.Generate(...)
01a9ef8c 0736f65f MyApp.Server.Document.GenerationManager.Generate(...)
01a9eff4 0725853d MyApp.Server.ServiceImplementation.DocumentService.GenerateDocumentById(GenerateDocumentByIdRequest)
[… removed ASP.Net framework stack trace …]
01a9f710 65fbe244 System.Web.HttpRuntime.ProcessRequestInternal(System.Web.HttpWorkerRequest)
01a9f744 65fbde92 System.Web.HttpRuntime.ProcessRequestNoDemand(System.Web.HttpWorkerRequest)
01a9f750 65fbc567 System.Web.Hosting.ISAPIRuntime.ProcessRequest(IntPtr, Int32)
01a9f900 79f2d05b [ContextTransitionFrame: 01a9f900] 
01a9f950 79f2d05b [GCFrame: 01a9f950] 
01a9faa8 79f2d05b [ComMethodFrame: 01a9faa8] 

[NB: I've had to edit that for brevity and wrapping]

Ok, so we’re dying in a 3^rd party component. But we’re still going to need to replicate this somehow because (presumably) it doesn’t do this all the time. Fortunately in this case all we need to know is the arguments supplied to our GenerateDocumentById() method to attempt to reproduce in dev. They’re in the dump too, so let’s see what they were.

Select that thread (12) using ~12s (or the Processes and Threads window, and no, I don’t know what the ‘s’ suffix means) and use !clrstack –p to view the stack with parameters. It’s a big list, but in there somewhere is the bit I’m interested in:

0:012> !clrstack –p
[… blah blah …]
01a9eff4 0725853d MyApp.Server.ServiceImplementation.DocumentService.GenerateDocumentById(GenerateDocumentByIdRequest)
     PARAMETERS:
          this = 0x02e596b0
          generateDocumentByIdRequest = 0x02e595b4

!dumpobject (or !do) then tells me the contents of that generateDocumentByIdRequest instance given it’s address as shown above (0x02e595b4):

0:012> !do 0x02e595b4
Name: MyApp.Server.ServiceContracts.GenerateDocumentByIdRequest
MethodTable: 05dcb5e0
EEClass: 06f935ec
Size: 12(0xc) bytes
   (C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\Temporary ASP.NET Files\blah blah blah\MyApp.Server.ServiceContracts.DLL)
Fields:
MT        Field    Offset    Type    VT Attr      Value   Name
790fedc4  40001c4  4         System.Int32 0  instance  8229991 documentId

The bit I care about on this is the documentId field itself, and being a value-type field (Int32) I can see the value right there: 8229991 (for a reference type like System.String I'd get another address to the instance itself, ie have to perform another !do).

And in this case, that was that. Well, it turned out to affect more than just this document ID, but at least I had enough to reproduce in the dev environment. And just in case anyone read the stack trace carefully, by the time we got to this stage Aspose already had a fix out for this issue, so I can’t really knock them. Finally whilst I was at it, I had a look at the objects in memory and began to wonder whether I’ve got a memory leak, so there’s probably a follow up to come.

Other SOS commands to look at:

!threadpool (also this is where the CPU utilization hides)
!eeheap –gc (see memory allocations that are subject to the GC)
!dumpheap –stat (see object allocations by type)
!help (gives help on the last extension loaded, so SOS commands in our case)

Other Resources:

Installing and Configuring IISState

SOS: It’s not just an ABBA Song Anymore: A good introductory article on MSDN with a dreadful title

If Broken It Is… Fix it You Should [Tess’s Blog]: This will all make a lot more sense now

Production Debugging for .NET Framework Applications: This 170 page document from Microsoft’s Patterns and Practices team is allegedly out of date, but is an invaluable reference, and what I based most of this article on

Debugging Microsoft® .NET 2.0 Applications: I haven’t read this MSPress book, but the v1 version got quite a lot of citations when I was researching this stuff

---

[1] So why did we take it? Because if it turns out it’s an IIS issue, and not your code, then someone will be able to get something out of it. Just not you.

Nesting Transactions in Stored Procedures (Part 1)

Nesting transactional stored procedures is an absolute minefield.

Many projects I've worked on have just avoided opening transactions within stored procs altogether, and rely entirely on ADO / ADO.Net transactions in the application. This is certainally a simple way to avoid the problem, but isn't always appropriate. At some point there's always a data integrity rule that you want enforced at the lowest possible level (to avoid it being accidentally bypassed - I'm thinking auditing, or the classic 'move money between accounts' scenario). Putting that rule in a stored proc, and making that the only point of call for writes to the table involved makes it pretty safe. This is even more important if it's not just your app using the database.

But nevertheless avoidance is appropriate in many cases, if only because handling the nesting of transactions within nested stored proc calls is a complex matter. To summarize the problems:

Rollback and commit are asymmetric in nested transaction scenarios

Rollback aborts the whole nested stack, whereas commit just decreases the nesting level (only the final commit actually commits anything). So error handling has to behave differently in the proc at the 'root' of the transaction than it does in nested procs. Ulp!

SQL error handling is hard, and poorly-understood

Even basic TSQL error handling is poorly understood: having an exception raised in the calling application but the rest of the stored proc continue executing is pretty counter-intuitive, right? Whadyamean the transaction committed anyway!?!

The fact is that most SQL errors don't affect batch completion without explicit handling, some abort the batch but not the transaction, and some abort both. Furthermore, XACT_ABORT changes which do which. Erk!


Fortunately we don't have to come up with a solution, because there are already well-defined patterns for handling both these issues. There's an excellent article on the subject on Code magazine that's been the basis for my approach for a few years now:

Handling SQL Server Errors in Nested Procedures by Ron Talmage (I use the multi-level model)

Unfortunately barely any developers appear to be aware of the problem, let alone the solution patterns. This is particularly problematic, because either of Ron's solutions must be applied consistently in order to work properly. If that's an issue, maybe you're best going back to the avoidance pattern.

Anyway, the big question for me is "how has this pattern been affected by TRY ... CATCH functionality in Sql Server 2005?". We'll look at that next time...

How to Disable Kerberos on Windows 2003 using NTAuthenticationProviders

If your IIS website / service

• Is running as an account other than NetworkService
  
• or isn't being accessed via the server's AD name (eg through an alternate URL, or load balanced alias)

...then Kerberos authentication will fail, because the client doesn't know who the server is to start the ticket exchange process.

In both cases getting the right SPN added into AD will fix things, but sometimes this can be problematic. However you can explicitly downgrade the virtual directory to only use NTLM authentication by setting the NTAuthenticationProviders property. There's no UI for this, so you have to set it on your IIsWebVirtualDir in the metabase, eg (for IIS 6 \ Windows 2003):

cscript C:\inetpub\adminscripts\adsutil.vbs //nologo SET W3SVC/1/Root/MyApp/NTAuthenticationProviders NTLM

Then everything (?) works again. Hooray.

This is discussed at the very bottom of this article: http://support.microsoft.com/kb/215383
NB: For Windows 2000 you can only set this at a site-level, not an application level, as the article outlines.

Working Directory Independence for Batch Files

%~p0\

(or: Many times your batch file will want to access resources in the same folder as the batch file. This can be tricky if the user calls the batch file from another folder, since the working directory is not the directory the batch file is in. Rather that resorting to pushd / popd everywhere, one can use the extended command line parameter handling to convert the full path to the batch file into a relative path to it’s container. %0 is the path the batch was called with, so %~p0 is the path to that location)

Blogsphere faux pas

Don't confuse Atwood and Achewood

Access denied when piping to :Null

It's quite a common thing in my batch files (yes, .BAT is still alive) to limit the verbosity of the output displayed to the user. It's a 'seeing-the-wood-for-the-trees' thing: I don't want to have the user waide through screens of output to know that a CACLS worked ok (and potentially miss something important)

So I quieten things down by piping to null:

aspnet_setreg -k:SOFTWARE\MyCompany\MyApp -u:"%username%" -p:"%password%" >:null

...and listen for ERRORLEVEL instead.

But recently I discovered that this fails with Access Denied if you don't have write permissions to the current directory (yes, you need write permissions to write to :null. Just don't ask). Not only that, but the whole statement fails, not just the piping bit.

So these days I pipe to %temp%\null instead.

When is a type not public?

[Keith's one this]

...when it's a nested type. So you have to do:

if(type.IsPublic || type.IsNestedPublic){
   // do stuff to public types
}

I can't even remember what we were doing when we ran into this, which (to me) indicates I'd better blog it before I forget about it altogether.

Encryption of connection credentials in .Net

In which I discuss techniques to manage the storage of a secret configuration setting (connection credentials) on a production webserver, when the development team (and their automated build/depoy process) shouldn't actually know the secret itself (and especially shouldn't store it in source control).

It's a bit of a ramble to myself as I think this stuff through.

What's new in .Net 2

In .Net 2, the configuration infrastructure includes support for encrypted connection strings out-of the box. One can simply call aspnet_regiis to encrypt (almost) any given section of a configuration file:

aspnet_regiis -pe "connectionStrings" -app "/MachineDPAPI" -prov "DataProtectionConfigurationProvider"

Pasted from <http://msdn2.microsoft.com/en-us/library/ms998280.aspx>

However

• This encrypts the whole section, which makes inspection and verification of other settings harder
• This requires the .config file to have the credentials in in the first place
• So either the credentials sit in source control at the dev end :-(
• Or the install process has to both prompt for them, and add them into the web.config before encryption. This is a pain, but manageable.

A variation on this technique is available for web farm scenarios, where the config is encrypted with a custom exportable RSA key.

aspnet_regiis -pe "connectionStrings" -app "/WebFarmRSA" -prov "CustomProvider"

aspnet_regiis -px "CustomKeys" "C:\CustomKeys.xml" -pri

Pasted from <http://msdn2.microsoft.com/en-us/library/ms998283.aspx>

Apart from being web-farm aware, this technique has the advantage that the public part of the key could be exported to the build server, and the settings encrypted prior to deployment

However this still means the build server (and by inference source control) contains the production credentials in plain text. This is generally what we want to avoid most - internal information leakage is far more likely than a compromise of the production environment, especially in an intranet scenario.

How did we used to manage this in Net 1.1?

Under .Net 1.1 the approach was to use the ASPNET_SETREG tool to move settings into the registry, under DPAPI encryption, then adjust the configuration using a special syntax that said 'read this from the registry instead'.

aspnet_setreg.exe -k:SOFTWARE\MY_SECURE_APP\identity -u:"yourdomainname\username" -p:"password"

Pasted from <http://support.microsoft.com/kb/329290>

identity impersonate="true"
userName="registry:HKLM\SOFTWARE\MY_SECURE_APP\identity\ASPNET_SETREG,userName"
password="registry:HKLM\SOFTWARE\MY_SECURE_APP\identity\ASPNET_SETREG,password"

Pasted from <http://support.microsoft.com/kb/329290>

This has the following advantages:

• Secret only needs to be known once, on initial install, and is thereafter stored in the registry
• Source control and build process doesn't need to know the secret
• Same config file deployed to both boxes in a web farm

Whilst under .net 1.1 this was only supported for specific keys:

identity userName= password=

processModel userName= password=

sessionState stateConnectionString= sqlConnectionString=

Pasted from <http://support.microsoft.com/kb/329290>

…a common pattern, and one I've used a couple of times, was to use aspnet_setreg to put arbitrary settings into the registry in an encrypted manner, and use custom DPAPI code to retrieve them. There was no equivalent of the DPAPI wrapper class 'DataProtection' in .Net 1.1, so Keith Brown provided one:

"Version 2.0 of the .NET Framework introduces a class called DataProtection that wraps DPAPI. It's simple to use; in fact, it looks almost exactly like the wrapper class I provided above. I've shown an example in figure 70.2."

Pasted from <http://pluralsight.com/wiki/default.aspx/Keith.GuideBook.HowToStoreSecretsOnAMachine>

So where are we going with this?

Well to my mind at least the .Net 1.1 registry approach was actually better than the .Net 2 model - it separated the configuration and the secret in a way that perfectly reflected the normal division of labour between development and infrastructure teams (where the devs are expected to provide the configs, but they shouldn't know the secret).

There should be no reason why one cannot use the .Net 1.1 technique (ie use ASPNET_SETREG to store the secret), whilst still using the .Net 2 DataProtection class to unencrypt it.

• This does require careful monitoring of the ACL on the registry key in question
• There's no real requirement to use ASPNET_SETREG over a custom exec (using the DPAPI), bar convenience. That having been said a custom EXE could set the registry key ACL at the same time, which I remember as being a headache otherwise.