ThreadStatic, CallContext and HttpContext in ASP.Net

Summary:
Even if you think you know what you're doing, it is not safe to store anything in a ThreadStatic member, CallContext or Thread Local Storage within an ASP.Net application, if there is the possibilty that the value might be setup prior to Page_Load (eg in IHttpModule, or page constructor) but accessed during or after.

[Update: Aug 2008 In view of the fairly large number of people continuing to link to this post I feel the need to clarify that this thread-swapping behaviour happens at a very specific point in the page lifecycle and not whenever-it-feels-like-it. My wording after the Jef Newson quote was unfortunate. That aside, I've been immensely gratified (and flattered) by the number of times I've seen this post cited within design discussions around dealing appropriately with HttpContext. I'm glad people found it useful.]

There's a lot of confusion about using how to implement user-specific singletons in ASP.Net - that is to say global data that's only global to one user or request. This is not an uncommon requirement: publishing Transactions, security context or other 'global' data in one place, rather than pushing it through every method call as tramp data can make for a cleaner (and more readable) implementation. However its a great place to shoot yourself in the foot (or head) if you're not careful. I thought I knew what was going on, but I didn't.

The preferred option, storing your singletons in HttpContext.Current.Items, is simple and safe, but ties the singleton in question to being used within an ASP.Net application. If the singleton's down in your business objects, this isn't ideal. Even if you wrap the property-access in an if statement

if(HttpContext.Current!=null){
  /* store in HttpContext */
}else{
  /* store in CallContext or ThreadStatic */
}

... then you've still got to reference System.Web from that assembly, which tends to encorage more 'webby' objects in the wrong place.

The alternatives are to use a [ThreadStatic] static member, Thread local storage (which pretty much amounts to the same thing), or CallContext.

The problems with [ThreadStatic] are well documented, but to summarize:

• Field initalizers only fire on the first thread
  
• ThreadStatic data needs explicit cleaning up (eg in EndRequest), because whilst the Thread's reachable, the ThreadStatic data won't be GC'd so you might be leaking resources.
  
• ThreadStatic data is only any good within a request, because the next request might come in on a different thread, and get someone else's data.
  

Scott Hanselman gets it right, that ThreadStatic doesn't play well with ASP.Net, but doesn't fully explain why.

Storage in CallContext alleviates some of these problems, since the context dies off at the end of the request and GC will occur eventually (though you can still leak resources until the GC happens if you're storing Disposables). Additionally CallContext is how HttpContext gets stored, so it must be ok, right?. Irrespective, you'd think (as I did) that provided you cleaned up after yourself at the end of each request, everthing would be fine:

"If you initialize a ThreadStatic variable at the beginning of a request, and you properly dispose of the referenced object at the end of the request, I am going to go out on a limb and claim that nothing bad will happen. You're even cool between contexts in the same AppDomain

"Now, I could be wrong on this. The clr could potentially stop a managed thread mid-stream, serialize out its stack somewhere, give it a new stack, and let it start executing. I seriously doubt it. I suppose that it is conceivable that hyperthreading makes things difficult as well, but I also doubt that."
Jef Newsom

Update: This was the misleading bit. I do explain further later on that this thread-swap can only happen between the BeginRequest and the Page_Load, but Jef's quote creates a very powerful image I failed to immediately correct. My bad.

Trouble is that's exactly what happens. Trouble is that's almost what happens. Under load ASP.Net can migrate inbound requests from its IO thread pool to a queue taken up by it's worker process thread pool:

So at some point ASP.NET decides that there are too many I/O threads processing other requests. [...] It just takes the request and it queues it up in this internal queue object within the ASP.NET runtime. Then, after that's queued up, the I/O thread will ask for a worker thread, and then the I/O thread will be returned to its pool. [...] So ASP.NET will have that worker thread process the request. It will take it into the ASP.NET runtime, just as the I/O thread would have under low load.
Microsoft ASP.NET Threading Webcast

Now I always knew about this, but I assumed it happened early enough in the process that I didn't care. It appears however that I was wrong. We've been having a problem in our ASP.Net app where the user clicks one link just after clicking another, and our app blows up with a null reference exception for one of our singletons (I'm using CallContext not ThreadStatic for the singleton, but it turns out it doesn't matter).

I did a bit of research about how exactly ASP.Net's threading works, and got conflicting opinions-masquerading-as-fact (requests are thread-agile within a request vs requests are pinned to a thread for their lifetime) so I replicated my problem in a test application with a slow page (sleeps for a second) and a fast page. I click the link for the slow page and before the page comes back I click the link for the fast page. The results (a log4net dump of what's going on) surprised me.

What the output shows is that - for the second request - the BeginRequest events in the HttpModule pipeline and the page constructor fire on one thread, but the Page_Load fires on another. The second thread has had the HttpContext migrated from the first, but not the CallContext or the ThreadStatic's (NB: since HttpContext is itself stored in CallContext, this means ASP.Net is explicitly migrating the HttpContext across). Let's just spell this out again:

• The thread switch occurs after the IHttpHandler has been created
• After the page's field initializers and constructor run
  
• After any BeginRequest, AuthenticateRequest, AquireSessionState type events that your Global.ASA / IHttpModules are using.
• Only the HttpContext migrates to the new thread
  

This is a major PITA, because as far as I can see it mean the only persistence option for 'ThreadStatic'esque behavior in ASP.Net is to use HttpContext. So for your business objects, either you're stuck with the if(HttpContext.Current!=null) and the System.Web reference (yuck) or you've got to come up with some kind of provider model for your static persistence, which will need setting up prior to the point that any of these singletons are accessed. Double yuck.

Please someone say it ain't so.

Appendix: That log in full:

[3748] INFO  11:10:05,239 ASP.Global_asax.Application_BeginRequest() - BEGIN /ConcurrentRequestsDemo/SlowPage.aspx
[3748] INFO  11:10:05,239 ASP.Global_asax.Application_BeginRequest() - threadid=, threadhash=, threadhash(now)=97, calldata=
[3748] INFO  11:10:05,249 ASP.SlowPage_aspx..ctor() - threadid=3748, threadhash=(cctor)97, threadhash(now)=97, calldata=3748, logicalcalldata=3748
[3748] INFO  11:10:05,349 ASP.SlowPage_aspx.Page_Load() - threadid=3748, threadhash=(cctor)97, threadhash(now)=97, calldata=3748, logicalcalldata=3748
[3748] INFO  11:10:05,349 ASP.SlowPage_aspx.Page_Load() - Slow page sleeping....

[2720] INFO  11:10:05,669 ASP.Global_asax.Application_BeginRequest() - BEGIN /ConcurrentRequestsDemo/FastPage.aspx
[2720] INFO  11:10:05,679 ASP.Global_asax.Application_BeginRequest() - threadid=, threadhash=, threadhash(now)=1835, calldata=
[2720] INFO  11:10:05,679 ASP.FastPage_aspx..ctor() - threadid=2720, threadhash=(cctor)1835, threadhash(now)=1835, calldata=2720, logicalcalldata=2720, threadstatic=2720

[3748] INFO  11:10:06,350 ASP.SlowPage_aspx.Page_Load() - Slow page waking up....
[3748] INFO  11:10:06,350 ASP.SlowPage_aspx.Page_Load() - threadid=3748, threadhash=(cctor)97, threadhash(now)=97, calldata=3748, logicalcalldata=3748
[3748] INFO  11:10:06,350 ASP.Global_asax.Application_EndRequest() - threadid=3748, threadhash=97, threadhash(now)=97, calldata=3748
[3748] INFO  11:10:06,350 ASP.Global_asax.Application_EndRequest() - END /ConcurrentRequestsDemo/SlowPage.aspx

[4748] INFO  11:10:06,791 ASP.FastPage_aspx.Page_Load() - threadid=2720, threadhash=(cctor)1835, threadhash(now)=1703, calldata=, logicalcalldata=, threadstatic=
[4748] INFO  11:10:06,791 ASP.Global_asax.Application_EndRequest() - threadid=2720, threadhash=1835, threadhash(now)=1703, calldata=
[4748] INFO  11:10:06,791 ASP.Global_asax.Application_EndRequest() - END /ConcurrentRequestsDemo/FastPage.aspx

The key bit is what happens when FastPage's Page_Load fires. The ThreadID is 4748, but the threadID I stored in HttpContext in the ctor is 2720. The hash code for the logical thread is 1703, but the one I stored in the ctor is 1835. All data I stored in the CallContext is gone (even that marked ILogicalThreadAffinative), but HttpContext is still there. As you'd expect, my ThreadStatic is gone too.