Mysterious Hang or The Great Deception of InvokeRequired

The Bug: Your Application Hangs

The Cause
Sample Application
The Solution
See Also

The Bug: Your Application Hangs!

On some not so lucky day several months ago I received a couple of user complaints that my Windows Forms application mysteriously hangs every now and then. This proved to be quite hard to reproduce. The hang occured infrequently and tended to happen when the user walks away from the computer. Even worse, the hang never occured under debugger. Once hung, the application window froze completely: it was not even possible to move it.

The version of the application with the hang was running under .NET 2.0. Interestingly, the older, .NET 1.1 version of the same application never hung.

Once we were able to attach debugger to the hung application, we found that the main GUI thread is stuck inside SystemEvents.OnUserPreferenceChanged() method waiting forever on some strange event. This was a big hint that allowed us to reproduce the problem almost at will. OnUserPreferenceChanged() is called in response to the WM_SETTINGCHANGE message. The application hangs when we change background bitmap, or do something else that causes broadcast of this message to all top-level Windows in the system.

The Cause

It's Nobody's Fault, Really

After several days of googling, intriguing debugging sessions, and examining tons of Windows Forms code via .NET reflector we discovered the chain of events that leads to the hang. It is definitely not a simple bug. It is a bizzare set of relatively innocent circumstances, that, when combined, lead to the deadly outcome. This is what makes this problem interesting.

We proceed by describing the pieces of the puzzle one by one, and then we will see how they work together. The contributors to the problem are:

Creation on Win32 window handls is deffered
InvokeRequired may return unexpected results
In .NET 2.0 each thread is assigned a synchronization context
In .NET 2.0 system events are handled using thread synchronization contexts
Some controls automatically subscribe to system events

Due to all of the above, GUI controls may be created on a worker thread that has wrong synchronization context. These controls subscribe to UserPreferenceChanged system event and it leads to blocking of main GUI thread when a user preference change occurs.

Deferred Creation of Win32 Windows Handles

In an attempt to improve performance, .NET defers creation of real Win32 windows as much as possible. Thus, if we simply do something like this:

Control ctrl = new Control();

it does not cause creation of Win32 window. Window handle will be created only when the control needs to be actualy shown on screen, or when the programmer explicitly requests the control's handle via the Handle property.

ctrl.Show(); // creates handle if control actually becomes visible
IntPtr handle = ctrl.Handle; // creates handle unconditionally

InvokeRequired May Lie to You

Here is a typical code snippet for marshalling notifications to the GUI thread:

delegate void MyHandlerDelegate(); void MyHandler() { // "The BeginInvoke dance" if (this.InvokeRequired) // assuming this descends from Control { BeginInvoke( new MyHandlerDelegate(MyHandler) ); return; } // assume we are on the main GUI thread ... do GUI stuff ... }

This code works fine most of the time, but not all the time. How does Control.InvokeRequired know whether invoke is required? We can find the answer by looking at reflector. Here is how (simplified) code for InvokeRequired looks like:

public bool InvokeRequired
{
    get
    {
        HandleRef hWnd;
        int lpdwProcessId;
        if (this.IsHandleCreated)
        {
            hWnd = new HandleRef(this, this.Handle);
        }
        else
        {
            Control wrapper = this.FindMarshalingControl();
            if (!wrapper.IsHandleCreated)
            {
                return false; // <==========
            }
            hWnd = new HandleRef(wrapper, wrapper.Handle);
        }
        int windowThreadProcessId = SafeNativeMethods.GetWindowThreadProcessId(hWnd, out lpdwProcessId);
        int currentThreadId = SafeNativeMethods.GetCurrentThreadId();
        return (windowThreadProcessId != currentThreadId);
    }
}

The code first locates a suitable Win32 window handle. If current control does not have a handle, FindMarshallingControl() will walk through the control's parents until it either finds a parent with a valid handle, or reaches the end of the hierarchy. The code then obtains Win32 thread ID of the handle's GUI thread and compares that thread ID to the ID of the current Win32 thread. Invoke is required if two IDs are not the same. An interesting thing happens if a suitable window handle is not found.

When neither the control nor any of its parents have a valid window handle, InvokeRequired returns false unconditionally.

In fact, there is no any other choice. There is no information about the control's window thread. When (and if) control's Win32 handle is created, that thread will become control's window thread. But this did not happen yet. If InvokeRequired returned true, user probably will do BeginInvoke() or Invoke() on the control, which will cause exception. Thus, returning false seems like the only reasonable choice under the circumstances.

This will obviously wreck havoc if a code doing the BeginInvoke dance is executed on a non-GUI thread. The "if" condition will return false, and the "GUI stuff" will end up being executed on a non-GUI thread leading to potentially disastrous concequences.

In real life, a situation when InvokeRequired cannot find a window handle may occur in at least two scenarios:

Early in the program startup, e.g. before calling Application.Run()
Early in the life of the control, e.g. while executing the control's constructor

A control that belongs to a form is typically initialized as follows:

class MyForm : Form
{
    MyControlClass myControl;

    public MyForm()
    {
        InitializeComponent();
    }

    void InitializeComponent()
    {
        myControl = new MyControlClass();    
        // at this point myControl has no window handle and no parent
        this.Controls.Add(myControl);
    }
   
}

While control's constructor is executed, the control has no window handle and no parent. If the constructor spawns an asynchronous handler that calls InvokeRequired from a non-GUI thread, there is a race condition. If the constructor exits and the control gets its parent before the handler calls InvokeRequired everything works fine. If, however, the constructor lingers on, and the handler calls InvokeRequired while the control is still an orphan, InvokeRequired will return false and bad things will happen.

Synchronization Contexts

.NET 2.0 introduces a new class System.Threading.SynchronizationContext. It is supposed to generalize thread synchronization mechanisms like Control.BeginInvoke. Each thread has a (lazy-initialized) synchronization context which is retrieved via SynchronizationContext.Current. The programmer can then do things like context.Post(someDelegate), which is supposed to be an asynchronous call, and context.Send(someDelegate), which supposed to be a synchronous, blocking call.

SynchronizationContext defines both the interface for synchronization contexts and the default implementation. In this default implementation Post() calls ThreadPool.QueueUserWorkItem() and Send() simply calls the parameter delegate.

Windows Forms define a special implementation of SynchronizationContext called WindowsFormsSynchronizationContext. In this implementation Post() calls BeginInvoke() on a special "marshalling control", and Send() calls Invoke() on the same control.

Windows Presentation Foundation and ASP.NET define their own implementations of SynchronizationContext.

If a thread does not specifically setup synchronization context and someone calls SynchronizationContext.Current, default implementation will be returned, something close to new SynchronizationContext(). However,

Creating any Windows Forms control on the thread automatically installs WindowsFormsSynchronizationContext

as current synchornization context. This is done in the constructor of the Control class.

Handling System Events

Microsoft.Win32.SystemEvents class defines a number of static events such as UserPreferenceChanged.

In .NET 1.1 these are regular C# events. Each event keeps a list of subscriber delegates. When specific event needs to be raised, subscriber delegates are directly called one by one. It is the subscriber's responsibility to implement any necessary thread synchronization.

.NET 2.0 tries to reduce the burden on the subscriber and builds certain thread synchronization into the system. Instead of keeping a plain list of subscriber delegates, in .NET 2.0 each event keeps list of delegates and corresponding thread contexts. In other words, when someone does

SystemEvents.UserPreferenceChanged += subscriberDelegate;

custom subscription code calls SycnhronizationContext.Current and adds a structure containing subscriberDelegate delegate and the synchronization context to the list of subscribers. When specific event needs to be raised, the system calls context.Send(subscriberDelegate) for each subscriber in order.

Keep in mind that this applies only to the events in the SystemEvents class, not to all events everywhere. SystemEvents class has custom add and remove handlers for its events that implement this behavior.

Automatic Subscription to System Events

Some Windows Forms controls automatically subscribe to SystemEvents.UserPreferenceChanged in their OnHandleCreated() method, or in other methods. This includes:

Form and all top-level controls
DataGridView
DateTimePicker
DomainUpDown
MonthCalendar
NumericUpDown
ProgressBar
RichTextBox

So, it is enough to execute code like this:

IntPtr handle = new Form().Handle;

new Form().Show();

to create subscription to SystemEvents.UserPreferenceChanged. Note, however, that it is not even necessary to create the form directly in your code. Someone else's code may do this for you. E.g., Infragistics library creates a hidden form that it uses for device context measurements. Thus, you may be doing an innocent looking Infragistics call, but it will create a Windows Form and subscribe to the UserPreferenceChanged event.

Putting It All Together

We have examined all pieces of the puzzle and are finally ready to see the whole picture. Here is the sequence of events that leads to the mysterious hang:

A control is created on the GUI thread. Creation of the window handle is deferred.

There is code on a worker thread that does the BeginInvoke dance.
The worker thread code is called while neither the control nor any of its parents have window handles.
InvokeRequired returns false on the worker thread.
The worker thread proceedes with the GUI calls.
One or more controls are created on the worker thread. This installs WindowsFormsSynchronizationContext on the thread.
Some GUI calls lead, directly or indirectly, to subscription to the SystemEvents.UserPreferenceChanged event.
SystemEvents class stores current synchronization context, which is WindowsFormsSynchronizationContext, along with the subscriber delegate.
User preference change occurs, e.g. desktop background bitmap is changed.
Windows procedure on the main GUI thread receives WM_SETTINGCHANGE message.
The code in SystemEvents class raises UserPreferenceChanged event. It examines the list of subscribers and calls Send() on the stored thread context.
WindowsFormsSynchronizationContext.Send() sends a message to the worker thread and blocks until the message is processed.
Since the worker thread does not have a message loop, the message never gets processed.
The main GUI thread is blocked forever. The application hangs.

What Happens under .NET 1.1

Under .NET 1.1 InvokeRequired behaves essentially in the same way. Thus, it may lie to you just like under .NET 2.0 and lead to execution of GUI code on a non-GUI thread.

From the other hand, .NET 1.1 does not have a concept of a thread synchronization context. The SystemEvents class stores plain delegates and calls them directly when a system event occurs. Therefore, while the behavior under .NET 1.1 is still dangerous, the mysterious hanging bug does not occur under .NET 1.1. However, if we take a .NET 1.1 program and run it under .NET 2.0 using <supportedRuntime> configuration attribute, the bug will occur.

Sample Application

Sample application that demonstrates the mysterious hanging bug: MysteriousHang.zip (32K).

Freezer

Freezer is a program that broadcasts a WM_SETTINGCHANGE message to all top-level windows in the system. This freezes applications that have the mysterious hanging bug. The version of .NET under which Freezer runs is not important.

MysteriousHang

MysteriousHang is an actual application that demonstrates the mysterious hanging bug. It is centered around the following piece of code:

private void WorkerThreadFunc()
{
    if (InvokeRequired)
    {
        BeginInvoke(new MethodInvoker(WorkerThreadFunc), null);
        return;
    }

    IntPtr handle = new Form().Handle;
}

This code is executed on a non-GUI thread and leads to the mysterious hanging bug under the right circumstances.

Structure of The Sample

In order to demonstrate all possible combinations, we need two solutions: MysteriousHang.sln for VS 2003 and MysteriousHang_2_0.sln for VS 2005. The former solution includes Freezer and MysteriousHange projects, the latter includes only MysteriousHange project, since one version of freezer is enough for our needs.

.NET 2.0 binary files go to the bin_2_0 directory. Unfortunately, there is no way to change the obj directory, so it is shared between the two versions. If you get internal compiler errors when compiling under VS 2003, manually clean the obj directory.

Reproducing the Bug

The mysterious hanging bug occurs only under .NET 2.0 and when the process is not debugged using VSHost. Different situations are summarized in the table below:

Compiler	Runs under	VS Host	Hangs
VS 2003	.NET 1.1	no	no
VS 2003	.NET 2.0	no	yes
VS 2005	.NET 2.0	no	yes
VS 2005	.NET 2.0	yes	no

Notes:

VS Host is not supported under VS 2003.
Programs compiled with VS 2005 cannot run under .NET 1.1.

To reproduce the bug:

Run MysteriousHang from the bin_2_0 folder. Make sure it says it will hang.
Run Freezer.
Press the "Freeze 'em! button
Wait until the hourglass cursor goes away (this may take a while).
MysteriousHang window is now frozen

To force .NET 1.1 application to run under .NET 2.0 uncomment the <supportedRuntime version="v2.0.50727"/> fragement in the application configuration file.

The Solution

A couple of weeks ago I received an e-mail telling me that I explained the problem very well, but the article never mentions the solution. This paragraph fixes the bug. :-)

There are several ways to avoid or at least detect the lying InvokeRequired. First of all, use early detection. If you believe that certain piece of code must execute only on the main GUI thread - verify that. The easiest way to do so is to give the main GUI thread a name and verify that the current thread has that name:

void Main()
{
    Thread.Current.Name = "MainGUI";
    ...
}

void EnsureMainGuiThread()
{
    if (Thread.Current.Name != "MainGUI")
    {
        throw new InvalidOperationException("This code must execute on the main GUI thread");
    }
}

void SomeMethod()
{
    if (InvokeRequired)
    {
        BeginInvoke(...);
        return;
    }

    EnsureMainGuiThread();
    ...
}

There are also some active steps you can take to prevent bad things from happening:

Force creation of the main form's Win32 handle as early as possible by calling its Handle property. You want the handle to exist before you make your first call to InvokeRequired.
Avoid calling InvokeRequired on child controls. Always use main window. If you don't want to depend on the main window class, pass main window instance as ISynchronizeInvoke interface.
Be especially careful when you do something asynchronously from your control's constructor. When you are in the constructor, the control is not yet attached to its parent, so InvokeRequired may lie to you, even if the parent has valid Win32 handle.