Observing Unhandled Exceptions
In .NET unhandled exceptions can be observed via AppDomain.UnhandledException event:
AppDomain.CurrentDomain.UnhandledException += myHandler; // C#
In Win32 unhandled exceptions are observed via SetUnhandledExceptionFilter call:
SetUnhandledExceptionFilter(myfilter); // C/C++
Is native or managed exception filter called first?
In a mixed native/managed environment it is possible to setup both filters, which begs the question which filter would be called first, and whether the second filter would be called at all.
The answer to this question is complicated and does not seem to be documented anywhere. The information below is for .NET Framework 4.0 and 4.5.
If a managed exception occurs in the main (first) thread of the application, native handler is called first. If native handler was not set, or if it returns EXCEPTION_CONTINUE_SEARCH, managed handler is called second. Otherwise managed handler is not called at all.
If a managed exception occurs in any other thread, including secondary UI threads, managed handler is called first, followed by native handler.
If a managed or native debugger is attached, managed handler is called first, and native handler is not called at all, for all threads. This makes the situation especially treacherous, since the behavior with and without debugger is different for main thread.
I am not sure whether this behavior is a bug or a feature. When initialized, CLR installs its own Win32 exception filter implemented by the CLRUEFManager class in mscoree.dll. “UEF” apparently stands for Unhandled Exception Filter. This class decides when to call managed unhandled exception logic and when to call “old” user-installed Win32 exception filter. To the best of my knowledge, the source code of this class is not public, and the reason why it works the way it works is not publicly known.
Strategy for reliably observing unhandled exceptions
If your application is mixed (managed/native), and the native part installs a Win32 unhandled exception filter, reliably observing managed unhandled exception from the main thread becomes a problem. Win32 UEF filter typically returns EXCEPTION_EXECUTE_HANDLER, because it has no reason to do otherwise. In fact, in our production application it went even further and called TerminateProcess(). For main thread Win32 filter is called first, and thus there is a very high chance that managed UnhandledException event will not be called.
This problem does not exist for secondary threads, because the managed exception filter typically does not terminate the application in its body, so both filters will be called as expected.
You can try to prevent unhandled exceptions altogether by putting try/catch on every managed/unmanaged boundary. However, I would not call this strategy reliable. It is complicated if you have large body of code, and pretty much impossible if you deal with mixed third party libraries.
Another strategy is to ensure Win32 unhandled exceptions filter always returns EXCEPTION_CONTINUE_SEARCH. This may be problematic if you deal with third party libraries that install their own UEF filters. However, even in this case not all is lost: after all, we are in the native world where nearly everything is possible (but nothing of interest is easy). For instance, you could override SetUnahdneldExceptionFilter() call with Detours and make sure you keep the top-level handler that always returns EXCEPTION_CONTINUE_SEARCH.
For WPF applications one can use Dispatcher.UnhandledException event. It will be correctly called before the native exception filter, even on main thread. In Windows Forms applications Application.ThreadException plays similar role. So, for those types of applications a hybrid approach would work: handle main thread managed exceptions in Dispatcher.UnhandledException, and all others in AppDomain.CurrentDomain.UnhandledException. If there is a chance that native UEF may return EXCEPTION_CONTINUE_SEARCH, the same exception may be passed to both Dispatcher and AppDomain events, so care must be taken not to do things twice in this case.
Application samples and use cases
Mixed.Exceptions.zip contains sample code that supports the above assertions. It has three folders:
– ClrHosting.Exceptions.Console
– ClrHosting.Exceptions.Wpf
– Exceptions.Console.ManagedC++
ClrHosting.Exceptions.Console
This is a console application that hosts its own CLR. There are two projects in the solution:
– ClrHosting: native C++ application
– ManagedCode: managed C# DLL
Native application calls managed code via ICLRRuntimeHost::ExecuteInDefaultAppDomain method. If an exception occurs inside the call, managed exception handler is not called. Instead, ExecuteInDefaultAppDmain returns a failing HRESULT. It is COR_E_APPLICATION (0x80131600) if thrown exception was derived from System.ApplicationException, and COR_E_EXCEPTION (0x80131500) if it was not.
However, asynchronous exception in a worker thread does result in calling managed AppDomain.UnhandledException, followed by the native UEF handler.
The application throws exception in the main thread by default. To cause it to throw exception in a worker thread, pass it a command line argument, e.g. ClrHosting.exe x.
ClrHosting.Exceptions.Wpf
This solution is a variation of console CLR hosting, but here the managed code creates a WPF window and the native code runs a message loop. WPF window has buttons to throw an exception in the UI thread, in a worker thread, or create a window on a new UI thread.
Main difference between this example and console example is that main thread exceptions do not occur directly inside ExecuteInDefaultAppDomain() call. Instead, they happen inside DispatchMessage(), which is part of the message loop.
For main thread exceptions Dispatcher.UnhandledException is called first, then native UEF handler, followed by the AppDomain.UnhandledException event.
For worker thread exceptions and secondary UI thread exceptions the picture is reverse: managed handler first, then native UEF handler.
Exceptions.Console.ManagedC++
This solution does not involve explicit CLR hosting. Instead it applies a IJW (“it just works”) technique for calling managed C++ from native C++ and vice versa. It has two projects: NativeCppApp.exe and ManagedCppDLL.dll. The EXE loads the DLL dynamically and calls the managed entry point using GetProcAddress. At this point the CLR is loaded implicitly into the process.
The behavior is pretty much the same as for the WPF application. For main thread native handler is called first, CLR handler is called second. For worker threads CLR handler is called first, native handler second.
Conclusion
I am sure the .NET/CLR team had some reasons to implement unhandled exceptions filtering the way they did. It does not look like a random bug. However, I am completely in the dark as to what these reasons could be. I would prefer that every thread behaved like a worker thread and called the CLR handler first. If that’s not possible, I would have appreciated a decent documentation, but I could not find anything on the subject. Still, the behavior appears to be consistent, so at least we know what to expect now.