I have just run into a problem: a test passes fine on the CI sever, but breaks on my machine. Reason: it gets a wrong config file. Root cause: when you run tests from multiple assemblies via Resharper test runner, it plays games with AppDomains for the sake of optimization. Let’s say you are testing assemblies A.DLL and B.DLL. If you run tests from B.DLL separately, they will be executed in an AppDomain that points to B.DLL.config. However, if you run tests from both A and B in one session, they all will be executed in a single AppDomain, most likely pointing to A.DLL.config, and thus tests from B.DLL may fail.
It turns out there is an option to turn off this optimization, but obviously it is not really easy to find among 10,000 Visual Studio options. Thankfully, someone raised a similar question on StackOverflow:
They even submitted a feature request to JetBrains:
https://youtrack.jetbrains.com/issue/RSRP-428958
In the comments to that request, JetBrains support says this is already fixed, but it’s not clear in which Resharper version. I sent a query to JetBrains support to find out, waiting for their response.