Problems with .NET unit testing

The marketing page for Rider quotes NUnit test support. Is this only for .Net Framework and not .Net Core projects? There is no template for a .Net core Nunit test project: 

For now I am using NunitLite to debug and run tests.

@Alex
NUnit dotnetcore is not working in the current release.

However it is already fixed in Re# and will get to Rider 2017.2.

https://youtrack.jetbrains.com/issue/RSRP-464912#comment=27-2312068

To add to this, in Rider 2018.1.3 the settings within the unit test runner settings (see screenshot) are not respected in the unit test runner itself. The "Automatic" option was still selected while the platform architecture was set to x64.

Kevin,

Thanks for your feedback! Do I understand correctly that you've changed Platform Type in the NuGet tool window (screenshot 2), but it was not changed accordingly in Unit Testing settings (screenshot 1)?

Not quite, the second window was the unit test settings. Before taking the screenshot it looks like I'd already fixed the setting, but the unit test tools (window?) didn't use the setting I'd set in the Settings window for the architecture. I took the screenshot after manually setting the arch to x64 and verifying it worked.

I seem to be unable to debug unit tests in

JetBrains Rider 2018.2.3
Build #RD-182.4231.496, built on September 13, 2018
Licensed to Gary Woodfine
Subscription is active until November 5, 2018
JRE: 1.8.0_152-release-1248-b8 amd64
JVM: OpenJDK 64-Bit Server VM by JetBrains s.r.o
Linux 4.15.0-36-generic

The debugging of Unit Tests was working this morning but it seems to just randomly stopped working.  I have shut down and restarted rider,  rebooted machine. bu the problem keeps recurring.  I am actually able to Run the tests, using Rider and Dotnet Test.  but when I try debug,

I Get Exception below.

I have tried to CLEAN , RESTORE and Rebuild several times.  I created a blank file api.tests.xunit.runner.json

System.IO.FileNotFoundException: Could not find file '/home/gary/code/rea-development/backend/candidate/tests/bin/Debug/netcoreapp2.1/api.tests.xunit.runner.json'.
File name: '/home/gary/code/rea-development/backend/candidate/tests/bin/Debug/netcoreapp2.1/api.tests.xunit.runner.json'
at Interop.ThrowExceptionForIoErrno(ErrorInfo errorInfo, String path, Boolean isDirectory, Func`2 errorRewriter)
at Interop.CheckIo(Error error, String path, Boolean isDirectory, Func`2 errorRewriter)
at Microsoft.Win32.SafeHandles.SafeFileHandle.Open(String path, OpenFlags flags, Int32 mode)
at System.IO.FileStream.OpenHandle(FileMode mode, FileShare share, FileOptions options)
at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options)
at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, Boolean useAsync)
at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share)
at System.IO.File.OpenRead(String path)
at Xunit.ConfigReader_Json.File_OpenRead(String path) in C:\Dev\xunit\xunit\src\xunit.runner.utility\Configuration\ConfigReader_Json.cs:line 172
at Xunit.ConfigReader_Json.LoadFile(String configFileName) in C:\Dev\xunit\xunit\src\xunit.runner.utility\Configuration\ConfigReader_Json.cs:line 141

Unable to edit the above comment 

But I am using .net core 2.1 

I noticed the debug output had some additional info.  I am going to attempt to recreate the unit project and see if the problem recurs

I tried creating a Fresh Unit Test project Using dotnet new xunit and wrote a simple unit test and the problem recurs.

Pdb file for assembly /usr/share/dotnet/shared/Microsoft.NETCore.App/2.1.5/System.IO.MemoryMappedFiles.dll was not found or failed to read
Loaded Assembly '/usr/share/dotnet/shared/Microsoft.NETCore.App/2.1.5/System.IO.Compression.dll'
Loading module /usr/share/dotnet/shared/Microsoft.NETCore.App/2.1.5/System.IO.Compression.dll in application domain 1:clrhost
Pdb file for assembly /usr/share/dotnet/shared/Microsoft.NETCore.App/2.1.5/System.IO.Compression.dll was not found or failed to read
Loaded Assembly '/usr/share/dotnet/shared/Microsoft.NETCore.App/2.1.5/System.Buffers.dll'
Loading module /usr/share/dotnet/shared/Microsoft.NETCore.App/2.1.5/System.Buffers.dll in application domain 1:clrhost
Pdb file for assembly /usr/share/dotnet/shared/Microsoft.NETCore.App/2.1.5/System.Buffers.dll was not found or failed to read
Loaded Assembly '/usr/share/dotnet/shared/Microsoft.NETCore.App/2.1.5/System.Text.Encoding.Extensions.dll'
Loading module /usr/share/dotnet/shared/Microsoft.NETCore.App/2.1.5/System.Text.Encoding.Extensions.dll in application domain 1:clrhost
Pdb file for assembly /usr/share/dotnet/shared/Microsoft.NETCore.App/2.1.5/System.Text.Encoding.Extensions.dll was not found or failed to read

How about removing 

/usr/share/dotnet/shared/Microsoft.NETCore.App 
And letting Rider or dotnet restore to restore it again?

Not sure where exactly where you want me to do this?

What I have tried is basically

sudo apt remove --purge dotnet*

Then I completely reinstalled dotnet using

sudo sh -c 'echo "deb [arch=amd64] https://packages.microsoft.com/repos/microsoft-ubuntu-bionic-prod bionic main" > /etc/apt/sources.list.d/dotnetdev.list'

sudo apt-get update
sudo apt-get install dotnet-sdk-2.1 

Howeever the problem remains

In the Rider Settings --> Build ... -> Toolset and Build

and reset all the options to custom then  dotnet restore  and try test run debug i still get the error

I meant deleting folder in the file-system:

/usr/share/dotnet/shared/Microsoft.NETCore.App 

Ok Cool.

After some more investigating yesterday, and trying out other projects which I have Unit Tests in. It is only one project which has the issue.

This leads me to believe that there must be something about this particular project's configuration which may be a little irregular. I'll do some more digging this morning and get back to you.

Interesting!  I have just tried a dotnet build on the solution.

I found a couple of warnings that point to the Unit Test Project. 

Do you know where the build log is stored? so I can view it?   I can't seem to find it

/usr/share/dotnet/sdk/2.1.403/Microsoft.Common.CurrentVersion.targets(2110,5): warning MSB3277: Found conflicts between different versions of "Microsoft.Extensions.DependencyModel" that could not be resolved. These reference conflicts are listed in the build log when log verbosity is set to detailed. [/home/gary/code/rea-development/backend/candidate/tests/api.tests/api.tests.csproj]
 api.tests -> /home/gary/code/rea-development/backend/candidate/tests/api.tests/bin/Debug/netcoreapp2.1/api.tests.dll

Build succeeded.

/usr/share/dotnet/sdk/2.1.403/Microsoft.Common.CurrentVersion.targets(2110,5): warning MSB3277: Found conflicts between different versions of "Microsoft.Extensions.DependencyModel" that could not be resolved. These reference conflicts are listed in the build log when log verbosity is set to detailed. [/home/gary/code/rea-development/backend/candidate/tests/api.tests/api.tests.csproj]
 1 Warning(s)
 0 Error(s)

It should be "Open log file" in the Build Toolwindow.

OK Cool!   Honestly didn't think of that!!   Thats a cool feature I was blissfully unaware of!  Thanks for letting me know

@gary woodfine did you ever figure this out? I'm running into the exact same issue: can't debug an xunit test.

I'm afraid I haven't as yet!

I still have the problem, I have a suspicion that it has something to do with referencing the API project, which is developed in .net core 2.1. I am able to debug other unit test projects within the same solution, and other solutions.   The only anomoly I can identify is that the particular project I have an issue with references a .net core API 2.1 project!

I'm not sure if this is correct, it's the only conclusion I can jump too.  I am still waiting for feedback

What project type are you having an issue with?

Hope this helps

Unfortunately it seems, that Rider does not fully support non-deterministic xUnit tests, like ones based on AutoFixture. It seems that it's a problem with pre-enumeration.

I've built a small example at https://github.com/drivenet/rider-xunit-discovery-problem

Load solution, restore packages, build the project.

Now, run all tests -- everything seems OK.

Now, try to run the "DetectedProperly" test only -- it works.

Then -- try to run the "NotDetectedProperly" test only -- it either becomes "Unknown, test nor run" or "Inconclusive".

Should I file a bug somewhere?

Xm,

Please file an issue on YouTrack: https://youtrack.jetbrains.com/issues/rider

Thank you!

https://youtrack.jetbrains.com/issue/RIDER-21081

Encountered the same problem as Gary. On newly created xUnit test project I started getting those File not found exceptions when I was trying to debug a test (any test in the project)

Creating a new xUnit project (in the same solution) didn't help. 

I went to the Run -> Stop on exception configuration and re-added the System.Exception. Pop up appeared saying that the exception type is already added. This was expected behaviour.

What was unexpected that after this I was able to F5 thru the FileNotFound exception and after that a break point was hit in the test method...

gary woodfine, Lukas Belovic

The issue was that the debugger didn't respect the "Just my code" setting in exception breakpoints. The workaround for Rider 2018.2.3 is either disable exception breakpoints (Run -> View Breakpoints) or step through all FileNotFound exceptions. In Rider 2018.3, we've fixed "Just my code" setting. If it is turned on, Rider won't stop on the exceptions thrown by xUnit.

Hope this helps!

I've run into an issue after upgrading. It appears as if the "Run <n> number of assemblies in parallel" option is not being honored. I have some Nunit test assemblies that cannot be run in parallel so I have that setting set to 1. In previous versions, my tests worked out fine. Since upgrading I'm getting massive test failures that seem to indicate that some of these assemblies are either not being unloaded properly. I would like to find out if there is a work around or if anyone has seen this issue.

I'm having this issue on 2019.1.1 on Mac OS X with .NET Core 2.2. I can confirm that it's stopping on intermediate code. To be fair, I told it to, but it's really confusing during startup. Especially because (as discussed elsewhere), it doesn't show the exception.

Unit tests work fine for me in NUnit 3.12.0 however any time I try to run tests with coverage, I get "Exit code 134" on all my tests and they are all marked as inconclusive.

Am I just doing something wrong here or should I submit a ticket?

Hi @John,

It seems that this is a Catalina OSX + Rider 2019.2 case. If so, please try Rider.2019.2.3 or Rider 2019.3EAP and let me know about the results. This issue is known and fixed: https://youtrack.jetbrains.com/issue/RIDER-33547.
If it does not help, we would be grateful if you file an issue on youtrack with collected logs.

Sofia.

We are unable to run tests without rebuilding all dependencies for dotnet core projects. Our integration test project depends on a huge number of other related projects and every time someone tries to run tests they have to wait for about 1 minute for the build to finish, despite the fact that no code has been changed. 

https://youtrack.jetbrains.com/issue/RIDER-35463

Somehow in my windows machine if you have a global.json configuration it can make the rider works weirdly, you can try to delete it if you have and see if it make it works

Unit tests would fail to start with error 134, and I had to reinstall the app to get it to work. No upgrade, already using the latest build. More details below.

```

Exit code is 134 (Unhandled exception. System.BadImageFormatException: Could not load file or assembly '/Applications/Rider.app/Contents/lib/ReSharper...

Last runner error: Process /usr/local/share/dotnet/dotnet:14817 exited with code '134': Unhandled exception. System.BadImageFormatException: Could not load file or assembly '/Applications/Rider.app/Contents/lib/ReSharperHost/TestRunner/netcoreapp2.0/ReSharperTestRunner32.dll'. An attempt was made to load a program with an incorrect format. File name: '/Applications/Rider.app/Contents/lib/ReSharperHost/TestRunner/netcoreapp2.0/ReSharperTestRunner32.dll'

```

This issue started today without any particular reason, I was just in the middle of a coding task, running multiple solutions, with multiple Rider instances. I tried to clean the repo, removing .idea, obj/bin folders, etc. I also uninstalled .NET 2.2x just in case.  The .NET app was working fine, but unit tests would fail to start. Every project is based on .NET Core 3.1 but tests seems to be looking for netcoreapp2.0

JetBrains Rider 2020.2
Build #RD-202.6397.244, built on August 10, 2020
Runtime version: 11.0.7+10-b944.20 x86_64
VM: OpenJDK 64-Bit Server VM by JetBrains s.r.o.
.NET Core 3.1.6
macOS 10.15.6

Same problem as @Devis described, with the same error message and a .net core 3.1 project.

Rider 2020.2 on MacOS 10.14.5. Tests were running fine then out of nowhere started failing with that exception.

Running Unit Tests with Coverage resolved the problem for the time being.