How to disable Rider from auto-adding *.cs files to a *.csproj?

My project uses a complex architecture where files inside a project folder might not necessarily be a part of that particular project. They also follow a different naming scheme (suffix before *.cs).

Is there a way I can prevent Rider from auto-adding files to the *.csproj file?