Choose references dynamically in .net core for Roslyn

Question

TL;DR

How do I let the runtime choose the right assemblies in .NET Core 5 for C# plugins compiled at runtime that involve .NET 4.7.2 code?

Context

I have a .NET 4.7.2 app on which certain modules behave differently based on some configurable plugins. I have the following code in a .NET 4.7.2 assembly that compiles C# plugins at runtime.

    public OperationResult<Assembly> CompileClass(string code, string[] references, string fileName, bool generateInMemory = true, bool includeDebugInformation = true)
    {
        OperationResult<Assembly> result = new OperationResult<Assembly> { Success = true };

        try
        {
            string pluginsFolder = Path.Combine(InsproConfiguration.GetSettings().PluginsFolder);
            bool keepSoureceFilesAfterCompiling = false;
#if (DEBUG)
            keepSoureceFilesAfterCompiling = true;
#endif

            if (!Directory.Exists(pluginsFolder)) 
            {
                Directory.CreateDirectory(pluginsFolder); 
            }

            using (CSharpCodeProvider compiler = new CSharpCodeProvider(new Dictionary<string, string> { { "CompilerVersion", "v4.0" } }))
            {
                CompilerParameters parameters = new CompilerParameters()
                {
                    GenerateInMemory = generateInMemory,
                    IncludeDebugInformation = includeDebugInformation,
                    OutputAssembly = Path.Combine(pluginsFolder, fileName) + ".dll",
                    CompilerOptions = "/debug:full",
                    TempFiles = new TempFileCollection { KeepFiles = keepSoureceFilesAfterCompiling }
                };
                parameters.ReferencedAssemblies.AddRange(references);
                CompilerResults compiledCode = compiler.CompileAssemblyFromSource(parameters, code);
                var errors = new StringBuilder();

                foreach (CompilerError error in compiledCode.Errors)
                {
                    errors.AppendLine($"Error in line {error.Line}, Column {error.Column}: {error.ErrorText}");
                }

                if (!string.IsNullOrEmpty(errors.ToString()))
                {
                    result.HandleFailure(errors.ToString());
                }

                result.ResultObject = compiledCode.CompiledAssembly;
            }
        }
        catch (Exception ex)
        {
            LogService.Current.LogError(ex);
        }

        return result;
    }

I'm now trying to upgrade to the code (slowly) to .NET 5.0 and I started with the UnitTests (one of the projects to which no other project has a reference). I've written the following code

public OperationResult<Assembly> CompileClassWithRoslyn(string code, List<string> referenceAssemblies, string assemblyName)
{
        OperationResult<Assembly> result = new OperationResult<Assembly>();

        try
        {
            //Set file name, location and referenced assemblies
            string pluginsFolder = Path.Combine(InsproConfiguration.GetSettings().PluginsFolder);
            SyntaxTree syntaxTree = CSharpSyntaxTree.ParseText(code);
            var trustedAssembliesPaths = ((string)AppContext.GetData("TRUSTED_PLATFORM_ASSEMBLIES")).Split(Path.PathSeparator);

            if (!referenceAssemblies.Any(a => a.Contains("mscorlib"))) 
            {
                 referenceAssemblies.Add("mscorlib.dll"); 
            }

            var references = trustedAssembliesPaths.Where(p => referenceAssemblies.Contains(Path.GetFileName(p)))
                                                        .Select(p => MetadataReference.CreateFromFile(p))
                                                        .ToList();

            CSharpCompilation compilation = CSharpCompilation.Create(
                assemblyName,
                syntaxTrees: new[] { syntaxTree },
                references: references,
                options: new CSharpCompilationOptions(OutputKind.DynamicallyLinkedLibrary));

            using (var ms = new MemoryStream())
            {
                EmitResult emitResult = compilation.Emit(ms);

                if (!emitResult.Success)
                {
                    IEnumerable<Diagnostic> failures = emitResult.Diagnostics.Where(diagnostic =>
                        diagnostic.IsWarningAsError ||
                        diagnostic.Severity == DiagnosticSeverity.Error);
                    result.HandleFailure(failures.Select(f => f.GetMessage()));
                }
                else
                {
                    ms.Seek(0, SeekOrigin.Begin);
                    Assembly assembly = Assembly.Load(ms.ToArray());                     
                }
            }
        }
        catch (Exception ex)
        {
            return result.HandleFailure(ex);
        }

        return result;
    }

In the old code, at

parameters.ReferencedAssemblies.AddRange(references);
CompilerResults compiledCode = compiler.CompileAssemblyFromSource(parameters, code);

the assemblies are chosen automatically by name by the runtime. In the new code mscorlib doesn't resolve correctly because I'm get an error:

Error CS0518: Predefined type 'System.Object' is not defined or imported

Answer 1

When compiling with Roslyn against .net5, the challenge is quite different from compiling against the legacy .net framework, because you have to reference reference assemblies not implementation assemblies. Many tips will lead you in the bad direction by letting you reference System.Private.CoreLib.dll, that is an implementation assembly. e.g. MetadataReference.CreateFromFile(typeof(object).Assembly.Location)

The code below references all (VB excepted) reference assemblies of .net 5

foreach (string dll in Api.GetFiles(@"C:\Program Files\dotnet\packs\Microsoft.NETCore.App.Ref\5.0.0\ref\net5.0", "*.dll"))
{
    if (!dll.Contains("VisualBasic"))
        references.Add(MetadataReference.CreateFromFile(dll));
}

If you use the Windows Forms compatibility pack (net5.0-windows), add these assemblies:

foreach (string dll in Api.GetFiles(@"C:\Program Files\dotnet\packs\Microsoft.WindowsDesktop.App.Ref\5.0.0\ref\net5.0\", "*.dll"))
{
    if (!dll.Contains("VisualBasic") && !dll.Contains("PresentationFramework") && !dll.Contains("ReachFramework"))
        references.Add(MetadataReference.CreateFromFile(dll));
}

With these references

1. the compilation occurs without errors
2. the produced assembly may be used in others projets without complaining about missing references (e.g. System.Private.CoreLib.dll)

All assemblies of the framework? When spying into the generated code, you will see that only the needed assemblies are referenced.

If the compilation has to run on machines where the above-mentioned directories don't exist, a possible solution is:

• embed all these reference assemblies as embedded resources
• use Assembly.GetExecutingAssembly().GetManifestResourceStream to read these embedded resources as stream
• fill byte[] with those streams
• use references.Add(MetadataReference.CreateFromImage(BytesFromResource(dll))); to add references