NET Framework 4.0 breaks Com Interop - vb6 calling c# dll failing when changing target framework from 3.5 to 4.0 or higher

Question

I have a C# custom control DLL that loads pdf in byte[] and render it in vb6 control which hosts my custom C# control.

This works without any issue when compiled against .NET framework 3.5 but stops working if I change the target framework to 4.0 or higher.

Here is my source code, can some one tell me what is needed for the COM interop to work again in .NET 4.0 or higher?

[ComVisible(true)]
[Guid("my guid 1")]
[InterfaceType(ComInterfaceType.InterfaceIsIDispatch)]
public interface IWrapperControl
{
    [DispId(10)]
    void LoadPDF(object data);
}

[ComVisible(true)]
[Guid("my guid 2")]
[ClassInterface(ClassInterfaceType.AutoDual)] // i know this is bad, but it works in .NET 3.5
public partial class WrapperControl : UserControl, IWrapperControl
{
    public WrapperControl()
    {
        InitializeComponent();
    }

    [DispId(10)]
    public void LoadPDF(object data)
    {
        var viewer = create my pdf viewer control from data; 
        Controls.Add(viewer);
    }

    [ComRegisterFunction]
    static void ComRegister(Type t)
    {    
        code removed    
    }

    [ComUnregisterFunction]
    static void ComUnregister(Type t)
    {
     code removed
    }
}

I've tried different ClassInterfaceTypes, I've tried adding ProgId and I've tried adding ComDefaultInterfaceAttribute typeof(IWrapperControl). Nothing seems to work. Clearly there is a breaking change in .NET 4 from .NET 3.5.