Unable to browse/refer a .net class library dll file (which is part of product deployment and dependency of an .Net exe file) from custom action

Question

I'm creating a new custom action in a basic MSI (installshield 2014) project. I've to call a public method in a managed .Net assembly abc.dll which gets deployed as part of the product deployment. abc.dll is part of a component named component1 which is part of the the feature feature1 in the setup design.

When I try to refer that assembly in the custom action creation wizard I mention its Location as Installed with the product. But on the Action Parameters step in custom action creation wizard when I try to browse abc.dll at the deployment path then I don't see it:

Although I'm able to see abc.dll while browsing it in the components as shown in the snapshot below. The abc.dll is present as part of component1 which gets deployed in %programfiles% path of the product.

On the other hand, I can see a pqr.exe file (which gets deployed as part of another component component2) in the custom action creation wizard as shown in the below snapshot:

Can anyone guide me why this might be happening?

Answer 1

I finally got the problem to my solution. It all boils down to how Installshield treats the packaging of .Net assembly and its dependencies.

Difference between how a .Net exe and its dependencies are referred by Installshield:

Whenever you add main project output e.g. pqr.exe in my case then it gets added into a component. When you browse your component you will see the source path of the exe under Link To column in terms of standard known variables of installshield environment e.g. [InstallDir]\ComponentName\ etc.

Now when installshield has to create the MSI package for deploying a .Net assembly e.g. pqr.exe in my case it also tries to package its dependencies as well for deployment. In my case prq.exe was depending upon abc.dll. But the caveat is that installshield doesn't maintain the list of dependencies of a .Net assembly statically inside its own *.ISM installer project file.

So abc.dll was also shown in the component but its path was coming as a fully qualified path on the disk where ISM file was getting built e.g. D:\mywork\MSIProject\. This is because abc.dll is actually not part of the ISM insataller project file. In fact, I opened the xml format of the ISM file in a text editor and tried to search for abc.dll and it was not there at all. So abc.dll is not present as an assembly to be deployed inside ISM installer definition file. But only while building the ISM file it tries to package all the dependencies along with the *.exe file.

All the dependencies should be present in the same root directory where *.exe file is present or installshield will not be able to package them.

Difference between how a .Net exe and its dependencies are packaged by installshield:

Another point of difference to note is that if you remove the pqr.exe from disk path where ISM file is getting built then ISM file will fail to build but if you remove abc.dll (which is not a physical part of the ISM definition file as it only a reference/dependency) then ISM file will still build successfully.

Difference between how a .Net exe and its dependencies are referred by a custom action in installshield:

As far as my actual problem goes, you can refer only those assemblies in your custom action which are part of a component in a direct manner. You cannot refer the assemblies which logically look like a part of the components only (as they are dependencies) but in reality they are being shown there just because they are dependencies of a managed .Net exe file.

Answer 2

I assume that InstallShield is validating that those binaries can be called as custom actions. An executable can be run as a custom action, so it shows up. A Dll cannot be arbitrarily called unless it exports an entry point with the required signature. There is no native support for Windows Installer to call managed code custom actions, so perhaps InstallShield provides a C++ shim to call (as do Visual Studio installer projects). Otherwise C++ Dlls need to have this signature:

UINT__stdcall CustomActionEntryPoint(MSIHANDLE hInstall)

Examples here:

https://www.simple-talk.com/dotnet/visual-studio/visual-studio-setup-projects-and-custom-actions/

https://www.codeproject.com/Articles/570751/DevMSI-An-Example-Cplusplus-MSI-Wix-Deferred-Custo

And as I said, your version of IS may have support for calling managed code Dlls under some other description of calling Dlls (as opposed to something like "calling a standard Windows Installer custom action").