OpenFileMapping not connecting to shared memory with one program but connects fine from another program

Question

I'm in the process of updating an old program (very old program) which last ran under Windows XP (32-bit). Two of the executables communicate through shared memory. One of these programs, I have updated to be 64-bit. The other is still 32-bit.

The 32-bit program creates some shared memory with the following lines of code:

HANDLE hFile = ::CreateFileMapping(INVALID_HANDLE_VALUE,
                  NULL,
                  PAGE_READWRITE,
                  0,            //high order
                  size,         //low order
                  name);

It then calls MapViewOfFile().

Both calls return success. The program is happy.

Then, just to confirm things are OK, I connect to the shared memory. These calls weren't in the original code, but I added them just to make sure things were working:

HANDLE hFile = ::OpenFileMapping(FILE_MAP_ALL_ACCESS,
    TRUE,
    name);

void* tmp = MapViewOfFile(hFile, FILE_MAP_ALL_ACCESS, 0, 0, size);

Both calls return success. That's in the 32-bit process.

The 64-bit process makes the exact same calls to OpenFileMapping() and MapViewOfFile() as above, with the same values, but OpenFileMapping() returns error 2 (ERROR_FILE_NOT_FOUND).

I've verified that the sizes are the same, the name is the same, neither of them are using Unicode. Neither of them use wide characters. The only difference is that the second program is 64-bit. Presumably the old 32-bit program worked fine.

I originally thought that the problem was the 32-bit vs 64-bits, but that doesn't appear to be the case.

I wrote a short little program to open the shared file.

#include "Windows.h"
#include "winbase.h"

int main()
{
    ::Sleep(20000);

    const char* name = "ProcServSM";
    const HANDLE hFile = ::OpenFileMapping(FILE_MAP_ALL_ACCESS,
        TRUE,
        name);

    ::Sleep(0);
}

It worked both in 32 and 64 bits. I compared visual studio settings between the original program and my test program. Didn't find any meaningful differences.

Windows 10, Visual studio 2019. All programs are debug builds.

One other piece of information is that the code that deals with the file mapping is in a library.

I've since modified my test program to make the same calls to the library that the original program did. The original program still fails, and the test program still works.

Answer 1

If the name doesn't exist yet, CreateFileMapping() creates a new mapping object with that name and returns a valid HANDLE to the object, and GetLastError() returns 0.

If the name already exists, CreateFileMapping() returns a valid HANDLE to the existing mapping object, and GetLastError() returns ERROR_ALREADY_EXISTS.

Unless the name belongs to an existing object that is not a file mapping, in which case CreateFileMapping() returns NULL and GetLastError() returns ERROR_INVALID_HANDLE instead.

On the other hand, OpenFileMapping() can fail and GetLastError() will return ERROR_FILE_NOT_FOUND, if CreateFileMapping() has not been called beforehand to create the mapping object with that name, or that object does not exist anymore when OpenFileMapping() is called. OpenFileMapping() does not create a mapping object. It can only access an existing mapping object.

The order of CreateFileMapping() and OpenFileMapping() calls is very important. The program that wants to use OpenFileMapping() needs to wait for the other program to call CreateFileMapping() first. However, if you don't know which program will try to access the mapping object first, then use CreateFileMapping() in both programs, don't use OpenFileMapping() at all.