QSharedMemory attaching to third-party non-Qt application shared memory in Linux where key is an integer

[kobbler248]

I am writing a Qt application that needs to attach to shared memory being created by a third-party application on Linux. The third party application accepts arguments to set the shared memory key as an unsigned integer. I have been able to successfully use the Linux shared memory API to attach with no problem and the IPC list shows the shared memory segment with the key that was passed to the third-party application..

I understand that, according to the Qt documentation, I should use setNativeKey() to accomplish this task, yet it takes a QString and I have been unable to decipher what it is doing with this QString to come up with the 32-bit key and therefore cannot predict what the key will end up being on the Qt side.

I have tried passing the same integer in many different ways as a QString as follows:

int extShmKey = 829;
QSharedMemory sharedMem;
QString qtNativeShmKey = QString::number(extShmKey); //Literal "829"
sharedMem.setNativeKey(qtNativeShmKey);

-and-

union SharedMemKeyTranslator{
quint32 key_uint;
char key_char[4];
} shMemKeyTrans;

shMemKeyTrans.key_uint = 829;
QString qtNativeShmKey = QString(shMemKeyTrans.key_char); //Literal "=\003"
sharedMem.setNativeKey(qtNativeShmKey);

Any help with this particular usage would be greatly appreciated.

[kobbler248]

After digging through the Qt source code for QSharedMemory I finally figured out why it requires a QString for the key.

For SystemV :
Qt is following the best practice for Linux shared memory key generation/usage by using the function ftok.

key_t ftok(const char *pathname, int proj_id)
This function literally stands for "file to key", takes a string to a filepath and integer for an ID (but only uses the first 8 bits of the ID), and creates a 32-bit hash from the two. Qt uses 'Q' (surprise, surprise) for the ID and whatever filepath you passed it via the QString. Qt also checks to see that the filepath is valid, but attempts to create the file for you so you aren't left hanging.

It would have been nice to have this level of detail in the documentation especially since not everybody implements using best practices. Also knowing that 'Q' is the seed that they are using would have also been a big help.

Now, all I have to do is generate a key using ftok with a filepath string and 'Q' as the ID. I can then pass the decimal equivalent to the third-party application and hopefully have no further issues with shared memory keys, QSharedMemory, and third-party implementations. At least, that's my hope.

For POSIX:
I'm still deciphering the implementation, but based on the configure script I don't think this is the default for a modern Linux distribution.

[SGaist]

Hi and welcome to devnet,

Might be a silly question but did you call attach after setNativeKey ?

[kobbler248]

Yes, a valid question. sharedMem.attach() returns QSharedMemory::NotFound in both cases.

As an experiment I called sharedMem.create(8) instead to see what the 32-bit key value was that it was using based on the QString I passed. (I used a small number for the size so it would be easy to find in the IPC list.)

For the decimal value 829, I was expecting to see the hex value of 0x0000033d on the shared memory segment in the list of IPCs, but the following is what was created:

Example1: 0x8e2b09dc

Example2: 0x772b09be

[kobbler248]

After digging through the Qt source code for QSharedMemory I finally figured out why it requires a QString for the key.

For SystemV :
Qt is following the best practice for Linux shared memory key generation/usage by using the function ftok.

key_t ftok(const char *pathname, int proj_id)
This function literally stands for "file to key", takes a string to a filepath and integer for an ID (but only uses the first 8 bits of the ID), and creates a 32-bit hash from the two. Qt uses 'Q' (surprise, surprise) for the ID and whatever filepath you passed it via the QString. Qt also checks to see that the filepath is valid, but attempts to create the file for you so you aren't left hanging.

It would have been nice to have this level of detail in the documentation especially since not everybody implements using best practices. Also knowing that 'Q' is the seed that they are using would have also been a big help.

Now, all I have to do is generate a key using ftok with a filepath string and 'Q' as the ID. I can then pass the decimal equivalent to the third-party application and hopefully have no further issues with shared memory keys, QSharedMemory, and third-party implementations. At least, that's my hope.

For POSIX:
I'm still deciphering the implementation, but based on the configure script I don't think this is the default for a modern Linux distribution.

[kobbler248]

I finally got around to testing this (at least from the SystemV perspective), and I can't get an equivalent key using this method. So either I missed something in Qt's SystemV implementation or the POSIX implementation is what's in the default libraries. (For the filepaths I tested, the difference always ended up in the most significant byte of the key.)

UPDATE: I was looking at the wrong version of the source (v5.10.1) for the version I have installed (v5.6.2) on my test system. These generate the key differently, with 5.6.2 having an additional level of hashing.

[SGaist]

Interesting analysis ! Thanks

Did you try to check the bug report system to see if there was something related to your use case ?

[kobbler248]

No, but I'll check that now.

I've resorted to using the SystemV API for now as my project does not yet require portability.

Thanks

[kobbler248]

From reading the bug list, there are a few bugs that addressed this issue. Unfortunately all have been closed with the current implementation standing as I found and identified above. So at this point it appears that the documentation could use a friendly update to include the details of the internal key generation so it can, at least, be duplicated without diving into the Qt source.

With regards to my failed experiments trying to recreate the key, I was looking at the wrong version of the source (5.10.1) for the version I have installed (5.6.2) on my test system. They are, indeed, different. If I have time, I will revisit using QSharedMemory and verify my findings.

I also verified that the SystemV API implementation is the default interface used on Linux.

I hope this may help others.

[kobbler248]

Tested the method described above using Qt 5.10.1 libraries. Keys now match. :-)

Unfortunately, this will require an inclusion of Linux system libraries in a Qt application in order to calculate the key that the QSharedMemory is already generating in order to make it available for a third-party process...

For now, I will keep using the SystemV API directly considering the other features that have not yet been made available in QSharedMemory.