Turn off OpenGL optimization via Qt Environment Variable

  • Qt version: 5.9
    OS: CentOS 7.4
    Nvidia version: 390.48-1
    OpenGL version string: 4.6.0 NVIDIA 390.48

    We running our software on a whitelisted system. Once we whitelist,
    we can't add new binaries to run.

    We have a number of applications that use Qt+OpenGL.

    Some of the OpenGL software is creating mmap'ed binaries in /tmp.
    They are files of the form /tmp/.gl* where the rest of the text
    is randomly generated.

    In the README file( http://us.download.nvidia.com/XFree86/Linux-x86/390.48/README/openglenvvariables.html#disableexecmem ),
    there is a section that reads:

    Disabling executable memory optimizations

    By default, the NVIDIA driver will attempt to use optimizations which rely
    on being able to write to executable memory. This may cause problems in
    certain system configurations (e.g., on SELinux when the "allow_execmem"
    boolean is disabled or "deny_execmem" boolean is enabled, and on
    grsecurity kernels configured with CONFIG_PAX_MPROTECT). When possible,
    the driver will attempt to detect when it is running on an unsupported
    configuration and disable these optimizations automatically. If
    the __GL_WRITE_TEXT_SECTION environment variable is set to 0,
    the driver will unconditionally disable these optimizations.

    We were hoping the messages would disappear when the variable was set.
    The messages look something like:

    May 15 17:46:02 <hostname> * prevented unauthorized execution of '/tmp/.gl96LhR8' by process <process>
    May 15 17:46:02 <hostname> * prevented unauthorized execution of '/tmp/.gl96LhR8' by process <process>
    May 15 17:46:02 <hostname> * prevented unauthorized execution of '<home>/.nv/.glNpZrDX' by process <process>
    May 15 17:46:02 <hostname> * prevented unauthorized execution of '<home>/.nv/.glNpZrDX' by process <process>

    If we set the variable to 0 in the environment, we see dozens and dozens of such messages
    when we launch our application. If we set the variable to 1 or don't set it at
    all, we see 4 such messages in the log when our application program starts up.

    The strace output indirectly associates libqtgraphicaleffectsprivate.so
    with the creation of the mmap'ed memory executable files.

    We executed 'ldd /path/to/Qt-5.9/qml/QtGraphicalEffects/private/libqtgraphicaleffectsprivate.so'
    and /usr/lib64/nvidia/libGL.so.1, /usr/lib64/nvidia/libGLX.so.0, and
    /usr/lib64/nvidia/libGL_dispatch.so.0, show up.

    I looked around in the Qt documentation starting with the graphical
    effects stuff which lead me to a Qt environment variable which tells
    Qt Quick not to use OpenGL:

    QT_QUICK_BACKEND="software" /path/to/our/application

    With this, the messages disappear from the logs. This is not
    an option, however. We need OpenGL.

    Another option to is to add our application program to an
    updaters list. This tells our whitelisting software the
    program is trusted to create other executables and run them.
    With that, the messages disappear from the log.

    However, this violates our cybersecurity requirements.
    We can't add any directory a trusted list and we have
    a very, very limited set of administrative programs
    that are on the updaters list.

    When we set the environment variable, __GL_WRITE_TEXT_SECTION=0,
    and run /bin/glxgears, a standard program that is supplied by the
    glx-utils package, the optimizations are prevented and hence
    the complaints from our whitelisting software. Without the
    environment variable, execution of /bin/glxgears results
    in the messages.

    We were able to reproduce the problem with one of the Qt example
    programs: /path/to/Qt-5.9/examples/quickcontrols2/texteditor/texteditor.

    We ran the above program while monitoring /var/log/messages.
    4 messages similar to the above were generated by the whitelisting software.

    When we set __GL_WRITE_TEXT_SECTION=0 and ran the same program,
    dozens and dozens of these messages were generated. The environment
    variable had the opposite effect from what we saw with the
    /bin/glxgears program.

    Is there a special way to set this variable due to locale
    or UTF considerations? We are using English for locale.

    Is there another way to disable these optimizatons through
    a Qt environment variable other than telling Qt not to use
    OpenGL via the QT_QUICK_BACKEND environment variable?

  • Lifetime Qt Champion

    Hi and welcome to devnet,

    Pretty interesting use case !

    Did you check whether there was already something about it in the bug report system ?

    In any case, I'd recommend bringing this to the interest mailing list. You'll find there Qt's developers/maintainers. This forum is more user oriented.

  • @SGaist I did check the bug database and found nothing there about opengl+whitelisting. I also checked for the environment variable and optimizations. There was nothing relevant.

    I will send to the 'interest mailing list'.

    Thankyou for your quick reply.

  • @SGaist I performed a post to the interest mailing list as you suggested. Should I receive some sort of reply? I also tried to sign up and have not gotten a response. Is membership constrained to developers in the Qt company?

  • Lifetime Qt Champion

    No, there’s no such restriction, but you should registre before posting.

  • @SGaist I did try registering twice. Have not heard back although the response could have gotten hung up in the SPAM filter. Have been checking that and haven't seen anything in there. I will keep trying.

  • @Sandra_Carney OK, I am in. I was trying to use my (new) corporate email. We are now part of Abbott. The old email worked. I will now goto the next step. Thanx.

Log in to reply