Xorg VRAM leak because of Qt/OpenGL Application

mwestphal

Hello board,

I am working on a complex Qt/OpenGL Application.
Xorg starts leaking in VRAM when i'm using the application and never release the memory, until I restart X of course.
This only started to appears once we rewrote the OpenGL part using QOpenGLWindow and widgetContainer instead of QOpenGLWidget.

$ nvidia-smi
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 390.48                 Driver Version: 390.48                    |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce GTX 105...  Off  | 00000000:01:00.0 Off |                  N/A |
| N/A   46C    P8     4W /  N/A |     50MiB /  4040MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
|    0     29628      G   /usr/lib/xorg-server/Xorg                     47MiB |
+-----------------------------------------------------------------------------+
$ ./myOpenGLQtBasedApp ... doing graphic stuff then exiting
$ nvidia-smi
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 390.48                 Driver Version: 390.48                    |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce GTX 105...  Off  | 00000000:01:00.0 Off |                  N/A |
| N/A   46C    P8     4W /  N/A |    110MiB /  4040MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
|    0     29628      G   /usr/lib/xorg-server/Xorg                    107MiB |
+-----------------------------------------------------------------------------+

The version of Xorg does not matter, tested a few.
The version of the driver does not matter, as long as it's nvidia, tested 340, 384, 390
The linux distribution does not matter, tested Ubuntu 16.04, 18.04, fedora
The de does not matter, tested Unity, Gnome-shell, Xfce, Lxde + Compton, Openbox + compton
The compositor used does not matter, but the leak disappear without a compositor.
The version of Qt does not matter, tested 5.8, 5.9, 5.10.
I did not test Wayland.

Do you know what could cause this behavior ?
Could this be a Qt bug ?
If not , do we miss something in out QOpenGLWindow reimplementation ? Somethine else ?

sierdzio

Please consider reporting this to https://bugreports.qt.io. Qt devs would appreciate a minimal compilable example together with the (splendid) data you've already provided.

mwestphal

Well, the minimum example require sadly to build the whole application. I was not able to reproduce it using only our QOpenGLWindow reimplementation. That is why I'm not sure that it is a bug in Qt our in our Application.

aha_1980

@mwestphal if you have a commercial licence, then the support can help you with priblems in your code also.

if you use open source Qt, then you have to provide a minimal example for bug reports, i.e. track down the actual problem (which is always good!)

mwestphal

Since the whole application is OpenSource and Free, I do not have any problem with sharing it, it is simply that I am afraid that a example with this so many lines will not be considered and quite hard to pin down for somebody not knowing the code.

I was more hoping for advices on how this could happen as it would help me track it down.

sierdzio

@mwestphal said in Xorg VRAM leak because of Qt/OpenGL Application:

I was more hoping for advices on how this could happen as it would help me track it down.

I have zero experience with QOpenGLWidget and containerWidget, unfortunately.

One recommendation that springs to mind when leaks are a problem: run your app through valgrind and / or clang's address sanitizer. They are often quite good at finding such issues.

mwestphal

@sierdzio : That we did, and actually corrected a few memory leaks in the process. But not related to this issue.
Since this is Xorg leaking on the VRAM, it would have been quite a wild luck to fix it this way anyway.

wrosecrans

Have you tried building your application with Address Sanitizer to see if it helps you identify the source of the leak? It is very helpful with that sort of problem.

aha_1980

@wrosecrans have you read the last two responses? ;)

wrosecrans

@aha_1980

D'oh. Maybe this plan to give up drinking so much coffee in the mornings wasn't such a great idea. :)

I guess I just think using Address Sanitizer is advice that is worth repeating! One thing I will add is that it's not clang specific -- recent versions of gcc will accept the same command line parameters to use it.

mwestphal

The issue was resolved thanks to an intense debugging session.

The leak was caused by NULL parenting the parent of the windowContainer containing the QVTKOpenGLWindow just before deletion.

This code was here before when we used a QOpenGLWidget and it caused no issue. In any case, NULL parenting a widget before deletion is useless so removing the line resolve the issue.

This leak shouldn't happen though, even in this situation, so I have opened a Qt issue to report it.

sierdzio

Great, thanks for investigating and reporting the bug!