Dump QImage raw pixel data into `std::vector<char>`

[Christian Ehrlicher]

In your first post you use a std::vector<char>() as container, in your last post a plain pointer. How do you allocate this memory?

valgrind is slow, yes - therefore I suggested AdressSanitizer or let it run for an hour and see what happens with valgrind.

[abmyii]

@SGaist said in Dump QImage raw pixel data into &#x60;std::vector<char>&#x60;:

Beware byteCount is obsolete.

Are you doing something multi-threaded ?

Yes, but not the image -> fb conversion.

@Christian-Ehrlicher said in Dump QImage raw pixel data into &#x60;std::vector<char>&#x60;:

In your first post you use a std::vector<char>() as container, in your last post a plain pointer. How do you allocate this memory?

valgrind is slow, yes - therefore I suggested AdressSanitizer or let it run for an hour and see what happens with valgrind.

The memcpy is in a function, and fb is a pointer to frame_data.data() as in the original example.

The full code is hosted on GitLab (https://gitlab.com/abmyii/ubports-mir-vnc-server/-/blob/e3be9585007ea2b663139fd2f5345745392cf9c7/mirvncserver.cpp) but is quite messy. The relevant bits are:

https://gitlab.com/abmyii/ubports-mir-vnc-server/-/blob/e3be9585007ea2b663139fd2f5345745392cf9c7/mirvncserver.cpp#L94-123 (The function for reading the pixels)
https://gitlab.com/abmyii/ubports-mir-vnc-server/-/blob/e3be9585007ea2b663139fd2f5345745392cf9c7/mirvncserver.cpp#L586-591 (The frame buffer initialisation)

https://gitlab.com/abmyii/ubports-mir-vnc-server/-/blob/e3be9585007ea2b663139fd2f5345745392cf9c7/mirvncserver.cpp#L619 (The function call)

There is a chance that the Bus Error is occurring at the VNC level though...

[abmyii]

I'm trying (and failing) to get it compiled with Clang + AddressSanitizer (linking errors - probably not converting from GCC correctly), but I was thinking - is there any reasonably fast method to copy the pixels without using malloc just to ensure that it is(n't) the malloc causing the issue?

[Christian Ehrlicher]

@abmyii said in Dump QImage raw pixel data into &#x60;std::vector<char>&#x60;:

copy the pixels without using malloc just to ensure that it is(n't) the malloc causing the issue?

What should be faster? When you allocate enough pixels (which you still did not check, at least in your code we've seen until now) than all is fine. Otherwise also valgrind would have told you.

[abmyii]

@Christian-Ehrlicher What I mean is that are there other methods that don't require malloc? I want to try those to see if using those methods causes the crashes to stop. As for allocating enough pixels I checked that some time ago and then removed the code but I've added it back in with this output (doesn't change throughout the execution of the program):

FB size: 1327104, Image byteCount: 1327104

Valgrind didn't crash (unfortunately) but there is obviously an issue somewhere, so now finding it isn't going to be easy. I do believe it's on the VNC side, not Qt, but is being induced somehow by the malloc.

Compiling with Clang instead of G++ isn't straightforward for me at my level of knowledge. On compiling I get the following warnings:

clang mirvncserver.cpp -c -std=c++11 -Wall -fpermissive -I/usr/include/mirclient -I/usr/include/mircommon -I/usr/include/mircore -I/usr/include/libevdev-1.0 -I/usr/include/arm-linux-gnueabihf/qt5 -fPIC -fsanitize=address -O1 -fno-omit-frame-pointer -g -o mirvncserver-clang -lboost_program_options -lpthread -lmirclient -lEGL -lxcb-glx -lGLESv2 -lmirserver -lmircore -levdev -lvncserver -lstdc++ -lQt5Gui -lQt5Core

clang: warning: -lboost_program_options: 'linker' input unused
clang: warning: -lpthread: 'linker' input unused
clang: warning: -lmirclient: 'linker' input unused
clang: warning: -lEGL: 'linker' input unused
clang: warning: -lxcb-glx: 'linker' input unused
clang: warning: -lGLESv2: 'linker' input unused
clang: warning: -lmirserver: 'linker' input unused
clang: warning: -lmircore: 'linker' input unused
clang: warning: -levdev: 'linker' input unused
clang: warning: -lvncserver: 'linker' input unused
clang: warning: -Z-reserved-lib-stdc++: 'linker' input unused
clang: warning: -lQt5Gui: 'linker' input unused
clang: warning: -lQt5Core: 'linker' input unused

And the output file is not executable mirvncserver-clang: ELF 32-bit LSB relocatable, ARM, EABI5 version 1 (SYSV), not stripped. I'm sure I'm making a very basic error, but I'm not sure what it is.

[Christian Ehrlicher]

@abmyii said in Dump QImage raw pixel data into &#x60;std::vector<char>&#x60;:

want to try those to see if using those methods causes the crashes to stop

You're aware that all functions which allocate memory is using malloc (C) or new (C++)? So what do you think you gain. If your allocated enough memory than all is fine. Otherwise use a debugger and examine your stack trace

Why do you want to compile your program with clang now? I said you should use the Address Sanitizer - no need to use clang.

[abmyii]

@Christian-Ehrlicher said in Dump QImage raw pixel data into &#x60;std::vector<char>&#x60;:

@abmyii said in Dump QImage raw pixel data into &#x60;std::vector<char>&#x60;:

want to try those to see if using those methods causes the crashes to stop

You're aware that all functions which allocate memory is using malloc (C) or new (C++)? So what do you think you gain. If your allocated enough memory than all is fine. Otherwise use a debugger and examine your stack trace

Why do you want to compile your program with clang now? I said you should use the Address Sanitizer - no need to use clang.

Ah, according to the instructions in the repo you must compile with clang (https://github.com/google/sanitizers/wiki/AddressSanitizer#using-addresssanitizer). How can I do that with G++?

Rather than mallocing I was thinking I could push_back or set the pixel data using something like fb[pos] = (char)qRed(pixel), but I would definitely prefer malloc if I can find the issue...

[Christian Ehrlicher]

@abmyii said in Dump QImage raw pixel data into &#x60;std::vector<char>&#x60;:

Rather than mallocing I was thinking I could push_back

Ok, please re-read and think over again. malloc and push_back don't have anything in common.

asan: http://stackoverflow.com/questions/37970758/ddg#40215639 for example

[abmyii]

@Christian-Ehrlicher said in Dump QImage raw pixel data into &#x60;std::vector<char>&#x60;:

Ok, please re-read and think over again. malloc and push_back don't have anything in common.

Sorry, perhaps I'm misunderstanding but that is the whole point - I want to test if using a different method other than malloc for copying the data will fix the issue.

Thank you for the link, I'll try that now.

[Christian Ehrlicher]

@abmyii said in Dump QImage raw pixel data into &#x60;std::vector<char>&#x60;:

other than malloc for copying the data will fix the issue.

Again: malloc does not copy anything. It allocates memory!

[abmyii]

@Christian-Ehrlicher Other than an initial issue:

=================================================================
==463==ERROR: AddressSanitizer: alloc-dealloc-mismatch (operator new vs free) on 0xb1800850
    #0 0xb6af11d7 in free (/usr/lib/arm-linux-gnueabihf/libasan.so.2+0x751d7)
    #1 0xafddfb31  (/usr/lib/arm-linux-gnueabihf/libhybris/linker/mm.so+0x9b31)
    #2 0xafde0b03  (/usr/lib/arm-linux-gnueabihf/libhybris/linker/mm.so+0xab03)
    #3 0xafde1153 in do_dlopen(char const*, int, android_dlextinfo const*) (/usr/lib/arm-linux-gnueabihf/libhybris/linker/mm.so+0xb153)
    #4 0xafddc5bd  (/usr/lib/arm-linux-gnueabihf/libhybris/linker/mm.so+0x65bd)

0xb1800850 is located 0 bytes inside of 33-byte region [0xb1800850,0xb1800871)
allocated by thread T0 here:
    #0 0xb6af1e4b in operator new(unsigned int) (/usr/lib/arm-linux-gnueabihf/libasan.so.2+0x75e4b)
    #1 0xb6fd9ce7  (/lib/ld-linux-armhf.so.3+0x27ce7)

SUMMARY: AddressSanitizer: alloc-dealloc-mismatch ??:0 free
==463==HINT: if you don't care about these warnings you may set ASAN_OPTIONS=alloc_dealloc_mismatch=0
==463==ABORTING

Which goes away by setting the variable it works absolutely fine... Now I'm even more confused...

Again: malloc does not copy anything. It allocates memory!

Ah, whoops, my mistake - and what a mistake, haha! That's what happens when you go from Python to C++... So I assume that the malloc sets the vector a new memory range to "look at"?

[Christian Ehrlicher]

@abmyii malloc allocates memory - nothing more.
The asan output is fine - no problems in your code. Only somwhere memory is allocated with new and deallocated with free() or allocated with malloc and deallocated with delete which is not allowed.

[abmyii]

@Christian-Ehrlicher That's good to hear! I wonder what's causing the issue, then - as soon as I remove -fsanitize=address it starts failing again. Can I leave -fsanitize=address in for a release binary? Even if it was OK, I'd need to find a way to fix the above issue when running without the env variable. I'll have a look to see if I can find the offending lines of code.

Edit: Didn't find any (especially with my untrained eyes...). The only malloc is the image one.

[Christian Ehrlicher]

@abmyii said in Dump QImage raw pixel data into &#x60;std::vector<char>&#x60;:

it starts failing again.

This really sounds more like a threading issue.

[abmyii]

@Christian-Ehrlicher How does having -fsantize=address cause it to stop failing? Also is there any way to debug it?

[Christian Ehrlicher]

@abmyii said in Dump QImage raw pixel data into &#x60;std::vector<char>&#x60;:

How does having -fsantize=address cause it to stop failing?

When it's a thread issue then the timing is now different

Also is there any way to debug it?

With a debugger, yes. Examine your threads when it crashes

[abmyii]

@Christian-Ehrlicher

(gdb) thread apply all bt

Thread 6 (Thread 0xb0dfdbc0 (LWP 4458)):
#0  0xb6e433a2 in ?? () from /usr/lib/arm-linux-gnueabihf/libvncserver.so.1
#1  0x00000c00 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

Thread 5 (Thread 0xb15fdbc0 (LWP 4457)):
#0  0xb69634e2 in select () at ../sysdeps/unix/syscall-template.S:84
#1  0xb6e282c6 in ?? () from /usr/lib/arm-linux-gnueabihf/libvncserver.so.1
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

Thread 4 (Thread 0xb20b0bc0 (LWP 4455)):
#0  0xb69634e2 in select () at ../sysdeps/unix/syscall-template.S:84
#1  0xb6e28868 in ?? () from /usr/lib/arm-linux-gnueabihf/libvncserver.so.1
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

Thread 3 (Thread 0xb2f0abc0 (LWP 4454)):
#0  __libc_do_syscall () at ../sysdeps/unix/sysv/linux/arm/libc-do-syscall.S:46
#1  0xb68badba in __pthread_cond_wait (cond=0x72f58, mutex=0x728f0) at pthread_cond_wait.c:186
#2  0xb2f93ff4 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

Thread 2 (Thread 0xb3dd9bc0 (LWP 4453)):
#0  0xb6961c00 in poll () at ../sysdeps/unix/syscall-template.S:84
#1  0xb684c7c8 in ?? () from /usr/lib/arm-linux-gnueabihf/libmircommon.so.7
#2  0xb6851038 in ?? () from /usr/lib/arm-linux-gnueabihf/libmircommon.so.7
#3  0xb6d9edc8 in ?? () from /usr/lib/arm-linux-gnueabihf/libstdc++.so.6
#4  0xb68b65b4 in start_thread (arg=0x0) at pthread_create.c:335
#5  0xb6967c5c in ?? () at ../sysdeps/unix/sysv/linux/arm/clone.S:89 from /lib/arm-linux-gnueabihf/libc.so.6
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

Thread 1 (Thread 0xb40ea000 (LWP 4441)):
#0  0xb69658d6 in munmap () at ../sysdeps/unix/syscall-template.S:84
#1  0xb692480e in munmap_chunk (p=<optimized out>) at malloc.c:2860
#2  0xb6ae6f12 in QImageData::~QImageData() () from /usr/lib/arm-linux-gnueabihf/libQt5Gui.so.5
#3  0xb6ae7122 in QImage::~QImage() () from /usr/lib/arm-linux-gnueabihf/libQt5Gui.so.5
#4  0x00019df6 in (anonymous namespace)::read_pixels(int, unsigned int, mir::geometry::Size const&, std::vector<char, std::allocator<char> >&) ()
#5  0x0001b130 in (anonymous namespace)::do_screencast((anonymous namespace)::EGLSetup const&, mir::geometry::Size const&, int, double, std::ostream&) ()
#6  0x0001c446 in main ()

Quite indecipherable to me but the last (#1) thread's backtrace looks promising...

[Christian Ehrlicher]

@abmyii said in Dump QImage raw pixel data into &#x60;std::vector<char>&#x60;:

#4 0x00019df6 in (anonymous namespace)::read_pixels(int, unsigned int, mir::geometry::Size const&, std::vector<char, std::allocator<char> >&) ()

What happens here?
Also try to install the debug symbols of libvncserver

[abmyii]

@Christian-Ehrlicher

void read_pixels(int bpp, GLenum format, mir::geometry::Size const& size, std::vector<char> &fb)
{
    auto width = size.width.as_uint32_t();
    auto height = size.height.as_uint32_t();

    // Read pixels into image (https://community.khronos.org/t/render-to-texture-under-qt/68430/3)
    QImage image(width, height, QImage::Format_RGBA8888);
    glReadPixels(0, 0, width, height, format, GL_UNSIGNED_BYTE, image.bits());

    if (mirror)
	image = image.mirrored(true, true);

    if (landscape) {
        //QPoint center = image.rect().center();
        QMatrix matrix;
        matrix.translate(0, 0);
        matrix.rotate(90);

        image = image.transformed(matrix);
    }

    std::cout << "FB size: " << fb.size() << ", Image byteCount: " << image.byteCount() << std::endl;
    std::memcpy(fb.data(), (const char *)image.constBits(), image.byteCount());
#ifdef DEBUG
    unsigned int sum = 0;
    unsigned int *pui = (unsigned int *) buffer;
    for (int i = 0; i < width * height; i++) {
        sum += *(pui + i);
    }
    printf("read pixels sum 0x%x\n", sum);
#endif
}

Sure, I'll try that.

[abmyii]

@Christian-Ehrlicher

Thread 6 "mirvncserver" received signal SIGBUS, Bus error.
[Switching to Thread 0xb0db8bc0 (LWP 3495)]
0xb6e433b2 in CheckSolidTile32 (cl=0xb1700470, needSameColor=0 '\000', colorPtr=0xb0db8438, h=16, w=16, y=-1231936575, x=20) at tight.c:611
611	tight.c: No such file or directory.

(gdb) thread apply all bt

Thread 6 (Thread 0xb0db8bc0 (LWP 3495)):
#0  0xb6e433b2 in CheckSolidTile32 (cl=0xb1700470, needSameColor=0 '\000', colorPtr=0xb0db8438, h=16, w=16, y=-1231936575, x=20) at tight.c:611
#1  CheckSolidTile (cl=cl@entry=0xb1700470, x=x@entry=0, y=y@entry=465, w=w@entry=16, h=h@entry=16, colorPtr=colorPtr@entry=0xb0db8438, 
    needSameColor=needSameColor@entry=0 '\000') at tight.c:570
#2  0xb6e45aae in SendRectEncodingTight (cl=cl@entry=0xb1700470, x=x@entry=0, y=433, w=w@entry=432, h=h@entry=335) at tight.c:388
#3  0xb6e45fba in SendRectEncodingTight (cl=cl@entry=0xb1700470, x=0, y=0, w=432, h=768) at tight.c:450
#4  0xb6e460f6 in rfbSendRectEncodingTight (cl=cl@entry=0xb1700470, x=<optimized out>, y=<optimized out>, w=<optimized out>, h=h@entry=768) at tight.c:275
#5  0xb6e2cf7c in rfbSendFramebufferUpdate (cl=cl@entry=0xb1700470, givenUpdateRegion=givenUpdateRegion@entry=0xb0200470) at rfbserver.c:3165
#6  0xb6e27e0c in clientOutput (data=0xb1700470) at main.c:497
#7  0xb68b65b4 in start_thread (arg=0x0) at pthread_create.c:335
#8  0xb6967c5c in ?? () at ../sysdeps/unix/sysv/linux/arm/clone.S:89 from /lib/arm-linux-gnueabihf/libc.so.6
Backtrace stopped: previous frame identical to this frame (corrupt stack?)