Dump QImage raw pixel data into `std::vector<char>`

[abmyii]

@Christian-Ehrlicher What I mean is that are there other methods that don't require malloc? I want to try those to see if using those methods causes the crashes to stop. As for allocating enough pixels I checked that some time ago and then removed the code but I've added it back in with this output (doesn't change throughout the execution of the program):

FB size: 1327104, Image byteCount: 1327104

Valgrind didn't crash (unfortunately) but there is obviously an issue somewhere, so now finding it isn't going to be easy. I do believe it's on the VNC side, not Qt, but is being induced somehow by the malloc.

Compiling with Clang instead of G++ isn't straightforward for me at my level of knowledge. On compiling I get the following warnings:

clang mirvncserver.cpp -c -std=c++11 -Wall -fpermissive -I/usr/include/mirclient -I/usr/include/mircommon -I/usr/include/mircore -I/usr/include/libevdev-1.0 -I/usr/include/arm-linux-gnueabihf/qt5 -fPIC -fsanitize=address -O1 -fno-omit-frame-pointer -g -o mirvncserver-clang -lboost_program_options -lpthread -lmirclient -lEGL -lxcb-glx -lGLESv2 -lmirserver -lmircore -levdev -lvncserver -lstdc++ -lQt5Gui -lQt5Core

clang: warning: -lboost_program_options: 'linker' input unused
clang: warning: -lpthread: 'linker' input unused
clang: warning: -lmirclient: 'linker' input unused
clang: warning: -lEGL: 'linker' input unused
clang: warning: -lxcb-glx: 'linker' input unused
clang: warning: -lGLESv2: 'linker' input unused
clang: warning: -lmirserver: 'linker' input unused
clang: warning: -lmircore: 'linker' input unused
clang: warning: -levdev: 'linker' input unused
clang: warning: -lvncserver: 'linker' input unused
clang: warning: -Z-reserved-lib-stdc++: 'linker' input unused
clang: warning: -lQt5Gui: 'linker' input unused
clang: warning: -lQt5Core: 'linker' input unused

And the output file is not executable mirvncserver-clang: ELF 32-bit LSB relocatable, ARM, EABI5 version 1 (SYSV), not stripped. I'm sure I'm making a very basic error, but I'm not sure what it is.

[Christian Ehrlicher]

@abmyii said in Dump QImage raw pixel data into &#x60;std::vector<char>&#x60;:

want to try those to see if using those methods causes the crashes to stop

You're aware that all functions which allocate memory is using malloc (C) or new (C++)? So what do you think you gain. If your allocated enough memory than all is fine. Otherwise use a debugger and examine your stack trace

Why do you want to compile your program with clang now? I said you should use the Address Sanitizer - no need to use clang.

[abmyii]

@Christian-Ehrlicher said in Dump QImage raw pixel data into &#x60;std::vector<char>&#x60;:

@abmyii said in Dump QImage raw pixel data into &#x60;std::vector<char>&#x60;:

want to try those to see if using those methods causes the crashes to stop

You're aware that all functions which allocate memory is using malloc (C) or new (C++)? So what do you think you gain. If your allocated enough memory than all is fine. Otherwise use a debugger and examine your stack trace

Why do you want to compile your program with clang now? I said you should use the Address Sanitizer - no need to use clang.

Ah, according to the instructions in the repo you must compile with clang (https://github.com/google/sanitizers/wiki/AddressSanitizer#using-addresssanitizer). How can I do that with G++?

Rather than mallocing I was thinking I could push_back or set the pixel data using something like fb[pos] = (char)qRed(pixel), but I would definitely prefer malloc if I can find the issue...

[Christian Ehrlicher]

@abmyii said in Dump QImage raw pixel data into &#x60;std::vector<char>&#x60;:

Rather than mallocing I was thinking I could push_back

Ok, please re-read and think over again. malloc and push_back don't have anything in common.

asan: http://stackoverflow.com/questions/37970758/ddg#40215639 for example

[abmyii]

@Christian-Ehrlicher said in Dump QImage raw pixel data into &#x60;std::vector<char>&#x60;:

Ok, please re-read and think over again. malloc and push_back don't have anything in common.

Sorry, perhaps I'm misunderstanding but that is the whole point - I want to test if using a different method other than malloc for copying the data will fix the issue.

Thank you for the link, I'll try that now.

[Christian Ehrlicher]

@abmyii said in Dump QImage raw pixel data into &#x60;std::vector<char>&#x60;:

other than malloc for copying the data will fix the issue.

Again: malloc does not copy anything. It allocates memory!

[abmyii]

@Christian-Ehrlicher Other than an initial issue:

=================================================================
==463==ERROR: AddressSanitizer: alloc-dealloc-mismatch (operator new vs free) on 0xb1800850
    #0 0xb6af11d7 in free (/usr/lib/arm-linux-gnueabihf/libasan.so.2+0x751d7)
    #1 0xafddfb31  (/usr/lib/arm-linux-gnueabihf/libhybris/linker/mm.so+0x9b31)
    #2 0xafde0b03  (/usr/lib/arm-linux-gnueabihf/libhybris/linker/mm.so+0xab03)
    #3 0xafde1153 in do_dlopen(char const*, int, android_dlextinfo const*) (/usr/lib/arm-linux-gnueabihf/libhybris/linker/mm.so+0xb153)
    #4 0xafddc5bd  (/usr/lib/arm-linux-gnueabihf/libhybris/linker/mm.so+0x65bd)

0xb1800850 is located 0 bytes inside of 33-byte region [0xb1800850,0xb1800871)
allocated by thread T0 here:
    #0 0xb6af1e4b in operator new(unsigned int) (/usr/lib/arm-linux-gnueabihf/libasan.so.2+0x75e4b)
    #1 0xb6fd9ce7  (/lib/ld-linux-armhf.so.3+0x27ce7)

SUMMARY: AddressSanitizer: alloc-dealloc-mismatch ??:0 free
==463==HINT: if you don't care about these warnings you may set ASAN_OPTIONS=alloc_dealloc_mismatch=0
==463==ABORTING

Which goes away by setting the variable it works absolutely fine... Now I'm even more confused...

Again: malloc does not copy anything. It allocates memory!

Ah, whoops, my mistake - and what a mistake, haha! That's what happens when you go from Python to C++... So I assume that the malloc sets the vector a new memory range to "look at"?

[Christian Ehrlicher]

@abmyii malloc allocates memory - nothing more.
The asan output is fine - no problems in your code. Only somwhere memory is allocated with new and deallocated with free() or allocated with malloc and deallocated with delete which is not allowed.

[abmyii]

@Christian-Ehrlicher That's good to hear! I wonder what's causing the issue, then - as soon as I remove -fsanitize=address it starts failing again. Can I leave -fsanitize=address in for a release binary? Even if it was OK, I'd need to find a way to fix the above issue when running without the env variable. I'll have a look to see if I can find the offending lines of code.

Edit: Didn't find any (especially with my untrained eyes...). The only malloc is the image one.

[Christian Ehrlicher]

@abmyii said in Dump QImage raw pixel data into &#x60;std::vector<char>&#x60;:

it starts failing again.

This really sounds more like a threading issue.

[abmyii]

@Christian-Ehrlicher How does having -fsantize=address cause it to stop failing? Also is there any way to debug it?

[Christian Ehrlicher]

@abmyii said in Dump QImage raw pixel data into &#x60;std::vector<char>&#x60;:

How does having -fsantize=address cause it to stop failing?

When it's a thread issue then the timing is now different

Also is there any way to debug it?

With a debugger, yes. Examine your threads when it crashes

[abmyii]

@Christian-Ehrlicher

(gdb) thread apply all bt

Thread 6 (Thread 0xb0dfdbc0 (LWP 4458)):
#0  0xb6e433a2 in ?? () from /usr/lib/arm-linux-gnueabihf/libvncserver.so.1
#1  0x00000c00 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

Thread 5 (Thread 0xb15fdbc0 (LWP 4457)):
#0  0xb69634e2 in select () at ../sysdeps/unix/syscall-template.S:84
#1  0xb6e282c6 in ?? () from /usr/lib/arm-linux-gnueabihf/libvncserver.so.1
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

Thread 4 (Thread 0xb20b0bc0 (LWP 4455)):
#0  0xb69634e2 in select () at ../sysdeps/unix/syscall-template.S:84
#1  0xb6e28868 in ?? () from /usr/lib/arm-linux-gnueabihf/libvncserver.so.1
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

Thread 3 (Thread 0xb2f0abc0 (LWP 4454)):
#0  __libc_do_syscall () at ../sysdeps/unix/sysv/linux/arm/libc-do-syscall.S:46
#1  0xb68badba in __pthread_cond_wait (cond=0x72f58, mutex=0x728f0) at pthread_cond_wait.c:186
#2  0xb2f93ff4 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

Thread 2 (Thread 0xb3dd9bc0 (LWP 4453)):
#0  0xb6961c00 in poll () at ../sysdeps/unix/syscall-template.S:84
#1  0xb684c7c8 in ?? () from /usr/lib/arm-linux-gnueabihf/libmircommon.so.7
#2  0xb6851038 in ?? () from /usr/lib/arm-linux-gnueabihf/libmircommon.so.7
#3  0xb6d9edc8 in ?? () from /usr/lib/arm-linux-gnueabihf/libstdc++.so.6
#4  0xb68b65b4 in start_thread (arg=0x0) at pthread_create.c:335
#5  0xb6967c5c in ?? () at ../sysdeps/unix/sysv/linux/arm/clone.S:89 from /lib/arm-linux-gnueabihf/libc.so.6
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

Thread 1 (Thread 0xb40ea000 (LWP 4441)):
#0  0xb69658d6 in munmap () at ../sysdeps/unix/syscall-template.S:84
#1  0xb692480e in munmap_chunk (p=<optimized out>) at malloc.c:2860
#2  0xb6ae6f12 in QImageData::~QImageData() () from /usr/lib/arm-linux-gnueabihf/libQt5Gui.so.5
#3  0xb6ae7122 in QImage::~QImage() () from /usr/lib/arm-linux-gnueabihf/libQt5Gui.so.5
#4  0x00019df6 in (anonymous namespace)::read_pixels(int, unsigned int, mir::geometry::Size const&, std::vector<char, std::allocator<char> >&) ()
#5  0x0001b130 in (anonymous namespace)::do_screencast((anonymous namespace)::EGLSetup const&, mir::geometry::Size const&, int, double, std::ostream&) ()
#6  0x0001c446 in main ()

Quite indecipherable to me but the last (#1) thread's backtrace looks promising...

[Christian Ehrlicher]

@abmyii said in Dump QImage raw pixel data into &#x60;std::vector<char>&#x60;:

#4 0x00019df6 in (anonymous namespace)::read_pixels(int, unsigned int, mir::geometry::Size const&, std::vector<char, std::allocator<char> >&) ()

What happens here?
Also try to install the debug symbols of libvncserver

[abmyii]

@Christian-Ehrlicher

void read_pixels(int bpp, GLenum format, mir::geometry::Size const& size, std::vector<char> &fb)
{
    auto width = size.width.as_uint32_t();
    auto height = size.height.as_uint32_t();

    // Read pixels into image (https://community.khronos.org/t/render-to-texture-under-qt/68430/3)
    QImage image(width, height, QImage::Format_RGBA8888);
    glReadPixels(0, 0, width, height, format, GL_UNSIGNED_BYTE, image.bits());

    if (mirror)
	image = image.mirrored(true, true);

    if (landscape) {
        //QPoint center = image.rect().center();
        QMatrix matrix;
        matrix.translate(0, 0);
        matrix.rotate(90);

        image = image.transformed(matrix);
    }

    std::cout << "FB size: " << fb.size() << ", Image byteCount: " << image.byteCount() << std::endl;
    std::memcpy(fb.data(), (const char *)image.constBits(), image.byteCount());
#ifdef DEBUG
    unsigned int sum = 0;
    unsigned int *pui = (unsigned int *) buffer;
    for (int i = 0; i < width * height; i++) {
        sum += *(pui + i);
    }
    printf("read pixels sum 0x%x\n", sum);
#endif
}

Sure, I'll try that.

[abmyii]

@Christian-Ehrlicher

Thread 6 "mirvncserver" received signal SIGBUS, Bus error.
[Switching to Thread 0xb0db8bc0 (LWP 3495)]
0xb6e433b2 in CheckSolidTile32 (cl=0xb1700470, needSameColor=0 '\000', colorPtr=0xb0db8438, h=16, w=16, y=-1231936575, x=20) at tight.c:611
611	tight.c: No such file or directory.

(gdb) thread apply all bt

Thread 6 (Thread 0xb0db8bc0 (LWP 3495)):
#0  0xb6e433b2 in CheckSolidTile32 (cl=0xb1700470, needSameColor=0 '\000', colorPtr=0xb0db8438, h=16, w=16, y=-1231936575, x=20) at tight.c:611
#1  CheckSolidTile (cl=cl@entry=0xb1700470, x=x@entry=0, y=y@entry=465, w=w@entry=16, h=h@entry=16, colorPtr=colorPtr@entry=0xb0db8438, 
    needSameColor=needSameColor@entry=0 '\000') at tight.c:570
#2  0xb6e45aae in SendRectEncodingTight (cl=cl@entry=0xb1700470, x=x@entry=0, y=433, w=w@entry=432, h=h@entry=335) at tight.c:388
#3  0xb6e45fba in SendRectEncodingTight (cl=cl@entry=0xb1700470, x=0, y=0, w=432, h=768) at tight.c:450
#4  0xb6e460f6 in rfbSendRectEncodingTight (cl=cl@entry=0xb1700470, x=<optimized out>, y=<optimized out>, w=<optimized out>, h=h@entry=768) at tight.c:275
#5  0xb6e2cf7c in rfbSendFramebufferUpdate (cl=cl@entry=0xb1700470, givenUpdateRegion=givenUpdateRegion@entry=0xb0200470) at rfbserver.c:3165
#6  0xb6e27e0c in clientOutput (data=0xb1700470) at main.c:497
#7  0xb68b65b4 in start_thread (arg=0x0) at pthread_create.c:335
#8  0xb6967c5c in ?? () at ../sysdeps/unix/sysv/linux/arm/clone.S:89 from /lib/arm-linux-gnueabihf/libc.so.6
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

[Christian Ehrlicher]

@abmyii said in Dump QImage raw pixel data into &#x60;std::vector<char>&#x60;:

std::vector<char> &fb

Is this used in another thread?

Otherwise I don't see anything obvious.

[abmyii]

@Christian-Ehrlicher I think the while loop is in another thread and VNC is accessing fb.data() throughout execution.

[Christian Ehrlicher]

@abmyii So you have your answer. Concurrent access to a memory region from two different threads.

[abmyii]

@Christian-Ehrlicher Ah I see, that makes perfect sense now! Is there a simple solution to this? I tried the following which didn't work:

std::vector<char> temp_fb(frame_size_bytes, 0);
std::memcpy(temp_fb.data(), (const char *)image.constBits(), image.byteCount());
fb = temp_fb