Dump QImage raw pixel data into `std::vector<char>`

Christian Ehrlicher

@abmyii said in Dump QImage raw pixel data into `std::vector<char>`:

How does having -fsantize=address cause it to stop failing?

When it's a thread issue then the timing is now different

Also is there any way to debug it?

With a debugger, yes. Examine your threads when it crashes

abmyii

(gdb) thread apply all bt

Thread 6 (Thread 0xb0dfdbc0 (LWP 4458)):
#0  0xb6e433a2 in ?? () from /usr/lib/arm-linux-gnueabihf/libvncserver.so.1
#1  0x00000c00 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

Thread 5 (Thread 0xb15fdbc0 (LWP 4457)):
#0  0xb69634e2 in select () at ../sysdeps/unix/syscall-template.S:84
#1  0xb6e282c6 in ?? () from /usr/lib/arm-linux-gnueabihf/libvncserver.so.1
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

Thread 4 (Thread 0xb20b0bc0 (LWP 4455)):
#0  0xb69634e2 in select () at ../sysdeps/unix/syscall-template.S:84
#1  0xb6e28868 in ?? () from /usr/lib/arm-linux-gnueabihf/libvncserver.so.1
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

Thread 3 (Thread 0xb2f0abc0 (LWP 4454)):
#0  __libc_do_syscall () at ../sysdeps/unix/sysv/linux/arm/libc-do-syscall.S:46
#1  0xb68badba in __pthread_cond_wait (cond=0x72f58, mutex=0x728f0) at pthread_cond_wait.c:186
#2  0xb2f93ff4 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

Thread 2 (Thread 0xb3dd9bc0 (LWP 4453)):
#0  0xb6961c00 in poll () at ../sysdeps/unix/syscall-template.S:84
#1  0xb684c7c8 in ?? () from /usr/lib/arm-linux-gnueabihf/libmircommon.so.7
#2  0xb6851038 in ?? () from /usr/lib/arm-linux-gnueabihf/libmircommon.so.7
#3  0xb6d9edc8 in ?? () from /usr/lib/arm-linux-gnueabihf/libstdc++.so.6
#4  0xb68b65b4 in start_thread (arg=0x0) at pthread_create.c:335
#5  0xb6967c5c in ?? () at ../sysdeps/unix/sysv/linux/arm/clone.S:89 from /lib/arm-linux-gnueabihf/libc.so.6
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

Thread 1 (Thread 0xb40ea000 (LWP 4441)):
#0  0xb69658d6 in munmap () at ../sysdeps/unix/syscall-template.S:84
#1  0xb692480e in munmap_chunk (p=<optimized out>) at malloc.c:2860
#2  0xb6ae6f12 in QImageData::~QImageData() () from /usr/lib/arm-linux-gnueabihf/libQt5Gui.so.5
#3  0xb6ae7122 in QImage::~QImage() () from /usr/lib/arm-linux-gnueabihf/libQt5Gui.so.5
#4  0x00019df6 in (anonymous namespace)::read_pixels(int, unsigned int, mir::geometry::Size const&, std::vector<char, std::allocator<char> >&) ()
#5  0x0001b130 in (anonymous namespace)::do_screencast((anonymous namespace)::EGLSetup const&, mir::geometry::Size const&, int, double, std::ostream&) ()
#6  0x0001c446 in main ()

Quite indecipherable to me but the last (#1) thread's backtrace looks promising...

Christian Ehrlicher

@abmyii said in Dump QImage raw pixel data into `std::vector<char>`:

#4 0x00019df6 in (anonymous namespace)::read_pixels(int, unsigned int, mir::geometry::Size const&, std::vector<char, std::allocator<char> >&) ()

What happens here?
Also try to install the debug symbols of libvncserver

abmyii

@Christian-Ehrlicher


void read_pixels(int bpp, GLenum format, mir::geometry::Size const& size, std::vector<char> &fb)
{
    auto width = size.width.as_uint32_t();
    auto height = size.height.as_uint32_t();

    // Read pixels into image (https://community.khronos.org/t/render-to-texture-under-qt/68430/3)
    QImage image(width, height, QImage::Format_RGBA8888);
    glReadPixels(0, 0, width, height, format, GL_UNSIGNED_BYTE, image.bits());

    if (mirror)
	image = image.mirrored(true, true);

    if (landscape) {
        //QPoint center = image.rect().center();
        QMatrix matrix;
        matrix.translate(0, 0);
        matrix.rotate(90);

        image = image.transformed(matrix);
    }

    std::cout << "FB size: " << fb.size() << ", Image byteCount: " << image.byteCount() << std::endl;
    std::memcpy(fb.data(), (const char *)image.constBits(), image.byteCount());
#ifdef DEBUG
    unsigned int sum = 0;
    unsigned int *pui = (unsigned int *) buffer;
    for (int i = 0; i < width * height; i++) {
        sum += *(pui + i);
    }
    printf("read pixels sum 0x%x\n", sum);
#endif
}

Sure, I'll try that.

abmyii

@Christian-Ehrlicher

Thread 6 "mirvncserver" received signal SIGBUS, Bus error.
[Switching to Thread 0xb0db8bc0 (LWP 3495)]
0xb6e433b2 in CheckSolidTile32 (cl=0xb1700470, needSameColor=0 '\000', colorPtr=0xb0db8438, h=16, w=16, y=-1231936575, x=20) at tight.c:611
611	tight.c: No such file or directory.

(gdb) thread apply all bt

Thread 6 (Thread 0xb0db8bc0 (LWP 3495)):
#0  0xb6e433b2 in CheckSolidTile32 (cl=0xb1700470, needSameColor=0 '\000', colorPtr=0xb0db8438, h=16, w=16, y=-1231936575, x=20) at tight.c:611
#1  CheckSolidTile (cl=cl@entry=0xb1700470, x=x@entry=0, y=y@entry=465, w=w@entry=16, h=h@entry=16, colorPtr=colorPtr@entry=0xb0db8438, 
    needSameColor=needSameColor@entry=0 '\000') at tight.c:570
#2  0xb6e45aae in SendRectEncodingTight (cl=cl@entry=0xb1700470, x=x@entry=0, y=433, w=w@entry=432, h=h@entry=335) at tight.c:388
#3  0xb6e45fba in SendRectEncodingTight (cl=cl@entry=0xb1700470, x=0, y=0, w=432, h=768) at tight.c:450
#4  0xb6e460f6 in rfbSendRectEncodingTight (cl=cl@entry=0xb1700470, x=<optimized out>, y=<optimized out>, w=<optimized out>, h=h@entry=768) at tight.c:275
#5  0xb6e2cf7c in rfbSendFramebufferUpdate (cl=cl@entry=0xb1700470, givenUpdateRegion=givenUpdateRegion@entry=0xb0200470) at rfbserver.c:3165
#6  0xb6e27e0c in clientOutput (data=0xb1700470) at main.c:497
#7  0xb68b65b4 in start_thread (arg=0x0) at pthread_create.c:335
#8  0xb6967c5c in ?? () at ../sysdeps/unix/sysv/linux/arm/clone.S:89 from /lib/arm-linux-gnueabihf/libc.so.6
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

Christian Ehrlicher

@abmyii said in Dump QImage raw pixel data into `std::vector<char>`:

std::vector<char> &fb

Is this used in another thread?

Otherwise I don't see anything obvious.

abmyii

@Christian-Ehrlicher I think the while loop is in another thread and VNC is accessing fb.data() throughout execution.

Christian Ehrlicher

@abmyii So you have your answer. Concurrent access to a memory region from two different threads.

abmyii

@Christian-Ehrlicher Ah I see, that makes perfect sense now! Is there a simple solution to this? I tried the following which didn't work:

std::vector<char> temp_fb(frame_size_bytes, 0);
std::memcpy(temp_fb.data(), (const char *)image.constBits(), image.byteCount());
fb = temp_fb

Christian Ehrlicher

Use mutexes or copy the data - I would suggest reading a little bit on how threading works. E.g. https://doc.qt.io/qt-5/threads-synchronizing.html - c++ uses the same mechanisms

abmyii

Whoops, I've been saying malloc (which is apparently thread-safe) all along when in fact the actual function I'm using is memcpy... Does that change things and/or can I use malloc instead?

JonB

@abmyii
Hence what @Christian-Ehrlicher has been saying above. malloc and memcpy do totally different things, it makes no sense to ask whether one can be used in place of the other....

abmyii

@JonB I understand - I got confused due to my unfamiliarity with the functions but I never asked if I could use malloc instead of memcpy - I asked if I could use push_back instead of malloc (sic).

JonB

@abmyii said in Dump QImage raw pixel data into `std::vector<char>`:

I never asked if I could use malloc instead of memcpy [sic.]

in fact the actual function I'm using is memcpy... Does that change things and/or can I use malloc instead? [*sic.]

If you say so! I'll leave others to untangle, who clearly understand what you are saying better than I :) Don't worry about explaining to me.

abmyii

@JonB OK, you got me! Too much to keep track of...

I tried this and it worked, but is quite a lot slower than memcpy:

std::vector<unsigned char> vectorBuffer(charBuffer, charBuffer + length);
fb = vectorBuffer;

JonB

@abmyii
I am lost in the complexities of your questions :) But I would suggest heeding @Christian-Ehrlicher's advice, he is usually right! He last offered:

Use mutexes or copy the data

So why not pursue that?

Christian Ehrlicher

@abmyii said in Dump QImage raw pixel data into `std::vector<char>`:

std::vector<unsigned char> vectorBuffer(charBuffer, charBuffer + length);
fb = vectorBuffer;

This does two memcpy for no good reason. Since I don't know what libvncserver function you use you're on your own. Read the documentation of them and also read what's needed for threadsafe access.

abmyii

@JonB Haha, you think you are the only one?!

I am planning to try the mutex idea - I haven't had a chance to look it up and figure out how to do it yet. A basic QMutex lock/unlock didn't fix it (though I probably applied it wrong)...

@Christian-Ehrlicher Ah, I see. What "libvncserver function" do you mean?

Christian Ehrlicher

@abmyii said in Dump QImage raw pixel data into `std::vector<char>`:

What "libvncserver function" do you mean?

You are using libvncserver, not me.

Giving up here now.

abmyii

Going to mark this thread as closed since it seems the issue lies with VNC, not Qt. Thank you all for your suggestions and helping me to isolate the problem.

@Christian-Ehrlicher Sorry for stressing you out - I thought you were referring to a specific function for some reason. I really need to slow down when reading and comprehend rather than just having a quick look and then replying which is causing confusion on all sides. Thank you again for your patience and assistance throughout.