Dump QImage raw pixel data into `std::vector<char>`
-
@Christian-Ehrlicher With valgrind it didn't crash, but also the program was a lot slower so perhaps when it runs at normal speed the issue occurs?
And yup, as for the
byteCount
, that's what I was referring to when I said I checked the sizes.image.byteCount()
andframe_buffer.size()
.(Replying to 2 in 1 because of the 600s posting limit...)
BTW thank you all for the speedy suggestions/replies!
-
Beware byteCount is obsolete.
Are you doing something multi-threaded ?
-
In your first post you use a std::vector<char>() as container, in your last post a plain pointer. How do you allocate this memory?
valgrind is slow, yes - therefore I suggested AdressSanitizer or let it run for an hour and see what happens with valgrind.
-
@SGaist said in Dump QImage raw pixel data into `std::vector<char>`:
Beware byteCount is obsolete.
Are you doing something multi-threaded ?
Yes, but not the image -> fb conversion.
@Christian-Ehrlicher said in Dump QImage raw pixel data into `std::vector<char>`:
In your first post you use a std::vector<char>() as container, in your last post a plain pointer. How do you allocate this memory?
valgrind is slow, yes - therefore I suggested AdressSanitizer or let it run for an hour and see what happens with valgrind.
The
memcpy
is in a function, andfb
is a pointer toframe_data.data()
as in the original example.The full code is hosted on GitLab (https://gitlab.com/abmyii/ubports-mir-vnc-server/-/blob/e3be9585007ea2b663139fd2f5345745392cf9c7/mirvncserver.cpp) but is quite messy. The relevant bits are:
https://gitlab.com/abmyii/ubports-mir-vnc-server/-/blob/e3be9585007ea2b663139fd2f5345745392cf9c7/mirvncserver.cpp#L94-123 (The function for reading the pixels)
https://gitlab.com/abmyii/ubports-mir-vnc-server/-/blob/e3be9585007ea2b663139fd2f5345745392cf9c7/mirvncserver.cpp#L586-591 (The frame buffer initialisation)https://gitlab.com/abmyii/ubports-mir-vnc-server/-/blob/e3be9585007ea2b663139fd2f5345745392cf9c7/mirvncserver.cpp#L619 (The function call)
There is a chance that the
Bus Error
is occurring at the VNC level though... -
I'm trying (and failing) to get it compiled with Clang + AddressSanitizer (linking errors - probably not converting from GCC correctly), but I was thinking - is there any reasonably fast method to copy the pixels without using
malloc
just to ensure that it is(n't) themalloc
causing the issue? -
@abmyii said in Dump QImage raw pixel data into `std::vector<char>`:
copy the pixels without using malloc just to ensure that it is(n't) the malloc causing the issue?
What should be faster? When you allocate enough pixels (which you still did not check, at least in your code we've seen until now) than all is fine. Otherwise also valgrind would have told you.
-
@Christian-Ehrlicher What I mean is that are there other methods that don't require
malloc
? I want to try those to see if using those methods causes the crashes to stop. As for allocating enough pixels I checked that some time ago and then removed the code but I've added it back in with this output (doesn't change throughout the execution of the program):FB size: 1327104, Image byteCount: 1327104
Valgrind didn't crash (unfortunately) but there is obviously an issue somewhere, so now finding it isn't going to be easy. I do believe it's on the VNC side, not Qt, but is being induced somehow by the
malloc
.Compiling with Clang instead of G++ isn't straightforward for me at my level of knowledge. On compiling I get the following warnings:
clang mirvncserver.cpp -c -std=c++11 -Wall -fpermissive -I/usr/include/mirclient -I/usr/include/mircommon -I/usr/include/mircore -I/usr/include/libevdev-1.0 -I/usr/include/arm-linux-gnueabihf/qt5 -fPIC -fsanitize=address -O1 -fno-omit-frame-pointer -g -o mirvncserver-clang -lboost_program_options -lpthread -lmirclient -lEGL -lxcb-glx -lGLESv2 -lmirserver -lmircore -levdev -lvncserver -lstdc++ -lQt5Gui -lQt5Core clang: warning: -lboost_program_options: 'linker' input unused clang: warning: -lpthread: 'linker' input unused clang: warning: -lmirclient: 'linker' input unused clang: warning: -lEGL: 'linker' input unused clang: warning: -lxcb-glx: 'linker' input unused clang: warning: -lGLESv2: 'linker' input unused clang: warning: -lmirserver: 'linker' input unused clang: warning: -lmircore: 'linker' input unused clang: warning: -levdev: 'linker' input unused clang: warning: -lvncserver: 'linker' input unused clang: warning: -Z-reserved-lib-stdc++: 'linker' input unused clang: warning: -lQt5Gui: 'linker' input unused clang: warning: -lQt5Core: 'linker' input unused
And the output file is not executable
mirvncserver-clang: ELF 32-bit LSB relocatable, ARM, EABI5 version 1 (SYSV), not stripped
. I'm sure I'm making a very basic error, but I'm not sure what it is. -
@abmyii said in Dump QImage raw pixel data into `std::vector<char>`:
want to try those to see if using those methods causes the crashes to stop
You're aware that all functions which allocate memory is using malloc (C) or new (C++)? So what do you think you gain. If your allocated enough memory than all is fine. Otherwise use a debugger and examine your stack trace
Why do you want to compile your program with clang now? I said you should use the Address Sanitizer - no need to use clang.
-
@Christian-Ehrlicher said in Dump QImage raw pixel data into `std::vector<char>`:
@abmyii said in Dump QImage raw pixel data into `std::vector<char>`:
want to try those to see if using those methods causes the crashes to stop
You're aware that all functions which allocate memory is using malloc (C) or new (C++)? So what do you think you gain. If your allocated enough memory than all is fine. Otherwise use a debugger and examine your stack trace
Why do you want to compile your program with clang now? I said you should use the Address Sanitizer - no need to use clang.
Ah, according to the instructions in the repo you must compile with clang (https://github.com/google/sanitizers/wiki/AddressSanitizer#using-addresssanitizer). How can I do that with G++?
Rather than
malloc
ing I was thinking I couldpush_back
or set the pixel data using something likefb[pos] = (char)qRed(pixel)
, but I would definitely prefermalloc
if I can find the issue... -
@abmyii said in Dump QImage raw pixel data into `std::vector<char>`:
Rather than mallocing I was thinking I could push_back
Ok, please re-read and think over again. malloc and push_back don't have anything in common.
asan: http://stackoverflow.com/questions/37970758/ddg#40215639 for example
-
@Christian-Ehrlicher said in Dump QImage raw pixel data into `std::vector<char>`:
Ok, please re-read and think over again. malloc and push_back don't have anything in common.
Sorry, perhaps I'm misunderstanding but that is the whole point - I want to test if using a different method other than
malloc
for copying the data will fix the issue.Thank you for the link, I'll try that now.
-
@abmyii said in Dump QImage raw pixel data into `std::vector<char>`:
other than malloc for copying the data will fix the issue.
Again: malloc does not copy anything. It allocates memory!
-
@Christian-Ehrlicher Other than an initial issue:
================================================================= ==463==ERROR: AddressSanitizer: alloc-dealloc-mismatch (operator new vs free) on 0xb1800850 #0 0xb6af11d7 in free (/usr/lib/arm-linux-gnueabihf/libasan.so.2+0x751d7) #1 0xafddfb31 (/usr/lib/arm-linux-gnueabihf/libhybris/linker/mm.so+0x9b31) #2 0xafde0b03 (/usr/lib/arm-linux-gnueabihf/libhybris/linker/mm.so+0xab03) #3 0xafde1153 in do_dlopen(char const*, int, android_dlextinfo const*) (/usr/lib/arm-linux-gnueabihf/libhybris/linker/mm.so+0xb153) #4 0xafddc5bd (/usr/lib/arm-linux-gnueabihf/libhybris/linker/mm.so+0x65bd) 0xb1800850 is located 0 bytes inside of 33-byte region [0xb1800850,0xb1800871) allocated by thread T0 here: #0 0xb6af1e4b in operator new(unsigned int) (/usr/lib/arm-linux-gnueabihf/libasan.so.2+0x75e4b) #1 0xb6fd9ce7 (/lib/ld-linux-armhf.so.3+0x27ce7) SUMMARY: AddressSanitizer: alloc-dealloc-mismatch ??:0 free ==463==HINT: if you don't care about these warnings you may set ASAN_OPTIONS=alloc_dealloc_mismatch=0 ==463==ABORTING
Which goes away by setting the variable it works absolutely fine... Now I'm even more confused...
Again: malloc does not copy anything. It allocates memory!
Ah, whoops, my mistake - and what a mistake, haha! That's what happens when you go from Python to C++... So I assume that the
malloc
sets the vector a new memory range to "look at"? -
@abmyii malloc allocates memory - nothing more.
The asan output is fine - no problems in your code. Only somwhere memory is allocated withnew
and deallocated withfree()
or allocated withmalloc
and deallocated withdelete
which is not allowed. -
@Christian-Ehrlicher That's good to hear! I wonder what's causing the issue, then - as soon as I remove
-fsanitize=address
it starts failing again. Can I leave-fsanitize=address
in for a release binary? Even if it was OK, I'd need to find a way to fix the above issue when running without the env variable. I'll have a look to see if I can find the offending lines of code.Edit: Didn't find any (especially with my untrained eyes...). The only
malloc
is the image one. -
@abmyii said in Dump QImage raw pixel data into `std::vector<char>`:
it starts failing again.
This really sounds more like a threading issue.
-
@Christian-Ehrlicher How does having
-fsantize=address
cause it to stop failing? Also is there any way to debug it? -
@abmyii said in Dump QImage raw pixel data into `std::vector<char>`:
How does having -fsantize=address cause it to stop failing?
When it's a thread issue then the timing is now different
Also is there any way to debug it?
With a debugger, yes. Examine your threads when it crashes
-
(gdb) thread apply all bt Thread 6 (Thread 0xb0dfdbc0 (LWP 4458)): #0 0xb6e433a2 in ?? () from /usr/lib/arm-linux-gnueabihf/libvncserver.so.1 #1 0x00000c00 in ?? () Backtrace stopped: previous frame identical to this frame (corrupt stack?) Thread 5 (Thread 0xb15fdbc0 (LWP 4457)): #0 0xb69634e2 in select () at ../sysdeps/unix/syscall-template.S:84 #1 0xb6e282c6 in ?? () from /usr/lib/arm-linux-gnueabihf/libvncserver.so.1 Backtrace stopped: previous frame identical to this frame (corrupt stack?) Thread 4 (Thread 0xb20b0bc0 (LWP 4455)): #0 0xb69634e2 in select () at ../sysdeps/unix/syscall-template.S:84 #1 0xb6e28868 in ?? () from /usr/lib/arm-linux-gnueabihf/libvncserver.so.1 Backtrace stopped: previous frame identical to this frame (corrupt stack?) Thread 3 (Thread 0xb2f0abc0 (LWP 4454)): #0 __libc_do_syscall () at ../sysdeps/unix/sysv/linux/arm/libc-do-syscall.S:46 #1 0xb68badba in __pthread_cond_wait (cond=0x72f58, mutex=0x728f0) at pthread_cond_wait.c:186 #2 0xb2f93ff4 in ?? () Backtrace stopped: previous frame identical to this frame (corrupt stack?) Thread 2 (Thread 0xb3dd9bc0 (LWP 4453)): #0 0xb6961c00 in poll () at ../sysdeps/unix/syscall-template.S:84 #1 0xb684c7c8 in ?? () from /usr/lib/arm-linux-gnueabihf/libmircommon.so.7 #2 0xb6851038 in ?? () from /usr/lib/arm-linux-gnueabihf/libmircommon.so.7 #3 0xb6d9edc8 in ?? () from /usr/lib/arm-linux-gnueabihf/libstdc++.so.6 #4 0xb68b65b4 in start_thread (arg=0x0) at pthread_create.c:335 #5 0xb6967c5c in ?? () at ../sysdeps/unix/sysv/linux/arm/clone.S:89 from /lib/arm-linux-gnueabihf/libc.so.6 Backtrace stopped: previous frame identical to this frame (corrupt stack?) Thread 1 (Thread 0xb40ea000 (LWP 4441)): #0 0xb69658d6 in munmap () at ../sysdeps/unix/syscall-template.S:84 #1 0xb692480e in munmap_chunk (p=<optimized out>) at malloc.c:2860 #2 0xb6ae6f12 in QImageData::~QImageData() () from /usr/lib/arm-linux-gnueabihf/libQt5Gui.so.5 #3 0xb6ae7122 in QImage::~QImage() () from /usr/lib/arm-linux-gnueabihf/libQt5Gui.so.5 #4 0x00019df6 in (anonymous namespace)::read_pixels(int, unsigned int, mir::geometry::Size const&, std::vector<char, std::allocator<char> >&) () #5 0x0001b130 in (anonymous namespace)::do_screencast((anonymous namespace)::EGLSetup const&, mir::geometry::Size const&, int, double, std::ostream&) () #6 0x0001c446 in main ()
Quite indecipherable to me but the last (#1) thread's backtrace looks promising...
-
@abmyii said in Dump QImage raw pixel data into `std::vector<char>`:
#4 0x00019df6 in (anonymous namespace)::read_pixels(int, unsigned int, mir::geometry::Size const&, std::vector<char, std::allocator<char> >&) ()
What happens here?
Also try to install the debug symbols of libvncserver