Video on StreamingVideoSurface is inverted, or not depending on whose PC runs it

flyfisher1952

Hi,

I have an odd situation regarding streaming video to a StreamingVideoSurface object. Another developer and I are doing a Qt app targeting Windows and MacOs. What we are seeing is when I run the app on my Windows 8 PC the video image is inverted, while the other developer has the image right side up. We also package up the video/audio stream and send it to a remote server. The video displayed there is the opposite of what our app displays. So my video is right side up and the other developer's image is upside down. We used code straight out of the tutorial to create the StreamingSurface class that follows:

@#include "streamingvideosurface.h"
#include <QVideoSurfaceFormat>
#include <QPainter>

StreamingVideoSurface::StreamingVideoSurface(QWidget *realSurface, QObject *parent) :
QAbstractVideoSurface(parent),
realSurface(realSurface)
{
}

QListQVideoFrame::PixelFormat StreamingVideoSurface::supportedPixelFormats(QAbstractVideoBuffer::HandleType handleType) const
{
if (handleType == QAbstractVideoBuffer::NoHandle) {
return QListQVideoFrame::PixelFormat()
<< QVideoFrame::Format_RGB32
<< QVideoFrame::Format_ARGB32
<< QVideoFrame::Format_ARGB32_Premultiplied
<< QVideoFrame::Format_RGB565
<< QVideoFrame::Format_RGB555;
} else {
return QListQVideoFrame::PixelFormat();
}
}

bool StreamingVideoSurface::start(const QVideoSurfaceFormat &format)
{
const QImage::Format imageFormat = QVideoFrame::imageFormatFromPixelFormat(format.pixelFormat());
const QSize size = format.frameSize();

if (imageFormat != QImage::Format_Invalid && !size.isEmpty()) {
    this->imageFormat = imageFormat;
    imageSize = size;
    sourceRect = format.viewport();

    QAbstractVideoSurface::start(format);

    realSurface->updateGeometry();
    updateVideoRect();

    return true;
} else {
    return false;
}

}

void StreamingVideoSurface::stop()
{
currentFrame = QVideoFrame();
targetRect = QRect();

QAbstractVideoSurface::stop();

realSurface->update();

}

bool StreamingVideoSurface::present(const QVideoFrame &frame)
{
if (surfaceFormat().pixelFormat() != frame.pixelFormat()
|| surfaceFormat().frameSize() != frame.size()) {
setError(IncorrectFormatError);
stop();

     return false;
 } else {
     emit framePresented(frame);

     currentFrame = frame;
     realSurface->repaint(targetRect);

     return true;
 }

}

void StreamingVideoSurface::updateVideoRect()
{
QSize size = surfaceFormat().sizeHint();
size.scale(realSurface->size().boundedTo(size), Qt::KeepAspectRatio);

targetRect = QRect(QPoint(0, 0), size);
targetRect.moveCenter(realSurface->rect().center());

}

void StreamingVideoSurface::paint(QPainter *painter)
{
if (currentFrame.map(QAbstractVideoBuffer::ReadOnly)) {
const QTransform oldTransform = painter->transform();
const QVideoSurfaceFormat sf = surfaceFormat();

// seems we need this after all
if (sf.scanLineDirection() == QVideoSurfaceFormat::BottomToTop) {
painter->scale(1, -1);
painter->translate(0, -realSurface->height());
}

    QImage image(
         currentFrame.bits(),
         currentFrame.width(),
         currentFrame.height(),
         currentFrame.bytesPerLine(),
         imageFormat);

    painter->drawImage(targetRect, image, sourceRect);

    painter->setTransform(oldTransform);

    currentFrame.unmap();
}

}

bool StreamingVideoSurface::isFormatSupported(const QVideoSurfaceFormat &format) const
{
const QImage::Format imageFormat = QVideoFrame::imageFormatFromPixelFormat(format.pixelFormat());
const QSize size = format.frameSize();

return imageFormat != QImage::Format_Invalid
     && !size.isEmpty()
     && format.handleType() == QAbstractVideoBuffer::NoHandle;

}
@

Any ideas on what we should look for to solve this issue would be greatly appreciated.

Mike

SGaist

Hi and welcome to devnet,

Are you all using the same Qt package ? Same hardware and drivers ?

flyfisher1952

HI,

We are and aren't. He is developing on MacOS while I'm using Windows. He runs Windows in a VM so he compiles and tests in 2 OS's. We both are using Logitech C920 webcams. We've sent the app to other Windows developers in our company and they have the same experience that the other developer has, the app shows the video right side up while the remote server shows it inverted. I'm not sure what their webcams are.

It doesn't seem to be tied to that since dev 2 can attach to the Mac built-in webcam and get the same results.

thanks,
MIke

SGaist

IIRC, currently on OS X the webcam image is not mirrored e.g. you look to the left and your image looks to the right (at least for the embedded camera)

flyfisher1952

This has to do with the image being upside down, not reversed.

SGaist

Can you explain what you mean by reversed ?

flyfisher1952

The image is upside down and backwards. In the following picture I am looking to my right while the image shows me looking left as well as being upside down. Guess I should have included the picture in the original post so it would be clear.

!https://www.dropbox.com/s/gzhee50debor2le/Capture.PNG(Me looking right, but upside down and reversed in the captured image.)!

SGaist

Sorry, but I can't see the image you posted

flyfisher1952

I tried to insert it as an image, here is the link.

https://www.dropbox.com/s/gzhee50debor2le/Capture.PNG

SGaist

Ok, so flipped horizontally and vertically. Just to be sure we're on the same base: you (or one of your colleagues) are getting this "doubly flipped" image on OS X even with the embedded iSight camera ? Correct ?

Can you add which version of Qt you are using and which version of OS X ?

flyfisher1952

I am on Windows 8.1 (all updates applied) and using Qt 5.3. My colleague sees the image unflipped in both directions but the video going to the remove server is flipped h & v.

I've sent my partner the link to this thread so he can put in his 2 cents :)

realrunner

I'm the colleague.

Here's the situation.

Running the untainted "QML Camera Example" or the "Camera Example" my colleague's preview in the viewfinder is flipped both horizontally and vertically. His specs, as stated above are Windows 8.1 with a Logitech C920 USB web cam.

If I run the same examples my image is just fine. I've tried with the following setups: OS X Mavericks w/ Face Time built in camera, OS X with Logitech C920, Windows 7 (in parallels) with both cameras and Windows 7 (direct, no vm) with both cameras. I've also created a build and distributed it to two other developers in our company running windows and their images are fine.

We're all running the latest stable Qt (5.3). I've dug through the code and examples and am unable to find any difference between our two setups.

Unfortunately the DirectShow plugin is pretty weak when it comes to webcam capture. There is no control over resolution, zoom, anything really, just preview and snapshot.

I'm going to start digging into the DirectShow plugin directly this week to try and figure out what's going on and perhaps, along the way, expose some of its missing functionality.

SGaist

Hi and welcome to devnet !

Then I wonder if there's a difference in the webcam drivers for 8.1 and 7. I don't remember if the WMF plugin has better support for webcam but it might be worth a try.

In the case you implement these missing functionalities and would like to submit them to the Qt project, you should directly start with the dev branch of QtMultimedia. Well you should take a look there anyway, IIRC, there's some work currently done on the DirectShow plugin.

realrunner

Looking at the WMF code it looks like it has no support for webcam capture. I'll try digging into the DirectShow code. If there is a difference, hopefully something is exposed there.

Thanks.

davecotter

this worked for me:
format.setScanLineDirection(QVideoSurfaceFormat::BottomToTop);