Convert YouTube (Flash) video frames to QImage problems
-
For a project I'm working on I need to convert a YouTube (Flash) video frame to a QImage. I'm using the method suggested by the Qt documentation for QWebPage but am experiencing problems and would appreciate some help. The method is straightforward and works great for web images but fails for videos under certain conditions (see below). I created a simple basic test that demonstrates the problem. The test code (sans debug statements) is:
MainWindow.h
@class MainWindow : public QMainWindow
{
Q_OBJECTpublic:
explicit MainWindow(QWidget *parent = 0);
~MainWindow();private:
Ui::MainWindow *ui;
private slots:
void loadDone(bool ok);
void loadfin();
};@MainWindow.cpp
@MainWindow::MainWindow(QWidget *parent) :
QMainWindow(parent),
ui(new Ui::MainWindow)
{
ui->setupUi(this);
ui->webView->settings()->setAttribute(QWebSettings::PluginsEnabled,true);
ui->webView->settings()->setAttribute(QWebSettings::LocalContentCanAccessRemoteUrls,true);// QUrl url("http://www.w3schools.com/tags/...");
// QUrl url("http://..happy2.png");
// QUrl url = QUrl::fromLocalFile("C:/...green.html");
// QUrl url = QUrl::fromLocalFile("C:/...green1.html");
QUrl url("http://www.youtube.com/watch?v=fwbH8dgASRc");
ui->webView->load(url);// QString ht = "<img >";
// QString ht = "YouTube embedded code";
// ui->webView->setHtml(ht);connect(ui->webView,SIGNAL(loadFinished(bool)),this,SLOT(loadDone(bool)));
}
MainWindow::~MainWindow()
{
delete ui;
}void MainWindow::loadDone(bool ok)
{
QTimer::singleShot(1000,this,SLOT(loadfin()));
}// end load donevoid MainWindow::loadfin()
{
QSize sz(ui->webView->page()->mainFrame()->contentsSize());
ui->webView->setFixedSize(sz);
ui->webView->page()->setViewportSize(sz);QImage image(sz,QImage::Format_ARGB32); QPainter painter(&image); ui->webView->page()->mainFrame()->render(&painter); image.save("webRendered.png"); ui->label->setPixmap(QPixmap::fromImage(image));
}// end loadfin
@The file paths have been abbreviated. Several test cases are hard coded and I add and delete comment lines as necessary to run the tests. The results puzzle me. The code correctly creates an image from from either a url or when the HTML is loaded using the webpage setHtml method but if the HTML contains a Flash video then things get puzzling.
The method fails completely if the reference to the flash video is generated from a local file or using the setHtml method (e.g., using the YouTube embedded HTML). Note: QWebView correctly displays the video in all cases. It is the render to the QImage that fails. However if the video is referenced directly over the web then the render sometimes works and sometimes doesn't.
Thinking that it may be a synchronization problem I added an 1 second delay using the loadDone slot but that doesn't make a difference. The various test cases I ran generated these conclusions (I actually ran more test cases than indicated in the above).
These results generate the following questions:
- Can anyone explain the above behavior?
- Does anyone have any suggestions as to how I can capture a Flash video frame in a QImage using Qt 4.8? I have searched the internet but have found no useful suggestions.
-
I think the url for local file, you have to put file:/// as prefix, I guess.
-
Making some progress. The problem appears to be a combination of a sync problem and render problem. A more efficient way to get a QPixmap is to use either the static function QPixmap::grabWidget(QWidget*) or the static function QPixmap::grabWindow(WId).
The grabWidget function has all the same problems as the QPage method outlined in the original post. The grabWindow function grabs screen pixels which isn't ideal but somewhat workable provided that the YouTube video is visible. Therein lies another problem.
In order to use the grabWindow method one has to wait until the YouTube player has displayed the video which takes some time. On my system it takes as long as 10 seconds provided that the internet connect is viable. Therefore one would like to have a signal when the display is visible.
At first glance it appears that the YouTube player api could provide this information. Using the api player reference ("https://developers.google.com/youtube/js_api_reference":https://developers.google.com/youtube/js_api_reference) it appeared to be straightforward by using the QtWebKit bridge. The YouTube player has a user defined JavaScript callback function onYouTubePlayerReady(playerid) which can be used to connect to C++ function that can then use the grabWindow method to save an image.
The problem is that the YouTube player doesn't call the callback function when I'm using QWebView to display the YouTube video. At this point I'm stumped. I don't know if the reason the player doesn't call the callback function is because of a security problem or something stupid that I've done.
For reference here is the html I'm currently using (just the latest in many variations I've tried):
@<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0//EN" "http://www.w3.org/TR/REC-html40/strict.dtd">
<html xml:lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>test</title>
</head>
<body >
<object id="objid" width="420" height="315">
<param id="paramid" name="movie" value="http://www.youtube.com/v/fwbH8dgASRc?enablejsapi=1&playapiid=player1&version=3&hl=en_US&rel=0"></param>
<param name="allowFullScreen" value="true"></param>
<param name="allowscriptaccess" value="always"></param>
<embed id="embid" src="http://www.youtube.com/v/fwbH8dgASRc?enablejsapi=1&playapiid=player2&version=3&hl=en_US&rel=0"
type="application/x-shockwave-flash" width="420"
height="315" allowscriptaccess="always"
allowfullscreen="true"></embed>
</object>
</body>
</html>@(modified from the suggested code on the YouTube site to include playerid and enablejsapi).
So the original post has morphed into the following two questions:
How can one detect that the YouTube player has cued the video for play since neither of the JavaScript methods onYouTubePlaterReader or onStateChange seem to work with QWebView?
Is it possible to somehow grab the video display without making the QWebView object visible?