Retrieving information from a Webview seemingly only reliable via pushButton - solved

RudderDuck

I don't get it. I'm trying to retrieve information from a website using a webView. The site uses javascript and Ajax which could be part of the problem. I've implemented a loop that gets quit as soon as the webView's loadFinished signal is received. I then try to read some info from a 'span' element.

The behaviour I don't understand is as follows:

When I load the page I call a function to retrieve some span information (output via qDebug). I then wait 5 seconds and call the function again (output via qDebug). Sometimes (not all the times) both attempts don't retrieve the requested information (output "").

However, when the requested information isn't retrieved I try and call the same function via a pushButton. This never fails to retrieve the requested information.

I don't understand why within MainWindow it sometimes fails but then always succeeds when pushing the button and calling the same function. I would like to retrieve the information without having to push a button outside MainWindow.

Here is my code.

.pro file

@#-------------------------------------------------

Project created by QtCreator 2014-11-17T15:44:22

#-------------------------------------------------

QT += core gui webkitwidgets

greaterThan(QT_MAJOR_VERSION, 4): QT += widgets

TARGET = WorkingExampleTwo
TEMPLATE = app

SOURCES += main.cpp
mainwindow.cpp

HEADERS += mainwindow.h

FORMS += mainwindow.ui@

mainwindow.h

@#ifndef MAINWINDOW_H
#define MAINWINDOW_H

#include <QMainWindow>
#include <QUrl>
#include <QWebFrame>
#include <QWebElement>
#include <QtWebKitWidgets/QWebView>

namespace Ui {
class MainWindow;
}

class MainWindow : public QMainWindow
{
Q_OBJECT

public:
explicit MainWindow(QWidget *parent = 0);
~MainWindow();

private slots:
void on_webView_loadFinished();
void on_pushButton_clicked();
void getInfo();

private:
Ui::MainWindow *ui;
};

#endif // MAINWINDOW_H
@

main.cpp

@#include "mainwindow.h"
#include <QApplication>

int main(int argc, char *argv[])
{
QApplication a(argc, argv);
MainWindow w;
w.show();

return a.exec&#40;&#41;;

}
@

mainwindow.cpp

@#include "mainwindow.h"
#include "ui_mainwindow.h"
#include <QThread>

MainWindow::MainWindow(QWidget *parent) :
QMainWindow(parent),
ui(new Ui::MainWindow)
{
ui->setupUi(this);
// define a loop that is quit as soon as the page has loaded
QEventLoop loop;
// connect the webview signal loadfinished to the loop's quit slot
QObject::connect(ui->webView, SIGNAL(loadFinished(bool)), &loop, SLOT(quit()));
QUrl url("http://www.somesite/somepage.html");

// load the url
ui->webView->load(url);
// show the page
ui->webView->show();
qDebug() << "loading page...";
// execute the loop
loop.exec&#40;&#41;;
qDebug(&#41; << "page loaded";
getInfo();
QThread::msleep(5000);
getInfo();

}

MainWindow::~MainWindow()
{
delete ui;
}
void MainWindow::on_webView_loadFinished(){
// do stuff
}

void MainWindow::on_pushButton_clicked(){
// do stuff
getInfo();
}

void MainWindow::getInfo(){
QWebFrame* frame = ui->webView->page()->currentFrame();
QWebElementCollection spans = frame->findAllElements("span");
foreach(QWebElement span, spans) {
if (span.attribute("class")=="h-init h-time-tick ng-valid ng-binding ng-dirty")
qDebug() << span.toPlainText();
}
}@

mainwindow only contains a pushButton and a webView.