[SOLVED] QtWebEngine headless?



  • I'm trying to use QWebEnginePage to grab some online content. I don't care about any GUI/visualisations just to extract pure data from HTML
    alas my console app crashes in QtWebEngine::initialize() on QOpenGLContext::create()

    AFAIK it is completely separate from QT and despite its module name 'webenginewidgets' it should not require any Qt Widgets the same way as WebKit does, right?

    Windows 7, 64 bit
    Qt 5.4 64 bit
    MSVS2013 64 bit

    QT += core webenginewidgets
    QT -= gui
    TARGET = contest
    CONFIG += console
    CONFIG -= app_bundle
    TEMPLATE = app
    SOURCES += main.cpp

    Everything works fine until I add
    QWebEnginePage page;
    page.load(QUrl("http://www.google.com"));

    and the it starts crashing on the very first line
    QCoreApplication a(argc, argv);

    with
    c:\work\build\qt5_workdir\w\s\qtbase\src\gui\kernel\qopenglcontext.cpp:596: error: Exception at 0x7fedf0d3142, code: 0xc0000005: read access violation at: 0x0, flags=0x0

    QWebEngine examples work fine - both QML and C++ based but they are GUI apps
    Where can I find any documentation about QWebEngine requirements - GUI App, OpenGL version
    HTH


  • Lifetime Qt Champion

    Hi,

    widgets here doesn't mean it needs QWidget (you can see that QtWebEnginePage is a QObject) however it probably expects either a QGuiApplication or a QApplication to work.



  • If you don't show() any QWidget, you can have a QApplication without any UI.

    Create a QWidget project but remove the
    @
    QWidget w;
    w.show();
    @

    lines from main() and you are good to go.



  • Thanks guys
    I figured it out but instead of starting with QWidget project I started with Console project and switched from QCoreApplication to QApplication.

    Can you point me to the right documentation, help pages etc. explaining this i.e. why it is not working with QCoreApplication?

    and now I'm facing different problem - I can't get page source using page->toHtml() on loadFinished()
    none of the 3 examples (functor, lambda, template) seem to work (of course in console app I don't have textEdit controls so I use QString instead) - all I get is an empty string.

    Again any documentation/examples how to get web page source code into class member variable?


  • Lifetime Qt Champion

    Because QCoreApplication is not meant to be used for GUI application there's QGuiApplication and QApplication for that. It's on the first line of it's "documentation":http://doc.qt.io/qt-5/qcoreapplication.html page

    Are you running an event loop to use signals and slots ?



  • Yes I'm running an event loop.
    and I receive loadFinished() signal
    I tried all 3 proposed methods
    the only difference is that I try to read the HTML in a class member variable



  • Thanks to SGaist's suggestion I've solved it.

    I was unintentionally blocking main event loop at a different place in my code
    although I was calling QApplication::processEvents();
    obviously not quite often.


  • Lifetime Qt Champion

    You're welcome !

    Were you using a busy loop ?



  • Yes I was using a busy loop calling QApplication::processEvents(); once in a while

    To summarize:

    1. Yes you can run QtWebEngine headless/in console but it requires QApplication and won't work with QCoreApplicaion

    2. Much alike WebKit it requires quite frequently access to main event loop i.e. do not block it for extensive periods of time

    3. I'm not sure if toHtml() shows original/loaded Html or 'generated' Html (after some time javascript usually alters original Html) and which Blink engine version is used (it could be rather old) but it doesn't render google's results properly.
      I need to investigate... any pointers, documentation...


Log in to reply
 

Looks like your connection to Qt Forum was lost, please wait while we try to reconnect.