QWebView check on valid load



  • Hello all, I have a trouble. In my app I use proxy, sometimes works completion signal to the load, but the page isn't fully loaded. How i can check it? Try to find "</html>"?



  • Hi bagipro:

    if you have a Qwebview widget in your .ui, you can do it, or similar:

    @ if(ui->qwebview->findText("</html>"))
    .........
    else
    ..............
    }@

    good luck.



  • Hehe, i do it firstly)
    QWebView automatically closes tags, but i did view.page()->mainFrame()->toHtml().contains("</html>");



  • ok, good input :p



  • Question is not solved ;p



  • I have this code in my project and it works fine. It isn't exactly the same case, but....

               @ connect(ui->webLogoLaboratorio,SIGNAL(loadFinished(bool)),this, SLOT(cargaFinalizadaLogo(bool)));
    
                    QUrl  uUrl;
                    QString cUrl = "http://xxxxxx.com/documentos/image/fotos/medicamento/"+e.text().trimmed()+"_1.jpg";
                    uUrl.setUrl(cUrl);
                    ui->webLogoLaboratorio->load(uUrl);
    

    void FrmInformacionFarmaco::cargaFinalizadaLogo(bool estado)
    {

    if(ui->webLogoLaboratorio->findText("Not Found"))
        ui->webLogoLaboratorio->setVisible(false);
    else
        ui->webLogoLaboratorio->setVisible(true);
    

    }@

    this waits for completly loaded webpage. And when is completly loaded it run CargaFinalizadaLogo metod, and test if xxxx text is in qWebview. In my case if image is not in server, it return 404- not found, If this text is present I hide control, otherwise it shows image

    I think is similar at your problem. good luck



  • Yes, but i have same code.
    For example, full page:
    @<html>
    <title>Hello world<title>
    <meta>some data</meta>
    [removed]script[removed]
    <table>
    <tr>
    <td>one</td>
    <td>two</td>
    <td>three</td>
    </tr>
    </table>
    </html>@

    But sometimes i get (all times differently)
    @<html>
    <title>Hello world<title>
    <meta>some data</meta>
    [removed]script[removed]
    </html>@



  • And if you open url in web browser like firefox it works fine all times?

    Make sure your php or asp code for ensure problem is not in your server code.

    Your example gives impresion that for some reason your server not have data to return.



  • Oh no, in browser all works correctly, it is not server's problem



  • Maybe, if you copy here your code, we will can find where is the problem



  • Hi,

    You can check whether all data was loaded using @void QNetworkReply::downloadProgress ( qint64 bytesReceived, qint64 bytesTotal ) [signal]@ signal.

    Hope it helps.



  • ArcNexus, I haven't server code, but i sure that is application bug.

    This is app code
    @QObject::connect(&view,SIGNAL(loadFinished(bool)),&loopLoad,SLOT(quit()));
    QObject::connect(&timer,SIGNAL(timeout()),&loopLoad,SLOT(quit()));@

    @view.load(QUrl("https://www.site.com/"));
    timer.start(SOCKS_TM);
    loopLoad.exec();
    if(!timer.isActive())
    {
    timer.stop();
    view.stop();
    }@

    MaximAlien, thanks, i will check



  • Can you comment timer lines? and test again, maybe the timer cause an exit before the html code has completly downloaded from server?. I think it is a possible cause.



  • No
    @if(!timer.isActive())
    {
    timer.stop();
    view.stop();
    }@
    It is check on timeout, if request takes longer than SOCKS_TM ms, then i do abort of an operation (i pasted not full code)



  • I see you've connected view's loadFinished signal...
    well you better connect view.page()->mainFrame() loadFinished
    because if a view which is a webpage contains more frames/iframes you could possible receive loadFinished from each one of the frames
    I guess you catch only the first loadFinished.

    There is no way to know when page finished loading...
    the best approach perhaps is to wait for let's say 3-5 seconds without data transfer and its not 100% correct. loadFinished actually tells us that HTML i.e. the code of that page (frame) just loaded nothing more. After the HTML source code of the page loaded then browser (or QWebView in our case) starts interpreting and running that code so... in a few (milli)seconds without transfer some JavaScript could start loading something, or some json or whatever so if a page has javascript that constantly loading/refreshing data (e.g. stock exchange data) you will receive loadFinished only once (per i/frame) yet JavaScript or json could constantly refresh it in the background - so when should we consider such a page fully loaded?!



  • ThatDude, likely! But i edited the code, but nothing has changed :( Often the page doesn't load until the end.

    @QWebFrame* frame = view.page()->mainFrame();@

    @QObject::connect(frame,SIGNAL(urlChanged(QUrl)),&loopUrlChanged,SLOT(quit()));
    QObject::connect(frame,SIGNAL(loadFinished(bool)),&loopLoad,SLOT(quit()));
    QObject::connect(&timer,SIGNAL(timeout()),&loopUrlChanged,SLOT(quit()));
    QObject::connect(&timer,SIGNAL(timeout()),&loopLoad,SLOT(quit()));@



  • I'm sure that signal it is general trouble, something work wrong .
    For example, @SIGNAL(loadFinished(bool))@ triggered, but url is NOT CHANGED

    I use evaluateJavaScript method
    @frame->evaluateJavaScript(QString("document.getElementById("login_email").value = "%1"").arg(email));
    frame->evaluateJavaScript(QString("document.getElementById("login_password").value = "%1"").arg(pass));
    frame->evaluateJavaScript("document.login_form.submit()");@

    And after
    @timer.start(SOCKS_TM);
    loopUrlChanged.exec();//отправка формы
    if(!timer.isActive())
    {
    timer.stop();
    view.stop();
    break;
    }

            timer.start(SOCKS_TM);
            loopLoad.exec&#40;&#41;;
            if(!timer.isActive(&#41;&#41;
            {
                timer.stop(&#41;;
                view.stop(&#41;;
                break;
            }@
    

    By the end, the page is the same

    What i doing wrong?



  • It will be eaiser to debug if you use QWebElement instead
    @ void Form::submitInfo()
    {
    QWebFrame *frame = ui->webView->page()->mainFrame();

        QWebElement firstName = frame->findFirstElement("#firstname");
     
        firstName.setAttribute("value","some text");
     
    }
    

    @
    OR
    @ QWebFrame *frame = ui->webView->page()->mainFrame();

        QWebElement firstName = frame->findFirstElement("#firstname");
        firstName.evaluateJavaScript("this.value='some text' ").
    

    @
    You could even do this
    @elem.setFocus()
    elem.evaluateJavaScript("this.click()")
    @



  • Yes, but trouble is not in evaluateJavaScript. I would like to say that it sometimes does not work, not all time. And the main problem in the wrong load. It does not load to the end



  • Well,
    perhaps you should wait a bit longer... if you use QWebElement you could wait until that element shows on page.

    Are you sure that data really hits your browser - you could use some tools to check that - Fiddler (web debugging proxy) or Wireshark (network protocol analyzer)



  • Ok, i will check (fiddler doesn't fix it because i use proxy, i need to use wireshark or windows network monitor).

    But see, there is all correctly?
    @QObject::connect(frame,SIGNAL(urlChanged(QUrl)),&loopUrlChanged,SLOT(quit()));
    QObject::connect(frame,SIGNAL(loadFinished(bool)),&loopLoad,SLOT(quit()));
    QObject::connect(&timer,SIGNAL(timeout()),&loopUrlChanged,SLOT(quit()));
    QObject::connect(&timer,SIGNAL(timeout()),&loopLoad,SLOT(quit()));@



  • [quote author="bagipro" date="1374304254"]Ok, i will check (fiddler doesn't fix it because i use proxy, i need to use wireshark or windows network monitor).[/quote]

    I remembered! My site using ssl (https), i can't fix it :(



  • had to be server :p

    some times yes, other times no..... mmmm that sounds at server problems....



  • [quote author="ArcNexus" date="1374324206"] had to be server :p
    some times yes, other times no..... mmmm that sounds at server problems....
    [/quote]
    Heh, I checked yandex.ru (normal size from 160 to 175 KB) using different proxy for each load and we see:
    !http://clip2net.com/clip/m0/1374349499-clip-30kb.png(1)!
    !http://clip2net.com/clip/m0/1374349970-clip-51kb.png(1)!



  • As I said...

    1. you do not wait long enough
    2. there is no way of knowing when the page is fully loaded
    3. HTTPS - Webkit has some troubles with https esp. on windows. I never had any trouble with that on linux
      Does you site use self-signed certificate? In that case you might want to add to to the list of know i.e. trusted authorities

    Can you show us same yandex.ru requests/responses after you change your timer to wait e.g. 3 min per each request i.e. do not rely on loadFInished ...
    Even ui->webView->page()->mainFrame() could fire multiple loadFinished signal if it has (static or dynamically created) internal frames or iframes
    so sitting and waiting only for the very first loadFinished won't help you much
    you might want to investigate different approaches using a timer:

    1. wait for say 3 sec. without any other loadFinished for mainFrame
    2. wait for say 3 sec. without any more data coming for that page/view
      QWebPage::totalBytes
      QWebPage::bytesReceived () const


  • ThatDude,
    Something like that
    @void MainWindow::waitLoad(QWebPage* page)
    {
    qint64 bytes1 = page->bytesReceived(),bytes2;
    QTimer timer;
    QEventLoop loop;
    QObject::connect(&timer,SIGNAL(timeout(QPrivateSignal)),&loop,SLOT(quit()));

    do
    {
        bytes2 = bytes1;
        timer.start(2000);
        loop.exec&#40;&#41;;
        bytes1 = page->bytesReceived();
    }
    while(bytes1 != bytes2);
    

    }@



  • Yep!
    That's like a double edged sword though - some pages just constantly load something in the background that's why I tend to prefer waiting for certain amount of time after last loadFinished (each new loadFinished just resets the timer)... this still suffers from things like
    @<META HTTP-EQUIV="Refresh" CONTENT="1">@

    in such cases a page timeout must be enforced

    P.S. Monitor both loadStarted and loadFinished on QWebView i.e. for all frames (it creates during its lifetime) and increment and decrement a counter, when counter reaches 0 wait a few seconds then consider page loading has finished otherwise reset counter(s).
    HTH



  • Whats wrong?
    @void MainWindow::waitLoad(QWebPage* page)
    {
    qint64 bytes=0;
    QTimer timer;
    QEventLoop loop;
    QObject::connect(&timer,SIGNAL(timeout()),&loop,SLOT(quit()));

    do
    {
        qDebug() << bytes << " " << page->totalBytes();
        bytes  = page->totalBytes();
        timer.start(2000);
        loop.exec&#40;&#41;;
    }
    while( bytes != page->bytesReceived() );
    

    }@

    @0 502701
    0 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238
    50238 50238 @



    1. You compare two different things
      page->totalBytes()
      AND
      page->bytesReceived()

    2. if you are downloading many pages simultaneously - you better keep count of these per-page



  • Oh yeah, it's my fault ;)
    But i checked and.. it did not succeed, i have many files like file1_start.html (after loopLoad and before that method, size of the file is 1 kb, 5% from all files) and file1_end.html (after loopLoad and after that method, size of the file is 1 kb) .

    Many files have start size 24.1 kb and end size 25kb, but i'm not waiting for that result ;(



  • !http://clip2net.com/clip/m0/1375083759-clip-48kb.png(bug)!

    All time i had a trouble.
    Idk, but after some hours of work my program crashes



  • You are lucky - mine crashed every few minutes with Qt5.x
    You could try using Qt 4.8.5 instead
    Based on my personal observations WebKit in Qt5 is quite buggy and crashes in 'random' places
    I've also hit that bug at WTF::randomNumber but with an earlier version pre 5.1 ...and until today I thought they've fixed it in 5.1

    The only QWebKit project I've ported to 5.1 I had to write a standalone app that restarts process when it crashes using QProcess etc.



  • That's awful ;(
    I planning to write this program on C# using standart liblary. Without dlls, bugs and random exceptions :3



  • But i need to fix the bug
    @Exception at 0x13fa54b, code: 0xc0000005: write access violation at: 0x1, flags=0x0 (first chance) in Qt5WebKit!WTF::randomNumber
    First chance exceptions are reported before any exception handling.
    This exception may be expected and handled.
    eax=00000000 ebx=0001f000 ecx=00000000 edx=0012a504 esi=00000000 edi=0242d7d8
    eip=013fa54b esp=0012a584 ebp=00010000 iopl=0 nv up ei pl zr na pe nc
    cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00210246
    Qt5WebKit!WTF::randomNumber+0x6b:
    Missing image name, possible paged-out or corrupt data.
    Missing image name, possible paged-out or corrupt data.
    Missing image name, possible paged-out or corrupt data.
    013fa54b 8935efbeadbb mov dword ptr ds:[0BBADBEEFh],esi ds:0023:bbadbeef=????????

    *** ERROR: Symbol file could not be found. Defaulted to export symbols for C:\Qt\Qt5.1.0\5.1.0\msvc2010\bin\Qt5Core.dll -
    s
    sException at 0x13fa54b, code: 0xc0000005: write access violation at: 0x1, flags=0x0 (first chance)
    dNOTE: INFERIOR SPONTANEOUS STOP
    sОстановлено.
    dState changed from InferiorRunOk(11) to InferiorStopOk(14) [master]
    Resolving symbol: Qt5WebKit!WTF::randomNumber...
    <x Qt5WebKit!WTF::randomNumber
    <l-t
    <l-s

    Obtained 0x13fa4e0 for Qt5WebKit!WTF::randomNumber (#1)
    <u 0x13fa4e0 0x13fa650
    Source options are 4:
    4/s - List source code at prompt
    Source options are 0:
    None

    dNOTE: INFERIOR RUN REQUESTED
    sПотребован запуск...
    dState changed from InferiorStopOk(14) to InferiorRunRequested(10) [master]
    <g

    dNOTE: INFERIOR RUN OK
    sВыполняется.
    dState changed from InferiorRunRequested(10) to InferiorRunOk(11) [master]
    (83c.f08): Access violation - code c0000005 (!!! second chance !!!)
    s
    sException at 0x13fa54b, code: 0xc0000005: write access violation at: 0x1, flags=0x0 in Qt5WebKit!WTF::randomNumber
    eax=00000000 ebx=0001f000 ecx=00000000 edx=0012a504 esi=00000000 edi=0242d7d8
    eip=013fa54b esp=0012a584 ebp=00010000 iopl=0 nv up ei pl zr na pe nc
    cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00200246
    Qt5WebKit!WTF::randomNumber+0x6b:
    Missing image name, possible paged-out or corrupt data.
    Missing image name, possible paged-out or corrupt data.
    Missing image name, possible paged-out or corrupt data.
    013fa54b 8935efbeadbb mov dword ptr ds:[0BBADBEEFh],esi ds:0023:bbadbeef=????????
    s
    sException at 0x13fa54b, code: 0xc0000005: write access violation at: 0x1, flags=0x0
    dNOTE: INFERIOR SPONTANEOUS STOP
    sОстановлено.
    dState changed from InferiorRunOk(11) to InferiorStopOk(14) [master]
    dUsing cached disassembly for 0x13fa54b (0x13fa4e0-0x13fa64f) in 'WTF::randomNumber'/ ''
    @

    !http://clip2net.com/clip/m0/1375639721-clip-33kb.png(1)!


Log in to reply
 

Looks like your connection to Qt Forum was lost, please wait while we try to reconnect.