QWebView check on valid load

bagipro

No
@if(!timer.isActive())
{
timer.stop();
view.stop();
}@
It is check on timeout, if request takes longer than SOCKS_TM ms, then i do abort of an operation (i pasted not full code)

ThatDude

I see you've connected view's loadFinished signal...
well you better connect view.page()->mainFrame() loadFinished
because if a view which is a webpage contains more frames/iframes you could possible receive loadFinished from each one of the frames
I guess you catch only the first loadFinished.

There is no way to know when page finished loading...
the best approach perhaps is to wait for let's say 3-5 seconds without data transfer and its not 100% correct. loadFinished actually tells us that HTML i.e. the code of that page (frame) just loaded nothing more. After the HTML source code of the page loaded then browser (or QWebView in our case) starts interpreting and running that code so... in a few (milli)seconds without transfer some JavaScript could start loading something, or some json or whatever so if a page has javascript that constantly loading/refreshing data (e.g. stock exchange data) you will receive loadFinished only once (per i/frame) yet JavaScript or json could constantly refresh it in the background - so when should we consider such a page fully loaded?!

bagipro

ThatDude, likely! But i edited the code, but nothing has changed :( Often the page doesn't load until the end.

@QWebFrame* frame = view.page()->mainFrame();@

@QObject::connect(frame,SIGNAL(urlChanged(QUrl)),&loopUrlChanged,SLOT(quit()));
QObject::connect(frame,SIGNAL(loadFinished(bool)),&loopLoad,SLOT(quit()));
QObject::connect(&timer,SIGNAL(timeout()),&loopUrlChanged,SLOT(quit()));
QObject::connect(&timer,SIGNAL(timeout()),&loopLoad,SLOT(quit()));@

bagipro

I'm sure that signal it is general trouble, something work wrong .
For example, @SIGNAL(loadFinished(bool))@ triggered, but url is NOT CHANGED

I use evaluateJavaScript method
@frame->evaluateJavaScript(QString("document.getElementById("login_email").value = "%1"").arg(email));
frame->evaluateJavaScript(QString("document.getElementById("login_password").value = "%1"").arg(pass));
frame->evaluateJavaScript("document.login_form.submit()");@

And after
@timer.start(SOCKS_TM);
loopUrlChanged.exec();//отправка формы
if(!timer.isActive())
{
timer.stop();
view.stop();
break;
}

        timer.start(SOCKS_TM);
        loopLoad.exec&#40;&#41;;
        if(!timer.isActive(&#41;&#41;
        {
            timer.stop(&#41;;
            view.stop(&#41;;
            break;
        }@

By the end, the page is the same

What i doing wrong?

ThatDude

It will be eaiser to debug if you use QWebElement instead
@ void Form::submitInfo()
{
QWebFrame *frame = ui->webView->page()->mainFrame();

    QWebElement firstName = frame->findFirstElement("#firstname");
 
    firstName.setAttribute("value","some text");
 
}

@
OR
@ QWebFrame *frame = ui->webView->page()->mainFrame();

    QWebElement firstName = frame->findFirstElement("#firstname");
    firstName.evaluateJavaScript("this.value='some text' ").

@
You could even do this
@elem.setFocus()
elem.evaluateJavaScript("this.click()")
@

bagipro

Yes, but trouble is not in evaluateJavaScript. I would like to say that it sometimes does not work, not all time. And the main problem in the wrong load. It does not load to the end

ThatDude

Well,
perhaps you should wait a bit longer... if you use QWebElement you could wait until that element shows on page.

Are you sure that data really hits your browser - you could use some tools to check that - Fiddler (web debugging proxy) or Wireshark (network protocol analyzer)

bagipro

Ok, i will check (fiddler doesn't fix it because i use proxy, i need to use wireshark or windows network monitor).

But see, there is all correctly?
@QObject::connect(frame,SIGNAL(urlChanged(QUrl)),&loopUrlChanged,SLOT(quit()));
QObject::connect(frame,SIGNAL(loadFinished(bool)),&loopLoad,SLOT(quit()));
QObject::connect(&timer,SIGNAL(timeout()),&loopUrlChanged,SLOT(quit()));
QObject::connect(&timer,SIGNAL(timeout()),&loopLoad,SLOT(quit()));@

bagipro

[quote author="bagipro" date="1374304254"]Ok, i will check (fiddler doesn't fix it because i use proxy, i need to use wireshark or windows network monitor).[/quote]

I remembered! My site using ssl (https), i can't fix it :(

ArcNexus

had to be server :p

some times yes, other times no..... mmmm that sounds at server problems....

bagipro

[quote author="ArcNexus" date="1374324206"] had to be server :p
some times yes, other times no..... mmmm that sounds at server problems....
[/quote]
Heh, I checked yandex.ru (normal size from 160 to 175 KB) using different proxy for each load and we see:
!http://clip2net.com/clip/m0/1374349499-clip-30kb.png(1)!
!http://clip2net.com/clip/m0/1374349970-clip-51kb.png(1)!

ThatDude

As I said...

you do not wait long enough
there is no way of knowing when the page is fully loaded
HTTPS - Webkit has some troubles with https esp. on windows. I never had any trouble with that on linux
Does you site use self-signed certificate? In that case you might want to add to to the list of know i.e. trusted authorities

Can you show us same yandex.ru requests/responses after you change your timer to wait e.g. 3 min per each request i.e. do not rely on loadFInished ...
Even ui->webView->page()->mainFrame() could fire multiple loadFinished signal if it has (static or dynamically created) internal frames or iframes
so sitting and waiting only for the very first loadFinished won't help you much
you might want to investigate different approaches using a timer:

wait for say 3 sec. without any other loadFinished for mainFrame
wait for say 3 sec. without any more data coming for that page/view
QWebPage::totalBytes
QWebPage::bytesReceived () const

bagipro

ThatDude,
Something like that
@void MainWindow::waitLoad(QWebPage* page)
{
qint64 bytes1 = page->bytesReceived(),bytes2;
QTimer timer;
QEventLoop loop;
QObject::connect(&timer,SIGNAL(timeout(QPrivateSignal)),&loop,SLOT(quit()));

do
{
    bytes2 = bytes1;
    timer.start(2000);
    loop.exec&#40;&#41;;
    bytes1 = page->bytesReceived();
}
while(bytes1 != bytes2);

}@

ThatDude

Yep!
That's like a double edged sword though - some pages just constantly load something in the background that's why I tend to prefer waiting for certain amount of time after last loadFinished (each new loadFinished just resets the timer)... this still suffers from things like
@<META HTTP-EQUIV="Refresh" CONTENT="1">@

in such cases a page timeout must be enforced

P.S. Monitor both loadStarted and loadFinished on QWebView i.e. for all frames (it creates during its lifetime) and increment and decrement a counter, when counter reaches 0 wait a few seconds then consider page loading has finished otherwise reset counter(s).
HTH

bagipro

Whats wrong?
@void MainWindow::waitLoad(QWebPage* page)
{
qint64 bytes=0;
QTimer timer;
QEventLoop loop;
QObject::connect(&timer,SIGNAL(timeout()),&loop,SLOT(quit()));

do
{
    qDebug() << bytes << " " << page->totalBytes();
    bytes  = page->totalBytes();
    timer.start(2000);
    loop.exec&#40;&#41;;
}
while( bytes != page->bytesReceived() );

}@

@0 502701
0 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238
50238 50238 @

ThatDude

You compare two different things
page->totalBytes()
AND
page->bytesReceived()
if you are downloading many pages simultaneously - you better keep count of these per-page

bagipro

Oh yeah, it's my fault ;)
But i checked and.. it did not succeed, i have many files like file1_start.html (after loopLoad and before that method, size of the file is 1 kb, 5% from all files) and file1_end.html (after loopLoad and after that method, size of the file is 1 kb) .

Many files have start size 24.1 kb and end size 25kb, but i'm not waiting for that result ;(

bagipro

!http://clip2net.com/clip/m0/1375083759-clip-48kb.png(bug)!

All time i had a trouble.
Idk, but after some hours of work my program crashes

ThatDude

You are lucky - mine crashed every few minutes with Qt5.x
You could try using Qt 4.8.5 instead
Based on my personal observations WebKit in Qt5 is quite buggy and crashes in 'random' places
I've also hit that bug at WTF::randomNumber but with an earlier version pre 5.1 ...and until today I thought they've fixed it in 5.1

The only QWebKit project I've ported to 5.1 I had to write a standalone app that restarts process when it crashes using QProcess etc.

bagipro

That's awful ;(
I planning to write this program on C# using standart liblary. Without dlls, bugs and random exceptions :3