Qt Forum

    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Search
    • Unsolved

    Update: Forum Guidelines & Code of Conduct

    Read data from website

    Qt WebKit
    2
    12
    3183
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • appollosputnik
      appollosputnik Banned last edited by

      In my website there is a table. I want to read the data from the 1st row 1st column of the table. Please tell me how can I read ? I have my QWebView. Please help. Thanks

      [code]
      connect(m_WebView, SIGNAL(loadFinished(bool)), this, SLOT(LoadFinishedCB(bool)));

      m_WebView->setUrl(QUrl("http://www.mywebsite.com"));
      

      [/code]

      In LoadFinishedCB I want to read the data. Please help.

      1 Reply Last reply Reply Quote 0
      • K
        KA51O last edited by

        something like this maybe?
        @
        QWebElementCollection AllFirstColumnEntries =m_WebView->page()->mainFrame()->findAllElements("table.TheTableYoureLookingFor td.FirstColumn"); // this returns all cells in the first column
        @
        this works for a table implemented like this:
        @
        <table class="TheTableYoureLookingFor" >
        <tr>
        <td class="FirstColumn"><p></p></td>
        <td class="SecColumn"><p></p></td>
        </tr>
        <tr>
        <td class="FirstColumn"><p></p></td>
        <td class="SecColumn"><p></p></td>
        </tr>
        <tr>
        <td class="FirstColumn"><p></p></td>
        <td class="SecColumn"><p></p></td>
        </tr>
        </table>
        @

        1 Reply Last reply Reply Quote 0
        • appollosputnik
          appollosputnik Banned last edited by

          Please check the code snippet below. I am trying to click on link in the page twice and then a table is loaded i want to read one data from the table from 1st row and 2nd column. Please help.
          [code]
          void MainWindow::LoadFinishedCB(bool ok)
          {
          QFile file("out.txt");
          if (!file.open(QIODevice::WriteOnly))
          return;
          QTextStream out(&file);
          button_click = m_WebView->page()->mainFrame()->documentElement().findAll("a");
          qDebug() << button_click.count() << "\n";

          button_click.at(5).evaluateJavaScript(
                      "var evObj = document.createEvent('MouseEvents');evObj.initEvent( 'click', true, true );this.dispatchEvent(evObj);"
                      );
          
          
          button_click.at(48).evaluateJavaScript(
                      "var evObj = document.createEvent('MouseEvents');evObj.initEvent( 'click', true, true );this.dispatchEvent(evObj);"
                      );
          

          //The above piece loads a new link in the same page.
          //At this stage how can I make the program to wait till the page is //refreshed ???

          out << m_WebView->page()->currentFrame()->toHtml() << "\n";
          
          QWebElementCollection tables = m_WebView->page()->currentFrame()->findAllElements("table [id=layer_showhost]");
          qDebug() << tables.count() << "\n";
          
          QWebElementCollection trs = tables.first().document().findAll("tr");
          qDebug() << trs.count() << "\n";
          
          
          for(int m=0; m<trs.count(); m++)
          {
            QWebElementCollection tds = trs.at(m).document().findAll("td");
            qDebug() << tds.count() << "\n";
          
            for(int n=0; n<tds.count(); n++)
            {
                QWebElementCollection spans = tds.at(n).document().findAll("span");
                qDebug() << spans.count() << "\n";
          
                for(int z=0; z<spans.count(); z++)
                {
                    out << spans.at(z).document().toPlainText() << " ";
                }
            }
          
          }
          

          }
          [/code]

          1 Reply Last reply Reply Quote 0
          • K
            KA51O last edited by

            The code you posted just prints out the number of tables found, the number of table rows found in the first table, the number of table cells for each row in the first table and the text found enclosed by a span tag in each of these cells.

            What exactly do you want from me now? Am I supposed to modify that piece of code you may have pulled somewhere of the internet so that it does what you specified above?

            Why don't you try to do that, its really not that complicated. If you have problems I will gladly help you, after you posted the modified code or the problems you encountered.

            Edit:
            Just read the comment in your code
            [quote]
            The above piece loads a new link in the same page.
            At this stage how can I make the program to wait till the page is refreshed ???
            [/quote]
            Just connect a custom slot (in which you could do all the stuff you do after the comment I quoted above) to the webviews signal loadFinished(bool ok).

            For example:
            @
            button_click.at(48).evaluateJavaScript(
            "var evObj = document.createEvent('MouseEvents');evObj.initEvent( 'click', true, true );this.dispatchEvent(evObj);"
            );
            //The above piece loads a new link in the same page.
            //At this stage how can I make the program to wait till the page is //refreshed ???

            //if you have another slot already connected to the signal loadFinished disconnect this slot first
            // you can reconnect it later
            connect(m_WebView, SIGNAL(loadFinished(bool), this, SLOT(doTheTableParsing()));
            }

            void YourClass::doTheTableParsing()
            {
            // here you can reconnect to the loadFinished() signal but first disconnect it from the new custom slot
            disconnect(m_WebView, SIGNAL(loadFinished(bool), this, SLOT(doTheTableParsing()));
            out << m_WebView->page()->currentFrame()->toHtml() << "\n";

            QWebElementCollection tables = m_WebView->page()->currentFrame()->findAllElements("table [id=layer_showhost]");
            qDebug() << tables.count() << "\n";
            
            QWebElementCollection trs = tables.first().document().findAll("tr");
            qDebug() << trs.count() << "\n";
            

            if(trs.size() > 0)
            {
            QWebElementCollection tdsInFirstTr = trs.at(0).document().findAll("td");
            if(tdsInFirstTr.size() > 0)
            {
            QWebElementCollection spanInSecondTDInFirstTR = tdsInFirstTr.at(1).document().findAll("span");
            if(spanInSecondTDInFirstTR .size() > 0)
            {
            qDebug() << spanInSecondTDInFirstTR .at(0).toPlainText() << "\n";
            }
            }
            }
            }
            @

            1 Reply Last reply Reply Quote 0
            • appollosputnik
              appollosputnik Banned last edited by

              loadFinished() signal is not being emitted second time. What can be done?

              1 Reply Last reply Reply Quote 0
              • K
                KA51O last edited by

                hmm. You can try some of the signals provided by QWebFrame (loadFinished(), pageChanged(),...) or QWebPage (contentsChanged(), loadFinished(), ..), if none of them work for your use case the only option I can think of would be a QTimer::singleShot(..).

                1 Reply Last reply Reply Quote 0
                • appollosputnik
                  appollosputnik Banned last edited by

                  ok thanks a lot friend let me try I will post again thanks

                  1 Reply Last reply Reply Quote 0
                  • appollosputnik
                    appollosputnik Banned last edited by

                    Can u give me your skype id I want to discuss with you. I am facing severe problem. Totally stuck. Thanks Sujan

                    1 Reply Last reply Reply Quote 0
                    • appollosputnik
                      appollosputnik Banned last edited by

                      I am writing the page as html. where the span values are question mark. no values are displayed.

                      [code]
                      out << m_WebView->page()->currentFrame()->toHtml() << "\n";
                      [/code]

                      var msg_proce = "<span style=color:red>??????...</span>";
                      var msg_success = "<span style=color:red>????</span>";
                      <span style=font-size:15px;color:red>??????????,<a >????</a></span>

                      HOW CAN I GET THE SPAN VALUES. PLEASE HELP.

                      1 Reply Last reply Reply Quote 0
                      • appollosputnik
                        appollosputnik Banned last edited by

                        loadFinished() signal is working friend for currentFrame().. Thanks a lot.

                        Please see the above post I am not able to get the span values. Please reply. Thanks Sujan

                        1 Reply Last reply Reply Quote 0
                        • K
                          KA51O last edited by

                          Happy to hear that you got the signal problem solved. About the span issue, I guess we should continue the discussion in "your new post.":http://qt-project.org/forums/viewthread/29334/

                          1 Reply Last reply Reply Quote 0
                          • appollosputnik
                            appollosputnik Banned last edited by

                            My problem is solved I am using a QTimer with singleshot for updating the site with click. Then it comes ok I am able to fetch data using QWebElement as per your suggestions. Thanks a lot friend.

                            1 Reply Last reply Reply Quote 0
                            • First post
                              Last post