Skip to content
  • Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Groups
  • Search
  • Get Qt Extensions
  • Unsolved
Collapse
Brand Logo
  1. Home
  2. Qt Development
  3. QtWebEngine
  4. A desktop app to retrieve values from a site and to perform actions such as button clicking
Forum Updated to NodeBB v4.3 + New Features

A desktop app to retrieve values from a site and to perform actions such as button clicking

Scheduled Pinned Locked Moved Unsolved QtWebEngine
11 Posts 3 Posters 3.0k Views 1 Watching
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • I Offline
    I Offline
    itsmaxdd
    wrote on last edited by itsmaxdd
    #1

    Hello QtForum,
    i need a suggestion about the right path to take in order to not overcompilate my project.
    My idea is to develop a desktop application with a gui but without a webpage frame, so it means the user is not able to surf the site. The application should provide a graphical interface such as labels and buttons. Labels contain values directly retrieved on the site using GetElementsById for example while buttons allow to simulate what the real site button would normally do.
    Of course i could even inject a js script to handle the logic.

    Is QtWebEngine the right solution?
    The site will require an authentication (user and pw)
    As i said i only need to read 2/3 values and press 2 buttons on the site.

    Regards,

    1 Reply Last reply
    0
    • SGaistS Offline
      SGaistS Offline
      SGaist
      Lifetime Qt Champion
      wrote on last edited by
      #2

      Hi and welcome to devnet,

      Do you really need to show that web page ? Depending on the tools you have at disposition, why not just write a client that does the job to grab data from the corresponding API.

      Interested in AI ? www.idiap.ch
      Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

      I 1 Reply Last reply
      0
      • SGaistS SGaist

        Hi and welcome to devnet,

        Do you really need to show that web page ? Depending on the tools you have at disposition, why not just write a client that does the job to grab data from the corresponding API.

        I Offline
        I Offline
        itsmaxdd
        wrote on last edited by itsmaxdd
        #3

        @SGaist Exactly, i dont really want to show the web page. The point is to have a desktop interface to a site. Which corresponding API are you referring to? Should i use QtNetworkAccessManager to perform GET/POST requests? In this case i wouldnt be able to use the DOM tree, so after issuing a GET request my only solution is to look for words or pattern in the plain text response?

        Or can i use this for example

        QWebPage page;
        page.mainFrame()->setHtml(source); // source is retrieved with GET
        QWebElement parse = page.mainFrame()->documentElement();
        QWebElement result = parse.findFirst("whatever");

        Regards,

        1 Reply Last reply
        0
        • SGaistS Offline
          SGaistS Offline
          SGaist
          Lifetime Qt Champion
          wrote on last edited by
          #4

          I thought your data would come from a REST API. Is it the case, or do you really need to grab a page and parse it ?

          Interested in AI ? www.idiap.ch
          Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

          I 1 Reply Last reply
          0
          • SGaistS SGaist

            I thought your data would come from a REST API. Is it the case, or do you really need to grab a page and parse it ?

            I Offline
            I Offline
            itsmaxdd
            wrote on last edited by
            #5

            @SGaist How do i recognize if it is based on a REST API?
            I thought i would have to parse the entire webpage after a GET request.

            Moreover, say you have a button which loads a function when clicked , this function has a text as input (from an html label) and it encodes it in a particular way (branded MD5 and so on).
            Can i clone the POST request so that i let the site use its own function without implementing it in my c++ GUI?

            1 Reply Last reply
            0
            • SGaistS Offline
              SGaistS Offline
              SGaist
              Lifetime Qt Champion
              wrote on last edited by
              #6

              What page are you trying to access ?

              Interested in AI ? www.idiap.ch
              Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

              1 Reply Last reply
              0
              • I Offline
                I Offline
                itsmaxdd
                wrote on last edited by itsmaxdd
                #7

                Local modem interface :-) Moreover, what if the server has an antiCSRF script? is the API able to handle it as a normal browser?

                1 Reply Last reply
                0
                • SGaistS Offline
                  SGaistS Offline
                  SGaist
                  Lifetime Qt Champion
                  wrote on last edited by
                  #8

                  So the admin page of the modem ?

                  Interested in AI ? www.idiap.ch
                  Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

                  I 1 Reply Last reply
                  0
                  • SGaistS SGaist

                    So the admin page of the modem ?

                    I Offline
                    I Offline
                    itsmaxdd
                    wrote on last edited by itsmaxdd
                    #9

                    @SGaist

                    Re: A desktop app to retrieve values from a site and to perform actions such as button clicking

                    Yes, i was pretty much able to achieve a good point. I've used QNetworkManager with QNetworkReply.
                    Now the doubt i have is about the correct design of a software which implements multiple and nested GET/POST request.

                    Say you have to run a login() function which requires a GET and a POST and then retrieveXYZ() which requires two GETs (and so on, scalable).

                    The way i was thinking to do it was like

                    mainwindow.cpp
                        //code
                        login();
                        retrieveXYZ();
                        //code
                    
                    Mainwindow::login(){ 
                        //code
                        connect(nam, SIGNAL(finished()), this, SLOT(onGetLoginFinished()));
                        nam->get(...);
                    }
                    
                    Mainwindow::onLoginFinished(){
                        //do stuff
                        connect(nam, SIGNAL(finished()), this, SLOT(onPostLoginFinished()));
                        nam->post(...);
                    }    
                    
                    Mainwindow::onPostLoginFinished(){
                        //do stuff
                    }
                    
                    Mainwindow::retrieveXYZ(){
                        //code
                        connect(nam, SIGNAL(finished()), this, SLOT(onGet1RetrieveFinished()));
                        nam->get();
                        //code
                    }
                    
                    Mainwindow::onRetrieveXYZFinished(){
                        //do stuff
                        connect(nam, SIGNAL(finished()), this, SLOT(onGet2RetrieveFinished()));
                        nam->get();
                    }    
                    

                    or should i use something like QSignalMapper ?
                    Which are the most correct/efficient way to do so? i've seen people using sender() cast but i didn't understand the point.

                    Basically i would like to retrieve the particular reply finished() signal rather than the general one (or of the qnam)

                    This method may work but it is not nice and clean to me

                    Is this the best we can get?

                    http://www.johanpaul.com/blog/2011/07/why-qnetworkaccessmanager-should-not-have-the-finishedqnetworkreply-signal/

                    Moving the connect approach to the reply?

                    1 Reply Last reply
                    0
                    • SGaistS Offline
                      SGaistS Offline
                      SGaist
                      Lifetime Qt Champion
                      wrote on last edited by
                      #10

                      We might be overthinking that stuff. What exactly are your users supposed to be doing ?

                      Interested in AI ? www.idiap.ch
                      Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

                      1 Reply Last reply
                      0
                      • S Offline
                        S Offline
                        sendevent
                        wrote on last edited by
                        #11

                        There are two general ways: to use the QtWebKit or the QtWebEngine.
                        There are memory usage and rendering issues with QtWebKit and it is deprecated since Qt 5.5, but its API is synchronous and simplier to use.
                        In case it is possible to use the 5.5 or an older version of Qt with precompiled QWebKit or you are able to build the module for target platform(s) manually (which could be a bit tricky), just try to open target web page in the QWebView (see the demo browser in the examples). If the page is rendered properly, interaction with inputs/buttons is adequate and there are no visible crashes/freezes/memory leaking - you can use the QWebKit :)

                        ...
                            connect( webView->page(), //-- or webView->page()->mainFrame(), depending on the DOM structure
                                SIGNAL(loadFinished(bool), this, SLOT(onWebContentLoaded(bool));
                        ...
                        
                        void ...::onWebContentLoaded(bool ok)
                        {
                            if(!isLogedIn()) //-- parse HTML to find markers or just check a flag actualized in the doLogIn();
                                doLogIn();
                            else
                                extractTheData();
                        }
                        
                        void ...::doLogIn()
                        {
                            if( QWebFrame *frame = webView->page()->mainFrame() )
                            {
                                QWebElement inputLogin = frame->findFirstElement( "#user" ); //-- Chrome->View Source (Ctrl+Shift+I)->Copy->Copy Selector
                                QWebElement inputPass = frame->findFirstElement( "#password" );
                                QWebElement buttonSubmit = frame->findFirstElement( "#login" );
                        
                                if(!inputLogin.isNull() 
                                && !inputPass.isNull()
                                && !buttonSubmit.isNull() )
                                {
                                    inputLogin.setAttribute("value", ui->lineEditLogin->text() ); //-- populate HTML's input via API
                                    inputPass.evaluateJavaScript( QString("this.value='%1'").arg( ui->lineEditPass->text())); //-- or via JS
                                    buttonSubmit.evaluateJavaScript( "this.click()" ); //-- or you can generate a key/mouse event
                                }
                            }
                        }
                        
                        

                        In case you are forced to use the QWebEngine, things are a bit more complicated: you have to use a QWebChannel to expose your QObject to the JS side, where all interaction/extraction will be done.

                        In both cases to keep alive all routines/events needed for correct page rendering while widget is hidden, set the webView's (or its parent's) attribute Qt::WA_DontShowOnScreen to true.

                        1 Reply Last reply
                        0

                        • Login

                        • Login or register to search.
                        • First post
                          Last post
                        0
                        • Categories
                        • Recent
                        • Tags
                        • Popular
                        • Users
                        • Groups
                        • Search
                        • Get Qt Extensions
                        • Unsolved