How to conutinue

SeaLongHoang

how to keep getting c_url from the code below without using requests lib

class Example:

    def __init__(self):

        self.doRequest()


    def doRequest(self):

        url = 'https://saytruyen.net/truyen-su-tro-lai-cua-phap-su-hac-am-sau-66666-nam.html'
        req = QtNetwork.QNetworkRequest(QUrl(url))

        self.nam = QtNetwork.QNetworkAccessManager()
        self.nam.finished.connect(self.handleResponse)
        self.nam.get(req)


    def handleResponse(self, reply):

        er = reply.error()

        if er == QtNetwork.QNetworkReply.NoError:

            bytes_string = reply.readAll()
            r = str(bytes_string, 'utf-8')
            html = HTML(html=r)
            rs = html.find("#list-chapter a", first = False)
            for i in reversed(rs):
                c_url = "https://saytruyen.net/" + i.attrs['href']
                

        else:
            print("Error occured: ", er)
            print(reply.errorString())

        QCoreApplication.quit()

SGaist

Hi,

What is your issue ?

SeaLongHoang

@SGaist Thank you for your time. My problem is that I don't know how to keep getting c_url using QtNetwork and QNetworkAccessManager. I'm hoping you can show me how to accomplish it.

SGaist

You are not doing anything with c_url, hence my question what is your actual issue ?

Keeping the generated urls somewhere as class members ? Using them ?

SeaLongHoang

@SGaist Oh, my bad, I intended to obtain the HTML code of the requested web page from c_url in the same way as the first url did.

eyllanesc

@SeaLongHoang You have to invoke back on request:

class Manager:
    def __init__(self):
        self.manager.finished.connect(self.handle_response)

    @cached_property
    def manager(self):
        return QtNetwork.QNetworkAccessManager()

    def start(self):
        self.start_request(
            QtCore.QUrl(
                "https://saytruyen.net/truyen-su-tro-lai-cua-phap-su-hac-am-sau-66666-nam.html"
            )
        )

    def start_request(self, url):
        request = QtNetwork.QNetworkRequest(url)
        self.manager.get(request)

    def handle_response(self, reply):
        err = reply.error()
        if err == QtNetwork.QNetworkReply.NoError:
            self.process(reply.readAll().encode("utf-8"))
        else:
            print("Error occured: ", err)
            print(reply.errorString())

    def process(self, data):
        html = HTML(html=data)
        rs = html.find("#list-chapter a", first=False)
        for i in reversed(rs):
            url = "https://saytruyen.net/" + i.attrs["href"]
            self.start_request(QtCore.QUrl(url))


def main():
    app = QtCore.QCoreApplication()

    manager = Manager()
    manager.start()

    app.exec_()


if __name__ == "__main__":
    main()

SeaLongHoang

@eyllanesc When I execute that code, I discover some bugs.

Traceback (most recent call last):
  File "c:\Users\ASUS\Videos\LEECH MANGA\test.py", line 40, in <module>
    manager = Manager()
  File "c:\Users\ASUS\Videos\LEECH MANGA\test.py", line 6, in __init__ 
    self.manager.finished.connect(self.handle_response)
AttributeError: 'function' object has no attribute 'finished'

jsulm

@SeaLongHoang said in How to conutinue:

self.manager.finished.connect(self.handle_response)

Then fix this line...

self.manager().finished.connect(self.handle_response)

SeaLongHoang

@jsulm Thank you for your advice, but it is still has a bug. When I run it, it returns this:

QObject::connect: No such signal QObject::finished(QNetworkReply*)

eyllanesc

@jsulm In my original code I use cached_property so you should not modify it but import it: from functools import cached_property

SeaLongHoang

@eyllanesc Thank you very much, however I believe you could examine your code more carefully at times, such as this lineself.process(reply.readAll().encode("utf-8")) it must be self.process(str(reply.readAll(), 'utf-8'))

jsulm

@SeaLongHoang said in How to conutinue:

however I believe you could examine your code more carefully at times

Well, people here are volunteers and don't have to write exact and tested code. You should adjust the code as needed if something is wrong there.

SeaLongHoang

@jsulm Okay, I get that, and I appreciate all of your assistance, but I have a little question for you: how can I continue to locate elements from that new request?

@eyllanesc said in How to conutinue:

   self.start_request(QtCore.QUrl(url))

jsulm

@SeaLongHoang said in How to conutinue:

how can I continue to locate elements from that new request?

What do you mean by that?
If you mean how to handle the response - that is what handle_response() slot does.

SeaLongHoang

@jsulm I mean, if I want to keep getting elements like this from new requests, what should I do? You may view the code below.

html = HTML(html=data)
rs = html.find("#list-chapter a", first=False)
for i in reversed(rs):
            url = "https://saytruyen.net/" + i.attrs["href"]
            self.start_request(QtCore.QUrl(url))

jsulm

@SeaLongHoang said in How to conutinue:

what should I do?

Nothing special. For each request handle_response() will be called which has parameter "reply".

SeaLongHoang

@jsulm What should I replace "...." with in this line html = HTML(html=....) after request to source code HTML from url self.start_request(QtCore.QUrl(url))

jsulm

@SeaLongHoang Please take a closer look at the code @eyllanesc gave you.