Manipulate all HTTP Requests

exzibit7

For months I've been trying to achieve something like this, but could not find any definite solution to it.
I've created a browser using QWebView, and I want to manipulate(tamper/truncate/append/delete) all the http data sent and received from the QWebView. Something like a HTTP sniffer but only for the QWebView.

For example, if the QWebView loads "http://www.google.com":http://www.google.com, it will further make these requests for the images,css,javascript on the page:
@https://lh3.googleusercontent.com/-U3Amb0g0kCE/AAAAAAAAAAI/AAAAAAAAAAA/6AX3CXIbeY8/s27-c/photo.jpg
http://www.google.com/images/srpr/logo3w.png
https://www.google.com/images/icons/product/chrome-48.png
https://www.google.com/extern_chrome/e5f7e389b8930964.js@
Depending on the situation I may want to manipulate any of the above requests dynamically. For example I may want to change the "photo.jpg" request to some other image, or add to extra lines to the "e5f7e389b8930964.js" file or even abort the "logo3w.png" request.

So far I have tried the following but had no success,

Subclass QNetworkAccessManager of the QWebView and override the createRequest() method.

Subclass QNetworkReply and manipulate the data requested.

So isn't there any way I can achieve this?

Thanks.

AcerExtensa

bq. So far I have tried the following but had no success,
Subclass QNetworkAccessManager of the QWebView and override the createRequest() method.
Subclass QNetworkReply and manipulate the data requested.

And why? What was the problem?

exzibit7

[quote author="AcerExtensa" date="1338996180"]bq. So far I have tried the following but had no success,
Subclass QNetworkAccessManager of the QWebView and override the createRequest() method.
Subclass QNetworkReply and manipulate the data requested.

And why? What was the problem? [/quote]

By overriding createRequest() I was able to abort requests completely or redirect them. But I wasn't able to tamper the received data as the QNetworkReply returned is asynchronous.

So I tried connecting to the finished slot of the QNetworkReply, but still could not tamper it because QNetworkReply is readonly.

So instead I subclassed QNetworkReply and tried to write data to it using QNetworkReply::open(QIODevice::ReadWrite) and QNetworkReply::write(const QByteArray &data). Even though the open() method returned true, the write() method returned -1 i.e no data was written. Still don't know the reason of this.

AcerExtensa

QNetworkReply inherits QIODevice, if you can subclass QNetworkReply with subclassed QIODevice, maybe then you can get access to data manipulation...

AcerExtensa

something like "this":http://gitorious.org/qtwebkit/performance/blobs/master/host-tools/mirror/main.cpp

Nemoar

It is related I guess:

Can WebKit replace “Firefox + NoScript + ImgLikeOpera”?
http://qt-project.org/forums/viewthread/20993/

dangelog

[quote author="exzibit7" date="1338996754"]
By overriding createRequest() I was able to abort requests completely or redirect them. But I wasn't able to tamper the received data as the QNetworkReply returned is asynchronous.

So I tried connecting to the finished slot of the QNetworkReply, but still could not tamper it because QNetworkReply is readonly.[/quote]

You need a "proxy" QNetworkReply that changes data (on the fly, if you wish), based on the original one. There should be an implementation somewhere in webkit tests, as well as one on Richard Moore's blog on KDE.

Nemoar

Using PySide, I block/tamper images like this:
(Relevant portions of classes follows:)

@
netMan = NetMan()

class WebView(QWebView):
def init(self):
QWebView.init(self)
self.page().setNetworkAccessManager(netMan)

class NetMan(QNetworkAccessManager):
def createRequest(self, op, req, data = None):
url = req.url().toString()
urll = url.lower()
for test in ['.jpg', '.jpeg', '.gif', '.png']:
if test in urll:
req.setUrl(QUrl('file:///web/missing.png'))
break

return QNetworkAccessManager.createRequest(self, op, req, data)

@