Download videos from youtube - help.
-
[quote author="skycrusher" date="1294345809"]Just one more question:
When I set a cookiejar for QNetworkAccessManager:
@
QNetworkAccessManager * manager = new QNetworkAccessManager(this);
manager->setCookieJar(&cookie_JAR);
@
Do I need to post the cookie info before downloading the video?[/quote]What do you mean by post the "cookie info"? You need to add the cookies to the cookie jar and set the cookie jar on the manager (in no particular order) before you use the manager to get the file. The manager will take care of sending the cookies based on the domain.
One question though (I must be missing something): how do you get the cookies on the first place? Just a simple request or there is some scrapping to find the cookies on the response? If a request returns the cookies (as cookies) you are all set for the second request.
-
Hello, fcrochik. Thanks for replying.
I thought I had to post the cookies names/values to download the video.
If you're saying that
@
manager->setCookieJar(&cookie_JAR);
@is enough, so be it! =)
But even so, I still can't download the video.Answering your question: I get the cookies (and the real video url) from a python script called "youtube-dl.":https://github.com/rg3/youtube-dl
To make things clearer, I'll show you an example:
In a command line:
@
./youtube-dl --cookies=cookiejarfile -g 'http://www.youtube.com/watch?v=OYjZK_6i37M&feature=fvst'
@
I get the link:
@
"http://v2.lscache6.c.youtube.com/videoplayback?ip=0.0.0.0&sparams=id,expire,ip,ipbits,itag,algorithm,burst,factor,oc:U0dYSVNTTl9FSkNNOF9LTVJB&fexp=905500&algorithm=throttle-factor&itag=35&ipbits=0&burst=40&sver=3&expire=1294380000&key=yt1&signature=267F9CBE2AEAF4D1459FBC7369BDCB88DEEF96F7.7E6725F492AB86449870565C68653D73C3F4131B&factor=1.25&id=3988d92bfea2dfb3"
@And this python script also writes a cookie jar file. This file's contents are:
@
# Netscape HTTP Cookie File
# http://www.netscape.com/newsref/std/cookie_spec.html
# This is a generated file! Do not edit..youtube.com TRUE / FALSE 1609716210 PREF f1=50000000&gl=US&hl=en
.youtube.com TRUE / FALSE 1315092210 VISITOR_INFO1_LIVE TnDv4gFkTIA
@
I read this file and parse the two cookies of it and add them to a QNetworkCookieJar.
I use this QNetworkCookieJar in a QNetworkAccessManager variable.
with QNetworkRequest, I use the video url and try to download. With no success. =(
The script really works. I tested with "Curl":https://github.com/shuber/curl . And the video was downloaded just fine.I'm trying to download the video with QT, but I don't know what's wrong. =/
-
It looks like you create the QNetworkCookieJar object on the stack, because you get the pointer with the & operator:
@
manager->setCookieJar(&cookie_JAR)
@If you do so, it is wrong. You have to create the cookie jar on the heap with new:
@
QNetworkCookieJar cookie_JAR = new QNetworkCookieJar(this)
// ...
manager->setCookieJar(cookie_JAR);
@The reason is in the API docs for "QNetworkAccessManager::setCookieJar() ":http://doc.qt.nokia.com/stable/qnetworkaccessmanager.html#setCookieJar:
bq. Note: QNetworkAccessManager takes ownership of the cookieJar object.
And being QObject based, it is always a good idea to create QNetworkCookieJar on the heap.
Also, the object you created on the stack might go out of scope (and be deleted) before QNetworkAccessManager accesses it.
-
[quote author="Volker" date="1294357703"]
And being QObject based, it is always a good idea to create QNetworkCookieJar on the heap.Also, the object you created on the stack might go out of scope (and be deleted) before QNetworkAccessManager accesses it.
[/quote]
Volker: good catch! But I am not sure this is the only problem. I would expect a seg fault...
Skycrusher: once you are taking the time to write this part as a Qt application why don't you rewrite the python script as well? It may actually make the overall process easier and certainly more portable.
-
If you want to a little crazier you can check out "cuteTube":http://talk.maemo.org/showthread.php?t=65854
I haven't tried and don't know how it does but it may actually be as simple as recompiling it for your platform.
-
fcrochik: This "cutetube" is awesome, but I want a program a little different than that. :)
I'm trying to make something like a "virtual video jukebox", but thanks anyway. ;)
I want this program the be Qt-based because I think with Qt/C++ it is easier to make GUIs.Volker:
[quote author="Volker" date="1294357703"]It looks like you create the QNetworkCookieJar object on the stack, because you get the pointer with the & operator:@
manager->setCookieJar(&cookie_JAR)
@If you do so, it is wrong. You have to create the cookie jar on the heap with new.
[/quote]
I just did the changes you told me. And the problem remains.
Let me show you the code:@
void Script_Youtube_Downloader::do_download()
{
qDebug("do_download");QNetworkAccessManager * manager = new QNetworkAccessManager(); QNetworkCookieJar * cookie_JAR = new QNetworkCookieJar(); QUrl url; QUrl passed_url; QList<QNetworkCookie> cookies; QNetworkRequest* request = new QNetworkRequest(); cookies = this->parse_my_cookiejar(this->get_cookie_info()); passed_url.setUrl(this->get_passed_url(), QUrl::StrictMode); url.setUrl(this->get_real_url(), QUrl::StrictMode); request->setUrl(url); request->setRawHeader("User-Agent", "Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.12) Gecko/20101028 Firefox/3.6.12"); request->setRawHeader("Accept-Charset", "ISO-8859-1,utf-8;q=0.7,*;q=0.7"); request->setRawHeader("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"); request->setRawHeader("Accept-Encoding", "gzip, deflate"); request->setRawHeader("Accept-Language", "en-us,en;q=0.5"); if (cookie_JAR->setCookiesFromUrl(cookies, url)) qDebug("cookie JAr OK!\n"); else qDebug("COOKIE_JAR_ERROR\n"); manager->setCookieJar(cookie_JAR); if (this->get_cookie_info().isEmpty()) qDebug("EMPTY cookie jar from python"); request->setUrl(url); reply = manager->get(*request); //"reply" is a public QNetworkReply attribute of this class reply->waitForReadyRead(-1); qDebug("Is it any redirection address?"); qDebug(reply->attribute(QNetworkRequest::RedirectionTargetAttribute).toUrl().toString().toUtf8()); connect(reply, SIGNAL(downloadProgress(qint64,qint64)), this, SLOT(printProgress(qint64, qint64))); connect(reply, SIGNAL(readyRead()), this, SLOT(printOk())); connect(manager, SIGNAL(finished(QNetworkReply*)), this, SLOT(videoDownloaded(QNetworkReply*))); connect(reply, SIGNAL(readChannelFinished()), this, SLOT(printOk()) ); connect(reply, SIGNAL(error(QNetworkReply::NetworkError)), this, SLOT(printNotOk(QNetworkReply::NetworkError)));}
@And the program debug outuput is:
@
/*
do_download
Is it any redirection address?An error HAS SERIOUSLY OCCURED: 202
Redirection URL is EMPTY...
And error occured while I was trying to download
Http attribute:
HttpStatusCode-->403
*/
@ -
Fcrochik,
Just removed the "waitForReadyRead" command.
The error continues.
Actually, 202 is an error from QNetworkReply (it means that the requested operation was denied by the server). Look what I wrote in the "printnotOk" slot:
@
void Script_Youtube_Downloader::printNotOk(QNetworkReply::NetworkError error)
{
qDebug("An error HAS SERIOUSLY OCCURED: " + QString::number(error).toUtf8());
}
@and error 403 is an error from the Http protocol (it means that the operation was denied by the server).
Look the "videoDownloaded" SLOT implementation:
@
void Script_Youtube_Downloader::videoDownloaded(QNetworkReply* reply)
{
qDebug("received "finished" SIGNAL.\n");QUrl url; QUrl redir; redir = reply->attribute(QNetworkRequest::RedirectionTargetAttribute).toUrl(); url = reply->url(); if(!url.isValid()) qDebug("Reply's url is not valid! Review your code."); if (redir.isEmpty()) { qDebug("Redirection URL is EMPTY..."); } if (reply->error()) { qDebug("And error occured while I was trying to download"); qDebug("Http attribute:"); qDebug("HttpStatusCode-->%d", reply->attribute(QNetworkRequest::HttpStatusCodeAttribute).toInt()); } else { if (saveToDisk("SALVA_DIABO!!!", reply)) qDebug("file saved to: SALVA_DIABO" ); }}
@ -
There is a nice technique that you can use to explore what is going on. Instead of using the connection manager use a QWebView. You can get access to the networkmanager set the cookiejar and then use the webview to navigate to the URL. This will let webkit in charge of all the request handling and you will be able to verify that the cookie/request handling is working.
p.s. Gostei do "Salva diabo"! (Portuguese)
-
I am having almost the same problem. I am decrypting the real URL of youtube video myself, using the same way as JDownloader, as follows:
-
Use QNetworkAccessManager to access the HTML content of the youtube URL, say: www.youtube.com/watch?v=tBF6MA0xfps
-
Analyze HTML and construct the video's real URL (a very long URL).
After the above two steps, I could get the video's real URL as well QNetworkCookieJar from the QNetworkAccessManager I used.
However, I got ERROR 202 from QNetworkReply when try to download from the real URL using the same QNetworkAccessManager and the same QNetworkCookieJar.
The decrypted URL could be directly played by Chrome, Safari, VLC, or downloaded by wget, curl, and libcurl. This means that accessing the decrypted URL does not require cookie magic.
I tried using the same or different QNetworkAccessManager with the same or different QNetworkCookieJar to access the decrypted URL. But I always got error 202.
There must be some bugs in QNetworkAccessManager.I also tried Qt 4.8.1 on Windows 7, and Qt 4.7.4 on Mac OS X 10.7, but got the same error 202. I think the bug is probably in Qt's OS-independent code path.
-
