Qt DevNet Web Service API

ZapB · wrote on 11 Apr 2011, 10:23

Thanks for the mods. I'll patch my source for win support with it.

Edit: I have applied the patch this in svn now.

ZapB · wrote on 22 Apr 2011, 14:51

Just a small update. The login dialog is now handled via QML too (the actual network stuff is still in C++ of course). It's in svn if anybody wants to grab the update. Here's a screenie (no bling-tastic animations yet though).

!http://gallery.theharmers.co.uk/upload/2011/04/22/20110422154652-693ce8de.png(QML Login Screen)!

ps I hope you don't mind me grabbing artistic assets from the site for use in this client. ;-)

mgran · wrote on 22 Apr 2011, 22:51

Sweet ZapB, I'll try out the updates once I get back from the Easter break and to a proper pc :)

ZapB · wrote on 23 Apr 2011, 13:31

OK another hour of playing with QML and the login page now behaves more like you would expect, including things like:

Enabling/disabling login button based on user name and password fields being populated

Visible feedback on login button status (enabled/disabled has focus/no focus)

Tab key navigation between elements

Plus it has some bling now - you'll have to run it to see it. ;-)

My opinion of QML so far is that it is very nice, but it will be a whole let better once we have one or more decent sets of components to use (desktop components would be really cool - and yes I have seen the blog with them in). It is instructional to develop your own from first principals but requires lots of digging in the docs for QML noobs like me.

Next step is to see if I can get this converted into a mobile app and try in on my N8. Now where did I put that SDK...

ZapB · wrote on 24 Apr 2011, 17:40

Sweet! I just managed to get this running on my N8. It's very awesome to see the same QML animations running on the phone - smoothly too! :D

It took a little bit of trial and error and I am still not quite sure what made it start to deploy properly and I can still not get qjson to build for the qt-simulator target. I'll write up some instructions on how to do this and commit my mods to the svn repo shortly.

Edit 1: Code to allow this to be built and deployed for Symbian is now in svn.

Edit 2: Figured out what was stopping me from running this on the device originally. I kept getting the "General OS Error" when trying to deploy and run my app. It turns out that it was because of insufficient capabilities in the qjson lib on which my app depends. By changing the TARGET.CAPABILITY line in qjson/src/src.pro to:

@
TARGET.CAPABILITY = NetworkServices ReadDeviceData WriteDeviceData
@

allows my app to load the qjson dll on the device (since my app needs access to the Internet). Read about "capabilities":http://wiki.forum.nokia.com/index.php/Capabilities for more info on this.

I have concerns about how this may scale if several apps want to use the same dll but with different capabilities but that's something to worry about for another day.

ZapB · wrote on 2 May 2011, 10:36

Marius, I wonder if you could help me with something please? I am trying to add avatar support to my little QtDN client and until the API supports returning the avatar image URL I thought I would take the opportunity to try using QXmlQuery to parse the xhtml of the front page of QtDN to extract the gravatar source url.

This is all going fine until I get this error from QXmlQuery:

@
Error FODC0002 in tag:trolltech.com,2007:QtXmlPatterns:QIODeviceVariable:inputDocument, at line 74, column 32: Expected '=', but got '>'.
@

Looking at the page source I see that line 74 is:

@
<div class="wrapper" {set_mode}>
@

which of course is not valid xhtml (or xml) as advertised by the doctype. I can work around this by stripping this out before I pass it onto the QXmlQuery of course but would you like me to report this as a bug? Or even better is there an easy fix you can apply to get rid of the above non-valid entry please?

Many thanks.

ZapB · wrote on 2 May 2011, 10:44

Another piece of invalid xml is found on line 190:

@
<div class="item"><a href="http://info.arcada.fi/sv/qtquick"><img src="http://www.arcada.fi/sites/www.arcada.fi/themes/arcada/media/separate_elements/main_navigation/logo.png"></a></div>
@

The error is:

@
Error FODC0002 in tag:trolltech.com,2007:QtXmlPatterns:QIODeviceVariable:inputDocument, at line 190, column 180: Opening and ending tag mismatch.
@

The img element is missing the empty tag specifier - ie it should have a "/" at the end of the img tag.

Edit: This input tag on line 532 is also missing a closing "/"

@
<input type="text" name="tag_search" id="tag_search_input" value="">
@

ZapB · wrote on 2 May 2011, 13:36

I found one other issue that prevents the main page from validating as well-formed xml. The javascript at the bottom of the page (starting on line 710):

@
function searchStart()
...
});
@

really needs to be wrapped in a CDATA section like this:

@
<![CDATA[
function searchStart()
...
});
]]>
@

I have put in place some hackish work-arounds to get over these non-conformances. Here is what I have done to scrape the gravatar image from the front page.

In the Profile::update() function (which is where I usually use QNAM to query the profile API) I also do this on the first call after a successful login:

@
static bool firstTime = true;
if ( firstTime )
{
// Load the front page so that we can scrape the avatar
firstTime = false;
QNetworkRequest avatarRequest;
avatarRequest.setUrl( QUrl( "http://developer.qt.nokia.com/" ) );
avatarRequest.setRawHeader( "User-Agent", "ZapBs QtDevNet Client 0.1" );

    QNetworkReply* avatarReply = m_nam->get( avatarRequest );
    connect( avatarReply, SIGNAL( finished() ),
             SLOT( _q_extractAvatar() ) );
    connect( avatarReply, SIGNAL( error( QNetworkReply::NetworkError ) ),
             SLOT( _q_onReplyError( QNetworkReply::NetworkError ) ) );
}

@

This just uses QNAM to fetch the xhtml document for the QtDN front page.

Edit: Had to split post as it was too large.

ZapB · wrote on 2 May 2011, 13:36

When the request has finished we go into the private slot _q_extractAvatar(). This does a few things:

Checks that we have a real QNetworkReply.

Reads the entire xhtml document from the reply, and schedules the reply for deletion.

Performs a few hacky replacements on the document to work around xml validation errors.

Uses QXmlQuery to find the appropriate img element for the avatar image, extracts the src attribute and stores this as an attribute of a trivial xml document as the output.

Uses some simple manipulations to get the actual avatar source url

Replaces the size from 48px which is used on the front page to 80px which I use in the client.

Emits the avatarUrlChanged() signal to nudge the QML component into action

Here's the code that does it:

@
void Profile::_q_extractAvatar()
{
qDebug() << Q_FUNC_INFO;

QNetworkReply* reply = qobject_cast<QNetworkReply*>( sender() );
if ( !reply )
    return;

// Hack around broken xhtml on QtDevNet front page
QByteArray input( reply->readAll() );
reply->deleteLater(); // Delete the reply when we go back to the event loop
input.replace( "{set_mode}",
               "" );
input.replace( "<img src="http://www.arcada.fi/sites/www.arcada.fi/themes/arcada/media/separate_elements/main_navigation/logo.png">",
               "<img src="http://www.arcada.fi/sites/www.arcada.fi/themes/arcada/media/separate_elements/main_navigation/logo.png" />" );
input.replace( "&lt;input type="text" name="tag_search" id="tag_search_input" value=""&gt;",
               "&lt;input type="text" name="tag_search" id="tag_search_input" value="" /&gt;" );
input.replace( "function searchStart",
               "&lt;![CDATA[function searchStart" );
input.replace( "});n[removed]",
               "});n]]>[removed]" );

QBuffer device;
device.setBuffer( &input );
device.open( QIODevice::ReadOnly );

// Our XQuery string
QString queryString( "declare default element namespace "http://www.w3.org/1999/xhtml";n"
                     "declare variable $inputDocument external;n"
                     "doc($inputDocument)//div[@class="frontpage-profile-image"]/a/img/n"
                     "<p>{@src}</p>" );

// Prepare the xml query object
QXmlQuery query;
query.bindVariable( "inputDocument", &device );
query.setQuery( queryString );
if ( !query.isValid() )
    return;

// We need a buffer to write the output to
QByteArray outArray;
QBuffer buffer( &outArray );
buffer.open( QIODevice::ReadWrite );

// Set up a formatter and execute the query
QXmlFormatter formatter( query, &buffer );
if ( !query.evaluateTo( &formatter ) )
    return;
buffer.close();
//qDebug() << "Query output:" << outArray.constData();

// Now get the actual source url that we need
QString needle( "src="" );
int index = outArray.indexOf( needle );
if ( index != -1 )
{
    QByteArray avatarSource = outArray.mid( index + needle.length() );
    int idx = avatarSource.indexOf( """ );
    if ( idx != -1 )
    {
        avatarSource = avatarSource.mid( 0, idx );
        avatarSource.replace( "size=48", "size=80" );
        m_avatarUrl.setUrl( QString::fromLatin1( avatarSource ) );
        emit avatarUrlChanged();
    }
}

}
@

I know I could have replaced the QXmlQuery stuff with a simple text search but I think this way is more robust. Plus it was a chance for me to use the XmlPatterns module which is still very new to me (which probably shows in my XQuery string).

ZapB · wrote on 2 May 2011, 13:37

Marius, it might also be nice to have the memberId returned as part of the profile query in the restful API. ie for my account it would include a json attribute of "memberId" : "3246".

So overall it would be nice if the profile json response looked like this:

@
{
"memberId" : "3246",
"gravatarHash" : "d4737278a208398242bdf2535701f8a3",
"current_rank":
{
"points":"494",
"points_last_7_days":"296",
"level":2,
"title":"Ant Farmer",
"image":"http://developer.qt.nokia.com/images/qtdn/level_images/level2.png"
},
"next_rank":
{
"points_remaining":47,
"level":3,
"title":"Hobby Entomologist",
"image":"http://developer.qt.nokia.com/images/qtdn/level_images/level3.png"
},
"badges": [
{
"name":"Nokia Certified Qt Developer",
"url":"http://developer.qt.nokia.com/images/qtdn/icon_certified_developer.png"
}]
}
@

By returning the gravatar hash rather than the actual URL, clients could construct their own url for custom sizes and default image fallbacks more easily.

goetz · wrote on 2 May 2011, 13:48

ZapB, bad news for you: Marius is on holidays for the next two weeks. I think it would be better to file a bug report, so these things are not missing.

ZapB · wrote on 2 May 2011, 13:59

Hi Volker, thanks for the heads-up. I'll file a bug report.

Edit: Created issues "QTWEBSITE-220":http://bugreports.qt.nokia.com/browse/QTWEBSITE-220 and "QTWEBSITE-221":http://bugreports.qt.nokia.com/browse/QTWEBSITE-221

troubalex · wrote on 2 May 2011, 14:36

Marius is on holidays. He promised to look at the forums from time to time but not sure when and how much time he will want to spend. ;)

In the meantime I'll poke one of our devs for you.

ZapB · wrote on 2 May 2011, 15:05

Thanks Alexandra. I've just noticed another problem on the front page (added a comment to "QTWEBSITE-220":http://bugreports.qt.nokia.com/browse/QTWEBSITE-220). The new forum post summary on the main page does not have reserved characters replaced by their corresponding xml entities. ie & should be "& a m p ;" Edit: had to add spaces to stop forum software from replacing it here too ;-)

I'll pause development of this client there for now whilst I document some of the things used in it as it makes a nice show-case for some C++/QML integration, QNAM usage, QXmlQuery usage, custom QML components etc. Once I get these wiki'fied and the API is reinstated I'll carry on to the next set of features (probably forum related stuff).

Thanks again!

gurudutt · wrote on 2 May 2011, 16:28

[quote author="ZapB" date="1304333076"]Another piece of invalid xml is found on line 190: @ <div class="item"><a href="http://info.arcada.fi/sv/qtquick"><img src="http://www.arcada.fi/sites/www.arcada.fi/themes/arcada/media/separate_elements/main_navigation/logo.png"></a></div> @ The error is: @ Error FODC0002 in tag:trolltech.com,2007:QtXmlPatterns:QIODeviceVariable:inputDocument, at line 190, column 180: Opening and ending tag mismatch. @ The img element is missing the empty tag specifier - ie it should have a "/" at the end of the img tag. Edit: This input tag on line 532 is also missing a closing "/" @ <input type="text" name="tag_search" id="tag_search_input" value=""> @[/quote]

All possible above stuffs are fixed,
for other fixes it has to wait for deployment in future. I will create a issue here and will try to do all possible fixes.

ZapB · wrote on 2 May 2011, 18:09

Hi Gurudutt. Thanks for the quick turnaround on those little fixes. Can you mark the task as a sub-task of "QTWEBSITE-220":http://bugreports.qt.nokia.com/browse/QTWEBSITE-220 please so that I can track it?

I appreciate that the other fixes will require changes to some php code and so must be tested and verified. Is the code for QtDevNet open source and on Gitorious? If so, I don't mind taking a poke around to see if I can fix them up.

Thanks again.

mgran · wrote on 3 May 2011, 03:40

Hey ZapB, I'm on vacation atm but Gurudutt is all over it :)

Looking forward to trying it on my N8 when I'm back!

Edit: Didn't see the last few replies, edited a bit

ZapB · wrote on 3 May 2011, 08:22

Hi Marius. No worries. Enjoy your holidays! I'm back at work today after having last week off. I've just registered with the Ovi store and once I test the current state of the app (when the API is re-enabled) I'll build it and publish it on Ovi (for free of course).

I've set up the start of a drupal-based "website":http://www.zerotau.co.uk/ so that I could apply for the Qt Ambassador program on the back of this little project. My "plotting library":http://developer.qt.nokia.com/forums/viewthread/4209/ is still a little way off being ready for release yet and I want to keep the source code private until the first public release - just so as to make a nice surprise :-).

ZapB · wrote on 5 May 2011, 10:55

Gurudutt, another problem today. Onthe front page again, line 226, col 181:

@
<div class="item"><img src="http://developer.qt.nokia.com/uploads/qt_quarterly/qt_ambassador_logo.png" alt="Qt Ambassador" width="150" height="201" alt="image" /></div>
@

the alt attribute is redefined leading to another error when using XQuery on that xhtml. Could you squash that bug too please? It looks like a simple static html one.

Many thanks.

gurudutt · wrote on 5 May 2011, 14:45

[quote author="ZapB" date="1304592956"]Gurudutt, another problem today. Onthe front page again, line 226, col 181:
the alt attribute is redefined leading to another error when using XQuery on that xhtml. Could you squash that bug too please? It looks like a simple static html one.

Many thanks.[/quote]

It should be fixed now.