Building media database

[jech]

Hello,

I'd like to ask the more experienced developers for an advice regarding my player. I've written a music player in PySide (Python 3.3 + QML 1.0) that I use on my tablet with Windows 7 in car. It works quite well according to my needs nut I'd like to develop it further. But I need to make a decision in which direction should I go. I'd like to switch to Qt5/QML 2 and also port my player to Android. To do this, I need to leave Python.

I decided to use Python because I know it and I'm totally unexperienced in C/C++. I think I could move most of the code to JS functions inside the QML. But the main part the player uses is the media database. I use a Python code, which seeks local filesystem, reads file information using MediaInfo library and stores the information in SQLite database.

AFAIK I can not access the local filesystem fro QML/JS, is this right? What would be the easiest way to get a list of all directories, subdirectories and files to the QML part of my application? Would it be a good idea to use the Audio element to read information about the media files or would it be too slow and I should rather stay with MediaInfo for this and rewrite the Python part to C/C++?

I will be grateful for any advice which will help me to choose the right direction.

[bobweaver]

I have wrote a number of media center thingys in qt/qml. Most of the time I use Sqlite to also store some of the info. I can read and write to these databases using QtQuick.LocalStorage in QML with some nifty JS work. I also tend to use QtMultimedia for my players. With there functions(start() stop() ect) I am writing a Huge Media center atm that uses Mythtv as a backend. Made a couple of tutorials also on youtube about QtMultimedia also . Along with gathering metadata and what not. I tend to Use TMDB.com and LastFM and TVDB for gathering info about media items. I like to Use XmlListModel to gather the information (check out youtube links) then insert that data into tables if they don't have that info.

Other things that I have used that have helped me out.

I made a plugin for using mysql sqlite postgre ect to make models from the data in the tables
I also use Like QFile QDir for reading, creating and writing to files/dirs via c++ to qml
I also find myself using things like QStandardPaths in C++ to setup things like syncing and what not to correct places/file paths, and scanning media, Like on start up from correct Dirs.

Links of Useful stuff
"Qt.Quick.LocalStorage":http://qt-project.org/doc/qt-5.0/qtquick/qmlmodule-qtquick-localstorage2-qtquick-localstorage-2.html
"QMultiMedia Audio":http://qt-project.org/doc/qt-5.0/qtmultimedia/audiooverview.html
"XmlListModel ":http://qt-project.org/doc/qt-5.0/qtquick/qmlmodule-qtquick-xmllistmodel2-qtquick-xmllistmodel-2.html
"SqlDataBase QML":https://github.com/bobweaver/sqlDatabase
"QDir":http://qt-project.org/doc/qt-5.0/qtcore/qdir.html
"QFile":http://qt-project.org/doc/qt-5.0/qtcore/qfile.html
"QStandardPaths":http://qt-project.org/doc/qt-5.0/qtcore/qstandardpaths.html
some tutorials
"intro to qtmultimedia":http://www.youtube.com/watch?v=Dh6ax1xVJD0
"using metadata":http://www.youtube.com/watch?v=5m4JvZFDrrQ
random
"QtMediaHub":http://qt-project.org/wiki/QtMediaHub
"alpha stuff for Myth qml":https://github.com/bobweaver/mythtv-qml
"TVDB":http://thetvdb.com/
"TMDB":http://www.themoviedb.org/

[jech]

Hi bobweaver,

thanks a lot for your advices and links. So I'll definitely have to learn some C++ to do the media scanning. I need to find a good tutorial on this. I always found some plain C guides, which claimed they are about C++. If I do something I want to do it right. Do you know any good tutorial which I could follow? If it also mentioned QtCreator as IDE, it would be great. :-)

[bobweaver]

as far as media scanning this is what I have done in the past (warning I am not a professional )

Set up the dir that you want to scan via QStandardPaths::MoviesLocation
and also make the user set there own if there is error in this &|| if they want more media dirs that are not /home/user/video or whatever the MoviesLocation is set as. You can do this via pop up. or how ever you want to implant that.

Now there are a couple of ways that you can get metadata. You can use XmlListModel then text would be equal to the filename
@XmlListModel {
source = "http://thetvdb.com/api/GetSeries.php?seriesname=" + text
//...
XmlRole{
//...
}
onStatusChanged: {
if (status == XmlListModel.Ready) console.log("Loaded"){
// Insert the Data Here to the table.
}
}

}@

That is just for TV stuff though you might want to write a script or use one that is out there already and alter it for your liking. 1st thing that comes to mind for me is.

http://wiki.xbmc.org/index.php?title=Supplemental_tools/Windows
http://www.mythtv.org/wiki/Tmdb.py
http://www.mythtv.org/wiki/Ttvdb.py

both are opensource and free to alter.

Here is a example of me using in qml (tmdb.py,ttvb.py)
https://github.com/bobweaver/mythtv-qml/blob/master/programs/frontend/qml/Mythbuntu-QML/themes/Mythbuntu/Recordings/MetaDataScanner.qml
as you can see in that example I am not inserting anything into thte database but you could if you like.

Ok so that takes care of default dirs. But what if the user has there own dirs that are like not under MoveLoacation? This is easy compared to what I thought it would be.

1. QSqlDataBase. learn how to open and write to database. If you look at your source dir for qt. Under <src(qt5)> / <yourVersion(5.1.2)> / <your compiler (gcc)> / examples / sql

you will see that there are plenty of examples that you can open the .pro file with qtcreator and hack at them.

2. Get to know QDir and learn how to call on Dirs and read from them (files or dirs).

3. QFile/QFileInfo/QString Use a foreach statement to run each file from your last function to get a list of all the movies then you can scan.

example:
somescanner.cpp
@
void lookForFiles()
{
QDir dir ;
dir.setPath("/some/path/to/movies/");
foreach (QFileInfo itm, dir.entryInfoList()) {
if (itm.isFile() ){
getName(itm.absoluteFilePath());
}
}
}

void getName(const QString &appFile)
{
//Here is where we make are scanning happen and push to DB
}
@

In you header File add Q_INVOKABLE to you functions so that you can call on them in QML. Like this

@
Q_INVOKABLE void lookForFiles()
@

Then in your main.cpp file or plugin file add the files that you have that are c++

@
#include "somescanner.h"
//...
qmlRegisterType<SomeScanner>("MetaDataScanner", 1, 0, "Scanner");
@

again there are other ways of doing things
now in qml import your C++

@
import MetaDataScanner 1.0

//...
Scanner{
id: scan
}
//.....
Component.onCompleted: {
scan.lookForFiles() // are Q_INVOKABLE
}
@

There are a ton of video tutorials out there on how to use all these things that I have suggested.

Some places to get tutorials

http://qt-project.org/videos

check out voidrealms tutorials also on youtube. he has a playlist that is over 100 videos all about c++ qt
http://www.youtube.com/playlist?list=PL2D1942A4688E9D63

I have also made some tutorials. all on QML though have not gotten to plugins. (my mic broke)
http://www.youtube.com/playlist?list=PLB22HyVdO1GkLFrvRi5vIo5XcWS0EflxD

Hope that this helps.

[jech]

Thanks a lot for your great tips and ideas.

I already looked at QDir and QFile and it seems to be pretty easy to use. I shouldn't have any problem with searching for files and directories.

But what bothers me is the database building and reading. I know how to use SQL in a QML file, I've already programmed a small tool with DB in QML. But my player should be a music player in the first place with quite a huge database (ten thousands of songs).

I can see 3 options:

1. Read the directory in C++ and then send the list with file paths to QML. In QML open each file, read it's information (audio/video, metadata etc.) But I'm afraid it would be very slow.

2. Use QMediaPlayer in C++ to scan file information. But again I'm afraid about the speed. Furthermore I don't know, how could QML access a database I create in C++. Is this possible at all?

3. Use MediaInfo just like now, which would cause additional dependency. More important again I have the problem with accessing database I created in C++ from QML.

I'll probably try to implement solution 1) and see if it's acceptable fast. Then I'll look into other possibilities. But I'm afraid i'll have to write all database related stuff in C++ anyway.

[bobweaver]

[quote author="jech" date="1380637460"]
2) Use QMediaPlayer in C++ to scan file information. But again I'm afraid about the speed. Furthermore I don't know, how could QML access a database I create in C++. Is this possible at all?
[/quote]

Maybe you would like to help with a simple plugin that I have been making.
It makes QSqlQueryModel out of data in a database and then returns it to qml model. This can be used for sql mysql sqlite postgre or anything that is used in QSqlDatabase::addDatabase . Here is a Link to the plugin

https://github.com/bobweaver/sqlDatabase

Things that need work.

• Setting the Query in Qml better
• Setting the connection to the database in a better way.
• Having more Checks and tests to make sure querys are safe lol
• settings for all the things in QSqlDatabase but in Qml
• Make the Choice of Driver to be Enumeration. Like databaseType: SqlConnection.SQlite
• Set Up count of the model to Notify in QML so there can be onCountChanged same for Status ect

I have only used it for mysql I am sure that it could be used for other drivers though.
Any help is great thanks. As I need it lol :)

[dfaure]

I see you mentionned TMDB: I'm starting a Qt5-based tmdb access library, since I couldn't find one anywhere. It's currently at http://quickgit.kde.org/?p=scratch/dfaure/libtmdbqt.git

(if you read this in the far future and you can't find it, try http://quickgit.kde.org/?p=libtmdbqt.git)