[SOLVED]screen/audio recording with ffmpeg using QProcess

musimbate

Hi,
I need to integrate screen casting and media playback in my qt application.Basically , we want to capture everything the user is doing on screen save it in video format ."something like camtasia studio":http://www.techsmith.com/camtasia.html but it doesn't need to be that advanced in editing . We want to save the recorded screen information in our custom video format . We also have to build a Qt based player to play that custom video format of ours .

My search has pointed me to these two frameworks "ffmpeg":http://ffmpeg.org/ and "gstreamer":http://www.gstreamer.com/ .I have no previous experience with multimedia programming so excuse my dumb questions if any .So here are my questions:

It is clear that these two can do media playback but not that clear when it comes to screen capturing .Can they help with my screen capturing needs as mentioned before?

2.My platform is windows x86 (Qt 4.8.6 under vs2010) ,which one is matches better?By this I mean ease of integration of the libraries and possibly less trouble (dependencies) when the user tries to run the final application.

3.I am completely new to multimedia programming,had never dealt with video/audio encoding and decoding before,so any idea that can help is highly welcome .

4.How well do these integrate with Qt.Suggestions.

Thank you for your time.

Dan.

SGaist

Hi,

"ffmpeg":https://trac.ffmpeg.org/wiki/Capture/Desktop
"gstreamer"http://gstreamer.freedesktop.org/data/doc/gstreamer/head/gst-plugins-good-plugins/html/gst-plugins-good-plugins-ximagesrc.html

Seems that is what you are looking for (haven't tried them so you'll have to test)

If you go the gstreamer way, there's the QtGStreamer library that you could try. It builds both with Qt 4 and Qt 5. There might wrappers for ffmpeg, I only remember QtAV which is a playback library.
Get you hands dirty using the examples provided/findable and the documentation. Each library is different, e.g. ffmpeg is pretty powerful but can be hard to get into. So the best thing to do is get a working example of what you'd like to achieve and play with it until it becomes exactly what you need.
Both example are pretty easy to integrate.

Hope it helps

musimbate

Thank you SGaist for the detailed reply,
I am going through gstreamer right now and would go for QtGStreamer when I have a stronger grasp of the basics.

Totaly agree with advice no 3).

Thanks again.

musimbate

Hello,
I have been wresting with gstreamer and it is good with playback on windows.The problem I have now is with screencasting .I looked at "ximagesrc":http://gstreamer.freedesktop.org/data/doc/gstreamer/head/gst-plugins-good-plugins/html/gst-plugins-good-plugins-plugin-ximagesrc.html doc and found that it only grabs x11 windows. Is there a similar gstreamer element that can work on other windowing systems ,namely windows? I am looking into ffmpeg for these needs for the moment .

Thank you for your time.

SGaist

Good question, I'd recommend asking this on the gstreamer forum/mailing list. I don't know whether they implemented something similar for windows. However, and it's just a wild idea I didn't test it, it might be possible using QtGstreamer. You should try asking them. AFAIK, the QtQuick and QtWidget elements are just sinks

echostorm

my suggestion is you can use the CLI version and use the standard commands you can find online but simply tweak them from your QT UI any reason why you need to link the library into your own app? I think you dont have to since all you want is screen capture... with regards to custom format just add it as a plugin for gstreamer... I believe this approach may make life easy for you.

basically you just need to create the front-end GUI for gstreamer the way VLC does with both gstreamer and ffmpeg but only for your specific plugin for both playback and screencap.

musimbate

Hello and thank you all for your help/suggestions,
I will look more into QtGstreamer when the time allows .

I would really like to give QtGsteamer a shot as it might be my best solution but I haven't been able to build it so far .I am also skeptical as it might have the same problem I have described "here":http://qt-project.org/forums/viewthread/48048/ on my system.As I have a deadline to hit , ffmpeg is the only option I got left ,and this is my current problem.

I am trying ffmpeg to capture screen and the command :

@
ffmpeg -f dshow -i video="UScreenCapture" out.mp4
@
is working fine from the command line .

But when I call it through QProcess like this

@
arguments<< "-f"<<"dshow"<< "-i"<< "video="UScreenCapture""<<"D:\ffmpeg\bin\out.mp4";
@

I get errors about formating.(The command prompt window disappers before I can see what it says).

Any help would be appreciated.

Thanks

echostorm

@QString pathtoffmpeg = "D:/ffmpeg/bin/ffmpeg";
QStringList ffmpegarguments;
ffmpegarguments << "-f" << "dshow" << "-i" << "video=UScreenCapture" << "D:\ffmpeg\bin\out.mp4";
myProcess->start(pathtoffmpeg, ffmpegarguments);@

That worked fine for me I also got the out.mp4 file so it works..

Dont forget to add error detection through channels like for example if file exists etc...

echostorm

ffmpeg doesn't need " " for video source btw... try it you will see it works without it also... if you just want to make sure that its working real fast without doing shutdown signal just add a duration for testing simply add this to arguments:
@ffmpegarguments << "-f" << "dshow" << "-t" << "10" << "-i" << "video=UScreenCapture" << "D:\ffmpeg\bin\out.mp4";@
this will record 10 seconds for you... btw thanks for this nice find I just used the UScreenCapture codec for my own project :D

musimbate

Hello echostorm ,
Thank you for your tip ,your format works for me to and I am glad UScreenCapture worked for your project .I have a few other issues with my project though.

1.I am getting the audio device with this statement
@
QString AudioDeviceName= QAudioDeviceInfo::availableDevices(QAudio::AudioInput).at(0).deviceName();
@

but when I pass the audio device together with with UScreenCapture to record audio and video alltogether like this:

@
arguments << "-f" << "dshow" << "-i" << "video=UScreenCapture" <<"-f"<< "dshow"<< "-i" <<AudioDeviceName <<"D:\ffmpeg\bin\out.mp4";
@
and it complains again with the formats :-(

I also tried to get the audioDeviceName myself and pass it the same way we did for UScreenCapture :
@
arguments << "-f" << "dshow" << "-i" << "video=UScreenCapture" <<"-f"<< "dshow"<< "-i" << "audio=麦克风 (Realtek High Definition Au" <<"D:\ffmpeg\bin\out.mp4";
@
and it didn't work eather .There are some chinese characters in the name in my audio device .Could this be related to some utf issues?

2.I have to provide the ability for the user to control the begining and the end of the recording process .My idea if for example when he presses the startRecord button I call ffmpeg and instruct it to start recording ,and when he presses stopRecord I instruct ffmpeg to stop recording .I can't figure out how to tell ffmpeg to stop recording from my Qt application . From the command prompt I simply type Ctrl + C and it stops but how do I do that form Qt

3.ffmpeg opens the black/white command line window when you call it from your gui application .How do I make sure it doesn't show up in my gui .Is there a way we can redirect ffmpeg output somewhere else or a way to hide the Command prompt window off the visible area of the screen?

If you dealt with these issues in your application ,I would be glad if you shared.

Thank you for the input .(and sorry for the long post :-)

musimbate

Hi,
Seems I have something for issue no 3 ,
from this "tutorial":http://www.bogotobogo.com/Qt/Qt5_QProcess_QFileDialog_QTextEdit_FFmpeg.php I could cook up some code to redirect output somewhere else:

I create the ffmpegRecordingProcess like this :
@
mTranscodingProcess = new QProcess(this);
@

Start it like this:
@
//Used here to do transcoding
arguments << "-i" << input << output;

   qDebug() << arguments;

   ffmpegRecordingProcess->setProcessChannelMode(QProcess::MergedChannels);
   ffmpegRecordingProcess->start(program, arguments);

@

with the process's readyReadStandardOutput() connected to my custom slot:
@
connect(mTranscodingProcess,SIGNAL(readyReadStandardOutput()),this,SLOT(readyReadStandardOutput()));
@

In my slot I can dump the output in a text edit like this:
@
void Widget::readyReadStandardOutput()
{
mOutputString.append(mTranscodingProcess->readAllStandardOutput());
ui->textEdit->setText(mOutputString);

}
@

This works in my dummy test app,Will integrate it into our application and see the results.

Hope this helps somebody someday .

echostorm

Before anything can you first check if your not facing a known bug with ffmpeg that happens on windows 7 64bit:

run this command ffmpeg -list_devices true -f dshow -i dummy

Do you get any error like couldnt enumerate audio devices if all is fine just copy and paste your device name exactly as it appears if you do face that error I think your going to have to compile ffmpeg with a patch if its ready or wait till bug is fixed...

echostorm

How about using virtual-audio-capturer since your working on windows only it seems, you will need JRE I think installed infact you can also use it for screen capture too its on github:

https://github.com/rdp/screen-capture-recorder-to-video-windows-free

it helped me alot...

echostorm

in-case your facing format problems this works 100% for me:

@ffmpegarguments << "-f" << "dshow" << "-t" << "60" << "-i" << "video=UScreenCapture:audio=virtual-audio-capturer" << "D:\ffmpeg\bin\audio_video_out.mp4";@

MAKE SURE you install the virtual audio capturer...

with the virtual audio capture you wont have any unicode problems hope this helps you and good luck!

As for my project I will go back to X264 but have to worry about the license issue seems im gona have to go multi-platform and my project is an Augmented Reality project will probably release my code when its polished and my patent pending is active :)

musimbate

First of all ,thanks for the input,

I came across virtual-audio-capturer before and noticed it only captures audio playing what is playing in your speakers ,its a kind of "record what you hear thing" .May be will come in helpful for the project someday .

I was able to record audio by going into audio settings and changing the name of my audio device to sth plain english (wasn't aware one could do that-----silly me !) ,with this I could record like this:
@
arguments << "-f" << "dshow" << "-i" << "video=UScreenCapture" <<"-f"<< "dshow"<< "-i" <<"audio=MicroPhone (Realtek High Defini" <<"D:\ffmpeg\bin\out.mp4";
@

Notice I changed the name of my audio device from 麦克风 to MicroPhone.

I also made an error in one of my previous posts:

I retrieved the name of the audio device and passed it to ffmeg like this:
@
QString AudioDeviceName= QAudioDeviceInfo::availableDevices(QAudio::AudioInput).at(0).deviceName();
arguments << "-f" << "dshow" << "-i" << "video=UScreenCapture" <<"-f"<< "dshow"<< "-i" <<AudioDeviceName <<"D:\ffmpeg\bin\out.mp4";

@

I should have appended audio = to the audio device name like this:
@
QString AudioDeviceName= QAudioDeviceInfo::availableDevices(QAudio::AudioInput).at(0).deviceName();
QString audioString= "audio="+AudioDeviceName;
arguments << "-f" << "dshow" << "-i" << "video=UScreenCapture" <<"-f"<< "dshow"<< "-i" <<audioString <<"D:\ffmpeg\bin\out.mp4";
@

Sorry for that I had a long day .

Do you have a case when you need to tell ffmpeg to stop recording?How do you do that in your project .I am kind of cornered on that here.

Thanks and good luck on your project.

echostorm

Ok im going to make life easy for you 8) with regards to the code im giving you this is just a dummy app for testing so I would advise you to keep the console windows visible because you could do this:
@myProcess->setProcessChannelMode(QProcess::ForwardedChannels);@
This will make sure you see whats going on with ffmpeg at this stage we really want to know if there are any errors or just whats really going on right?

You will notice that if you send kill() or QProcess::terminate will not work cause in the end the file you get from ffmpeg is corrupt if you look carefully ffmpeg requires you to use q to end application and save proper file so a simple trick would be:
@myProcess->write("q");
myProcess->closeWriteChannel();@
you should call this to end the ffmpeg process once your done with the recording...
Dont forget you will also still have to end your main process too (the parent Qprocess) just remember that.

I hope this helps.

musimbate

Thanks a bunch,
It is recording and stopping nice and clean now .I edited the title of the thread to reflect what is discussed in here.

echostorm

if you also want to test this real fast you can use a timer:
@QElapsedTimer timer;
timer.start();
do {
myProcess->write("q");
} while (timer.elapsed() < 6000);@

and you should do this after you start the QProcess you will see in aprox. 6 seconds the ffmpeg will get the quit signal and stop recording it will save the file properly and will be working fine I guess I did a very crude and simple example but you get the idea now if you still need help I don't mind helping out but will be busy now till weekend wish you best of LUCK...

echostorm

Ok great just noticed it worked out for you after I already posted... glad it worked out :)