Qt Forum

    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Search
    • Unsolved

    Android: QAbstractVideoSurface not triggered by QMediaPlayer to grab frames from video

    Mobile and Embedded
    4
    14
    7745
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • A
      and.fo last edited by

      Hi there,

      I'm trying to grab videos from a video file using a QMediaPlayer in combination with a derivation of QAbstractVideoSurface, in order to get single frames in the present-method.

      It works well on windows, but on Android the methods in the derived QAbstractVideoSurface are not called. No error in output visible and no error message in the error-method. The isActive method returns false, so it seems that the methods in the the derived classes are not called.

      The start-methods and all other methods are not called. The same code works on windows. Frames are grabbed.

      Any ideas? is this an bug in QT 5.2 -> I'm using the beta version.

      1 Reply Last reply Reply Quote 0
      • Z
        z.emb last edited by

        I have the same problem with QCamera instead of QMediaPlayer. The Camera switches into ActiveState, even the flashlight works, but no calls to the derived VideoSurface, no start(), no present().

        Instead I get "E/Camera : Error 1001" as error.

        The exactly same code works on the Desktop. Using Qt5.2

        1 Reply Last reply Reply Quote 0
        • M
          MartynW last edited by

          I too have much the same problem with the released Qt 5.2 and QCamera (I extended the example http://qt-project.org/doc/qt-4.8/multimedia-videowidget.html). It works on Windows but not Android (Nexus 7).

          Weird thing is for Windows camera->isCaptureModeSupported(QCamera::CaptureViewfinder) returns false, but QAbstractVideoSurface::present() gets called, whereas it returns true for Android but present() does not get called as it should.

          For Android start() gets called and then present() gets called just the once and then nothing. From start() I see format.pixelFormat() is Format_BGR32 which is also the format in present() but the frame.startTime() is set to -1, which doesn't seem obviously right.

          Sounds like it may be a bug.

          1 Reply Last reply Reply Quote 0
          • M
            MartynW last edited by

            Yep, its been reported in https://bugreports.qt-project.org/browse/QTBUG-34888 and looks like we may get a fix in 5.2.1 according to https://bugreports.qt-project.org/browse/QTBUG-35416.

            Meanwhile, weirdly, simply referencing the frames handle() method within your present() method seems to kick the triggering process past the first frame.
            @
            qDebug() << frame.handleType();
            QVariant vHandle = frame.handle();
            qDebug() << "Frame: " << frame.startTime() << " format: " << frame.pixelFormat()
            << " valid?: " << frame.isValid() << " size " << frame.width()
            << " x " << frame.height();
            @

            gives:
            @
            D/Qt ( 7020): ConsumerVideoSurface.cpp:145 (virtual bool ConsumerVideoSurface::present(const QVideoFrame&)): UserHandle(1001)
            D/Qt ( 7020): ConsumerVideoSurface.cpp:165 (virtual bool ConsumerVideoSurface::present(const QVideoFrame&)): Frame: -1 format: Format_BGR32 valid?: true size 960 x 720
            @

            1 Reply Last reply Reply Quote 0
            • J
              johnc last edited by

              I am getting a similar issue.

              Putting in the frame.handle() call in my present() method kick started it, but nothing is painting.

              I think I must be hitting the raw frame issue as outlined in one of the bug reports.

              In the paint function it fails on the frame.map() function. It seems that on Android it is unable to copy the raw frame data over to accessible memory.

              Does anyone know how to get around this issue? We need raw access so we can paint to a specific control, and also run a Qr reader over the frames.

              The code runs fine on iOS so the error is definitely just Android specific.

              1 Reply Last reply Reply Quote 0
              • M
                MartynW last edited by

                Supposedly you will be able to get the frame using QVideoProbe in 5.2.1 to be released beginning of Feb time (?) or the proper fix in 5.3 end of April time. Meantime it doesn't look possible (see http://qt-project.org/forums/viewthread/36973/) . Unless someone knows differently!

                1 Reply Last reply Reply Quote 0
                • M
                  MartynW last edited by

                  Yes, QVideoProbe works fine in 5.2.1 (don't forget to call frame.map()) and returns image frames in NV21 format, altho the orientation does not change and is at 90 degrees when in portrait mode. It gives consecutive frames even when you don't call frame.handle() in the VideoSurface (which is best as the callbacks don't return anything useful).

                  1 Reply Last reply Reply Quote 0
                  • A
                    and.fo last edited by

                    Grabbing frames is still not working for me on Android.

                    Android version is 4.4.3. Qt version 5.2.1 as Qt 5.3. crashes for another reason.

                    Either with a custom QAbstractVideoSurface or the QVideoProbe there is no callback triggered to "present" or "videoFrameProbed".

                    In case of QVideoProbe the following function returns false on Android and Windows7.

                    @
                    probe.setSource(&m_player)}
                    @

                    How did you manage it? Is the frame grabbing working on Android?

                    1 Reply Last reply Reply Quote 0
                    • M
                      MartynW last edited by

                      Yes, it works fine for Windows an Android. I vaguely remember setSource returning false at one time, I think I just got the camera name wrong. I show a snippet of what I have below in the hope it may help.

                      BTW, I think the 'proper' fix never made it into 5.3.

                      @ QVideoProbe *m_pProbe;

                      // Search for system camera devices: We pick the first if any specified one is not found
                      foreach (const QByteArray &deviceName, QCamera::availableDevices())
                      {
                          QString description = m_pCamera->deviceDescription(deviceName);
                          if (cameraDevice.isEmpty())
                              cameraDevice = deviceName;
                      
                          if (description.compare(strCameraName, Qt::CaseInsensitive) == 0)
                              cameraDevice = deviceName;
                      
                          qDebug() << "Camera: " << description; 
                      }
                      
                      // Example device on Windows: "@device:pnp:\\?\usb#vid_0c45&pid_6409&mi_00#7&373d688d&0&0000#{65e8773d-8f56-11d0-a3b9-00a0c9223196}\global"
                      // Example device on Nexus 7 2012: "front", description "Front-facing camera"
                      
                      if (cameraDevice.isEmpty())
                          m_pCamera = new QCamera;
                      else
                          m_pCamera = new QCamera(cameraDevice);
                      
                      m_pConsumerVideoSurface = new ConsumerVideoSurface(this);
                      m_pCamera->setViewfinder(m_pConsumerVideoSurface);
                      connect(m_pConsumerVideoSurface, SIGNAL(signalOnFrame(QImage)), this , SLOT(slotOnFrame(QImage)), Qt::DirectConnection);
                      if (bIsVideoProbeRequired)
                      {
                          m_pProbe = new QVideoProbe(this);
                          m_pConsumerVideoSurface->SuppressVideo(true);
                      }
                      
                      
                      m_pProbe = new QVideoProbe(this);
                      if (m_pProbe != NULL)
                      {
                          m_pProbe->setSource(m_pCamera); // Returns true, hopefully.
                          connect(m_pProbe, SIGNAL(videoFrameProbed(QVideoFrame)), this, SLOT(slotOnProbeFrame(QVideoFrame)), Qt::DirectConnection);
                      }
                      m_pCamera->start();
                      

                      @

                      1 Reply Last reply Reply Quote 0
                      • A
                        and.fo last edited by

                        Thanks a lot. I will have a try :-)

                        But in may application I'm not grabbing frames from the camera but from the mediaplayer. I'm working on frames from a video playback file.

                        Maybe this is not working on Android? Do you have any experience with that?

                        1 Reply Last reply Reply Quote 0
                        • M
                          MartynW last edited by

                          Sorry, no, I was only interested in the camera, but I thought QVideoProbe was meant to be working generally so I would expect you to be ok.

                          1 Reply Last reply Reply Quote 0
                          • A
                            and.fo last edited by

                            Thanks again!

                            I found this post, mentioning QVideoProbe is working only for the camera on Android, not for the MediaPlayer.

                            http://lists.qt-project.org/pipermail/interest/2014-February/011125.html

                            1 Reply Last reply Reply Quote 0
                            • M
                              MartynW last edited by

                              It seems to be a bit of a work in progress with changes proposed then delayed - see "https://bugreports.qt-project.org/browse/QTBUG-35416":https://bugreports.qt-project.org/browse/QTBUG-35416.

                              1 Reply Last reply Reply Quote 0
                              • J
                                johnc last edited by johnc

                                Is there any further update to this or solution?

                                The bug reports all seem to say to use QVideoProbe instead, but I can't get that to work on Android ( 4.1.2 ) and QT 5.4.
                                If I leave out setViewFinder I get QCamera::InvalidRequestError

                                Putting setViewFinder in removes all errors, the camera changes state from Loading, to Loaded, to Active.
                                But then after this no frames are captured.

                                I have seen a number of examples like this one http://www.programering.com/a/MTN3IjMwATQ.html
                                But all seem to have the same result, no frames.

                                1 Reply Last reply Reply Quote 0
                                • First post
                                  Last post