Important: Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

pyqt5 draw above videowidget?



  • Hi guys:
    I want to Know if it's possible I draw a block above a video, like two layers, the drawing layer is above the video layer.
    Thanks a lot!



  • Hey guys:
    I solved my problem.
    The solution is send a signal to change the item position in callback function of eyetracker rather than directly change it in call callback function of eyetracker which is in secondary thread.
    Details in this Link
    cheers!



  • What did your google web search tell you about it?



  • @Kent-Dorfman
    Hi kent:
    sorry for late reply.
    I do googled it before, and I find some an example.
    What I want to do is visualize the gaze data(where the user is looking at) from a eye tracker(60hz) while watching a video.
    it works, but the video always get stuck sometime (the audio is normal, the the frames stuck, both video content and ellipse), can you help me out here?
    Thank you a lot!

    import os
    import time
    
    from PyQt5 import QtCore, QtGui, QtWidgets, QtMultimedia, QtMultimediaWidgets
    import tobii_research as tr
    import numpy as np
    """
    code from https://stackoverflow.com/questions/53899740/how-to-draw-qtgraphicsview-on-top-of-qvideowidget-with-transparency
    """
    class Widget(QtWidgets.QWidget):
    
        def __init__(self, parent=None):
    
            super(Widget, self).__init__(parent)
    
            #first window,just have a single button for play the video
            self.resize(256, 256)
            self.btn_play = QtWidgets.QPushButton(self)
            self.btn_play.setGeometry(QtCore.QRect(100, 100, 28, 28))
            self.btn_play.setObjectName("btn_open")
            self.btn_play.setText("Play")
            self.btn_play.clicked.connect(self.Play_video)#click to play video
            #
    
            self._scene = QtWidgets.QGraphicsScene(self)
            self._gv = QtWidgets.QGraphicsView(self._scene)
            #construct a videoitem for showing the video
            self._videoitem = QtMultimediaWidgets.QGraphicsVideoItem()
            #add it into the scene
            self._scene.addItem(self._videoitem)
    
            # assign _ellipse_item is the gaze data, and embed it into videoitem,so it can show above the video.
            self._ellipse_item = QtWidgets.QGraphicsEllipseItem(QtCore.QRectF(0, 0, 40, 40), self._videoitem)
            self._ellipse_item.setBrush(QtGui.QBrush(QtCore.Qt.black))
            self._ellipse_item.setPen(QtGui.QPen(QtCore.Qt.red))
            self._scene.addItem(self._ellipse_item)
            self._gv.fitInView(self._videoitem)
    
            self._player = QtMultimedia.QMediaPlayer(self, QtMultimedia.QMediaPlayer.VideoSurface)
            self._player.setVideoOutput(self._videoitem)
            file = os.path.join(os.path.dirname(__file__), "video.mp4")#video.mp4 is under the same dirctory
            self._player.setMedia(QtMultimedia.QMediaContent(QtCore.QUrl.fromLocalFile(file)))
    
            #get eye tracker
            self.eyetrackers = tr.find_all_eyetrackers()
            self.my_eyetracker = self.eyetrackers[0]
    
        def gaze_data_callback(self, gaze_data_):
    
            #for now, I don't know the coordinate system,just randomly assign the gaze data to test the functionality
            self._ellipse_item.setPos(float(np.random.choice(range(300, 500))), float(np.random.choice(range(400, 500))))
            print("time.time()::{}".format(time.time()))
    
    
        def Play_video(self):
            self.my_eyetracker.subscribe_to(tr.EYETRACKER_GAZE_DATA, self.gaze_data_callback, as_dictionary=True)
            size = QtCore.QSizeF(1920.0, 1080.0)#I hope it can fullscreen the video
            self._videoitem.setSize(size)
            self._gv.showFullScreen()
            self._player.play()
    
    if __name__ == '__main__':
        import sys
        app = QtWidgets.QApplication(sys.argv)
        w = Widget()
        w.show()
        sys.exit(app.exec_())
    
    


  • @fsluckyM said in pyqt5 draw above videowidget?:

    it works, but the video always get stuck sometime

    Why are you adding a video to graphics scene, and not playing the video into a widget?
    You then modify that graphics scene with you gaze handler?
    How does it know layering if they both go into the same graphics scene?

    If it were me:

    1. I'd use an overlay layout manager where the video is on the bottom layer in a basic widget, and the gaze drawing is done in an overlayed widget in the same layout manager.
    2. I'd do this in C++ and not in python. Your ability to optimize for speed increases if you use C++.


  • @Kent-Dorfman
    Hi Kent:
    I have not much qt experience, is there a overlay layout exist?
    I only know QBoxLayout,QStackedLayout.

    Thanks a lot



  • Hi Kent:
    I simplified my task, every time eye tracker return data, I update the position of ellipse. when I run this code by cmd, it warn me this "QObject::startTimer: Timers cannot be started from another thread". The whole test process is like this video, any hint?
    Thank you!

    import os
    import time
    
    from PyQt5 import QtCore, QtGui, QtWidgets, QtMultimedia, QtMultimediaWidgets
    import tobii_research as tr
    import config
    import numpy as np
    from PyQt5.QtWidgets import *
    from PyQt5.QtGui import *
    from PyQt5.QtCore import *
    
    class Widget(QtWidgets.QWidget):
    
        def __init__(self, parent=None):
    
            super(Widget, self).__init__(parent)
    
            self.btn_open = QtWidgets.QPushButton(self)
            self.btn_open.setGeometry(QtCore.QRect(100, 100, 100, 100))
            self.btn_open.setObjectName("btn_open")
            self.btn_open.setText("Play")
            self.btn_open.clicked.connect(self.Play_video_2)
    
            self._scene = QtWidgets.QGraphicsScene()
            self.MainWindow=QMainWindow()
            self._gv = QtWidgets.QGraphicsView(self._scene,self.MainWindow)
            self.MainWindow.setCentralWidget(self._gv)
    
            self._ellipse_item=QtWidgets.QGraphicsEllipseItem(QtCore.QRectF(0, 0, 40, 40))
            self._ellipse_item.setBrush(QtGui.QBrush(QtCore.Qt.black))
            self._ellipse_item.setPen(QtGui.QPen(QtCore.Qt.red))
            self._scene.addItem(self._ellipse_item)
    
            self.eyetrackers = tr.find_all_eyetrackers()
            self.my_eyetracker = self.eyetrackers[0]
            self.flag=1
    
        def gaze_data_callback(self, gaze_data_):
               self._ellipse_item.setPos(self.flag,self.flag)
               self.flag+=1
    
        def Play_video_2(self):
            self.my_eyetracker.subscribe_to(tr.EYETRACKER_GAZE_DATA, self.gaze_data_callback, as_dictionary=True)
            #self.MainWindow.showFullScreen()
            self.MainWindow.resize(512,512)
            self.MainWindow.show()
    
    if __name__ == '__main__':
        import sys
        app = QtWidgets.QApplication(sys.argv)
    
        w = Widget()
        w.show()
        sys.exit(app.exec_())
        #w.my_eyetracker.unsubscribe_from(tr.EYETRACKER_GAZE_DATA, w.gaze_data_callback)
    
    

  • Banned

    Okay I have not fixed any issues this might have since you did not include the only thing besides PyQt5 that you import that being this tobii_research and subsequently I could not run this to debug it. That being said I did majorly cleaned it up as clean code is always best to work from.

    # Qt Imports should come first
    from PyQt5.QtGui import *
    from PyQt5.QtCore import *
    from PyQt5.QtWidgets import *
    from PyQt5.QtMultimedia import *
    from PyQt5.QtMultimediaWidgets import *
    
    # All other imports go here
    import tobii_research as tr
    
    class AppWidget(QWidget):
        def __init__(self):
          # Do not use Super( ) unless you are aware of its 3 main issues
          # since 99% of the time you will not even need it 
            QWidget.__init__(self)
    
            self.btnOpen = QPushButton(self)
            self.btnOpen.setGeometry(QRect(100, 100, 100, 100))
            self.btnOpen.setText('Play')
            self.btnOpen.clicked.connect(self.PlayVideo)
    
            self.GrphScene = QGraphicsScene()
          # Normally you do not define a QMainWindow within a QWidget
          # it is typically done the other way around so why are you??
            self.MainWindow = QMainWindow()
            self.GrphView = QGraphicsView(self.GrphScene, self.MainWindow)
            self.MainWindow.setCentralWidget(self.GrphView)
    
            self.GrphEllipsItem = QGraphicsEllipseItem(QRectF(0, 0, 40, 40))
            self.GrphEllipsItem.setBrush(QBrush(Qt.black))
            self.GrphEllipsItem.setPen(QPen(Qt.red))
            self.GrphScene.addItem(self.GrphEllipsItem)
    
            self.EyeTrackers = tr.find_all_eyetrackers()
            self.MyEyeTracker = self.EyeTrackers[0]
            self.Flag = 1
    
        def GazeDataCallback(self, gaze_data_):
            self.GrphEllipsItem.setPos(self.Flag, self.Flag)
            self.Flag += 1
    
        def PlayVideo(self):
            self.MyEyeTracker.subscribe_to(tr.EYETRACKER_GAZE_DATA, self.GazeDataCallback, AsDictionary=True)
            #self.MainWindow.showFullScreen()
            self.MainWindow.resize(512,512)
            self.MainWindow.show()
        
    if __name__ == '__main__':
        MainEvntHndlr = QApplication([])
    
        MainApp = AppWidget()
        MainApp.show()
    
        MainEvntHndlr.exec()
    
      # If anyone wants more extensive free help I run an online lab-like classroom-like 
      # message server feel free and drop by you will not be able to post until I clear 
      # you as a student as this prevents spammers so if interested here is the invite
      # https://discord.gg/3D8huKC
    


  • @Denni-0
    Hi Denni:
    I am sorry I didn't post in very clean way, I am in quite hurry.
    I am using tobii eye tracker for get gaze data. the
    GazeDataCallback is the callback function from tobii_research package, it will be called every time when eyetracker return a data, the frequency of tracker is 60hz, which mean this function will be called about every 16.7milliseconds (may slightly change because of the lantency). It is controled by subscribe_to() and unsubscribe_from() function.

    Thank you for you help.(taking a nap now, 4.am here. sorry for late reply in advance)


  • Banned

    That is fine but without being able to have access to this tobii_research library I cannot run your code as written, however, on the flip-side if I ran the code without this tobii_research and put in some kind of widget filler -- I do not believe it would display the Open button as you never place that within the QMainWindow which is the only thing that appears you .show()

    Also as denoted within the somewhat cleaned up code -- one normally does not put a QMainWindow within a QWidget wrapper and if my observations are correct you ought not be doing it that way either and you should have done something like the following (see below). As for the open button all-things-considered I would have probably made that a MenuToolBar item instead as then it would not affect your viewing pane

    def CenterPanel(QWidget):
        def __init__(self):
            QWidget.__init__(self)
    
    # This would then contain your btnOpen, GraphScene, GraphView, GraphEllipseItem and EyeTracker
    
    def MainWindow(QMainWindow):
        def __init__(self):
            QMainWindow.__init__(self)
    
            CenterPane = CenterPanel(self)
            self.setCentralWidget(CenterPane)
    


  • Hey guys:
    I solved my problem.
    The solution is send a signal to change the item position in callback function of eyetracker rather than directly change it in call callback function of eyetracker which is in secondary thread.
    Details in this Link
    cheers!


  • Banned

    Okay but be careful that you do not use that code provided verbatim eyllanesc has stated that he does not care that what he posts is poor quality code (which that example he posted is poor) and while he can get things to work his solutions are not always the best nor are his explanations always accurate.

    Also be aware that you cannot put anything based on a QWidget within a QThread nor can you put certain QGui objects within a QThread (although they are not fully explicit about which ones) and lastly the current documentation for how to use a QThread I believe is still wrong as you never sub-class a QThread



  • Hi Denni:
    Thank you for you reply.
    Wow, I didn't realize there are many potential problem out there. I am beginner in Qt, Can you point out where should I modify my code effectively and reasonably?
    I am not using the code from eyllanesc verbatim, but I do use the idea he told me — "send signal of change the QgraphicsItem position in callback function which is in eyetracker built-in thread (I don't know term of "eyetracker built-in thread" is correct or not, but I think eyetracker is a hardware which work in it's own way)"
    Thank you!


  • Banned

    @fsluckyM yea there are a lot of undocumented and hidden documented items about using python and python-Qt that many wanna-be-programmers are simply unaware of -- or worse are aware of but just do not care such as with eyllanesc. That is why I conduct a free python-qt classroom for folks that truly want to learn quality programming concepts. I have sent you chat in case you might be interested in that otherwise ask more specific questions and such and I will do my best to answer them or point you in the direction to get an answer


Log in to reply