Skip to content
  • Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Groups
  • Search
  • Get Qt Extensions
  • Unsolved
Collapse
Brand Logo
  1. Home
  2. Qt Development
  3. Language Bindings
  4. QByteArray to string?
Forum Updated to NodeBB v4.3 + New Features

QByteArray to string?

Scheduled Pinned Locked Moved Solved Language Bindings
python3pyqt5
29 Posts 4 Posters 33.9k Views 3 Watching
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • S SGaist
    15 Nov 2017, 20:02

    Hi,

    IIRC, you can also use QString, something like: string = QString(qba).

    Not with PyQt 5 as it seems

    [edit: updated after getting new informations SGaist]

    J Offline
    J Offline
    JonB
    wrote on 16 Nov 2017, 10:52 last edited by JonB
    #6

    @SGaist said in Python3/PyQt5.x QByteArray to string?:

    Hi,

    IIRC, you can also use QString, something like: string = QString(qba).

    Yeah, you see, I checked and I don't see how you can do this? Where is the QString to use from PyQt, that's exactly the point?? from PyQt5.QtCore import QString doesn't find anything... I'd love to be able to do it that way, but it's back to my understanding that PyQt deliberately hides QString and you have to use Python str instead?

    Like see the depressing answer to https://riverbankcomputing.com/pipermail/pyqt/2015-September/036470.html:

    is there some other way to get a hold of a real QString with PyQt5?

    No.

    Elsewhere I have come across from PyQt4.QtCore import QString. So I'm thinking PyQt5 actually removed that ability... :(

    https://riverbankcomputing.com/pipermail/pyqt/2014-January/033561.html informs us the reason is:

    It's more Pythonic to use Python strings than Qt strings.

    Great! :(

    1 Reply Last reply
    -1
    • S Offline
      S Offline
      SGaist
      Lifetime Qt Champion
      wrote on 16 Nov 2017, 11:29 last edited by
      #7

      My bad, I based my answer on the gotchas article and the QString constructor list. It would be worth mentioning in that article that QString has been remove from PyQt5.

      By the way QStringList is also in this case.

      All in all, your trick seems to be the best solution. You can also post in the Riverbank mailing to see if they have any suggestions as best practice to convert QByteArray to string.

      Interested in AI ? www.idiap.ch
      Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

      J 1 Reply Last reply 16 Nov 2017, 11:46
      1
      • S SGaist
        16 Nov 2017, 11:29

        My bad, I based my answer on the gotchas article and the QString constructor list. It would be worth mentioning in that article that QString has been remove from PyQt5.

        By the way QStringList is also in this case.

        All in all, your trick seems to be the best solution. You can also post in the Riverbank mailing to see if they have any suggestions as best practice to convert QByteArray to string.

        J Offline
        J Offline
        JonB
        wrote on 16 Nov 2017, 11:46 last edited by JonB
        #8

        @SGaist
        Yeah, that's precisely the conclusion I came to too! Hence the question for confirmation it really is that obscure :)

        Riverbank have obviously already made clear what they want to do, having explicitly apparently changed it from PyQt4 to PyQt5, so there seems little point in asking them. Unless I am mistaken, they won't much actively respond to a PyQt query from me?

        1 Reply Last reply
        0
        • S Offline
          S Offline
          SGaist
          Lifetime Qt Champion
          wrote on 16 Nov 2017, 13:32 last edited by
          #9

          Why wouldn't they ? You are not asking them to bring back QString, just what they recommend as best practice to convert a QByteArray to a python string following your example.

          Interested in AI ? www.idiap.ch
          Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

          J 1 Reply Last reply 16 Nov 2017, 15:14
          0
          • S SGaist
            16 Nov 2017, 13:32

            Why wouldn't they ? You are not asking them to bring back QString, just what they recommend as best practice to convert a QByteArray to a python string following your example.

            J Offline
            J Offline
            JonB
            wrote on 16 Nov 2017, 15:14 last edited by
            #10

            @SGaist
            I meant, I thought I have tried posting to PyQt forum before, and just no replies. Maybe I'm mistaken though...

            1 Reply Last reply
            -1
            • J Offline
              J Offline
              jazzycamel
              wrote on 16 Nov 2017, 16:18 last edited by
              #11

              The standard python3 way to handle this would be:

              print(qba.decode('utf8'))
              

              This is actually to do with changes between Python versions 2 and 3. In Python 2 str could be used for both text and binary data and was considered 'brittle' by the core devs., so in Python 3 it was decided that str (incl. unicode) would be used only for text and bytes would be used for binary (see more here). Therefore, in PyQt5, when a Qt type containing binary data (QByteArray, the clue is in the name) is converted to a native type, bytes is used rather than str giving the developer the choice of which encoding to use if it is string data.

              Also, as an active member of the PyQt mailing list, I can say it is normally pretty responsive and helpful so, in future, please think of giving us a second chance :).

              Hope this helps :)

              For the avoidance of doubt:

              1. All my code samples (C++ or Python) are tested before posting
              2. As of 23/03/20, my Python code is formatted to PEP-8 standards using black from the PSF (https://github.com/psf/black)
              J 1 Reply Last reply 16 Nov 2017, 16:51
              3
              • J jazzycamel
                16 Nov 2017, 16:18

                The standard python3 way to handle this would be:

                print(qba.decode('utf8'))
                

                This is actually to do with changes between Python versions 2 and 3. In Python 2 str could be used for both text and binary data and was considered 'brittle' by the core devs., so in Python 3 it was decided that str (incl. unicode) would be used only for text and bytes would be used for binary (see more here). Therefore, in PyQt5, when a Qt type containing binary data (QByteArray, the clue is in the name) is converted to a native type, bytes is used rather than str giving the developer the choice of which encoding to use if it is string data.

                Also, as an active member of the PyQt mailing list, I can say it is normally pretty responsive and helpful so, in future, please think of giving us a second chance :).

                Hope this helps :)

                J Offline
                J Offline
                JonB
                wrote on 16 Nov 2017, 16:51 last edited by JonB
                #12

                @jazzycamel
                Thank you, but I'm sorry, I don't see how. The whole point is the PyQt 4 to 5 changes document (or is it Python 2 to 3, I can't recall) is that it says QByteArray.decode() method was removed? It's not there if I try to use it. Have you tried your suggestion with PyQt5/Python3?

                1 Reply Last reply
                -1
                • J Offline
                  J Offline
                  jazzycamel
                  wrote on 16 Nov 2017, 16:57 last edited by
                  #13

                  @JNBarchan
                  Apologies, that should have been

                  print(qba.data().decode('utf8'))
                  

                  (That'll teach me to read things properly...!)

                  For the avoidance of doubt:

                  1. All my code samples (C++ or Python) are tested before posting
                  2. As of 23/03/20, my Python code is formatted to PEP-8 standards using black from the PSF (https://github.com/psf/black)
                  J 1 Reply Last reply 16 Nov 2017, 17:24
                  3
                  • J jazzycamel
                    16 Nov 2017, 16:57

                    @JNBarchan
                    Apologies, that should have been

                    print(qba.data().decode('utf8'))
                    

                    (That'll teach me to read things properly...!)

                    J Offline
                    J Offline
                    JonB
                    wrote on 16 Nov 2017, 17:24 last edited by JonB
                    #14

                    @jazzycamel
                    OK, that does work, thank you! Now then, may I ask:

                    1. QByteArray.data() returns bytes. Where was I supposed to come across documentation for bytes.decode() (e.g. in PyQt?)? [EDIT: I'm a newbie to both Python & Qt. I spend my time looking around the Qt documentation to do this stuff. I'm beginning to guess this is a Python issue, not Qt, but it's a lot to take in!]

                    2. (Because of #1) I don't know the arguments to decode(). I have used my utf-8 and your utf8 and as far as I can see both work the same. Which is "right"/"preferable"?

                    3. Can you comment (briefly :) ) on why decode() vs str(encoding=...) is preferable/nicer/more Pythonic?

                    1 Reply Last reply
                    0
                    • J Offline
                      J Offline
                      jazzycamel
                      wrote on 16 Nov 2017, 17:40 last edited by jazzycamel
                      #15
                      1. bytes is a python standard type and is fully documented in the python docs, the particular information you require re. bytes.decode() can be found here.
                      2. In the documentation linked above you will find a link to Standard Encodings (also part of the python docs) which will tell you all you ever wanted to know about encodings (and more!). utf-8 and utf8 are simply aliases of one another, both are perfectly acceptable (as detailed/listed in the docs) as are U8 and UTF (I think...!).
                      3. Semantics, but Python is considered to be primarily an object-oriented language and therefore you should use an objects own methods (yes, bytes and str are objects as are all 'types' in Python) rather than a function. In fact, the str() function just invokes an objects own __str__() method as that defines how the object should be represented as a string (true for all types).

                      For the avoidance of doubt:

                      1. All my code samples (C++ or Python) are tested before posting
                      2. As of 23/03/20, my Python code is formatted to PEP-8 standards using black from the PSF (https://github.com/psf/black)
                      J 1 Reply Last reply 16 Nov 2017, 17:58
                      3
                      • J jazzycamel
                        16 Nov 2017, 17:40
                        1. bytes is a python standard type and is fully documented in the python docs, the particular information you require re. bytes.decode() can be found here.
                        2. In the documentation linked above you will find a link to Standard Encodings (also part of the python docs) which will tell you all you ever wanted to know about encodings (and more!). utf-8 and utf8 are simply aliases of one another, both are perfectly acceptable (as detailed/listed in the docs) as are U8 and UTF (I think...!).
                        3. Semantics, but Python is considered to be primarily an object-oriented language and therefore you should use an objects own methods (yes, bytes and str are objects as are all 'types' in Python) rather than a function. In fact, the str() function just invokes an objects own __str__() method as that defines how the object should be represented as a string (true for all types).
                        J Offline
                        J Offline
                        JonB
                        wrote on 16 Nov 2017, 17:58 last edited by
                        #16

                        @jazzycamel
                        Yep, all good stuff, makes sense, thank you very much!

                        As I edited against #1, I now realise that certain things from Qt via PyQt require me to look at Python documentation rather than Qt.

                        Since you happen to be here, and are so kind, would you care to comment on one issue which was raised in posts above. In PyQt 4, apparently, you could go s = QString() if you wanted to. Is it indeed correct that in PyQt 5 there really is no such thing as QString anywhere, and you have to deal in Python types like str in every situation? (Doubtless same applies to, say, QByteArray type and bytes, and for other such Qt types where you have decided only to allow the Python type.)

                        Finally, don't suppose you could make Python be just like C# instead for me, then I'd be much happier? ;-)

                        J 1 Reply Last reply 16 Nov 2017, 20:02
                        0
                        • S Offline
                          S Offline
                          SGaist
                          Lifetime Qt Champion
                          wrote on 16 Nov 2017, 19:52 last edited by
                          #17

                          @jazzycamel long time no see ! Thanks for the thorough explanation :-)
                          Parts of it would be a welcome addition to the PyQt5 documentation.

                          Interested in AI ? www.idiap.ch
                          Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

                          1 Reply Last reply
                          0
                          • J JonB
                            16 Nov 2017, 17:58

                            @jazzycamel
                            Yep, all good stuff, makes sense, thank you very much!

                            As I edited against #1, I now realise that certain things from Qt via PyQt require me to look at Python documentation rather than Qt.

                            Since you happen to be here, and are so kind, would you care to comment on one issue which was raised in posts above. In PyQt 4, apparently, you could go s = QString() if you wanted to. Is it indeed correct that in PyQt 5 there really is no such thing as QString anywhere, and you have to deal in Python types like str in every situation? (Doubtless same applies to, say, QByteArray type and bytes, and for other such Qt types where you have decided only to allow the Python type.)

                            Finally, don't suppose you could make Python be just like C# instead for me, then I'd be much happier? ;-)

                            J Offline
                            J Offline
                            jazzycamel
                            wrote on 16 Nov 2017, 20:02 last edited by
                            #18

                            @JNBarchan
                            There is indeed no such thing as QString() in PyQt5. It shouldn't be necessary as the library takes care of type marshalling between the Python and Qt (C++) types. In fact, while there is a QVariant(), its generally not necessary to use it for the same reason. QByteArray() does exist also, but I would steer clear of it if possible and let PyQt5 deal with via bytes().

                            No, I will never (and no one else should!) ever make Python like C#!! :)

                            For the avoidance of doubt:

                            1. All my code samples (C++ or Python) are tested before posting
                            2. As of 23/03/20, my Python code is formatted to PEP-8 standards using black from the PSF (https://github.com/psf/black)
                            J 1 Reply Last reply 30 Nov 2017, 09:47
                            3
                            • J jazzycamel
                              16 Nov 2017, 20:02

                              @JNBarchan
                              There is indeed no such thing as QString() in PyQt5. It shouldn't be necessary as the library takes care of type marshalling between the Python and Qt (C++) types. In fact, while there is a QVariant(), its generally not necessary to use it for the same reason. QByteArray() does exist also, but I would steer clear of it if possible and let PyQt5 deal with via bytes().

                              No, I will never (and no one else should!) ever make Python like C#!! :)

                              J Offline
                              J Offline
                              JonB
                              wrote on 30 Nov 2017, 09:47 last edited by
                              #19

                              @jazzycamel , or anyone else

                              Having implemented qba.data().decode('utf8') as directed, I have now come across a situation where the QByteArray data returned by QProcess.readAllStandardOutput() from an OS command run under Windows causes the Python/PyQt code to generate a UnicodeDecodeError error, as detailed in my post https://forum.qt.io/topic/85493/unicodedecodeerror-with-output-from-windows-os-command

                              This makes it impossible to convert the data, blocking the whole behaviour of my usage.

                              My belief is that this would not be happening at all from C++ where I would simply use whatever methods of QByteArray/QString or the language. The problem is precisely is that I am being forced to use a "Python/PyQt" way of doing this, causing the error in Python/PyQt only, which is exactly why I didn't want to have to do that but cannot get access to the necessary types/methods of Qt from PyQt...?

                              1 Reply Last reply
                              -1
                              • S Offline
                                S Offline
                                SGaist
                                Lifetime Qt Champion
                                wrote on 30 Nov 2017, 14:16 last edited by
                                #20

                                Can you show the code you use ?

                                Interested in AI ? www.idiap.ch
                                Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

                                J 1 Reply Last reply 30 Nov 2017, 15:59
                                0
                                • S SGaist
                                  30 Nov 2017, 14:16

                                  Can you show the code you use ?

                                  J Offline
                                  J Offline
                                  JonB
                                  wrote on 30 Nov 2017, 15:59 last edited by JonB
                                  #21

                                  @SGaist
                                  I promise you all you'll see is a QByteArray being returned with the sub-process's output, and I'm trying to convert that to a QString to put into a QTextEdit. That's all the question is. And I get a UnicodeDecodeError, probably when robocopy echoes the name of a file which has that 0x9c character in it via PyQt's decode():

                                  can't decode byte 0x9c in position 32: invalid start byte
                                  

                                  So presumably all you have to do is create a QByteArray, put a 0x9c in its first byte, and try qba.data().decode('utf8'). That's what this thread is about.

                                  This whole issue where I'm discussing the code is in https://forum.qt.io/topic/85493/unicodedecodeerror-with-output-from-windows-os-command. If you'd be kind enough to look at that, I think that's a more appropriate place to discuss the code than here? If you still want more code there, let me know, and I'll supply.

                                  1 Reply Last reply
                                  -1
                                  • S Offline
                                    S Offline
                                    SGaist
                                    Lifetime Qt Champion
                                    wrote on 30 Nov 2017, 20:01 last edited by
                                    #22

                                    I don't have a Windows machine at hand. Doing this on macOS yields correct results

                                    from PyQt5.QtCore import QByteArray
                                    ba = QByteArray()
                                    ba.append(u"\u009C")
                                    PyQt5.QtCore.QByteArray(b'\xc2\x9c')
                                    ba.data().decode('utf-8')
                                    '\x9c'
                                    ba.data().decode('utf-16')
                                    '鳂'
                                    

                                    Interested in AI ? www.idiap.ch
                                    Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

                                    J 1 Reply Last reply 30 Nov 2017, 20:19
                                    0
                                    • S SGaist
                                      30 Nov 2017, 20:01

                                      I don't have a Windows machine at hand. Doing this on macOS yields correct results

                                      from PyQt5.QtCore import QByteArray
                                      ba = QByteArray()
                                      ba.append(u"\u009C")
                                      PyQt5.QtCore.QByteArray(b'\xc2\x9c')
                                      ba.data().decode('utf-8')
                                      '\x9c'
                                      ba.data().decode('utf-16')
                                      '鳂'
                                      
                                      J Offline
                                      J Offline
                                      JonB
                                      wrote on 30 Nov 2017, 20:19 last edited by JonB
                                      #23

                                      @SGaist
                                      I'm afraid I don't believe that relates to the situation.

                                      I now have information from the client:

                                      The exception occurs (only) when a filename robocopy encounters --- robocopy is echoing filenames as it goes --- contains the £ (UK pound sterling) character (I am in the UK, you may not be). In that situation, ba.data().decode('utf-8') (where ba is the QByteArray from QProcess.readAllStandardOutput()) results in:

                                      Unhandled Exception:
                                      
                                      'utf-8' codec can't decode byte 0x9c in position 32: invalid start byte
                                      
                                      <class 'UnicodeDecodeError'>
                                      File "C:\HJinn\widgets\messageboxes.py", line 289, in processReadyReadStandardOutput
                                      output = output.data().decode('utf-8')
                                      

                                      Now, armed with that information:

                                      • In a Command Prompt I type in: echo £ > file
                                      • I dump the file and I see: 9C 20 0D 0A
                                      • So the £ character is single byte with value 0x9C
                                      1 Reply Last reply
                                      -1
                                      • S Offline
                                        S Offline
                                        SGaist
                                        Lifetime Qt Champion
                                        wrote on 30 Nov 2017, 20:24 last edited by
                                        #24

                                        What do you get if you use unicode_escape in place of utf-8 ?

                                        Interested in AI ? www.idiap.ch
                                        Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

                                        J 2 Replies Last reply 30 Nov 2017, 20:30
                                        0
                                        • S SGaist
                                          30 Nov 2017, 20:24

                                          What do you get if you use unicode_escape in place of utf-8 ?

                                          J Offline
                                          J Offline
                                          JonB
                                          wrote on 30 Nov 2017, 20:30 last edited by
                                          #25

                                          @SGaist
                                          I don't know, because I don't have access to the code right now, but I will tomorrow.

                                          Thank you, your suggestion is much more like what I have been looking for. We are now discussing the argument to decode():

                                          • I believe utf-8 is definitely right for Linux, where I develop.
                                          • I'm beginning to learn (whether I like it or not) that it is not for Windows.
                                          • Under Windows utf-8 does work 99% of the time, but not always, and now I know not for the £ character.
                                          • I believe that either latin-1 or windows_1252 may be able to handle this correctly.
                                          • I will also try your unicode_escape if you think it's worthwhile.
                                          1 Reply Last reply
                                          -1

                                          • Login

                                          • Login or register to search.
                                          • First post
                                            Last post
                                          0
                                          • Categories
                                          • Recent
                                          • Tags
                                          • Popular
                                          • Users
                                          • Groups
                                          • Search
                                          • Get Qt Extensions
                                          • Unsolved