Qt Forum

    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Search
    • Unsolved

    Update: Forum Guidelines & Code of Conduct


    Qt World Summit: Early-Bird Tickets

    Solved QTextStream writing cellida characters (é,è,ë,ĉ,ç) to disk appear as ?

    General and Desktop
    2
    7
    1462
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • A
      arozon last edited by arozon

      Hi. A little bit of background on my situation starts off with the general idea of the application. I am trying to build a data entry application using web sockets with both a mobile and desktop version.

      Using a websocket server running on windows and built through qt i receive qstrings and process my data without any problems however there seems to be a bug when a websocket server is running on a windows machine and interpreting QStrings from a mac0S or iOS client

      Characters such as:

      À é è ê ę ç
      

      Appear as ? When parsed by the windows server.. i believe this is because of the nature of characters on mac and ios using cedilla combining character.

      However strings with such characters sent from an android or pc linux client are not lost in parsing.

      I am using TextMessages to pass information. Any help would be greatly appreciated :D

      Edit:
      Please see post 6, the problem is caused when writing cellida characters to disk using QTextStream

      raven-worx 1 Reply Last reply Reply Quote 0
      • A
        arozon last edited by

        Fixed, the solution was quite simple but I still believe this may be a bug. As cellida characters received from a macOS or iOS environment fail to be autoDetected as UTF-8, and the fallback to system on windows renders the characters as question marks to data on disk.

        Solution:

        bool FileIO::writeFile(const QString& data, const QString& filename)
        {
            qDebug() << data;
            qDebug() << filename;
            if (filename.isEmpty())
                return false;
        
            QFile file(cDir.path() + cDir.separator() + filename);
            if (!file.open(QFile::WriteOnly | QFile::Truncate))
                return false;
        
            QTextStream out(&file);
            out.setCodec("UTF-8"); //Makes sure to always force UTF-8
            out << data;
        
            file.close();
        
            return true;
        }
        
        1 Reply Last reply Reply Quote 1
        • raven-worx
          raven-worx Moderators @arozon last edited by

          @arozon

          1. are all clients built by you using Qt?
          2. how exactly are you sending (encoode) the data sent?
          3. how exactly do you receive (decode) the received data on the windows server?
          4. where do you see the '?'?

          For example you should send UTF-8 binary data over the web-socket and do proper encoding on the other side from UTF-8.

          --- SUPPORT REQUESTS VIA CHAT WILL BE IGNORED ---
          If you have a question please use the forum so others can benefit from the solution in the future

          1 Reply Last reply Reply Quote 3
          • A
            arozon last edited by

            @raven-worx Yes all clients are from Qt

            I am receiving a json stringifyied object where a string is received like this

            {
            "messageIndex":"0",
            "Mautent":["mess":"djhkgs","ax":"568hfduy4"], 
             "data": "{'name':'TéÈç', ...}"
            }
            

            This data is received from a TextInput and then sent from any client on the different OS's through sendTextMessage();

            From there on I receive the object on a windows machine running a Qt websocketServer, search the string to verify authentication, either parse or not based on the result.

            However text pulled from a macOS or iOS client results in a question mark where cellida characters were placed.

            I believe that for example ç character which is encoded in utf-8 as 0xc3 0xa7 gets passed as 0x63 0xcc 0xa7

            raven-worx 1 Reply Last reply Reply Quote 0
            • raven-worx
              raven-worx Moderators @arozon last edited by

              @arozon said in WebSocket Server on windows misrepresenting strings with cellida characters (é,è,ë,ĉ,ç) from macOS:

              "data": "{'name':'TéÈç', ...}"

              so you have a JSON object which has another JSON object but the nested JSON data is transfered as string?? o.O
              Why?!

              Also you didn't show how you are composing your JSON data and also how you are decomposing it.

              --- SUPPORT REQUESTS VIA CHAT WILL BE IGNORED ---
              If you have a question please use the forum so others can benefit from the solution in the future

              1 Reply Last reply Reply Quote 1
              • A
                arozon last edited by arozon

                @raven-worx Not quite, but yeah. Its a json object where one of the objects holds a string representing the data that can be easily parshed directly. I didnt make it over complicated as strings are very small and the data entry will not be excessive. I perform a handshake before hand, but thats irrelevant.

                In terms of setting data quite simple..

                // SENDING THE DATA TO THE SERVER (SEND AS TEXT MESSAGE)
                
                function getDataString() {
                var mess = JSON.parse('{"messageIndex":"","Mautent":["mess":"","ax":""],  "data": ""}');
                var dataobj = JSON.parse('{"name":"", ...}');
                dataobj.name = ftextname.text; //TéÈç
                ...
                mess.data = JSON.stringify(dataobj);
                ...
                return JSON.stringify(mess);
                }
                
                
                // PARSING THE RECEIVED STRING
                
                function parseMessage(message) {
                try {
                    var parsed = JSON.parse(message);
                    // authenticate
                    if (auth) {
                    file.writeFile(parsed.data, getRandomName());
                    }
                  } catch (e) {
                    // Oh well, but whatever...
                  }
                }
                
                // ON THE C++ SIDE (file.writeFile) (QML PLUGIN)
                
                bool FileIO::writeFile(const QString& data, const QString& filename)
                {
                    if (filename.isEmpty())
                        return false;
                
                    QFile file(cDir.path() + cDir.separator() + filename);
                    if (!file.open(QFile::WriteOnly | QFile::Truncate))
                        return false;
                
                    QTextStream out(&file);
                    out << data;
                
                    file.close();
                
                    return true;
                }
                
                1 Reply Last reply Reply Quote 0
                • A
                  arozon last edited by

                  After working through more debug, the culprit seems to be when saving the file. Up to that point my string remains intact.
                  Changing the main topic to QTextStream encoding. When writing the stream to the disk, cellida characters are lost.

                  bool FileIO::writeFile(const QString& data, const QString& filename)
                  {
                      if (filename.isEmpty())
                          return false;
                  
                      QFile file(cDir.path() + cDir.separator() + filename);
                      if (!file.open(QFile::WriteOnly | QFile::Truncate))
                          return false;
                  
                      QTextStream out(&file);
                      out << data;
                  
                      file.close();
                  
                      return true;
                  }
                  
                  1 Reply Last reply Reply Quote 0
                  • A
                    arozon last edited by

                    Fixed, the solution was quite simple but I still believe this may be a bug. As cellida characters received from a macOS or iOS environment fail to be autoDetected as UTF-8, and the fallback to system on windows renders the characters as question marks to data on disk.

                    Solution:

                    bool FileIO::writeFile(const QString& data, const QString& filename)
                    {
                        qDebug() << data;
                        qDebug() << filename;
                        if (filename.isEmpty())
                            return false;
                    
                        QFile file(cDir.path() + cDir.separator() + filename);
                        if (!file.open(QFile::WriteOnly | QFile::Truncate))
                            return false;
                    
                        QTextStream out(&file);
                        out.setCodec("UTF-8"); //Makes sure to always force UTF-8
                        out << data;
                    
                        file.close();
                    
                        return true;
                    }
                    
                    1 Reply Last reply Reply Quote 1
                    • First post
                      Last post