Simple encryption output is not complete

[jsulm]

@tomy Why don't you simply encode binary data using QByteArray?

[tomy]

@jsulm
What binary data do you mean? Do you mean encoding the deciphered string into readable text inside convert/hex2char?

Another new issue is that the project works fine for English messages with size 4 chars but not for messages of the same size in other languages, say, eastern ones like Japanese, Arabic etc! :(
Any reason why, please?

[jsulm]

@tomy What I mean is: do not work on character level but pure binary data. For example if your string can be encoded as UTF-8 you can get QByteArray using https://doc.qt.io/qt-5/qstring.html#toUtf8

[tomy]

@jsulm

The encipher/decipher functions work on each byte of the data given to it apparently. You mean to store each 4 chars (8 bytes) of string into a QByteArray and then send it to the those functions!?
I've not mastered the TEA algorithm, so changing it for me is hard.

The more important problem for me at the time is to know why the project works for English letters but not eastern ones while QChar is two bytes and unicode! :(
Qt, according to my experience on using it up to now, has worked fine with strings in various languages.

[VRonin]

@tomy said in Simple encryption output is not complete:

The more important problem for me at the time is to know why the project works for English letters but not eastern ones

It's written in the docs. TestClass::hex2char calls toLatin1() and, from the docs:

Note: It is not possible to distinguish a non-Latin-1 character from a Latin-1 0 (NUL) character.

[tomy]

@VRonin
Yeah, you're right, thanks.

I use the unicode() version as suggested by docs to be preferred:
return ch >= '0' && ch <= '9' ? ch.unicode() - '0' : ch.toLower().unicode() - 'a' + 10;, but again the same functionality and no change in the output! :(

For both toLatin1() and unicode() I get at times the error: ASSERT: "size_t(i) < size_t(size())" in file C:/Qt/6.2.3/mingw_64/include/QtCore/qstring.h, line 1212 .

[hskoglund]

@tomy Hi I googled the TEA algorithm, actually I think what you're implementing is https://en.wikipedia.org/wiki/XTEA.

Anyway, I created a simple Qt console app using what @jsulm suggested: a binary representation, although I didn't use QByteArray instead opting for two unions (yeah I know pretty old-school :-)
Here's the main.cpp (I copied the encipher() and decipher() functions from Wikipedia and hardwired the # of rounds to 32):

#include <QCoreApplication>

// see https://en.wikipedia.org/wiki/XTEA
unsigned int num_rounds = 32;

void encipher(uint32_t v[2], uint32_t const key[4]) {
    unsigned int i;
    uint32_t v0=v[0], v1=v[1], sum=0, delta=0x9E3779B9;
    for (i=0; i < num_rounds; i++) {
        v0 += (((v1 << 4) ^ (v1 >> 5)) + v1) ^ (sum + key[sum & 3]);
        sum += delta;
        v1 += (((v0 << 4) ^ (v0 >> 5)) + v0) ^ (sum + key[(sum>>11) & 3]);
    }
    v[0]=v0; v[1]=v1;
}

void decipher(uint32_t v[2], uint32_t const key[4]) {
    unsigned int i;
    uint32_t v0=v[0], v1=v[1], delta=0x9E3779B9, sum=delta*num_rounds;
    for (i=0; i < num_rounds; i++) {
        v1 -= (((v0 << 4) ^ (v0 >> 5)) + v0) ^ (sum + key[(sum>>11) & 3]);
        sum -= delta;
        v0 -= (((v1 << 4) ^ (v1 >> 5)) + v1) ^ (sum + key[sum & 3]);
    }
    v[0]=v0; v[1]=v1;
}

// show QStrings on the console
void qputs(QString s)
{
    puts(qUtf8Printable(s));
}

// for hex dumping
void dumpUint32(QString s,uint32_t* u,int count /* of uint32_t */)
{
    qputs(s + QByteArray::fromRawData((char *)u,count * 4).toHex(' '));
}

// main
int main()
{
    QString sSwedishPangram = "Flygande bäckasiner söka hwila på mjuka tuvor.";
    QString sPolishPangram  = "Zbłaźń mżystość ględów hiperfunkcją,";
    const int maxQChars = 100;

    union // for the key
    {
        char ak[16] = "Qt is the best!";
        uint32_t key[4];
    };
    dumpUint32("This is the key: ",key,4); // show it in hex

    union // for the data
    {
        QChar    ac[maxQChars] = {'\0'};  // fill it with zeros
        uint32_t av[maxQChars / 2];
    };

// stuff the data union
    int nQChars = 0;
    for (auto c : sSwedishPangram + sPolishPangram)
    {
        ac[nQChars++] = c;
        if (nQChars >= maxQChars)
            qFatal("out of room");
    }

    qputs(QString("\nQChars before encrption: ") + QString::fromRawData(ac,nQChars));
    dumpUint32("In hex before encryption: ",av,nQChars / 2);

// encrypt 4 QChars (or 2 uint32_t) at the time in a loop
    for (int c = 0; (c < nQChars); c += 4)
        encipher(av + c / 2,key);

    dumpUint32("\nIn hex after encryption: ",av,nQChars / 2);

// decrypt 4 QChars (or 2 uint32_t) at the time in a loop
    for (int c = 0; (c < nQChars); c += 4)
        decipher(av + c / 2,key);

    dumpUint32("\nIn hex after decryption: ",av,nQChars / 2);
    qputs(QString("\nQChars before encrption: ") + QString::fromRawData(ac,nQChars));
    auto s = QString::fromRawData(ac,nQChars);
    qputs("\nand the reconstructed QString: " + s);

    qputs("Have a nice day :-)");
 }

Hope you'll find it useful!

[hskoglund]

Forgot to mention: on Windows, the console output (or the Application output in Qt Creator) does not show Unicode characters very well. To see them in all their glory, for example you can start the app in a CMD window and pipe the output to a text file. Open it in Notepad, voila.

[tomy]

@hskoglund

Thanks for your help and I appreciate that. The TEA version used in my code is not XTEA by the way, albeit there're a couple of different versions of TEA out there! :) And since, as I said earlier, I've not scrutinized the algorithm well, because it works fine just like an API, changing it or working on another version of that takes likely much time for me.
So if you agree let's not let the earlier issue get buried.

If I enter "Turn" as input, I get "Turn" as output, using my current code and in convert.
These are also a couple of print messages:

Message before incryption: Turn
Message after incryption: 3004859800 98801674
Message after decryption: 7667796 7209074
Message decrypted, in hex: 7667796 7209074
Final message ('convert'): "T\u0000ur\u0000n"

But if "Zbłaźń" is entered, nothing will be given as output on the app and our print messages become:

Message before incryption: Zb?a??
Message after incryption: 1719136445 1717795074
Message after decryption: 6422618 6357314
Message decrypted, in hex: 6422618 6357314
Final message ('convert'): "Z\u0000bB\u0001a"

Message after incryption: 3673089947 2926484279
Message after decryption: 21234042 2097184
Message decrypted, in hex: 21234042 2097184
ASSERT: "size_t(i) < size_t(size())" in file C:/Qt/6.2.3/mingw_64/include/QtCore/qstring.h, line 1212

And the program terminates immediately!

And as a new attempt using part of your code in convert:

void TestClass::convert(const QString str)
{
    QString msg { str};
    QChar ac[100] = {'\0'};  // fill it with zeros
    int nQChars = 0;

    for(auto ch : msg) // we enter less than 100 chars
        ac[nQChars++] = ch;

    auto s = QString::fromRawData(ac,nQChars);

    msg.clear();

    for (size_t i = 0; i < s.size(); i += 2)
        msg += static_cast<char>(hex2char(s[i]) << 4 | hex2char(s[i + 1]));

    std::reverse(msg.begin(), msg.end());

    qInfo() << "Final message ('convert'):" << msg << "\n\n";

    message += msg;
    emit messageChanged(message);  
}

I tested the project for both the words "Turn" and "Zbłaźń", but the output and print messages are exactly the same as above!

I have got a simple question. Do I necessarily need to use a different version of the TEA algorithm in my Qt project to get it to support both western and eastern languages?

Isn't there any easier solution for that problem?

[hskoglund]

Hi, which version of TEA you use shouldn't matter, they're all based on xoring the bits of the key and the text. So to support both western and eastern languages, it doesn't matter exactly how TEA works.

Try to simplify the code as much as possible and reuse all of the functions that are already in Qt, like converting to/from hex. When you use stuff from Qt instead of writing your own, there's less risks of bugs.
Also try to avoid using casts like static_cast<char> (in my example main.cpp above I only use one (char *) cast in the dumpUint32() function.)

About the new code in the convert() function above, note that you tear down and reassemble the same str/msg i.e. fromRawData() just gives you the same QString back. And why reversing it, just adds an extra complexity.

I used the union trick in my main.cpp above so that I don't have to worry about exactly how western or eastern languages look like in Unicode, i.e. I store the QChars inside the union and then I use the uint32_t array to access and xor the bits with encipher()/decipher(). Also note that when encipher() has run, the QChars inside the union most likely are invalid so only use the uint32_t access of them!

[tomy]

This is a simplified version of the project:
.h:

class TestClass : public QObject
{
    Q_OBJECT
    Q_PROPERTY(QString message READ getMessage WRITE setMessage NOTIFY messageChanged);

public:
    explicit TestClass(QObject *parent = nullptr);

    const QString& getMessage() const { return message;}
    void setMessage(const QString&);

public slots:
    void encipher(const unsigned long* const, unsigned long* const, const unsigned long* const);
    void decipher(const unsigned long* const v, unsigned long* const w, const unsigned long* const k);
    void convert(const QString);

signals:
    void messageChanged(QString);

private:
    QString message;
};

.cpp:

#include "testclass.h"
#include <QTextStream>

TestClass::TestClass(QObject *parent)
    : QObject{parent} {  }

void TestClass::encipher(const unsigned long* const v, unsigned long* const w,
                         const unsigned long* const k)
{
    static_assert(sizeof(long) == 4, "size of long wrong for TEA");
    unsigned long y = v[0];
    unsigned long z = v[1];
    unsigned long sum = 0;
    const unsigned long delta = 0x9E3779B9;

    for (unsigned long n = 32; n-- > 0; ) {
        y += (z << 4 ^ z >> 5) + z ^ sum + k[sum & 3];
        sum += delta;
        z += (y << 4 ^ y >> 5) + y ^ sum + k[sum >> 11 & 3];
    }

    w[0] = y;
    w[1] = z;
}

void TestClass::decipher(const unsigned long* const v, unsigned long* const w,
                         const unsigned long* const k)
{
    static_assert(sizeof(long) == 4, "size of long wrong for TEA");
    unsigned long y = v[0];
    unsigned long z = v[1];
    unsigned long sum = 0xC6EF3720;
    const unsigned long delta = 0x9E3779B9;

    for (unsigned long n = 32; n-- > 0; ) {
        z -= (y << 4 ^ y >> 5) + y ^ sum + k[sum >> 11 & 3];
        sum -= delta;
        y -= (z << 4 ^ z >> 5) + z ^ sum + k[sum & 3];
    }

    w[0] = y;
    w[1] = z;
}

void TestClass::setMessage(const QString& newMessage)
{
    QString key {"bs"};
    while (key.size() < 16)
        key += ' '; // pad the key

    const unsigned long* k = reinterpret_cast<const unsigned long*>(key.data());

    QChar inbuf[4]; // 4 QChars, 8 bytes suitable for encipher function
    unsigned long* inptr = reinterpret_cast<unsigned long*>(inbuf);

    unsigned long encOutptr[2]; // two 4-byte array, totally 8 bytes
    unsigned long decOutptr[2]; // two 4-byte array, totally 8 bytes

    int count = 0;
    for (const auto ch : newMessage)
        inbuf[count++] = ch;

    encipher(inptr, encOutptr, k);

    QString encipheredMessage;
    QTextStream ts1 {&encipheredMessage};
    ts1 << encOutptr[0] << ' ' << encOutptr[1];

    encOutptr[0] = encipheredMessage.sliced(0, encipheredMessage.indexOf(' ')).toULong();
    encOutptr[1] = encipheredMessage.sliced(encipheredMessage.indexOf(' ') +1).toULong();

    decipher(encOutptr, decOutptr, k); // Decrypt data using the key

    QString decryptedMessage;
    QTextStream ts2 {&decryptedMessage};
    ts2 << Qt::hex << decOutptr[1] << decOutptr[0];

    convert(decryptedMessage);
}

void TestClass::convert(const QString str)
{
    auto ba { QByteArray::fromHex(str.toUtf8()) };
    QString msg;

    for(int i=0; i< ba.size(); ++i)
        msg += *(ba.data()+i);

    std::reverse(msg.begin(), msg.end());
    message = msg;
    emit messageChanged(message);
}

1. Since each QChar is two bytes - to support unicode characters- and the encipher/decipher functions accept arrays of 8 bytes, so we need to send them arrays of 4 QChars, as written in the code. Right?
2. In convert I removed cast and tied to use built-in Qt facilities. The project works fine with messages up to 4 Ascii characters, for example: Test or Hi!, but couldn't find any Qt facility to convert hex data to human readable text in eastern languages). Any idea for this part, please?