Display a std::string in a QLineEdit in QT

amadeok

Hello

I'm making a c++ program that searches inside files.

I'm using std::ifstream and std::find() to search, so the positions i get with ifstream.tellg() are the byte positions of the words in the ifstream.

if i want to display the results in a QLineEdit for example, i'd have to convert to QString, which is utf16, which means that the byte position i got before with std::find() will be useless, specially if the text has special characters, for example:
"àòaa" in utf8 has 6 bytes, but in utf16 it has 8 bytes, and the difference adds up with the multi-byte characters of utf8.

So is there a way to display a std::string on a QLineEdit without converting to QString? Many files are in utf8 and I would rather not have to convert them all the utf16

thanks

JonB

@amadeok said in Display a std::string in a QLineEdit in QT:

So is there a way to display a std::string on a QLineEdit without converting to QString?

Since QLineEdit ultimately needs a QString, and since QString has QString QString::fromStdString(const std::string &str) I think you will want

lineEdit->setText(QString::fromStdString(stdString));

JKSH

@amadeok I'm not sure that counting bytes in a UTF-8 string makes sense, since a byte does not represent a "character" (grapheme, to be precise)

You can use QString::indexOf() to search a QString.

Many files are in utf8 and I would rather not have to convert them all the utf16

How big are these files? What device are you running on?

amadeok

@JonB said in Display a std::string in a QLineEdit in QT:

@amadeok said in Display a std::string in a QLineEdit in QT:

So is there a way to display a std::string on a QLineEdit without converting to QString?

Since QLineEdit ultimately needs a QString, and since QString has QString QString::fromStdString(const std::string &str) I think you will want
lineEdit->setText(QString::fromStdString(stdString));

@JonB
hi thanks for reply.
Yes but say if i need to highlight a word, or display a line whose position i found by looking at the original utf8 encoded string, that position i found with std::find() will not lead me to the same position on the converted utf16 text.

@JKSH said in Display a std::string in a QLineEdit in QT:

@amadeok I'm not sure that counting bytes in a UTF-8 string makes sense, since a byte does not represent a "character" (grapheme, to be precise)

You can use QString::indexOf() to search a QString.

Many files are in utf8 and I would rather not have to convert them all the utf16

How big are these files? What device are you running on?

Hi, the application is for PC, linux or windows.
The files could be any size, depending on what the user is searching, that's why i'm concerned with having to convert all

JKSH

@amadeok said in Display a std::string in a QLineEdit in QT:

Hi, the application is for PC, linux or windows.
The files could be any size, depending on what the user is searching, that's why i'm concerned with having to convert all

Unless the file is multiple gigabytes, you won't notice the conversion overhead on modern PCs. And if the file is multiple gigabytes, then it can't fit in a QLineEdit.

One possible approach is to do your original search in std::string. Then when you find it, convert the small section around your target to QString and then use QString::indexOf() for highlighting. Put the substring in the QLineEdit, rather than your whole file.

amadeok

@JKSH
Hi
Yes converting only the lines that were found could be an option, or if i need to display the whole file, to convert everything gradually according to where the user is scrolling.
I did some testing on a 10mb .txt file:

std::wstring wreadFile(const char* filename)
{
	std::wifstream wif(filename);
	wif.imbue(std::locale(std::locale::empty(), new std::codecvt_utf8<wchar_t>));
	std::wstringstream wss;
	wss << wif.rdbuf();
	return wss.str();
}

std::string readFile(const char* filename)
{
	std::ifstream in(filename);
	std::stringstream ss;
	ss << in.rdbuf();
	return ss.str();
}

on my pc the second function which reads normally the utf8 file takes about 130ms, while the first that converts to wstring while reading took about 750ms.

JonB

@amadeok
My 2 cents. Ultimately you want to display the string in a QLineEdit. Convert the incoming to QString, either in total or incrementally as you go along, and do the searching in the QString. This business of insisting on using std::find() and then converting positions is unhelpful. If you can't find/display it right in a QString it won't be right in the QLineEdit anyway.

JKSH

@amadeok said in Display a std::string in a QLineEdit in QT:

on my pc the second function which reads normally the utf8 file takes about 130ms, while the first that converts to wstring while reading took about 750ms.

That doesn't tell us anything useful, because QString is completely different from std::wstring.

I extended your test on a 13MB .txt file (created by duplicating https://www.w3.org/2001/06/utf-8-test/UTF-8-demo.html 10x):

QString qreadFile(const char* filename)
{
	QFile src(filename);
	src.open(QFile::ReadOnly|QFile::Text);
	return QString::fromUtf8(src.readAll());
}

QString qreadFile2(const QString& filename)
{
	QFile src(filename);
	src.open(QFile::ReadOnly|QFile::Text);

	QTextStream stream(&src);
	stream.setCodec("UTF-8");

	return stream.readAll();
}

std::wstring wreadFile(const char* filename)
{
	std::wifstream wif(filename);
	wif.imbue(std::locale(std::locale::empty(), new std::codecvt_utf8<wchar_t>));
	std::wstringstream wss;
	wss << wif.rdbuf();
	return wss.str();
}

std::string readFile(const char* filename)
{
	std::ifstream in(filename);
	std::stringstream ss;
	ss << in.rdbuf();
	return ss.str();
}

On my PC (MSVC 2019 32-bit):

Reading std::wstring took 676ms
Reading std::string took 135ms
Reading QString took 53ms (both versions)

So reading the file as a QString is faster than reading it as a std::string.