range of ASCII characters which can be displayed by Qt

Sillimon · 20 Feb 2018, 10:11

Hello all !
I'm currently working on reading cards information and i'm having troubles with some data i get.
One of the cards i'm reading is containing Latin-1 ASCII char and more. When i get them in a byteArray, i'm checking the values with the "Locals and expressions" features and i got this :
[0] '1/2' 189 0xbd
[1] nothing 22 0x16
[0] 'ï' 239 0xef
etc...

As you can see, some of them doesn't even have a char representation to be displayed...
Those are probably the control characters which is why they cannot be displayed but i wanted to be sure of me so i'm asking the question :
Considering that i have to display everything but those "empty-like" character, does someone know the range of the ASCII that cannot be displayed in a Qt interface ?

I tried something like that to manage it but that's an hypothetic value :

QChar latinChar = readData.at(k);
                if(latinChar < 32)
                {
                    alpha = ".";
                }
                else if(readData.at(k) == '\0')
                {
                    alpha = ".";
                }
                else
                {
                    alpha = readData.at(k);
                }

Thanks a lot for answers and knowledge !

EDIT : seems that char after 127 (included) ain't working either ?

jsulm · 20 Feb 2018, 10:31

@Sillimon Did you try http://doc.qt.io/qt-5/qstring.html#fromLatin1-1 to convert the byte array to QString?
Qt UI should be able to show any printable ASCII characters.
If your input contains non printable ASCII characters then you will need to remove them first.

Paul Colby · wrote on 20 Feb 2018, 10:14

Hi @Sillimon,

Have a look at QChar::isPrint() - looks like it does what you want :)

PS Latin-1 and ASCII are similar, but not the same, and 127 is not a printable character, in either set.

Cheers.

Sillimon · 20 Feb 2018, 10:35

@jsulm
Yep, i tried it... and a lot of others like unicode too but they didn't worked. Perhaps because, as you said, they contain non-printable characters so they can't convert. But in this case, QLatin1Char should have worked right ?

Also, i have to take those non-printable characters in count as long as i have to represent them by a ' . '

@Paul-Colby
I tried it but it worked too good... It worked for those empty characters AND the other latin1 character i used to display correctly (just like ï) so i gave up on it !

You mean Latin-1 is an "add-on" of ASCII ? Like the continuity of the ASCII ?
Yeah, 127 is the DEL right ? but for 128+ char ? They seem totally printable to me...

aha_1980 · 20 Feb 2018, 12:06

@Sillimon

You mean Latin-1 is an "add-on" of ASCII ? Like the continuity of the ASCII ?

Indeed, Latin-1 is one addon to ASCII, but there are much more and they are incompatible. That's why Unicode was invented - to solve all the incompatible encodings.

If your QByteArray is really containing Latin1-characters, try to create a QString from them: QString str = QString::fromLatin1(byteArray);.

As QString is Unicode (UTF-16), you should now be able to correctly read your data.

Paul Colby · wrote on 20 Feb 2018, 11:29

So I had a bit of a play, and the problem is with this line:

QChar latinChar = readData.at(k);

The problem here, is that you are relying on a QChar constructor which is interpreting the byte as Unicode (interestingly it appears that the docs are wrong on that point). If you were to define QT_NO_CAST_FROM_ASCII in your project (which I always do - its best practice, in my opinion), then this code wouldn't even compile. I suggest you read up on QT_NO_CAST_FROM_ASCII at http://doc.qt.io/qt-5/qstring.html#QT_NO_CAST_FROM_ASCII

Anyway, you can get what you want (so far as my testing shows) but using QChar::fromLatin1() explicitly, or QLatin1Char. eg

    // Setup a byte array with all possible 8-bit values.
    for (uint c = 0; c < 256; ++c)
        readData.append(c);

    // Interpret each byte as a Latin-1 char, and see if its printable.
    for (int i = 0; i < readData.size(); ++i) {
        //const QChar c = readData.at(i); ///< This is not the char you're looking for.
        const QChar c = QLatin1Char(readData.at(i)); ///< This is good.
        //const QChar c = QChar::fromLatin1(readData.at(i)); ///< Also a good option.
        qDebug() << i << (c.isPrint() ? c.toLatin1() : '.') << c.isPrint();
    }

Output looks like:

0 . false
1 . false
2 . false
3 . false
4 . false
5 . false
6 . false
7 . false
8 . false
9 . false
10 . false
11 . false
12 . false
13 . false
14 . false
15 . false
16 . false
17 . false
18 . false
19 . false
20 . false
21 . false
22 . false
23 . false
24 . false
25 . false
26 . false
27 . false
28 . false
29 . false
30 . false
31 . false
32   true
33 ! true
34 " true
35 # true
36 $ true
37 % true
38 & true
39 ' true
40 ( true
41 ) true
42 * true
43 + true
44 , true
45 - true
46 . true
47 / true
48 0 true
49 1 true
50 2 true
51 3 true
52 4 true
53 5 true
54 6 true
55 7 true
56 8 true
57 9 true
58 : true
59 ; true
60 < true
61 = true
62 > true
63 ? true
64 @ true
65 A true
66 B true
67 C true
68 D true
69 E true
70 F true
71 G true
72 H true
73 I true
74 J true
75 K true
76 L true
77 M true
78 N true
79 O true
80 P true
81 Q true
82 R true
83 S true
84 T true
85 U true
86 V true
87 W true
88 X true
89 Y true
90 Z true
91 [ true
92 \ true
93 ] true
94 ^ true
95 _ true
96 ` true
97 a true
98 b true
99 c true
100 d true
101 e true
102 f true
103 g true
104 h true
105 i true
106 j true
107 k true
108 l true
109 m true
110 n true
111 o true
112 p true
113 q true
114 r true
115 s true
116 t true
117 u true
118 v true
119 w true
120 x true
121 y true
122 z true
123 { true
124 | true
125 } true
126 ~ true
127 . false
128 . false
129 . false
130 . false
131 . false
132 . false
133 . false
134 . false
135 . false
136 . false
137 . false
138 . false
139 . false
140 . false
141 . false
142 . false
143 . false
144 . false
145 . false
146 . false
147 . false
148 . false
149 . false
150 . false
151 . false
152 . false
153 . false
154 . false
155 . false
156 . false
157 . false
158 . false
159 . false
160   true
161 ¡ true
162 ¢ true
163 £ true
164 ¤ true
165 ¥ true
166 ¦ true
167 § true
168 ¨ true
169 © true
170 ª true
171 « true
172 ¬ true
173 . false
174 ® true
175 ¯ true
176 ° true
177 ± true
178 ² true
179 ³ true
180 ´ true
181 µ true
182 ¶ true
183 · true
184 ¸ true
185 ¹ true
186 º true
187 » true
188 ¼ true
189 ½ true
190 ¾ true
191 ¿ true
192 À true
193 Á true
194 Â true
195 Ã true
196 Ä true
197 Å true
198 Æ true
199 Ç true
200 È true
201 É true
202 Ê true
203 Ë true
204 Ì true
205 Í true
206 Î true
207 Ï true
208 Ð true
209 Ñ true
210 Ò true
211 Ó true
212 Ô true
213 Õ true
214 Ö true
215 × true
216 Ø true
217 Ù true
218 Ú true
219 Û true
220 Ü true
221 Ý true
222 Þ true
223 ß true
224 à true
225 á true
226 â true
227 ã true
228 ä true
229 å true
230 æ true
231 ç true
232 è true
233 é true
234 ê true
235 ë true
236 ì true
237 í true
238 î true
239 ï true
240 ð true
241 ñ true
242 ò true
243 ó true
244 ô true
245 õ true
246 ö true
247 ÷ true
248 ø true
249 ù true
250 ú true
251 û true
252 ü true
253 ý true
254 þ true
255 ÿ true

Cheers.

Sillimon · 21 Feb 2018, 07:26

@aha_1980 said in range of ASCII characters which can be displayed by Qt:

Indeed, Latin-1 is one addon to ASCII, but there are much more and they are incompatible. That's why Unicode was invented - to solve all the incompatible encodings.

So unicode is basically a universal representation type, taking everything in count ? Doesn't the others encoding types lose a bit of their purpose ?
I found this website with that interesting array, even though i'm not fully/easily understanding everything :
Unicode Data Transfer Formats

I didn't read everything yet since i do not have the time right now but they all seems... really similar ?
I mean... the range they cover is usually similar and does the octets number make such a big difference ?
I think imma need some reading time this evening...

If your QByteArray is really containing Latin1-characters, try to create a QString from them: QString str = QString::fromLatin1(byteArray);.

As QString is Unicode (UTF-16), you should now be able to correctly read your data.

Actually, i think i'm displaying everything that could be displayed...
I checked on wikipedia : ISO/IEC 8859
And i can display everything but the grey ones wich seems normal actually.

@Paul-Colby
Yep ! It works perfectly ! Thanks a lot ! =)

aha_1980 · wrote on 21 Feb 2018, 07:26

@Sillimon said in range of ASCII characters which can be displayed by Qt:

So unicode is basically a universal representation type, taking everything in count ? Doesn't the others encoding types lose a bit of their purpose ?

In theory, yes. In practice, I think we will have to deal with them for some time more, especially Windows still uses these encodings a lot (e.g. if you create a console application), while the Windows API itself is Unicode 16-bit (UTF-16).