DTLS. "dh key too small" but TLS works

BrMisha

I have two Qt-based applications (client and server) which use DTLS and TLS connections. That works fine on Ubuntu and Windows 10. Now i have tried to build the server's part for a raspberry pi 4 (raspbian). TLS works fine but a DTLS handshake stops with error.
On server side (raspbian):

14102410:SSL routines:dtls1_read_bytes:sslv3 alert handshake failure

On client (Ubuntu 20):

141A318A:SSL routines:tls_process_ske_dhe:dh key too small

The client's error is quite popular and i have found many solutions fix that (add 2 string into the /etc/ssl/openssl.cnf) but that doesn't affect

As i know this error is about server's side has lower minimal key length but why TLS work in this case?
Thanks

BrMisha

After long time of tests i have established connection.
Raspberry uses DHE-PSK-AES256-GCM-SHA384 cipher (dont know why).
I set "m_serverConfiguration.setCiphers({QSslCipher("ECDHE-PSK-CHACHA20-POLY1305")})" and that works but that is temporary solution.

PS: ECDHE-PSK-CHACHA20-POLY1305 chooses when server works on my desctop (ubuntu)

By the way, i have tried to connect from console (from desctop):

openssl s_client -dtls1_2 -connect 192.168.0.103:3000 -psk 70736b -psk_identity 6eccf762dfb624c9b915f4b34238431c.

and see similar error:

misha@misha-xigmater:~$ openssl s_client -dtls1_2 -connect 192.168.0.103:3000 -psk 70736b -psk_identity 6eccf762dfb624c9b915f4b34238431c
CONNECTED(00000003)
Can't use SSL_get_servername
140501839738176:error:141A318A:SSL routines:tls_process_ske_dhe:dh key too small:../ssl/statem/statem_clnt.c:2149:
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 453 bytes and written 561 bytes
Verification: OK
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : DTLSv1.2
    Cipher    : 0000
    Session-ID: 
    Session-ID-ctx: 
    Master-Key: 
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    Start Time: 1629932016
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
    Extended master secret: yes

But there is all ok if i connect from raspberry (where server works):

CONNECTED(00000003)
Can't use SSL_get_servername
---
no peer certificate available
---
No client certificate CA names sent
Server Temp Key: DH, 1024 bits
---
SSL handshake has read 751 bytes and written 822 bytes
Verification: OK
---
New, TLSv1.2, Cipher is DHE-PSK-AES256-GCM-SHA384
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : DTLSv1.2
    Cipher    : DHE-PSK-AES256-GCM-SHA384
    Session-ID: 6F647EAB003A3669714D295D45BA190F0B01A1212E59FBB946FD8097298EFC23
    Session-ID-ctx: 
    Master-Key: 63A0046E9BB5E5DE9EA76BB822319DABF543F13A0AE864B84D405AB955705F0A3E5286892D07BC194D13B19DFF5E31BA
    PSK identity: 6eccf762dfb624c9b915f4b34238431c
    PSK identity hint: None
    SRP username: None
    TLS session ticket lifetime hint: 7200 (seconds)
    TLS session ticket:
    0000 - 90 7f 32 5d 7f 65 48 50-14 19 41 da 13 7b 67 c6   ..2].eHP..A..{g.
    0010 - 7c 73 3d 57 0f 4e bc 6a-58 81 a2 0f 05 62 6e ee   |s=W.N.jX....bn.
    0020 - f4 83 d4 a7 c7 10 3b 0b-9e 09 1c 01 e1 87 0b 6b   ......;........k
    0030 - a7 33 64 97 21 e3 e2 d7-ef 13 4e 7e 26 ee e1 84   .3d.!.....N~&...
    0040 - 46 c2 bc 92 21 ef ad 05-f3 f6 b5 62 9a bc 19 14   F...!......b....
    0050 - 9f 8f 68 8c f0 f2 6e 39-fe df eb a2 0d 7e 27 70   ..h...n9.....~'p
    0060 - df b6 f1 5d c7 93 75 9d-4f 7f 2e 44 9b 07 ad b5   ...]..u.O..D....
    0070 - ac ce 07 c3 e9 51 be 6e-0b ec 77 37 a5 7e 75 33   .....Q.n..w7.~u3
    0080 - 0f 7e 64 a8 ab 29 57 aa-35 eb a7 bd 88 49 4b 3e   .~d..)W.5....IK>
    0090 - 00 b7 66 0b f1 51 d2 f7-2e 2b 07 48 4e d0 fb 78   ..f..Q...+.HN..x
    00a0 - 44 fe 4b 8f 13 92 a3 ca-c1 2a b7 58 d0 74 bd 8a   D.K......*.X.t..
    00b0 - 1f cb fa 6c 22 9f 98 fb-24 a0 d4 b5 65 34 dc 23   ...l"...$...e4.#

    Start Time: 1629932165
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
    Extended master secret: yes