How to Store UDP packet for some second in RAM
-
You can use an easier solution to allocate QByteArray
std::vector<QByteArray> byte_arrays(50); //access it like this qDebug()<<byte_arrays[0]; //0 mean you access first QByteArray in the vector //no need to free the QByteArray because std::vector do it for you, who say resource management is hard in c++?In most of the cases, following statement is correct
allocate resource on stack is better than smart pointer, allocate on smart pointer is better than raw pointer@tham said in How to Store UDP packet for some second in RAM:
allocate resource on stack is better than smart pointer, allocate on smart pointer is better than raw pointer
Yes, although with Qt we have to remember that smart pointers are objects which manage lifetime of other objects, and in Qt the QObject based class system does it with parents. On the other hand some important classes (like QByteArray) are not QObject based. In Qt programming you can do
new MyQObjectBasedClass(pointerToParent);and the new pointer isn't get lost. But this is getting offtopic...
-
Hi,
Why not just send them by batch rather than trying to read them all there ?
[Moved in from duplicate thread ~kshegunov]
-
@Alexanov said in How to Store UDP packet for some second in RAM:
I want to store for 15 Second all packet in RAM and then store in HDD in *.txt file. (15 second * 800MB =12GB)
Do you have that amount of memory (in addition to what the system requires), as your error seems to indicate you don't?
@kshegunov said in How to Store UDP packet for some second in RAM:
addition
Yes , I have 128G RAM and 1TB HDD. I did not have any problem with my Hardware.
Although I saw the task manager ,when my program allocate 1G Memory , OS terminate that process and send signal to Qt and say "bad alloc()".My Qustion is: what do you do? if you want to accumulate UDP received packet for a few second and then store in HDD???
-
@Alexanov 1Gbit ethernet isn't 1GBytes/second, it's 1GBits/second which is 1/8 of 1GBytes/second.
It's of course possible to create QByteArrays in a loop with new, however you have to store the pointers somewhere. Or maybe you can use QByteArrayList.
-
Hi,
Why not just send them by batch rather than trying to read them all there ?
[Moved in from duplicate thread ~kshegunov]
-
Rather than store that much data in RAM and then dump everything in your HDD, make the chunk smaller and write more regularly.
-
Okay ,I did it and worked fine but , this method is stopgap solution,
because at second Step i want to replace 1G Ethernet card to 10G. When I replaced cards , "file.write" line (in my code) worked very slowly than packets were coming !(I lost some packets when my program was writing in file , the speed of write is more slower than speed of incoming data)!!!!!
do you have any solution for this ?
Thanks :) -
Only one: adapt your hardware.
You can't expect to dump GB worth of data each second on a single spinning disks hard drive (even solid state drive). Use something like a RAID to withstand the throughput.
-
Only one: adapt your hardware.
You can't expect to dump GB worth of data each second on a single spinning disks hard drive (even solid state drive). Use something like a RAID to withstand the throughput.
-
Okay ,I did it and worked fine but , this method is stopgap solution,
because at second Step i want to replace 1G Ethernet card to 10G. When I replaced cards , "file.write" line (in my code) worked very slowly than packets were coming !(I lost some packets when my program was writing in file , the speed of write is more slower than speed of incoming data)!!!!!
do you have any solution for this ?
Thanks :)@Alexanov said in How to Store UDP packet for some second in RAM:
the speed of write is more slower than speed of incoming data
In addition to what SGaist said, also consider moving various tasks into separate threads. So that your network thread will continue reading data into buffer while your IO thread writes to HDD.
If, however, you constantly (per second) get more data than your hard disks can write... there is no software solution for this (unless you can go clever with the data: analyze it and extract only important parts, compress it etc.).
-
You can use an easier solution to allocate QByteArray
std::vector<QByteArray> byte_arrays(50); //access it like this qDebug()<<byte_arrays[0]; //0 mean you access first QByteArray in the vector //no need to free the QByteArray because std::vector do it for you, who say resource management is hard in c++?In most of the cases, following statement is correct
allocate resource on stack is better than smart pointer, allocate on smart pointer is better than raw pointer
