Memory leaks and memory fragmentation
-
wrote on 27 Jun 2023, 11:25 last edited by
I am developing software and I find that the memory keeps going up, my program needs to use new and delete frequently, I suspect a memory leak or memory fragmentation. I checked with vld and found no reports of memory leaks, but memory was still slowly increasing. I couldn't tell if this was normal. I monitored memory by using the performance monitor that came with windows to monitor private bytes in process and by looking at committed memory in Task Manager Details. Is there any way but until this rise is normal? And are there any methods or tools to monitor and identify problems? My development and deployment environment is under windows, because the software needs to run for a long time, even memory fragmentation may have some impact, hope to get help, thank you very much
-
I am developing software and I find that the memory keeps going up, my program needs to use new and delete frequently, I suspect a memory leak or memory fragmentation. I checked with vld and found no reports of memory leaks, but memory was still slowly increasing. I couldn't tell if this was normal. I monitored memory by using the performance monitor that came with windows to monitor private bytes in process and by looking at committed memory in Task Manager Details. Is there any way but until this rise is normal? And are there any methods or tools to monitor and identify problems? My development and deployment environment is under windows, because the software needs to run for a long time, even memory fragmentation may have some impact, hope to get help, thank you very much
@BoLin said in Memory leaks and memory fragmentation:
performance monitor that came with windows
This is not a proper tool for this.
On Linux you would typically use Valgrind to look for memory leaks, not sure about Windows. -
I am developing software and I find that the memory keeps going up, my program needs to use new and delete frequently, I suspect a memory leak or memory fragmentation. I checked with vld and found no reports of memory leaks, but memory was still slowly increasing. I couldn't tell if this was normal. I monitored memory by using the performance monitor that came with windows to monitor private bytes in process and by looking at committed memory in Task Manager Details. Is there any way but until this rise is normal? And are there any methods or tools to monitor and identify problems? My development and deployment environment is under windows, because the software needs to run for a long time, even memory fragmentation may have some impact, hope to get help, thank you very much
wrote on 27 Jun 2023, 11:32 last edited by@BoLin said in Memory leaks and memory fragmentation:
even memory fragmentation may have some impact
As @jsulm says for the overall picture. We often have people coming saying "Qt leaks", turns out it does not. Just saying.
But I just wanted to say: it's one thing to look for and resolve if there are any memory leaks. But if you expect a C++ program to not cause "memory fragmentation", at least potentially, good luck, I don't think that will be easy to affect....
-
I am developing software and I find that the memory keeps going up, my program needs to use new and delete frequently, I suspect a memory leak or memory fragmentation. I checked with vld and found no reports of memory leaks, but memory was still slowly increasing. I couldn't tell if this was normal. I monitored memory by using the performance monitor that came with windows to monitor private bytes in process and by looking at committed memory in Task Manager Details. Is there any way but until this rise is normal? And are there any methods or tools to monitor and identify problems? My development and deployment environment is under windows, because the software needs to run for a long time, even memory fragmentation may have some impact, hope to get help, thank you very much
-
I am developing software and I find that the memory keeps going up, my program needs to use new and delete frequently, I suspect a memory leak or memory fragmentation. I checked with vld and found no reports of memory leaks, but memory was still slowly increasing. I couldn't tell if this was normal. I monitored memory by using the performance monitor that came with windows to monitor private bytes in process and by looking at committed memory in Task Manager Details. Is there any way but until this rise is normal? And are there any methods or tools to monitor and identify problems? My development and deployment environment is under windows, because the software needs to run for a long time, even memory fragmentation may have some impact, hope to get help, thank you very much
@BoLin about what quantities are we actually talking about here ?
my program needs to use new and delete frequently
how frequently ? the OS is "smart" and as such it gets annoyed quite fast. If you constantly request and free memory it is going to say "Fine keep it! I'll check later when you have made up your mind."
-
wrote on 28 Jun 2023, 11:01 last edited by
@Chris-Kawa I took a look at the software today, but the specific usage is still not clear. Have you ever used this software to detect memory fragmentation? If so, is it convenient to tell what program is being used? Because I did test this, I was using a single thread to continuously new and delete2 bytes, but that seems hard to test. And when the memory is really applied for, it is generally allocated 4k*n bytes, and I can hardly find the problem by simply observing the "commit memory" of the program.
-
@BoLin about what quantities are we actually talking about here ?
my program needs to use new and delete frequently
how frequently ? the OS is "smart" and as such it gets annoyed quite fast. If you constantly request and free memory it is going to say "Fine keep it! I'll check later when you have made up your mind."
wrote on 28 Jun 2023, 11:11 last edited by@J-Hilk Hello! I'm so glad you could reply. My program currently new and delete about ten times per second, which is about two bytes of new space each time, and then free up. About 3 seconds will be a new large space, about new510241024 bytes. I don't know how to tell if there are memory leaks and memory fragmentation for long running programs. Thank you very much for your answer.
-
@Chris-Kawa I took a look at the software today, but the specific usage is still not clear. Have you ever used this software to detect memory fragmentation? If so, is it convenient to tell what program is being used? Because I did test this, I was using a single thread to continuously new and delete2 bytes, but that seems hard to test. And when the memory is really applied for, it is generally allocated 4k*n bytes, and I can hardly find the problem by simply observing the "commit memory" of the program.
@BoLin said in Memory leaks and memory fragmentation:
Because I did test this, I was using a single thread to continuously new and delete2 bytes, but that seems hard to test
That's a pointless test. As you observed Windows allocates memory in pages, not bytes. Juggling 2 bytes in and out is basically nothing and will be in the same page all day long if you don't do any other allocations. The OS does not immediately page the memory out, so if you're reallocating the same small amount over and over the fragmentation does not increase significantly.
I haven't used MTuner for analyzing fragmentation, but I wouldn't be so sure it's the culprit.As a side note - if you know you're allocating this same 2 bytes over and over and you're worried about it so much then wouldn't it make sense to stop doing that? Keep it around and overwrite, or use small preallocated pool for it if you have multiple of those.
-
J JonB referenced this topic on 28 Jun 2023, 16:42
-
@BoLin said in Memory leaks and memory fragmentation:
Because I did test this, I was using a single thread to continuously new and delete2 bytes, but that seems hard to test
That's a pointless test. As you observed Windows allocates memory in pages, not bytes. Juggling 2 bytes in and out is basically nothing and will be in the same page all day long if you don't do any other allocations. The OS does not immediately page the memory out, so if you're reallocating the same small amount over and over the fragmentation does not increase significantly.
I haven't used MTuner for analyzing fragmentation, but I wouldn't be so sure it's the culprit.As a side note - if you know you're allocating this same 2 bytes over and over and you're worried about it so much then wouldn't it make sense to stop doing that? Keep it around and overwrite, or use small preallocated pool for it if you have multiple of those.
wrote on 29 Jun 2023, 00:14 last edited by@Chris-Kawa Is there an effective method for troubleshooting and monitoring memory fragmentation and memory leaks under Windows? How can I judge whether the current status is normal? I hope to receive your guidance
1/9