Freeing file handles created by Qt
-
Moving everything into a VM should be a last resort. The kernel (SYSTEM PID 4) has an open handle to stockfish.exe (I cannot find any process accessing stockfish.exe, at least not with process explorer). I did launch stockfish both from the cmd line and arena (that launched stockfish) but closed them afterwards (none of these processes are open).
Local debugging in Qt worked fine but execution coverage was poor, so I tried running stockfish from arena and attaching gdb using Qt. None of the breakpoints were reached, instead code execution got into ntdll.dll at which it became unresponsive (F10 and F11 did no longer work). From that point I could no longer do interact with stockfish.exe.
I would expect that you can find stockfish.exe or arena in your task list then. I just tried process explorer and I could kill a process launched by another application. However, typically I am using task manager, because I am used to it.
Since I do not know arena nor stockfish I cannot really help here. However, I think that the problem most likely stems from arena and that it does not handle things correctly. Also gdb may still not release everything.
-
Hi
Im not really sure what handles you refer to ?
If you mean being able to override the running exe with a new one, then it normally works fine.
There is also this option, you can check if on
You can try setting it to all.
-
Hi
Im not really sure what handles you refer to ?
If you mean being able to override the running exe with a new one, then it normally works fine.
There is also this option, you can check if on
You can try setting it to all.
@mrjj @koahnig
I checked my config and it seems correct (stop applications for the same project is a default setting (at least in my case)). I did attach gdb in Qt to stockfish.exe and never got to any breakpoints (that were reachable when running stockfish from Qt) but instead got into ntdll.dll after which Qt became unresponsive to step commands. Is there a better way of attaching gdb to a process? (without losing the handle to it (process explorer shows that the kernel still has an open but invalid handle to the binary)).This is a known issue in 64-bit versions of windows but there does not seem to be any workarounds or fixes for win 10.
I guess this case is most similar with this answer.
-
@mrjj @koahnig
I checked my config and it seems correct (stop applications for the same project is a default setting (at least in my case)). I did attach gdb in Qt to stockfish.exe and never got to any breakpoints (that were reachable when running stockfish from Qt) but instead got into ntdll.dll after which Qt became unresponsive to step commands. Is there a better way of attaching gdb to a process? (without losing the handle to it (process explorer shows that the kernel still has an open but invalid handle to the binary)).This is a known issue in 64-bit versions of windows but there does not seem to be any workarounds or fixes for win 10.
I guess this case is most similar with this answer.
@Sebi1729
I never seen this issue on the forum so I have zero idea what is causing this.
Does stockfish do anything a normal desktop app would not do ?
Deep systems calls?
You could try the debugger log if not tried that already
With Visual studio, Q seems to kill debugging but not sure if that applies to your case.
(since its the mingw version) -
@Sebi1729
I never seen this issue on the forum so I have zero idea what is causing this.
Does stockfish do anything a normal desktop app would not do ?
Deep systems calls?
You could try the debugger log if not tried that already
With Visual studio, Q seems to kill debugging but not sure if that applies to your case.
(since its the mingw version) -
@mrjj
I noticed that my Views tab us greyed out. Where does the Qt store logs by default? Stockfish at most writes to files in a local directory; it doesn't have any fancy logic. -
@mrjj
I noticed that my Views tab us greyed out. Where does the Qt store logs by default? Stockfish at most writes to files in a local directory; it doesn't have any fancy logic.@Sebi1729 Did you try to disable your anti-virus application?
-
@mrjj
I noticed that my Views tab us greyed out. Where does the Qt store logs by default? Stockfish at most writes to files in a local directory; it doesn't have any fancy logic.@Sebi1729
Windoze is notorious for having file handles left open/locked preventing overwrite, and it often does boil down to some AV. Make sure any & all AV is disabled and test.You really ought to be able to use something like
procexpto see what executables/handles are open by whom etc. If you have runstockfish.exeit only takes one thread inside it to still be running to prevent deletion. Try a littlekillprogram to see if that terminates it correctly if it's still running.The other possibility (hack) to get you going is that often Windoze allows open file rename where it does not allow delete/overwrite. You might try renaming to a temp file prior to re-link.
-
@Sebi1729
Windoze is notorious for having file handles left open/locked preventing overwrite, and it often does boil down to some AV. Make sure any & all AV is disabled and test.You really ought to be able to use something like
procexpto see what executables/handles are open by whom etc. If you have runstockfish.exeit only takes one thread inside it to still be running to prevent deletion. Try a littlekillprogram to see if that terminates it correctly if it's still running.The other possibility (hack) to get you going is that often Windoze allows open file rename where it does not allow delete/overwrite. You might try renaming to a temp file prior to re-link.
@JNBarchan said in Freeing file handles created by Qt:
Windoze is notorious for having file handles left open/locked preventing overwrite, and it often does boil down to some AV.
Actually it really boils down to "programmers" that don't take care to run a graceful shutdown sequence. Those people, from my observations, are often one and the same as the ones that believe leaking memory from global scope variables on program exit is just fine "because the OS will clean it up".
@Sebi1729 said in Freeing file handles created by Qt:
Is there a better way of attaching gdb to a process?
You can attach QtCreator, from the menu
Debug > Start Debugging > Attach to Running Application ...select the process from the list. Make sure you're using a debug build so you get the debug information. -
Yes, I disabled all antiviruses I was aware of.
@JNBarchan
I managed to close the handle using process hacker (it comes with a driver which can close any open handle). However, this is not a long term solution since handles are recycled on windows (which likely ends up in file corruption).
I tried this in two cases; attaching gdb to stockfish before starting it and while running. The end result was the same: I got into ntdll.dll and started stepping through machine level code (F10/F11) and at some point gdb started to hang. Upon exit I could no longer reattach nor rebuild stockfish. Source code was not available during step by step debugging. Right now I am thinking of recording a few games and replaying the UCI transcript to stockfish from Qt (in the first case I was launching stockfish with an external application (chess) Arena).
-
Yes, I disabled all antiviruses I was aware of.
@JNBarchan
I managed to close the handle using process hacker (it comes with a driver which can close any open handle). However, this is not a long term solution since handles are recycled on windows (which likely ends up in file corruption).
I tried this in two cases; attaching gdb to stockfish before starting it and while running. The end result was the same: I got into ntdll.dll and started stepping through machine level code (F10/F11) and at some point gdb started to hang. Upon exit I could no longer reattach nor rebuild stockfish. Source code was not available during step by step debugging. Right now I am thinking of recording a few games and replaying the UCI transcript to stockfish from Qt (in the first case I was launching stockfish with an external application (chess) Arena).
@Sebi1729 said in Freeing file handles created by Qt:
I got into ntdll.dll and started stepping through machine level code (F10/F11) and at some point gdb started to hang.
You should get access to the call stack when you attach to a debug build, you can then inspect what called what.
Upon exit I could no longer reattach nor rebuild stockfish.
Who's exit? After you've attached the debugger you can close the instance by
Shift+F5, but if the program has acquired a global mutex/semaphore or any other global resource this may be left uncleaned (although the debugger/OS should do it in principle).Right now I am thinking of recording a few games and replaying the UCI transcript to stockfish from Qt (in the first case I was launching stockfish with an external application (chess) Arena).
What exactly are you trying to do?
-
Yes, I disabled all antiviruses I was aware of.
@JNBarchan
I managed to close the handle using process hacker (it comes with a driver which can close any open handle). However, this is not a long term solution since handles are recycled on windows (which likely ends up in file corruption).
I tried this in two cases; attaching gdb to stockfish before starting it and while running. The end result was the same: I got into ntdll.dll and started stepping through machine level code (F10/F11) and at some point gdb started to hang. Upon exit I could no longer reattach nor rebuild stockfish. Source code was not available during step by step debugging. Right now I am thinking of recording a few games and replaying the UCI transcript to stockfish from Qt (in the first case I was launching stockfish with an external application (chess) Arena).
@Sebi1729 said in Freeing file handles created by Qt:
@JNBarchan
I managed to close the handle using process hacker (it comes with a driver which can close any open handle). However, this is not a long term solution since handles are recycled on windows (which likely ends up in file corruption).
The intention was not necessarily to repeatedly use an external close-handle-program, it was to identify which process has the handle open to diagnose the issue.
-
@JNBarchan said in Freeing file handles created by Qt:
Windoze is notorious for having file handles left open/locked preventing overwrite, and it often does boil down to some AV.
Actually it really boils down to "programmers" that don't take care to run a graceful shutdown sequence. Those people, from my observations, are often one and the same as the ones that believe leaking memory from global scope variables on program exit is just fine "because the OS will clean it up".
@Sebi1729 said in Freeing file handles created by Qt:
Is there a better way of attaching gdb to a process?
You can attach QtCreator, from the menu
Debug > Start Debugging > Attach to Running Application ...select the process from the list. Make sure you're using a debug build so you get the debug information. -
Stockfish is open source, so you can build and modify it yourself. I really don't get why you're trying to reverse engineer the engine ...
-
Stockfish is open source, so you can build and modify it yourself. I really don't get why you're trying to reverse engineer the engine ...
@kshegunov
I'm not trying to reverse engineer per-se but rather have a deeper understanding on how moves are computed given a game state; I'm planning on improving the engine with some machine learning algorithms. -
Merge double posts later. Apparently, McAfee can't exclude directories from scanning. It even blocks the linker from visual studio thus making any windows box useless for (binary application) development.
I fixed it by:
-
uninstalling McAfee
-
flaming the cretins who thought this was a good idea
-
