Program freezing when dealing with file

[TheDestroyer]

Hello guys, :)

I created a simple GUI program that does what is called "down-sampling". This program simply reads a huge file (at least 1 GB) and writes it after reducing its size by skipping some parts of it.

The problem is the following. Whenever I click the "Start" button for the process to start, the whole program freezes and becomes not responding till it finishes. I'd like to have the possibility of showing progress for the process or display something readable which is not possible due to the freezing problem. I'm using fstream to read and write the files. I tried to have the reading and writing functions independent of the GUI class, but this didn't help.

How would I do this? any ideas?

Thank you in advance!

[TheDestroyer]

Forgot to mention. The process is done as FIFO. So a unit is read from a file, and a smaller unit from it is written again to another file. It's not that the whole file is read into memory (which is impossible sometimes due to the size of the files that need this operation).

[Franzk]

The file processing blocks the event loop. There is a number of options for this

Make the worker threaded

Do the work in small steps, based on timer events

Within your loop, call QApplication::processEvents()

For more information read about "Keeping the GUI Responsive":http://doc.qt.nokia.com/qq/qq27-responsive-guis.html

[TheDestroyer]

Thanks for the fast answer, buddy!

I see the best solution for this is threading. Because it doesn't involve "pausing" the operation.

but anyway, doesn't using processEvents() make the process slower by a significant factor? does this also pause the operation? how does this "physically" work?

[Franzk]

[quote author="TheDestroyer" date="1305004111"]
I see the best solution for this is threading. Because it doesn't involve "pausing" the operation.[/quote]That is the one you should go for then. Be sure to read about the "intended use of QThread":http://labs.qt.nokia.com/2010/06/17/youre-doing-it-wrong/.

[quote author="TheDestroyer" date="1305004111"]but anyway, doesn't using processEvents() make the process slower by a significant factor? [/quote] Absolutely. GUI stuff is handled in between steps of your operation. If all you care about is keeping your GUI responsive, and the speed with which the operation is finished is not an issue, this is a possibility (although not recommended, but that's an other story).

[quote author="TheDestroyer" date="1305004111"]does this also pause the operation? how does this "physically" work?[/quote]Your operation gives the event loop space to process any events that may be pending. This means that your process can be interrupted by timer events, paint events and whatnot.

[andre]

It obviouly slows down a little bit, but not by too much. What happens is that when you call processEvents(), the eventloop processes any pending events by invoking the event handlers on the objects the events are meant for. That includes draw events. Note that you can filter which events you want processed, and you probably should do that. While the eventloop is bussy doing that (and the objects the event was send to are bussy handling those events (like draw events), your actual work is not being done. So yes, there is a pause, but it will probably be short.

Note that if you handle this with threading, the result is not all that much different (depending on the machine you run the program on). On a single core, threads are also run in chunks sequentially. Note that that is probably not an issue though, as I suspect your process will be IO-bound. That is: the reading and writing to disk is what is taking up the time, not the actual processing. The CPU is not all that active, I suspect (check with some system monitor). That means that you can probably yield control to another thread without loosing much or any performance.

"This wiki page":http://developer.qt.nokia.com/wiki/Threads_Events_QObjects explains a lot about the different options you have, and the most common pitfals with using threads in combination with signals and slots.

[TheDestroyer]

Thanks for the answers guys. I'm very grateful :D

I still have another question. I'm used to openMP for threading. Is QThread any better? I'm still new to Qt. In other words, is it worth it to start a new thing with QThread?

[Franzk]

I wouldn't say either of them is better. QThread fits better within the Qt development model, but that doesn't mean you have to use it. If you want a bit more abstraction from threads, look into QtConcurrent.

In the end you should go for what you prefer, so just experiment a bit with the options. You'll then find out what works best for you.

[TheDestroyer]

Thaaaaaaaaaaaaaaaaaaaanks :)

[andre]

I think for what you are doing: lineary going through a file, QtConcurrent is probably not the right tool for the job, though it is obviously possible. I think putting your worker code in a QObject, moving that object onto a QThread instance, and starting the reduction process using a signal would be the most straitforward way to do this. You can give the object signals to signal -process- progress, and connect to those from your GUI.

I have no experience with openMP, so I have no idea how it fits with Qt.

[TheDestroyer]

Thanks for the advice! I'll give it a shot :D