Handling large amounts of data

[clarkb7]

My app has to keep track of (potentially) several GB of data. This data is in the form of numbers (qreal) and strings (QString), which represents heap usage(allocs, staleness, etc), callstacks, etc of an application.

Currently I am storing structs on the heap to keep track of the data. Would switching to a model/view style benefit me at all? Does this framework provide methods to keep memory usage down?

Or should I use some sort of packing infrastructure to compress the data I don't need at the moment, and just keep a cache of recently accessed data.

Only a fraction of the data needs to be accessed at any given moment (1-15%), so overall memory usage is really what I'm concerned with.

Any other suggestions would be greatly appreciated, thanks.

[SGaist]

Hi,

What about using a database ? That would allow you to avoid using enormous amount of memory for unused data and use the model view paradigm.

If you need to keep a cache of current data, you could create a custom model to access them that could also fetch the data from the database as needed.

Hope it helps