keep sensible information in ram in case of power outage



  • My application works with sensible data in RAM (a bunch of different variable types). Such data changes very often (roughly every second) and must not be lost in case of power outage. Once power and application is up again, data must be restored from somewhere. What should I do? which is my best option on a standard x86 platform?

    • Keep data in RAM and save it continuously to disk? (doesn't it degrade disk life cycle?)
    • Keep data in a file and access the file directly to write and read? (isn't it slow and also degrade life cycle?)
    • Use mapped file to virtual memory? (how do I know when SO writes data in phisical memory to disk?)

    I am also looking for other options that probably I dont know I can make us of.
    Thank you very much


  • Qt Champions 2016

    Hi and welcome

    I assume having a UPS is out of the question ?

    Some notes:
    Writing on SSD/flash type storage will kill it over time
    but for say a sata disk, continuously writing is not (that much) a problem.

    Writing to disk directly , can trash your file in case of power outage if you in the middle of writing it.
    Some file systems like ext3-4 provides protection/helps with this issue.



  • You will need some sort of hardware solution to make this work unless you narrow down the kinds of failures.

    Whatever you write to the disk may not actually make it there if the failure happens at a bad time. If disk caching is enabled whatever you write is actually sitting in memory for a period of time and not actually written to the disk. A failure while you are writing the file is pretty much a guarantee the backup will be useless.

    If you are only concerned about the software state and not from hardware (software shuts down or closes and it must restart as though it never shut off) then you can look for other methods to deal with this. Maybe a second application/service that runs on the computer which keeps a copy of the data in its own memory space? On startup you seek out this running software and get your data. Periodically you send your data to this other program.

    A UPS is a good solution for hardware (a common failure for hardware is a loss of power). Using a laptop that can run on internal batteries is the same idea as having a UPS.



  • Write the new file with a new name, something iso 8601-ish. Include a checksum so you can verify the file contents, and decide to go to the next backup file when reading if the file is corrupted. This way you'll not be overwriting the same sectors over-and-over in a HDD.
    You might need to read the documentation of the OS to create un-cached files, if that is even possible.
    Do not acknowledge the data before you've actually written the data.
    Every now and then you might want to delete older files.

    The only data you can lose is the data actually being written to disk, I'd estimate the age of such data a few 100 mS old.
    Typical mutexes problem. The data is not valid between lock&unlock, and is never valid when locked.
    You might want to take a look at Git, they use the filesystem as database, and it's almost impossible to corrupt the database by killing the process, taking out the disk or switching off power. It can always recover itself to the state before the interrupted modification.

    But, what is datasource, and how many data is involved? kB's, MB's, hundreds of MB's?


  • Moderators

    @ivanperino said:

    My application works with sensible data in RAM (a bunch of different variable types). Such data changes very often (roughly every second) and must not be lost in case of power outage. Once power and application is up again, data must be restored from somewhere. What should I do?

    I second @Rondog's final solution.

    To protect against power loss, it is better to use a hardware solution than a software solution. Use an Uninterruptible Power Supply.


  • Moderators

    Interestingly no one came up with the most obvious answer today: Put it in The Cloud(TM)! SCNR


  • Qt Champions 2016

    @Wieland
    Well the "sensible data" statement made me drop the idea :)



  • I want sincerelly to say thankyou all. All comments from you all have been helpful.
    I omitted some information just to avoid a large post.

    • UPS is not a posibility.
    • Data must be retained locally because of certification constraints.
      Similar systems use NVRAM device (maybe through PCIe) mounted as a HD.

    I was trying to preclude the use of such a device.
    Thank you all again, this forum REALLY help.


Log in to reply
 

Looks like your connection to Qt Forum was lost, please wait while we try to reconnect.