Skip to content
  • Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Groups
  • Search
  • Get Qt Extensions
  • Unsolved
Collapse
Brand Logo
  1. Home
  2. Special Interest Groups
  3. C++ Gurus
  4. [Solved] Allocate and read from memory and write to a File

[Solved] Allocate and read from memory and write to a File

Scheduled Pinned Locked Moved C++ Gurus
6 Posts 3 Posters 2.1k Views
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • P Offline
    P Offline
    pushpa416
    wrote on last edited by
    #1

    Hi There,

    I got the data stored in memory in a 2D format which is 8000rows and 70000Columns, both rows and columns are variable numbers, the maximum rows i can go up to 8192 and maximum columns can go upto 200000columns.

    now I have to read and store the data in a txt file for 8000rows and 70000Columns.

    By doing like following I could be able to read (8000rows and 33000Columns) or (5000rows and 63000columns) without any Problem.

    @
    unsigned short buf;
    int fd;
    buf = (unsigned short )malloc(6001024
    1024);

    fd = FOS_Open(0);
    FOS_Read_Data((uint32 *)buf, 0, 0, sel_last_column, sel_last_row, 2); // read function provided
    FOS_Close(fd);

    uint32 i,j;

    QFile res("cotdr.txt"); // writing the resultant array of values to a txt file
    if (res.open(QIODevice::WriteOnly | QIODevice::Truncate)) {
    QTextStream result(&res);
    for(i = 0; i < (lel_last_column)*(sel_last_row);i+=sel_last_column)
    {
    for (j=sel_first_column; j<(sel_last_column); j++)
    {
    result << buf[i+j] << QChar(',');
    }
    result << "\n";
    }
    free(buf);
    @

    everything is on Linux based system and i am doing all this under push_button_clicked.
    if at all I increase the malloc size or number of rows or number of columns, Gui Crashes immeadiately with the above code.
    Could someone please suggest me how i could overcome this, example is much appreciated.

    Thanks

    [edit: added missing coding tags @ SGaist]

    1 Reply Last reply
    0
    • L Offline
      L Offline
      leonardo.ramosantos
      wrote on last edited by
      #2

      Hi there,

      Crash can happen because malloc gets a wide free memory space. Thus, malloc will look for an area of memory that fits the size you want. However, you may not have all the space you want, then your system goes down.
      What you can do is test the buf variable to make sure if malloc worked. Try this:

      @ buf = (unsigned short ) malloc(60010241024);
      if (buf) {
      /
      here you put your code */
      } else {
      qDebug() << "malloc didn't work!!!";
      }@

      If Buf is not valid, in fact, malloc failed to allocate memory.

      I hope I have helped you.

      1 Reply Last reply
      0
      • SGaistS Offline
        SGaistS Offline
        SGaist
        Lifetime Qt Champion
        wrote on last edited by
        #3

        Hi,

        Are you also aware that you are trying to allocate 600MB at a time ? Is your application 32bit ? Do you have other such allocation in it ?

        Also, since you need to re-parse the data anyway, why not read it one row at a time ?

        Interested in AI ? www.idiap.ch
        Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

        1 Reply Last reply
        0
        • P Offline
          P Offline
          pushpa416
          wrote on last edited by
          #4

          Hi both,

          when i print the buf message it doesn't show the fail message still failed(Gui Exits). it is a 32 bit system. i don't have any other allocations of 600MB.

          SGaist, does reading one row at time means (used while loop instead of for loop still fails), sorry I am bit new to the programming(believe that was correct).

          How Else i can read the data without allocating 600MB? if you don't mind could you please give me an example?

          Thanks,

          1 Reply Last reply
          0
          • SGaistS Offline
            SGaistS Offline
            SGaist
            Lifetime Qt Champion
            wrote on last edited by
            #5

            I don't know how FOS_Read_Data works but yes the idea is to call it repeatedly to get the data you want line by line.

            Also, your current allocation is a bit strange

            @
            unsigned short buf;
            buf = (unsigned short )malloc(6001024
            1024); // You don't take into account the size of short

            fd = FOS_Open(0);
            FOS_Read_Data((uint32 *)buf, 0, 0, sel_last_column, sel_last_row, 2); // You are telling that the buffer is uint32 however from your allocation it should be unsigned short which size is not be the same as uint32
            @

            Interested in AI ? www.idiap.ch
            Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

            1 Reply Last reply
            0
            • P Offline
              P Offline
              pushpa416
              wrote on last edited by
              #6

              Hi SGaist,

              Thank you for your help, in the FOS_Read_Data structure the buf was defined as default unsigned32, so i have to use as it is. and the values stored in the memory was 16bit value so i was used unsigned short to read. but anyway i could be able to solve it finally.

              Thank you

              1 Reply Last reply
              0

              • Login

              • Login or register to search.
              • First post
                Last post
              0
              • Categories
              • Recent
              • Tags
              • Popular
              • Users
              • Groups
              • Search
              • Get Qt Extensions
              • Unsolved