How to pass a 2D array without memory issue ?



  • Hello,
    i am working on image analysis.
    I need to transfert the pixels values into an array for specific grid analysis.

    So i have a function in an external class called gdalfile like this :

    GDAL FILE**

    float **gdalfile::GetFileAsAnArray()
     {
    
    .....
         float *pafScanline;
    .....
     // nbPixel = nb column * nb line
    //pafScanline  is a buffer used by the gdal librairie to get the image value
         pafScanline = (float *) CPLMalloc(sizeof(float)*nbPixel);
    
    // GDAL function which put the pixel value in pafScanline 
     ....
    //--> SO FAR NO PROBLEM
    
    
    // Creat the array
     int count(0);
     int nColonnes=nXSize;
     int  nLignes=nYSize;
    
    //dynamic 2D array
     float **t;
      t = new float* [ nLignes ];
      for (int i=0; i < nLignes; i++)
        t[i] = new float[ nColonnes ];
    
     // fill the array with pafScanline value
      for (int i=0; i < nLignes; i++)
        for (int j=0; j < nColonnes; j++)
        {
        t[i][j] = pafScanline[count];
        count++;
        }
    
      return t ;
    

    Here is the main.cpp
    MAIN**

    //constructors 
    ........
    
    // Get the array values
    
        float** p= gdalTest.GetFileAsAnArray();
    
    //Some analysis
         for (int i=0; i < gdalTest.getYLine(); i++)
           for (int j=0; j < gdalTest.getXColonne(); j++)
           {
    
           p[i][j]= .......
    ......
    
           }
    

    After around 100000 loop, i got this :
    ERROR 2: CPLMalloc(): Out of memory allocating 65536 bytes.

    I guess this comes from the way i pass mya array.

    Could you help me ?

    Regards


  • Moderators

    @TremblayGIS What loop do you mean? Do you have a loop in main where you constantly allocating memory? You can just free the memory before allocating new:

    for(...) {
    float** p= gdalTest.GetFileAsAnArray();
    
    //Some analysis
    for (int i=0; i < gdalTest.getYLine(); i++)
    for (int j=0; j < gdalTest.getXColonne(); j++)
    {
    
       p[i][j]= .......
    }
    delete[] p;
    }
    ......
    
       }
    
    

  • Moderators

    @TremblayGIS If this array always has same dimensions, then no need to allocate all the time: just allocate once and then fill it with data.



  • p[i][j]= ....... the problem is somewhere in the ....... part. Looks like you are having a memory leak that is eating away all your available RAM.

    on the other hand, this is 2017 and it's C++, why are you using such a convoluted code?

    #include <boost/multi_array.hpp>
    #include <memory>
    #include <cassert> //optional, for assert()
    boost::multi_array<float, 2> gdalfile::GetFileAsAnArray(){
    const int nColonnes=nXSize;
    const int  nLignes=nYSize;
    std::unique_ptr<float[]> pafScanline(new float[nbPixel]);
    // GDAL function which put the pixel value in pafScanline (use pafScanline.get() for the raw pointer)
    assert(nLignes*nColonnes<=nbPixel);
    boost::multi_array<float, 2> t(boost::extents[nLignes][nColonnes]);
    int count=0;
    for (int i=0; i < nLignes; i++)
        for (int j=0; j < nColonnes; j++)
            t[i][j] = pafScanline[count++];
    }
    return t;
    }
    

    or if you don't want to use boost:

    #include <vector>
    #include <memory>
    #include <cassert> //optional, for assert()
    std::vector< std::vector<float> > gdalfile::GetFileAsAnArray(){
    const int nColonnes=nXSize;
    const int  nLignes=nYSize;
    std::unique_ptr<float[]> pafScanline(new float[nbPixel]);
    // GDAL function which put the pixel value in pafScanline (use pafScanline.get() for the raw pointer)
    assert(nLignes*nColonnes<=nbPixel);
    std::vector<std::vector<float> > t(nLignes, std::vector<float>(nColonnes));
    int count=0;
    for (int i=0; i < nLignes; i++)
        for (int j=0; j < nColonnes; j++)
            t[i][j] = pafScanline[count++];
    }
    return t;
    }
    

    @jsulm said in How to pass a 2D array without memory issue ?:

    delete[] p;

    Small note, malloc needs free, not delete



  • Thank you so much for your help
    i'm gone a try that !

    regards



  • Hello, in fact i tried tis


    cpp :


    ...

    // 2D : column & lines dimensions
         const int nColonnes;
         const int  nLignes;
    
     //array. nbPixel = nLignes*nColonnes
        float pafScanline[nbPixel];
    
        // fill pafScanline with trivial value
          for(int i(0); i<nbPixel; ++i)
          {
              pafScanline[i] = i*3 ;
          }
    
    //fill  vector with pafScanline  
     std::vector<std::vector<float> > t(nLignes, std::vector<float>(nColonnes));
     int count=0;
     for (int i=0; i < nLignes; i++)
         for (int j=0; j < nColonnes; j++)
            {
             t[i][j] = pafScanline[count++];
            }
    
    return t
    

    main :


    int compteur(0);
    std::vector< std::vector<float> > p;
    p= gdalTest.GetFileAsAnArray();
    
     for (int i=0; i < gdalTest.getYLine(); i++)
       for (int j=0; j < gdalTest.getXColonne(); j++)
       {
       p[i][j]= p[i][j]+1500;
       compteur++;
           qDebug()<< p[i][j];
       }
    
    

    but after a while my program crashed (compteur = about 100 000 ) i got a crashed. i gess it's due to a memory issue.

    so i tried this to optimize the memory by passing the reference not the values :


    CPP :


    std::vector<std::vector<float> >* gdalfile::GetFileAsAnArray()
     {
    // 2D : column & lines dimensions
         const int nColonnes;
         const int  nLignes;
    
     //array. nbPixel = nLignes*nColonnes
        float pafScanline[nbPixel];
      
        // fill pafScanline with trivial value
          for(int i(0); i<nbPixel; ++i)
          {
              pafScanline[i] = i*3 ;
          }
    
    std::vector<std::vector<float> >* t(nLignes, std::vector<float>(nColonnes)); 
    
     int count=0;
    //loop to fill the vector
     for (int i=0; i < nLignes; i++)
     {
           t[i]=new std::vector<float>(nColonnes);
         for (int j=0; j < nColonnes; j++)
            {
             t[i][j] = pafScanline[count++];
            }
    }
    
      return t ;
    

    I GOT THIS ERROR :
    // D:\Documents\codeblocks\testTableau\gdalfile.cpp|43|error: expression list treated as compound expression in initializer [-fpermissive]|
    //D:\Documents\codeblocks\testTableau\gdalfile.cpp|43|error: cannot convert 'std::vector<float>' to 'std::vector<std::vector<float> >*' in initialization|

    would you have some ideas about what is wrong. I just want to pass the reference of the vector in order to avoid to copy it.

    Regards



  • this std::vector<std::vector<float> >* t(nLignes, std::vector<float>(nColonnes)); should become auto* t=new std::vector<std::vector<float> >(nLignes, std::vector<float>(nColonnes)); and this t[i]=new std::vector<float>(nColonnes); should be t[i]=std::vector<float>(nColonnes); but this will not improve anything in your program. std::vector is move constructable and move assignable so it doesn't make a copy when you call p= gdalTest.GetFileAsAnArray();

    your problem is somewhere else as the piece of code that crashes does not allocate any memory. Since you removed all my assertions from your code I'll have to ask you to check what nbPixel , nLignes and nColonnes contain



  • Ok thank you for your prompt help
    i'm gone a try that
    regards



  • @TremblayGIS said in How to pass a 2D array without memory issue ?:

    i'm gone a try that

    Try what? my point was that you code does not need to be changed but that the error was coming from somewhere else, like nbPixel ,nLignes or nColonnes


Log in to reply
 

Looks like your connection to Qt Forum was lost, please wait while we try to reconnect.