Error sending Iplimage using TCP

[ahmed kato]

I am trying to capture frames from a Logitech HD cam connected to a Raspberry Pi through usb, the RP is running arch linux and I am using OpenCV C api and a TCP client.

The TCP server is running c++(QT) under ubuntu.

I am storing the frame->imageData in a buffer and then send the data in the buffer.

this is a proof of concept that this idea works fine where tmpframe should be the new frame constructed on the server:

@CvCapture *capture = cvCaptureFromCAM(1);
cvNamedWindow( "mywindow", CV_WINDOW_AUTOSIZE );
int i =1;

   while ( 1 ) {
     // Get one frame
     IplImage* frame = cvQueryFrame( capture );
     CvSize size;
     size.height = 480;
  size.width = 640;
     IplImage* tmpframe = cvCreateImageHeader(size, IPL_DEPTH_8U, 3); //create a new frame
     if ( !frame ) {
       //fprintf( stderr, "ERROR: frame is null...\n" );
       getchar();
       break;
     }else
     {
         //std::cout<<i<<std::endl;
         char buffer[frame->imageSize];
         snprintf(buffer,frame->imageSize,"%s",frame->imageData);
         //printf("frame->height= %s\n",buffer);
         
         tmpframe->imageData = buffer;
         printf("data %s\n",tmpframe->imageData);
         
         //snprintf(IDbuffer,10,"%d",frame->ID);
         //printf("frame->ID= %s\n",IDbuffer);
         i++;
     }
     cvShowImage( "mywindow", tmpframe );@

actually this code is running perfectly,

Here is my client.c file:

@#include <opencv/cv.h>
#include <opencv/highgui.h>
#include <stdio.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <netdb.h>

void error(char *msg)
{
perror(msg);
exit(0);
}

int main(int argc,char *argv[])
{

int sockfd,portno,n;
struct sockaddr_in serv_addr;
struct hostent *server;
//char buffer[999999];

if(argc <3)
{
fprintf(stderr,"usage %s hostname portname port\n",argv[0]);
exit(0);
}

portno = atoi(argv[2]);
sockfd = socket(AF_INET , SOCK_STREAM,0);

if(sockfd < 0)
{
error("ERROR OPENING SOCKET");
}

server = gethostbyname(argv[1]);
if(server == NULL)
{
fprintf(stderr,"ERROR,NO SUCH HOST\n");
exit(0);
}

bzero((char*)&serv_addr,sizeof(serv_addr));
serv_addr.sin_family = AF_INET;

bcopy((char*)server->h_addr,(char*)&serv_addr.sin_addr.s_addr,server->h_length);
serv_addr.sin_port = htons(portno);

if(connect(sockfd,&serv_addr,sizeof(serv_addr)) < 0)
{
error("ERROR CONNECTING");
}
CvCapture *capture = cvCaptureFromCAM(1);
// Show the image captured from the camera in the window and repeat
int i =1;

   while ( 1 ) {
     // Get one frame
     IplImage* frame = cvQueryFrame( capture );
     if ( !frame ) {
       //fprintf( stderr, "ERROR: frame is null...\n" );
       getchar();
       break;
     }else
     {
         i++;
     }
     //bzero (buffer,999999);
     //strncpy(buffer,frame->imageData,sizeof(frame->imageData) -1);
     char *msg = frame->imageData;
     char buffer[frame->imageSize];
     bzero (buffer,frame->imageSize);
     snprintf(buffer,frame->imageSize,"%s",msg);

n=write(sockfd,buffer,frame->imageSize);
if(n <0)
{
error("ERROR READING FROM SOCKET");
}
//printf("%s\n",buffer);

}
return 0;
}
@

and this is how I am receiving data on my server:

@void HostConnector::readyRead()
{
QByteArray Data = socket->readAll();

CvSize size;
size.height = 480;
size.width = 640;
IplImage *frame = cvCreateImageHeader(size, IPL_DEPTH_8U, 3);

 
frame->imageData = Data.data();
cvShowImage( "mywindow", frame );

}@

But the server is crashing !!
a segmentation fault at @cvShowImage( "mywindow", frame );@

[mrdebug]

Have you tryed to send the QImage buffer instead of lplImage?
In this situation I have implemented h264 compression and http trasport (with FFMpeg, OpenCv and a custom http server using Qt).

[tobias.hunger]

What is the question?

How is this related to Qt at all?

[ahmed kato]

@mrdebug:

bq. Have you tryed to send the QImage buffer instead of lplImage?
In this situation I have implemented h264 compression and http trasport (with FFMpeg, OpenCv and a custom http server using Qt).

no I need to send some frames not a stream also I will perform some image proc before sending using opencv

@Tobias:

bq. What is the question?
How is this related to Qt at all?

I think that the error is in the QByteArray since the proof of concept is working fine, Qt is not receiving the correct data or I am using frame->imageData = Data.data(); wrong

[Scylla]

I think the problem is that the image size is larger than the tcp frame size (1500 bytes brutto). So you will get more then one package. You need the size of the data and send this first, so the other side knows how many data to receive. Thats the way I did it.

[koahnig]

Are you sure that the complete image has been received when you are doing your readAll?

readyRead signal is triggered when information is available. However, that does not mean that the information reception is already completed. With larger amount of data, it is likely that the signal is triggered several times. Also slow connection with fast computers make you receiving only portions of the data. In an extreme case the readyRead might be triggered with every byte received.

[ahmed kato]

@scylla
[quote author="Scylla" date="1357580551"]I think the problem is that the image size is larger than the tcp frame size (1500 bytes brutto). So you will get more then one package. You need the size of the data and send this first, so the other side knows how many data to receive. Thats the way I did it.[/quote]

I think you are right frame->imageSize = 921600.

[MuldeR]

Are you sure that code is okay? Especially the last line!

@CvSize size;
size.height = 480;
size.width = 640;
IplImage *frame = cvCreateImageHeader(size, IPL_DEPTH_8U, 3);
frame->imageData = Data.data();@

I'm not an OpenCV expert. But to my understanding, cvCreateImageHeader() will allocate and initialize the IplImage structure. The 'imageData' field of that struct will contain the pointer to the memory that has been allocated to store the image date. But your code does not copy the image data from the QByteArray's buffer into the memory buffer to which frame->imageData is pointing. Instead, you are overwriting(!) the pointer frame->imageData so that it now points to the QByteArray's internal data buffer! I'm not sure if it is intended by IplImage to reassign the 'imageData' pointer. And it obviously is a memory leak. But most important the pointer that's now in frame->imageData will become invalid(!) as soon as the QByteArray object is either destroyed or modified.

Summary: After your HostConnector::readyRead() returns, frame->imageData points to undefined memory !!!

--

See also:
[quote]char * QByteArray::data()
Returns a pointer to the data stored in the byte array [...] The pointer remains valid as long as the byte array isn't reallocated or destroyed. For read-only access, constData() is faster because it never causes a deep copy to occur.