[SOLVED]Problems with transfer



  • Hello to everyone who spend his own time to read this.

    I have some difficulties with file transfer. The byte array reach his point, but not entirely. Half bytes get with messge from client to server, but other half reach server after data stream has run low. I have an idea how fix this, but i can't get what's wrong? It's not in proffesional way to put patch on such thing, so i will be glad to get some explanations and instuгctions.

    Here is a code of client:
    @
    void Dialog::on_pbSendFile_clicked()
    {
    QString fileNameAndPath;
    fileNameAndPath=dialog.getOpenFileName(this,tr("Select file"),"/home/house15",tr("Text Files(*.txt *.doc .docx);;Image files(.png *.jpg *.jpeg *.gif *.bmp)"));
    sendFile= new QFile(fileNameAndPath);

        if(sendFile->open(QIODevice::ReadOnly))
            {
                QByteArray data;
                QDataStream out(&data, QIODevice::WriteOnly);
                out<<(quint16)0;
                out<<MyClient::comSendFile;
                QString s;//string for list of users, for which the message intends
                foreach(QListWidgetItem *l, ui->lw_users->selectedItems())
                    {
                        if(!l->text().isEmpty())
                            {
                                s+=l->text()+",";
                                s.remove(s.length()-1,1);
                            }
                    }
                 if(!s.isEmpty())
                     {
                         out<<s;
                         QString fileNameSendingFile = QFileInfo((*sendFile)).fileName();
                         out<<fileNameSendingFile;
                         out<<sendFile->size();
                         ui->progressBar->setRange(0,(int)sendFile->size());
                         ui->progressBar->setValue(0);
                         QByteArray sending_file;
                         quint64 send=0;
                          _sok->waitForBytesWritten();
                         while(!sendFile->atEnd())
                           {
                                sending_file=sendFile->read(SIZE_FOR_SENDING_FILE);
                                 out<<sending_file;
                                 send+= SIZE_FOR_SENDING_FILE;
                                ui->progressBar->setValue(send);
                            }
                         out.device()->seek(0);
                         quint16 sizeOfDataArray= (quint16)(data.size()-sizeof(quint16));
                         out<<sizeOfDataArray;
                         _sok->write(data);
                         AddToLog(tr("File was sended"),Qt::blue);
                     }
                 emit endOfTransmition();
            }
        else
            {
                QMessageBox::information(this,tr("Sending file"),tr("File cannot be open for reading"));
            };
    

    }@

    Here is code of server:
    @

    QDataStream in(_sok);
    in.setVersion(QDataStream::Qt_4_7);
    if (_blockSize == 0)
    {
    if (_sok->bytesAvailable() < (int)sizeof(unsigned long))
    {return;}
    in >> _blockSize;
    qDebug() << "_blockSize"<<tr(" now ") << _blockSize;
    }
    if (_sok->bytesAvailable() < _blockSize)
    { return;}
    else
    {
    _blockSize = 0;
    }

        quint8 command;
        in >> command;
        qDebug() << tr("Received command ") << command<<tr(" from ")<<this->_name;
        if (!_isAutched && command != comAutchReq)
        {return;}
    
        switch(command)
        {
           ***
    

    case comSendFile:
    {
    QString users_in;
    in>>users_in>>nameOfFile>>filesize;
    qDebug()<<"Now in stream: "<<nameOfFile<<" for "<<users_in<<" in size "<<filesize;
    QStringList listOfUsers = users_in.split(",");
    if(!listOfUsers.isEmpty())
    {
    QString savePath = "/***/";
    QDir dir;
    dir.mkpath(savePath);
    QString nameOfDir;
    nameOfDir=savePath+listOfUsers.first();
    if(!dir.exists(nameOfDir))
    dir.mkpath(nameOfDir);
    file=new QFile(nameOfDir+"/"+nameOfFile);
    file->open(QIODevice::WriteOnly);
    sizeReceiviedFile=0;
    char block[SIZE_BLOCK_FOR_SEND_FILE];
    while(!in.atEnd())
    {
    quint64 toFile = in.device()->read(block, sizeof(block));
    sizeReceiviedFile+=toFile;
    file->write(block,toFile);
    }

                            if(!listOfUsers.isEmpty())
                                if(listOfUsers.size()>1)
                                    {
                                        ***
                                    }
                        }
                    else
                        {
                            emit errorInFileTransfer();
                        }
                }
                break;
    
    }
    

    @

    Any info in additives to the above-stated under first requirement.

    Thanks for your attention in advance.



  • If I get it right you are sending all the file at once. So only a part of the file gets to the destination and the other part is lost or does the other part comes after a delay or what? I'm unable to figure out where the input socket is...



  • Other part comes after first, but not in same datastream. Didn't observe a delay. It's all about second part of file, which not in same datastream with first one. Why datastream with first part has ended before end file it self?

    Input socket in header file. I use QTcpSockets.



  • Some random thoughts on your code, that might or might not be related to your actual problem.

    Your while loop reading the file is superfluous. As you store everything in a byte array anyways, read it in one go:

    @
    out << sendFile->readAll();
    @

    Your progress bar won't be visible anyways, as reading to memory is super fast.

    Then, why do you poke around at the beginning of the byte array? Just store the little pieces in their variables of their own and send it using subsequent write calls. There's no need to introduce such error prone extra magic.

    Your user list will never contain a comma, you remove it immediately after you have added it. Why not collect the usernames in a [[Doc:QStringList]] and call join(",") on that in case it's not empty.

    You send the file only if the user list is not empty. Why do your users have to choose a file beforehand?

    To ensure that the last bytes written to your device are really sent, call flush() on the socket.

    And last, a debugging hint:
    Instead of sending to a socket, write your data to a file (or do both) and check if it contains the expected results.



  • [quote author="Volker" date="1323714461"]Some random thoughts on your code, that might or might not be related to your actual problem.

    Your while loop reading the file is superfluous. As you store everything in a byte array anyways, read it in one go:

    @
    out << sendFile->readAll();
    @

    Your progress bar won't be visible anyways, as reading to memory is super fast.

    Then, why do you poke around at the beginning of the byte array? Just store the little pieces in their variables of their own and send it using subsequent write calls. There's no need to introduce such error prone extra magic.

    Your user list will never contain a comma, you remove it immediately after you have added it. Why not collect the usernames in a [[Doc:QStringList]] and call join(",") on that in case it's not empty.

    You send the file only if the user list is not empty. Why do your users have to choose a file beforehand?

    To ensure that the last bytes written to your device are really sent, call flush() on the socket.

    And last, a debugging hint:
    Instead of sending to a socket, write your data to a file (or do both) and check if it contains the expected results.[/quote]

    Thanks for advice.

    Unfortunately ploblem still in there. Simptoms are all same - part of file in one datastream, other part in another... Brain storm in continue...



  • It's hard to say what's going wrong. Do you work with QThreads, BTW?



  • No, i don't... yet.

    Well, thanks anyway for suggestion. I solve this problem by simply adapting my server part for Reception of several data packets. It's work. Not perfectly, but work.

    Thanks everybody =)



  • By the way, maybe anybody can help with following from the aforesaid problem. It's a same transfer, but now it's about a content.

    I finally have transmeted file from client to server, but files reach the damaged. This critical for image files. On text files i can show example of corruption:

    Here is original:
    [Далеко не все диалоги удается вспомнить и записать. Часто мы ведем длинные разговоры, сидя в лодке или шагая по побережью. Чтобы "не потерять" высказанные Шаманом мысли, приходится записывать их уже своим текстом. Привожу примеры таких текстов, которые я планирую издать отдельно
    [49].]По публикации: Серкин В.П. Проблемы системы образования коренных народов Северо-Востока России в конце ХХ в. - Магадан: Кордис, 1998.

    And here is my problem:
    [Далеко не все диалоги удается вспомнить и записать. Часто мы ведем длинные разговоры, сидя в лодке или шаH°±їќLi¶hД.фяЗ¶X±±ї­¶гая по побережью. Чтобы "не потерять" высказанные Шаманом мысли, приходится записывать их уже своим текстом. Привожу примеры таких текстов, которые я планирую издать отдельно

    I think, even without knowledge of russian, you can see what thing is. What it can be? Coding? Data corruption?

    My current version code of client:
    @void Dialog::on_pbSendFile_clicked()
    {
    //AddToLog("Selected file: "+dialog.getOpenFileName());
    QString fileNameAndPath;
    fileNameAndPath=dialog.getOpenFileName(this,tr("Select file"),"/home/house15",tr("Text Files(*.txt *.doc .docx);;Image files(.png *.jpg *.jpeg *.gif *.bmp)"));
    sendFile= new QFile(fileNameAndPath);

        if(sendFile->open(QIODevice::ReadOnly))
            {
                QByteArray data;
                QDataStream out(&data, QIODevice::WriteOnly);
                out<<(quint16)0;
                out<<MyClient::comSendFile;
                QString s;
                foreach(QListWidgetItem *l, ui->lw_users->selectedItems())
                    {
                        if(!l->text().isEmpty())
                            {
                                s+=l->text()+",";
                                s.remove(s.length()-1,1);
                            }
                    }
                 if(!s.isEmpty())
                     {
                         out<<s;
                         QString fileNameSendingFile = QFileInfo((*sendFile)).fileName();
                         out<<fileNameSendingFile;
                         out<<sendFile->size();
                         ui->progressBar->setRange(0,(int)sendFile->size());
                         ui->progressBar->setValue(0);
                         char sending_block[SIZE_FOR_SENDING_FILE];
    
                         quint64 send=0;
                         while(!sendFile->atEnd())
                         {
                                 qint64 partOfFile= partOfFile= sendFile->read(sending_block,sizeof(sending_block));
                
                                out.writeRawData(sending_block,partOfFile);
                          }
                         out.device()->seek(0);
                         quint16 sizeOfDataArray= (quint16)(data.size()-sizeof(quint16));
                         out<<sizeOfDataArray;
    
                         _sok->write(data);
                         _sok->flush();
    
                         QString testfilelocaltion = "/home/house15/Desktop/MyMigratingFiles/testfile.txt";
                         QFile testFile&#40;testfilelocaltion&#41;;
                         testFile.open(QIODevice::WriteOnly);
                         testFile.write(data);
                         testFile.close();
                         AddToLog(tr("File was sended"),Qt::blue);
                     }
            }
        else
            {
                QMessageBox::information(this,tr("Sending file"),tr("File cannot be open for reading"));
            };
    

    }@

    Code of server is still same. Thanks.



  • It's hard to say what's going wrong. Is the contents of the file on the sending side ok?

    Are the payload and data sizes on the receiver's side correct?

    Also, make sure to not read to many bytes. You always read a block full of bytes if available. If your file does not end on a block boundary (which is very likeley) and if you're sending another file directly after the first one using the same socket, then you read the header and the start of the second file into your buffer and append it to your first file.

    The receiver code isn't robust at all. If you run into the receiver method and the socket did not yet receive all bytes, you will loose the file at best or, worse, read random garbage. As you have read the blocksize already, it will not be available an a second time entering the receiver code. The whole design of the receiver is broken.



  • Ok, now i have some progress.

    Answer on your questions:
    Payload and data sizes on receiver's side are correct. More than that - now it's one of conditions of a method ending of transmition.

    I have completely refused from pulling bytearray over part char array. Now it's a bytearray on both sides... and it's logical, i think =)

    Most importtant - code on receiver has been modified.

    But there is a still one small problem. After file was received, in beginning i have a strange symbols, which corrupt content of file, On text file it's not critical, that i will not tell about image.

    Some examples. Here is original:
    @Хохот шамана
    Владимир Серкин

    "Хохот шамана" - это книга психолога, который решил отправиться "в люди". Ему тесны границы не только академической психологии, но и целого ряда новых и новейших нетрадиционных психологических практик. Предлагаемое автором видение мира - это поиск необщих путей развития.@

    What i have in test file on sending side:
    @—ш
    c l i e n t 2 %>E>B (0<0=0 . t x t —µ —µХохот шамана
    Владимир Серкин

    "Хохот шамана" - это книга психолога, который решил отправиться "в люди". Ему тесны границы не только академической психологии, но и целого ряда новых и новейших нетрадиционных психологических практик. Предлагаемое автором видение мира - это поиск необщих путей развития.@

    And what on receiver:
    @—µХохот шамана
    Владимир Серкин

    "Хохот шамана" - это книга психолога, который решил отправиться "в люди". Ему тесны границы не только академической психологии, но и целого ряда новых и новейших нетрадиционных психологических практик. Предлагаемое автором видение мира - это поиск необщих путей развития.@

    My apology if you were tired of Russian =)

    Now i guess i khow what i think about - i think a same. It's a part of data from sending side about this message. But i stil don't khow how it get there.

    Code of sending and receiving side i will result further.



  • Code of sending side:

    @void Dialog::on_pbSendFile_clicked()
    {
    QString fileNameAndPath;
    fileNameAndPath=dialog.getOpenFileName(this,tr("Select file"),"/home/house15",tr("Text Files(*.txt *.doc .docx);;Image files(.png *.jpg *.jpeg *.gif *.bmp)"));
    sendFile= new QFile(fileNameAndPath);

        if(sendFile->open(QIODevice::ReadOnly&#41;)
            {
                QByteArray data;
                QDataStream out(&data, QIODevice::WriteOnly);
                out<<(quint16)0;
                out<<MyClient::comSendFile;
                QString s;
                foreach(QListWidgetItem *l, ui->lw_users->selectedItems())
                    {
                        if(!l->text().isEmpty())
                            {
                                s+=l->text()+",";
                                s.remove(s.length()-1,1);
                            }
                    }
                 if(!s.isEmpty())
                     {
                         out<<s;
                         QString fileNameSendingFile = QFileInfo((*sendFile)).fileName();
                         out<<fileNameSendingFile;
                         quint64 sendingfilesize=sendFile->size();
                         out<<sendingfilesize;
                         ui->progressBar->setRange(0,sendingfilesize);
                         ui->progressBar->setValue(0);
                        QByteArray sending_file;
                         sendFile->seek(0);
                        quint64 send=0;
                         while(!sendFile->atEnd())
                         {
                                 sending_file=sendFile->readAll();
                                 quint64 partOfFile=sending_file.size();
                                 out<<sending_file;
                                 send+= partOfFile;
                                ui->progressBar->setValue(send);
                          }
    
                         out.device()->seek(0);
                         quint16 sizeOfDataArray= (quint16)(data.size()-sizeof(quint16));
                         out<<sizeOfDataArray;
    
                         _sok->write(data);
                         _sok->flush();
    
                         QString testfilelocaltion = "/home/house15/Desktop/MyMigratingFiles/testfile.txt";
                         QFile testFile&#40;testfilelocaltion&#41;;
                         testFile.open(QIODevice::WriteOnly&#41;;
                         testFile.write(data);
                         testFile.close();
                         AddToLog(tr("Test file was create and now is full"),Qt::blue);
                     }
            }
        else
            {
                QMessageBox::information(this,tr("Sending file"),tr("File cannot be open for reading"));
            };
    

    }

    @



  • Code of receiver side:

    @void MyClient::onReadyRead()
    {
    QDataStream in(_sok);
    in.setVersion(QDataStream::Qt_4_7);
    if(sizeReceiviedFile<filesize)
    {receiveFile(in);}
    else
    {
    if (_blockSize == 0)
    {
    if (_sok->bytesAvailable() < (int)sizeof(unsigned int))
    {return;}

                        in >> _blockSize;
                        qDebug() << "_blockSize"<<tr(" now ") << _blockSize;
                    }
    
                if (_sok->bytesAvailable() < _blockSize)
                    { return;}
                else
                    {
                        //можно принимать новый блок
                        _blockSize = 0;
                    }
                quint8 command;
                in >> command;
                qDebug() << tr("Received command ") << command<<tr(" from ")<<this->_name;
    
                if (!_isAutched && command != comAutchReq)
                    {return;}
    
                switch(command)
                    {
                    ***
                    case comSendFile:
                        {
                            QString users_in;
                            in>>users_in>>nameOfFile>>filesize;
                            listOfUsers = users_in.split(",");
                            if(!listOfUsers.isEmpty())
                                {
                                    QString savePath = "/home/house15/Desktop/MyMigratingFiles/";
                                    QDir dir;
                                    dir.mkpath(savePath);
                                    QString nameOfDir;
                                    nameOfDir=savePath+listOfUsers.first();
                                    if(!dir.exists(nameOfDir))
                                        dir.mkpath(nameOfDir);
                                    file=new QFile&#40;nameOfDir+"/"+nameOfFile&#41;;
                                    file->open(QIODevice::WriteOnly);
                                    receiveFile&#40;in&#41;;
                                }
                            else
                                {
                                    emit errorInFileTransfer();
                                }
    
                        }
                        break;
                   ***
                    }
            }
    }
    

    void MyClient::receiveFile(QDataStream &in)
    {
    QByteArray block;
    while(!in.atEnd())
    {
    block= in.device()->readAll();
    quint64 toFile=block.size();
    sizeReceiviedFile+=toFile;
    file->write(block);
    }
    if(sizeReceiviedFile>=filesize)
    {
    if(listOfUsers.size()>1)
    {
    QFile *dublicateFile=new QFile[listOfUsers.count()-1];
    for(int i=0;i<listOfUsers.count()-1;i++)
    {
    dublicateFile = new QFile[i];
    dublicateFile->copy("/home/house15/Desktop/MyMigratingFiles/"+listOfUsers.first()+"/"+nameOfFile,"/home/house15/Desktop/MyMigratingFiles/"+listOfUsers.at(i+1)+"/"+nameOfFile);
    }
    }
    MyClient::endOfReceiveFile();
    }
    }@



  • Ok, i finally fugure out what was wrong with my sending file. Problem was in DataStream. Maybe it's a usual way for writing data in array through QDataStream - get starting write with "\0"-symbol, but i swear - it was a mystery for me.

    One unfortunate symbol in the file beginning was a symbol of the end of a file. His origin isn't known. When was starting data recording in a file, passing a stream, a problem was solved.

    Here some code. That stream creates an error:
    @QByteArray sending_file;
    sendFile->seek(0);
    quint64 send=0;
    while(!sendFile->atEnd())
    {
    sending_file=sendFile->readAll();
    quint64 partOfFile=sending_file.size();
    out<<sending_file;
    send+= partOfFile;
    ui->progressBar->setValue(send);
    }@

    And all of that was transform into:
    @ QByteArray data;
    QDataStream out(&data, QIODevice::WriteOnly);

                           ***
                                ui->progressBar->setRange(0,sendingfilesize);
                                ui->progressBar->setValue(0);
                                sendFile->seek(0);
                                quint64 send=0;
    
                                out.device()->seek(0);
                                quint16 sizeOfDataArray= (quint16)(data.size()-sizeof(quint16));
                                out<<sizeOfDataArray;
    
                                _sok->write(data);
                                _sok->flush();
                                out.device()->reset();
                                data.clear();
    
    
                                while(!sendFile->atEnd())
                                    {
                                        data=sendFile->readAll();
                                        quint64 partOfFile=data.size();
                                        send+= partOfFile;
                                        ui->progressBar->setValue(send);
                                    }
    
    
                                _sok->write(data);
                                _sok->flush();@
    

    Down with intermediaries =)

    Many thanks for that time that you have given to the decision of my problem. Awfully I apologize for the English - has just now seen, I am how much awful him I own.

    Once again thanks. I will hope it to somebody will help.


Log in to reply
 

Looks like your connection to Qt Forum was lost, please wait while we try to reconnect.