QFile::size() returning incorrect value when reading through a LAN on Windows.
-
[quote author="Andre" date="1300030795"]Do you get the same issues if you map the shared folder to a network drive in Windows?[/quote]
I'll try that. The thing is that it's hard to debug since it happens randomly (or at least I don't know the trigger). Just been half an hour until it occurred again.
EDIT: Debugging as suggested. Mapping the shared folder to a Windows' Network Drive (Z:)
The buffer/flush issue is still there. Going to wait a few minutes to see if the size issue appears.
-
[quote author="ivan.todorovich" date="1300031823"]
[quote author="Andre" date="1300030795"]Do you get the same issues if you map the shared folder to a network drive in Windows?[/quote]I'll try that. The thing is that it's hard to debug since it happens randomly (or at least I don't know the trigger). Just been half an hour until it occurred again.[/quote]
Yeah, those are very hard to debug issues, I know. What I am wondering though, is if it is really a good idea to keep a file, that is being altered by another process and that you access over a network open for so long. Personally, I would think that that is a recepe for trouble. There are just too many things that can go wrong. Perhaps you could considder using a QFileSystemWatcher instead?
-
Ok, same thing.
size went out of sync using the network drive.
What's strange is that I was running 3 instances of the app:
Using Network Drive (Z:/file.log).
Using normal folder sharing (//192.168.1.102/shared/file.log).
Using local folder (Executing the app directly in the server).
Nº 1 failed six times. (seems even worst, probably but luck though)
Nº 2 failed two times. (although previous test show it can)
Nº 3 didn't failedSo, this is making me believe there is no trigger. (if it were, shouldn't it affect all instances at the same time?)
-
Just a thought... I assume it isn't an option to have a watcher app run locally on the server? That app could watch the logfile, and provide an interface to connect with over the network (perhaps using QxtRPCPeer or something like that?)
Might proof more reliable than directly accessing files over the network... I do realize that that would be a work-around, not a solution.
-
[quote author="Andre" date="1300032579"]
Yeah, those are very hard to debug issues, I know. What I am wondering though, is if it is really a good idea to keep a file, that is being altered by another process and that you access over a network open for so long. Personally, I would think that that is a recepe for trouble. There are just too many things that can go wrong. Perhaps you could considder using a QFileSystemWatcher instead?
[/quote]
Yes, you might be right. Although theoretically it should work..
I'll try using a QFileSystemWatcher and opening the file only as needed..I could even fix this doing something like this: (haven't test it though)
This way it will only parse lines on size change.@
int CLogWatcher::getNewLines()
{if (!m_file->isOpen()) { qWarning() << tr("LogWatcher: Log file is closed."); return -1; } // if this is 1, it means the log file has been cleared // and we have to start reading from the beggining if (m_file->pos() > m_file->size()) { qWarning() << tr("LogWatcher: File pointer was bigger than file size. -> %1/%2") .arg(m_file->pos()) .arg(m_file->size()); m_file->seek(0); }static int oldSize;
if (m_file->size() > oldSize) {int lineCount = 0; while (!m_file->atEnd()) { QString line = m_file->readLine(); if (!line.trimmed().isEmpty()) { lineCount++; emit gotLine(line); } } oldSize = m_file->size();}
return lineCount;}
@Since QFile::atEnd() checks for buffered data to return, maybe I'm getting the lines even before the file size attr is updated. (?)
Sounds strange, but that's the only theory I have right now. -
[quote author="Andre" date="1300033834"]Just a thought... I assume it isn't an option to have a watcher app run locally on the server? That app could watch the logfile, and provide an interface to connect with over the network (perhaps using QxtRPCPeer or something like that?)
Might proof more reliable than directly accessing files over the network... I do realize that that would be a work-around, not a solution. [/quote]
Not an option. In this particular case, the point of using LAN file access it to leave the server intact.
In a future, I need to be able to support ftp also. -
[quote author="ivan.todorovich" date="1300034188"][quote author="Andre" date="1300033834"]Just a thought... I assume it isn't an option to have a watcher app run locally on the server? That app could watch the logfile, and provide an interface to connect with over the network (perhaps using QxtRPCPeer or something like that?)
Might proof more reliable than directly accessing files over the network... I do realize that that would be a work-around, not a solution. [/quote]
Not an option. In this particular case, the point of using LAN file access it to leave the server intact.
In a future, I need to be able to support ftp also.[/quote]OK, just tossing ideas around :)
-
[quote author="ivan.todorovich" date="1300034077"]
[...]
Since QFile::atEnd() checks for buffered data to return, maybe I'm getting the lines even before the file size attr is updated. (?)
Sounds strange, but that's the only theory I have right now.
[/quote]
Following this theory, I'm trying this:@
int CLogWatcher::getNewLines()
{if (!m_file->isOpen()) { qWarning() << tr("LogWatcher: Log file is closed."); return -1; } // if this is 1, it means the log file has been cleared // and we have to start reading from the beggining if (m_file->pos() > m_file->size()) { qWarning() << tr("LogWatcher: File pointer was bigger than file size. -> %1/%2") .arg(m_file->pos()) .arg(m_file->size()); m_file->seek(0); } int lineCount = 0; //while (!m_file->atEnd()) { while (m_file->pos() < m_file->size()) { QString line = m_file->readLine(); if (!line.trimmed().isEmpty()) { lineCount++; emit gotLine(line); } } return lineCount;}
@So far so good
-
Bleh, same error.
And to declare it official - I HAVE NO CLUE OF WHY THIS IS HAPPENING >.<
How the hell can (m_file->pos() > m_file->size()) be TRUE if I only read on (m_file->pos() < m_file->size())
It makes no sense! (at least to me) :/ -
[quote author="Andre" date="1300040355"]Perhaps you should file a bugreport? [/quote]
Yes I will.. thanks for your help!
-
[quote author="Volker" date="1300056406"]I suspect it to be some caching problem on the OS level (could be some race condition too).[/quote]
I don't know what you mean by race condition but I agree on the OS caching. Although using the same algorithm works on PHP just fine (using filesize(), fopen(), feof() and fgets()) and on Ruby too (file(), filestats() to find the size, tell() for the cursor pos and readlines()), so maybe the framework should prevent this issue somehow.
Tomorrow I'll take a deeper look at QFile's source code to see if I can find something supicious (I doubt it though).
Also if the problem persists I'll try a different aproach. (Haven't tried QFileSystemWatcher as suggested by Andre yet..)
It's really annoying to debug, after the last modification it failed only ONCE, and then worked for 3-4 hours until I closed it without problems!