How to replace certain text in a file as a QMAKE_POST_LINK action?

[Bart_Vandewoestyne]

As part of a QMAKE_POST_LINK action, I would like to replace certain strings in a file with other strings. A Google search brought me to https://doc.qt.io/qt-5/qmake-function-reference.html but for as far as I can see and understand, these functions are useful if you want to replace certain strings when generating the build files out of your .pro file. What I want to do is a platform-independent QMAKE_POST_LINK action that replaces certain strings in a text file. Can anyone point me in the right direction? Does qmake provide anything like that?

[Bart_Vandewoestyne]

In the meanwhile, I have found that in features/spec_post.prf there exists a QMAKE_STREAM_EDITOR variable, so I tried

QMAKE_POST_LINK += $$QMAKE_STREAM_EDITOR -i 's/foo/footest/' ${DST_FILE}$$escape_expand(\n\t)
QMAKE_POST_LINK += $$QMAKE_STREAM_EDITOR -i 's/bar/bartest/' ${DST_FILE}$$escape_expand(\n\t)

but if I check the Post-Build Event in my VS2015 project, I see that the above is transformed to

$(QMAKE) -install sed -i s/foo/footest/ C:\some_file.txt
$(QMAKE) -install sed -i s/bar/bartest/ C:\some_file.txt

which results in errors when building the project. From the error, I see that the QMAKE variable is not expanded... so either that is the problem, or there is something else wrong with the sed commands. I have no idea why the QMAKE variable is not expanded... shouldn't qmake know about that?

[Bart_Vandewoestyne]

Looking at the doSed() function in https://github.com/qt/qtbase/blob/5.12/qmake/main.cpp my educated guess is that the syntax that I need is

QMAKE_POST_LINK += $$QMAKE_STREAM_EDITOR -e 's/foo/test/g' $$shell_path($${DST_FILE})$$escape_expand(\n\t)

However, two problems remain:

1. The above gets expanded to

   $(QMAKE) -install sed -e s/foo/test/g C:\some_file.txt
   if errorlevel 1 goto VCEnd
   

   in my .vcxproj file and it does not work (maybe because the $(QMAKE) does not get expanded, maybe because the if errorlevel line is not on the same line... any ideas here???)
2. If I use QMAKE_QMAKE -install sed instead of $$QMAKE_STREAM_EDITOR and replace my QMAKE_POST_LINK action by

   QMAKE_POST_LINK += $$QMAKE_QMAKE -install sed -e 's/foo/footest/g' $$shell_path($${DST_FILE})$$escape_expand(\n\t)
   

   then I do have my qmake.exe and it looks like the post link sed replace action gets executed, but it appears like I'm in an infinite loop... and I don't understand why.

[Bart_Vandewoestyne]

I've investigated this a bit further by trying to get qmake's sed implementation work on my Windows 10 cmd window. This is what I tried:

I have a file with some content:

C:\SVN>type test.txt
Bart

I'm using qmake from Qt 5.12.2 (we are building Qt ourselves)

C:\SVN>qmake --version
QMake version 3.1
Using Qt version 5.12.2 in
C:/SVN/foo/bar/blah/blash/ThirdParty/Qt/qt-install/lib

Then I try to use qmake's sed implementation using the following invocations:

C:\SVN>qmake -install sed -e 's/Bart/test/g' test.txt
Error: unrecognized sed command '''

That one fails, but the following two ones seem to 'work', but with
strange behavior:

C:\SVN>qmake -install sed -e "s/Bart/test/g" test.txt

or

C:\SVN>qmake -install sed -e s/Bart/test/g test.txt

The last two invocations seem to trigger something, but instead of
just seeing 'test' on my standard output, it seems like the program is
in an infinite loop and I don't really see anything printed to the
output... all I see is a cursor that blinks and flashes...

Is my invocation of qmake's sed command correct? What am I still doing wrong here?

[aha_1980]

@Bart_Vandewoestyne I have no idea, but I'd look in the generated Makefile. Maybe you find a hint there what's going on...

[Bart_Vandewoestyne]

Apparently, it has to do with the line endings of the file test.txt. If the line endings are Windows line endings as follows:

$ hexdump.exe -c test.txt
0000000   B   a   r   t      \r  \n   T   e   s   t  \r  \n
000000d

then the command

qmake -install sed -e s/Bart/Bert/g test.txt

does not work. If the line endings are unix-style line endings like:

$ hexdump.exe -c test.txt
0000000   B   a   r   t      \n   T   e   s   t  \n
000000b

then it works:

C:\>qmake -install sed -e s/Bart/Bert/g test.txt
Bert
Test

I tried changing line 157 in https://github.com/qt/qtbase/blob/5.12/qmake/main.cpp from

        } else if (!(f = fopen(inFile, "r"))) {

to

        } else if (!(f = fopen(inFile, "rt"))) {

but that did not solve the problem :-( Any further suggestions?

[Kent-Dorfman]

but that did not solve the problem :-( Any further suggestions?

Don't use non-platform-native tools. sed is a UNIX command. If you are working under windows then make sure you are only using windoze tools. As for cygwin...I have no idea whether it expects input files to be in win or unix formats because I don't use it if at all possible. other possible ideas: convert all your text files into one format and make sure all the tools recognize them: dos2unix and unix2dos

[Bart_Vandewoestyne]

@Kent-Dorfman said in How to replace certain text in a file as a QMAKE_POST_LINK action?:

Don't use non-platform-native tools. sed is a UNIX command. If you are working under windows then make sure you are only using windoze tools.

For as far as I understand it $$QMAKE_STREAM_EDITOR and friends (QMAKE_COPY, etc...) are there to allow people to do certain operations in a platform-independent way from their qmake .pro file. On Windows using msbuild as makefile generator, $$QMAKE_STREAM_EDITOR expands to qmake -install sed, as can be seen in mkspecs/spec_post.prf:

equals(MAKEFILE_GENERATOR, MSBUILD) \
|equals(MAKEFILE_GENERATOR, MSVC.NET) \
|isEmpty(QMAKE_SH) {
    ...
    QMAKE_STREAM_EDITOR     = $(QMAKE) -install sed
   ...
} else {
    ...
    QMAKE_STREAM_EDITOR     = sed
    ...
}

Looking at the source code of qmake (https://github.com/qt/qtbase/blob/5.12/qmake/main.cpp) more specifically between lines 421 and lines 423 we have

    // Workaround for inferior/missing command line tools on Windows: make our own!
    if (argc >= 2 && !strcmp(argv[1], "-install"))
        return doInstall(argc - 2, argv + 2);

from which I conclude that the qmake -install sed option for qmake is a (in my opinion valuable) workaround for sed not being by default available on Windows. It's a simple sed implemented in a few lines of Qt/C++. So that is why I'm currently trying out commands like

qmake -install sed -e s/foo/bar/g test.txt

because I want to use that for replacing certain text in a QMAKE_POST_LINK action in a platform-independent way.

I am now at the point where I figured out that this works if test.txt has unix line endings, but the above fails if test.txt has windows line endings. I think it must have to do with the following code from https://github.com/qt/qtbase/blob/5.12/qmake/main.cpp:

for (const char *inFile : qAsConst(inFiles)) {
    FILE *f;
    if (!strcmp(inFile, "-")) {
        f = stdin;
    } else if (!(f = fopen(inFile, "r"))) {
        perror(inFile);
        return 1;
    }
    QTextStream is(f);
    while (!is.atEnd()) {
        QString line = is.readLine();
        for (int i = 0; i < substs.size(); i++)
            line.replace(substs.at(i).from, substs.at(i).to);
        puts(qPrintable(line));
    }
    if (f != stdin)
        fclose(f);
}

I tried opening inFile using fopen(inFile, "rt") instead of fopen(inFile, "r") but that didn't seem to solve the problem.

I do think this should work for files with both unix and Windows line endings. One cannot expect users to first change their line endings before they can use the qmake -install sed command (in my opinion...)

So if anyone sees why the above code works only for unix line endings and not for Windows line endings, I would be happy to hear!

PS: I'm not sure, but maybe it has to do with the fact that we build our Qt ourselves? Can anyone tell me if this problem also exists for the pre-compiled qmake?

[Bart_Vandewoestyne]

I just got a reply from the Qt people, pointing me to https://bugreports.qt.io/browse/QTBUG-80443 that should fix this issue.