It appears that the problem is indeed a change in XCode4, which perhaps might be rightly called a bug. If I select one of these red folders or files, I see in the File Inspector that the "Location" control is set to "nothing". So, XCode doesn't know how to properly access the file/folder. By setting the "Location" control to "Relative to Group" or "Relative to Project", the file paths are correctly resolved.
For example, here's a thread of perhaps some relevance: http://www.cocoabuilder.com/archive/xcode/314464-xcode-4-preference-for-children-paths.html
In lieu of a "bug fix" from Apple on this (if it indeed is a bug), it seems that qmake would need to programatically set this "Location" setting, when it constructs the XCode project. I suspect that someone familiar with the qmake code for XCode generation might be able to provide a workaround/fix on the qmake side rather easily?