How to find symbols in extern code files or projects and get their "type" (code analysis library)?

[Qtpp]

Hello experts,

I'm looking for a slim and free code analysis library to use in my app.

Actually, I don't want to completely "analyse" the foreign project/code file(s) in terms of logic. And I don't care about any flaws, memleaks or want to provide any CD/CI, like SonarQube or what other mighty tools/software suites are around.

I just want to detect symbols and find links to their corresponding classes and appearances, if this is possible.
Doesn't have to be 100% accurate.

Something like:

Given class Foo with files Foo.h and Foo.cpp

// Foo.h

#include "bar.h"

class Foo
{

private:

     Bar *bar_;

};

// ##############################################
// Foo.cpp

#include "foo.h"
#include "helloWorld.h"

{
    bar_ = new Bar();
    HelloWorld hw;
}

myApp.exe -"Foo.h" -"Foo.cpp"

Output like:

n symbols/variables found:
    - bar_, class "Bar"
    - hw, class "HelloWorld"
    - this, class "Foo"
    - [ ... ]

QtCreator uses clang/clazy for its full analysis, right?!
So my idea is just a part of it? It can work?
I don't want to compile the code I'm going to analyse, if possible (is it?!)
I'm wondering if there is any tool/library which does something like that?

Might be a silly question, but I couldn't find anything helpful.

Looking forward for any input
Thanks.

[FeRDNYC]

@Qtpp said in How to find symbols in extern code files or projects and get their "type" (code analysis library)?:

I don't want to compile the code I'm going to analyse, if possible (is it?!)

You probably want to at least come close to compiling the code, because... well, basically, because you have to, in order to determine what symbols the code actually produces.

There are questions that have to be answered, like what header files will be loaded by #includes, what the actual value of any preprocessor symbols would actually be at compile time, what the result of various #if/#defined tests would therefore be, etc. Those are most easily answered by at least preprocessing the code, if not fully compiling it.

(And also, if the code can't be compiled, that's probably a sign that it may not be worth trying to analyze either. Because the process of converting source code into symbols is exactly what a compiler does.)

Most clang analysis tools require the path to a compile_commands.json for the files it's analyzing, so that it can pick up compile command lines, and know which directories to look in for header files and etc.

Without knowing those things, you can't really interpret a given source file completely, because things like preprocessor conditionals create internal inconsistency if all branches are taken at once.

[Qtpp]

@FeRDNYC said in How to find symbols in extern code files or projects and get their "type" (code analysis library)?:

You probably want to at least come close to compiling the code, because... well, basically, because you have to, in order to determine what symbols the code actually produces.

Hey, thanks for your answer.
Yea I thought so, which I definitely won't do.
However my very first idea was to simply parse the code files, which also works to a certain extent only, since it's game over as soon as you face code like:

auto obj = getObject();

or

test ( helloWorld->getHello()->getWorld() );

Unfortunately only having the code files, you never gonna know what type of object auto obj is (returned by getObject) or what helloWorld->getHello()->getWorld() returns. Plus the defines or the other includes you mention.
Might work for locals and member variables though.

@FeRDNYC said in How to find symbols in extern code files or projects and get their "type" (code analysis library)?:

(And also, if the code can't be compiled, that's probably a sign that it may not be worth trying to analyze either. Because the process of converting source code into symbols is exactly what a compiler does.)

I thought of something that might be helpful for the whole Qt community. It should be possible to use it on individual files (that use Qt code) and entire projects, but... well... :(

When I make my project a QtCreator plugin, so that it's not standalone but integrated into the "coding process".
I have access to the symbols, clang/clazy and compile output, no?!
THAT would actually be the best and easiest way, I guess.

What do you think?

[SGaist]

Hi,

Are you trying to reimplement something like the clang code model from Qt Creator ?

[Qtpp]

@SGaist said in How to find symbols in extern code files or projects and get their "type" (code analysis library)?:

Are you trying to reimplement something like the clang code model from Qt Creator ?

Hi, not quite.
At least this is not my intention just for the sake of doing it.

I think I would be happy if I can access it from my QtCreator Plugin. Don't know if it's a good idea to go this way and if it helps to solve my issue. That's why I'm asking.

Are you following the whole thread here?
My only, but not so easy (as I found out and as @FeRDNYC told me) goal is it to extract variables (symbols) from code files or projects and do something with that information. I may not need all of them, and the user should decide which ones to include or exclude.
To be 100% accurate, I am only interested in the particular classes and objects (i.e. their class names) that appear in a file.

So, does a QtCreator Plugin have access to the "clang code model from QtCreator"?

If so, how can I do it?

[SGaist]

From your description, it looks like you are after libclang to parse the code and extract information.

I don't think the code model exposes stuff but you should check the sources to confirm.

[Qtpp]

@SGaist said in How to find symbols in extern code files or projects and get their "type" (code analysis library)?:

From your description, it looks like you are after libclang to parse the code and extract information.

That is looking promising.
So I can use my own clang to parse and inspect the file(s)?!
And the clang / code model used by the IDE is just read-only? You don't have access to it and can't gain access in some way by installing or modifying something?!

But thanks, that helped. Will try libclang then... looks complicated to use tho :/

[SGaist]

Libclang is just a library.

As I wrote, I currently don't know how the code model is implemented and if you can tap in it. However, you can check that, the code of Qt Creator is available.

[Qtpp]

@SGaist said in How to find symbols in extern code files or projects and get their "type" (code analysis library)?:

As I wrote, I currently don't know how the code model is implemented and if you can tap in it. However, you can check that, the code of Qt Creator is available.

Thanks for your reply.

I've checked QtCreator on Github and was able to find the clangcodemodel plugin here. Meanwhile I realized that QtCreator itself is made out of many plugins for each purpose.

Before I start to learn the whole thing for nothing, is it somehow possible to reach out to one of the QtCreator maintainers/devs, some person who knows all the internal QtCreator things well, so he/she can confirm or contradict that it's doable?!
Because it feels like I can look at this code for month without finding out if it's worth to create a QtCreator plugin for my use case.
A clear "yes" or "no, because...." would help a lot.

PS:
While scrolling through the GitHub folders I saw a plugin called "cppcheck" in there. Stumbled upon this "software" during my research on how to analyse and extract stuff from C++ code.
Don't know exactly what the purpose of this is and if it's used by clang or maybe can be used in my case instead of clang?!
Will continue to research :)

[jsulm]

@Qtpp said in How to find symbols in extern code files or projects and get their "type" (code analysis library)?:

is it somehow possible to reach out to one of the QtCreator maintainers/devs

https://lists.qt-project.org/listinfo/qt-creator

[Qtpp]

@SGaist said in How to find symbols in extern code files or projects and get their "type" (code analysis library)?:

Libclang is just a library.

Made some progress in this case.
However most guides describe how to extend clang through your own custom written clang-tools (which requires re-compiling clang every time)...
I was thinking more about a standalone program which links (maybe statically) clang and accesses already existing API/tools like

• https://clang.llvm.org/docs/LibASTMatchersTutorial.html
• https://clang.llvm.org/docs/LibASTMatchers.html

This must also be possible right? Did not find anything like that as most guides show how to write clang "plugins"?!

[SGaist]

cppcheck is a static analyzer however I don't know how it's currently implemented.

As for writing an application with libclang, I think their tutorial should get you started.

[Christian Ehrlicher]

clazy is also using libclang