Cleanup and speed-up of type resolution

[oxisto]

This PR should speed up type handling immensely. The list of first oder types is now a map of a list instead of a list. The issue is that we have a lot of "types" (remember, a "type" is a unique combination of a scope and a name), we potentially have many scopes and a simple "myType" can exist as multiple Type nodes before the type resolver harmonises this. In order to at least have unique types per scope, we iterate existing types and return one that matches the name/scope combination. This was previously done over the complete list and is now only restricted to those types whos name matches in the first place.

This is more a workaround than a complete solution to the problem of types but should at least be significantly faster now.

Benchmarks

As a benchmark I parsed http://github.com/openstack/glance with default passes (without CDG/PDG because they are broken currently for Python). This speeds up the parsing from 68s to 32s.
MacBook Air, 13-inch, M3, 2024, 16 GB

Before:

Metric	Value
Number of files translated	368
TranslationManager: Executing Language Frontend	16643 ms
CompressLLVMPass: Executing Pass	2 ms
ImportResolver: Executing Pass	547 ms
TypeResolver: Executing Pass	924 ms
TypeHierarchyResolver: Executing Pass	342 ms
CXXExtraPass: Executing Pass	123 ms
GoExtraPass: Executing Pass	308 ms
PythonAddDeclarationsPass: Executing Pass	1196 ms
ResolveCallExpressionAmbiguityPass: Executing Pass	3891 ms
JavaExternalTypeHierarchyResolver: Executing Pass	2 ms
EvaluationOrderGraphPass: Executing Pass	654 ms
JavaImportResolver: Executing Pass	0 ms
SymbolResolver: Executing Pass	39116 ms
DFGPass: Executing Pass	842 ms
ControlFlowSensitiveDFGPass: Executing Pass	3120 ms
DynamicInvokeResolver: Executing Pass	219 ms
PrepareSerialization: Executing Pass	376 ms
FilenameMapper: Executing Pass	175 ms
TranslationManager: Translation into full graph	68491 ms

After:

Metric	Value
Number of files translated	368
TranslationManager: Executing Language Frontend	14279 ms
CompressLLVMPass: Executing Pass	2 ms
ImportResolver: Executing Pass	544 ms
TypeResolver: Executing Pass	431 ms
TypeHierarchyResolver: Executing Pass	168 ms
CXXExtraPass: Executing Pass	162 ms
GoExtraPass: Executing Pass	333 ms
PythonAddDeclarationsPass: Executing Pass	1158 ms
ResolveCallExpressionAmbiguityPass: Executing Pass	2054 ms
JavaExternalTypeHierarchyResolver: Executing Pass	1 ms
EvaluationOrderGraphPass: Executing Pass	654 ms
JavaImportResolver: Executing Pass	0 ms
SymbolResolver: Executing Pass	8537 ms
DFGPass: Executing Pass	712 ms
ControlFlowSensitiveDFGPass: Executing Pass	3031 ms
DynamicInvokeResolver: Executing Pass	191 ms
PrepareSerialization: Executing Pass	423 ms
FilenameMapper: Executing Pass	171 ms
TranslationManager: Translation into full graph	32865 ms

[maximiliankaul]

Speedup (start to end cpg parsing (frontend + passes)) against current main is about 4x with a larger Python test case 👍

[oxisto]

Unfortunately, while this speeds up python, this slows down the CXXExtraPass extremely :(

[oxisto]

Unfortunately, while this speeds up python, this slows down the CXXExtraPass extremely :(

Solved

[sonarqubecloud]

Quality Gate passed

Issues
13 New issues
0 Accepted issues

Measures
0 Security Hotspots
80.9% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

[codecov]

Codecov Report

Attention: Patch coverage is 81.25000% with 27 lines in your changes missing coverage. Please review.

Project coverage is 74.77%. Comparing base (21aeba7) to head (d4d29d6).
Report is 2 commits behind head on main.

Files with missing lines	Patch %	Lines
...lin/de/fraunhofer/aisec/cpg/passes/TypeResolver.kt	66.66%	2 Missing and 3 partials ⚠️
...main/kotlin/de/fraunhofer/aisec/cpg/TypeManager.kt	76.92%	1 Missing and 2 partials ⚠️
...fer/aisec/cpg/frontends/java/DeclarationHandler.kt	0.00%	3 Missing ⚠️
...ain/kotlin/de/fraunhofer/aisec/cpg/ScopeManager.kt	88.23%	1 Missing and 1 partial ⚠️
...kotlin/de/fraunhofer/aisec/cpg/graph/types/Type.kt	66.66%	2 Missing ⚠️
...n/de/fraunhofer/aisec/cpg/passes/SymbolResolver.kt	94.73%	0 Missing and 2 partials ⚠️
...lin/de/fraunhofer/aisec/cpg/passes/CXXExtraPass.kt	80.00%	1 Missing and 1 partial ⚠️
...r/aisec/cpg/frontends/java/JavaLanguageFrontend.kt	66.66%	2 Missing ⚠️
...hofer/aisec/cpg/graph/types/FunctionPointerType.kt	0.00%	1 Missing ⚠️
.../fraunhofer/aisec/cpg/graph/types/ReferenceType.kt	0.00%	1 Missing ⚠️
... and 4 more

Additional details and impacted files

Files with missing lines	Coverage Δ
...tlin/de/fraunhofer/aisec/cpg/TranslationManager.kt	72.30% <100.00%> (+0.72%)	⬆️
...e/fraunhofer/aisec/cpg/frontends/LanguageTraits.kt	100.00% <ø> (ø)
...otlin/de/fraunhofer/aisec/cpg/graph/TypeBuilder.kt	66.66% <100.00%> (+0.66%)	⬆️
...de/fraunhofer/aisec/cpg/graph/types/NumericType.kt	75.00% <100.00%> (ø)
.../de/fraunhofer/aisec/cpg/graph/types/StringType.kt	66.66% <100.00%> (ø)
...de/fraunhofer/aisec/cpg/graph/types/UnknownType.kt	100.00% <100.00%> (ø)
.../fraunhofer/aisec/cpg/frontends/cxx/CPPLanguage.kt	90.06% <100.00%> (+0.06%)	⬆️
...er/aisec/cpg/frontends/golang/ExpressionHandler.kt	88.14% <100.00%> (ø)
...ofer/aisec/cpg/frontends/java/ExpressionHandler.kt	72.52% <100.00%> (ø)
...ec/cpg/passes/JavaExternalTypeHierarchyResolver.kt	78.12% <100.00%> (ø)
... and 15 more

... and 5 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[oxisto]

I will split this up into multiple issues/PRs since this became unmanageable.