Skip to content

Concurrency support & lazy generation support for cartesian product generation #125

New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Closed
shivasurya opened this issue Sep 7, 2024 · 1 comment
Assignees
Labels
enhancement New feature or request help wanted Extra attention is needed

Comments

@shivasurya
Copy link
Owner

shivasurya commented Sep 7, 2024

Is your feature request related to a problem? Please describe.
Similar to CodeQL performance optimization, #116 supports filtering near to the data source and now it warrants optimization using concurrency via multi thread/goroutine and stream result to prevent memory pressure.

When benchmarked against signalapp/Signal-Android, 18,000 methods and 54,000 method invocations - I believe concurrency and streaming support should solve memory pressure issue.

Describe the solution you'd like
Implement goroutine to generate cartesian product

Describe alternatives you've considered

  • Using prolog, datalog like native database support
  • Apache Calcite for relational algebra evaluation and query optimization

Use case
What will this feature enable for you?
Support larger source code repositories such as signal-android

Additional context
N/A

@shivasurya shivasurya added enhancement New feature or request help wanted Extra attention is needed labels Sep 7, 2024
@shivasurya shivasurya self-assigned this Sep 7, 2024
@shivasurya
Copy link
Owner Author

Cartesian product is the last effort. By introducing class based filter backed by storage layer #194 will render cartesian product ineffective solution.

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
enhancement New feature or request help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

1 participant