CodeQL provides powerful built-in analyses and program representations, such as AST, CFG, data flow, call graphs, and class hierarchies. However, these representations are currently primarily accessed through QL queries after a CodeQL database has been created.
For more complex analyses or for extending CodeQL's existing analyses (e.g., custom data-flow or alias analyses), implementing everything directly in QL can become quite difficult and cumbersome.
Currently, I use the following workaround:
- Build a CodeQL database for the target project.
- Write basic QL queries to export selected facts to CSV., such as:
- Interested AST nodes and their relationships
- Class/interface information and inheritance relationships
- Call sites and call relationships
- Other program facts relevant to my analysis
Define my own schema and load the CSV facts into memory.
Implement more sophisticated analyses using Python and custom algorithms.
This works, but it requires an additional export/import layer and also means that I have to manually reconstruct program representations that CodeQL already maintains internally. Would it be possible to provide an official Python SDK/API (or another programmatic API) that allows users to directly access the facts stored in a CodeQL database?
For example, something along the lines of:
db = codeql.Database("my-project-db")
ast = db.ast()
cfg = db.cfg()
dataflow = db.dataflow()
classes = db.class_hierarchy()
calls = db.call_graph()The exact API is not important; the key idea is that Python code could directly access the program facts represented in the CodeQL database, without first exporting them through QL queries. This would make it possible to use CodeQL as a powerful program representation and fact extraction backend, while implementing more complex or experimental analyses in Python.
CodeQL provides powerful built-in analyses and program representations, such as AST, CFG, data flow, call graphs, and class hierarchies. However, these representations are currently primarily accessed through QL queries after a CodeQL database has been created.
For more complex analyses or for extending CodeQL's existing analyses (e.g., custom data-flow or alias analyses), implementing everything directly in QL can become quite difficult and cumbersome.
Currently, I use the following workaround:
Define my own schema and load the CSV facts into memory.
Implement more sophisticated analyses using Python and custom algorithms.
This works, but it requires an additional export/import layer and also means that I have to manually reconstruct program representations that CodeQL already maintains internally. Would it be possible to provide an official Python SDK/API (or another programmatic API) that allows users to directly access the facts stored in a CodeQL database?
For example, something along the lines of:
The exact API is not important; the key idea is that Python code could directly access the program facts represented in the CodeQL database, without first exporting them through QL queries. This would make it possible to use CodeQL as a powerful program representation and fact extraction backend, while implementing more complex or experimental analyses in Python.