This is an example application which uses OpenAI, Databricks Unity Catalog and DBSQL to convert natural language queries into SQL and execute it against DBSQL.
Read more about this example project here.
- Databricks Unity Catalog provides metadata about catalogs, schemas and tables
- OpenAI API provides API to generate SQL statements from the natural language
- Databricks SQL is the execution engine to run the generated SQL statements
- React for the frontend application
- React Bootstrap for layout and components style
- React Syntax Highlighter for SQL syntax highlighting
- Axios - HTTP client, used for backend communication
- React Bootstrap Table - for table visualization
- FastAPI for the backend server
- Databricks SQL Connector to communicate with DBSQL endpoint
- OpenAI Python API - for API calls from backend
git clone https://github.com/renardeinside/databricks-uc-semantic-layer.git- Get your DBSQL endpoint coordinates from the UI
- Get your OpenAI API key
- Generate the data using the job defined in
./uc-semantic-layer:
- Configure the catalog and schema in
./uc-semantic-layer/conf/data_preparation.yml- Run the job (either on the interactive or as a job):
cd uc-semantic-layer
# install dbx and other relevant libraries
pip install -r unit-requirements.txt
# optional - configure dbx to use another profile (by default it uses the DEFAULT one)
dbx configure -e default --profile=<some-other-profile-name>
# this is to execute on an interactive cluster
dbx execute --job=semantic-layer-data-preparation --cluster-name=<interactive-cluster-name>
# this is to launch on automated one, please configure your node_type_id in conf/deployment.yml
dbx deploy --job=semantic-layer-data-preparation --files-only
dbx launch --job=semantic-layer-data-preparation --as-run-submit --trace
cd .. # back to the project root- Setup the relevant variables in
.envfile (check.env.examplefor a reference). - Start the services:
make launch- Open http://localhost:3000 and enjoy the app!