feat(sqllab): Format sql #25344

justinpark · 2023-09-19T22:27:47Z

SUMMARY

This commit adds "Format SQL" feature in sql editor which helps to format sql code in the sql editor.
This commit also adds a new shortcut (ctrl + shift + f) for this feature.

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

after--format-sql.mov

TESTING INSTRUCTIONS

Go to SQL Lab and type a long-run sql
Hit Ctrl + Shift + f or click "Format SQL" in the action dropdown
Check the formatted sql

ADDITIONAL INFORMATION

Has associated issue:
Required feature flags:
Changes UI
Includes DB Migration (follow approval process in SIP-59)
- Migration is atomic, supports rollback & is backwards-compatible
- Confirm DB migration upgrade and downgrade tested
- Runtime estimates and downtime expectations provided
Introduces new feature or API
Removes existing feature or API

justinpark · 2023-09-19T22:31:40Z

superset-frontend/src/SqlLab/components/AceEditorWrapper/index.tsx

@@ -144,8 +144,10 @@ const AceEditorWrapper = ({
  };

  const onChangeText = (text: string) => {
-    setSql(text);
-    onChange(text);
+    if (text !== sql) {


This change prevents the overwrite the sql by setQueryEditorAndSaveSqlWithDebounce

michael-s-molina

Thank you for the PR @justinpark! I left some first-pass comments.

superset/sqllab/api.py

superset-frontend/src/SqlLab/actions/sqlLab.js

superset/sqllab/api.py

michael-s-molina · 2023-09-21T10:50:52Z

/testenv up

github-actions · 2023-09-21T10:52:57Z

@michael-s-molina Ephemeral environment spinning up at http://34.220.51.49:8080. Credentials are admin/admin. Please allow several minutes for bootstrapping and startup.

justinpark · 2023-10-12T16:27:47Z

@michael-s-molina could you stamp this PR if the updates looks okay?

justinpark · 2023-10-12T20:40:34Z

@michael-s-molina I also set up SQLLAB_FORMAT_SQL_OPTIONS in config to setup custom SQL format option.

justinpark · 2023-10-17T17:47:26Z

I also set up SQLLAB_FORMAT_SQL_OPTIONS in config to setup custom SQL format option.

Revert this config option after offline discussion

michael-s-molina · 2023-10-24T23:33:04Z

/testenv up

github-actions · 2023-10-24T23:37:23Z

@michael-s-molina Ephemeral environment spinning up at http://18.237.171.147:8080. Credentials are admin/admin. Please allow several minutes for bootstrapping and startup.

michael-s-molina · 2023-10-24T23:34:42Z

superset-frontend/src/SqlLab/components/SqlEditor/index.tsx

+        func: () => {
+          formatCurrentQuery();
+        },


Suggested change

func: () => {

formatCurrentQuery();

},

func: formatCurrentQuery,

michael-s-molina · 2023-10-24T23:37:20Z

superset/sqllab/api.py

+        action=lambda self, *args, **kwargs: f"{self.__class__.__name__}" f".format",
+        log_to_statsd=False,
+    )
+    def format(self) -> FlaskResponse:


Maybe rename the endpoint to format_sql?

hughhhh · 2023-10-25T03:46:56Z

superset-frontend/src/SqlLab/actions/sqlLab.js

+export function formatQuery(queryEditor) {
+  return function (dispatch, getState) {
+    const { sql } = getUpToDateQuery(getState(), queryEditor);
+    return SupersetClient.post({


why couldn't we just us a npm package to do this work vs. creating a whole new endpoint?

why couldn't we just us a npm package to do this work vs. creating a whole new endpoint?

@hughhhh We chose server-side formatting over client-side formatting for several reasons:

It reduces the need to maintain additional NPM dependencies.

It reduces the package size.

It ensures that SQL formatting is consistent across surfaces in "view query", using sqlparse.

If you and other community members believe that an NPM package solution is better, I am happy to make changes.

@betodealmeida @eschutho @michael-s-molina do you have an opinion on the frontend approach?

There seems to be a number of pros with having this on the backend including: custom formatting, dialect specific formatting, and consistency throughout the application.

@villebro has experience working with frontend SQL formatting and ran into a number of issues.

michael-s-molina

LGTM

john-bodley · 2023-11-02T20:33:38Z

superset/sqllab/api.py

+        try:
+            model = self.format_model_schema.load(request.json)
+            result = sqlparse.format(
+                model.get("sql", ""), reindent=True, keyword_case="upper"


If the sql field is required (per the schema) is model["sql"] suffice?

github-actions · 2023-11-03T16:38:43Z

Ephemeral environment shutdown and build artifacts deleted.

pull-request-size bot added the size/L label Sep 19, 2023

justinpark requested review from betodealmeida, john-bodley, eschutho and michael-s-molina September 19, 2023 22:29

justinpark commented Sep 19, 2023

View reviewed changes

michael-s-molina reviewed Sep 20, 2023

View reviewed changes

superset/sqllab/api.py Outdated Show resolved Hide resolved

superset-frontend/src/SqlLab/actions/sqlLab.js Outdated Show resolved Hide resolved

superset/sqllab/api.py Outdated Show resolved Hide resolved

justinpark force-pushed the feat--sqllab-format-sql branch 3 times, most recently from a61a3e0 to 881f020 Compare October 12, 2023 15:18

justinpark added 5 commits October 17, 2023 13:46

feat(sqllab): Format sql

83cea9a

openapi spec

b0b6bca

remove line

53ef737

api schema ref and line ordering

033e41f

remap keyboard shortcut

bf9ad0e

justinpark force-pushed the feat--sqllab-format-sql branch from 50ab10a to bf9ad0e Compare October 17, 2023 17:47

michael-s-molina reviewed Oct 24, 2023

View reviewed changes

hughhhh reviewed Oct 25, 2023

View reviewed changes

justinpark added 2 commits October 26, 2023 10:49

change endpoint pathname

f18f7ae

fix format_sql for fetchMock

49a9da5

michael-s-molina approved these changes Nov 2, 2023

View reviewed changes

john-bodley reviewed Nov 2, 2023

View reviewed changes

update model.get("sql") by model["sql"]

d341c71

pylint

7ef8a51

justinpark merged commit 24a2213 into apache:master Nov 3, 2023

cccs-rc pushed a commit to CybercentreCanada/superset that referenced this pull request Mar 6, 2024

feat(sqllab): Format sql (apache#25344)

f40038d

mistercrunch added 🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels 🚢 3.1.0 labels Mar 8, 2024

sfirke pushed a commit to sfirke/superset that referenced this pull request Mar 22, 2024

feat(sqllab): Format sql (apache#25344)

e3dcad5

vinothkumar66 pushed a commit to vinothkumar66/superset that referenced this pull request Nov 11, 2024

feat(sqllab): Format sql (apache#25344)

3aab80a

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(sqllab): Format sql #25344

feat(sqllab): Format sql #25344

justinpark commented Sep 19, 2023 •

edited

Loading

justinpark Sep 19, 2023

michael-s-molina left a comment

michael-s-molina commented Sep 21, 2023

github-actions bot commented Sep 21, 2023

justinpark commented Oct 12, 2023

justinpark commented Oct 12, 2023

justinpark commented Oct 17, 2023

michael-s-molina commented Oct 24, 2023

github-actions bot commented Oct 24, 2023

michael-s-molina Oct 24, 2023

michael-s-molina Oct 24, 2023

hughhhh Oct 25, 2023

justinpark Oct 31, 2023 •

edited

Loading

john-bodley Nov 1, 2023

michael-s-molina left a comment

john-bodley Nov 2, 2023 •

edited

Loading

github-actions bot commented Nov 3, 2023

feat(sqllab): Format sql #25344

feat(sqllab): Format sql #25344

Conversation

justinpark commented Sep 19, 2023 • edited Loading

SUMMARY

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

TESTING INSTRUCTIONS

ADDITIONAL INFORMATION

justinpark Sep 19, 2023

Choose a reason for hiding this comment

michael-s-molina left a comment

Choose a reason for hiding this comment

michael-s-molina commented Sep 21, 2023

github-actions bot commented Sep 21, 2023

justinpark commented Oct 12, 2023

justinpark commented Oct 12, 2023

justinpark commented Oct 17, 2023

michael-s-molina commented Oct 24, 2023

github-actions bot commented Oct 24, 2023

michael-s-molina Oct 24, 2023

Choose a reason for hiding this comment

michael-s-molina Oct 24, 2023

Choose a reason for hiding this comment

hughhhh Oct 25, 2023

Choose a reason for hiding this comment

justinpark Oct 31, 2023 • edited Loading

Choose a reason for hiding this comment

john-bodley Nov 1, 2023

Choose a reason for hiding this comment

michael-s-molina left a comment

Choose a reason for hiding this comment

john-bodley Nov 2, 2023 • edited Loading

Choose a reason for hiding this comment

github-actions bot commented Nov 3, 2023

justinpark commented Sep 19, 2023 •

edited

Loading

justinpark Oct 31, 2023 •

edited

Loading

john-bodley Nov 2, 2023 •

edited

Loading