-
Notifications
You must be signed in to change notification settings - Fork 465
New issue
Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? # to your account
feat(server): Backfill missing_fields on _nango_config #3050
Conversation
…-2168/missing-fields-backfill
packages/database/lib/migrations/20241122212401_integration_backfill_missing_fields.cjs
Outdated
Show resolved
Hide resolved
packages/database/lib/migrations/20241122212401_integration_backfill_missing_fields.cjs
Outdated
Show resolved
Hide resolved
packages/database/lib/migrations/20241122212401_integration_backfill_missing_fields.cjs
Show resolved
Hide resolved
In Slack we discussed adding |
packages/database/lib/migrations/20241122212401_integration_backfill_missing_fields.cjs
Outdated
Show resolved
Hide resolved
packages/database/lib/migrations/20241122212401_integration_backfill_missing_fields.cjs
Show resolved
Hide resolved
packages/database/lib/migrations/20241122212401_integration_backfill_missing_fields.cjs
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we estimate how fast the migration gonna be in prod? How many integrations do we have to go through?
.whereIn('provider', appLinkProviders) | ||
.whereRaw("NOT (missing_fields @> '{app_link}')") | ||
.update({ missing_fields: knex.raw("array_append(missing_fields, 'app_link')") }); | ||
}; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You can make the script faster by skipping deleted one (about 25% in prod)
@TBonnin There are only 16240 in prod, so this should run pretty quick, especially since I was able to remove any need to decrypt fields. |
|
GitGuardian id | GitGuardian status | Secret | Commit | Filename | |
---|---|---|---|---|---|
8205899 | Triggered | Generic Password | dde28f4 | packages/database/lib/getDbConfig.ts | View secret |
🛠 Guidelines to remediate hardcoded secrets
- Understand the implications of revoking this secret by investigating where it is used in your code.
- Replace and store your secret safely. Learn here the best practices.
- Revoke and rotate this secret.
- If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.
To avoid such incidents in the future consider
- following these best practices for managing and storing secrets including API keys and other credentials
- install secret detection on pre-commit to catch secret before it leaves your machine and ease remediation.
🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.
<!-- Describe the problem and your solution --> Waiting on #3050 Shows missing field errors on the integrations page and highlights the errors on the tab: <img width="1382" alt="image" src="https://github.com/user-attachments/assets/377167b2-2b49-4a55-8b28-76530eccb1a6"> And in the integrations list: <img width="1381" alt="image" src="https://github.com/user-attachments/assets/60d92ff1-00f8-4c11-8646-b69f3f5c21ac"> <!-- Issue ticket number and link (if applicable) --> https://linear.app/nango/issue/NAN-2168/surface-integrationsconnections-errors-in-nango-ui <!-- Testing instructions (skip if just adding/editing providers) --> - Set up a new integration that needs configuration, but don't configure it - View the integration page. You should see a yellow dot in the settings tab and a banner saying what fields need to be configured
Fills out
missing_fields
as best as we can based on_nango_config
. Unfortunately theCUSTOM
auth_method
won't be able to be backfilled performantly, but all others were possible.For all of these, the prior PR will cause them to show errors in the future when created or edited, so I think we're safe here without having to decrypt all of the
CUSTOM
auth based ones.https://linear.app/nango/issue/NAN-2168/surface-integrationsconnections-errors-in-nango-ui
How I tested
missing_fields
in the DB as an easy way to handle that.