Skip to content

Commit 00b6cc7

Browse files
committed
feat: Periodicaly check if domain name changed and close connections to old database.
wip: periodic check for domain change wip: periodic checks wip: close sockets on instance closed wip: readme for below
1 parent 8434d03 commit 00b6cc7

File tree

9 files changed

+276
-24
lines changed

9 files changed

+276
-24
lines changed

.github/workflows/tests.yml

+4-4
Original file line numberDiff line numberDiff line change
@@ -154,7 +154,7 @@ jobs:
154154
MYSQL_DB:${{ vars.GOOGLE_CLOUD_PROJECT }}/MYSQL_DB
155155
POSTGRES_CONNECTION_NAME:${{ vars.GOOGLE_CLOUD_PROJECT }}/POSTGRES_CONNECTION_NAME
156156
POSTGRES_USER:${{ vars.GOOGLE_CLOUD_PROJECT }}/POSTGRES_USER
157-
POSTGRES_USER_IAM_NODE:${{ vars.GOOGLE_CLOUD_PROJECT }}/POSTGRES_USER_IAM_NODE
157+
POSTGRES_IAM_USER:${{ vars.GOOGLE_CLOUD_PROJECT }}/POSTGRES_USER_IAM_NODE
158158
POSTGRES_PASS:${{ vars.GOOGLE_CLOUD_PROJECT }}/POSTGRES_PASS
159159
POSTGRES_DB:${{ vars.GOOGLE_CLOUD_PROJECT }}/POSTGRES_DB
160160
POSTGRES_CAS_CONNECTION_NAME:${{ vars.GOOGLE_CLOUD_PROJECT }}/POSTGRES_CAS_CONNECTION_NAME
@@ -183,7 +183,7 @@ jobs:
183183
MYSQL_DB: "${{ steps.secrets.outputs.MYSQL_DB }}"
184184
POSTGRES_CONNECTION_NAME: "${{ steps.secrets.outputs.POSTGRES_CONNECTION_NAME }}"
185185
POSTGRES_USER: "${{ steps.secrets.outputs.POSTGRES_USER }}"
186-
POSTGRES_USER_IAM_NODE: "${{ steps.secrets.outputs.POSTGRES_USER_IAM_NODE }}"
186+
POSTGRES_IAM_USER: "${{ steps.secrets.outputs.POSTGRES_IAM_USER }}"
187187
POSTGRES_PASS: "${{ steps.secrets.outputs.POSTGRES_PASS }}"
188188
POSTGRES_DB: "${{ steps.secrets.outputs.POSTGRES_DB }}"
189189
POSTGRES_CAS_CONNECTION_NAME: "${{ steps.secrets.outputs.POSTGRES_CAS_CONNECTION_NAME }}"
@@ -279,7 +279,7 @@ jobs:
279279
MYSQL_DB:${{ vars.GOOGLE_CLOUD_PROJECT }}/MYSQL_DB
280280
POSTGRES_CONNECTION_NAME:${{ vars.GOOGLE_CLOUD_PROJECT }}/POSTGRES_CONNECTION_NAME
281281
POSTGRES_USER:${{ vars.GOOGLE_CLOUD_PROJECT }}/POSTGRES_USER
282-
POSTGRES_USER_IAM_NODE:${{ vars.GOOGLE_CLOUD_PROJECT }}/POSTGRES_USER_IAM_NODE
282+
POSTGRES_IAM_USER:${{ vars.GOOGLE_CLOUD_PROJECT }}/POSTGRES_USER_IAM_NODE
283283
POSTGRES_PASS:${{ vars.GOOGLE_CLOUD_PROJECT }}/POSTGRES_PASS
284284
POSTGRES_DB:${{ vars.GOOGLE_CLOUD_PROJECT }}/POSTGRES_DB
285285
POSTGRES_CAS_CONNECTION_NAME:${{ vars.GOOGLE_CLOUD_PROJECT }}/POSTGRES_CAS_CONNECTION_NAME
@@ -302,7 +302,7 @@ jobs:
302302
MYSQL_DB: "${{ steps.secrets.outputs.MYSQL_DB }}"
303303
POSTGRES_CONNECTION_NAME: "${{ steps.secrets.outputs.POSTGRES_CONNECTION_NAME }}"
304304
POSTGRES_USER: "${{ steps.secrets.outputs.POSTGRES_USER }}"
305-
POSTGRES_IAM_USER: "${{ steps.secrets.outputs.POSTGRES_USER_IAM_NODE }}"
305+
POSTGRES_IAM_USER: "${{ steps.secrets.outputs.POSTGRES_IAM_USER }}"
306306
POSTGRES_PASS: "${{ steps.secrets.outputs.POSTGRES_PASS }}"
307307
POSTGRES_DB: "${{ steps.secrets.outputs.POSTGRES_DB }}"
308308
POSTGRES_CAS_CONNECTION_NAME: "${{ steps.secrets.outputs.POSTGRES_CAS_CONNECTION_NAME }}"

README.md

+74
Original file line numberDiff line numberDiff line change
@@ -444,6 +444,80 @@ variables. Here is a quick reference to supported values and their effect:
444444
- `GOOGLE_CLOUD_QUOTA_PROJECT`: Used to set a custom quota project to Cloud SQL
445445
APIs when defined.
446446

447+
## Configure your DNS Records
448+
449+
Add a DNS TXT record for the Cloud SQL instance to a **private** DNS server
450+
or a private Google Cloud DNS Zone used by your application.
451+
452+
**Note:** You are strongly discouraged from adding DNS records for your
453+
Cloud SQL instances to a public DNS server. This would allow anyone on the
454+
internet to discover the Cloud SQL instance name.
455+
456+
For example: suppose you wanted to use the domain name
457+
`prod-db.mycompany.example.com` to connect to your database instance
458+
`my-project:region:my-instance`. You would create the following DNS record:
459+
460+
- Record type: `TXT`
461+
- Name: `prod-db.mycompany.example.com` – This is the domain name used by the application
462+
- Value: `my-project:region:my-instance` – This is the instance name
463+
464+
### Configure the connector
465+
466+
Configure the connector as described above, replacing the connector ID with
467+
the DNS name.
468+
469+
Adapting the MySQL + database/sql example above:
470+
471+
```js
472+
import mysql from 'mysql2/promise';
473+
import {Connector} from '@google-cloud/cloud-sql-connector';
474+
475+
const connector = new Connector();
476+
const clientOpts = await connector.getOptions({
477+
domainName: 'prod-db.mycompany.example.com',
478+
ipType: 'PUBLIC',
479+
});
480+
481+
const pool = await mysql.createPool({
482+
...clientOpts,
483+
user: 'my-user',
484+
password: 'my-password',
485+
database: 'db-name',
486+
});
487+
const conn = await pool.getConnection();
488+
const [result] = await conn.query(`SELECT NOW();`);
489+
console.table(result); // prints returned time value from server
490+
491+
await pool.end();
492+
connector.close();
493+
```
494+
495+
### Automatic failover using DNS domain names
496+
497+
For example: suppose application is configured to connect using the
498+
domain name `prod-db.mycompany.example.com`. Initially the private DNS
499+
zone has a TXT record with the value `my-project:region:my-instance`. The
500+
application establishes connections to the `my-project:region:my-instance`
501+
Cloud SQL instance. Configure the connector using the `domainName` option:
502+
503+
Then, to reconfigure the application to use a different database
504+
instance, change the value of the `prod-db.mycompany.example.com` DNS record
505+
from `my-project:region:my-instance` to `my-project:other-region:my-instance-2`
506+
507+
The connector inside the application detects the change to this
508+
DNS record. Now, when the application connects to its database using the
509+
domain name `prod-db.mycompany.example.com`, it will connect to the
510+
`my-project:other-region:my-instance-2` Cloud SQL instance.
511+
512+
The connector will automatically close all existing connections to
513+
`my-project:region:my-instance`. This will force the connection pools to
514+
establish new connections. Also, it may cause database queries in progress
515+
to fail.
516+
517+
The connector will poll for changes to the DNS name every 30 seconds by default.
518+
You may configure the frequency of the connections using the Connector's
519+
`failoverPeriod` option.
520+
447521
## Support policy
448522

449523
### Major version lifecycle

src/cloud-sql-instance.ts

+67-1
Original file line numberDiff line numberDiff line change
@@ -14,13 +14,26 @@
1414

1515
import {IpAddressTypes, selectIpAddress} from './ip-addresses';
1616
import {InstanceConnectionInfo} from './instance-connection-info';
17-
import {resolveInstanceName} from './parse-instance-connection-name';
17+
import {
18+
isSameInstance,
19+
resolveInstanceName,
20+
} from './parse-instance-connection-name';
1821
import {InstanceMetadata} from './sqladmin-fetcher';
1922
import {generateKeys} from './crypto';
2023
import {RSAKeys} from './rsa-keys';
2124
import {SslCert} from './ssl-cert';
2225
import {getRefreshInterval, isExpirationTimeValid} from './time';
2326
import {AuthTypes} from './auth-types';
27+
import {CloudSQLConnectorError} from './errors';
28+
29+
// Private types that describe exactly the methods
30+
// needed from tls.Socket to be able to close
31+
// sockets when the DNS Name changes.
32+
type EventFn = () => void;
33+
type DestroyableSocket = {
34+
destroy: (error?: Error) => void;
35+
once: (name: string, handler: EventFn) => void;
36+
};
2437

2538
interface Fetcher {
2639
getInstanceMetadata({
@@ -42,6 +55,7 @@ interface CloudSQLInstanceOptions {
4255
ipType: IpAddressTypes;
4356
limitRateInterval?: number;
4457
sqlAdminFetcher: Fetcher;
58+
failoverPeriod?: number;
4559
}
4660

4761
interface RefreshResult {
@@ -74,9 +88,13 @@ export class CloudSQLInstance {
7488
// The ongoing refresh promise is referenced by the `next` property
7589
private next?: Promise<RefreshResult>;
7690
private scheduledRefreshID?: ReturnType<typeof setTimeout> | null = undefined;
91+
private checkDomainID?: ReturnType<typeof setInterval> | null = undefined;
7792
/* eslint-disable-next-line @typescript-eslint/no-explicit-any */
7893
private throttle?: any;
7994
private closed = false;
95+
private failoverPeriod: number;
96+
private sockets = new Set<DestroyableSocket>();
97+
8098
public readonly instanceInfo: InstanceConnectionInfo;
8199
public ephemeralCert?: SslCert;
82100
public host?: string;
@@ -98,6 +116,7 @@ export class CloudSQLInstance {
98116
this.ipType = options.ipType || IpAddressTypes.PUBLIC;
99117
this.limitRateInterval = options.limitRateInterval || 30 * 1000; // 30 seconds
100118
this.sqlAdminFetcher = options.sqlAdminFetcher;
119+
this.failoverPeriod = options.failoverPeriod || 30 * 1000; // 30 seconds
101120
}
102121

103122
// p-throttle library has to be initialized in an async scope in order to
@@ -153,6 +172,14 @@ export class CloudSQLInstance {
153172
return Promise.reject('closed');
154173
}
155174

175+
// Lazy instantiation of the checkDomain interval on the first refresh
176+
// This avoids issues with test cases that instantiate a CloudSqlInstance.
177+
if (this?.instanceInfo?.domainName && !this.checkDomainID) {
178+
this.checkDomainID = setInterval(() => {
179+
this.checkDomainChanged();
180+
}, this.failoverPeriod);
181+
}
182+
156183
const currentRefreshId = this.scheduledRefreshID;
157184

158185
// Since forceRefresh might be invoked during an ongoing refresh
@@ -312,9 +339,48 @@ export class CloudSQLInstance {
312339
close(): void {
313340
this.closed = true;
314341
this.cancelRefresh();
342+
if (this.checkDomainID) {
343+
clearInterval(this.checkDomainID);
344+
this.checkDomainID = null;
345+
}
346+
for (const socket of this.sockets) {
347+
socket.destroy(
348+
new CloudSQLConnectorError({
349+
code: 'ERRCLOSED',
350+
message: 'The connector was closed.',
351+
})
352+
);
353+
}
315354
}
316355

317356
isClosed(): boolean {
318357
return this.closed;
319358
}
359+
async checkDomainChanged() {
360+
if (!this.instanceInfo.domainName) {
361+
return;
362+
}
363+
364+
const newInfo = await resolveInstanceName(
365+
undefined,
366+
this.instanceInfo.domainName
367+
);
368+
if (!isSameInstance(this.instanceInfo, newInfo)) {
369+
// Domain name changed. Close and remove, then create a new map entry.
370+
this.close();
371+
}
372+
}
373+
addSocket(socket: DestroyableSocket) {
374+
if (!this.instanceInfo.domainName) {
375+
// This was not connected by domain name. Ignore all sockets.
376+
return;
377+
}
378+
379+
// Add the socket to the list
380+
this.sockets.add(socket);
381+
// When the socket is closed, remove it.
382+
socket.once('closed', () => {
383+
this.sockets.delete(socket);
384+
});
385+
}
320386
}

src/connector.ts

+6
Original file line numberDiff line numberDiff line change
@@ -44,6 +44,7 @@ export declare interface ConnectionOptions {
4444
ipType?: IpAddressTypes;
4545
instanceConnectionName: string;
4646
domainName?: string;
47+
failoverPeriod?: number;
4748
limitRateInterval?: number;
4849
}
4950

@@ -129,6 +130,7 @@ class CloudSQLInstanceMap extends Map<string, CacheEntry> {
129130
const entry = this.get(key);
130131
if (entry) {
131132
if (entry.isResolved()) {
133+
await entry.instance?.checkDomainChanged();
132134
if (!entry.instance?.isClosed()) {
133135
// The instance is open and the domain has not changed.
134136
// use the cached instance.
@@ -154,6 +156,7 @@ class CloudSQLInstanceMap extends Map<string, CacheEntry> {
154156
ipType: opts.ipType || IpAddressTypes.PUBLIC,
155157
limitRateInterval: opts.limitRateInterval || 30 * 1000, // 30 sec
156158
sqlAdminFetcher: this.sqlAdminFetcher,
159+
failoverPeriod: opts.failoverPeriod,
157160
});
158161
this.set(key, new CacheEntry(promise));
159162

@@ -257,6 +260,9 @@ export class Connector {
257260
tlsSocket.once('secureConnect', async () => {
258261
cloudSqlInstance.setEstablishedConnection();
259262
});
263+
264+
cloudSqlInstance.addSocket(tlsSocket);
265+
260266
return tlsSocket;
261267
}
262268

system-test/pg-connect.cjs

+12-12
Original file line numberDiff line numberDiff line change
@@ -20,13 +20,13 @@ const {Client} = pg;
2020
t.test('open connection and retrieves standard pg tables', async t => {
2121
const connector = new Connector();
2222
const clientOpts = await connector.getOptions({
23-
instanceConnectionName: String(process.env.POSTGRES_CONNECTION_NAME),
23+
instanceConnectionName: process.env.POSTGRES_CONNECTION_NAME,
2424
});
2525
const client = new Client({
2626
...clientOpts,
27-
user: String(process.env.POSTGRES_USER),
28-
password: String(process.env.POSTGRES_PASS),
29-
database: String(process.env.POSTGRES_DB),
27+
user: process.env.POSTGRES_USER,
28+
password: process.env.POSTGRES_PASS,
29+
database: process.env.POSTGRES_DB,
3030
});
3131
t.after(async () => {
3232
try {
@@ -48,14 +48,14 @@ t.test('open connection and retrieves standard pg tables', async t => {
4848
t.test('open IAM connection and retrieves standard pg tables', async t => {
4949
const connector = new Connector();
5050
const clientOpts = await connector.getOptions({
51-
instanceConnectionName: String(process.env.POSTGRES_CONNECTION_NAME),
52-
ipType: "PUBLIC",
53-
authType: "IAM",
51+
instanceConnectionName: process.env.POSTGRES_CONNECTION_NAME,
52+
ipType: 'PUBLIC',
53+
authType: 'IAM',
5454
});
5555
const client = new Client({
5656
...clientOpts,
57-
user: String(process.env.POSTGRES_USER_IAM_NODE),
58-
database: String(process.env.POSTGRES_DB),
57+
user: process.env.POSTGRES_IAM_USER,
58+
database: process.env.POSTGRES_DB,
5959
});
6060
t.after(async () => {
6161
try {
@@ -82,9 +82,9 @@ t.test(
8282
});
8383
const client = new Client({
8484
...clientOpts,
85-
user: String(process.env.POSTGRES_USER),
86-
password: String(process.env.POSTGRES_CAS_PASS),
87-
database: String(process.env.POSTGRES_DB),
85+
user: process.env.POSTGRES_USER,
86+
password: process.env.POSTGRES_CAS_PASS,
87+
database: process.env.POSTGRES_DB,
8888
});
8989
t.after(async () => {
9090
try {

system-test/pg-connect.mjs

+3-3
Original file line numberDiff line numberDiff line change
@@ -50,12 +50,12 @@ t.test('open IAM connection and retrieves standard pg tables', async t => {
5050
const connector = new Connector();
5151
const clientOpts = await connector.getOptions({
5252
instanceConnectionName: String(process.env.POSTGRES_CONNECTION_NAME),
53-
ipType: "PUBLIC",
54-
authType: "IAM",
53+
ipType: 'PUBLIC',
54+
authType: 'IAM',
5555
});
5656
const client = new Client({
5757
...clientOpts,
58-
user: String(process.env.POSTGRES_USER_IAM_NODE),
58+
user: String(process.env.POSTGRES_IAM_USER),
5959
database: String(process.env.POSTGRES_DB),
6060
});
6161
t.after(async () => {

system-test/pg-connect.ts

+1-1
Original file line numberDiff line numberDiff line change
@@ -58,7 +58,7 @@ t.test('open IAM connection and retrieves standard pg tables', async t => {
5858
});
5959
const client = new Client({
6060
...clientOpts,
61-
user: String(process.env.POSTGRES_USER_IAM_NODE),
61+
user: String(process.env.POSTGRES_IAM_USER),
6262
database: String(process.env.POSTGRES_DB),
6363
});
6464
t.after(async () => {

0 commit comments

Comments
 (0)