Feat: add traffic mesh proxy flag #1196

erezrokah · 2020-09-02T13:06:03Z

Fixes https://github.com/netlify/traffic-mesh/issues/806

This is a big PR since I did some refactoring to the dev command in the process since it was very hard to follow.
The commits are small though.

I'll add comments to clarify the changes.

This PR consists of 3 main changes:

Extract common logic to download binary clients (this was used in the tunnel client and now shared with traffic mesh agent).
Extract the different dev command responsibilities (starting functions server, framework server, Netlify dev proxy, tunnel client, etc) into separate functions/files.
Traffic Mesh flag implementation which was quite small.

When enabling traffic mesh some test are still failing - looking into those.

erezrokah · 2020-09-02T13:07:43Z

src/lib/exec-fetcher.js

@@ -0,0 +1,90 @@
+const fs = require('fs-extra')


Most of the logic here was extracted from https://github.com/netlify/cli/blob/93468a3fc43db5d637b30b9cbb21042da8626962/src/utils/live-tunnel.js

And made generic so we can re-use it.

erezrokah · 2020-09-02T13:08:34Z

src/lib/exec-fetcher.js

+    })
+    return outdated
+  } catch (e) {
+    if (exists) {


Added a small fix to use existing version if checking for updates failed.
That can happen when reaching GitHub API limit:
netlify/gh-release-fetch#3

erezrokah · 2020-09-02T13:10:19Z

src/lib/exec-fetcher.test.js

+  }
+})
+
+const packages = [


Feels a bit strange to have this here and not in a dedicated test file (one for the tunnel client and one for the traffic mesh).
However I prefer it this way since we're actually testing the exec-fetcher

erezrokah · 2020-09-02T13:12:50Z

src/lib/settings.js

@@ -0,0 +1,15 @@
+const path = require('path')


I've started a new concept of libs instead of utils that we currently have.
It seems that most of our current utils are not used for sharing code, but just to extract functionality from the different commands.
However some commands/utils need to share the same code - so I started putting those parts under lib.

erezrokah · 2020-09-02T13:14:34Z

src/utils/live-tunnel.js

-  if (isWindows()) {
-    return true
-  }
+const startLiveTunnel = async ({ siteId, netlifyApiToken, localPort, log }) => {


createTunnel and connectTunnel were called separately in the dev command.
I grouped them into startLiveTunnel

erezrokah · 2020-09-02T13:16:13Z

src/utils/proxy.js

@@ -0,0 +1,346 @@
+const path = require('path')


This was mostly copied as is from the dev command

erezrokah · 2020-09-02T13:19:46Z

src/commands/dev/index.js

@@ -384,8 +29,11 @@ async function startDevServer(settings, log) {
      },
    })

-    server.start(function() {
-      log(`\n${NETLIFYDEVLOG} Server listening to`, settings.frameworkPort)
+    await new Promise(resolve => {


Added some code to wait until the server is ready

erezrokah · 2020-09-02T13:20:18Z

src/utils/proxy.js

+}
+
+async function startProxy(settings = {}, addonUrls, configPath, projectDir) {
+  const functionsServer = settings.functionsPort ? `http://localhost:${settings.functionsPort}` : null


The original startProxy had some code to wait for the framework server port and functions server port:

cli/src/commands/dev/index.js

Line 170 in 93468a3

try {

cli/src/commands/dev/index.js

Line 178 in 93468a3

if (functionsDir && settings.functionsPort) {

I removed it in favour of:

Using waitPort in the function that starts the framework server: https://github.com/netlify/cli/pull/1196/files#diff-8a2f65e3aede3ea9696e546766f50f03R82

When starting the functions server - wait on a promise until the server is ready: https://github.com/netlify/cli/pull/1196/files#diff-cf2bd94d7688f0711bbd931e6c5e0397R373

erezrokah · 2020-09-02T13:21:07Z

src/utils/proxy.js

+    proxy.ws(req, socket, head)
+  })
+
+  return new Promise(resolve => {


Also added this code to wait for the dev server to be ready

erezrokah · 2020-09-02T13:22:25Z

src/utils/proxy.js

+
+  return new Promise(resolve => {
+    server.listen({ port: settings.port }, () => {
+      resolve(`http://localhost:${settings.port}`)


The original startProxy returned an object with url and port as properties:

cli/src/commands/dev/index.js

Line 235 in 93468a3

return { url: `http://localhost:${settings.port}`, port: settings.port }

but only url was used.
Simplified the function to return just the url as a string

erezrokah · 2020-09-02T13:23:07Z

src/utils/serve-functions.js

@@ -318,4 +310,77 @@ async function serveFunctions(dir, siteInfo = {}) {
  return app
 }

-module.exports = { serveFunctions }
+const getBuildFunction = ({ functionBuilder, log }) => {


Moved this chunk of code from the dev command

erezrokah · 2020-09-02T13:23:28Z

src/utils/serve-functions.js

+    const functionsServer = await serveFunctions(settings.functions, siteInfo)
+
+    await new Promise(resolve => {
+      functionsServer.listen(settings.functionsPort, err => {


Added this to wait until the server is ready

erezrokah · 2020-09-02T13:24:17Z

src/utils/traffic-mesh.js

@@ -0,0 +1,70 @@
+const path = require('path')


This is very similar to the tunnel client code

erezrokah · 2020-09-02T13:25:30Z

src/utils/traffic-mesh.js

+  )
+
+  try {
+    const open = await waitPort({ port, output: 'silent', timeout: 30 * 1000 })


Wait for the agent to be ready (the traffic mesh agent writes to a log file and not stdout) so waiting on the port works well.

erezrokah · 2020-09-02T13:26:09Z

tests/command.dev.test.js

-      path: 'index.html',
-      content: '<h1>⊂◉‿◉つ</h1>',
-    })
+const testMatrix = [


The changes in this file is for support a test matrix with different args (and creating unique test names)

ehmicky

🚀

erezrokah · 2020-09-02T15:01:10Z

@ehmicky wow that was fast. Going to disable the new failing test (they are failing to due GitHub API limit reached on CI).

erezrokah added 11 commits September 2, 2020 14:38

refactor: move os.homedir usage to a dedicated lib

8b9f02c

refactor: extract release downloading into a separate lib

c6036a2

feat(traffic-mesh): add traffic mesh installation

81ca361

refactor: move serve functions code to utils file

7fdc804

refactor: rename startDevServer to startFrameworkServer

7c200f2

refactor: extract startProxy into a separate file

099f1d3

refactor(command-dev): extract logic to functions

bb121d0

feat(command-dev): add forward proxy integration

b4448f2

test(command-dev): add traffic mesh tests

34a79ea

fix(traffic-mesh): use public release version

a2941b2

test(command-dev): disable failing tests

788e51d

github-actions bot added the type: feature code contributing to the implementation of a feature and/or user facing functionality label Sep 2, 2020

erezrokah mentioned this pull request Sep 2, 2020

Use traffic mesh proxy #1193

Closed

erezrokah commented Sep 2, 2020

View reviewed changes

src/utils/proxy.js

@@ -0,0 +1,346 @@

const path = require('path')

Copy link

Contributor Author

erezrokah Sep 2, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was mostly copied as is from the dev command

erezrokah commented Sep 2, 2020

View reviewed changes

src/utils/traffic-mesh.js

@@ -0,0 +1,70 @@

const path = require('path')

Copy link

Contributor Author

erezrokah Sep 2, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is very similar to the tunnel client code

erezrokah commented Sep 2, 2020

View reviewed changes

test(exec-fetcher): avoid reaching GitHub API limit in tests

6114da5

erezrokah requested a review from ehmicky September 2, 2020 13:40

ehmicky approved these changes Sep 2, 2020

View reviewed changes

test(exec-fetcher): disable failing test

868f21b

erezrokah merged commit 894ed44 into master Sep 2, 2020

erezrokah deleted the feat/add_traffic_mesh_proxy branch September 2, 2020 15:22

snyk-bot mentioned this pull request Sep 15, 2021

[Snyk] Fix for 1 vulnerabilities hixio-mh/cli-1#38

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feat: add traffic mesh proxy flag #1196

Feat: add traffic mesh proxy flag #1196

erezrokah commented Sep 2, 2020 •

edited

Loading

erezrokah Sep 2, 2020

erezrokah Sep 2, 2020

erezrokah Sep 2, 2020

erezrokah Sep 2, 2020

erezrokah Sep 2, 2020

erezrokah Sep 2, 2020

erezrokah Sep 2, 2020

erezrokah Sep 2, 2020

erezrokah Sep 2, 2020

erezrokah Sep 2, 2020

erezrokah Sep 2, 2020

erezrokah Sep 2, 2020

erezrokah Sep 2, 2020

erezrokah Sep 2, 2020

erezrokah Sep 2, 2020

ehmicky left a comment

erezrokah commented Sep 2, 2020

Feat: add traffic mesh proxy flag #1196

Feat: add traffic mesh proxy flag #1196

Conversation

erezrokah commented Sep 2, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ehmicky left a comment

Choose a reason for hiding this comment

erezrokah commented Sep 2, 2020

erezrokah commented Sep 2, 2020 •

edited

Loading