mesh-vpn: fully abstract VPN methods #2102

blocktrron · 2020-08-20T17:11:15Z

This fully abstracts VPN methods, making gluon-mesh-vpn-fastd and
gluon-mesh-vpn-tunneldigger completely self-contained.

Provide a LUA interface for generic interacting with VPN methods in
gluon-mesh-vpn-core and web packages.

This also adds the ability to install tunneldigger and fastd to the same
image, selecting the VPN method based on the selected domain.

package/gluon-mesh-vpn-core/luasrc/usr/lib/lua/gluon/mesh-vpn/core.lua

neocturne · 2020-09-06T15:44:52Z

I see a few conceptual issues that need to be discuss:

Should the chosen VPN implementation only be a matter of site/domain config, or should is be possible for the user to change this (for example turning the security/performance mode of gluon-web-mesh-vpn-fastd into a generic switch hat selects between fastd and tunneldigger)?
There are a few places in this PR that query a specific method whether it is enabled. This doesn't make sense to me - VPN enable status should be a single value in /etc/config/gluon. The enabled flags of the individual VPN implementation config files are dependent values that are populated based on the global enable status and the selected VPN implementation; they should only be set, but never be queried by Gluon's scripts (except for migration purposes, in case the global enable status doesn't exist yet)

package/gluon-mesh-vpn-fastd/luasrc/usr/lib/lua/gluon/mesh-vpn/method/fastd.lua

package/gluon-mesh-vpn-core/luasrc/usr/lib/lua/gluon/mesh-vpn/core.lua

package/gluon-mesh-vpn-fastd/luasrc/usr/lib/lua/gluon/mesh-vpn/method/fastd.lua

blocktrron · 2020-09-09T03:57:49Z

@NeoRaider

For the first point, I would propose to leave it to the site configuration for now. The goal of this PR is not to add this functionality but to remove the core VPN package dependencies on the specific VPN implementation packages.

Your second remark is correct. I've simply didn't correctly interpret the context and it's goal for which the specific VPN states are extracted. I'll fix that.

package/gluon-config-mode-mesh-vpn/luasrc/lib/gluon/config-mode/reboot/0100-mesh-vpn.lua

package/gluon-config-mode-mesh-vpn/luasrc/lib/gluon/config-mode/wizard/0300-mesh-vpn.lua

package/gluon-mesh-vpn-core/luasrc/usr/lib/lua/gluon/mesh-vpn/core.lua

package/gluon-mesh-vpn-core/luasrc/lib/gluon/mesh-vpn/update-config

package/gluon-mesh-vpn-fastd/luasrc/usr/lib/lua/gluon/mesh-vpn/method/fastd.lua

neocturne · 2020-10-21T22:00:36Z

package/gluon-mesh-vpn-core/luasrc/lib/gluon/mesh-vpn/update-config

 end
-uci:save('tunneldigger')


You are removing the code that disables the unavailable VPNs, this is not correct.

Hmm, I wrote the code this way so that even VPNs that are not installed anymore get disabled (for the case when for example fastd is installed, but gluon-mesh-vpn-fastd isn't).

What we really want is a generic facility that allows removed packages to clean up after themselves on firmware upgrades, that way we wouldn't need a full list of packages that could exist. I have a few ideas how to solve this...

package/gluon-mesh-vpn-fastd/luasrc/usr/lib/lua/gluon/mesh-vpn/method/fastd.lua

package/gluon-mesh-vpn-core/luasrc/usr/lib/lua/gluon/mesh-vpn/core.lua

CodeFetch · 2021-02-16T16:47:48Z

@NeoRaider

Should the chosen VPN implementation only be a matter of site/domain config, or should is be possible for the user to change this (for example turning the security/performance mode of gluon-web-mesh-vpn-fastd into a generic switch hat selects between fastd and tunneldigger)?

I've been working on IPSec using L2TP where I'd like to be able to let the user configure the encryption this way. Please make it as generic as possible or we will rip it out again, later.

blocktrron · 2021-04-28T21:42:50Z

Fixed all outstanding comments.

Regarding the teardown of old VPN methods - I tend to leave it as - is for now. This might not be perfect, but we remain consistent in the behavior.

For the future, we can have a teardown manager which manages teardown in case a package does no exist anymore. The goal of allowing a third-party VPN method from a community feed is still possible this way, they have to take care of teardown themselves though.

@AiyionPrime Not build / ru ntested yet although the linter is happy.

AiyionPrime · 2021-04-29T11:52:29Z

Thanks!
I just compiled and installed it on one of my cudy wr1000 (ramips-mt76x8).
http://[2001:678:978:114:b64b:d6ff:fe22:4ebc]/cgi-bin/status

You five got access to it, welcome to my kitchen; obviously leave the rest of the network untouched.

blocktrron · 2021-05-17T19:08:17Z

Addressed comments and rebased on #2223

AiyionPrime · 2021-05-19T11:58:11Z

There are currently 17 occurrences of proto, are they meant to stay?

AiyionPrime · 2021-05-19T14:11:15Z

Looks better, I'll test later, when the dust has settled a little.

blocktrron · 2021-05-19T21:12:26Z

Works from my testing.

AiyionPrime · 2021-05-20T08:42:51Z

Compiles and gluon-reconfigure does it's job. I'll close blocktrron#6 then.

AiyionPrime

Just reread the code once more.

blocktrron · 2021-05-22T04:52:20Z

package/gluon-web-admin/files/lib/gluon/config-mode/view/admin/info.html

-	if meshvpn_enabled then
-		pubkey = util.trim(util.exec('/etc/init.d/fastd show_key mesh_vpn'))
+	if has_vpn and vpn.enabled() then
+		local _, active_vpn = vpn.get_active_provider()


resolved in 98a1c19

blocktrron · 2021-05-22T04:53:04Z

package/gluon-mesh-vpn-tunneldigger/luasrc/lib/lua/gluon/mesh-vpn/provider/tunneldigger.lua

+end
+
+function M.uci_sections()
+	return {'tunneldigger'}


These methods are now obsolete anyways, as uci::commit is called globally on reconfigure anyways. I removed them now.

resolved in 98a1c19

blocktrron · 2021-05-22T04:52:14Z

package/gluon-config-mode-mesh-vpn/luasrc/lib/gluon/config-mode/reboot/0100-mesh-vpn.lua

-		pubkey = util.trim(util.exec("/etc/init.d/fastd show_key mesh_vpn"))
+if vpn.enabled() then
+	local _, active_vpn = vpn.get_active_provider()
+	pubkey = active_vpn.public_key()


resolved in 98a1c19

package/gluon-mesh-vpn-core/luasrc/usr/lib/lua/gluon/mesh-vpn.lua

+	local out = {}
+
+	for _, v in ipairs(util.glob('/lib/gluon/mesh-vpn/provider/*')) do
+		table.insert(out, v:match('([^/]+)$'))


This fully abstracts VPN methods, making gluon-mesh-vpn-fastd and gluon-mesh-vpn-tunneldigger completely self-contained. Provide a LUA interface for generic interacting with VPN methods in gluon-mesh-vpn-core and web packages. This also adds the ability to install tunneldigger and fastd to the same image, selecting the VPN method based on the selected domain. Signed-off-by: David Bauer <mail@david-bauer.net>

AiyionPrime

Looks good to me.
@NeoRaider or @mweinelt may you review once more?

AiyionPrime · 2021-05-26T08:47:30Z

I compiled the code before the weekend; and just did so again. It's deployed on http://meshvpnabstracttest.n.ffh.zone/cgi-bin/status.
You all still should have access to it.

blocktrron · 2021-05-26T19:15:33Z

Verified initial setup & upgrades are working in a fastd environment.

Let's do this.

The Gluon Tunneldigger protocol handler hasn't received the care and attention a core package, especially a VPN protocol, should. Due to the works in freifunk-gluon#2102 it's no longer required to maintain VPN handlers in the Gluon Core and they can be instead included via a package feed. Users who wish to continue using a supported core Gluon package are encouraged to take a look at fastd in conjunction with it's null@l2tp method. It offers roughly the same performance (it's using the same principle of operation) and supports connections via IPv6 as a bonus. In general it should be possible to change the VPN used protocol asynchronously with a normal update. If Gluon users wish to continue using tunneldigger the recomendation is to participate in the development of tunneldigger inside our community-packages.

The Gluon Tunneldigger protocol handler hasn't received the care and attention a core package, especially a VPN protocol, should. Due to the works in #2102 it's no longer required to maintain VPN handlers in the Gluon Core and they can be instead included via a package feed. Users who wish to continue using a supported core Gluon package are encouraged to take a look at fastd in conjunction with it's null@l2tp method. It offers roughly the same performance (it's using the same principle of operation) and supports connections via IPv6 as a bonus. In general it should be possible to change the VPN used protocol asynchronously with a normal update. If Gluon users wish to continue using tunneldigger the recomendation is to participate in the development of tunneldigger inside our community-packages.

The Gluon Tunneldigger protocol handler hasn't received the care and attention a core package, especially a VPN protocol, should. Due to the works in freifunk-gluon#2102 it's no longer required to maintain VPN handlers in the Gluon Core and they can be instead included via a package feed. Users who wish to continue using a supported core Gluon package are encouraged to take a look at fastd in conjunction with it's null@l2tp method. It offers roughly the same performance (it's using the same principle of operation) and supports connections via IPv6 as a bonus. In general it should be possible to change the VPN used protocol asynchronously with a normal update. If Gluon users wish to continue using tunneldigger the recomendation is to participate in the development of tunneldigger inside our community-packages.

blocktrron force-pushed the mesh-vpn-abstraction branch from c3b5ddd to ba34418 Compare August 20, 2020 17:14

blocktrron mentioned this pull request Aug 20, 2020

Gluon mesh vpn wireguard vxlan freifunk-gluon/community-packages#6

Merged

rotanid added the 0. type: enhancement The changeset is an enhancement label Aug 22, 2020

rotanid requested a review from neocturne September 2, 2020 22:15

neocturne reviewed Sep 6, 2020

View reviewed changes

package/gluon-mesh-vpn-core/luasrc/usr/lib/lua/gluon/mesh-vpn/core.lua Outdated Show resolved Hide resolved

neocturne reviewed Sep 6, 2020

View reviewed changes

package/gluon-mesh-vpn-fastd/luasrc/usr/lib/lua/gluon/mesh-vpn/method/fastd.lua Outdated Show resolved Hide resolved

neocturne reviewed Sep 6, 2020

View reviewed changes

package/gluon-mesh-vpn-core/luasrc/usr/lib/lua/gluon/mesh-vpn/core.lua Outdated Show resolved Hide resolved

neocturne reviewed Sep 6, 2020

View reviewed changes

package/gluon-mesh-vpn-fastd/luasrc/usr/lib/lua/gluon/mesh-vpn/method/fastd.lua Outdated Show resolved Hide resolved

blocktrron force-pushed the mesh-vpn-abstraction branch from ba34418 to dc13502 Compare September 9, 2020 03:53

blocktrron force-pushed the mesh-vpn-abstraction branch 2 times, most recently from 8fff333 to d84c0a1 Compare September 9, 2020 04:07

rotanid added this to the 2020.3 milestone Sep 13, 2020

rotanid requested a review from neocturne October 19, 2020 21:02

neocturne requested changes Oct 21, 2020

View reviewed changes

rotanid added the 2. status: waiting-on-author Waiting on some action from the author label Oct 24, 2020

lemoer mentioned this pull request Jan 4, 2021

Add package gluon-mesh-vpn-wireguard using wgpeerselector #2168

Merged

AiyionPrime added a commit to AiyionPrime/gluon that referenced this pull request Apr 24, 2021

squashme: resolved freifunk-gluon#2102 (comment)

7030e12

AiyionPrime added a commit to AiyionPrime/gluon that referenced this pull request Apr 24, 2021

squashme: resolved freifunk-gluon#2102 (comment)

5553180

AiyionPrime added a commit to AiyionPrime/gluon that referenced this pull request Apr 24, 2021

squashme: resolved freifunk-gluon#2102 (comment)

7c48114

mweinelt mentioned this pull request Apr 27, 2021

Add Wireguard Support to gluon #1534

Closed

2 tasks

blocktrron force-pushed the mesh-vpn-abstraction branch 4 times, most recently from 3688b9c to fae8219 Compare April 28, 2021 21:35

blocktrron force-pushed the mesh-vpn-abstraction branch from fae8219 to 251d550 Compare April 29, 2021 23:22

blocktrron force-pushed the mesh-vpn-abstraction branch from e4b5414 to ef5fe30 Compare May 17, 2021 14:31

github-actions bot added 3. topic: config-mode This is about the configuration mode 3. topic: fastd 3. topic: multidomain 3. topic: package Topic: Gluon Packages 3. topic: tunneldigger This is about tunneldigger, a L2TP brokering solution labels May 17, 2021

blocktrron force-pushed the mesh-vpn-abstraction branch from ef5fe30 to fca24cc Compare May 19, 2021 11:42

github-actions bot removed the 3. topic: multidomain label May 19, 2021

blocktrron force-pushed the mesh-vpn-abstraction branch 2 times, most recently from 330ed59 to 4d737b8 Compare May 19, 2021 13:44

AiyionPrime requested changes May 20, 2021

View reviewed changes

blocktrron force-pushed the mesh-vpn-abstraction branch from 4d737b8 to 3ee6a02 Compare May 22, 2021 04:53

blocktrron force-pushed the mesh-vpn-abstraction branch from 3ee6a02 to 98a1c19 Compare May 22, 2021 04:54

AiyionPrime approved these changes May 26, 2021

View reviewed changes

blocktrron merged commit 71c3b83 into freifunk-gluon:master May 26, 2021

blocktrron deleted the mesh-vpn-abstraction branch May 26, 2021 19:15

herbetom mentioned this pull request Dec 18, 2023

gluon-mesh-vpn-tunneldigger: drop package #3109

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

mesh-vpn: fully abstract VPN methods #2102

mesh-vpn: fully abstract VPN methods #2102

blocktrron commented Aug 20, 2020

neocturne commented Sep 6, 2020

blocktrron commented Sep 9, 2020

neocturne Oct 21, 2020

neocturne Oct 21, 2020

CodeFetch commented Feb 16, 2021

blocktrron commented Apr 28, 2021

AiyionPrime commented Apr 29, 2021

blocktrron commented May 17, 2021

AiyionPrime commented May 19, 2021

AiyionPrime commented May 19, 2021

blocktrron commented May 19, 2021

AiyionPrime commented May 20, 2021

AiyionPrime left a comment

This comment was marked as resolved.

blocktrron May 22, 2021

AiyionPrime May 26, 2021

This comment was marked as resolved.

blocktrron May 22, 2021

AiyionPrime May 26, 2021

This comment was marked as resolved.

blocktrron May 22, 2021

AiyionPrime May 26, 2021

This comment was marked as resolved.

This comment was marked as resolved.

AiyionPrime left a comment

AiyionPrime commented May 26, 2021

blocktrron commented May 26, 2021

		end
		uci:save('tunneldigger')

mesh-vpn: fully abstract VPN methods #2102

mesh-vpn: fully abstract VPN methods #2102

Conversation

blocktrron commented Aug 20, 2020

neocturne commented Sep 6, 2020

blocktrron commented Sep 9, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

CodeFetch commented Feb 16, 2021

blocktrron commented Apr 28, 2021

AiyionPrime commented Apr 29, 2021

blocktrron commented May 17, 2021

AiyionPrime commented May 19, 2021

AiyionPrime commented May 19, 2021

blocktrron commented May 19, 2021

AiyionPrime commented May 20, 2021

AiyionPrime left a comment

Choose a reason for hiding this comment

This comment was marked as resolved.

Choose a reason for hiding this comment

Choose a reason for hiding this comment

This comment was marked as resolved.

Choose a reason for hiding this comment

Choose a reason for hiding this comment

This comment was marked as resolved.

Choose a reason for hiding this comment

Choose a reason for hiding this comment

This comment was marked as resolved.

This comment was marked as resolved.

AiyionPrime left a comment

Choose a reason for hiding this comment

AiyionPrime commented May 26, 2021

blocktrron commented May 26, 2021