Light nodes and light mode of operation - meta + timeline

[gbalint]

https://hackmd.io/s/ry1kbJZfm

In order to be useful on mobile phones and other light hardware (storage and bandwidth impaired) swarm must support a "light mode".

Light on storage:
A swarm node without redundant local storage should be able to advertise to the network that it does not intend to store any chunks. Such a node will not be synced-to by the other nodes.
The node will count as a regular peer in the ordinary Kademlia routing as far as retrieval requests are concerned. It will receive, forward and process retrieval requests (and it may chose to cache the chunks).
However, it will not count towards any node's MostProxBin. This is because this bin is used for redundant storage and the light node will not be storing.

Light on storage and bandwidth:
A node should also be able to advertise that in addition to having no storage, it is also limited in bandwidth (eg a phone not on wifi). When a node indicates that it is in a bandwidth impaired state, its peers will no longer forward retrieval requests to the node. In practice the node is using its peers as swarm gateways.
The node does not qualify as a regular peer in either the MostProxBin nor any other Kademlia bin.

Perhaps it is wise to allow various swarm components to be modular, to be enabled/disabled by nodes at will. This could be part of the initial handshake, or they could also change their status later (such as when a phone connects to wifi, it might start forwarding requests)

So nodes would advertise services:
syncing: on/off (off means non-storing node)
forwarding: on/off (off means bandwidth impaired)
receipting: on/off (on means node is SWEAR registered and will issue storage receipts)
etc.

https://github.com/ethersphere/swarm/wiki/Light-mode-of-operation

see the 3rd network paper http://swarm-guide.readthedocs.io/en/latest/introduction.html#orange-papers

network depends on #628

Swarm client - light mode

Light mode refers to a special mode of operation necessitated by clients need to low resources or balanced services in the network.

Connectivity related issues
How services are launched differently

• syncing
• request forwarding
  Light nodes should not be counted regarding proximity

Types of resource constraints

• CPU > depend on number of connection
• memory > depend on number of connection
• storage
• bandwidth > depend on number of connection
• battery > depend on number of connection

Phases

acceptance criteria for all phases:

• able to create and run a simulation in the simulation framework with an arbitrary mixture of light and normal nodes
• documentation

Phase 0

Goal: able to start a light node

• advertise itself as a light node
• normal nodes would not send retrieval requests
• light nodes reject all retrieval requests
• use minimal storage size

Task 0 - DONE

• Add configuration flag for light nodes
  • goal: it can be set from command line or environment variable that the node is a light node
  • We agreed that setting the light node won't set the storage size, which has its separate config variable
  • We agreed that setting the light node won't change connection parameters in this Task, but only when working on the Kademlia connections (Task 1)
• Communicate that the node is the light node in the handshake
  • goal: nodes in the network understand that who is a light node and who is a swarm normal node
  • swarm: Added lightnode flag ethereum/go-ethereum#17291

Task 1 - DONE

• Light nodes will not send retrieval requests to the swarm light nodes
• Don't save light nodes to addressbook
• Nodes do not want to connect to light nodes - partly result of not saving to address book,
  • to make sure light nodes are not advertised to peers via NotifyPeer (see TASK 4 discovery part)
  • goal: Light node should connect to the network through a normal node, ie., swarm light node has no swarm light node connection

Task 2 - DONE

• Disable syncing, reject subscription requests
• Reject retrieval requests from other nodes

Task 3

• Create a template Makefile for crossplatform compilation
  • goal: developers can build swarm node for mobile platforms
  • TODO: It still needs to be clarified what is the expected output of the mobile build? Do we want a library or a standalone Swarm mobile application? If it's a library, what is the API? If it's an app, what is the expected functionality?
• Some experiments we (@agazso) hacked on a branch: https://github.com/helmethair-co/keydrop-go

Task 4

• light nodes should not count towards nearest neighbours, ie., make sure syncing nodes are connected to the right number of non-lite peers
  • lightnodes should not be accounted for when calculating the neighborhood depth
  • lightnode should not be accounted for when calculating minimum bucket size
  • create special annotation for lightnodes in Kademlia table output
• lightnodes should not participate in message forwarding as part of the forwarding Kademlia
• light nodes should not be advertised in discovery via NotifyPeer
• light nodes should be omitted from the response to SubPeerMessage
• use ENR and prevent spurious p2p discovery
• light mode should be the default operation - light node should be default mode of operation #974
  • flag default implementation
  • This needs to be documented and advertised

Phase 1

goal: Able to switch light/normal node operation on the fly

If we don't change the connectivity strategy for light nodes to fill the kademlia, then even pss message receival can work.

Notes

How others, like mobile bittorrent clients solved this?

Meeting Notes

• communication channel: orange-lounge
• weekly sync from on Thursday 1500 (Budapest time) 28th of July, 2018 on swarm hangout: https://hangouts.google.com/hangouts/_/ethereum.org/swarm
• follow our work on a github board

Resources:
https://github.com/ethersphere/swarm/wiki/Light-mode-of-operation
#458

use cases

• use swarm client on mobile devices
• switch swarm client to active/passive mode for the node owners who just would like to keep the account balance

[agazso]

After discussing Task4 with @justelad, we identified some subtasks that are need to be done:

1. lightnodes should not appear in Peerpot NN set
2. lightnodes should not be accounted for when calculating the neighborhood depth
3. for example a Bin with a single lightnode should be considered empty
4. create special annotation for lightnodes in Kademlia table output
5. lightnodes should not participate in message forwarding as part of the forwarding Kademlia

We were also thinking to create a unified struct for storing conns, addrs and other information (such as a Lightnode flag). Then the different calculations would become easy in the case of a Lightnode.

One question remains if we do this: right now there is a Register function in kademlia.go which is used to register BzzAddr, either when loading the addresses from the database (loadPeers in hive.go) and more importantly from discovery when a peerMsg is arriving (handlePeerMsg in discovery.go). Right now the latter is more problematic, because during discovery the nodes are forwarding lightnode addresses, and with Register they became saved as well. So if we could somehow avoid sending lightnode addresses in discovery, we would be good. However in the peerMsg struct we don't have information about lightnodes, so maybe we should add this on the protocol level.

What do you think of these approaches of the problem?

[zelig]

1. PeerPot is a testing construct only, but yes if it is used to test depth and NN correctness in mixed node networks then yes it should skip light nodes
2. yes
3. yes
4. yes
5. already implemented by Mark 😄

i dont think a uniffied struct is a good idea

6. dont forward light nodes in discovery
   is part of task 2 and I thought this one was done, but it is not i guess
   No need to change either peerMsg or Register AFAICT

[zelig]

@agazso also note this issue #1018 which is about eliminating BzzAddr as well as the bzz handshake and use ENR instead