Skip to content

Releases: ipfs/kubo

v0.11.0-rc2

03 Dec 20:55
Compare
Choose a tag to compare
v0.11.0-rc2 Pre-release
Pre-release

Tracking Issue: #8343

v0.11.0-rc1

01 Dec 19:19
v0.11.0-rc1
Compare
Choose a tag to compare
v0.11.0-rc1 Pre-release
Pre-release

Tracking Issue: #8343

v0.10.0

01 Oct 19:23
v0.10.0
64b532f
Compare
Choose a tag to compare

go-ipfs v0.10.0 Release

We're happy to announce go-ipfs 0.10.0. This release brings some big changes to the IPLD internals of go-ipfs that make working with non-UnixFS DAGs easier than ever. There are also a variety of new commands and configuration options available.

As usual, this release includes important fixes, some of which may be critical for security. Unless the fix addresses a bug being exploited in the wild, the fix will not be called out in the release notes. Please make sure to update ASAP. See our release process for details.

🛠 TLDR: BREAKING CHANGES

  • ipfs dag get
    • default output changed to dag-json
    • dag-pb (e.g. unixfs) field names changed - impacts userland code that works with dag-pb objects returned by dag get
    • no longer emits an additional new-line character at the end of the data output
  • ipfs dag put
    • defaults changed to reduce ambiguity and surprises: input is now assumed to be dag-json, and data is serialized to dag-cbor at rest.
    • --format and --input-enc were removed and replaced with --store-codec and --input-codec
    • codec names now match the ones defined in the multicodec table
    • dag-pb (e.g. unixfs) field names changed - impacts userland code that works with dag-pb objects stored via dag put

Keep reading to learn more details.

🔦 Highlights

🌲 IPLD Levels Up

The handling of data serialization as well as many aspects of DAG traversal and pathing have been migrated from older libraries, including go-merkledag and go-ipld-format to the new go-ipld-prime library and its components. This allows us to use many of the newer tools afforded by go-ipld-prime, stricter and more uniform codec implementations, support for additional (pluggable) codecs, and some minor performance improvements.

This is significant refactor of a core component that touches many parts of IPFS, and does come with some breaking changes:

  • IPLD plugins:
    • The PluginIPLD interface has been changed to utilize go-ipld-prime. There is a demonstration of the change in the bundled git plugin.
  • The semantics of dag put and dag get change:
    • dag get now takes the output-codec option which accepts a multicodec name used to encode the output. By default this is dag-json, which is a strict and deterministic subset of JSON created by the IPLD team. Users may notice differences from the previously plain Go JSON output, particularly where bytes are concerned which are now encoded using a form similar to CIDs: {"/":{"bytes":"unpadded-base64-bytes"}} rather than the previously Go-specific plain padded base64 string. See the dag-json specification for an explanation of these forms.
    • dag get no longer prints an additional new-line character at the end of the encoded block output. This means that the output as presented by dag get are the exact bytes of the requested node. A round-trip of such bytes back in through dag put using the same codec should result in the same CID.
    • dag put uses the input-codec option to specify the multicodec name of the format data is being provided in, and the store-codec option to specify the multicodec name of the format the data should be stored in at rest. These formerly defaulted to json and cbor respectively. They now default to dag-json and dag-cbor respectively but may be changed to any supported codec (bundled or loaded via plugin) by its multicodec name.
    • The json and cbor multicodec names (as used by input-enc and format options) are now no longer aliases for dag-json and dag-cbor respectively. Instead, they now refer to their proper multicodec types. cbor refers to a plain CBOR format, which will not encode CIDs and does not have strict deterministic encoding rules. json is a plain JSON format, which also won't encode CIDs and will encode bytes in the Go-specific padded base64 string format rather than the dag-json method of byte encoding. See https://ipld.io/specs/codecs/ for more information on IPLD codecs.
    • protobuf is no longer used as the codec name for dag-pb
    • The codec name raw is used to mean Bytes in the IPLD Data Model
  • UnixFS refactor. The dag-pb codec, which is used to encode UnixFS data for IPFS, is now represented through the dag API in a form that mirrors the protobuf schema used to define the binary format. This unifies the implementations and specification of dag-pb across the IPLD and IPFS stacks. Previously, additional layers of code for file and directory handling within IPFS between protobuf serialization and UnixFS obscured the protobuf representation. Much of this code has now been replaced and there are fewer layers of transformation. This means that interacting with dag-pb data via the dag API will use different forms:
    • Previously, using dag get on a dag-pb block would present the block serialized as JSON as {"data":"padded-base64-bytes","links":[{"Name":"foo","Size":100,"Cid":{"/":"Qm..."}},...]}.
    • Now, the dag-pb data with dag-json codec for output will be serialized using the data model from the dag-pb specification: {"Data":{"/":{"bytes":"unpadded-base64-bytes"}},"Links":[{"Name":"foo","Tsize":100,"Hash":{"/":"Qm..."}},...]}. Aside from the change in byte formatting, most field names have changed: dataData, linksLinks, SizeTsize, CidHash. Note that this output can be changed now using the output-codec option to specify an alternative codec.
    • Similarly, using dag put and a store-codec option of dag-pb now requires that the input conform to this dag-pb specified form. Previously, input using {"data":"...","links":[...]} was accepted, now it must be {"Data":"...","Links":[...]}.
    • Previously it was not possible to use paths to navigate to any of these properties of a dag-pb node, the only possible paths were named links, e.g. dag get QmFoo/NamedLink where NamedLink was one of the links whose name was NamedLink. This functionality remains the same, but by prefixing the path with /ipld/ we enter data model pathing semantics and can dag get /ipld/QmFoo/Links/0/Hash to navigate to links or /ipld/QmFoo/Data to simply retrieve the data section of the node, for example.
    • ℹ See the dag-pb specification for details on the codec and its data model representation.
    • ℹ See this detailed write-up for further background on these changes.

Ⓜ Multibase Command

go-ipfs now provides utility commands for working with multibase:

$ echo -n hello | ipfs multibase encode -b base16 > file-mbase16
$ cat file-mbase16
f68656c6c6f

$ ipfs multibase decode file-mbase16
hello

$ cat file-mbase16 | ipfs multibase decode
hello

$ ipfs multibase transcode -b base2 file-mbase16
00110100001100101011011000110110001101111

See ipfs multibase --help for more examples.

🔨 Bitswap now supports greater configurability

This release adds an Internal section to the configuration file that is designed to help advanced users optimize their setups without needing a custom binary. The Internal section is not guaranteed to be the same from release to release and may not be covered by migrations. If you use the Internal section you should be making sure to check the config documentation between releases for any changes.

🐚 Programmatic shell completions command

ipfs commands completion bash will generate a bash completion script for go-ipfs commands

📜 Profile collection command

Performance profiles can now be collected using ipfs diag profile. If you need to do some debugging or have an issue to submit the collected profiles are very useful to have around.

🍎 Mac OS notarized binaries

The go-ipfs and related migration binaries (for both Intel and Apple Sillicon) are now signed and notarized to make Mac OS installation easier.

👨‍👩‍👦 Improved MDNS

There is a completed implementation of the revised libp2p MDNS spec. This should result in better MDNS discovery and better local/offline operation as a result.

🚗 CAR import statistics

dag import command now supports --stats option which will include the number of imported blocks and their total size in the output.

🕸 Peering command

This release adds swarm peering command for easy management of the peering subsystem. Peer in the peering subsystem is maintained to be connected at all times, and gets reconnected on disconnect with a back-off.

See ipfs swarm peering --help for more details.

Changelog

Full Changelog
  • github.com/ipfs/go-ipfs:
    • fuse: load unixfs adls as their dagpb substrates
    • enable the legacy mDNS implementation
    • test: add dag get --ouput-codec test
    • change ipfs dag get flag name from format to output-codec
    • test: check behavior of l...
Read more

v0.10.0-rc2

28 Sep 20:41
v0.10.0-rc2
Compare
Choose a tag to compare
v0.10.0-rc2 Pre-release
Pre-release

Tracking Issue: #8176

v0.10.0-rc1

23 Aug 15:22
v0.10.0-rc1
Compare
Choose a tag to compare
v0.10.0-rc1 Pre-release
Pre-release

Tracking Issue: #8176

v0.9.1

22 Jul 00:45
v0.9.1
dc2715a
Compare
Choose a tag to compare

go-ipfs v0.9.1 Release

This is a small bug fix release resolving the following issues:

  1. A regression where the empty CID bafkqaaa could not resolve on gateways #8230
  2. A panic on OpenBSD #8211
  3. High CPU usage with QUIC #8256
  4. High memory usage with TCP #8219
  5. Some pubsub issues (libp2p/go-libp2p-pubsub#427, libp2p/go-libp2p-pubsub#430)
  6. Updated WebUI to v2.12.4
  7. Fixed the snap deployment #8212

Changelog

Full Changelog
  • github.com/ipfs/go-ipfs:
    • chore: update deps
    • feat: webui v2.12.4
    • test: gateway response for bafkqaaa
    • fix: downgrade mimetype dependency
    • update go-libp2p to v0.14.3
    • bump snap to build with Go 1.16
  • github.com/libp2p/go-libp2p (v0.14.2 -> v0.14.3):
  • github.com/libp2p/go-libp2p-pubsub (v0.4.1 -> v0.4.2):
    • release priority locks early when handling batches
    • don't respawn writer if we fail to open a stream; declare it a peer error
    • batch process dead peer notifications
    • use a priority lock instead of a semaphore
    • do the notification in a goroutine
    • emit new peer notification without holding the semaphore
    • use a semaphore for new peer notifications so that we don't block the event loop
    • don't accumulate pending goroutines from new connections
    • Make close concurrent safe
    • Fix close of closed channel
  • github.com/libp2p/go-libp2p-quic-transport (v0.11.1 -> v0.11.2):
    • update quic-go to v0.21.2
  • github.com/libp2p/go-tcp-transport (v0.2.2 -> v0.2.4):
  • github.com/lucas-clemente/quic-go (v0.21.1 -> v0.21.2):
    • update qtls to include the crypto/tls fix of Go 1.16.6 / 1.15.14
    • cancel the PTO timer when all Handshake packets are acknowledged
    • update to Go 1.17rc1
    • update Ginkgo to v1.16.4 and Gomega to v1.13.0 (lucas-clemente/quic-go#3139)
  • github.com/multiformats/go-multiaddr (v0.3.2 -> v0.3.3):

❤ Contributors

Contributor Commits Lines ± Files Changed
vyzo 8 +205/-141 12
Marten Seemann 7 +127/-74 11
gammazero 2 +43/-5 3
Steven Allen 1 +13/-2 1
Adin Schmahmann 3 +13/-2 3
Marcin Rataj 2 +9/-1 2
Aaron Bieber 1 +6/-2 1

v0.9.0

23 Jun 02:00
v0.9.0
179d1d1
Compare
Choose a tag to compare

go-ipfs v0.9.0 Release

We're happy to announce go-ipfs 0.9.0. This release makes go-ipfs even more configurable with some fun experiments to boot. We're also deprecating or removing some uncommonly used features to make it easier for users to discover the easy ways to use go-ipfs safely and efficiently.

As usual, this release includes important fixes, some of which may be critical for security. Unless the fix addresses a bug being exploited in the wild, the fix will not be called out in the release notes. Please make sure to update ASAP. See our release process for details.

🔦 Highlights

📦 Exporting of DAGs via Gateways

Gateways now support downloading arbitrary IPLD graphs via the /api/v0/dag/export endpoint. This endpoint works in the same way as the ipfs dag export command.

One major thing this enables is ability to verify data downloaded from public gateways. If you go to https://somegateway.example.net/ipfs/bafyexample you are using the old school HTTP transport, and trusting that the gateway is being well behaved. However, if you download the graph as a DAG archive then it is possible to verify that the data you downloaded does in fact match bafyexample.

Additionally, it was previously quite painful to download things other than UnixFS (files + directories) using gateways. It is now possible to download arbitrary IPLD graphs from gateways, making them useful as a general-purpose alternative to p2p transports.

This opens exciting opportunities in areas like thin clients, mobile browsers and IoT devices, which now can delegate IPFS resolution to any public gateway, and have ability to verify that the data received matches the requested hash.

☁ Custom DNS Resolvers

Resolution of DNS records for DNSLink and DNSAddrs means that names are sent in cleartext between the operating system and the DNS server provided by an ISP. In the past, the only way to customize DNS resolution in IPFS stack was to set up own DNS proxy server.

There is now the ability to customize DNS resolution and override the default resolver from the OS with DNS over HTTPS (DoH) one. We made it really flexible: override can be applied globally, or per specific TLD/FQDN. Examples can be found in the documentation.

👪 Support for non-ICANN DNSLink names

Building off of the support for custom DNS resolvers it is now possible to create DNSLink names not handled by ICANN and choose how that domain name will be resolved. An example of this is how ENS is supported, despite .eth not being an ICANN TLD you can point .eth to any ENS resolver you want (including a local one).

While go-ipfs may have some DoH defaults for a few popular non-ICANN DNSLink names (e.g. ENS), you are free to use any protocol for a naming system and as long as it exposes a DNSLink record via a DNS endpoint you can make it work.

🖥️ Updated to the latest WebUI

Our web interface now includes experimental support for pinning services, and various updates to Files and Peers screens.

Remote pinning services added via the ipfs pin remote service add command are already detected, one can also add one from Settings screen, and it will appear in Set pinning interface on the Files screen.

Data presented on the Peers screen can now be copied by simply clicking on a specific cell, and a list of open streams gives better insight into how a local node interacts with a specific peer.

See release notes for ipfs-webui v2.12 for screenshots and more details.

🔑 IPNS keys can now be exported via the CLI without stopping the daemon

ipfs key export no longer requires interrupting ipfs daemon

🕸 Experimental DHT Client and Provider System

An area of go-ipfs that has been historically tricky is how go-ipfs finds who has the data they are looking for. While the IPFS Public DHT is only one of the ways go-ipfs can find data it tends to be an important one. While since go-ipfs v0.5.0 the time to find content in the network has dropped significantly the time to put/get IPNS records or for a node to advertise the content it has still has much room for improvement.

We have been doing some experimenting and have an alternative DHT client that essentially trades off some resources and in return is much more performant. We have also included with the experimental DHT client a bulk provider system that takes advantage of the new client to more efficiently do many advertisements at a time

This work is quite new and still under development, however, the results so far have been promising especially for users with lots of data who have otherwise been having difficulty advertising their data into the IPFS Public DHT

As described in the experimental features documentation the experimental client can be enabled using the command below (or modifying the config file).

ipfs config --json Experimental.AcceleratedDHTClient true

A few things to take note of when AcceleratedDHTClient is enabled:

  • go-ipfs will likely use more resources then previously
  • DHT queries will not be usable (i.e. finding which peers have some data, finding where a particular peer is, etc.) for the first 5-10 minutes of operation depending on your network conditions
  • There is an ipfs stats provide command that will help you track your provide/reprovide usage, if you are providing lots of data you may want to consider how to reduce the amount you are providing (e.g. Reprovider Strategies and/or Strategic Providing)

See the documentation for more details.

🚶‍♀️ Migrations

Migrations are now individually packaged

While previously the go-ipfs repo migration binary was monolithic and contained all migrations from previous go-ipfs versions the binaries are now packaged individually. However, the fs-repo-migrations binary is still there to help those who manually upgrade their repos to download all the individual migrations.

This means faster download times for upgrades, a much easier time building migrations for those who make use of custom plugins, and an easier time developing new migrations going forward.

Configurable migration downloads enable downloading over IPFS

Previously the migration downloader built into go-ipfs downloaded the migrations from dist.ipfs.io. While users could use tools like ipfs-update to download the migrations over IPFS or manually download the migrations (over IPFS or otherwise) themselves, this is now automated and configurable. Users can choose to download the migrations over IPFS or from any specified IPFS Gateway.

The configurable migration options are described in the config file documentation, although most users should not need to change the default settings.

The main benefit here is that users behind restrictive firewalls, or in offline/private deployments, won't have to run migrations manually, which is especially important for desktop use cases where go-ipfs is running inside of IPFS Desktop and Brave.

🍎 Published builds for Apple M1 hardware

Go now supports building for Darwin ARM64, and we are now publishing those builds

👋 Deprecations and Feature Removals

The ipfs object commands are now deprecated

In the last couple years most of the Object API's commands have become fulfillable using alternative APIs.

The utility of Object API's is limited to data in UnixFS-v1 (dag-pb) format. If you are still using it, it is highly recommended that you switch to the DAG ipfs dag (supports modern data types like dag-cbor) or Files ipfs files (more intuitive for working with dag-pb) APIs.

While the Object API and commands are still usable they are now marked as deprecated and hidden from users on the command line to discourage further use. We also updated their --help text to point at the modern replacements.

X-Ipfs-Gateway-Prefix is now deprecated

IPFS community moved towards dedicated Origins (DNSLink and subdomain gateways) which are much easier to isolate and reason about.

Setting up Gateway.PathPrefixes and X-Ipfs-Gateway-Prefix is no longer necessary and support will be removed in near future.

Proquints support removed

A little known feature that was not well used or documented and was more well known for the error message Error: not a valid proquint string users received when trying to download invalid IPNS or DNSLink names (e.g. https://dweb.link/ipns/badname). We have removed support for proquints as they were out of place and largely unused, however proquints are valid multibases so if there is renewed interest in t...

Read more

v0.9.0-rc2

28 May 00:20
v0.9.0-rc2
Compare
Choose a tag to compare
v0.9.0-rc2 Pre-release
Pre-release

Tracking Issue: #8058

v0.9.0-rc1

14 May 13:27
v0.9.0-rc1
Compare
Choose a tag to compare
v0.9.0-rc1 Pre-release
Pre-release

Tracking Issue: #8058

v0.8.0

19 Feb 00:38
v0.8.0
ce693d7
Compare
Choose a tag to compare

go-ipfs 0.8.0 Release

We're happy to announce go-ipfs 0.8.0! This release is focused on making it easier to work with pins. We have some snazzy new features around being able to ask remote services to pin data for you, and modifying large pin sets is much faster than ever before.

🔦 Highlights

🧷 Remote pinning services

There is now support for asking remote services to pin data for you.

This comes with a redesign of how we're thinking about pinning and includes some commonly requested features such as:

  • Pins can have names (and coming soon metadata)
  • Data can be pinned in either the foreground or background
  • Pins can be searched by name, CID, or status

Command-line users benefit from ipfs pin remote commands, which simplify remote pinning operations. The built-in pinning service API client executes all the necessary remote calls under the hood. HTTP API users have access to new commands under /api/v0/pin/remote.

As long a service supports the vendor-agnostic IPFS Pinning Service API, it can be used directly in go-ipfs.

Examples include:

ipfs pin remote service add mysrv https://my-service.example.com/api-endpoint myAccessToken
ipfs pin remote service ls --stat # confirm service mysrv is available

ipfs pin remote add /ipfs/bafymydata --service=mysrv --name=myfile  # will block until status is pinned
ipfs pin remote ls --service=mysrv --name=myfile
ipfs pin remote rm --serivce=mysrv --name=myfile

ipfs pin remote add /ipfs/bafymydata2 --service=mysrv --name=myfile2 --background  # queue pin request and finish instantly
ipfs pin remote ls --service=mysrv --cid=bafymydata2 --status=queued,pinning,pinned,failed
ipfs pin remote rm --serivce=mysrv --cid=bafymydata2 --status=queued,pinning,pinned,failed

More examples can be found under ipfs pin remote --help

A few notes:

  • Remote pinning services work with recursive pins. This means commands like ipfs pin remote ls will not list indirectly pinned CIDs.
  • By default, only finished, successful pins are listed. To list or remove pending/failed pins, pass explicit status: --status=queued,pinning,pinned,failed
  • While pinning service data is stored in the configuration file it cannot be edited directly via the ipfs config commands due to the sensitive nature of pinning service API keys. The ipfs pin remote service commands can be used for interacting with remote service settings.
  • An OpenAPI ipfs-pinning-service.yaml makes it easy to create or generate a compatible client/server. Anyone can implement it and allow for pin management.

🏠 Remote MFS pinning policy

Every service added via ipfs pin remote service add can be tasked to update a pin every time MFS root changes:

$ ipfs config --json Pinning.RemoteServices.mysrv.Policies.MFS.Enable

To avoid flooding remote service with many updates, go-ipfs will send them at most once every five minutes.

Details about customizing behavior of this feature can be found in configuration docs.

📌 Faster local pinning and unpinning

The pinning subsystem has been redesigned to be much faster and more flexible in how it tracks pins. For users who are working with many pins this will lead to a big speed increase in listing and modifying the set of pinned items as well as decreased memory usage.

Part of the redesign was setup to account for being able to interact with local pins the same way we can now interact with remote pins (e.g. names, being allowed to pin the same CID multiple times, etc.). Keep posted for more improvements to pinning.

🔒 DNSLink names on https:// subdomains

Previously DNSLink names would have trouble loading over subdomain gateways with HTTPS support since there is no way to get multilevel wildcard certificates (e.g. en.wikipedia-on-ipfs.org.ipns.dweb.link cannot be covered by TLS wildcard cert for *.ipns.dweb.link). Therefore, when trying to load DNSLink names over https:// subdomains go-ipfs we now forward to an inlined DNS name. Since DNS labels cannot contain . in them they are escaped using -.

/ipns/en.wikipedia-on-ipfs.org
ipns://en.wikipedia-on-ipfs.org
https://dweb.link/ipns/en.wikipedia-on-ipfs.org
https://en-wikipedia--on--ipfs-org.ipns.dweb.link 👈 a single DNS label, no TLS error 🔒

Note: The last redirect is specific to HTTPS, and is triggered only when X-Forwarded-Proto: https header is present.
Recipes for setting up your own public gateway can be found in configuration docs.

💨 QUIC update

QUIC support has received a number of upgrades, including the ability to take advantage of larger UDP receive buffers for increased performance.

Linux users may notice a logged error on daemon startup if your system needs extra configuration to allow IPFS increase the buffer size. A helpful link for resolving this is in the log message as well as here.

👋 No more Darwin 386 builds

Go 1.15 (the latest version of Go) no longer supports Darwin 386 and so we are dropping support as well.

Changelog

Full Changelog
Read more