BSR tags for googleapis/googleapis are missing

[andremarianiello]

My buf commands have started producing

Failure: buf.build/googleapis/googleapis:8abb121752d4454db5546c3b4653b450

Here is my buf.lock file

# Generated by buf. DO NOT EDIT.
version: v1
deps:
  - remote: buf.build
    owner: envoyproxy
    repository: protoc-gen-validate
    commit: dc09a417d27241f7b069feae2cd74a0e
  - remote: buf.build
    owner: googleapis
    repository: googleapis
    commit: 8abb121752d4454db5546c3b4653b450
  - remote: buf.build
    owner: grpc-ecosystem
    repository: grpc-gateway
    commit: ff83506eb9cc4cf8972f49ce87e6ed3e

https://buf.build/googleapis/googleapis/tags/main contains only two tags, both from today

Seems like something deleted all tags from before today? Does anyone have any idea what is going on?

[bufdev]

We recently made a change to our managed buf.build/googleapis/googleapis repository before certifying BSR as v1 (even though googleapis is technically just a module, we know that it's a commonly-used module that we maintain, so we consider this to be important not to break post-v1). googleapis contains over 3800 files, mostly relating to Google's core APIs. This causes numerous issues for most users, such as timeouts when installing packages with huge swaths of unused code to pull in just a few files. Not only is this a lot of code over the network and on disk, but it can cause issues in editors that try to parse all of that code!

However, there are only ~30 files used by 99.999% of developers, and these files are the most common dependency in the Protobuf ecosystem. For example:

• google.type, which defines useful messages such as DateTime and Money.-
• google.rpc, for interacting with gRPC.
• google.api, for defining APIs, such as with grpc-gateway.

We considered these commonly used packages and included a subset of them into the new, slim googleapis repository.

We hate breaking anyone, even in an alpha, and our apologies that it's causing any difficulties here. The issues with googleapis were serious enough that we felt that we had to do this in this extraordinary circumstance. We're coming out of beta extremely soon, and once we do, we take our responsibility not to break customers (free users or otherwise) very seriously, we appreciate you riding the rough road for the moment!

[andremarianiello]

Thanks for the explanation! I have a couple follow-up questions if you don't mind:

• Was this announced somewhere? I would like to # to be notified of breakages like this in the future.
• Is the original full repo available somewhere? (containing the rarely used modules)
• Once you are out of alpha/beta, how will breaking changes like this be handled?

[bufdev]

• Instead of announcing this directly, we put a custom error message produced by the backend in this circumstance, as well as updated the docs at https://docs.buf.build/faq#googleapis-failure. We thought that this would be better than a mass email, but in hindsight, we likely should have done both, our apologies.
• The source repo is always available at github.com/googleapis/googleapis. In terms of the Buf module, that is not retained, as retaining it would have continued the stability issues for our customers who depend on googleapis.
• Once we're out of beta on the BSR, we won't have breaking changes like this, so that won't be a concern.

A slight technical note though: modules can break, which is something the publisher is responsible for (just as source code can break on GitHub), and we can't control third-party module content of course. However, googleapis is a commonly-used dependency that we do publish ourselves, and we understand its importance to the Protobuf ecosystem, so post-v1, we won't be making changes like this to googleapis and deleting previous commits. Of course though, Google could choose to break some of the Protobuf definitions, and we can't control that, but we doubt this will happen for this commonly-used definitions.

[andremarianiello]

I have just one more follow-up question: What is the recommended course of action for users who relied on proto files that are no longer part of googleapis/googleapis?

[andremarianiello]

Side note: I didn't get the custom error message, probably because my version of buf is too old, so that's on me.

[bufdev]

What files are you depending on outside of the set now posted? Can you provide a list?

[andremarianiello]

Probably none - I don't know for sure since I haven't version bumped yet. I was asking on behalf of the .001% of users who have this problem, or on behalf of my future self if I want to use those extra protos at some point in the future.

[dragon3]

I've just got this issue.

Failure: buf.build/googleapis/googleapis:04ad98c82478417784639b43e71c6b4c: does not exist

We depend on google/pubsub/v1/pubsub.proto but it doesn't exist in the new module...

Is there any way to get the original full repository? or any other workaround?

[ran-huang]

Same issue here.

Failure: buf.build/googleapis/googleapis:8f04c867ddf14e13a231d5119b8ebb8b: does not exist

[ben-swit]

i have same issue
how to fix it ? so much confuse when i run buf lint only shown Failure: buf.build/googleapis/googleapis:d8957b4333cc4523b3802d0af0e059e3: does not exist

[ben-swit]

i solved buf mod update under all buf.yaml

[zhy1]

I fixed this in my project using the solution below:

• deleted buf.lock
• add buf mod update before the buf lint

[wingsofovnia]

Now we are getting:

❯ buf generate
Failure: internal error, contact support, reference code: 556938

The page https://buf.build/googleapis/googleapis fails with

An error occurred: unexpected server response, missing reference

Update: it briefly reappeared and disappeared again

[bufdev]

The above was an unrelated issue, this is now fixed.