2 out of 3 proxies are broken

[giantplaceholder]

Hi there, this is just an FYI.

There are three proxies used for downloading the relay's list: one on Github, one on Bitbucket, and one via another app.

Right now only Github mirror is working (although it lags behind a bit behind onionoo.torproject.org), whilst Bitbucket one is severely outdated (last updated in April), and the other one is outright broken (script can't parse the output).

[ValdikSS]

whilst Bitbucket one is severely outdated

Fixed, thanks.

and the other one is outright broken (script can't parse the output).

Yeah, it's broken for some reason. I'll take a look.

[giantplaceholder]

Yeah, it's broken for some reason. I'll take a look.

I suspect that the issue is whatever UA is being used, since plain curl grabs the list with no issues.

[ValdikSS]

I suspect that the issue is whatever UA is being used, since plain curl grabs the list with no issues.

Nope, curl doesn't grab either. It seems that CORS proxy adds '=' in the end if the query by mistake. Some sites work, but most don't.

curl 'https://icors.vercel.app/?https%3A%2F%2Fonionoo.torproject.org%2Fdetails%3Ftype%3Drelay%26running%3Dtrue%26fields%3Dfingerprint%2Cor_addresses%2Ccountry' -v

…
< server: Vercel
< strict-transport-security: max-age=63072000; includeSubDomains; preload
< x-final-url: https://onionoo.torproject.org/details?type=relay&running=true&fields=fingerprint,or_addresses,country=
< x-request-url: https://onionoo.torproject.org/details?type=relay&running=true&fields=fingerprint,or_addresses,country=
< x-vercel-cache: MISS
< x-vercel-id: arn1::sfo1::q79b7-1734548861974-f6def2ea47a6
< content-length: 464
…

Note = after country, which should not be there. It doesn't work for me

[giantplaceholder]

I suspect that the issue is whatever UA is being used, since plain curl grabs the list with no issues.

Nope, curl doesn't grab either. It seems that CORS proxy adds '=' in the end if the query by mistake. Some sites work, but most don't.

curl 'https://icors.vercel.app/?https%3A%2F%2Fonionoo.torproject.org%2Fdetails%3Ftype%3Drelay%26running%3Dtrue%26fields%3Dfingerprint%2Cor_addresses%2Ccountry' -v

…
< server: Vercel
< strict-transport-security: max-age=63072000; includeSubDomains; preload
< x-final-url: https://onionoo.torproject.org/details?type=relay&running=true&fields=fingerprint,or_addresses,country=
< x-request-url: https://onionoo.torproject.org/details?type=relay&running=true&fields=fingerprint,or_addresses,country=
< x-vercel-cache: MISS
< x-vercel-id: arn1::sfo1::q79b7-1734548861974-f6def2ea47a6
< content-length: 464
…

Note = after country, which should not be there. It doesn't work for me

I've checked - seems that https://api.codetabs.com/v1/proxy/?quest= works for this particular case. I've build your scanner manually, and it seems to work fine and grabs the list succesfully.

Tbh, I've know idea whether they have any rate limit or not, but I guess it'll be fine, since this is only one of the options, and the client is not the app per se, but the user's machine in itself.

https://universal-cors-proxy.glitch.me works as well, but only if symbols are encoded (i.e. %2F instead of slash, %2C instead of comma and so on). This might be the issue with the app on Vercel, btw.