2-step verification (OAuth) not supported in mobileclient or webclient

[jakealstad]

https://support.google.com/accounts/answer/180744?hl=en

I plan to investigate what it will take to implement this.

[simon-weber]

Sounds good. It's been a long time since I've looked at auth stuff, and I'm really not sure how 2-factor interacts with clientlogin apart from app-specific passwords.

As a heads up, the musicmanager client does support 2-factor -- I should update the docs to make this clearer.

[unforced]

I haven't had a chance to look at the code yet, but is it possible to do anything with OAuth? That seems like the better solution to this problem.

[simon-weber]

I'd love to have OAuth support for the webclient/mobileclient -- plaintext login is awful. However, last I checked, neither of them supported it.

I'd be thrilled to be proved wrong! =)

[jfeldstein]

+1 for being able to oauth into google music. It would pave the way for amazing apps (without requiring users hand over their google passwords)

[simon-weber]

@jfeldstein you can kind of get away with building apps from the musicmanager currently, since it's got oauth and song listing/streaming. Managing device ids is a pain, though.

[simon-weber]

The newish iOS app uses OAuth and uses a set of scopes I've never seen before. I haven't looked at the android client recently, but maybe it uses them as well.

Anyone want to check it out?

[bartolkaruza]

Can you list the scopes that you found in the IOS app? Thanks.

[simon-weber]

@bartolkaruza The main scope is https://www.googleapis.com/auth/skyjam.

[bartolkaruza]

I attempted to use that scope, which gave me an OAuth token succesfully. When adding the token as a bearer to the Authentication header on the sj/v1.3 requests, the response changed from a 401 to a 403: Access not configured. This suggests that OAuth would work if you could somehow get your app to have official access to Google Music through the API console?

[simon-weber]

It's worth a shot, though I don't think it would work: I'd imagine that only the iOS endpoints (which are totally different from any others) are configured to work with that scope.

Let me know how it goes, though! You should be able to hook up a new app in the console to that scope.

[NullVoxPopuli]

I would like this too. :-)

[simon-weber]

I managed to intercept a mostly-successful OAuth flow on Android today! Unfortunately, there's some bug that prevents the last step from actually working -- hitting the auth endpoint for google music results in Error=ServerError.

Here's hoping that step just works when I recreate it on my own.

[simon-weber]

I looked into this a bit more, and I don't think we should get our hopes up =(

The flow I observed is the android-specific flow for connecting a Google account. The end result:

• doesn't provide auth tokens in a way that normal app code can consume
• isn't limited in scope to just google music

Yes, it would allow users to use two-factor without an app-specific password, but it would come at considerable UX cost and would still give the third party app access to basically all the user's google data.

---

So, to sum up the current situation:

• web: no real hope of getting OAuth. Two-factor possible, but annoying to implement.
• android: no OAuth currently, but maybe in the future. Two-factor technically possible today, but at terrible UX cost.
• music manager: OAuth/two-factor implemented! ✨ But, very limited set of operations 💩
• iOS: OAuth/two-factor possible (I think). Expensive to implement -- I'd need an iOS device and a few days off =P

[simon-weber]

I've confirmed that the iOS app performs OAuth in a totally normal way =)

Also, while the iOS protocol is very different on the surface, the actual methods and params are basically the same.

[sauyon]

I have an iOS device, so I can (hopefully?) try to set up OAuth. ClientLogin is deprecated and we need to move away from it anyway.

[ghost]

Hey @simon-weber,

I'm trying to go the OAuth route on Android after authenticating the user with OAuth by way of AccountManager.

I'm using the scope oauth2:https://www.google.com/accounts/OAuthLogin the way many apps like Andlytics do to access services that don't have a specific scope.

I've successfully gotten logged in and have a logged in httpclient - the response I get after the initial login contains a full Play Music page with a bunch of cookies (GAPS, LSID, HSID, SSID, APISID, SAPISID, ACCOUNT_CHOOSER, NID, SID, sjsaid, xt).

I've tried modifying the API so that I can pass this 'authenticated' httpclient in and use it for making requests, but no luck so far. I see you do quite a bit of manipulation of headers and the like, and that you're using an 'Auth' token that you get after logging in the user with email and password, which, as you've said, we can't get through OAuth.

Any thoughts on how I might get this to work?

[simon-weber]

Hey @sciencepro! Sorry for the late response, I was out of town.

Woah, I really didn't think AccountManager would work. That's great news!

Any thoughts on how I might get this to work?

What do you mean by "authenticated httpclient"? Are you manually attaching an OAuth header (like session.Musicmanager) or are the cookies you've got sufficient?

[ghost]

Should have followed up on this - turns out that using some deprecated methods, I was able to get the right Auth token straight from AccountManager. I have this guy to thank for the idea: https://github.com/tilal6991/UnofficialGoogleMusicAndroidAPI

So now instead of using username/password to log into the api, I use this token.

[simon-weber]

Awesome! (Thanks @tilal6991!)

This is the same OAuth "ubertoken" method that Chrome Sync uses, yes? I remember reading about this a while back (maybe here) and don't remember why I abandoned that effort.

Would the same flow be possible directly from Python?

[ghost]

I'm not familiar with Chrome Sync but yes, it's the same ubertoken idea. First read about it here http://nelenkov.blogspot.com/2012/11/sso-using-account-manager.html

Not sure if it could be done with Python alone, seems that AccountManager is needed to get the initial login

[simon-weber]

Did a bit more digging. Unfortunately, this flow doesn't help us outside of Android. It's the one I saw earlier that has backend restrictions we can't get around.

[simon-weber]

The android app now uses the has_permission flow for builtin apps described at http://sbktech.blogspot.com/2014/01/inside-android-play-services-magic.html. I have a feeling that the AccountManager/playservices parts can't easily be spoofed without an android device, but there might be an accompanying normal oauth scope for google music now.

[ghost]

Will ClientLogin's upcoming deprecation be a problem? This is the scheme that's being used with normal username/password login, correct?

http://googledevelopers.blogspot.com/2015/02/reminder-clientlogin-shutdown-scheduled.html

[simon-weber]

Right, plaintext auth for both the webclient and mobileclient uses clientlogin.

Here's the current state of things (as compared to #168 (comment)):

• web: we use clientlogin. unknown what google uses.
• android: we use clientlogin. google uses oauth, though it's unclear whether we can also use it.
• music manager: we and google both use oauth.
• iOS: we don't use this api. google uses oauth in a way that I think would work for us.

[ghost]

Thanks for the clarification. What I'm not clear on is if the method I'm using of getting an auth token from AccountManager on Android falls under ClientLogin or if it's something else.

Hopefully by April another one of these methods will show some promise. I'll see what I can start exploring when I get a chance.

[simon-weber]

AccountManager for google music is definitely capable of using oauth - that's what I referred to in #168 (comment) - but I'm not sure if there's a specific way you have to interact with it on android to do that.

[simon-weber]

It looks like other code has the master token flow working outside of android. Some notes:

• I think they took the client_sig from the gsf app and are posing as it. Like the linked article describes, this is used in place of a client id.
• I'm not sure why they don't need to provide an androidId. They link to a library that can create a fake android id, so maybe they need one later on.

[simon-weber]

I've got the master token flow working!

I should have a working mobileclient in a few days.

[simon-weber]

b0e05e6 gets things working again (and doesn't give the insecure apps warning). I've yet to try 2fa, but it should be possible to support it now, too.

[simon-weber]

2fa support will be a bit weird. Here's what the 2fa flow looks like:

• master token request results in an error that returns a url for the user to visit
• the url has the 2fa form
• on success, the client is redirected to a page that sets the master token as a cookie

Notice that the last step doesn't actually show the value in a user-readable format (unlike, say, the Music Manager's oauth flow that has the "copy and paste this value" copy). I see two options to support this:

• make login optionally accept a master token. when gmusicapi sees the 2fa error, give the url to client code. client code tells users to visit url, enter code, then copy out the cookie value (which could be made easier with a bookmarklet) and provide it in a login call.
• make login optionally accept a 2fa code. when gmusicapi sees the 2fa error, it posts the code to the url itself, then retrieves the cookie.

The first option is what the music manager does, but having users deal with cookies is super weird. The second option seems easier for everyone, but could be flaky if the login form changes often.

[ghost]

This is really awesome, thanks for the update.
Regarding 2FA, depending on the scenario, copying out the cookie might not be so bad, but will this still work if the cookie is domain restricted?

[ghost]

I would hope whatever system is in use, it works on systems that don't have
a web browser or desktop installed. My setup is a headless raspberrypi, I
suspect there are a lot of headless systems these days running this.

With regards to 2FA does application specific passwords not mitigate this?
On 15 May 2015 01:42, "Simon Weber" notifications@github.com wrote:

2fa support will be a bit weird. Here's what the 2fa flow looks like:

master token request results in an error that returns a url for the user
to visit
the url has the 2fa form
on success, the client is redirected to a page that sets the master token
as a cookie

Notice that the last step doesn't actually show the value in a
user-readable format (unlike, say, the Music Manager's oauth flow that has
the "copy and paste this value" copy). I see two options to support this:

make login optionally accept a master token. when gmusicapi sees the 2fa
error, give the url to client code. client code tells users to visit url,
enter code, then copy out the cookie value (which could be made easier with
a bookmarklet) and provide it in a login call.
make login optionally accept a 2fa code. when gmusicapi sees the 2fa
error, it posts the code to the url itself, then retrieves the cookie.

The first option is what the music manager does, but having users deal
with cookies is super weird. The second option seems easier for everyone,
but could be flaky if the login form changes often.

—
Reply to this email directly or view it on GitHub.

[ghost]

Is there any reason why a similar approach wouldn't work for Webclient? I'm going to give it a try. I still need to use Webclient to obtain a deviceId.

[ghost]

Actually, I realize the androidId is required to do the login, so it's sort of a chicken and egg problem.

[mario-tux]

Given the knowledge of device_id, I can confirm that mc.login on develop branch works. On gmusicproxy I use function get_registered_devices() to get the device ids list. Now, it is broken and there is no way to discover such keys on a fresh installation.

[thebigmunch]

@diraimondo: Technically, you can login with a bogus device ID, but, of course, that's not really a great idea.

And while I am an Android user, many people are not. So having it take an Android ID only (or naming it that way if it takes others) is also a bit of a usability issue. I've had to put some things on hold for a bit due to this as well : (

[mario-tux]

The requirement for a valid Android device id was already there on the mobileclient api before.

[thebigmunch]

The requirement for a valid Android device id was already there on the mobileclient api before.

No. The requirement of a valid mobile device id (Android, iOS or even Chrome app) was already there for streaming, not specifically Android.

[siebert]

I'm wondering if gmusicapi already works using an application-specific password on accounts with 2-step verification enabled. If so, why bother at all implementing a proper 2-step verification workflow?

[simon-weber]

My understanding is that app-specific passwords were a hack because clientlogin didn't support 2fa, and that they won't work now. I think they're not needed when everything uses oauth in the right way (mobileclient doesn't).

[mario-tux]

I would like to use a proper 2fa authentication with gmusicapi without application-specific passwords. Is it already working?

[siebert]

App-specific passwords are a kind of hack to enable access to 2fa protected google accounts for applications which don't support 2fa. They still work for me and are necessary to use a gmail account in Thunderbird among others.

[simon-weber]

The current mobileclient interface seems to support the formerly ios-specific skyjam scope now, so we can merge this into #426 (comment).