missing "mail" in response with Microsoft SSO

[arribatec-cloud-1]

I have set up an application in Azure with credentials.

When I try to log in using said credentials as per the examples, the call fails with a missing key error:

ERROR: KeyError('mail')
Traceback (most recent call last):
  File "/whatever/routes/sso_microsoft.py", line 54, in microsoft_callback
    user = await microsoft_sso.verify_and_process(request)
  File "/usr/local/lib/python3.9/site-packages/fastapi_sso/sso/base.py", line 212, in verify_and_process
    return await self.process_login(
  File "/usr/local/lib/python3.9/site-packages/fastapi_sso/sso/base.py", line 292, in process_login
    return await self.openid_from_response(content)
  File "/usr/local/lib/python3.9/site-packages/fastapi_sso/sso/microsoft.py", line 45, in openid_from_response
    return OpenID(email=response["mail"], display_name=response["displayName"], provider=cls.provider)
KeyError: 'mail'

The code looks like this:

from fastapi import APIRouter, Depends
from fastapi_sso.sso.microsoft import MicrosoftSSO
from starlette.requests import Request
import logging
import os
import pprint

logger = logging.getLogger(__name__)

allow_insecure_http = ("1" == os.environ.get("OAUTHLIB_INSECURE_TRANSPORT", "0"))

# documentation https://pypi.org/project/fastapi-sso/

sso_microsoft_route = APIRouter(
	  prefix="/sso/microsoft"
	, tags = ["sso"]
	#,dependencies=[Depends(get_token_header)]
	, responses={404: {"description": "Not found"}}
)

MICROSOFT_SSO_DEBUG = os.environ.get("MICROSOFT_SSO_DEBUG")
MICROSOFT_SSO_REDIRECT_URL = os.environ.get("MICROSOFT_SSO_REDIRECT_BASE_URL")
MICROSOFT_SSO_TENANT = os.environ.get("MICROSOFT_SSO_TENANT")
MICROSOFT_SSO_CLIENT_ID = os.environ.get("MICROSOFT_SSO_CLIENT_ID")
MICROSOFT_SSO_CLIENT_SECRET = os.environ.get("MICROSOFT_SSO_CLIENT_SECRET")


if MICROSOFT_SSO_DEBUG:
	logger.info(f"  MICROSOFT_SSO_REDIRECT_URL: {MICROSOFT_SSO_REDIRECT_URL}")
	logger.info(f"        MICROSOFT_SSO_TENANT: {MICROSOFT_SSO_TENANT}")
	logger.info(f"     MICROSOFT_SSO_CLIENT_ID: {MICROSOFT_SSO_CLIENT_ID}")
	logger.info(f" MICROSOFT_SSO_CLIENT_SECRET: {MICROSOFT_SSO_CLIENT_SECRET}")

microsoft_sso = MicrosoftSSO(
	  client_id = MICROSOFT_SSO_CLIENT_ID
	, client_secret = MICROSOFT_SSO_CLIENT_SECRET
	, tenant = MICROSOFT_SSO_TENANT
	, allow_insecure_http = allow_insecure_http
	, scope = ["openid"]
)


@sso_microsoft_route.get("/login")
async def microsoft_login(request: Request):
	with microsoft_sso:
		return await microsoft_sso.get_login_redirect(redirect_uri = request.url_for("microsoft_callback"))


@sso_microsoft_route.get("/callback")
async def microsoft_callback(request: Request):
	user = None
	with microsoft_sso:
		try:
			user = await microsoft_sso.verify_and_process(request)
		except Exception as e:
			logger.exception(f"ERROR: {pprint.pformat(e)}")
	if not user:
		logger.warning("NO USER")
		return None
	return {
		"id": user.get("id"),
		"picture": user.get("picture"),
		"display_name": user.get("display_name"),
		"email": user.get("email"),
		"provider": user.get("provider"),
	}

[arribatec-cloud-1]

I investigated this further and .... well let's just say MicrosoftSSO has no error handling what-so-ever. I might submit a PR at some point if I get it to work.

If you need a patch straight away, put this in the top of MicrosoftSSO.openid_from_response():

from fastapi_sso.sso.base SSOLoginError
error = response.get("error")
		if error:
			raise SSOLoginError(401, f"Error '{pprint.pformat(error)}' returned from Microsoft")

[tomasvotava]

I believe some tenants require to ask for email scope directly, it is now a default in 0.8.0 https://github.com/tomasvotava/fastapi-sso/releases/tag/0.8.0

Could you test if this resolves the problem for you?

[arribatec-cloud-1]

I did not test your suggestion, but it is aligning well with what I found what worked for me in the end; to omit the scope parameter from MicrosoftSSO constructor altogether. I used to have it set to ["openid"] which would override the default of ["openid", "User.Read"]. It could also just be luck/timing of the 10 times I created and changed my application settings and credentials inside Azure... I guess just providing an example/documentation would solve this issue. Anyways, thanks for looking at my issue! Lennart Rolland Senior Cloud Consultant ***@***.*** +4797688353

…

________________________________ From: Tomas Votava ***@***.***> Sent: Friday, November 24, 2023 6:36 PM To: tomasvotava/fastapi-sso ***@***.***> Cc: Lennart Rolland ***@***.***>; Author ***@***.***> Subject: Re: [tomasvotava/fastapi-sso] missing "mail" in response with Microsoft SSO (Issue #81) You don't often get email from ***@***.*** Learn why this is important<https://aka.ms/LearnAboutSenderIdentification> I believe some tenants require to ask for email scope directly, it is now a default in 0.8.0 https://github.com/tomasvotava/fastapi-sso/releases/tag/0.8.0 Could you test if this resolves the problem for you? — Reply to this email directly, view it on GitHub<#81 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/A5HPLK4KR2B5SQL5EGKA6DTYGDLKDAVCNFSM6AAAAAA6PLOXVKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMRVHE2DSNZVGY>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[tomasvotava]

That's a good idea, I've added a simple post to guide users who struggle with this as well, thanks!

https://tomasvotava.github.io/fastapi-sso/how-to-guides/key-error/

[bolau]

The key error of the OP is not a problem anymore. But still, I don't get an email address. Even with the scopes email or User.Read.All, it's not returned by MS, their "mail" field is null:

{
   '@odata.context': 'https://graph.microsoft.com/v1.0/$metadata#users/$entity', 
   'businessPhones': [], 
   'displayName': 'Boris Lau', 
   'givenName': 'Boris', 
   'jobTitle': None, 
   'mail': None, 
   'mobilePhone': None, 
   'officeLocation': None, 
   'preferredLanguage': 'en-US', 
   'surname': 'Lau', 
   'userPrincipalName': 'lau@XXXXX',
   'id':XXXXX'
}

Resulting in:

id='XXXXX'
email=None
first_name='Boris'
last_name='Lau'
display_name='Boris Lau'
picture=None
provider='microsoft'

Interestingly, my mail address is contained in the field userPrincipalName.

[tomasvotava]

Hi @bolau! I am afraid since e-mail is really considered personal data these days, more and more openid providers will make it more difficult to retrieve it on behalf of a user. E.g. Apple has a configuration that allows the user to generate a temporary e-mail for each service, therefore you'll never actually be able to retrieve the user's real e-mail address and won't even be able to tell. Nevertheless, I believe in Microsoft's case there are lots of settings on tenant level that can play part in whether you get the e-mail address or not.
I'd recommend double-checking whether your Azure Application Registration has admin authorization granted (for some permissions you will se that it's not required, but things change when it is).
We are using the /me endpoint to retrieve the data https://github.com/tomasvotava/fastapi-sso/blob/master/fastapi_sso/sso/microsoft.py#L45
It is documented here, you can get your access token like this:

@app.get("/callback")
async def login_callback(request: Request):
    with sso:
        openid = await sso.verify_and_process(request)
    print(sso.access_token)

Then you can use the access token to play around with the API in curl / Postman or MS Graph Explorer. In the explorer, you can provide your own access token (obtained with fastapi) and try to come up with a combination of fields and params that will yield what you need. I will play around as well, but in my tenant and my app, the mail field is actually retrieved, so it's hard for me to debug.

[bolau]

Hi Tomas, thanks for your reply. I tried the Graph Explorer, but couldn't get the email address out of it.
Here's a temporary solution, which modifies fastapi_sso/sso/microsoft.py, but :

    async def openid_from_response(self, response: dict, session: Optional["httpx.AsyncClient"] = None) -> OpenID:
        email = response.get("mail")
        if email is None and "@" in response.get("userPrincipalName"):
            email = response.get("userPrincipalName")
        return OpenID(
            email=email,
            display_name=response.get("displayName"),
            provider=self.provider,
            id=response.get("id"),
            first_name=response.get("givenName"),
            last_name=response.get("surname"),
        )

I don't want to make this a pull request though, since this is most likely not a "correct" solution. I guess my account just doesn't have a proper e-mail address linked to it. Which seems weird, but anyhow :)

[tomasvotava]

In that case, you could try passing convert_response = False to verify_and_process. This will return the original response from /me endpoint instead of the OpenID instance and you can grab the userPrincipalName directly from there without the need to edit or subclass MicrosoftSSO.

SSOBase.verify_and_process