AzureAD Graph API - User / Group Sync

[Oscar Stankard]

Hi there,

Trying to get the Microsoft Graph API connection set up to allow us to sync usernames but seem to be having issues whether using the secret in the json or using the KeySafe method.

When trying to set up the KeySafe method and hitting the 'connect' button it says that it's been unable to connect ("Unexpected Exception: Failed to load the requested oauth2 config") and when configuring the key values and data in the .json config file itself the debug log reports endpoint returning a 401 error ("2022/04/12 12:05:49 [DEBUG] [SCRIPT] Generating Bearer Token 2022/04/12 12:05:49 [ERROR]  [Azure] BearerToken Error: Invalid HTTP Response: 401").

I have recopied the client ID, client secret/value, and tenent ID several times as errors with seems to be the most widely reported reason for this error but to no avail.  The executable/zip appears to be the current one linked to on github by the instructions at https://wiki.hornbill.com/index.php?title=Azure_User_Import and https://wiki.hornbill.com/index.php?title=Azure_App_Registration_Instructions.  I'm not entirely sure we've populated all the required values on the KeySafe given how empty it's looking after we've plumbed in all the details we are aware of and given the error message but not sure how we'd even determine what the other values should be.

Please can anyone advise what we might try next or that we might have overlooked, don't seem to see any KeySafe logs at https://wiki.hornbill.com/index.php?title=Hornbill_KeySafe either unfortunately.

Thanks for any thoughts you might have.

 

Kind regards,

Oscar

[SamS]

Hi @Oscar Stankard,

If you are seeing a "Connect"-button within the KeySafe entry, then you likely have set up a "Microsoft:OAuth2"-Type.

Please use a regular "OAuth2" - displays as "oAuth 2.0".

For running the Azure User Import, you will need to fill in the KeySafe form as detailed in the Azure User Import Wiki entry.

Don't forget to pair the API key to the Keysafe entry - and, when creating the Azure App, don't forget that the System Administrator needs to confirm the permissions.

[Oscar Stankard]

Hi there,

When making a KeySafe key I only see one option named "oAuth 2.0" and when I enter the details stipulated on the Azure User Import instructions the Connect button appears after saving changes to the KeySafe entry.

Have gone through again from scratch in case have made errors and seem to be making the same ones this time round too whatever they are, other screens captured for second pair of eyesness.

Thanks for your help.

 

Kind regards,

Oscar

[SamS]

Hi @Oscar Stankard,

What happens when you add the API Key you are using for the import to the "API Key Permissions"-list, and try the import again?

Don't forget to marry the API key to the Keysafe entry - and, when creating the Azure App, don't forget that the (Sys) Admin needs to confirm the permissions.

[Oscar Stankard]

Hi there,

So under the part of the KeySafe config called API Key Permissions, if I click on the green + symbol it just states that there're no permissions found to add: "No API Keys Found".

How can I best check on the current API Key and KeySafe entry marrying and that the permissions you mention are confirmed?  I'm quite sure I've pressed the buttons to confirm the AzureAD user and group read permissions when creating the app registration and API key and they all showed a green tick after, each time we attempted to set up the API.  The Azure AD page warns me that I should be using MSAL rather than ADAL but no idea what either are.

Thanks for your help.

[SamS]

Hi @Oscar Stankard,

Within "Add Permission from User API Keys" search for the user you have created the API key for and then select that API key to include it with the KeySafe entry.

[Oscar Stankard]

Hi there,

Thanks for your help, have added the Hornbill API key to the KeySafe config and saved the changes but the connect button and result when pressed remains the same unfortunately.  Made a new API key just in case as i noted the one in there already didn't have an expiry date but seems to be the same.

I've seemingly got the non-KeySafe version working but I'm keen to get the KeySafe version functioning as described due to the increased security it offers, but this should at least prove the API details are valid.  If I press on and try and use KeySafe entry 6 in the config the logging seems to suggest the tennant is not suitably configured in the KeySafe config:

 

2022/04/20 15:47:45 [DEBUG] Loading Config File: C:\Entertainer\Hornbill AzureAD Graph/conf.json
2022/04/20 15:47:45 [MESSAGE] Loading KeySafe Authentication Data: 6
2022/04/20 15:47:45 [ERROR] Error Loading KeySafe Authentication: The API key being used does not have permission to access this keysafe record
2022/04/20 15:47:45 [ERROR] Error Decoding LDAP Server Authentication: unexpected end of JSON input
2022/04/20 15:47:45 [DEBUG] Tenant found in KeySafe:
2022/04/20 15:47:45 [ERROR] azure Tenant is not set
2022/04/20 15:47:45 [ERROR] Please Check your Configuration: goAzure2HUserImport

 

But the Azure App Registration Tenant ID is filling the Hornbill "API Endpoint" box as I read is required, and it's also the same TenantID that works when supplied through the non-KeySafe route.

 

Thanks for your help.

Kind regards,

Oscar

[SamS]

Hi @Oscar Stankard,

The "Connect"-button will NOT work - there is no "destination" to check against. It is also NOT needed.

The "Connect"-button is a recent addition, so I tried my test config against a new oAuth configuration entry and that also (still) worked (just in case the "Connect"-button brought something else with it).

According to the error message, the API key you are using is NOT the same as configured against the keysafe entry, which explains why the Tenant ID/API Endpoint isn't being returned.

The way to check whether the "Azure AD API Key" APIKey Permission is indeed the one you are using with the utility is to add the API key you are using to the list (again). IF it matches the API key there, then you will get a "The specified permission is already granted"-error - otherwise it will add the API key to the list.

[Oscar Stankard]

Hi there,

Great thanks for your help Sam, it looks like setting the specific API key from the specific user on the Hornbill side was the missing link when setting up the KeySafe entry.

We've since tried a few imports to try and refine the user filter but the connection is failing stating:

[WARN] No Unique Identifier set for this record  &map[businessPhones:[] givenNam
e:.087 id:0041d-752f-015b3d7c5f mail:087@ourdomain.com mailN
ickname:087 surname:ourname userPrincipalName:087@ourdomain.com]

This user seems to be normal in so far as we've spotted and no other lines are logged after this warn event and this seems to happen when we've selected .keysearch or .userprincipalname as the UserDN value.

 

Here's the current cof.json in case anyone can spot the problem:

{
    "APIKey": "",
    "InstanceId": "",
    "AzureConf": {
        "KeySafeID": 6,
        "Tenant": "",
        "ClientID": "",
        "ClientSecret": "",
        "UserFilter": "",
        "UserID": "",
        "UserProperties": [
            "givenName"
            , "surname"
            , "businessPhones"
            , "mail"
            , "mailnickname"
        ],
        "APIVersion": "v1.0",
        "Search": "users",
        "UsersByGroupID": [
        ]
    },
    "User": {
        "Operation":"Both",
        "UserDN": "{{.keysearch}}",
        "HornbillUniqueColumn": "h_user_id",
        "AccountMapping": {
            "UserID": "{{.mailnickname}}",
            "LoginID": "{{.mailnickname}}",
            "EmployeeID": "",
            "UserType": "basic",
            "Name": "{{.givenName}} {{.surname}}",
            "Password": "",
            "FirstName": "{{.givenName}}",
            "LastName": "{{.surname}}",
            "JobTitle": "",
            "Site": "",
            "Phone": "",
            "Email": "{{.mail}}",
            "Mobile": "",
            "AbsenceMessage": "",
            "TimeZone": "",
            "Language": "",
            "DateTimeFormat": "",
            "DateFormat": "",
            "TimeFormat": "",
            "CurrencySymbol": "",
            "CountryCode": ""
        },
        "Type": {
            "Action": "Both"
        },
        "Status": {
            "Action": "Both",
            "Value": "active"
        },
        "Role": {
            "Action": "Both",
            "Roles": [
                "Basic User Role"
            ]
        },
        "ProfileMapping": {
            "MiddleName": "",
            "JobDescription": "",
            "Qualifications": "",
            "Interests": "",
            "Expertise": "",
            "Gender": "",
            "Dob": "",
            "Nationality": "",
            "Religion": "",
            "HomeTelephone": "",
            "SocialNetworkA": "",
            "SocialNetworkB": "",
            "SocialNetworkC": "",
            "SocialNetworkD": "",
            "SocialNetworkE": "",
            "SocialNetworkF": "",
            "SocialNetworkG": "",
            "SocialNetworkH": "",
            "PersonalInterests": "",
            "homeAddress": "",
            "PersonalBlog": "",
            "Attrib1": "",
            "Attrib2": "",
            "Attrib3": "",
            "Attrib4": "",
            "Attrib5": "",
            "Attrib6": "",
            "Attrib7": "",
            "Attrib8": ""
        },
        "Manager": {
            "Action": "__clear__"
            , "AzField": "userPrincipalName"
            , "HornbillUniqueColumn":"h_user_id"
        },
        "Image": {
            "Action": "Both",
            "UploadType": "uri",
            "InsecureSkipVerify": false,
            "ImageType": "png",
            "ImageSize": "504",
            "URI": ""
        },
        "Site": {
            "Action": "Both",
            "Value": ""
        },
        "Org": [
        ]
    }
}

 

Thanks very much for any thoughts anyone has.

 

Kind regards,

Oscar

 

[SamS]

Hi @Oscar Stankard,

keysearch is not an Azure Field, please use userPrincipleName or other relevant field instead.

I'll remove that reference from the documentation - the conf.json combined with the utility DOES have a more suitable example

[Oscar Stankard]

Hi there,

We tried .userPrincipalName first but thought to try the .keysearch option after having no luck.  Happy to use any alternate value you might suggest, also tried .mailnickname just in case but believe it will be needing the version that includes the domain in the UserDN value?

The existing userID/logonID/employeeID values for this record in Service Manager apear to be the short/domainless version and the record in Azure AD appears to have the mailnickname populated.  If I use the "startswith(displayName,'Oscar')" in the UserFilter to try with a different record it seems to fail in the same way.

I have tried it again but with the proper userPrincipalName capitalisation in case it was case sensitive but to no effect.  Previous runs seem to have got past this record and gone on to list all users to the log file but it's getting stuck at the first record now.  If I update the account mapping values UserID/LoginID/EmployeeID to use the .userprincipalname instead of .mailnickname value it seems to work without this WARN so perhaps I can use that value but with our domain taken out instead using one of those inline 'startswith' functions?

Thanks for your help.

Kind regards,

Oscar

 

[Oscar Stankard]

Hi there,

Have found that you need to have a capital N in mailNickname in the Azure as well as mapping fields and then it works as hoped.  On to some testing!

 

Kind regards,

Oscar

[Oscar Stankard]

Hi there,

The script seems to permit all users listed from the Graph call to be created or updated to either 'basic' or full 'user' users, but we need some to be basic and some to be full?  When trying to run the import it's currently trying to 'downgrade' the helpdesk users, leaving the value blank or setting a 'none' update mode doesn't appear to be an option.

So if we need to run 2 separate conf.json, one to look at the 'user' and one to do the 'basic' ones, we need a way to exclude the helpdesk users from the basic sync, and then exclusively the helpdesk users in a second config set for 'user' level users?  Users that have documents assigned to them seem to reject the attempt to 'downgrade' so perhaps we could just ensure every team member has a document assigned to block this?

In experimenting with the (Graph?) filter syntax that's exposed to us by the userfilter value, it seems it doesn't support the 'ne' operator, or give the ability to reference 'complex' properties like the licence collection?  Looking through the documentation I can't seem to see a way to set the user filter to exclude the dozen or so helpdesk menbers from the wider results?  Hopefully I'm overlooking something but at the moment the only path I can see is to populate 2 Azure AD groups with the 2 types of users we want to import/sync with a separate script not limited by Graph, and use that as the UsersByGroupID filter for 2 separate basic/user sync runs to Service Manager?

Thanks for your help.

Kind regards,

Oscar

[SamS]

Hi @Oscar Stankard,

You could indeed manage it with multiple configuration files and filters.

That being said, there is a "Type" configuration which allows you manage the user type.

Setting :

 "Type": {
            "Action": "Create"
        },

Set to "Create" in combo with a basic user in the regular configuration will make all new users to be "Basic", but, importantly will NOT modify the current status of full "user" users. So, with this (single) configuration, you just need to manually upscale the necessary users when they start.

[Oscar Stankard]

Hi there,

Ah good stuff I had not twigged that this 'Type' key was to change the update mode of the 'UserType' value, presuming the update mode under the 'User' section was the one that goverend it.

So all the other values are hopefully the same between all 'basic' and 'user' users (the "Basic User Role" role can be applied to both types of users without issue as far as I can think?) and we can bare to add the userusers-specific roles and change user type manually for the handfull of service desk changes we have day to day.

Let's see if this ticks the box for the service desk team, thanks for your help.

 

Kind regards,

Oscar