LDAP Import not updating user's Roles

[samwoo]

Hello,

I have a few questions:

1. I'm not sure if it's been asked but is it possible to have this import tool look at multiple DSN's within the conf?
2. How does Hornbill / LDAP_Import deal with users that do not exist in AD any more or are Archived? (for example if someone left the council)
3. Further investigation with regards to SSO is underway, how does Auto-Provisioning work with AD and Users? For example does it update if the user's details have been changed on the fly? Does this still require LDAP_Import to be run if Auto-Provisioning is turned on? (Sorry i dont have a great deal of knowledge in this area so might be asking the wrong question)

Thanks,

Samuel

[samwoo]

Hello,

Another question - is it possible to use the "Filter": "(objectClass=user)", to do a "NOT" filter?

For example if we have usernames with .admin, could we filter to bring back everything EXCEPT anything with *.admin in the username Attribute? Can this be extended to other Attributes?

Thanks,

Samuel

[TrevorKillick]

Hi Samuel

Its possible but not something we have very much expertise around, there are some examples posted by other customers with more complex filters for examples this post by Gareth has a NOT in the filter using !
 

Hope this helps

Kind Regards

Trevor Killick

[Guest]

On 7/29/2016 at 3:32 PM, samwoo said:

Hello,

Another question - is it possible to use the "Filter": "(objectClass=user)", to do a "NOT" filter?

For example if we have usernames with .admin, could we filter to bring back everything EXCEPT anything with *.admin in the username Attribute? Can this be extended to other Attributes?

Thanks,

Samuel

Hi Sam,

In addition to Trevors post, and to answer some of the other questions as well:

1) Multiple DSNs will require separate conf files (unless you are referring to subtrees of a DSN - in which case you can use the "Scope" option which is described on the wiki page here)

2) In regards to Archiving, it depends on what you do to the users in LDAP. If you set them to "disabled", you could have a new conf file, that runs after your standard import ones, that actively searches for the users who have been set to disabled (using a filter such as: (UserAccountControl:1.2.840.113556.1.4.803:=2)  ). Then in this Conf file, using the option called "UserAccountStatus" - you can enable this, set it to update and change the status to "archived" as per below:

   "UserAccountStatus":{
       "Action":"Update",
       "Enabled": true,
       "Status":"archived"

What this will do is find any disabled users in your AD, and update the status of these in Hornbill to "archived" automatically. In Hornbill, you should always archive your users and never delete them.

3) Auto Provisioning is only used for the initial creation of users, it does not perform any updates. If you have an LDAP import script configured, there is no need to enable Auto Provisioning as the LDAP Import Utility results in the same outcome, but with more flexibility and configuration options. 

4) Assuming by "User Name", the attribute in LDAP you are referrng to is the sAMAccountName, the LDAP query you need to filter out the .admin users will be along the lines of: (&(objectClass=*)(!(sAMAccountName=*.admin))) - ensure to test this though, because as Trevor mentioned this falls outside of our typical expertise. 

I hope this helps

Regards

Bob

[samwoo]

@TrevorKillick and @bob_dickinson

Thank you both for responding. I have improved on this by download Softera LDAP Browser, i am now visually able to define Filters, then copy the Filter string into the conf files.

I've also created a Batch file which will run LDAP_Import a few times, but for each run i've changed the conf file it looks at... and it works flawlessly (will take dryrun=true off once testing is completed)

Quote

- LDAP_Import.bat -
ldap_import.exe -dryrun=true -workers=4 -file=Desktop.json
ECHO ---------------------------------
ldap_import.exe -dryrun=true -workers=4 -file=NetworkServices.json
ECHO ---------------------------------
ldap_import.exe -dryrun=true -workers=4 -file=Applications.json
ECHO ---------------------------------
ldap_import.exe -dryrun=true -workers=4 -file=AdministrationTOP.json
ECHO ---------------------------------
ldap_import.exe -dryrun=true -workers=4 -file=Users.json

 

I have a request though... i was wondering if we could have a new feature where we can define what word to prefix at the front the log file as running this batch will produce multiple log files in the same format, with no visible way of know which conf file it was running for.

So to use the example BAT file above, i would expect it could look something like the following (for example):

Quote

- LDAP_Import.bat -
ldap_import.exe -dryrun=true -workers=4 -file=Desktop.json -prefix=Desktop
ECHO ---------------------------------
ldap_import.exe -dryrun=true -workers=4 -file=NetworkServices.json -prefix=NetworkServices
ECHO ---------------------------------
ldap_import.exe -dryrun=true -workers=4 -file=Applications.json -prefix=Applications
ECHO ---------------------------------
ldap_import.exe -dryrun=true -workers=4 -file=AdministrationTOP.json -prefix=AdminTop
ECHO ---------------------------------
ldap_import.exe -dryrun=true -workers=4 -file=Users.json -prefix=Users

Then the log file will look like the following (for example) Desktop_LDAP_User_Import_2016-08-03T10-50-54+01-00.log

Thanks,

Samuel

[TrevorKillick]

Hi Samuel

Glad you got it working, I will see if we can get this feature in for you in the next couple of weeks. 

I will post back here once a new version is available.

Kind Regards

Trevor Killick

[samwoo]

Hi Trevor,

Cheers for that I will keep an eye out for this feature.

I have yet another question... I have been updating our AD so we can pull "Basic" users and "Users" from certain areas. I have enabled the flag "UpdateUserType" , and have already has "UserRoleAction" set to "Create".

For users that were Basic are now updating to User is fine, but for the extra job roles that I need assigned to some of them... it's not working:

Quote

2016/08/03 11:42:19 [DEBUG] Add Role to User: Self Service User
2016/08/03 11:42:19 [DEBUG] Add Role to User: Basic User Role
2016/08/03 11:42:19 [DEBUG] Add Role to User: MyLibrary User
2016/08/03 11:42:19 [DEBUG] Add Role to User: Asset Management User
2016/08/03 11:42:19 [DEBUG] Add Role to User: Collaboration Role
2016/08/03 11:42:19 [DEBUG] Add Role to User: Change Management User
2016/08/03 11:42:19 [DEBUG] Add Role to User: Incident Management User
2016/08/03 11:42:19 [DEBUG] Add Role to User: mailbox_it_service_desk
2016/08/03 11:42:19 [DEBUG] Add Role to User: My Boards
2016/08/03 11:42:19 [DEBUG] Add Role to User: Service Request User
2016/08/03 11:42:20 [ERROR] Unable to Assign Role to User: Error assigning one or more roles

Does that allow updating of current job roles by adding any new non-existing ones?

Thanks,

Samuel

[TrevorKillick]

Hi Samuel

Log prefix is available in the latest version:

https://github.com/hornbill/goLDAPUserImport/releases/tag/v2.0.4

As for this error is there anything in the server log?

Kind Regards

Trevor Killick

[samwoo]

12 minutes ago, TrevorKillick said:

Hi Samuel

Log prefix is available in the latest version:

https://github.com/hornbill/goLDAPUserImport/releases/tag/v2.0.4

As for this error is there anything in the server log?

Kind Regards

Trevor Killick

Quote

2016/08/04 15:11:50 [DEBUG] ---- XMLMC LDAP Import Utility V2.0.3 ----
2016/08/04 15:11:50 [DEBUG] Flag - Config File AdministrationTOP.json
2016/08/04 15:11:50 [DEBUG] Flag - Zone eur
2016/08/04 15:11:50 [DEBUG] Flag - Dry Run false
2016/08/04 15:11:50 [DEBUG] Flag - Workers 4
2016/08/04 15:11:50 [WARN] 2.0.3 is not latest, you should upgrade to 2.0.4 by downloading the latest package Here https://github.com/hornbill/goLDAPUserImport/releases/tag/v2.0.4
2016/08/04 15:11:50 [DEBUG] Loading Config File: C:\TEMP\Hornbill - Testing/AdministrationTOP.json
2016/08/04 15:11:50 [DEBUG] Instance Endpoint https://eurapi.hornbill.com/****************/xmlmc/
2016/08/04 15:11:51 [DEBUG] Attempting Connection to LDAP... 
Server: ******************************
Port: ***
Type: 
Skip Verify: false
Debug: false
2016/08/04 15:11:51 [DEBUG] Creating LDAP Connection
2016/08/04 15:11:51 [DEBUG] LDAP Search Query 
{Server:************************ UserName:******************* Password:************* Port:*** ConnectionType: InsecureSkipVerify:false Scope:1 DerefAliases:1 SizeLimit:0 TimeLimit:0 TypesOnly:false Filter:(&(objectClass=user)(&(!(sAMAccountName=*GCSX*))(!(objectClass=Computer))(!(sAMAccountName=*test*))(!(sAMAccountName=*.admin))(givenName=*))) DSN:ou=administration,ou=users,ou=users & desktops,ou=***********,DC=************,DC=******,DC=***** Debug:false} ----
2016/08/04 15:11:51 [DEBUG] LDAP Results: 2
2016/08/04 15:11:51 [DEBUG] Processing Users
2016/08/04 15:11:51 2016/08/04 14:11:51 [DEBUG] Buffer For Job: 1 - Worker: 2 - User: maredw
2016/08/04 14:11:51 [DEBUG] Update User: maredw
2016/08/04 14:11:51 [DEBUG] LDAP Attribute for Site Lookup: [physicalDeliveryOfficeName]
2016/08/04 14:11:51 [DEBUG] Looking Up Site *************
2016/08/04 14:11:51 [DEBUG] Site Lookup found Id 1
2016/08/04 14:11:51 [ERROR] Unable to Load LDAP Attribute: telephoneNumber For Input Param: [telephoneNumber]
2016/08/04 14:11:51 [ERROR] Unable to Load LDAP Attribute: mobile For Input Param: [mobile]
2016/08/04 14:11:51 [DEBUG] Add Role to User: Self Service User
2016/08/04 14:11:51 [DEBUG] Add Role to User: Basic User Role
2016/08/04 14:11:51 [DEBUG] Add Role to User: MyLibrary User
2016/08/04 14:11:51 [DEBUG] Add Role to User: Asset Management User
2016/08/04 14:11:51 [DEBUG] Add Role to User: Collaboration Role
2016/08/04 14:11:51 [DEBUG] Add Role to User: Change Management User
2016/08/04 14:11:51 [DEBUG] Add Role to User: Incident Management User
2016/08/04 14:11:51 [DEBUG] Add Role to User: mailbox_it_service_desk
2016/08/04 14:11:51 [DEBUG] Add Role to User: My Boards
2016/08/04 14:11:51 [DEBUG] Add Role to User: Service Request User
2016/08/04 14:11:51 [ERROR] Unable to Assign Role to User: Error assigning one or more roles
2016/08/04 14:11:51 [DEBUG] Processing User Profile Data maredw
2016/08/04 14:11:51 [DEBUG] LDAP Attribute for Manager Lookup: [manager]
2016/08/04 14:11:51 [ERROR] Unable to Load LDAP Attribute: manager For Input Param: [manager]
2016/08/04 14:11:51 [DEBUG] User Profile Update Success

2016/08/04 15:11:51 2016/08/04 14:11:51 [DEBUG] Buffer For Job: 2 - Worker: 1 - User: stecul
2016/08/04 14:11:51 [DEBUG] Update User: stecul
2016/08/04 14:11:51 [DEBUG] LDAP Attribute for Site Lookup: [physicalDeliveryOfficeName]
2016/08/04 14:11:51 [DEBUG] Looking Up Site IMT ********
2016/08/04 14:11:51 [DEBUG] Site Lookup found Id 
2016/08/04 14:11:51 [ERROR] Unable to Load LDAP Attribute: telephoneNumber For Input Param: [telephoneNumber]
2016/08/04 14:11:51 [ERROR] Unable to Load LDAP Attribute: mobile For Input Param: [mobile]
2016/08/04 14:11:51 [DEBUG] Add Role to User: Self Service User
2016/08/04 14:11:51 [DEBUG] Add Role to User: Basic User Role
2016/08/04 14:11:51 [DEBUG] Add Role to User: MyLibrary User
2016/08/04 14:11:51 [DEBUG] Add Role to User: Asset Management User
2016/08/04 14:11:51 [DEBUG] Add Role to User: Collaboration Role
2016/08/04 14:11:51 [DEBUG] Add Role to User: Change Management User
2016/08/04 14:11:51 [DEBUG] Add Role to User: Incident Management User
2016/08/04 14:11:51 [DEBUG] Add Role to User: mailbox_it_service_desk
2016/08/04 14:11:51 [DEBUG] Add Role to User: My Boards
2016/08/04 14:11:51 [DEBUG] Add Role to User: Service Request User
2016/08/04 14:11:51 [ERROR] Unable to Assign Role to User: Error assigning one or more roles
2016/08/04 14:11:51 [DEBUG] Processing User Profile Data stecul
2016/08/04 14:11:51 [DEBUG] LDAP Attribute for Manager Lookup: [manager]
2016/08/04 14:11:51 [ERROR] Unable to Load LDAP Attribute: manager For Input Param: [manager]
2016/08/04 14:11:51 [DEBUG] User Profile Update Success

2016/08/04 15:11:51 [DEBUG] Updated: 0
2016/08/04 15:11:51 [DEBUG] Updated Skipped: 2
2016/08/04 15:11:51 [DEBUG] Created: 0
2016/08/04 15:11:51 [DEBUG] Created Skipped: 0
2016/08/04 15:11:51 [DEBUG] Profiles Updated: 2
2016/08/04 15:11:51 [DEBUG] Profiles Skipped: 0
2016/08/04 15:11:51 [DEBUG] Time Taken: 983.4362ms
2016/08/04 15:11:52 [DEBUG] ---- XMLMC LDAP Import Complete ---- 
 

That's all i get from the smallest conf result (it does the same thing with more users in other OU's too which is set in other conf files)

[TrevorKillick]

Hi Samuel

Thanks i mean the EspServerService.log which is available in the admin tool, if you filter by Error you should see an explanation as to why the API call failed.

Kind Regards

Trevor Killick

[samwoo]

Hi Trevor,

Sorry I got it now. These are the only errors that I get when running the smallest conf file:

Quote

1572907	04-Aug-16 2:22 PM	error	perf	13772	admin:userAddRole() Operation Invocation results: failure (478175232 B, 9 ms, 0 kB, 0 ms, 0 kB)
1572905	04-Aug-16 2:22 PM	error	sql	13772	MySQL Error 1062: Duplicate entry 'stecul-Self Service User' for key 'PRIMARY'
1572891	04-Aug-16 2:22 PM	error	perf	7752	admin:userAddRole() Operation Invocation results: failure (478175232 B, 10 ms, 0 kB, 0 ms, 0 kB)
1572889	04-Aug-16 2:22 PM	error	sql	7752	MySQL Error 1062: Duplicate entry 'maredw-Self Service User' for key 'PRIMARY'
1572875	04-Aug-16 2:22 PM	error	perf	8968	admin:userUpdate() Operation Invocation results: failure (478175232 B, 5 ms, 0 kB, 0 ms, 0 kB)
1572869	04-Aug-16 2:22 PM	error	perf	8940	admin:userUpdate() Operation Invocation results: failure (478175232 B, 5 ms, 0 kB, 0 ms, 0 kB)

 

[TrevorKillick]

Hi Samuel

The errors suggest the users already have the Role?

Is that the case or are you saying the expected roles are not assigned?

Normally we catch an error relating to things like this and discard them unfortunately in this case the error our server returns is a bit generic.  

Kind Regards

Trevor Killick

[samwoo]

Hi Trevor,

If we look at maredw - these are his job roles after running the above import (please note maredw had these job roles since he was set up on the system manually quite some time ago):

The job roles i specified to assign are not showing up. I am unable to test as the user is currently unavailable but ideally would like to see it appear here. This happens with everyone who do not have at least one of the my the Job Roles against them.

Thanks,

Samuel

[TrevorKillick]

Hi Samuel

Ok the other thing to check is make sure each of the roles you specify has the Privilege level of Basic as you are assigning them to Basic Users and they can only have roles assigned to them that have the Privilege level of Basic and not User.



Kind Regards

Trevor Killick

[samwoo]

6 minutes ago, TrevorKillick said:

Hi Samuel

Ok the other thing to check is make sure each of the roles you specify has the Privilege level of Basic as you are assigning them to Basic Users and they can only have roles assigned to them that have the Privilege level of Basic and not User.



Kind Regards

Trevor Killick

maredw has a user type of "user" (as does everyone else i'm assigning these job roles to). Doesnt having the Basic User Role assigned to a "user" have any effect on adding additional Job Roles?

Thanks,

Samuel

[TrevorKillick]

Hi Samuel,

No thats fine, i assumed after seeing the Basic User Role that you were importing basic users. There is no restriction on adding a role with lower privileges than a user.

Kind Regatds

Trevor Killick

[samwoo]

On 19/05/2016 at 1:42 PM, TrevorKillick said:

Hi Samuel

Not a problem glad its all working for you now.

Regards

Trevor

Thanks Trevor. 

Is there a table where I can do a select query to truly see what Job Roles a user has? I am wondering whether or not it is actually assigning it to them but it's not showing in the Job Roles area.

And also now  i'm thinking about it, the last time I posted on here about it to give users the "Self Service Portal" role it actually added the Job Roles to via the LDAP Import, but not this time. Maybe something has changed? 

Thanks,

Samuel

[TrevorKillick]

Hi Samuel

Try This:

SELECT * FROM h_sys_accounts_roles



I haven't changed anything since may when version 2.0.0 was released other than the prefix i added yeteruday for you.

Kind Regards

Trevor Killick

[samwoo]

Hi Trevor,

Thanks. I have run the script for maredw who should have the extra Job roles from the import and it hasnt worked  He only has the Self Service User and Basic User role applied.

I'm at a loss... will keep checking anyway.

Thanks,

Samuel

[TrevorKillick]

Hi Samuel

Can you zip up the config the latest log file from the import and if possible download the ESPServerService.log file from the admin tool and ping it over to me i will take a look through them.

Kind Regards
Trev

[samwoo]

Hi Trevor,

Thank you. I have attached the file.

Many thanks,

Samuel

Hornbill_-_LDAP_Testing.zip

[TrevorKillick]

Hi Samuel

Looking through the logs i think there is an issue when you try and run the api admin::userAddRoles if one or more roles are already assigned then the whole api call returns a failure, i thought it used to work and silently fail if a role was already assigned.

I have asked our platform team to take a look, as soon as i have something to resolve this issue i will let you know.

Kind Regards

Trevor Killick

[samwoo]

Hi Trevor,

Thank you very much for investigating this. I will still use the LDAP imports to update the user information as I'm now confident (thank you) that it wont affect any current job roles assigned to a user.

I will keep an eye out for any updates to this issue.

Cheers,

Samuel

[TrevorKillick]

Hi Samuel

Just to keep you updated the Server Team have fixed the underlying issue causing the Updating of User Roles to fail, this will be fixed in an up coming release of the Platform. Unfortunately due to the nature of some changes that made there way into the server before this fix it could be a week or two before we see the particular build pushed to live. 

As soon as i have a build version i will update the post as well as posted when its been pushed to live.

No changed are required for this fix so it as soon as the server build is released the import will start working correctly.

Kind Regards

Trevor Killick

[samwoo]

On 05/08/2016 at 8:34 PM, TrevorKillick said:

Hi Samuel

Just to keep you updated the Server Team have fixed the underlying issue causing the Updating of User Roles to fail, this will be fixed in an up coming release of the Platform. Unfortunately due to the nature of some changes that made there way into the server before this fix it could be a week or two before we see the particular build pushed to live. 

As soon as i have a build version i will update the post as well as posted when its been pushed to live.

No changed are required for this fix so it as soon as the server build is released the import will start working correctly.

Kind Regards

Trevor Killick

Hi Trevor,

Urgent question - has the updated build version been pushed to live?

We have gone Live with the Self Service Portal today and lots and lots of users are not able to log calls due to them not having the Self Service User role. I've checked the LDAP_Import and it's clearly stating that the role should be added / updated against them.

Is this script ok for me to run until this issue is resolved?

INSERT h_sys_accounts_roles (h_user_id, h_role)
SELECT  h_user_id, 'Self Service User' 
FROM    h_sys_accounts 
where   h_class = 3 
and     h_account_status = 0 
and     h_user_id not in
        (
            SELECT h_user_id 
            FROM h_sys_accounts_roles 
            where h_role in ('Self Service User')
        )

We have stopped all non-priority phone calls coming into the Service Desk and these users without the Job Role are now unable to log calls.

Thanks,

Samuel