Jump to content

Ldap Import troubleshooting (v3.0.3)

Lyonel

By Lyonel
in System Administration

 Share

Recommended Posts

Lyonel Mentor

Lyonel

Posted
Posted

Hi @TrevorKillick,

I am testing out the new version of the LDAP import (as recommended to me in another post), but I am having some problems...

Latest log I have (instance id and api key replaced here for security reasons): 

2018/04/13 14:11:08 [MESSAGE] ---- XMLMC LDAP Import Utility V3.0.3 ----
2018/04/13 14:11:08 [MESSAGE] Flag - config test-lyonel
2018/04/13 14:11:08 [MESSAGE] Flag - logprefix 
2018/04/13 14:11:08 [MESSAGE] Flag - dryrun true
2018/04/13 14:11:08 [MESSAGE] Flag - instanceid XXXXXX
2018/04/13 14:11:08 [MESSAGE] Flag - apikey XXXXXXXXXXXXXXXXXXXXXXXXXXXX
2018/04/13 14:11:08 [MESSAGE] Flag - apitimeout 60
2018/04/13 14:11:08 [MESSAGE] Flag - workers 1

2018/04/13 14:11:08 [MESSAGE] Loading Configuration Data: test-lyonel
2018/04/13 14:11:08 [MESSAGE] Loading LDAP Authetication Data: 1
2018/04/13 14:11:08 [MESSAGE] Log Level 2
2018/04/13 14:11:08 [MESSAGE] Page Size 1000

2018/04/13 14:12:08 [ERROR] Unable to Query Accounts List Cant read the body of the response
2018/04/13 14:12:08 [ERROR] Unable to Query Accounts List unexpected end of JSON input
2018/04/13 14:12:08 [ERROR] Unable to Query Accounts List The element <application> was not expected at location '/methodCall/params/application'
2018/04/13 14:13:10 [ERROR] Unable to Query Accounts List Cant read the body of the response
2018/04/13 14:13:10 [ERROR] Unable to Query Accounts List unexpected end of JSON input
2018/04/13 14:13:10 [ERROR] Unable to Query Accounts List The element <application> was not expected at location '/methodCall/params/application'
2018/04/13 14:13:13 [MESSAGE] Import Complete
2018/04/13 14:13:13 [MESSAGE] Accounts Proccesed: 1
2018/04/13 14:13:13 [MESSAGE] Created: 0
2018/04/13 14:13:13 [MESSAGE] Updated: 0
2018/04/13 14:13:13 [MESSAGE] Status Updates: 0
2018/04/13 14:13:13 [MESSAGE] Profiles Updated: 1
2018/04/13 14:13:13 [MESSAGE] Images Updated: 0
2018/04/13 14:13:13 [MESSAGE] Groups Added: 0
2018/04/13 14:13:13 [MESSAGE] Groups Removed: 0
2018/04/13 14:13:13 [MESSAGE] Roles Added: 0
2018/04/13 14:13:13 [MESSAGE] Time Taken: 2m5s
2018/04/13 14:13:13 [MESSAGE] Total Traffic: 32
2018/04/13 14:13:13 [MESSAGE] ---- XMLMC LDAP Import Complete ----

Note: type in line 11 "Loading LDAP Authetication Data: 1" => missing a "n" in Authentication :)

Now in Debug mode: 

2018/04/13 14:26:52 [MESSAGE] ---- XMLMC LDAP Import Utility V3.0.3 ----
2018/04/13 14:26:52 [MESSAGE] Flag - config test-lyonel
2018/04/13 14:26:52 [MESSAGE] Flag - logprefix 
2018/04/13 14:26:52 [MESSAGE] Flag - dryrun true
2018/04/13 14:26:52 [MESSAGE] Flag - instanceid XXXXXX
2018/04/13 14:26:52 [MESSAGE] Flag - apikey XXXXXXXXXXXXXXXXXXXXXXXXXXXXX
2018/04/13 14:26:52 [MESSAGE] Flag - apitimeout 60
2018/04/13 14:26:52 [MESSAGE] Flag - workers 1

2018/04/13 14:26:53 [MESSAGE] Loading Configuration Data: test-lyonel
2018/04/13 14:26:53 [MESSAGE] Loading LDAP Authetication Data: 1
2018/04/13 14:26:53 [MESSAGE] Log Level 1
2018/04/13 14:26:53 [MESSAGE] Page Size 1000

2018/04/13 14:26:53 [DEBUG] Processing Old Log Files Current Retention Set to: 10
2018/04/13 14:26:53 [DEBUG] Query LDAP For Users
2018/04/13 14:26:53 [DEBUG] Creating LDAP Connection
2018/04/13 14:26:53 [DEBUG] LDAP Results: 1

2018/04/13 14:26:53 [DEBUG] Processing LDAP User Data
2018/04/13 14:26:53 [DEBUG] LDAP Users Processed: 1

2018/04/13 14:26:53 [DEBUG] Loading Users from Hornbill
2018/04/13 14:26:53 [DEBUG] Loading User Accounts List Offset: 0
2018/04/13 14:26:54 [DEBUG] Loading User Accounts List Offset: 1000
2018/04/13 14:26:54 [DEBUG] Loading User Accounts List Offset: 2000
2018/04/13 14:26:55 [DEBUG] Loading User Accounts List Offset: 3000
2018/04/13 14:27:55 [ERROR] Unable to Query Accounts List Cant read the body of the response
2018/04/13 14:27:55 [ERROR] Unable to Query Accounts List unexpected end of JSON input
2018/04/13 14:27:55 [DEBUG] Loading User Accounts List Offset: 4000
2018/04/13 14:27:55 [ERROR] Unable to Query Accounts List The element <application> was not expected at location '/methodCall/params/application'
2018/04/13 14:27:55 [DEBUG] Loading User Accounts List Offset: 5000
2018/04/13 14:27:55 [DEBUG] Loading User Accounts List Offset: 6000
2018/04/13 14:27:56 [DEBUG] Loading User Accounts List Offset: 7000
2018/04/13 14:27:56 [DEBUG] Users Loaded: 5094
2018/04/13 14:27:56 [DEBUG] Skipping Loading Roles Due to Config
2018/04/13 14:27:56 [DEBUG] Loading Sites from Hornbill
2018/04/13 14:27:56 [DEBUG] Loading Sites List Offset: 0
2018/04/13 14:27:56 [DEBUG] Sites Loaded: 97
2018/04/13 14:27:56 [DEBUG] Loading Orgs from Hornbill
2018/04/13 14:27:56 [DEBUG] Loading Orgs List Offset: 0
2018/04/13 14:27:56 [DEBUG] Orgs Loaded: 47
2018/04/13 14:27:56 [DEBUG] Loading User Orgs from Hornbill
2018/04/13 14:27:56 [DEBUG] Loading User Accounts Orgs List Offset: 0
2018/04/13 14:27:56 [DEBUG] Loading User Accounts Orgs List Offset: 1000
2018/04/13 14:27:56 [DEBUG] Loading User Accounts Orgs List Offset: 2000
2018/04/13 14:27:56 [DEBUG] Loading User Accounts Orgs List Offset: 3000
2018/04/13 14:27:57 [DEBUG] Loading User Accounts Orgs List Offset: 4000
2018/04/13 14:27:57 [DEBUG] Loading User Accounts Orgs List Offset: 5000
2018/04/13 14:27:57 [DEBUG] Loading User Accounts Orgs List Offset: 6000
2018/04/13 14:27:57 [DEBUG] Loading User Accounts Orgs List Offset: 7000
2018/04/13 14:27:57 [DEBUG] Loading User Accounts Orgs List Offset: 8000
2018/04/13 14:27:57 [DEBUG] Loading User Accounts Orgs List Offset: 9000
2018/04/13 14:27:57 [DEBUG] Loading User Accounts Orgs List Offset: 10000
2018/04/13 14:27:57 [DEBUG] Loading User Accounts Orgs List Offset: 11000
2018/04/13 14:27:57 [DEBUG] Loading User Accounts Orgs List Offset: 12000
2018/04/13 14:27:57 [DEBUG] Loading User Accounts Orgs List Offset: 13000
2018/04/13 14:27:58 [DEBUG] Loading User Accounts Orgs List Offset: 14000
2018/04/13 14:27:58 [DEBUG] Loading User Accounts Orgs List Offset: 15000
2018/04/13 14:27:58 [DEBUG] Loading User Accounts Orgs List Offset: 16000
2018/04/13 14:27:58 [DEBUG] User Orgs Loaded: 1

2018/04/13 14:27:58 [DEBUG] Processing User Data
2018/04/13 14:27:58 [DEBUG] Auto Generated Password for: XXXXXXXXXXXX - YYYYYYYYYY
2018/04/13 14:27:58 [DEBUG] LDAP Attribute for Site Lookup: [physicaldeliveryofficename]
2018/04/13 14:27:58 [DEBUG] Looking Up Site XXXXXXXXXXXXXX
2018/04/13 14:27:58 [DEBUG] Found Site in Cache94
2018/04/13 14:27:58 [DEBUG] LDAP Attribute for Org Lookup: [extensionAttribute1]
2018/04/13 14:27:58 [DEBUG] Looking Up Org 
2018/04/13 14:27:58 [DEBUG] Unable to Find Organsiation 
2018/04/13 14:27:58 [DEBUG] User: XXXXXXXXXXXXX
	Create: true 
	Update: false 
	Update Type: false 
	Update Profile: false 
	Update Site: false
	Update Status: false 
	Roles Count: 0 
	Update Image: false 
	Groups: 0

2018/04/13 14:27:58 [DEBUG] User Data Processed: 1
2018/04/13 14:27:58 [DEBUG] Finalising User Data
2018/04/13 14:27:58 [DEBUG] User Create: XXXXXXXXXXXXXX
2018/04/13 14:27:58 [DEBUG] User Create XML YYYYYYYYYYYYYYY (can be provided in PM)
2018/04/13 14:27:58 [MESSAGE] Import Complete
2018/04/13 14:27:58 [MESSAGE] Accounts Proccesed: 1
2018/04/13 14:27:58 [MESSAGE] Created: 1
2018/04/13 14:27:58 [MESSAGE] Updated: 0
2018/04/13 14:27:58 [MESSAGE] Status Updates: 0
2018/04/13 14:27:58 [MESSAGE] Profiles Updated: 0
2018/04/13 14:27:58 [MESSAGE] Images Updated: 0
2018/04/13 14:27:58 [MESSAGE] Groups Added: 0
2018/04/13 14:27:58 [MESSAGE] Groups Removed: 0
2018/04/13 14:27:58 [MESSAGE] Roles Added: 0
2018/04/13 14:27:58 [MESSAGE] Time Taken: 1m5s
2018/04/13 14:27:58 [MESSAGE] Total Traffic: 32
2018/04/13 14:27:58 [MESSAGE] ---- XMLMC LDAP Import Complete ----

Also, it does not seem to take into account the logprefix value... and anything after that for that matter. 

Here is the command line I run:  

LDAP_User_Import.exe -instanceid XXXXX -apikey XXXXXXX -config test-lyonel -dryrun true [-logprefix testLR -version true]

Any idea on why it crashes in the middle of the process? Is it cause for concern?

Link to comment
Share on other sites
TrevorKillick Experienced

TrevorKillick

Posted
Posted

@Lyonel

It looks like depends on the other traffic on the box 1000 users a time can be too much so I'm going to change the default to something like 500 for now.

I will try and reproduce locally now i can see the issue and try and see if there does need to be an upper limit placed on this pace size.

Kind Regards

Trevor Killick

Link to comment
Share on other sites
Lyonel Mentor

Lyonel

Posted
  • Author
Posted

@TrevorKillick I think there is a small bug with the organisation section in the Hornbill Admin UI:

image.thumb.png.a315f603f6207a446a4cba18309a02dc.png

image.png.7dde6dd5debffd8b9a7315c5401b3fb6.png

Both options "Division" and "Company" have a value of 4 (instead of 4 and 5 respectively).

When selecting the option "Company" and then go to the debug tab:

image.png.d4e5b6266ffc2b6f70e6a83f55cd97ca.png

I would expect to see "Type": 5

If you get a chance to include wiht the next admin console release? That would be fab! I cannot progress my testing with this issue unfortunately :(

But to finish on a good news, thanks to this new version of the tool I managed to reduce the number of imports we need to do from 34 to 21 !! :rolleyes:

Link to comment
Share on other sites
Lyonel Mentor

Lyonel

Posted
  • Author
Posted

Thanks @TrevorKillick! Much appreciated, I will keep an eye out for the next release and resume my testing then.

By the way, I really like v3 of the tool. Much better than the previous version. I have to admit that as a developer myself, I very much appreciate the continuous improvement, effort and time you guys put in improving things (even if they work just fine). Good job ! :D

  • Like 1
Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share
Go to topic listing
×
×
  • Create New...