Asset Update script not working for me

[Frank Reay]

I am using the Asset Update script from the wiki here CSV Asset Import - Hornbill

I am doing this manually via the CMD prompt. I have managed to run but it does not seem to be able to find the asset I want to update (currently I am just testing on a single asset before I extend it to my long list). My config file says this:

{
  "APIKey": "xxxx",
  "InstanceId": "xxxx",
  "AssetIdentifier":"h_name",
  "CSVAssetIdentifier":"name",
  "CSVCommaCharacter": ",",
  "CSVLazyQuotes": false,
  "CSVFieldsPerRecord": 0,
  "CSVCarriageReturnRemoval": false,
  "LogSizeBytes":1000000,
  "AssetTypes": {
      "NET - Firewall": "c:\\Firewall.csv"
  },
  "AssetGenericFieldMapping":{
      "h_name":"",
      "h_site":"",
      "h_asset_tag":"",
      "h_acq_method":"",
      "h_actual_retired_date":"",
      "h_beneficiary":"",
      "h_building":"",
      "h_company_name":"",
      "h_cost":"",
      "h_cost_center":"",
      "h_country":"",
      "h_created_date":"",
      "h_deprec_method":"",
      "h_deprec_start":"",
      "h_description":"",
      "h_disposal_price":"",
      "h_disposal_reason":"",
      "h_floor":"",
      "h_geo_location":"",
      "h_invoice_number":"",
      "h_location":"",
      "h_location_type":"",
      "h_maintenance_cost":"",
      "h_maintenance_ref":"",
      "h_notes":"",
      "h_operational_state":"",
      "h_order_date":"",
      "h_order_number":"",
      "h_owned_by":"",
      "h_owned_by_name":"",
      "h_product_id":"",
      "h_received_date":"",
      "h_residual_value":"",
      "h_room":"",
      "h_scheduled_retire_date":"",
      "h_supplier_id":"[supplierId]",
      "h_supported_by":"[supportedBy]",
      "h_used_by":"",
      "h_used_by_name":"",
      "h_version":"",
      "h_warranty_expires":"",
      "h_warranty_start":""
  },
  "AssetTypeFieldMapping":{
        "h_description":"",
        "h_idx_ref":"",
        "h_mac_address":"",
        "h_manufacturer":"",
        "h_name":"",
        "h_net_ip_address":"",
        "h_physical_disk_size":"",
        "h_serial_number":""

  }
}
 

You will see that I am just trying to update 2 fields. I originally loaded the Supported By data into the Supplier ID field in error. Supplier ID is a strange field that cannot be edited manually so I need to delete the data in that field and put the same data into Supported By. It looks like Supplier ID is to be used for some future integration (which would be useful for us) so I think I really do need to delete the data from there.

This gave me the output file attached which in particular says this:

2021/01/16 17:08:15 [DEBUG] Loading Config File: C:\asset_import/conf_networkDevice.json
2021/01/16 17:08:18 [ERROR] API Call failed when retrieving Asset Class:Post /data/?method=entityBrowseRecords2: unsupported protocol scheme ""
2021/01/16 17:08:18 [DEBUG] API XML: <params><application>com.hornbill.servicemanager</application><entity>AssetsTypes</entity><searchFilter><column>h_name</column><value>NET - Firewall</value></searchFilter><maxResults>1</maxResults></params>
2021/01/16 17:08:18 [ERROR] Could not get Asset Class and Type. Please check AssetType within your configuration file:EOF

Later on in the file I can see that it has picked up my csv.

The Asset Type looks good to me

Has there been some system change as it seems to require an Asset Class but that is only given in the config file name. It seems to be complaining about an unsupported protocol scheme. I have tried renaming the Asset Type without spaces but that made no difference. 

The file does say 'Created Skipped: 1' but if I try to run it for real then it does nothing.

Asset_Import_2021-01-16T17-08-15Z_0.log

[Victor]

@Frank Reay did you have the CSV file open on your machine when you tried the import? "unsupported protocol scheme" usually occurs when the CSV file does not exist (not the case here I beleive) or when the file is opened somewhere else. Can you confirm this please?

[Frank Reay]

I can confirm that I have run an online checker on the json and that looked good.

What I have noticed is that all the fields in the csv (Template file for Network Devices) are not replicated in the json. I presume that will not cause an issue. The 3 fields I am referencing are in both the csv and json.

[Frank Reay]

@Victor I am fairly sure I had it closed. But I will try again.

[Frank Reay]

Tried again with the json and csv files definitely not open. Same response. You can see below that I am using the networkDevice json file so that is the correct Asset Class.

The exe I am running is from the _x64 zip (the 64 bit Cleaner Utility zip worked fine for me). So that also looks OK.

 

[SamS]

Hi @Frank Reay,

Can we confirm there are no other (network; eg firewall(*)/proxy server(**)) related configuration which might be interfering with the network traffic? Trying to obtain the asset class is the first thing the script is trying to do and it is communicating with Hornbill to do so.

(*) the list of servers to whitelist is the same as for our other scripts

(**) If you are using a proxy server, same as for other scripts, have you set HTTP_PROXY?

[Frank Reay]

Hi @SamS

I am currently working from home. I have zscaler (Internet protection) on my machine which I have disabled (although we don't think that would cause a problem anyway. That has made no difference. But there is nothing now between me and Hornbill.

I should highlight that I ran the CleanerUtility before Christmas without any issues. 

I noticed that you have just placed a new version of the exe in GitHub. I tried to run that and got a horrible error:

panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xc0000005 code=0x0 addr=0x0 pc=0x64e3b1]

[SamS]

Hi @Frank Reay,

I'll look into the particular issue you report, but the latest release is not relevant to your particular issue (hence no note of that in my previous response here).

[SamS]

Hi @Frank Reay,

I have just double-checked the release I uploaded and that isn't failing on my side. I would venture that the issue is thus environmental. If we have successfully excluded the network, then we are looking at either an issue with the configuration, the .csv file OR both.

IF the configuration file is as you gave it above, then the obvious field missing is the h_name - to match the updates to the assets you are looking to update.

Also, assume the headers/fieldnames  of the .csv are case-sensitive (just in case it is "supplierid" in the source).

[Frank Reay]

I have done more testing (without success) - I keep getting the same response that it cannot find the Asset Class or Type.

- I have changed Asset Type name so that it does not have a space or - (did that in the app and the config file).

- I have used a different config file for a different Asset Type requesting a very simple update. Same response.

- I have added [name] in the 2 places that h_name appears in the config file (under AssetGeneric... and  AssetSpecificFieldMapping).

- I have check case sensitivity "h_supported_by":"[supportedBy]",

- I have removed [supplierId] from that field to make the update even simpler.

- I have generated a new API key for the admin user. I have also tried using one of my own API keys.

I think you are correct that something is failing very early on as this is the first error:

2021/01/18 19:15:41 [DEBUG] Loading Config File: c:\asset_import/conf_networkDevice.json
2021/01/18 19:15:41 [ERROR] API Call failed when retrieving Asset Class:Post /data/?method=entityBrowseRecords2: unsupported protocol scheme ""
2021/01/18 19:15:41 [DEBUG] API XML: <params><application>com.hornbill.servicemanager</application><entity>AssetsTypes</entity><searchFilter><column>h_name</column><value>NET - Firewall</value></searchFilter><maxResults>1</maxResults></params>
2021/01/18 19:15:41 [ERROR] Could not get Asset Class and Type. Please check AssetType within your configuration file:EOF

I presume it knows to look for the Asset Class of networkDevice from the json file name? The above shows that it has been told the Asset Type is NET - Firewall but it can't find that either.

Can we see if API calls have been logged and could explain why they cannot retrieve the Asset Class and Type?

 

[Frank Reay]

@SamSI had a thought overnight and so this morning I deliberately put in an incorrect API key. I got the same response as before.

So I am clearly not making a connection to our Instance.

I think the next step is to try the CleanUtility which worked for me before Christmas. I will try that shortly - I won't rush it as it makes me nervous everytime I use it!

 

[Frank Reay]

@SamS Oh that's interesting!!!!

c:\HBCleanerUtility>hornbillCleaner.exe -instance=Landmark -apikey=xxxxx
2021/01/19 08:46:22 Error Decoding Zone Info File: invalid character '<' looking for beginning of value
The provided instance ID [Landmark] could not be found.

I didn't have this problem before Christmas (I am running from home as before ie via CMD Prompt with same API Key for admin user). I don't know what it means by invalid character'<' (is it referring to something in the exe?).

However it can't find our Instance name which is rather fatal.

[Frank Reay]

@SamSHi Sam. Is there an update on this?

[SamS]

Hi @Frank Reay,

No - no update. As you reported that the cleaner program now did not work, I assumed you realised that issue is environmental/network.

On further inspection: it appears that you are using "Landmark" as your Instance  ID in the command line, it will need to be "landmark" (all lowercase). If the cleaner program works, then we can rule out network issue, but if you also have "Landmark" as your instanceId in your CSV Asset import, then that will need lowercasing as well.

[Frank Reay]

Hi @SamS,

Yes lowercase cracked it. How depressing but at least it's all working now.

Thanks for your input.

 

[SamS]

Hi @Frank Reay,

Just to be on the safe side: did it work for BOTH (i.e. the cleaner AND the CSV Asset Import)?

[Frank Reay]

@SamS

Yes. When I log into HB support it is with instance name of Landmark so I just used that :-(

[Victor]

@Frank Reay it seems our documentation for import tools is slightly incomplete. The LDAP import tool documentation (and a few others) do specify the below, which is missing from the documentation for other tools:

instanceid - This is the name of your Hornbill instance and can be found within the URL you use to navigate to it: live.hornbill.com/[instance name]/. E.g. if the URL you use to access your instance is live.hornbill.com/arescomputing/, then your instance id would be "arescomputing". Remember, this value is also case sensitive. (https://wiki.hornbill.com/index.php?title=LDAP_User_Import)

I'll look into updating the wiki documentation with this information for all import tools.

[Frank Reay]

Thanks @Victor

I think that may save some embarrassment for other people!