Unable to access Hornbill?

[Adrian Simpkins]

Hi we are getting the below error code when accessing hornbill?

[carlt]

Hi

We're getting the same error

[Adam@Greggs]

We've got similar issues - status.hornbill.com is showing everything is running.

[Alisha]

Yes, we've been having issues this morning as well.

[Trevor Tinsley]

I'm receiving reports of general sluggishness and issues submitting service requests. Is this being investigated or should I add to the growing pile of incidents?

Edited November 29, 2019 by Trevor Tinsley
Adding screenshots

[Max Williams]

We are currently having issues too at the University of Portsmouth

[Nanette]

Dear all, 

We are currently investigating this with the highest of priority, as soon as there is any further update we will let you know.

Kind regards, Nanette

[Adam@Greggs]

We're able to get in occasionally, but I've noticed multiple automated calls have been duplicated. Possibly just a tidy-up afterwards, but may be related to whatever the issue is.

[Keith Stevenson]

All,

Thanks for the post. We have now identified the root cause and resolved this. A full RCA will be provided shortly, but briefly an index was removed on a given table that caused a number of frequently ran queries to take much longer than anticipated. The index has been re-added to all instances. 

 

Kind Regards

 

[nasimg]

@Keith Stevenson

Can you let me know when the RCA is available, I need to add it to my major incident report before I can close it.

Nasim

[Deen]

@nasimg RCA below:

At 07:40 on Friday we noticed slower than usual SQL query responses. The root cause was found to be the inadvertent removal of an index during the prior night's update. This index was removed in error, as there were other query dependencies that required the index to be in place for some large data sets. The issue was identified, diagnosed and resolved for all affected instances by 09:18 the same morning, and there was no data loss. We recognize this issue caused some of our customers disruptions and for that, we unreservedly apologize, thank you for your patience and understanding.