Deen

@John C I believe she did. Although I am not yet sure of the root cause.

This was added in error to the release notes and the notes have been corrected this morning to reflect this, as the feature is not actually present in the current release. I suspect it will be in an upcoming release although I don't yet have the details of when that might be.

@Darren Rose I take it we are talking about widgets here rather than actual reports? We have looked at some of the heavier ones and the largest contains the following in the query: h_itsm_requests.h_datelogged >= '2019-04-30 00:00:00' Is there a need for the query to bring back requests from early 2019? If this can be reduced to a weekly or monthly value that should significantly reduce the query time. Also looking at the query two of the fields h_fk_servicename and h_catalog don't have indexes, you could swap them with h_fk_serviceid and h_catalog_id as that should also help to speed up the query and get it under the current threshold.

@Adrian Simpkins If you could provide the details of the two impacted requests we can look into this further. It might be best if you do this via a new request though.

@Caroline I believe if the following Service Manager setting is set as required then the user that created the update should be able to edit it, as long as the conditions in the description below are met:

@Ben Maddams I've sent you an email, if you could reply to that mail with the requested passcode we can take a closer look into this for you.

@Trevor Tinsley Below is the RCA for last weeks outage: "At 16:26 BST on 14th June our monitoring systems detected a loss of connectivity to one of our datacentres (MDH - Maidenhead). This was a temporary loss of communication between the firewalls brought about by a temporary loss of power to a switch during routine maintenance. The power outage should not have caused any issues, but on this occasion, all remaining firewalls incorrectly took on master status. Steps were taken to resolve the issue by demoting secondary firewalls and this was completed at 16:42. The total impacted time was 16 minutes.A planned update to the firewalls which includes changes to mitigate this situation is already in place and will be scheduled over the coming months.The chance of the error occurring again is extremely low and we apologise for the inconvenience this will have caused."

@Adrian Simpkins @Daniel Good to hear you are back up and running.

Further changes have been made to correct the problem, if anyone still affected can flush their browser cache please and let me know if the problems persist.

Unfortunately although the patch did correct one of the issues there were further issues discovered here that needed to be addressed. Therefore a fix is being rolled out now and should be deployed to all shortly.

All, the patch has now been deployed, let me know if any of the problems persist. If you do continue to see issues be sure to flush your browser cache before trying again.

A patch is now in the process of being deployed to all instances. I will post confirmation once this has been completed.

All, we are aware of the issue and are currently working on a fix. I will have a further update for you shortly.

@Trevor Tinsley I will have that for you shortly and will let you know here when it is ready.

@Adam Toms I should have an RCA early tomorrow for you.