Jump to content

Keith Stevenson

Root Admin
 View Profile  See their activity

  • Posts

    2,614

  • Joined

  • Last visited

  • Days Won

    40

 Content Type 

Posts posted by Keith Stevenson

    email issue

    in Service Manager

    Posted

    Giuseppe,

    Thanks for the reply. From the logs we can see that this stopped working at 00:10 this morning. Can you confirm that you can login to Outlook Portal as servicedesk@datalogic.com (rather than login as another user and switch to the mailbox).

    If so then re-entering the Password on the mail connector properties in Hornbill Admin might resolve this.

     

    Kind regards

    Keith Stevenson

    email issue

    in Service Manager

    Posted

    Giuseppe,

    Thanks for the post. The error message is stating that Outlook  cannot verify the Login credentials supplied. This may occur as the account has changed or password expired. 

    This is a MS office account error.

    You should be able to confirm this with your Office365 Admin

    Kind regards

     

    Unable to Access HB

    in Service Manager

    Posted

    All

    At around 10:35 this morning, during a routine update to a backed server an error occurred in our subscription checker. This caused a number of instances to temporarily loose the count of available subscriptions for a subset of applications.

    The error was quickly identified and resolved by 10:37 (Change backed out as per change request plan).

    The type of update that occurred on the backend server happens around once a week as we continuously develop\enhance our service and this is the first time an error has been witnessed. The issue was not seen in our development or beta environments during testing.

    Inorder to prevent issues like this in future we are already looking at having additional tests in place.

     

    Kind Regards

    • Like 2

    Extremely High Bandwidth from Mailbox to Hornbill

    in Service Manager

    Posted

    Micheal,

    Thanks for the reply.  Your instance is configured for us to Download email from your internal mail server via IMAP, so we would be connecting\pulling and uploading nothing (I suppose your router may count the commands we use to connect\view IMAP mailbox and download anything it finds as uploads but thats going to be far less than 37.4GB - probably < 10mb for entire day).

    Can you provide the report as a private message? Perhaps its the wording thats wrong?

    Kind Regards

    Keith Stevenson

    unable to log on to Service Manager

    in Service Manager

    Posted

    Prathmesh,

    Thanks for the confirmation. We noticed at 8:39 that one of our nodes was acting strange and refusing connections. Investigations showed that this was the result of a deadlock on the back end database. The lock was cleared around 8:48 and once done your instance could again connect.

    We are investigating the root cause and will add additional monitoring to ensure that this will not occur again.

    Kind Regards

    Keith Stevenson

    • Like 1

    Hornbill Down?

    in Service Manager

    Posted

    All,

    Following the above we have now made changes to the Analytics engine and Admin tool used to manage this (Prevents multiple joins that would return an extraordinary number of records). This is currently on our beta testing stream and once tested will be rolled out to all.

    We are also changing the way services handle large data sets (In excess of 30GB) and adding additional self correcting mechanisms to avoid issues (These will be rolled out as part of our continuous delivery of the weeks ahead) . Looking even further ahead we already have plans to overhaul reporting that will drastically reduce the impact of any large data sets.

    Kind Regards

    Keith Stevenson

    • Like 1

    Hornbill Down?

    in Service Manager

    Posted

    All,

    We have just witnessed the issue again (Same subset of customers). This occurrence of the issue has now been resolved. The root cause is that a set of temporary resources was exhausted causing a bottleneck whilst other requests waited. The resolve was to clear the block.

    This issue has become apparent due to a recent addition to the Service Manager application and advance analytics, which allows the creation of free form reports\joins. If these are not carefully crafted they can create unexpected large volume sets.

    We are working to identify any instance that may suffer from this issue and will be contacting the primary\secondary contacts for each instance to provide guidance on report creation whilst we investigate a more permanent solution.

    We apologies for any inconvenience.

    Kind Regards

    Keith Stevenson

     

     

    • Like 1

    Hornbill Down?

    in Service Manager

    Posted

    All,

    Thanks for the posts. We can inform you that our monitoring system detected an issue at 13:05 which effected a subset of instances . The root cause was identified and steps taken to resolve at 13:08, however it was then also found that this required a restart of 1 of the backened servers which took approximately 5 mins to restart.  All services were fully restored at 13:18.

     

    We are still investigating the root cause and will provide a further update with analysis and steps we will undertake to ensure issue cannot happen again shortly.

     

    Kind regards

    • Like 1

    Failed to initialize application - operation timed out

    in Service Manager

    Posted

    Alex,

    Glad to hear that this is resolved. We have now identified the root cause (issue with stale Database connections getting reused rather than creating new ones) and will change our code to ensure that this doesnt happen again.

    We will also add additional monitoring checks to ensure that if such an event occurred in the future we are notified (and can take action) before it becomes an issue for yourselves.

    Kind regards

    Keith Stevenson

     

     

    • Like 2

    Accessibility requirement - Change field colour on assets

    in Service Manager

    Posted

    Reena,

    One option that might help whilst we review the possibility of changes would be to make use of a font designed for Dyslexia. This font (available from https://www.dyslexiefont.com/en/dyslexia-font/) is free for home\personal use and in Firefox (or chrome via extension) its a simple task to change all web pages to use this rather than ours. (See https://support.mozilla.org/en-US/kb/change-fonts-and-colors-websites-use)

    You can also change the background colour to help (Usually to Yellow) which should achieve your requirements.

    If the above helps im sure we can look at adding more options for accessibility in the future.

    Kind Regards

    Keith Stevenson

    service.hornbill.com DOWN

    in Service Manager

    Posted

    Everton1878

    Sorry to hear that your having trouble. There should be no issue  as the DNS record for the above was fully propagated last week and the TTL on any old record should have passed easily by now.   Can you do a NSLOOKLUP for service.hornbill.com from the CMD prompt on an effected machine and post the results as this will show us what DNS server your using, the records TTL etc. If you can also confirm this happens in all browsers that would help (Chrome does its own caching of DNS)

     

    Kind regards

     

     

    service.hornbill.com DOWN

    in Service Manager

    Posted

    All,

    Following on from the above posts and as clarification.  We monitor all service endpoints but have been monitoring them via their IP addresses and not using their FQDN (domain name), we also monitor our DNS service provider and to now that has worked ok. We made a decision to transition over to CloudFlare to take advantage of their superior service as well as future global caching to help accelerate our application content, this was executed last night. Unfortunately we misconfigured the DNS entry for service.hornbill.com but our monitoring continued to show the service as available because we were monitoring the service via its IP address and not its FQDN. We have now reviewed our monitoring setup and re-configured to use the domain name rather than IP address for all services to ensure this gap in our monitoring does not happen in the future. Sorry for any inconvenience caused.
     
    Kind Regards
     
    Keith Stevenson
     
     
     
    • Like 1

    Our instance isn't wirking

    in Service Manager

    Posted

    Adrian,

    Sorry to hear that your having problems. Ive looked at the server and although your instance was taking a large amount of RAM about 30 minutes ago, it seems to have settled down. From the logs I can see users logged it and performing actions. Can you confirm that you are still having issues?

    As for the reason why the RAM was high, the logs state that an analyst was processing an attachment with large number of attachments, but I will need our development team to investigate further. 

    Look forward to your response.

    Kind Regards

    Keith Stevenson

×
×
  • Create New...