Recent communication from Hornbill regarding breach

[sjg]

Due to Hornbills recent announcement regarding a breach to their internal network, can I ask if there's any way to obtain log information to track any unusual activity to our Hornbill instance ? 

 

 

[Steve Giller]

Your current log files are available from the Admin view under Platform Configuration->Logs.

[Gerry]

@sjg

As per the announcement (and for anyone else's benefit looking at this), I can confirm that there has been no compromise of customer instances or any production instances that run the customer instances.  If this ever does change we will be the first to let you know, but as of now, the impact of the issue is only Hornbill's own internal network, which is 100% separate (physically and logically) from the production networks. 
Gerry

[Keith Stevenson]

@sjg
To also clarify what was said above. 

We can provide the logs however without a clear understanding of what is normal and understanding of what the logs "mean" then it wont help to detect a problem. We have a number of ways of monitoring instances to detect abnormal functions. 

So this is an overview of how SIEM works 

We collect and store 100s of metrics every 15 seconds which gives us an understanding of what is normal for any instance and instances in general. 

Because people and large groups of people are creatures of habit which is further enforced by work patterns and routines we are able to see patterns in the data that lend themselves to statistical analysis. 

An example of this is API counts. 

People generally do the same thing every day , they get up in the morning at same time (the alarms set), they goto work at the same time (its in their contract) and they perform the same actions at nearly the same time every day (First thing read your email, then read posts, then check BBC, then go for coffee, then do them process new calls, then off hold calls etc) . The pattern repeats and because of this we see on every instance a similar number of API calls per hour (within 1 standard deviation of normal) every day during the week and another pattern during the weekend.  This is further enhanced by the use of automation , and things like scheduled reports or scheduled jobs which also always happen at the same time. 

If we take these counts of API per hour for a given instance we see the pattern similar to the example below, and we can then look for anomalies in the data every hour when comparing to previous 6 or 12 months for that hour.  Any change to peoples work patterns may initially cause alerts but over a longer enough period these become the new normal. 

We can also look for daily, weekly , shift and other patterns and anything that is statistically significant (1 or more Standard deviations from the normal ) is investigated. 

We perform this analysis every hour of every day on all instances and anything found gets flagged to the cloud team SIEM workspace for review.  A manual review is then used to try and understand the events and then any remaining concerns escalated to the contacts for the given instance. 

We perform the same statistical analysis on Load, Database, data in\Out and other metrics as well as API count to have a understanding of what is "Normal" and what is not. 

In the below image, it shows a fingerprint for a demo instance , the patterns we look for (its compared data ) , and what happens when an alert is generated.

This is just 1 of many ways the instances and the infrastructure supporting it are monitored for patterns\weirdness. 

Kind Regards 

Hornbill Cloud Team

[Dan Munns]

Has a 3rd party security team been tasked with any data forensics or is all the remediation/investigation being handled in-house? 

[Gerry]

Hi Dan,

Currently in-house, we have a good handle on the situation thus far, but have a lot of things to go through.  If there is any customers-specific data exposure we are prioritising these for immediate disclosure which we should have done in the next day or two.  As previously stated, none of the customer networks, data storage or systems we compromised, only administrative data (files more specifically) that we have on our general network file servers where exposed, we are currently working through that content to determine if there is customer-specific PII or other data that needs to be reported back to individual customers. 

 

Gerry