encoding Hornbill ESP Message Processing Error: Invalid Characters

[Matthew Puglisi]

My colleagues and I have seen this as a recurring issue and have noticed many of our Chinese emails being processed in this way. This is a serious issue as it means that many of our Chinese-speaking customers are not getting serviced properly and likely are not even aware of the issue. This then results in a bad impression of our Service Desk from the perspective of our Chinese-speaking customers. This may also affect other languages as well.

For the Admins, here are some Message tracking IDs for various errors we have received:

ERROR: Failed to add message to the inbound queue. Operation[mail::addMessageToQueue] The XMLXC request message contains invalid characters for the UTF-8 encoding scheme

imap4-Service Desk-550493b7b08253a5
imap4-Service Desk-e41dbdd809c08196
imap4-Service Desk-2068a108ca7b42ba
imap4-Service Desk-99ac61b2227ace7e
imap4-Service Desk-1693a278d8a669d7
imap4-Service Desk-d5b6b907649d15dd
imap4-Service Desk-90039ab599533746
imap4-Service Desk-60bcdcb437a19ee4
imap4-Service Desk-c2cd672e8e73bda0
imap4-Service Desk-616bbb1a172ba219
imap4-Service Desk-29299283b066c275
imap4-Service Desk-578086da095bc570
imap4-Service Desk-5c8286e31ad14d97
imap4-Service Desk-5562b92897d31e0b
imap4-Service Desk-5d5e63c9640554b6
imap4-Service Desk-32e7b9d8b9e0938c
imap4-Service Desk-662d4c87a064dd62

BONUS ERROR: Illegal byte sequence

imap4-Service Desk-a079ae1ca8eb5d2a

As you can see, we are getting a LOT of UTF-8 encoding errors.

SUGGESTION: Furthermore, I would submit that these error messages are effectively useless to our support team as it abstracts away the useful content into a cryptic ASCII-based RFC822 formatted text file attachment. It would be FAR better, as a service tech, if you took the opposite approach. That is, to display the message in its original form (replacing only the unsupported characters with question marks or squares; as we commonly see where unsupported characters were transcoded), and instead attach the error message for reference with the Hornbill Development Team. This way, we can at least have enough of the original message in tact to be able to support our customers while working through the technical issue on the backend with the Hornbill Development Team.

[Victor]

@Matthew Puglisi did you have a look at the thread started by your colleague where we discussed and explained the exact same "issue" you describe here?

 

[Matthew Puglisi]

11 hours ago, Victor said:

Did you have a look at the thread started by your colleague where we discussed and explained the exact same "issue" you describe here?

No, in fact I did not. I looked through several threads, so I've no idea how I missed that one. I apologize. I'll carry on the discussion there. Thanks!