Genii Weblog

Defect detection in Notes documents

Tue 15 Aug 2017, 12:53 PM



by Ben Langhinrichs

CoexLinks family of products: CoexLinks Fidelity, CoexLinks Migrate and CoexLinks Journal

Very soon, we are releasing a new version of all three of our CoexLinks products, CoexLinks Fidelity, CoexLinks Migrate and CoexLinks Journal. Aside from other features and bug fixes, they will share a new feature called Defect Detection. While the challenge for most document rendering (to MIME in this case) is faithfully reproducing the content of the email and including the envelope information in the desired form, some Notes emails have corruptions and defects which make the job harder.

There are four major defects (and a few smaller ones not worth mentioning):

  • Broken inline images. A variety of corruptions in images including zero-length data, missing image segments and incorrect image type (e.g., a GIF is marked as a JPEG) leave images broken in both the Notes client and the rendered document. We are able to detect and repair or partially repair about 75% of these corrupted images.
  • Compressed attachments with incorrect sizes. These are difficcult to detect because you can open the attachment or save it to disk from the Notes client, so you don't know you have an issue. But since the uncompressed size is incorrect, the document will be truncated and corrupted when emailed or when it is rendered by most tools including the Domino rendering engine. We can fix 100% of these corruptions.
  • Hotspots with invalid ends. In some versions of Notes, URL hotspots and other hotspots inside sections or table cells were left without a closing record. While they appear fine in Notes, they render with either large parts of the Body content missing, or with everything to the end showing as a URL link. We can fix about 95% of these corruptions.
  • Invalid stored image URLs. These corruptions are an artifact of the external MIME to internal MIME rendering, so mostly appear with received MIME emails or forwarded/replied to MIME emails. The fix is fairly simply, so we can fix 100% of these corruptions.

Whether you are sending email to customers, reading your own mail from a mobile or web interface, migrating an entire database or journaling mail to a third party vault, it is better to have defect detection in place so that unusual does not become the irretrievable.

Copyright © 2017 Genii Software Ltd.

What has been said:


1087.1. Uwe Brahm (08/16/2017 11:46 AM)

Obvious Questions come up:

- Does IBM know the cause of these corruptions?

- Is there one or more PMRs for them?

- Is there any version of Notes that does it correctly?

Uwe


1087.2. Ben Langhinrichs (08/16/2017 12:08 PM)

Uwe, I want to be clear, these are not all necessarily current issues. I know at least one appears to be caused by a well regarded third party product, but I don't know which versions of that product or of Domino lead to the issue. I know that IBM doesn't consider the hotspot issue a corruption. The rendering engines built in have numerous issues that IBM would see as far more important than these if they had any motivation to fix them.

There is no version of Notes that fixes all of these, but even if there were, it wouldn't help much. If you want to migrate the past 10-15 years of email using CoexLinks Migrate, you have emails created in a variety of different versions with a variety of different third party addins. Nothing IBM has will fix the problems in existing emails, which is why we need to fix them in our product as best we can. IBM wouldn't take on a PMR to fix problems that may or may not still be being created, and which may or may not be caused by a third party addin, especially when the goal is to export those emails out of Notes to either a vault or another email system.

I hope that clarifies things.

Ben