Maintaining email chains
Email chains are important for keeping track of email conversations and properly represent them in Jira as issues and their comments.
By default, messages that are part of a particular mail thread are identified, and the corresponding email thread is maintained based on the Jira issue key appearing in the subject of the messages. Even if the issue key is not included in the subject, issues can still be identified based on the incoming email with the help of the Message-ID.
A Message-ID is a globally unique message identifier that refers to a particular version of a particular message. It is generated and set in the header part of the email by the email server that dispatches it.
Note: According to the RFC-5322 standard Message-IDs are not mandatory (header) attributes of an email, but it is highly recommended to provide a Message-ID, as most service providers comply with this standard. See the standard here.
Typical use cases
To better understand the need for managing mail threads, consider the following use cases.
Use case 1
Anne sends an email to Bill and Corp, where Corp is the mail account that is processed by Jira. The email does not contain any references to an existing issue in Jira.
The issue is created in Jira by our Mail Handler.
Bill receives the email and uses the Reply-All function to send a reply to Anne and implicitly to Corp as well, since Corp was on the original email.
The reply email reaches Anne and Jira, but as it lacks a valid issue key, a new issue is created again by the Mail Handler, resulting in a duplicate.
Note: The preferred (or even the expected) behavior is to save Bill’s response as a comment in the issue created from Anne’s original email.
Use case 2
Anne sends an email to Bill and Corp, where Corp is the mail account that is processed by Jira. The email does not contain any reference to an existing issue in Jira.
The issue is created in Jira by our Mail Handler.
Bill receives the email and replies to Anne and Daniel (using the Reply or the Forward option of his mailer app), but does not include Corp in the recipient list (i.e. explicitly removes it).
Daniel receives the email and replies to Anne, but not to Bill and Corp.
Anne receives the email and replies to Daniel’s response, but at the same time, she decides to send the message also to Corp (she adds it to the recipients again).
The reply email reaches Daniel and Jira, but as it lacks a valid issue key, a new issue is created again by the Mail Handler, resulting in a duplicate.
Note: The preferred (or even the expected) behavior was to save Anne’s response from step 5 as a comment in the issue created from Anne’s original email.
In contrast to the previous example, the mail sub-thread in step 6 is completely detached from the original (main) thread started in steps 1 and 2.
If in the meantime somebody replies from Jira to the original message (to Anne or Bill or both), a completely independent sub-thread is created possibly containing several replies of its own.
On the other hand, these two sub-threads will never be mixed again (even if there are some common recipients every now and then), the only common item is the original message sent by Anne in step 1.
Assuming that the initial subject stated in the original email sent by Anne in step 1 is valid and relevant in any message of any sub-thread, it is a justified expectation to have them assigned to the very same Jira issue.
Use case 3
Anne sends an email to Bill and Daniel. The email does not contain any reference to an existing issue in Jira.
Bill receives the email and decides to have a Jira issue created from the topic, therefore he replies both to Anne and adds Corp (the mail account that is processed by Jira) as a CC recipient. Besides, he removes Daniel from the recipient list.
The issue is created in Jira by our Mail Handler.
Anne receives the email but replies to Bill only (instead of Reply-All, she uses the Reply function). As a consequence, Jira is not notified about the update, it does not know about the reply message.
Daniel also receives the original email from Anne, then he also decides to have a Jira issue created from the topic (he does not know about the already created issue as he was excluded from the reply emails in the main thread) so he replies to both Anne and Bill, and adds Corp as a CC recipient.
The reply email reaches Anne, Bill and Corp (the mail handler in Jira), but as it lacks any valid issue key, a new issue is created again by the Mail Handler, resulting in a duplicate.
Note: The preferred (or even the expected) behavior was to save Daniel's response from step 5 as a comment in the issue created from Bill’s reply in step 2 and 3.
In this last example we saw an incident when Jira was not involved in the conversation from the very beginning, but it was notified from two different sub-threads at different stages during the flow of conversation. It is also expected to get those replies to appear as comments in the very same Jira issue.
The occurrence of such duplicate, but seemingly unrelated Jira issues is a recurring problem - especially when managing lots of customer requests - and causes confusion in many cases, as cohesive messages and their history are administered in separate Jira issues.
In addition to the propositions above, it is extremely unlikely for mail threads to have a linear history. There is an almost unlimited number of combinations for creating branches, that is, sub-threads in a mail thread, where some of the original recipients are excluded or other recipients are added (or re-added) on the way.
The main purpose of mail chain management is to identify such mail threads correctly and preserve the cohesion of the messages in a thread that do not have any issue keys in the subject of emails. In other words, mail chaining exists to avoid creating duplicate issues.
To get around the problem of ensuring the cohesion of email threads, please follow the instructions below.
Configuring email thread management
To support email thread management based on Message-IDs do the following:
1. Go to Incoming Mails --> INCOMING MAIL HANDLERS --> Mail Handlers.
2. Click on the three dots next to the name of the mail handler and choose Edit.
3. In the Edit Mail Handler dialog, make sure the Enable issue lookup by mail headers toggle is checked under the Find issue tab.
Result: Your email chains are automatically maintained.
Now the handler attempts to find issues by extracting Message-IDs from the References or In-Reply-To email headers of the reply emails. If found, the reply mail’s body is added as a comment, which is the desired behavior in most use cases.
Important: In order for this rule to take effect, the message subjects must not contain any existing issue keys, otherwise the issues will be matched by the "issue key in subject" rule due to its precedence. This may lead to confusion in particular situations, if the two rules would otherwise result in different issue resolutions that should not happen in an ideal setting.
Operation principles
If possible, for all incoming emails, all available Message-IDs that can be extracted from the headers of the current incoming email are saved to their respective Jira issues, even if this particular lookup is disabled (i.e. if the matching issue is found based on the “issue key in subject” rule or based on a JQL filter condition).
By implementing this kind of retroactive Message-ID storage, the Jira issue will also know about the preceding part of the mail thread, even if Jira was getting into the loop at a later stage.
By disabling the Enable issue lookup by mail headers option, the matching issue will no longer be found based on the Message-IDs. This may result in creating new issues, as described in the use cases section above. However, re-enabling this lookup will then possibly return multiple issues for the given search, which can be problematic by nature.
Handling multiple matches
Provided that the mail chaining feature is turned on, following a phasing-out mechanism multiple matches are not possible anymore. However, if either the mail chaining feature is temporarily disabled or if duplicate issues already exist for a particular mail thread in the system at the time of the introduction of this feature, the problem of multiple matches must be addressed.
As email processing runs in the background without user interaction, our primary goal is to provide a systematic approach to an unambiguous resolution of the matching issue in most possible cases. That is, we try to identify which issue the message should be assigned to.
In case of multiple results, the issue with the lowest issue ID is automatically selected. As most probably an issue with the lowest issue ID was created the earliest, it should be the original entry point for the given topic. Therefore all other issues can be considered as duplicates.
Searching issues manually by Message-ID
The list of the associated Message-IDs are stored as a JSON array in a Jira entity property of the given issue. It is possible to check the saved data from Jira’s user interface too, as this entity property was made publicly searchable. In order to do this, navigate to Filters --> Advanced issue search, and enter any of the following expressions by substituting with the appropriate data:
To search for issues based on a known Message-ID (e.g. “messageId1“) that appears in the associated mail thread, enter the following JQL search query:
To search for issues based on a known Message-ID (e.g. “messageId2“) that was the entry point of the given mail thread for the mail handler, enter the following JQL search query:
The expressions above can also be combined, e.g. to search for issues that have any of the Message-IDs (“messageId1“ or “messageId2“) in their mail threads, use the following JQL search query:
This manual search feature is a great tool to double check your issue's metadata and start troubleshooting if you encounter anything out of the ordinary, possibly due to multiple matches (e.g. it's not the expected issue that is being processed by the mail handler). In the case of “mail thread equivalent duplicates”, you need to do the clean up among the issues to ensure correct behavior.
Note: Normally, an ever-increasing record is built from the Message-ID list of a continued mail thread. However, if the size of the Message-ID list exceeds 32 kB, only the most relevant Messsage-IDs are preserved, the less relevant ones are discarded. Usually, this won’t bring the mail chaining feature to a halt, as a mail thread can also be identified typically based on one of the “most relevant” Message-IDs only.
Special-purpose Message-IDs:
The
initialMessageId:the entry point of an email thread from the mail handler’s point of view, which is stored in a dedicated field.The Message-ID is of the “root message”, the original message of the entire email thread, which is usually the first item in the Message-ID list featured in the References header.
Message-IDs in outgoing emails
The outgoing emails sent by Email This Issue contain Message-IDs specific for Jira (as Message-IDs are generated by the service provider’s own choice). This Message-ID contains the numeric issueId to easily identify the associated issue of any message sent by Jira later on.
Important: Since we use this feature to identify the associated issue for reply messages, the References header is currently not populated by the app, so all the previous Message-IDs of the thread are not copied over. This may lead to confusion with certain email clients, as such outgoing mails and their replies can be misconceived by the clients and regarded as separate email threads, therefore they might lose track. However, this won’t cause any malfunction at the Jira side: as soon as any reply is posted to the mail handler again, the respective issue will be found and commented, as expected.
Last updated
