Structured Email: Problem Statement and Areas of Work

Introduction Email is a successful and widespread technology which is likely to remain important, even if new vendors and technologies continuously challenge its role. Email, to some extent, can be considered a victim of its own success. Interrelations of various components have created a relatively stable software ecosystem, which makes it difficult to introduce larger improvements. While many RFCs have updated and extended initial email specifications, the major inner workings of email remain widely unchanged since their inception. This documents lines out certain issues with email content that might be addressed by standardization work. The proposed approach aims to enable novel ways of how users and automated programs can interact via email while retaining downwards compatibility with existing email standards. For additional background, see also [I-D.happel-structured-dynamic-email-00].

Problem statement

Email content is not machine-readable A large share of today's emails is of transactional nature, i.e., sent by automated agents or processes to human users. While those emails often have relatively clear semantics (such as Invoice or Reservation), the medium of those emails is still human-readable text. In order to help users with managing and processing their emails more efficiently, a machine-readable representation of email content can be an important building block.

Using email as a "personal API" Beyond improving the handling of existing email interactions, it is worthwhile to note that email can be considered a prime technology in the recent discussion about data sovereignty, which aims to give back control to users over their personal data. Email is inherently open and decentralized. It is probably the only widely-used and standardized technology which seamlessly connects the local data space of users (i.e., emails on their PCs, mobile devices, or private hosted mailboxes) with the public internet. It might even be considered a ubiquitous "personal API" of internet users, which to date is mostly based on text-based instructions ("John, could you please send me this presentation?"). Along these lines, structured email might serve as an enabling technology for granting users additional sovereignty in date storage and exchange when used to interact with internet services.

Slow adoption of existing solutions In a mature technological space such as email, any novel approach such as structured email needs to address the issue of user adoption. Existing attempts to structure email interaction can be distinguished in standards-based and vendor-driven approaches. Several RFCs address very particular problems such as meeting workflows , message delivery notifications , mailing list subscriptions , message metadata or message content interpretation . Creating independent RFCs for each possible type of structured email interaction might not be a feasible approach for both standards makers and client developers. With respect to vendors, Google (, ) and Microsoft () have made attempts to structure email interactions. Even though some other vendors support, e.g., , a closer look shows that those implementations are mostly incompatible among each other (, , ). Besides a lack of standardization (and according tool support), widespread adoption also suffers from various sender restrictions, including manual approval processes.

Areas of work This section tries to identify and structure areas of work to address the aforementioned topics. We distinguish some core work, additional, and optional topics.

Core areas of work

Enable email clients to understand email content As a primary building block for structured mail, there needs to be a specification on how to represent email content or metadata about email content in a machine-readable form. Such an approach needs to be downwards compatible with existing practices and clients, probably in a similar way the "multipart/alternative" MIME part type is used for HTML email ("text/html"). for instance, is leveraging this approach by adding an additional MIME part in conjunction with "text/plain" and "text/html". Regarding content semantics, is using a subset of types in a JSON-LD representation. Ideally an open approach to structured email would allow for decentralized extensions of content semantics.

Enable users to compose structured email To sustain the decentralized and interactive nature of email, it is insufficient to just allow professional email senders to send structured emails to users. Instead, users need to be enabled to answer and also to initiate structured email exchanges. (initiated by Google) and Microsoft Actionable Messages are approaches that allow users to reply with a structurd answer based on input forms. However, the required input is not machine-understandable, making it difficult to assist users in data entry. Both approaches allow responses to be sent to a dedicated HTTP API only, but not as a regular email response. Both approaches also do not allow users to initiate a structured email exchange. For this purpose, a discovery mechanism would be required, which allows users resp. their email clients to determine which types of structured email content an intented email recipient is willing to accept besides standard email.

Cross-cutting areas of work

Internet Message Format extensions While structured email should be designed to maintain downwards-compatibility with the existing email technology stack, there might be some smaller helpful extensions:

Additional email header information might inform clients about the presence of structured email content.
It might be helpful to distinguish structured email markup which represents the complete message content (in the sense of "multipart/alternative") versus markup which provides partial annotations or metadata to regular email content. For the latter case, a standardized option to help clients identify such MIME parts (e.g., based on a MIME part header) might be useful.
A distinct characteristic of email technology is the multitude of processing options on both server-side or on client-side with multiple possible email clients operating on one account. This challenge is, e.g., also present in the email filtering design space /. Accordingly, there might be use cases in which users may want to direct structured email processing to certain clients (e.g., travel information to the mobile email app) in which additional header information might help guide processing.

Trust (Semi-)automated processing of structured email requires increased scrutiny with respect to security issues. Accordingly, current vendor-specific approaches (//) require DKIM and/or SPF set up in addition to a manual sender registration process. While the latter is already a clear inhibitor of adoption, this process will not work if users shall be enabled to send structured mail by themselves - a scenario without a central gatekeeper. Accordingly, suitable measures of trust need to be in place. Candidate approaches might be a consolidation of existing email trust indicators, specific concepts of "trusted senders", or drawing inspiration from sources such as the ACME protocol .

Interaction with HTTP backend APIs Structured email may benefit from certain server-side (HTTP) APIs, serving different purposes. An API for the discovery if an intended receiver supports structured email has already been discussed (see ). Additional APIs could be useful to allow synchronous responses of structured data in certain uses cases in which an asynchronous message response cycle would be infeasible. All current vendor-specific approaches (//) allow for some sort of direct data transmission against an HTTP backend API.

Optional areas of work

Email search and filtering Machine-understandable interpretation of email content could also be used to improve some existing practices based on text/keyword-based processing of email content. In particular, the IMAP SEARCH command () and email filtering might benefit from corresponding extensions. However, those might also be subject to work of other IETF working groups.

Interaction with other data types and external APIs Email clients have grown over years to support various types of user data beyond email. This typically involves contacts/address books, calendars, tasks, notes, or files. Besides the case of iMIP , interrelation between email and other data types is vendor-specific, if present at all. Structured email could help to enable easier interaction within email clients, with other applications on the same machine and with services on the internet (see also ). Ongoing work in the JMAP working group could overlap with some of these aspects. The W3C Solid initiative is another loosely related

Email storage formats Several vendors use machine learning or data extraction approaches (see also ) to derive information about email content which is similar to or complementary to structured email content. For interoperability and data portability, it would be helpful to standardize storage for such metadata. This might also help with current portability issues stemming from user-defined metadata such as IMAP flags ().

Common practices in email processing While structured email aims to provide a user-driven approach to make email content machine-understandable, there may be a number of more technical issues related to email-processing which might be addressed en route. Besides consideration of already existing RFCs (see ), this may involve currently unstandardized issues such as email signatures, vacation notices or "noreply" addresses. Those are probably more specific application areas of structured email, since they can impact multiple layers of email processing and corresponding standards.

Out of scope areas of work This section lists aspects which are relevant to structured mail but which might not be addressed by formal standardization work in the scope of this proposal.

Data extraction Several service providers or tools (e.g., ) apply data extraction techniques to derive structured data from textual email content. Data extraction is therefore an interesting approach to help bootstrapping the adoption of structured email, until it has become common practice among email senders. While cooperation on shared data extractors might be a useful activity for parties interested in structured email, it is probably not a subject for standardization.

UI design Prescription of how user interfaces should display structured email or organize interaction with it are - besides illustration - typically out of scope of RFCs. As in the case of data extraction, there might however be a shared interest among vendors to collaborate on certain best practices.

Privacy considerations Since email content often contains personal data, it is subject to privacy considerations. From a high level perspective, privacy issues in structured email should not significantly differ to privacy issues in existing email standards. On a more fine granuar level, structured email might both raise certain novel privacy issues (i.e., since structured data is more easy to process and share), but it could also improve certain privacy concerns. For instance, it could make certain data extraction practives (see ) obsolete, which currently cannot distingish senstive from non-sentitive parts of an email.

Security considerations As a problem statement document, no particular security considerations apply as such. Similar to privacy considerations, security issues might also widely overlap with those of existing email standards. See for the discussion of some general security aspects with respect to structured email.

IANA Considerations This document has no IANA actions at this time.