A publish-subscribe architecture for the Constrained Application Protocol (CoAP)

Introduction The Constrained Application Protocol (CoAP) supports machine-to-machine communication across networks of constrained devices and constrained networks. CoAP uses a request/response model where clients make requests to servers in order to request actions on resources. Depending on the situation the same device may act either as a server, a client, or both. One important class of constrained devices includes devices that are intended to run for years from a small battery, or by scavenging energy from their environment. These devices have limited reachability because they spend most of their time in a sleeping state with no network connectivity. Another important class of nodes are devices with limited reachability due to middle-boxes like Network Address Translators (NATs) and firewalls. For these nodes, the client/server-oriented architecture of REST can be challenging when interactions are not initiated by the devices themselves. A publish/subscribe-oriented architecture where nodes are separated by a broker and data is exchanged via topics might fit these nodes better. This document applies the idea of publish-subscribe to Constrained RESTful Environments. It introduces a broker that allows to create, discover subscribe and publish on topics. The broker enables store-and-forward data exchange between CoAP endpoints, thereby facilitating the communication of nodes with limited reachability, providing simple many-to-many communication, and easing integration with other publish/subscribe systems.

Terminology This specification requires readers to be familiar with all the terms and concepts that are discussed in and . Readers should also be familiar with the terms and concepts discussed in and . The URI template format is used to describe the REST API defined in this specification. This specification makes use of the following terminology:

publish-subscribe (pub/sub):: A messaging paradigm in which messages are published to a broker, and potential receivers can subscribe to a broker to receive messages. Message producers do not need to know where the message will be eventually sent. The broker matches publications and subscriptions, and delivers publications to subscribed receivers.
publishers and subscribers:: CoAP clients can act as publishers or as subscribers. Publishers propose topics for creation and send CoAP messages (publications) to the broker on specific topics. Subscribers have an ongoing observation relation (subscription) to a topic. Publishers and subscribers do not need to have any knowledge of each other, but they must know the topic they are publishing and subscribing to.
topic collection:: A resource collection is a group of related resources that share a common base URI. In this case the the topic collection contains resources of the type "topic resource". CoAP clients can discover and interact with the resources in a collection by sending CoAP requests to the URI of the collection.
topic resource:: An entry within a topic collection in a broker. Topics are created and configured before any data can be published. CoAP clients can propose new topics to be created, but it is up to the broker to decide whether and how a topic is created. The broker also decides the URI of each topic resource and of the topic-data resource when hosted at the broker. The creation, configuration, and discovery of topics at a broker are specified in . Interactions about the topic-data are defined in .
topic-data resource:: Topic resources contain a property called "topic-data". The topic-data resource is a CoAP URI used by publishers and subscribers to publish (PUT) and subscribe (GET with Observe) to data (see ).
broker:: A CoAP server that hosts one or more topic collections containing topic resources. The broker is responsible for the store-and-forward of state update representations when the topic-data URI points to a resource hosted on the broker. The broker is also responsible of handling the topic lifecycle as defined in . The creation, configuration, and discovery of topics at a broker is specified in .

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here.

CoAP Publish-Subscribe Architecture shows a simple Publish/Subscribe architecture over CoAP. In it, publishers submit their data over a RESTful interface to a broker-managed resource (topic) and subscribers observe this resource using . Resource state information is updated between the CoAP clients and the broker via topics. Topics are created by the broker but the initial configuration can be proposed by a client, normally a publisher. The broker is responsible for the store-and-forward of state update representations between CoAP clients. Subscribers observing a resource will receive notifications, the delivery of which is done on a best-effort basis.

Publish-subscribe architecture over CoAP | | |--------> | subscriber | |___________| | | '--------> |____________| . | | . . | broker | . ___________ | | observe ____________ | | publish | | .--------- | | | publisher | ---------> | | |--------> | subscriber | |___________| |__________| '--------> |____________| ]]> This document describes two sets of interactions, interactions to configure topics and their lifecycle (see ) and interactions about the topic data (see ). Topic resource interactions are discovery, create, read configuration, update configuration, delete configuration and handle the management of the topics. Topic data interactions are publish, subscribe, unsubscribe, read and are oriented on how data is transferred from a publisher to a subscriber.

Managing Topics shows the resources of a Topic Collection that can be managed at the Broker.

Resources of a Broker The Broker exports a topic-collection resource, with resource type "core.ps.coll" defined in of this document. The interfaces for the topic-collection resource is defined in .

Pub-Sub Topics The configuration side of a "publish/subscribe broker" consists of a collection of topics. These topics as well as the collection itself are exposed by a CoAP server as resources (see ). Each topic has a topic and a topic data resources. The topic resource is used by a client creating or administering a topic. The topic data resource is used by the publishers and the subscribers to a topic.

Topic and topic-data resources of a topic

Collection Representation Each topic resource is represented as a link, where the link target is the URI of the topic resource. Each topic-data is represented as a link, where the link target is the URI of the topic-data resource. A topic-data link is an entry within the topic resource called 'topic_data' (see ). The list can be represented as a Link Format document . The link to each topic resource specifies the link target attribute 'rt' (Resource Type), with value "core.pubsub.conf" defined in this document.

Topic Creation and Configuration A CoAP client can create a new topic by submitting an initial configuration for the topic (see ). It can also read and update the configuration of existing topics and delete them when they are no longer needed (see ). The configuration of a topic itself consists of a set of properties that can be set by a client or by the broker.

Topic Properties The CBOR map includes the following configuration parameters, whose CBOR abbreviations are defined in of this document.

'topic_name': A required field used as an application identifier. It encodes the topic name as a CBOR text string. Examples of topic names include human-readable strings (e.g., "room2"), UUIDs, or other values.
'topic_data': A required field containing the CoAP URI of the topic data resource for subscribing to a pubsub topic. It encodes the URI as a CBOR text string.
'resource_type': A required field used to indicate the resource type associated with topic resources. It encodes the resource type as a CBOR text string. The value should be "core.ps.conf".
'media_type': An optional field used to indicate the media type of the topic data resource. It encodes the media type as a CBOR text string (e.g.,"application/json").
'target_attribute': An optional field used to indicate the attribute or property of the topic_data resource. It encodes the attribute as a CBOR text string. Example attributes include "temperature".
'expiration_date': An optional field used to indicate the expiration date of the topic. It encodes the expiration date as a CBOR text string. The value should be a date string in ISO 8601 format (e.g., "2023-03-31T23:59:59Z"). The pubsub system can use this field to automatically remove topics that are no longer valid.
'max_subscribers': An optional field used to indicate the maximum number of subscribers allowed for the topic. It encodes the maximum number as a CBOR integer. If this field is not present, there is no limit to the number of subscribers allowed. The pubsub system can use this field to limit the number of subscribers for a topic.

Topic Resource Representation A topic is represented as a CBOR map containing the configuration properties of the topic as top-level elements. Unless specified otherwise, these are defined in this document and their CBOR abbreviations are defined in .

Default Values Below are the defined default values for the topic parameters:

'topic_name': There is no default value. This field is required and must be specified by the client or broker.
'topic_data': There is no default value. This field is required and must be specified by the client or broker.
'resource_type': The default value is "core.ps.conf".
'media_type': The default value is an empty string, indicating that no media type is specified.
'target_attribute': The default value is an empty string, indicating that no attribute is specified.
'expiration_date': The default value is an empty string, indicating that no expiration date is specified. If this field is not present, the topic will not expire automatically.
'max_subscribers': The default value is -1, indicating that there is no limit to the number of subscribers allowed. If this field is not present, the pubsub system will not limit the number of subscribers for the topic.

Discovery Discovery involves that of the Broker, topic collections, topic resources and topic data.

Broker Discovery CoAP clients MAY discover brokers by using CoAP Simple Discovery, via multicast, through a Resource Directory (RD) or by other means specified in extensions to . Brokers MAY register with a RD by following the steps on Section 5 of with the resource type set to "core.ps" as defined in of this document. Brokers SHOULD expose a link to the entry point of the pubsub API at their .well-known/core location . The specific resource path is left for implementations, examples in this document may use the "/ps" path. Example: GET Uri-Path: ./well-known/core Resource-Type: core.ps <= 2.05 Content ;rt=core.ps;ct=40 ]]>

Topic Discovery A Broker can offer a topic discovery entry point to enable clients to find topics of interest. The resource entry point thus represents a collection of related resources as specified in and is identified by the resource type "core.ps.coll". The interactions with topic collections are further defined in . A topic collection is a group of topic resources that define the properties of the topics themselves (see Section ). Each topic resource is represented as a link to its corresponding resource URI. The list can be represented as a Link Format document . Topic resources are identified by the resource type "core.ps.conf". Within each topic resource there is a set of configuration properties (see Section ). The 'topic_data' property contains the URI of the topic data resource that a CoAP client can subscribe to. Resources exposing resources of the topic data type are expected to use the resource type 'core.ps.data'.

Topic Collection Interactions These are the interactions that can happen at the topic collection level.

Retrieving all topics A client can request a collection of the topics present in the broker by making a GET request to the collection URI. On success, the server returns a 2.05 (Content) response with a representation of the list of all topic resources (see Section ) in the collection. Depending on the permission set each client MAY receive a different list of topics that they are authorized to read. Example: 0.01 GET Uri-Path: ps Uri-Path: tc <= 2.05 Content Content-Format: 40 (application/link-format) ;rt="core.ps.conf", ;rt="core.ps.conf" ]]>

Getting Topics by Properties A client can filter a collection of topics by submitting the representation of a topic filter (see Section ) in a FETCH request to the topic collection URI. On success, the server returns a 2.05 (Content) response with a representation of a list of topics in the collection (see Section ) that match the filter in CoRE link format . Example: 0.05 FETCH Uri-Path: ps Uri-Path: tc Content-Format: TBD (application/pubsub+cbor) { "resource_type" : "core.ps.conf", "target_attribute" : "temperature" } <= 2.05 Content Content-Format: 40 (application/link-format) ;anchor="coap://[2001:db8::2]/ps/tc/h9392", ;anchor="coap://[2001:db8::2]/ps/tc/f3192" ]]>

Creating a Topic A client can add a new topic to a collection of topics by submitting a representation of the initial topic resource (see Section ) in a POST request to the topic collection URI. On success, the server returns a 2.01 (Created) response indicating the topic URI of the new topic and the current representation of the topic resource. If a topic manager is present in the broker, the topic creation may require manager approval subject to certain conditions. If the conditions are not fulfilled, the manager MUST respond with a 4.03 (Forbidden) error. The response MUST have Content-Format set to "application/core-pubsub+cbor". The broker MUST respond with a 4.00 (Bad Request) error if any received parameter is specified multiple times, invalid or not recognized. A CoAP endpoint creating a topic may specify a 'topic_data' URI different than that used by the broker. The broker may then simply forward the observation requests to the 'topic_data' URI. If the 'topic_data' is empty the broker will assign a resource for a publisher to publish to. 0.02 POST Uri-Path: ps Uri-Path: tc Content-Format: TBD2 (application/core-pubsub+cbor) TBD (this should be a CBOR map with the mandatory parameters) { "topic_name" : "living_room_sensor" "resource_type" : "core.ps.conf" } <= 2.01 Created Location-Path: ps/h9392 Content-Format: TBD2 (application/core-pubsub+cbor) TBD (this should be a CBOR map) { "topic_name" : "living_room_sensor", "topic_data" : "coap://[2001:db8::2]/ps/data/6578616d706c65" "resource_type" : "core.ps.conf" } ]]>

Topic Resource Interactions These are the interactions that can happen at the topic resource level.

Getting a topic resource A client can read the configuration of a topic by making a GET request to the topic resource URI. On success, the server returns a 2.05 (Content) response with a representation of the topic resource. The response has as payload the representation of the topic resource as specified in . If a topic manager (TBD) is present in the broker, retrieving topic information may require manager approval subject to certain conditions (TBD). If the conditions are not fulfilled, the manager MUST respond with a 4.03 (Forbidden) error. The response MUST have Content-Format set to "application/core-pubsub+cbor". The response payload is a CBOR map, whose possible entries are specified in and use the same abbreviations defined in . Example: 0.01 GET Uri-Path: ps Uri-Path: tc Uri-Path: h9392 <= 2.05 Content Content-Format: TBD2 (application/core-pubsub+cbor) { "topic_name" : "living_room_sensor", "topic_data" : "coap://[2001:db8::2]/ps/data/6578616d706c65", "resource_type": "core.ps.conf", "media_type": "application/senml-cbor", "target_attribute": "temperature", "expiration_date": "", "max_subscribers": -1 } ]]>

Getting part of a topic resource by filter A client can read the configuration of a topic by making a FETCH request to the topic resource URI with a filter for specific parameters. This is done in order to retrieve part of the current topic resource. The request contains a CBOR map with a configuration filter or 'conf_filter', a CBOR array with CBOR abbreviation. Each element of the array specifies one requested configuration parameter of the current topic resource (see ). On success, the server returns a 2.05 (Content) response with a representation of the topic resource. The response has as payload the partial representation of the topic resource as specified in . If a topic manager (TBD) is present in the broker, retrieving topic information may require manager approval subject to certain conditions (TBD). If the conditions are not fulfilled, the manager MUST respond with a 4.03 (Forbidden) error. The response payload is a CBOR map, whose possible entries are specified in and use the same abbreviations defined in . Both request and response MUST have Content-Format set to "application/core-pubsub+cbor". Example: 0.05 FETCH Uri-Path: ps Uri-Path: tc Uri-Path: h9392 Content-Format: TBD2 (application/core-pubsub+cbor) { "conf_filter" : [topic_data, media_type] } <= 2.05 Content Content-Format: TBD2 (application/core-pubsub+cbor) { "topic_data" : "coap://[2001:db8::2]/ps/data/6578616d706c65", "media_type": "application/senml-cbor" } ]]>

Updating the Topic Resource A client can update the configuration of a topic by submitting the representation of the updated topic (see Section 3.1.3) in a PUT or POST request to the topic URI. Any existing properties in the configuration are overwritten by this update. On success, the server returns a 2.04 (Changed) response and the current full resource representation. The broker may chose not to overwrite parameters that are not explicitly modified in the request. Example: 0.03 PUT Uri-Path: ps Uri-Path: tc Uri-Path: h9392 Content-Format: TBD2 (application/core-pubsub+cbor) { "topic_name" : "living_room_sensor", "topic_data" : "coap://[2001:db8::2]/ps/data/6578616d706c65", "target_attribute": "temperature", "expiration_date": "2023-04-28T23:59:59Z", "max_subscribers": 2 } <= 2.04 Changed Content-Format: TBD2 (application/core-pubsub+cbor) TBD (this should be a CBOR map) { "topic_name" : "living_room_sensor", "topic_data" : "coap://[2001:db8::2]/ps/data/6578616d706c65", "resource_type": "core.ps.conf", "media_type": "application/senml-cbor", "target_attribute": "temperature", "expiration_date": "2023-04-28T23:59:59Z", "max_subscribers": 2 } ]]>

Deleting a Topic Resource A client can delete a topic by making a CoAP DELETE request on the topic resource URI. On success, the server returns a 2.02 (Deleted) response. When a topic resource is deleted, the broker SHOULD also delete the topic data resource, unsubscribe all subscribers by removing them from the list of observers and returning a final 4.04 (Not Found) response as per Section 3.2. Example: 0.04 DELETE Uri-Path: ps Uri-Path: tc Uri-Path: h9392 <= 2.02 Deleted ]]>

Publish/Subscribe Unless a topic is configured to use a different mechanism, publish/ subscribe is performed as follows: A publisher publishes to a topic by submitting the data in a PUT request to a broker-managed "topic data resource". This causes a change to the state of that resources. Any subscriber observing the resource at that time receives a notification about the change to the resource state. Observations are maintained and terminated as specified in . As shown in , each topic contains two resources: topic resource and topic data. In that section we explained the creation and configuration of the topic resources. This section will explain the management of topic data resources. A topic data resource does not exist until some initial data has been published to it. Before initial data has been published, the topic data resource yields a 4.04 (Not Found) response. If such a "half created" topic is undesired, the creator of the topic can simply immediately publish some initial placeholder data to make the topic "fully created". URIs for topic resources are broker-generated. URIs for topic-data MAY also be broker-generated or client-generated. There does not need to be any URI pattern dependence between the URI where the data exists and the URI of the topic resource. Topic resource and data resources might even be hosted on different servers.

Topic Lifecycle When a topic is newly created, it is first placed by the server into the HALF CREATED state (see ). In this state, a client can read and update the configuration of the topic and delete the topic. A publisher can publish to the topic data resource. However, a subscriber cannot yet observe the topic data resource nor read the latest data.

Lifecycle of a Topic | | <------------------ | | ------------. Create |___| ------------------> |___| <-----------' \ Publish / Subscribe | ^ \ ___ / | ^ Read/ | | '--> | | <--' | | Read/ Update | | Delete |___| Delete | | Update '-' Topic Topic '-' DELETED ]]> After a publisher publishes to the topic for the first time, the topic is placed into the FULLY CREATED state. In this state, a client can read, update and delete the topic; a publisher can publish to the topic data resource; and a subscriber can observe the topic data resource. When a client deletes a topic resource, the topic is placed into the DELETED state and shortly after removed from the server. In this state, all subscribers are removed from the list of observers of the topic data resource and no further interactions with the topic are possible. When a client deletes a topic data, the topic is placed into the HALF CREATED state, where clients can read, update and delete the topic and awaits for a publisher to begin publication.

Rate Limiting The server hosting a data resource may have to handle a potentially very large number of publishers and subscribers at the same time. This means the server can easily become overwhelmed if it receives too many publications in a short period of time. In this situation, if a client is sending publications too fast, the server SHOULD return a 4.29 (Too Many Requests) response . As described in , the Max-Age option in this response indicates the number of seconds after which the client may retry. The Broker MAY stop publishing messages from the client for the indicated time. When a client receives a 4.29 (Too Many Requests) response, it MUST NOT send any new publication requests to the same topic data resource before the time indicated by the Max-Age option has passed.

Topic Data Interactions TBD: intro and image that shows a topic data URI hosted in a different endpoint than the broker

Publish A topic must have been created in order to publish data to it (See Section ) and be in the half-created state in order to the publish operation to work (see ). A client can publish data to a topic by submitting the data in a PUT request to the 'topic_data' URI as indicated in its topic resource property. Please note that the 'topic_data' URI is not the same as the topic URI used for configuring the topic (see ). On success, the server returns a 2.04 (Updated) response. However, when data is published to the topic for the first time, the server instead MUST return a 2.01 (Created) response and set the topic in the fully-created state (see ). If the request does not have an acceptable content format, the server returns a 4.15 (Unsupported Content Format) response. If the client is sending publications too fast, the server returns a 4.29 (Too Many Requests) response . Example of first publication: 0.03 PUT Uri-Path: ps Uri-Path: data Uri-Path: 6578616d706c65 Content-Format: 110 { "n": "temperature", "u": "Cel", "t": 1621452122, "v": 23.5 } <= 2.01 Created ]]> Example of subsequent publication: 0.03 PUT Uri-Path: ps Uri-Path: data Uri-Path: 6578616d706c65 Content-Format: 110 { "n": "temperature", "u": "Cel", "t": 182734122, "v": 22.5 } <= 2.04 Updated ]]>

Subscribe A client can subscribe to a topic by sending a CoAP GET request with the Observe set to '0' to subscribe to resource updates. . On success, the broker MUST return 2.05 (Content) notifications with the data. If the topic is not yet in the fully created state (see ) the broker SHOULD return a response code 4.04 (Not Found). The following response codes are defined for the Subscribe operation:

Success:: 2.05 "Content". Successful subscribe with observe response, current value included in the response.
Failure:: 4.04 "Not Found". Topic does not exist.

TBD: Do we want to treat max_clients as an error? If the 'max_clients' parameter has been reached, the server must treat that as specified in section 4.1 of . The response MUST NOT include an Observe Option, the absence of which signals to the subscriber that the subscription failed. Example: 0.01 GET Uri-Path: ps Uri-Path: data Uri-Path: 6578616d706c65 Observe: 0 <= 2.05 Content Content-Format: 110 Observe: 10001 Max-Age: 15 [...SenML data...] <= 2.05 Content Content-Format: 110 Observe: 10002 Max-Age: 15 [...SenML data...] <= 2.05 Content Content-Format: 110 Observe: 10003 Max-Age: 15 [...SenML data...] ]]>

Unsubscribe A CoAP client can unsubscribe simply by cancelling the observation as described in Section 3.6 of . The client MUST either use CoAP GET with the Observe Option set to 1 or send a CoAP Reset message in response to a notification.

Delete topic data resource A publisher MAY delete a topic by making a CoAP DELETE request on the 'topic_data' URI. On success, the server returns a 2.02 (Deleted) response. When a topic_data resource is deleted, the broker SHOULD also delete the 'topic_data' parameter in the topic resource, unsubscribe all subscribers by removing them from the list of observers and return a final 4.04 (Not Found) response as per Section 3.2. The topic is then set back to the half created state as per . Example: 0.04 DELETE Uri-Path: ps Uri-Path: data Uri-Path: 6578616d706c65 <= 2.02 Deleted ]]>

Read latest data A client can get the latest published topic data by making a GET request to the 'topic_data' URI in the broker. Please note that discovery of the 'topic_data' parameter is a required previous step (see ). On success, the server MUST return 2.05 (Content) response with the data. If the target URI does not match an existing resource or the topic is not in the fully created state (see ), the broker SHOULD return a response code 4.04 (Not Found). If the broker can not return the requested content format it MUST return a response code 4.15 (Unsupported Content Format). Example: 0.01 GET Uri-Path: ps Uri-Path: data Uri-Path: 6578616d706c65 <= 2.05 Content Content-Format: 110 Max-Age: 15 { "n": "temperature", "u": "Cel", "t": 1621452122, "v": 23.5 } ]]>