Talk CoAP to me – IoT over Bluetooth Low Energy

In this article, I will describe the setup of an Android app and a sensor with a focus on the Bluetooth communication between the two devices. We chose to use the Constrained Application Protocol (CoAP) as the communication protocol. CoAP was designed to be a perfect match for IoT use-cases as it has a much smaller overhead in comparison to other protocols such as HTTP. However, CoAP was not designed with Bluetooth Low Energy (BLE) in mind. To implement the setup of our Android app, we had to find solutions for problems such as a limited packet size of BLE and limitations when working with streams of data. I will outline our approach in the first part of this article and then provide some actual examples of the implementation for CoAP over BLE.

A sensor is connected to a gateway application over Bluetooth Low Energy

The Constrained Application Protocol

Before diving into the details of the Android implementation, let us first get an understanding of the Constrained Application Protocol.

In 2014, the IETF published RFC 7252 describing a communication protocol made for the Internet of things:

“The Constrained Application Protocol (CoAP) is a specialized web transfer protocol for use with constrained nodes and constrained networks in the Internet of Things. The protocol is designed for machine-to-machine (M2M) applications such as smart energy and building automation.”

The overarching design paradigm for CoAP is simplicity. It was designed to run on devices with very little computing power (it can be implemented on sensors with as little as 10KiB of RAM). Most implementations of CoAP use UDP as a transport protocol as it adds only a small overhead (compared to e.g. TCP).

The basic concept of CoAP is very similar to the well know and widely used REST over HTTP model. Similar to REST, a CoAP client can access resources on a CoAP server using methods such as GET, POST, PUT or DELETE. As such, CoAP can be easily mapped to HTTP and vice-versa.

For a more detailed look on the specification of CoAP, I recommend checking out the following resources:

• coap.technology/
• tools.ietf.org/html/rfc7252
• Video at droidcon.com

Since the introduction of CoAP, quite a large number of implementations was published for a variety of languages. You can find an overview of popular libraries here.

CoAP implementations exist for many languages

The large majority of these implementations have one thing in common: they implement the CoAP as a protocol on top of UDP (and in some cases, TCP).

This brings us to the main topic of this article – we want to communicate with a sensor (= a CoAP server) over Bluetooth Low Energy. What changes in comparison to a usage over UDP?

CoAP over Bluetooth Low Energy (BLE)

In order to understand the difficulties of adapting CoAP to Bluetooth, one must understand some of the basics of Bluetooth Low Energy.

Two important terms when working with Bluetooth Low Energy are ATT and GATT. ATT, the Attribute Protocol, is a very low-level mechanism that defines how to transfer a unit of data. A unit of data is called an Attribute. On top of ATT, an additional layer of abstraction is defined, called GATT, the Generic Attribute Profile. GATT introduces the concepts of Profiles, Services and Characteristics that are well known to any developer who has worked with BLE. Typically, a BLE client discovers the services of a BLE server and then reads from or writes to specific characteristics.

GATT defines Profiles, Services, and Characteristics

The BLE standard defines many profiles for common use cases, such as a Heart Rate Profile tailored to fitness sensors.

With these concepts in mind, we can have a detailed look at the two challenges we faced while implementing CoAP over BLE.

Challenge #1 – Packet Size

As mentioned in the intro to CoAP, the RFC specification is focused on CoAP over UDP. For example, this becomes apparent in the section specifying the payload length: “The payload data extends from after the marker to the end of the UDP datagram” [1]. Looking at the specification of UDP, we quickly discover that one UDP packet can have a size of up to 65535 bytes. On the other hand, the maximum size of an ATT packet (and thus of the data unit behind GATT) is 512 bytes. Even more, in Android the ATT packet size (called MTU) is by default set to 23 bytes.

UDP packets can be much larger than GATT packets

What are the consequences of these size limits? If we wanted to send or receive a CoAP message with a payload greater than 512 bytes, we’d have to manually split up the message and take care of the transmission of multiple parts.

Other open questions come to mind: How can we map CoAP requests to GATT characteristics? How to structure the GATT services? Is there one characteristic for each CoAP endpoint?

Luckily, there is a solution that circumvents these questions: Instead of implementing a complex application-side logic, we use a special kind of GATT Service called UART Service. This service emulates a serial interface and offers two characteristics, one for reading and on for writing (Rx and Tx). These two characteristics can be used to transfer a stream of data without size restrictions. When writing to the server’s receiving characteristic Rx, our CoAP message is automatically split (by Android) into small packets with a payload of 20 bytes. The CoAP server can simply read from the received stream of small packets and reconstruct a complete CoAP message.

The UART service works like a stream of small chunks

Challenge #2 – Delimiting packets in a stream

By using the stream-like serial interface of the UART service, we avoid a lot of work on application-side logic. However, with this decision, a new challenge appears: How can the receiver of the stream reconstruct the containing CoAP messages? After all, we send small chunks of messages on a stream of bytes, so one layer below the actual abstraction of CoAP.

The answer is as simple as it is old: We apply the “Serial Line Internet Protocol” (or SLIP) before writing the resulting bytes to the UART stream. SLIP was published in RFC 1055 [2] in 1988 as an easy and lightweight way of encapsulating IP packets. The algorithm is extremely simple: appending a special END byte (0xC0) to each message allows the receiver to separate incoming bytes by searching for the occurrence of this special byte. Three other bytes are used to escape the END byte if it occurs in the regular payload. Thus, SLIP can be summarized as follows:

• Append the special “END” byte to each message
• If the END byte occurs in the data to be sent, the two-byte sequence ESC and ESC_END is sent instead
• If the ESC byte occurs in the data, the two-byte sequence ESC and ESC_ESC is sent

A special END byte delimits single messages

This algorithm is an easy and fast-to-implement way of separating our CoAP messages in a stream of bytes.

fun encode(data: ByteArray): ByteArray {

    val encoded = ByteArrayOutputStream()

 

    data.forEach { byte ->

        when (byte.toInt()) {

            0xC0 -> {

                encoded.write(0xDB)

                encoded.write(0xDC)

            }

            0xDB -> {

                encoded.write(0xDB)

                encoded.write(0xDD)

            }

            else -> encoded.write(byte.toInt())

        }

    }

 

    encoded.write(0xC0)

    return encoded.toByteArray()

}

Putting it together

Let’s take a look at how the described steps and pieces fall into place.

CoAP over Bluetooth Low Energy, separated into a CoAP layer and a BLE layer

First off, we decided to use a UART service implementation to send and receive bytes between the two devices. This circumvents the question of how to map CoAP endpoints to specific BLE characteristics and allows us to send CoAP messages of arbitrary size. In order to delimit single CoAP messages in the UART stream, we apply the SLIP algorithm which appends a special END byte to each message.

From the perspective of the Android app, each message thus goes through the same steps: First, a CoAP request message is constructed with the required headers, options, payload, and token. The token is saved in conjunction with a RequestType in order to know how to parse the payload of incoming CoAP messages.

/** CoAP layer - construct token and CoAP message **/
fun requestTemperature() {
    val token = generateToken()
    sendCoapRequest(
        coapMessageFactory.createGetRequest("device/temperature", token)
    )
    pendingRequests[token] = RequestType.TEMPERATURE
}

The CoAP message object is then converted to a Byte Array, which is encoded by the SLIP algorithm. The resulting Byte Array is written to the UART Rx characteristic of the remote device. Android takes care of splitting the message into smaller chunks.

/** BLE layer - encode and write message **/
fun sendCoapRequest(coapMessage: CoapMessage) {
    coapMessage.toByteArray()?.let { rawData->
        val encodedMessage = slipAdapter.encode(rawData)
        writeCharacteristic(uartRxCharacteristic, encodedMessage)
    }
}

When receiving new data, small chunks of bytes arrive in our app at the subscriber for the Tx characteristic of the remote device (in onReceive()). The new data is appended to an internal buffer. With each new update, the SLIP algorithm searches for the occurrence of the END byte in this buffer and splits the stream into complete messages. As a last step, we clear the completed messages from the buffer and keep only the remaining, still incomplete bytes.

/** BLE layer - receive and decode bytes **/

var buffer = ByteArray(0)

fun onReceive(newData: ByteArray?) {

    newData?.let { data ->

        buffer += data

        val decoded: DecodeResult = slipAdapter.decode(buffer)

        scope.launch {

            decoded.messages.forEach { m ->

                responseChannel.send(m)

            }

        }

        buffer = buffer.copyOfRange(result.endIndex + 1, buffer.size)

    }

}

Each complete message is parsed as a CoAP message. The app then checks the received CoAP token to determine if the received message is an expected response. If this check succeeds, the payload is further processed by the business components of the app (in handleExpectedMessage()).

/** CoAP layer - parse and handle CoAP message **/
fun handleMessage(data: ByteArray) {
    val coapMessage = coapMessageFactory.fromByteArray(data)
    coapMessage?.let { message ->
        val token = message.token
        val requestType = pendingRequests[token]
        return if (requestType == null) {
            handleUnexpectedMessage(message)
        } else {
            handleExpectedMessage(requestType, message, token)
        }
    } ?: logger.error("Unable to parse CoAP message")
}

Since our initial implementation of CoAP over BLE a few months ago, the solution has proven to be easy to maintain and extend.

Although developers initially need to learn how to work with CoAP, this is quickly accomplished due to the similarity to REST over HTTP. In my opinion, relying on an established standard protocol such as CoAP has many advantages over rolling your own BLE communication protocol.

Have you worked with CoAP? What are your thoughts and what were your challenges?

References

[1] https://tools.ietf.org/html/rfc7252#section-3

[2] https://tools.ietf.org/html/rfc1055

---

About the author