Logo Computer scientist,
engineer, and educator
• Articles • Articles about computing • Articles about software development • Apache integration software

Writing an MQTT client C for ActiveMQ from the ground up

This article explains how to use the client runtime provided by the Mosquitto MQTT library to communicate with ActiveMQ, or with any other MQTT-aware message broker, using a C program. An alternative strategy for interfacing C code with ActiveMQ is to use the CMS library (see this article). Using MQTT has several advantages, however, not least of which is that the Mosquitto MQTT library is available pre-built for many platforms; even if you have to build it from source, it's a lot easier and quicker than builing CMS.

This article assumes a reasonable knowledge of C programming, and a familiarity with using command-line tools to build and run code. As ever, the use of nothing more than the command-line in this article is intended to ensure that no details are concealed. Some knowledge of JMS will be helpful, because I frequently compared MQTT features with their equivalent JMS features.

About MQTT

MQTT is a lightweight, simple protocol for publish-subscribe messaging. Its main application is asynchronous inter-process communication, although its low overheads also make it practicable for decoupling software subsystems in the same process.

MQTT is popular in telementry and machine-to-machine applications, because support for it can (with care) add only a few Mb to an application's memory footprint. An MQTT message broker implemented in C, such as Mosquitto, might use only 10Mb or so. This compares very favourably indeed with Java-based message brokers. Of course, memory usage — of any broker — depends to some extent on client load, and on the kind of message persistence that is demanded by the application.

As a protocol, MQTT offers a small subset of the features supported by most Java-based brokers. The protocol offers no support for batches or transactions, very limited support for managing a message backlog (that is, queueing), a very limited notion of non-volatile messaging, and many brokers do not support authentication (most modern ones do). The MQTT payload format is unspecified; this is very different from JMS, where the application must specify the payload content (text, bytes, map.) An MQTT message is simply a variable-length byte array. So, whilst a Java-based message broker might support MQTT at the protocol level, it isn't safe to assume that Java-based clients of that broker will interoperate with C/C++ MQTT clients — this really depends on the kinds of messages that are used, and the way that data is encoded. See the section "C/Java interoperability" below.

The Apache CMS library offers a more complete, C-based messaging runtime, which is broadly API-compatible with JMS (provided the underlying protocol has the necessary capabilities.) CMS, however, is a large library — at least a 20Mb memory footprint — MQTT is adequate for a wide range of applications where CMS would be impractical.

Obtaining the Mosquitto MQTT client runtime (and server)

Obtaining a pre-built library

For Linux, you might be able to obtain the Mosquitto client runtime, and the server if necessary, from a standard repository. On Fedora, for example, I can do:
# yum install mosquito-devel
In fact this installs the server as well, but you're not forced to use it.

Be aware that if you install from a repository, you will be limited to the library features that were felt necessary by the package maintainer. So if, for example, you need SSL support and it was not included, it probably won't be possible to add it. Conversely, if you don't want the overhead introduced by including SSL support, you won't be able to take it out. For the record, the Fedora version of Mosquitto does not have SSL support built in.

Building from source

If Mosquitto is not available for your distribution, or you are running something other than Linux, or you need to enable or disable particular features, you will probably need to build from source. The source for the Mosquitto server and client runtime (both in one bundle) can be obtained from the Mosquitto website.

Mosquitto has certain dependencies, most of which are likely to be present on the build host already (particularly if it is Linux.) A notable exception is C-ares, which is a library for asynchronous DNS operations. This feature, along with a number of others that might be of limited application, can be disabled by editing config.mk before building. One setting you will probably want to disable is WITH_UUID. This functionality is only used for improving the randomness of client ID generation, and enabling it imposes a whole bunch of other dependencies, many of which will be irrelevant to messaging applications.

By default, the build process will install files in /usr/local, which won't suit everybody. This, again, can be changed in config.mk — look for the line "prefix=/usr/local".

After editing the configuration file, building and installation should be as simple as:

$ make
# make install 

Note that Mosquitto comes with very versatble MQTT test clients — mosquitto_pub and mostquitto_sub, for publishing and subscribing respectively. These work with MQTT on ActiveMQ, and might be useful for testing if your own code seems not to.

Enabling MQTT support in ActiveMQ

This should be straightforward — it's just a matter of enabling the protocol. In activemq.xml, add this entry to transportConnectors:
  <transportConnector name="mqtt" uri="mqtt://${bindAddress}:1883"/>
You aren't forced to use port 1883, but this is the conventional one. ActiveMQ takes care of mapping MQTT operations to JMS operations, so far as that is possible. There is no support in MQTT for a lot of what JMS provides (transactionality, for example), or for most of the extensions to JMS offered by ActiveMQ (such as message grouping). However, if you've already determined that your application's messaging needs can be met by MQTT, that won't be a problem.

Using the Mosquitto MQTT client API

Unlike JMS, there is no standards-based API for MQTT. Different library vendors provide different APIs. This undoubtedly creates a problem with portability — moving from one client library to another is likely to be fiddly. However, because MQTT itself is so straightforward, this is unlikely to be conceptually difficult, merely time-consuming and tiresome. The conceptual simplicity does mean that a substantial application can wrap the specific MQTT implementation, and provide a vendor-neutral messaging interface to the rest of the application.

THe Mosquitto client API is intended for use in C applications, and is not object-oriented. Ordinary C primitives are used for data structures. Text strings, for example, are represented as a traditional char[]. Where character encoding is relevant — and it nearly always is these days — these strings are interpreted as UTF-8 encoded (this is a stipulation of the protocol, not the library). The message payload is not an object of any kind, merely a byte array. It is up to the application to interpret the payload as it sees fit, including handling encoding transformation of text messages where necessary.

The example below shows a simple example — posting a small number of text (char[]) messages to a single topic. I hope the comments are clear enough to explain what is going on. A few notes follow the listing.

#include <errno.h>
#include <fcntl.h>
#include <stdio.h>
#include <unistd.h>
#include <stdlib.h>
#include <string.h>
#include <mosquitto.h>

// Server connection parameters
#define MQTT_HOSTNAME "localhost"
#define MQTT_PORT 1883
#define MQTT_USERNAME "admin"
#define MQTT_PASSWORD "admin"
#define MQTT_TOPIC "test"

/*
 * Start here
 */
int main (int argc, char **argv)
  {
  struct mosquitto *mosq = NULL;

  // Initialize the Mosquitto library
  mosquitto_lib_init();

  // Create a new Mosquitto runtime instance with a random client ID,
  //  and no application-specific callback data.  
  mosq = mosquitto_new (NULL, true, NULL);
  if (!mosq)
    {
    fprintf (stderr, "Can't initialize Mosquitto library\n");
    exit (-1);
    }

  mosquitto_username_pw_set (mosq, MQTT_USERNAME, MQTT_PASSWORD);

  // Establish a connection to the MQTT server. Do not use a keep-alive ping
  int ret = mosquitto_connect (mosq, MQTT_HOSTNAME, MQTT_PORT, 0);
  if (ret)
    {
    fprintf (stderr, "Can't connect to Mosquitto server\n");
    exit (-1);
    }

  int i;
  char text[20];
  for (i = 0; i < 10; i++)
    {
    sprintf (text, "Hello, World %d", i);
    // Publish the message to the topic
    ret = mosquitto_publish (mosq, NULL, MQTT_TOPIC,
      strlen (text), text, 0, false);
    if (ret)
      {
      fprintf (stderr, "Can't publish to Mosquitto server\n");
      exit (-1);
      }
    }

  // We need a short delay here, to prevent the Mosquitto library being
  //  torn down by the operating system before all the network operations
  //  are finished.
  sleep (1);

  // Tidy up
  mosquitto_disconnect (mosq);
  mosquitto_destroy (mosq);
  mosquitto_lib_cleanup();

  return 0;
  }
A few things to note about this example:
  • Passing a NULL value for the username to mosquitto_username_pw_set() disables authentication. However, ommiting the call completely will have the same effect. Authentication of any kind is a recent addition to the MQTT protocol, and some earlier brokers may fail to work at all if presented with authentication data. Note that MQTT does not seem to treat failed authentication as an error condition — messages are silently dropped.
  • MQTT supports a rudimentary notion of quality-of-service (QoS), and some of the Mosquitto APIs take a single integer argument to specify the QoS level. In this example, a QoS value of zero has arbitrarily been used. It is unlikely that ActiveMQ would interpret the QoS argument in a meaningful way (in fact, message prioritization is a hard problem in any message broker that tries to offer high throughput using consumer pre-fetch.)
  • A text string supplied as a payload need not be null-terminated, and the terminating null will not be stored in this example, because the return value of strlen() does not count the null. However, the API calls that retrieve messages appear tacitly to put a null on the end of the message payload in memory, whatever the data type. This behaviour should probably not be relied on in a real application, however — nothing in the documentation suggests that this is deliberate.
  • The final argument to mosquitto_publish() indicates whether the message should be retained. A retained message is delivered to a subscriber as soon as it connects; a message that is not marked as retained will only be delivered to clients when they are actually connected. Marking a message as retained does not imply any form of persistence — only one message is retained, per topic. However, MQTT does have the notion of a persistent session, very much like a durable subscription in JMS messaging. With a persistent session, certainly classes of messages are stored when a subscriber is off-line; this storage may be persistent, depending on the application.
  • Support for persistent sessions, among other things, requires that each client have a unique ID. This ID can be passed to mosquitto_new(), but the library will randomly generate one if none is supplied (and persistent sessions are not required.)
  • The Mosquitto library supports a whole range of callback functions, to indicate the status or progress of operations. No callbacks are used in this example; but a message consumer will typically implement at least one: a callback that is invoked when new data is published. Such a callback is presented in the code sample below.
The code needed to subscribe and consume is mostly the same as that for publishing. The example below shows the salient features (the full source is in the download bundle).
/*
 * my_message_callback. 
 * Called whenever a new message arrives
 */
void my_message_callback(struct mosquitto *mosq, void *obj,
    const struct mosquitto_message *message)
  {
  // Note: nothing in the Mosquitto docs or examples suggests that we
  //  must free this message structure after processing it,
  //  (nor that we can assume that it is null-terminated.)
  printf ("Got message: %s\n", (char *)message->payload);
  }

  //...

  mosquitto_subscribe (mosq, NULL, MQTT_TOPIC, 0);

  // Specify the function to call when a new message is received
  mosquitto_message_callback_set (mosq, my_message_callback);

  // Wait for new messages
  mosquitto_loop_forever (mosq, -1, 1);

Compiling and running the example code

The full code is available from the Download section at the end of this article. Naturally, you will need to change the connection parameters to match those of your server before building, and you might need to change the authentication paramters, or remove them altogether if you are using an old MQTT broker that does not support authentication.

The download bundle contains a Makefile to build the samples — one (test-pub.c) publishes ten messages, and another (test-sub.c) subscribes and consumes them.

The command lines, however, are very simple:

 
$ gcc -s -Wall -o test-pub test-pub.c -lmosquitto -lssl -lcrypto -pthread -lcares
If you disabled SSL and asynchronous DNS support when you built the Mosquitto library (or they were not included by the package maintainer), you need not specify -lssl, -lcrypto, or -lcares.

If your Mosquitto library — typically libmosquitto.so on Linux — is not in one of the standard, system locations, you can use the -L switch to gcc to indicate the directory where it can be found.

To run the example, just start the message broker, and execute

$ ./test-sub
in one console session and
$ ./test-pub
in another. You should see the subscriber emit some "Hello, World!" messages whenever the publisher executes.

C/Java interoperability

I have alluded several times to the fact that the MQTT message payload is simply a byte array. If your application uses only MQTT, and only C or C++ clients, then the handling of the payload is entirely a matter for the developer. The client library will not, and the broker should not, tamper with the payloads in flight.

With a mixture of C and Java clients, however, more care is required. Of the JMS message types available, MapMessage and ObjectMessage are likely to be of no relevance to a C program. BytesMessage maps nicely onto the byte array that MQTT imposes, so that is also unlikely to be a problem. The fiddly one is the TextMessage. It's extremely common in messaging applications to pass XML payloads in a TextMessage, and then the vexed question of character encoding arises.

The MQTT library will simply return to its application the message payload the broker gives it. Thus it is up to the broker to do something useful with character encoding, if an MQTT client consumes a Java-based TextMessage. The JVM has its own internal representation of text strings, but this is not normally exposed. Typically, when a Java application wants to manipulate the actual bytes that make up a string, it will ask the JVM to supply them with some particular encoding (UTF-8, ASCII, etc).

Whatever format ActiveMQ stores Java strings in internally, it supplies them to MQTT clients encoded according to the platform encoding. On Linux this is nearly always UTF-8 these days. In practice this means that a client that uses the Mosquitto library can generally assume that any text messages it receives have UTF-8 encoding. Of course, most of the first 127 characters are the same as ASCII in this encoding.

The GNU standard C library has no built-in functions (of which I am aware) for manipulating UTF-8 strings, or for converting them to any other representation. However, there are a number of other libraries that can do such conversions if necessary.

For message brokers other than ActiveMQ, that have a Java API, the way in which a Java TextMessage will be supplied to an MQTT client depends on the implementation and, as this kind of fine detail is rarely documented, some trial-and-error will likely be required to find out.

Conclusion

MQTT is a simple, fast alternative to more elaborate messaging protocols and libraries that is suitable for some applications. This article has explained how to implement MQTT clients in C using the Mosquitto library. MQTT is not intended to be complex, but it does have a number of important features that are not demonstrated by my simple examples, such as persistent sessions and wills (yes, in the sense of last-will-and-testament.) See the Mosquitto documentation page form more information on these topics.

Downloads

C source code bundle
Copyright © 1994-2015 Kevin Boone. Updated Feb 04 2016