Best practices to design APIs with AMQP

As we all know, there's this hype around microservices that we cannot ignore. It is true, developing microservices help us decouple our systems, test and work with them better.
Besides HTTP, you can create APIs with tons of other protocols and content types. Here is where I tell you a little experience about creating APIs with AMQP, using RabbitMQ (or any other message broker that supports AMQP 0.9.1).
First, there are a few concepts that you must understand, before we continue to create our apis, but, if you feel confortable with AMQP already, you should easily skip the next main topic.

Basic AMQP concepts

Nomenclature

Virtual Hosts, Exchanges, Queues and Routing Keys should be written in lower case and words must be separated by dots. For instance:

colors
blue
dark.blue
green.dark.blue

Virtual Hosts

Virtual Hosts are used to isolate resources. A determined resource (e.g. exchanges, queues) can be used by more than one virtual host.

Routing keys

Routing keys are strings informed by the producer and used by exchanges to redirect messages to queues. Not all exchanges use routing key to route the messages.

Exchanges

Exchanges receive and redirect messages to queues. One exchange does not keep messages, it just distributes them.

Type of exchanges

Direct: A direct exchange redirects a received message to a queue based on its routing key. The sent routing key must equals the binding routing key.

direct

Fanout: Fanout exchanges redirect all the messages to all binded queues.

fanout

Topic: Topic exchanges work almost like direct exchanges, but, you can apply simple patterns to match a binding.

* substitutes exactly one word.
# substitutes zero or more words

topic

For the example above, the following patterns would match to Queue 3:

dark.red
light.blue
dark.blue
dark.green.blue

But, those would not:

light.red
dark.green.red

Header: Header exchanges works like direct exchanges, but, rather than using a routing key to match bindings, it uses the headers from the sent message.

headers

Queues

A queue stores and sends messages to consumers. You should never send a message directly to a queue, you should always send the message to an exchange so then it can redirect to the right(s) queue(s).

Bindings

You can either bind exchanges to queues or other exchanges. If the exchange is not fanout, the binding will use routing keys or headers, otherwise there's no need for routing informations.

Consumers

Consumers listen to queues and receive messages from them.

Acknowledgement

You can define either auto or manual message acknowledgement. If it is true, as soon as a consumer receives a message, the broker will mark it as consumed, otherwise, if it is manual, it will wait until you mark the message as acknowledged or not.
Using manual mode you can either sent a positive or a negative acknowledgement.

Designing APIs

If you fully understood the above concepts, we can go futher: designing APIs with AMQP, based on all those concepts that will already know.

Scenario

Ok, we need to define what we need to do here, so, here is a simple scenario for us to work with: credit card transacions.

We will create three services: create, cancel and fetch a transaction.

Besides that, we also need to keep the events from the services that create side-effects, such as create and cancel.

Exchanges as domain models

Since the scenario is very simple, the only domain model we will have is the transaction domain, so, that's the exchange we will create.
Also, for the sake of the exemple, let's keep it simple and have a direct exchange. This type of exchange will fit perfectly in this case.

Queues as actions

We need to: create, cancel or fetch a transaction, so, we can easily say that each one of them is a different queue:

create.transaction
cancel.transaction
fetch.transaction

You probably noticed that, that's not the only actions that we will have. There are one more: save events, so, there's another queue:

save.transaction.side.effect.event

Ok, there will be four queues then.

But, since sending a message is asynchronous, how will fetch.transaction work?

You probably will send a message with a 'X-Reply-To' header. This way you will provide a routing key for the 'fetch.transaction' queue consumer reply if there's a transaction or not for you to consume.
If you don't want to, there's no problem in creating HTTP services to fetch resources instead of creating AMQP services.

Routing key as operations

There are three operations: create, cancel and fetch. Each one of them will be a routing key.

Bindings

Here is where everything comes together:

If we
Send a message with a routing key 'create',
We must send the message to: create.transaction and save.transaction.side.effect.event queues.
So 'create' routing key binds exchange transaction to create.transaction and save.transaction.side.effect.event queues.

If we
Send a message with a routing key 'cancel',
We must send the message to: cancel.transaction and save.transaction.side.effect.event queues.
So 'cancel' routing key binds exchange transaction to cancel.transaction and save.transaction.side.effect.event queues.

If we
Send a message with a routing key 'fetch',
We must send the message to: fetch.transaction queue.
So 'fetch' routing key binds exchange transaction to fetch.transaction queue.

The table below shows the above bindings in a cleaner way:

Exchange Routing Key Queues
transaction create create.transaction, save.transaction.side.effect.event
transaction cancel cancel.transaction, save.transaction.side.effect.event
transaction fetch fetch.transaction

So, do we forget about HTTP?

Nop. I wouldn't recommend external clients to integrate with your APIs through AMQP. We cannot deny, HTTP is much simpler and more well known than AMQP.
Also, "fetch" endpoints are harder to create using AMQP, since it is an async protocol and you will need to create more "callback" consumers on the fly, it is a common choice not to use AMQP but HTTP in this case.
As much as I hate to say this, I have to: there's no silver bullet. Besides, AMQP and HTTP can do an amazing job together, so, that's where I leave it to you to decide where to use which one.

Finalizing

I hope this could be helpful to you. I with I could learn the easy way how to design AMQP apis, but, that was a bit painful, at least at first. =P

Any questions or opinions, please, leave a comment.

[]'s

Gabriel Francisco

Software Engineer at 99, 24 years, under graduated in Computer Science and graduated in Service-oriented Software Engineering. Like playing guitar once in a while. Oh, and I'm kind of boring.

São Paulo

comments powered by Disqus