yhdev/dat/doc-easyipc

=pod

(Published on B<2014-07-29>.)

=head1 The Problem

How do you implement communication between two or more processes? This is a
question that has been haunting me for at least 6 years now.  Of course, this
question is very broad and has many possible answers, depending on your
scenario.  So let me get more specific by describing the problem I want to
solve.

What I want is to write a daemon process that runs in the background and can be
controlled from other programs or libraries. The intention is that people can
easily write custom interfaces or quick scripts to control the daemon.  The
service that the daemon offers over this communication channel can be thought
of as its primary API, in this way you can think of the daemon as a persistent
programming library. This concept is similar to existing programs such as
L<btpd|https://github.com/btpd/btpd>, L<MPD|http://www.musicpd.org/>,
L<Transmission|https://www.transmissionbt.com/> and
L<Telepathy|http://telepathy.freedesktop.org/wiki/> - I'll get back to these
later. 

More specifically, the most recent project I've been working on that follows
this pattern is L<Globster|https://dev.yorhel.nl/globster>, a remotely
controllable Direct Connect client (if you're not familiar with Direct Connect,
think of it as IRC with some additional file sharing capabilities built in).
While the problem I describe is not specific to Globster, it still serves as an
important use case. I see many other projects with similar IPC requirements.

The IPC mechanism should support two messaging patterns: Request/response and
asynchronous notifications. The request/response pattern is what you typically
get in RPC systems - the client requests something of the daemon and the daemon
then replies with a response. Asynchronous notifications are useful in allowing
the daemon to send asynchronous status updates to the client, such as incoming
chat messages or file transfer status. Lack of support for such notifications
would mean that a client needs to continuously poll for updates, which is
inefficient.

So what I'm looking for is a high-level IPC mechanism that handles this
communication.  Solutions are evaluated by the following criteria, in no
particular order.

=over

=item B<Easy>

And with I<easy> I refer to I<ease of use>. As mentioned above, other people
should be able to write applications and scripts to control the daemon. Not
many people are willing to invest days of work just to figure out how to
communicate with the daemon.

=item B<Simple>

Simplicity refers to the actual protocol and the complexity of the code
necessary to implement it. Complex protocols require complex code, and complex
code is hard to maintain and will inevitably contain bugs. Note that I<simple>
and I<easy> are very different things and often even conflict with each other.

=item B<Small>

The IPC implementation shouldn't be too large, and shouldn't depend on huge
libraries. If you need several megabytes worth of libraries just to send a few
messages over a socket, you're doing it wrong.

=item B<Language independent>

Control the daemon with whatever programming language you're familiar with.

=item B<Networked>

A good solution should be accessible from both the local system (daemon running
on the same machine as the client) and from the network (daemon and client
running different machines).

=item B<Secure>

There's three parts in having a secure IPC mechanism. One part is to realize
that IPC operates at a I<trust boundary>; The daemon can't blindly trust
everything the client says and vice versa, so message validation and other
mechanisms to prevent DoS or information disclosure on either part are
necessary.

Then there the matter of I<confidentiality>. On a local system, UNIX sockets
will provide all the confidentiality you can get, so that's trivial. Networked
access, on the other hand, requires some form of transport layer security.

And finally, we need some form of I<authentication>. There should be some
mechanism to prevent just about anyone to connect to the daemon. A
coarse-grained solution such as file permissions on a local UNIX socket or a
password-based approach for networked access will do just fine for most
purposes.  Really, just keep it simple.

=item B<Fast>

Although performance isn't really a primary goal, the communication between the
daemon and the clients shouldn't be too slow or heavyweight. For my purposes,
anything that supports about a hundred messages a second on average hardware
will do perfectly fine. And that shouldn't be particularly hard to achieve.

=item B<Proxy support>

This isn't really a hard requirement either, but it would be nice to allow
other processes (say, plugins of the daemon, or clients connecting to the
daemon) to export services over the same IPC channel as the main daemon. This
is especially useful in implementing a cross-language plugin architecture. But
again, not a hard requirement, because even if the IPC mechanism doesn't
directly support proxying, it's always possible for the daemon to implement
some custom APIs to achieve the same effect. This, however, requires extra work
and may not be as elegant as a built-in solution.

=back

Now let's discuss some existing solutions...


=head1 Custom Protocol

Why use an existing IPC mechanism in the first place when all you need is
UNIX/TCP sockets? This is the approach taken by
L<btpd|https://github.com/btpd/btpd>, L<MPD|http://www.musicpd.org/>
(L<protocol spec|http://www.musicpd.org/doc/protocol/index.html>) and older
versions of Transmission (see their L<1.2x
spec|https://trac.transmissionbt.com/browser/branches/1.2x/doc/ipcproto.txt>).
Brpd hasn't taken the time to documented the protocol format, suggesting it's
not really intended to be used as a convenient API (other than through their
btcli), and Transmission has since changed to a different protocol. I'll mainly
focus on MPD here.

MPD uses a text-based request/response mechanism, where each request is a
simple one-line command and a response consists of one or more lines, ending
with an C<OK> or C<ACK> line. There's no support for asynchronous
notifications, although that could obviously have been implemented, too. Let's
grade this protocol...

=over

=item B<Easy?> Not really.

Although MPD has conventions for how messages are formatted, each individual
message still requires custom parsing and validation. This can be automated by
designing an
L<IDL|https://en.wikipedia.org/wiki/Interface_description_language> and
accompanying code generator, but writing one specific for a single project
doesn't seem like a particularly fun task.

The protocol, despite its apparent simplicity, is apparently painful enough to
use that there is a special I<libmpdclient> library to abstract away the
communication with MPD, and interfaces to this library are available in many
programming languages. If you have access to such an application-specific
library for your language of choice, then sure, using the IPC mechanism is easy
enough. But that applies to literally any IPC mechanism.

Ideally, such a library needs to be written only once for the IPC mechanism in
use, and after that no additional code is needed to communicate with
services/daemons using that particular IPC mechanism. Code re-use among
different projects is great, yo. It also doesn't scale very well when extending
the services offered by daemon,  any addition to the API will require
modifications to all implementations.

=item B<Simple?> Definitely.

I only needed a quick glance at the MPD protocol reference and I was able to
play a bit with telnet and control my MPD. Writing an implementation doesn't
seem like a complex task. Of course, this doesn't necessarily apply to all
custom protocols, but you can make it as simple or complex as you want it to
be.

=item B<Small?> Sure.

This obviously depends on how elaborate you design your protocol. If you have a
large or complex API, the size of a generic message parser and validator can
easily compensate for the custom parser and validator needed for each custom
message. But for a simple APIs, it's hard to beat a custom protocol in terms of
size.

=item B<Language independent?> Depends.

Of course, a socket library is available to most programming languages, and in
that sense any IPC mechanism built on sockets is language independent. This is,
as such, more of an argument as to how convenient it is to communicate with the
protocol directly rather than with a library that abstracts the protocol away.
In the case of MPD, the text-based protocol seems easy enough to use directly
from most languages, yet for some reason most people prefer language-specific
libraries for MPD. 

If you design a binary protocol or anything more complex than simple
request/response message types, using your protocol directly is going to be a
pain in certain languages, and people will definitely want a library specific
to your daemon for their favourite programming language. Something you'll want
to avoid, I suppose.

=item B<Networked?> Sure enough.

Just a switch between UNIX sockets and TCP sockets. Whether a simple solution
like that is a good idea, however, depends on the next point...

=item B<Secure?> Ugh.

Security is hard to get right, so having an existing infrastructure that takes
care of most security sensitive features will help a lot. Implementing your own
protocol means that you also have to implement your own security, to some
extent at least.

Writing code to parse and validate custom messages is error-prone, and a bug in
this code could make both the daemon and the client vulnerable to crashes and
buffer overflows. A statically-typed abstraction that handles parsing and
validation would help a lot.

For networked communication, you'll need some form of confidentiality. MPD does
not seem to support this, so any networked access to an MPD server is
vulnerable to passive observers and MITM attacks. This may be fine for a local
network (presumably what it is intended to be used for), but certainly doesn't
work for exposing your MPD control interface to the wider internet. Existing
protocols such as TLS or SSH can be used to create a secure channel, but these
libraries tend to be large and hard to use securely. This is especially true
for TLS, but at least there's L<stunnel|https://www.stunnel.org/> to simplify
the implementation - at the cost of less convenient deployment.

In terms of authentication, you again need to implement this yourself. MPD
supports authentication using a plain-text password. This is fine for a trusted
network, but on an untrusted network you certainly want confidentiality to
prevent a random observer from reading your password.

=item B<Fast?> Sure.

Existing protocols may have put more effort into profiling and implementing
various optimizations than one would typically do with a custom and
quickly-hacked-together protocol, but still, it probably takes effort to design
a protocol that isn't fast enough.

=item B<Proxy support?> Depends...

Really depends on how elaborate you want to be. It can be very simple if all
you want is to route some messages, it can get very complex if you want to
ensure that these messages follow some format or if you want to reserve certain
interfaces or namespaces to certain clients. What surprised me about the MPD
protocol is that it actually has L<some support for
proxying|http://www.musicpd.org/doc/protocol/ch03s11.html>. But considering the
ad-hoc nature of the MPD protocol, the primitiveness and simplicity of this
proxy support wasn't too surprising. Gets the job done, I suppose.

=back

Overall, and as a rather obvious conclusion, a custom protocol really is what
you make of it. In general, though, it's a lot of work, not always easy to use,
and a challenge to get the security part right.



=head1 D-Bus

D-Bus is being used in L<Transmission|https://www.transmissionbt.com/> and is
what I used for L<Globster|https://dev.yorhel.nl/globster>. 

On a quick glance, D-Bus looks I<perfect>. It is high-level, has the messaging
patterns I described, the L<protocol
specification|http://dbus.freedesktop.org/doc/dbus-specification.html> does not
seem I<overly> complex (though certainly could be simplified), it has
implementations for a number of programming languages, has support for
networking, proxying is part of normal operation, and it seems fast enough for
most purposes. When you actually give it a closer look, however, reality isn't
as rose-colored.

D-Bus is designed for two very specific use-cases. One is to allow local
applications to securely interact with system-level daemons such as
L<HAL|https://en.wikipedia.org/wiki/HAL_(software)> (now long dead) and
L<systemd|http://freedesktop.org/wiki/Software/systemd/>, and the other
use-case is to allow communication between different applications inside one
login session. As such, on a typical Linux system there are two D-Bus daemons
where applications can export interfaces and where messages can be routed
through. These are called the I<system bus> and the I<session bus>.

=over

=item B<Easy?> Almost.

The basic ideas behind D-Bus seem easy enough to use. The fact that is has
type-safe messages, interface descriptions and introspection really help in
making D-Bus a convenient IPC mechanism.

The main reasons why I think D-Bus isn't all that easy to use in practice is
due to the lack of good introductionary documentation and the crappy state of
the various D-Bus implementations. There is a L<fairly good
article|https://pythonhosted.org/txdbus/dbus_overview.html> providing a
high-level overview to D-Bus, but there isn't a lot of material that covers how
to actually use D-Bus to interact with applications or to implement a service.

On the implementations, I have had rather bad experiences with the actual
libraries.  I've personally used the official libdbus-1, which markets itself a
"low-level" library designed to facilitate writing bindings for other
languages. In practice, the functionality that it offers appears to be too
high-level for writing bindings (L<GDBus|https://developer.gnome.org/glib/>
doesn't use it for this reason), and it is indeed missing a lot of
functionality to make it convenient to use directly. I've also played around
with Perl's L<Net::DBus|http://search.cpan.org/perldoc?Net%3A%3ADBus> and was
highly disappointed. Not only is the documentation rather incomplete, the
actual implementation has more bugs than features. And instead of building on
top of one of the many good event loops for Perl (such as
L<AnyEvent|http://search.cpan.org/perldoc?AnyEvent>), it chooses to implement
L<its own event
loop|http://search.cpan.org/perldoc?Net%3A%3ADBus%3A%3AReactor>. The existence
of several different libraries for Python doesn't incite much confidence,
either.

I was also disappointed in terms of the available tooling to help in the
development, testing and debugging of services. The L<gdbus(1)> tool is useful
for monitoring messages and scripting some things, but is not all that
convenient because D-Bus has too many namespaces and the terrible Java-like
naming conventions make typing everything out a rather painful experience.
L<D-Feet|http://live.gnome.org/DFeet/> offers a great way to explore services,
but lacks functionality for quick debugging sessions. I L<made an
attempt|http://g.blicky.net/dbush.git/> to write a convenient command-line
shell, but lost interest halfway. :-(

D-Bus has the potential to be an easy and convenient IPC mechanism, but the
lack of any centralized organization to offer good implementations,
documentation and tooling makes using D-Bus a pain to use.

=item B<Simple?> Not quite.

D-Bus is conceptually easy and the message protocol is alright, too. Some
aspects of D-Bus, however, are rather more complex than they need to be.

I have once made an attempt to fully understand how D-Bus discovers and
connects to the session bus, but I gave up halfway because there are too many
special cases. To quickly summarize what I found, there's the
C<DBUS_SESSION_BUS_ADDRESS> environment variable which could point to the
(filesystem or abstract) path of a UNIX socket or a TCP address. If that
variable isn't set, D-Bus will try to connect to your X server and get the
address from that. In order to avoid linking everything against X libraries, a
separate L<dbus-launch> utility is spawned instead. Then the bus address could
also be obtained from a file in your C<$HOME/.dbus/> directory, with added
complexity to still support a different session bus for each X session. I've no
idea how exactly connection initiation to the system bus works, but my
impression is that a bunch of special cases exist there, too, depending on
which init system your OS happens to use.

As if all the options in connection initiation aren't annoying enough, there's
also work on L<kdbus|https://lwn.net/Articles/580194/>, a Linux kernel
implementation to get better performance. Not only will kdbus use a different
underlying communication mechanism, it will also switch to a completely
different serialization format. If/when this becomes widespread you will have
to implement and support two completely different protocols and pray that your
application works with both.

On the design aspect there is, in my opinion, needless complexity with regards
to naming and namespaces. First there is a global namespace for I<bus names>,
which are probably better called I<application names>, because that's usually
what they represent.  Then, there is a separate I<object> namespace local to
each bus name.  Each object has methods and properties, and these are
associated with an I<interface name>, in a namespace specific to the particular
object. Despite these different namespaces, the convention is to use a full and
globally unique path for everything that has a name. For example, to list the
IM protocols that Telepathy supports, you call the C<ListProtocols> method in
the C<org.freedesktop.Telepathy.ConnectionManager> interface on the
C</org/freedesktop/Telepathy/ConnectionManager> object at the
C<org.freedesktop.Telepathy> bus. Fun times indeed. I can understand the
reasoning behind most of these choices, but in my opinion they found the wrong
trade-off.

Another point of complexity that annoys me is the fact that an XML format is
used to describe interfaces. Supporting XML as an IDL format is alright, but
requiring a separate format for an introspection interface gives me the
impression that the message format wasn't powerful enough for such a simple
purpose.  The direct effect of this is that any application wishing to use
introspection data will have to link against an XML parser, and almost all
conforming XML parser implementations are as large as the D-Bus implementation
itself.

=item B<Small?> Kind of.

C<libdbus-1.so.3.8.6> on my system is about 240 KiB. It doesn't cover parsing
interface descriptions or implementing a D-Bus daemon, but still covers most of
what is needed to interact with services and to offer services over D-Bus.
It's not I<that> small, but then again, libdbus-1 was not really written with
small size in mind. There's room for optimization.

=item B<Language independent?> Sure.

D-Bus libraries exist for a number of programming languages.

=item B<Networked?> Half-assed.

D-Bus I<officially> supports networked connections to a D-Bus daemon. Actually
using this, however, is painful. Convincing L<dbus-daemon(1)> to accept
connections on a TCP socket involves disabling all authentication (it expects
UNIX credential passing, normally) and requires adding an undocumented C<<
<allow_anonymous/> >> tag in the configuration (I only figured this out from
reading the source code). 

Even when you've gotten that to work, there is the problem that D-Bus isn't
totally agnostic to the underlying socket protocol. D-Bus has support for
passing UNIX file descriptors over the connection, and this of course doesn't
work over TCP. While this feature is optional and easily avoided, some services
(I can't find one now) use UNIX fds in order to keep track of processes that
listen to a certain event. Obviously, those services can't be accessed over the
network.

=item B<Secure?> Only locally.

D-Bus has statically typed messages that can be validated automatically, so
that's a plus.

For local authentication, there is support for standard UNIX permissions and
credential passing for more fine-grained authorization. For remote
authentication, I think there is support for a shared secret cookie, but I
haven't tried to use this yet.

There is, as with MPD, no support at all for confidentiality, so using
networked D-Bus over an untrusted network would be a very bad idea anyway.

=item B<Fast?> Mostly.

The messaging protocol is fairly lightweight, so no problems there. I do have
to mention two potential performance issues, however.

The first issue is that the normal mode of operation in D-Bus is to proxy all
messages through an intermediate D-Bus daemon. This involves extra context
switches and message parsing passes in order to get one message from
application A to application B. I believe it is I<officially> supported to
bypass this daemon and to communicate directly between two processes, but after
my experience with networking I am wary of trying anything that isn't part of
how D-Bus is I<intended> to be used. This particular performance issue is what
kdbus addresses, so I suppose it won't apply to future Linux systems.

The other issue is that a daemon that provides a service over D-Bus does not
know whether there exists an application that is interested receiving its
notifications. This means that the daemon always has to spend resources to send
out notification messages, even if no application is actually interested in
receiving them. In practice this means that the notification mechanism is
avoided for events that may occur fairly often, and an equally inefficient
polling approach has to be used instead. It is possible for a service provider
to keep track of interested applications, but this is not part of the D-Bus
protocol and not something you would want to implement for each possible event.
I've no idea if kdbus addresses this issue, but it would be stupid not to.

=item B<Proxy support?> Yup.

It's part of normal operation, even.

=back

D-Bus has many faults, some of them are by design, but many are fixable. I
would have contributed to improving the situation, but I get the feeling that
the goals of the D-Bus maintainers are not at all aligned with mine. My
impression is that the D-Bus maintainers are far too focussed on their own
specific needs and care little about projects with slightly different needs.
Especially with the introduction of kdbus, I consider D-Bus too complex now to
consider it worth the effort to improve. Starting from scratch seems less work.


=head1 JSON/XML RPC

While I haven't extensively used JSON-RPC or XML-RPC myself, it's still an
interesting alternative to study.
L<Transmission|https://www.transmissionbt.com/> uses JSON-RPC
(L<spec|https://trac.transmissionbt.com/browser/trunk/extras/rpc-spec.txt>) as
its primary IPC mechanism, and L<RTorrent|http://rakshasa.github.io/rtorrent/>
has support for an optional XML-RPC interface. (Why do I keep referencing
torrent clients? Surely there are other interesting applications? Oh well.)

The main selling point of HTTP-based IPC is that it is accessible from
browser-based applications, assuming everything has been setup correctly. This
is a nice advantage, but lack of this support is not really a deal-breaker for
me. Browser-based applications can still use any other IPC mechanism, as long
as there are browser plugins or some form of proxy server that converts the
messages of the IPC mechanism to something that is usable over HTTP. For
example, both solutions exist for D-Bus, in the form of the L<Browser DBus
Bridge|http://sandbox.movial.com/wiki/index.php/Browser_DBus_Bridge> and
L<cloudeebus|https://github.com/01org/cloudeebus>. Of course, such solutions
typically aren't as convenient as native HTTP support.

Since HTTP is, by design, purely request-response, JSON-RPC and XML-RPC don't
generally support asynchronous notifications. It's possible to still get
asynchronous notifications by using
L<WebSockets|https://en.wikipedia.org/wiki/WebSocket> (Ugh, opaque stream
sockets, time to go back to our L<custom protocol|/Custom Protocol>) or by
having the client implement a HTTP server itself and send its URL to the
service provider (This is known as a
L<callback|https://duckduckgo.com/?q=web%20service%20callback> in the
L<SOAP|https://en.wikipedia.org/wiki/SOAP> world. I have a lot of respect for
developers who can put up with that crap).  As I already hinted, neither
solution is simple or easy.

Let's move on to the usual grading...

=over

=item B<Easy?> Sure.

The ubiquity of HTTP, JSON and XML on the internet means that most developers
are already familiar with using it. And even if you aren't, there are so many
easy-to-use and well-documented libraries available that you're ready to go in
a matter of minutes.

Although interface description languages/formats exist for XML-RPC (and
possibly for JSON-RPC, too), I get the impression these are not often used
outside of the SOAP world. As a result, interacting with such a service tends
be weakly/stringly typed, which, I imagine, is not as convenient in strongly
typed programming languages.

=item B<Simple?> Not really.

Many people have the impression that HTTP is somehow a simple protocol. Sure,
it may look simple on the wire, but in reality it is a hugely bloated and
complex protocol. I strongly encourage everyone to read through L<RFC
2616|https://tools.ietf.org/html/rfc2616> at least once to get an idea of its
size and complexity. To make things worse, there's a lot of recent activity to
standardize on a next generation HTTP
(L<SPDY|https://en.wikipedia.org/wiki/SPDY> and L<HTTP
2.0|https://en.wikipedia.org/wiki/HTTP_2.0>), but I suppose we can ignore these
developments for the foreseeable future for the use case of IPC.

Of course, a lot of the functionality specified for HTTP is optional and can be
ignored for the purpose of IPC, but that doesn't mean that these options don't
exist. When implementing a client, it would be useful to know exactly which
HTTP options the server supports. It would be wasteful to implement compression
support if the server doesn't support it, or keep-alive, or content
negotiation, or ranged requests, or authentication, or correct handling for all
response codes when the server will only ever send 'OK'. What also commonly
happens is that server implementors want to support as much as possible, to the
point that you can have JSON or XML output, depending on what the client
requested.

XML faces a similar problem. The format looks simple, but the specification has
a bunch of features that hardly anyone uses. In contrast to HTTP, however, a
correct XML parser can't just decide to not parse C<< <!DOCTYPE ..> >> stuff,
so it I<has> to implement some of this complexity.

On the upside, JSON is a really simple serialization format, and if you're
careful enough to only implement the functionality necessary for basic HTTP, a
JSON-RPC implementation I<can> be somewhat simple.

=item B<Small?> Not really.

What typically happens is that implementors take an existing HTTP library and
build on top of that. A generic HTTP library likely implements a lot more than
necessary for IPC, so that's not going to be very small. RTorrent, for example,
makes use of the not-very-small L<xmlrpc-c|http://xmlrpc-c.sourceforge.net/>,
which in turn uses L<libcurl|http://curl.haxx.se/> (400 KiB, excluding TLS
library) and either the bloated L<libxml2|http://xmlsoft.org/> (1.5 MiB) or
L<libexpat|http://www.libexpat.org/> (170 KiB). In any case, expect your
programs to grow by a megabyte or more if you go this route.

Transmission seems rather less bloated. It uses the HTTP library that is built
into L<libevent|http://libevent.org/> (totalling ~500 KiB, but libevent is also
used for other networking parts), and a simple JSON parser can't be that large
either. I'm sure that if you reimplement everything from scratch for the
purpose of building an API, you could get something much smaller. Then again,
even if you manage to shrink the size of the server that way, you can't expect
all your users to do the same.

If HTTPS is to be supported, add ~500 KiB more. TLS isn't the simplest
protocol, either.

=item B<Language independent?> Yes.

Almost every language has libraries for web stuff.

=item B<Networked?> Definitely.

In fact, I've never seen anyone use XML/JSON RPC over UNIX sockets.

=item B<Secure?> Alright.

HTTP has built-in support for authentication, but it also isn't uncommon to use
some other mechanism (based on cookies, I guess?).

Confidentiality can be achieved with HTTPS. There is the problem of verifying
the certificate, since I doubt anyone is going to have certificates of their
local applications signed by a certificate authority, but there's always the
option of trust-on-first use. Custom applications can also include a
fingerprint of the server certificate in the URL for verification, but this
won't work for web apps.

=item B<Fast?> No.

JSON/XML RPC messages add significant overhead to the network and requires more
parsing than a simple custom solution or D-Bus. I wouldn't really call it
I<fast>, but admittedly, it might still be I<fast enough> for most purposes.

=item B<Proxy support?> Sure.

HTTP has native support for proxying, and it's always possible to proxy some
URI on the main server to another server, assuming the libraries you use
support that. It's not necessarily simple to implement, however.

=back

The lack of asynchronous notifications and the overhead and complexity of
JSON/XML RPC make me stay away from it, but it certainly is a solution that
many client developers will like because of its ease of use.


=head1 Other Systems

There are a more alternatives out there than I have described so far. Most of
those were options I dismissed early on because they're either incomplete
solutions or specific to a single framework or language. I'll still mention a
few here.

=head2 Message Queues

In the context of IPC I see that message queues such as
L<RabbitMQ|https://www.rabbitmq.com/> and L<ZeroMQ|http://zeromq.org/> are
quite popular. I can't say I have much experience with any of these, but these
MQs don't seem to offer a solution to the problem I described in the
introduction. My impression of MQs is that they offer a higher-level and more
powerful alternative to TCP and UDP. That is, they route messages from one
endpoint to another. The contents of the messages are still completely up to
the application, so you're still on your own in implementing an RPC mechanism
on top of that. And for the purpose of building a simple RPC mechanism, I'm
convinced that plain old UNIX sockets or TCP will do just fine.

=head2 Cap'n Proto

I probably should be spending a full chapter on L<Cap'n
Proto|http://kentonv.github.io/capnproto/> instead of this tiny little section,
but I'm simply not familiar enough with it to offer any deep insights. I can
still offer my blatantly uninformed impression of it: It looks very promising,
but puts, in my opinion, too much emphasis on performance and too little
emphasis on ease of use. It lacks introspection and requires that clients have
already obtained the schema of the service in order to interact with it. It
also uses a capability system to handle authorization, which, despite being
elegant and powerful, increases complexity and cognitive load (though I
obviously need more experience to quantify this). It still lacks
confidentiality for networked access and the number of bindings to other
programming languages is limited, but these problems can be addressed.

Cap'n Proto seems like the ideal IPC mechanism for internal communication
within a single (distributed) application and offers a bunch of unique features
not found in other RPC systems.  But it doesn't feel quite right as an easy API
for others to use.

=head2 CORBA

CORBA has been used by the GNOME project in the past, and was later abandoned
in favour of D-Bus, primarily (I think) because CORBA was deemed too L<complex
and incomplete|http://dbus.freedesktop.org/doc/dbus-faq.html#corba>. A system
that is deemed more complex than D-Bus is an immediate red flag. The L<long and
painful history of CORBA|http://queue.acm.org/detail.cfm?id=1142044> also makes
me want to avoid it, if only because that makes it very hard to judge the
quality and modernness of existing implementations.

=head2 Project Tanja

A bit over two years ago I was researching the same problem, but from a much
more generic angle. The result of that was a project that I called Tanja.  I
described its concepts L<in an earlier
article|https://dev.yorhel.nl/doc/commvis>, and wrote an incomplete
L<specification|http://g.blicky.net/tanja.git/> along with implementations in
L<C|http://g.blicky.net/tanja-c.git/>, L<Go|http://g.blicky.net/tanja-go.git/>
and L<Perl|http://g.blicky.net/tanja-perl.git/>. I consider project Tanja a
failure, primarily because of its genericity. It supported too many
communication models and the lack of a specification as to which model was
used, and the lack of any guarantee that this model was actually followed, made
Tanja hard to use in practice. It was a very interesting experiment, but not
something I would actually use. I learned the hard way that you sometimes have
to move some complexity down into a lower abstraction layer in order to keep
the complexity in check at higher layers of abstraction.


=head1 Conclusions

This must be the longest rant I've written so far.

In any case, there isn't really a perfect IPC mechanism for my use case. A
custom protocol involves reimplementing a lot of stuff, D-Bus is a pain, and
JSON/XML RPC are bloat.

I am still undecided on what to do. I have a lot of ideas as to what a perfect
IPC solution would look like, both in terms of features and in how to implement
it, and I feel like I have enough experience by now to actually develop a
proper solution. Unfortunately, writing a complete IPC system with the required
utilities and language bindings takes B<a lot> of time and effort.  It's not
really worth it if I am the only one using it.

So here is my plea to you, dear reader: If you know of any existing solutions
I've missed, please tell me. If you empathize with me and want a better
solution to this problem, please get in touch as well! I'd love to hear about
projects which face similar problems and have similar requirements.