Tech

A blog post talking about the history of the privacy-focused Tox protocol.

Background

After the 2013 Snowden US government leaks, it's no secret that many people, including those from the general public, have become quite uncomfortable about the topic of government surveillance.

Up until then, there was always an air of acceptance among everyone that the government was spying on them and that, most likely, all digital communications were being harvested by it somehow, but nobody gave the thought too much thinking.

Well, Snowden changed this and, in the wake of publications of classified materials that showed just how much the US government was eavesdropping on everyone, including domestically on US citizens, it became clear that the idea of being spied upon suddenly lost all its humor in the public's eyes.

Programs such as PRISM became part of the public consciousness and technologies that many had taken for granted, such as Skype, became the target of much distrust all of a sudden.

People were suddenly concerned about their online privacy, and felt betrayed by the revelations.

And so, as a consequence, in June 2013, the first commit was published on github by a user named irungentoo, a commit for a repository named toxcore.

And so was the Tox protocol born.

Design goals

The protocol, in its infancy, strived to achieve some very straight forward goals:

1. It was supposed to be entirely a peer-to-peer protocol, meaning that unlike many other instant messaging protocols devised up until that point (such as Whatsapp, Signal, Telegram etc.), the tox protocol will not rely on any central service at all, outside of the barebones bootstrap nodes which would be used to get the ball rolling

2. It would be an end-to-end encrypted messaging system, meaning that the only players involved in the conversation would be the ones that would have the means of decrypting it

3. Once a contact's friend request is accepted, the two clients would immediately connect directly to each other, without relying on any relays or intermediaries whatsoever (except if any of the contacts decides to use Tor to mask their IPs for additional privacy)

The Snowden leaks revealed that the main reason digital communication was prone to being eavesdropped on was that the most famous and common instant messaging communication programs relied on servers to relay the messages between the participants. This means that the NSA only needed to go to the server operators to convince them to handle these messages to them, either voluntarily or via use of legal coercion.

So the Tox protocol solved this dilemma by simply getting rid of servers altogether. You can't easily spy on everyone if people are directly connecting to each other to talk, without central intermediaries.

A good analogy is the advent of telephone companies. It's easy for the government to spy on phone conversations because, ultimately, there are only a handful of phone companies in any country, so they just need to compromise all of them and then they can access the phone conversations of millions of people. This is possible because all these millions of people rely on just a handful of companies for all their communication.

The less companies there are to compromise, the easier it is for the government to breach the service.

Drawbacks

The idea, was a good one. There were some caveats though.

Who came first? The chicken or the egg?

The main issue that hampered Tox's growth was the fact that Tox, by design, was very privacy focused.

Yes, in theory, you could use your real name as your tox profile account's name, you could post your email and phone number in your tox details as well for all your contacts to see.

But, in practice, most people used an anonymous username that was very difficult for others to guess. Moreover, the protocol didn't even mandate for the registration of an email address or a phone number. Basically, the protocol allowed for full anonymity at all times.

This was by design like this.

The issue with this was that there was no easy way to find your friends even if they also used tox.

There was no directory where you could search people by name, email address, phone number or even tox username at all.

Instead, if you wanted to talk with someone over tox, you first had to share your Tox ID with them, which is this long 76 character long hexadecimal string, that they would then use to find you over the internet and send you a friend invite.

Once you accepted the invite, your tox client would connect directly with theirs over the internet, negotiate a secret encryption key with them and then use this to encrypt all your communications with each other.

The key would only exist on your device and theirs, never leaked to any third party at all.

Needless to say, this was a cumbersome process, and it made finding new people a complete and utter hassle. Not only this, but it opened the door for a chicken and an egg dilemma, because if you needed to securely talk with someone, you first had to give them your tox ID (or they had to give you theirs) over a secure private channel before you even started talking over tox.

But in order to do that, you needed to have a private trusted communication channel between the two of you already to send the tox ID through, so what even was the point of tox if you already had that?

Offline messages? What's that?

Another, glaring shortcoming that the tox protocol suffered from, due to its server-less architecture, was the lack of offline messaging functionality.

Skype, Teams, Signal and all these other instant messaging platforms have servers that are, inherently, trusted by all the clients by design.

Servers might not seem like that much of a huge deal, but it allows for useful features like offline messaging to happen without having to overly engineer a very complicated solution.

Basically, if Bob wants to send Alice a message over Skype, for example, but Alice is offline at the time, Bob can send the message, the message gets recorded and timestamped by Skype servers which are, by design, always online, and then Bob can do other things in the meantime, even go offline as well, knowing that the message has been sent.

Now, even if Bob may have gone offline in the meantime, Alice may come online, connect to a Skype server and, as soon as the server sees her coming online, it remembers that Bob had tried to send her a message when she was offline, and sends the message to her now.

Bob doesn't need to be online for any of this. The Skype server did the job for him behind the scenes. This is what's known as offline messaging.

Tox doesn't have servers, though, so none of this is possible.

I'm sure, technically, this can be done in a peer-to-peer application too, if you put enough thought into designing a clever solution.

As long as there are other peers for you to connect to, you can engineer a solution in which they store the message themselves, instead of relying on a server, and relay it somehow to Alice when she gets online, but then you have to design a propagation protocol so that the message is kept alive while peers come online and go offline randomly, make sure that a malicious peer doesn't just flood the network with bogus offline messages meant to DoS all other peers and other such nonsense.

The point is, designing a solution that doesn't rely on servers is not easy and tox just decided to take the easiest approach out: just avoid supporting offline messages entirely.

What this means is that in Tox, if you wish to send any of your contacts a message, both you and the contact in question have to be online!

Sure, tox can hide this fact by queuing the message locally on your computer, waiting for your contact to come online to send it to him but, if you decide to shut down your computer during this time while they're still offline, they won't be getting your message while your computer is shut down, even if they will come online in the meantime.

Basically, all your offline messages to your contact will ever be sent to them only during the brief period when both you and them are online at the same time.

This makes people who live on opposite sides of the planet, and who have huge time zone differences between them, very difficult to communicate with each other over tox, as one is usually offline sleeping while the other is online, and vice versa.

Worse, if you have an urgent message you really need someone over tox to read, your only recourse is to keep your computer online and not sleeping at all times, until they get online, for the message to be delivered.

This is not only a huge waste of power but, many times, it's impractical. Basically, in order to mitigate the lack of servers, communicating parties have to turn their clients into servers themselves.

And, not only this, but because Tox is a trustless protocol by design and peers are designed not to trust each other, even if they are directly communicating with one another, a message that is being received by Alice at a later time than it was when being sent by Bob (i.e. an offline message), gets timestamped by Alice's tox client as the time of it being received by Alice, not the time it had been sent by Bob to Alice.

Or, in simpler words, if Bob sent Alice a message, but Alice was offline on Tox for an entire week afterwards so she couldn't receive it, when Alice does finally come online on Tox and receives Bob's message, the message is recorded in Alice's client as having been sent at the time Alice came online, not a week prior when it had actually been sent by Bob.

This is because, Bob could have hacked his own Tox client to lie to Alice about when he had sent the message, in which case his client could claim that the message had been sent a month prior, or even a year prior. Without a trusted third party server to corroborate the sending event, Alice's client has no way of knowing if what Bob's client says is true, nor can Bob's client even prove that he had sent the offline message at the time he claims he has and not earlier or later.

As such, in Tox, the offline messages you receive from a contact are timestamped on your end as the time you actually receive them, not at the time your contact claims to have sent them to you.

This is the issue with software that's inherently distrustful by design. You always end up lacking features that software with trusted servers have.

Have more than one computer? Sucks to be you!

Oh, this one's a doozy.

You know how, on Skype or Microsoft Teams, you just have to login to your account and then you can send messages from literally any internet-connected computer at your disposal?

Like, let's say you send a message to your boss on at work, close your computer to go on lunch break, and then, while you're gone and eating, you decide to see if your boss answered by just logging into Teams via your phone and check.

You can do this because your Teams account is stored somewhere in a database and, regardless of where you connect to Teams servers from, whether it's your work Desktop machine, your Android phone or your grandma's laptop, the servers are always the same and the database that they use to store information about you is also always the same. Only the Teams clients are different.

Well, Tox only has clients. It has no servers, no databases, no anything.

Basically, if you want to share your Tox conversations across machines, you're pretty much out of luck.

OK, in theory, there's nothing in the Tox protocol that prevents a Tox client from somehow implementing a solution to synchronize conversations across multiple computers using peer-to-peer technology. Maybe someone, someday, will actually implement this and I'll take my words back.

But, in practice, I've personally never seen this done.

Only once did I move my qTox profile from a Linux laptop to my Windows desktop by copying the profile folder on a thumb drive and, thankfully, everything went smoothly and without any bugs whatsoever. That way, I've effectively moved my encrypted Tox conversations across machines.

However, it's worth noting that, at least back at the time, this wasn't officially supported by qTox, meaning that it could have very well not worked. Or, even if it did work, a future update could make it not work anymore.

Basically, if you want to use Tox on multiple computers, the official fully supported way of doing it, is to just generate a new Tox profile on each and every one of them. And that means you'll have to re-add all your contacts across all of them, every one of your contacts will have to accept a separate friend request for each computer you use tox on, your friends will have you listed multiple times in their contacts list, once for each of your computers and, even with all of this, none of your chats will be synchronized across your devices, meaning that different computers will have entirely separate conversation histories.

This.IS.A.NIGHTMARE.

If you ever wonder why the Tox protocol was never successful, it's not because it was buggy or it lacked advanced features; it's because, by design, it couldn't implement some of the most basic features that most people expect by default from any instant messaging app.

Its greatest strength, the fact that it had no servers or central database, was also its downfall: no servers means no simple way of inter-device data syncing, offline messaging or central user directories to add friends from.

This is why Tox failed

Lack of support for niche Linux distros for certain Tox clients

This is more of a niche thing, as most software doesn't support Linux anyways, but the user base that most Tox clients pandered to, was the privacy oriented, corporation hating, free software loving Linux community.

Sure, there were Tox clients that were geared only towards Windows too, but those were very rare.

So you'd think, given their primary user base, that many client developers would go out of their way to ensure good support for most distros. Well, you'd be wrong in thinking that.

Or at least, I was wrong about this with a tox client named qTox.

qTox was one of the more popular clients out there, and it was my client of choice because it had the widest operating system support of all clients.

So, naturally, that was my first choice for a client.

I also have to point out that I'm a Fedora linux user. I use Fedora Workstation as my daily driver on my personal laptop, and I love this OS, with all its flaws and shortcomings.

One day, I upgraded to Fedora 36, as that was the latest release at the time and then, as usual, I went ahead and enabled RPM fusion repositories on my system.

Then, from RPM fusion, I installed qTox on my system.

Well, wouldn't you know it, I was getting an error upon trying to start the program.

The error? A library called libvpx.so.6 was missing on my system. Of course, I didn't get this error message while trying to start qTox normally from my launcher, I had to try to start qTox from the terminal, just so that I would get a printout on why it was failing to start in the first place on the console.

Well, wouldn't you know it, apparently Fedora 36 upgraded its system libraries and instead of coming preinstalled with libvpx.so.6, as qTox seemed to be expecting, it came with libvpx.so.7 instead, which was entirely different.

I mean, I know RPM Fusion was a third party repository and that people shouldn't expect much quality control from stuff in it but, isn't the entire point of a package manager that it was supposed to solve dependency issues like this?

And yes, I tried creating a symbolic link named libvpx.so.6 to libvpx.so.7, expecting it to work out of the box, but it wouldn't. The program would still crash immediately upon start-up with an even uglier error message.

The point is, while Fedora is indeed a bit niche, it's still one of the most popular Linux distros on the planet. You'd think the development team for qTox would try to pre-emptively fix issues like these before people would make the upgrade.

And, for the record, I didn't do the upgrade the exact day Fedora 36 came out. I usually wait a couple of weeks before I upgrade, so they had more than enough time to sort this out. The fact is, they didn't care.

Granted, qTox is just one Tox client. Their development team doesn't develop c-toxcore or any of the many other Tox clients on the planet, so they are just one party at fault here.

And, despite this issue, qTox also offered an AppImage that worked out of the box so I could continue to use qTox even after this.

But still, it's disheartening when you realize that this is the type of bugs you encounter quite often when trying to use Tox.

The titan has fallen

With all of these shortcomings, and without obvious technical solutions in sight, the Tox protocol has seen an excruciatingly slow but painful death.

It bled users year after year, as more and more privacy focused individuals sought to use other software suites that promised privacy but which also offered the benefits of centralized services, like Signal.

Don't get me wrong, I despise Signal as much as the next person, and the fact that I still have to have a phone number in order to use the service is extremely infuriating. But, at the end of the day, Signal is easier to use than Tox. And that fact is simply indisputable.

Couple that with the fact that Signal also is open source, much like most tox clients are, and you really have no reason to prefer Tox over Signal.

As time went on, developers, for one reason or another, started abandoning their tox projects, one after the other.

People simply didn't seem to care about peer-to-peer protocols anymore and, as the Snowden leaks were slowly fading out of the general population's consciousness, so too did the volunteers working on the myriad of tox clients all around the world.

And, after many years, the most popular tox client out there, qTox, had its official repository on github frozen, with the developers leaving behind a message that they're planning on abandoning the project.

Keep in mind, there are still many tox clients out there, and the main project, c-toxcore, the one that actually implements most of the functionality offered by Tox, is still maintained to this day.

But c-toxcore is just a platform-agnostic library that implements the Tox protocol itself. A library is worthless if you don't have front-end clients to expose its functionality.

That's what all the tox clients are supposed to do. Now, qTox is abandoned, so that's out of the question.

If you go to the Tox protocol's wikipedia page, you'll see a table with the most popular tox clients out there, as well as a column in that table mentioning whether they're still supported or not.

And, at least as of right now, most clients reported on that page are said to have been abandoned.

The most popular Tox client still being maintained right now is one named Toxic, a C client implementation relying on the Ncurses library. Issue with this one is that it's reliant on Unix functionality, meaning that it doesn't work out of the box on Windows.

Sure, technically savvy people can go out of their ways to make it work on Windows, either by compiling the source code using Cygwin or maybe using the Linux subsystem for Windows that's available under Windows 11 but, at the end of the day, most normal people won't go through this stuff when there's Skype, Microsoft Teams, Slack and many other alternatives available at their fingertips.

qTox was the last Tox client that still supported Windows out of the box and now that it's also abandoned, a large portion of desktop users will don't have the option anymore to use Tox, sadly.

Sure, this is a huge loss for Windows users, but it's an even larger loss for Tox, as now, a lot of people won't even consider using the protocol anymore, since they won't be able to use it to communicate with friends and family that do use Windows.

The protocol itself is maintained by the previously mentioned c-toxcore github project, which only maintains the library that does all the heavy work behind the scenes and which is used by Tox clients.

The library's latest stable version, 0.2.17, as of the posting of this blog post, was published more than a year ago at this point (13 months, to be exact).

The developers never said that it was being abandoned too, but, personally, if a piece of software doesn't get any updates for more than a year, I really start to wonder if it's still being maintained.

Technically there's also another Tox client that supports Windows called yat, but as of the writing of this blog post, I tried installing it myself and all installation links lead me to a website called www.lovecry.pt that seems to be down.

I also tried to reach that website a week ago and I didn't have any success back then either.

So yeah, I'm not getting my hopes up anymore.

At this point, I'm convinced the Tox protocol is either destined to die sooner or later, as nobody cares about mass surveillance anymore to go through the hoops that is using Tox, or, best case scenario, it becomes a protocol mainly used by third party clients that work only on Unix operating systems and used by a very niche community of privacy focused nerds.

For the past decade that I've been using it, I've never heard anyone mention Tox in day to day conversations, as an alternative to Skype or Discord, and now I'm more than sure I never will.

And with the advent and promotion of the federated communication protocol Matrix, there's even less of an incentive for people to seek out Tox nowadays.

In the end, one really has to wonder: how long does it take for a project to die?

This blog is federated. If you wish to follow this tech blog, please use the following Fediverse handle: @tech@blog.transistor.one

Blog post by Alexandru Pentilescu.

You may contact me at alexandru.pentilescu@disroot.org

Optionally, you may also encrypt your emails to me using the following PGP key: 0xFF49E5748BD42A6A6A7DECFDD38B28DF9F7497A2

Download that key from any keyserver you wish