01 August 2017

"End-to-end encryption"

The question of regulating encrypted communication has come up again. I was going to write again about how the politicians don't understand the technologies, and they probably don't, but if they did, what would they do about it?  The details are too complex to debate on TV news. What percentage of the viewing public even knows what public-key encryption is?

Politicians often talk as if "end-to-end encryption" is a technology, and one which is rare and might practically be banned. There are then huge arguments about whether such banning would be good or bad, which leave me somewhat bemused.

Of course, "end-to-end encryption" is no more a technology than "driving to a friend's house" is a technology. Cars and roads and driving are technologies, driving to a friend's house, or to a restaurant, or to work, are social or economic practices that make use of the technology.

Similarly, sending encrypted messages is a technology. sending "end-to-end" encrypted messages is not a technology, it's just sending encrypted messages to an intended end recipient. Whether a particular message is "end-to-end" encrypted depends on who the end is.

The soundbites talk about one kind of messaging: messages sent person-to-person from a sender to a recipient via a service provider like Whatsapp, Microsoft or Google.

In 2017, most data sent over the internet that is at all personal is encrypted. Huge efforts have been made over the last five or so years to get to this stage, yet the debates about encryption have not even touched on the fact. Data in motion seems to be invisible. The encryption used to send the messages is very strong; again, a few years ago, there were quite a few bugs in commonly used implementations, but efforts have been made to find and fix such bugs, and while there are likely to be some left, it is plausible that nearly all such encrypted messages are unbreakable even by the most powerful national security organisations.

However, the way most of these services work today is that the sender makes a connection to the service provider and authenticates himself with a password. The Service Provider also authenticates itself to the sender with a certificate, though that's mostly invisible. The sender then sends their message encrypted to the Service Provider, which decrypts it and stores it. Later (or simultaneously) the recipient makes a connection to the Service Provider the same way, and the Service Provider encrypts the message and sends it to the recipient. This is fundamentally the same whether we are talking about messaging apps, chat, or email, and whether the devices used are computers, phones or tablets.

Anyway, call this method 1. Service Provider Mediated

A few of these services now have an extra feature. The sender's app first encrypts the message in a way that con only be decrypted by the recipient, then encrypts it again to send to the Service Provider. The Service Provider decrypts one level of encryption, but not the second. When the recipient connects, the Service Provider re-encrypts the already encrypted message and sends to the recipient. The recipient decrypts the message twice, once to get what the Service Provider had stored, and then again to get what the sender originally wrote.

That is why the politicians are talking about Whatsapp, Telegram and so on.

This is method 2. Service Provider Mediated, with provided end-to-end encryption

An important question here is who keeps track of the encryption keys. If the Service Provider has that responsibility, then it can support interception by giving the sender the wrong encryption key; one that it or the government can reverse. If the sender keeps the recipient's encryption key, that is not possible, the Service Provider receives no messages that it is able to decrypt.

Going back to method 1, if the Service Provider doesn't guide the end-to-end encryption, it's still possible to add it with special software for the sender and recipient. This is awkward for the users and has never caught on in a big way, but it's the method that the authorities used to worry about, decades back.

Method 3. Service Provider Mediated with independent end-to-end encryption

There are plenty more. The sender connects to the Service Provider and indicates, via an encrypted message, what recipient they want to message. The Service Provider replies with an endpoint that the sender can connect to. The sender then directly connects to the recipient and transmits an encrypted message, which the recipient decrypts.

This peer-to-peer messaging isn't fundamentally different in technology from the end-to-end encrypted scenario. In both cases the actual networking is "store-and-forward": An intermediary receives data, stores it, and then transmits it to either another intermediary or the recipient. The only difference is how long the data is stored from; a typical router will store the data for only a fraction of a second before transmitting and deleting it, whereas a Service Provider's application server will store it at least until the recipient connects to retrieve it, and quite likely will archive it permanently. (Note there are regulations in some jurisdictions that require Service Providers to archive it permanently, but that applies to their application servers and not to routers, which handle orders of magnitude more data, most of which is transient).

It's not always obvious to the user whether a real-time connection is mediated or not. Skype calls were originally peer-to-peer, and Microsoft changed it to mediated after they bought Skype. The general assumption is that this was at the behest of the NSA to enable interception, though I've not seen any definitive evidence.

Another thing about this kind of service is that the Service Provider does not need nearly as much resource as one that's actually receiving all the messages their users send. There could be a thousand different P2P services, in any jurisdiction. With WebRTC now built into browsers, it's easy to set one up.

Method 4. Service Provider directed peer-to-peer.

It's not actually hard to be your own Service Provider. The sender can put the message on his own server, and the recipient can connect to the sender's server to receive it. Or, the sender can connect to the recipient's server, and send the message to that. In either case, the transmission of the messages (and it's only one transmission over the public internet, not two as in the previous cases) will be encrypted.

As with method 2,  the Service Provider might manage the encryption keys for the user, or the user's app might retain encryption keys for the correspondents it has in its directory.

The software is all free and common. Creating a service requires a little knowledge, but not real expertise. I estimate it would take me 90 minutes and cost £10 to set up a publicly-accessible email, forum and/or instant messaging service, using software that has been widespread for many years, and that uses the same secure encryption that everything else on the internet uses. Whether this counts as "end to end encryption" depends entirely on what you count as an "end".  If I want the server to be in my house instead of a cloud data centre in the country of my choice, it might cost me £50 instead of £10, and it's likely to have a bit more downtime. That surely would make it "end-to-end", at least for messages for which I am either the sender or the recipient.

This is getting easier and more common, as internet speeds improve, connected devices proliferate, and distrust of the online giants' commercial surveillance practices grows. There have been one or two "server in a box" products offered which you can just buy and plug in to get this kind of service -- so far they have been dodgy, but there is no technical barrier to making them much better. Even if such a server is intended and marketed simply as a personal backup/archive solution, it is nevertheless in practice a completely functional messaging platform. The difference between an application that saves your phone photos to your backup drive and a full chat application is just a little bit of UI decoration, and so software like owncloud designed to do the first just throws in the second because it's trivial.

That is Method 5. Owned server

There are several variants covered there. The user's own server might be on their own premises, or might be rented from a cloud provider. If rented, it might be a physical machine or a virtual machine. The messages might be encrypted with a key owned by the recipient, or encrypted with a key configured for the service, or both, or neither. Whether owned or rented, the server might be in the same country as the user, or a different country. Each of these makes a significant difference from the point of view of an investigating agency wanting to read the messages.

Investigating authorities aren't only concerned with encryption, though, they also want to know who is sending or receiving a message, even if they can't read it. This could make the politicians' opposition to mediated end-to-end encryption more reasonable: the Service Providers allow users to connect to their servers more or less anonymously. Using peer-to-peer or personal cloud services, the data is secure but the identity of the recipients of messages is generally easier to trace. The Service Providers give the users that the authorities are interested in a crowd of ordinary people to hide among.

It's easy to sneer at Amber Rudd, but can you imagine trying to describe a policy on this in a TV interview, or in the House of Commons? Note I've skipped over some subtle questions.

Even if you could, you probably wouldn't want to. Why spell out, "We want to get cooperation from Facebook to give us messages, but we're not stupid, we know that if the terrorists buy a £100 off-the-shelf NAS box and use that to handle their messages, that won't help us"?

Summary: kinds of messaging practice

Service Provider mediated non-end-to-end

Data accessible to authorities: with co-operation of Service Provider
Identity accessible to authorities: IP addresses obtainable with co-operation of Service Provider but can be obscured by onion routing / using public wifi etc
User convenience: very convenient

Service Provider mediated end-to-end

Data accessible to authorities: No
Identity accessible to authorities: IP addresses obtainable with co-operation of Service Provider but can be obscured by onion routing / using public wifi etc
User convenience: very convenient

End-to-end layered over Service Provider (e.g. PGP mail)

Data accessible to authorities: No
Identity accessible to authorities: IP addresses obtainable with co-operation of Service Provider but can be obscured by onion routing / using public wifi etc
User convenience: very inconvenient, all users must use special software, do key management


Data accessible to authorities: No
Identity accessible to authorities: IP addresses directly accessible by surveillance at either endpoint or at ISP
User convenience: fiddly to use, need to manage directories of some kind

Personal Internet Service (Hosted)

Data accessible to authorities: With the cooperation of the host, which could be in any country
Identity accessible to authorities: IP addresses directly accessible by surveillance at either endpoint or at ISP
User convenience: Significant up-front work required by one party, but very easy to use by all others. Getting more convenient.

Personal Internet Service (on-site)

Data accessible to authorities: If they physically seize the computer
Identity accessible to authorities: IP addresses directly accessible by surveillance at either endpoint or at ISP
User convenience: Significant up-front work required by one party, but very easy to use by all others. Getting more convenient.

Appendix: Things I can think of but have skipped over to simplify
  • Disk encryption -- keys stored or provided from outside at boot
  • Certificate spoofing, certificate pinning
  • Client applications versus web applications 
  • Hostile software updates
  • Accessing data on virtual servers through hypervisor

11 July 2017

Revisiting the Program

Alrenous has played the Thesis 11 card:

Alrenous @Alrenous  2h2 hours ago
 Finally, if you're really confident in your philosophy, it should move you action. Or why bother?
You moved to China. Good work.
Edit: I totally misread Alrenous here: he's not saying "Change the world", he's saying "change your own life/environment". So the below, while still, in my view, true and important, is not particularly relevant to his point. Oh well.

He makes a valid point that good knowledge cannot be achieved without trying things:

Alrenous @Alrenous  3h3 hours ago
 Have to be willing to fail to do something new. Something new is patently necessary. NRx isn't willing to fail. That's embarrassing.

The problem with this is that neoreaction is the science of sovereignty. Like, say, the science of black holes, it is not really possible for the researcher with modest resources to proceed by experiment, valuable though that would be.

We have ideas on how to use and retain sovereignty, but less to say about how to achieve it. There is a great deal of prior art on how to gain power via elections, guerrilla warfare, coup d'état, infiltration; we don't really have much of relevance to add to it.

We could do experiments in this area, by forming a political party or a guerrilla army or whatever, but that's a long way from our core expertise, and though we would like to experiment with sovereignty, attempting to get sovereignty over the United States to enable our experiments is possibly over-ambitious. We could hope to gain some small share of power, but we believe that a share of power is no good unless it can be consolidated into sovereignty.

Given that we do not have special knowledge of achieving power, it seems reasonable that we should produce theory of how power should be used, and someone better-placed to get power and turn it into sovereignty should run their military coup or whatever, and then take our advice. That's what we care about, even if cool uniforms would be better for getting chicks.

I put this forward as a goal in 2012. 

This is an ambitious project, but I think it is genuinely a feasible route to implementing our principles. Marxism's successes in the 20th Century didn't come because its theories were overwhelmingly persuasive; they came because Marxism had theories and nobody else did.

Since then, we have seen Steve Bannon, who apparently has at least read about and understood Moldbug, in a position of significant power in the Trump administration. We have seen Peter Thiel also with some kind of influence, also with at least sympathies towards NRx. These are not achievements in the sense that in themselves they make anything better. But they are experimental validations of the strategy of building a body of theory and waiting for others to consume it.

I have for the last few days been suggesting that Mark Zuckerberg could win the presidency as a moderate technocrat who will save the country from Trump and the Alt-Right Nazis, consolidate power beyond constitutional limits, as FDR did, and reorganise FedGov along the lines of Facebook Inc. This outcome is, frankly, not highly probable, but I insist that it is not absurd. One of the things that controls the possibility of this sort of outcome is whether people in positions of influence think it would be a good thing or a bad thing. If, with our current level of intellectual product we can get to the point of 2017 Bannon, is it not plausible that with much more product, of higher quality, much more widely known and somewhat more respectable, the environment in DC (or London or Paris) could be suitable for this sort of historically unremarkable development to be allowed to happen?

This, presumably, is the strategy the Hestia guys are pursuing with Social Matter and Jacobite, and I think it is the right one. We are at a very early stage, and we have a long way to go before a smooth takeover of the United States would be likely, though in the event of some exceptional crisis or collapse, even our immature ideas might have their day. But we do have experimental feedback of the spread of our ideas to people of intelligence and influence: if we had ten Ross Douthats, and ten Ed Wests, and ten Peter Thiels, discussing the same ideas and putting them into the mainstream, we would have visible progress towards achieving our goals.

17 June 2017

Trophic Cascade

I've been blogging for 13 years, and my first post was about Islam in Europe :

I believed then that danger of Islam was exaggerated, by people who I normally agreed with such as Eric Raymond

I've changed my view on many things since then, from being a by-the-book Libertarian to something I had to find a new name for.

Only one thing that I wrote back then is definitely now not true:
The Muslim immigrants to Britain are integrating slowly into British culture.

This 2005 piece by me comes off looking especially bad now

This does not mean that Islam is dying out, just that, like Christianity, it is evolving into a form that makes less conflict with the practicalities of living in a developed society. I expect that in a hundred years Moslems will continue to recite the Koran and observe Ramadan, but what I am calling the "primitive" elements -- intolerance of Western practices of commerce, sexual behaviour, freedom of expression, whatever -- will have died out.

Among Moslems in the West, as well as the more Westernised Moslem countries like Turkey, this is already the case for the majority. And this is why the "primitives" are angry.

File that under "overtaken by events." I did say then that it was more important for the West to be seen to win in Iraq than to achieve anything concrete, so maybe if that had been done then things would look different today. Perhaps what I predicted was at that time still possible, but whether I was wrong about that or not, the reality today is utterly different. It is moderate Islam that is declining, globally, not Islamism.

"Integration" now going backwards. Possibly that had already begun in 2004 and I hadn't noticed, but I suspect it is something new.

Many of my online homies say that "moderate Islam" is a myth or mirage -- that the history of Islam shows that it is inherently and inevitably violent and expansionist. Pitched against liberals who say that Christianity has an equally violent and aggressive history, they certainly have the better of their argument. But while the leftists are ignoring everything before the 1800s, the rightists are ignoring everything since. There was very little Islamist violence in the 20th Century. The Partition of India was a free-for-all. The major Islamic states, Egypt and Turkey, were secular socialist-nationalist in character.

Contrary to my previous assertions, the situation is getting worse not better, but it is still noticeable that Islamist terrorists in Britain are not in their national origins representative of Britain's Muslim population. The ringleader of the 2005 train bombers was from a typical British-Pakistani background, but most of the others have come from Africa or the Middle East. Even Butt seems atypical since he came to the country as a refugee -- most British Pakistanis did not come as refugees, but as Commonwealth migrants back in the 70s and families thereafter. Britain has been granting asylum to very few Pakistanis -- 77 in the last quarter [pdf] .

Pakistani immigration was encouraged for economic reasons up until 1971, and since then it has been family-based. However, their numbers have increased tenfold over those 45 years, from 120,000 to 1.2 million. That's plausible as bringing in existing family members plus marrying more and having two generations of children, but it's towards the high end of what you would estimate. If there's another significant contributor to that tenfold expansion I don't know what it is. 

Striking as those numbers are, my point is that those "normal British Pakistanis" are not the Islamic terrorists in Britain. They really are the "moderate Muslims" that are alleged not to exist (The child prostitution gangs such as the Rotherham one, on the other hand, are exactly from that typical background, one reason why I see that as a totally separate issue). My biggest worry is that by adding significant numbers of African and Middle Eastern jihadis into the mix, the whole British Pakistani culture could be shifted. The Muslim population of Britain doubled between 2005 and 2015 (per Ed West)  and the non-Pakistani Muslim population was probably multipled several times. This was the effect of the "rubbing noses in diversity" -- the Labour government changing the demographics of the country not even out of strategy but out of vulgar spite. That was a development I failed to imagine.

Waiting for Islam to become more moderate is no longer on the table. Forcing Islam to become more moderate is, I believe, thoroughly achievable with sensible policies. The fundamental is for law and society to be at least as tough on expression of tribalism from Muslims as they are on expression of tribalism from natives. This is currently very far from the case. I try to stay out of day-to-day politics, so when I retweet other right-wingers, it's usually because they're highlighting this disparity:

Twitter Moment

The other side of that is this story: In Germany, Syrians find mosques too conservative

Mosques in Western countries are now more extremist than those elsewhere in the world. This is a straightforward holiness spiral -- within a community, you can gain status by professing stronger allegiance to that community's symbols than anyone else does. In a functioning community, this tendency is moderated by the practical demands of society. But, even the large, stable, Pakistani communities in Britain are not truly functional -- they are subsidised and supported by the wider society.

The wider society -- the liberal West -- is deeply opposed to putting any restraint whatsoever on the puritanism growing within the community. They are like the naive conservationists of the past who believed that by keeping out all predators they were allowing an ecosystem to flourish naturally, when in fact they were unbalancing it towards a destructive tipping point. It is natural and universal for religious extremism to come into conflict with its neigbours and be pushed back by them.

Basically, what I'm saying is that Tommy Robinson is a natural predator, and by suppressing him, liberal society is producing a Trophic Cascade in the extremist ecosystem.

It's not only in a minority community that this mechanism should happen. I asked on Twitter, is there any Islamic country where the mosques are not subject to state supervision of doctrine? In majority Islamic communities, the pushback in favour of practicality comes from the state. Again, a liberal Western state disclaims any responsibility for pushing back on Islam, though it is a job that I understand most Islamic states consider necessary.

Update: It should go without saying that continuing to increase the Muslim population is also destabilising. As well as increasing the imbalance, in itself it is a sign of weakness which makes extremism more attractive and moderation less attractive. I am not saying any more than that it is not (yet) necessary to undertake more drastic measures such as mass deportations of long-standing residents. Since the continued importation of Muslims is the same political process as the active protection of extremism from its natural opposition, ending one means also ending the other.