[messaging] Matrix.. is Federation at odds with Privacy?

[carlo von lynX]

On Fri, Apr 17, 2015 at 12:49:02PM +0100, Matthew Hodgson wrote:
> However, I'm afraid the world is not so black and white - there are
> also scenarios where it's frustrating for users that their contacts
> are stuck in different apps.  For instance,
> Viber/WhatsApp/Messenger/Kik/WeChat/LINE are all incredibly similar,
> and I have friends on all of them.  Forcing everyone to install all
> apps and having group chats scattered across them is creating a
> completely horrid UX for end users.  Users should have freedom to
> choose their preferred UX if they desire.

But again, the companies you mention have no interest in
making themselves interchangeable unless they have a strong
reason to believe they will be the winners of such a change.
And users.. well.. users cannot have what doesn't make the
companies some money - unless some law says so.

> >If people want to reach *any* other user, they pick Facebook or e-mail.
> >An additional federated system stands no chance of getting established
> >here since Facebook is already reducing the importance of e-mail. So
> >the great integrating platforms are already there.
> 
> Again, I think this is an errantly black & white interpretation:
> 
>  * Facebook could easily move from dominance, just like the many
> waves of change that have preceded it.  There's absolutely room for
> new comms technology, whether it obfuscates metadata or federates or
> whatever - so long as it actually differentiates and builds a
> community.

Yes, either by a new platform with outstanding new features that
everyone goes wild about, or because a law changes the rules of the
game, or because all competitors join forces. But not because a small 
company wrote code to implement a proposed open standard - that is
more likely to become the 16th competing open standard.

>  * Facebook may be replacing email for social contact (alongside a
> whole range of technologies like SMS, WhatsApp etc), but email is
> here to stay just as much as the PSTN as a genuinely ubiquitous
> federated technology for a huge range of other use cases - for
> instance, it's literally still the only standard federated
> technology out there for businesses to exchange data!

Sure, even IRC isn't dead yet, although the usage numbers are
below 0.01% of the worldwide chat market. I know twenty-year-olds
that have *FIVE* addresses in their email address book. They only
use it because it is being forced upon them by *FIVE* external
authorities. Everything else happens elsewhere.

Sad, but a reality we should start come to terms with.

> Again, *some* trendsetting users seek out fun new silos to
> trailblaze (e.g. crypto enthusiasts proudly enthusing about their
> latest metadata-obfuscating messaging app ;)
> 
> The majority of the rest of the population get pulled behind, trying
> to keep up, and it creates a problem of creeping terrible UX;
> inextensible and closed communication platforms; vendor lock-in; and
> is completely contrary to the idea of data liberation and letting
> users pick the apps & services *THEY* want.

Yes, you said the same thing. And yet this is what happens most of
the time, since the idea of data liberation isn't glued into the
rules of the market. It's just in some idealist heads.

> >And among all the huge problems we have with the Internet.. the
> >threats it is posing to democracy (see Assange/Appelbaum/Maguhn/Zimmermann
> >2012), crying about the cool kids having forced you into installing
> >5 different chat apps is quite a distraction in the area of irrelevance.
> 
> Well, I genuinely applaud you & GNUnet for your mission to give
> folks the ability to protect themselves from malicious governments.

No no don't applaud me since we're still not delivering as much
as many others have delivered and especially I should be doing
less talking. Or maybe that's my job but anyhow...

> Meanwhile, Matrix's mission is to provide a simple extensible
> platform to allow realtime data interchange with decentralised
> persistence: basically a read/write Web with pubsub.  And just as
> the Web itself is huge vehicle for social justice and democracy
> (even without metadata obfuscation and even without crypto!!), I
> believe Matrix can make the world a better place too.

But, if it is without metadata obfuscation and without crypto,
then it doesn't need a complicated multicast pubsub scheme over
federation. It can simply use the cloud! You can use two clouds
for redundancy, but you really don't need to re-invent cloud
technology in a federated manner if you have nothing to hide.

> This is simply orthogonal to metadata privacy: if folks want
> metadata privacy they should use a different system (for now).  And
> hopefully down the line Matrix can sprout metadata-protecting
> flavours (just as PSYC1 is evolving to PSYC2), or we can work out a
> way to make PSYC2 and Matrix play nice together whilst not
> compromising the security model, or similar.
> 
> But I don't buy that metadata privacy should be table stakes for any
> new communication tech.  Sure, it's a great differentiator for
> GNUnet.  But there are other useful features users might value too.

That's kind of back to my initial question. Trying to understand
what use is left for federation. If you're not about privacy, you're
better off using the cloud.

> >I don't know of any case where this old federation legend actually
> >proved true. Did federation ever help people getting out of the
> >stranglehold of a specific service provider? Migrating your email
> >address is a pain in most cases, even if you pay for .forward services -
> >you still have large parts of your social surroundings using that old
> >address rather than your current one. XMPP has never had a functioning
> >and generally implemented protocol for migrating accounts. PSYC at least
> >has the necessary _redirect_permanent message code, but even we left
> >the implementation of that in the TODO file.
> 
> This is a failure of old-school federation systems like
> SMTP/XMPP/PSYC.  To be honest, we've also punted on the
> migration/porting question in v1 of Matrix (but it's on the radar
> for v2 or v3).  It's worth noting that it's a *really* useful
> feature (however painful the process is) for consumers on GSM.

Wait wait... please consider the migration scenario in a distributed
system. You move your configuration files from one device to another.
The new device announces your public key to the network. You have
already completed the migration.

Now explain again how ANY federation migration will ever be as
simple as that. Ever. And don't say PSYC failed at that just because
nobody cared to implement it. At least we created the preconditions. ;)

> It's worth noting that Matrix's semantics of federation really are
> nothing like SMTP and XMPP.  The building block in Matrix is
> *synchronising history*, not message passing.  This makes both
> interoperability and federation much more compelling: if I'm
> basically using different apps as different UI/UX for viewing the
> same decentralised conversation database, the motivation to move
> between apps (or services, in future) and pick the best app becomes
> much stronger. Just like users love picking their preferred email
> client or GSM handset, I expect them to love picking their favourite
> Matrix client... without losing their identity or conversation
> history.

Yes, same here with distributed pubsub and state. We are coding the
same stuff. In our case it doesn't run on company servers and users
can change UIs as they like. In your case they depend on companies
to play by the rules as there probably are several possibilities to 
practice lock-in.

> Have you ever seen how painful it is to migrate users to new
> intranet tools?  Not to mention that the world of business (by which

You mean against their will?

> i basically mean 'professional interaction') is dedicated to
> millions of different islands (organisations) desperately trying to
> collaborate on different projects, and discovering that email is
> still about the only thing they actually have in common which lets
> them own and manage their own IP/data.

Yes. And since humanity hasn't solved this one, it should try
with metadata protection in place from the start...  ;)

> >>Well, Hangouts & FB Messenger are both WebRTC-based these days, so I
> >>don't have to wait for a new Faceboogle :)
> >
> >Why on Earth should they care to interop? One of them is the stronger
> >and has an interest in draining the other. And they both have no
> >interest in letting small businesses have a piece of the cake.
> 
> The monomania of trying to compete with FB via G+ seems to be
> fading.

No no, it's not about social networks. It's about the time you
spend on *their* web pages with *their* ads embedded. So why
should they let anyone else have a piece of the web-telephony
market? You sound like you're trying to defy basic laws of
capitalism.

> Sure, everyone and their dog can design their own signalling scheme.
> And if you categorically will never need persistence, federation,
> e2e crypto, or any of the other goodies that Matrix provides, then
> perhaps it'll be quicker to write your own rather than use an
> existing library.  Arguing that "anyone can invent their own library
> if it serves their purpose" seems a little specious ;)

Well, I am arguing that either Faceboogle will always opt for
offering *all* of those features out of their own cloud backbone
rather than interopping with strangers that might suddenly attract
attention and steal a piece of the cake. And I especially like
the "e2e crypto" there in your list, knowing that no stateless
federation (aka the web and its webrtc) architecture can guarantee 
REAL end-to-end crypto to people.

> So, the reason I came across PSYC however many years ago was
> (briefly) running a psyced for precisely this purpose.  I think the
> main reason why it had limited uptake is that the UI/UX of the app

What UI? People were using either IRC or XMPP for something the
respective protocols weren't designed for, so we added some dirty
hacks allowing you to send command lines to the psyced. There was
no UI there and thus no UX.

The first real PSYC clients only materialized about ten years ago
and they were written in Macromedia Flash. I presume you never saw
one of those. The early clients were all command-line oriented
because we hadn't fleshed out the parts of the protocol necessary
to make buttons, panels and menues.

> felt aggressively non-mainstream: an extreme poweruser tool without
> any reassuring visible end-user facing benefits/community/glossiness

Well it was federation at its max: letting you idle on as many servers
as you liked, allowing everyone to run their own server and host chats.
Like IRC, but without the oligarchy and the chanwars - like XMPP, but
with better chatroom control as IRCers expect.

As I said, stuff nobody really needs. Federation. We've been there,
we bought the t-shirt, now we know it was badly spent time. We still
use it each day as our chatrooms are among the best in signal to noise
ratio, but that's just because secushare isn't ready for use.

> to showcase its benefits on make anyone but those 1000 geeks want to
> use it.  Also I seem to remember that there were some weird
> impedance mismatches between the various protocols - I forget the
> specifics now, but I ended up writing off the XMPP<->PSYC<->IRC
> bridging as an interesting experiment that wasn't really intended
> for primetime, and that the majority of the effort looked to be
> going into the pure-PSYC side of things.

Of course. IRC and XMPP are fundamentally incompatible and psyced
could not possibly resolve that elegantly other than by asking
you to install a native PSYC client.

> I agree that regulation is one way to force folks to use an open standard.
> 
> But the assertion that open standards formed to attack market
> leaders will always suck and lose relevance is just bogus.  If a

Oh yeah? ISPs formed an alliance and kicked out Compuserve & co.
When Netscape was bossing the web, Microsoft got in bed with
everyone at W3C until the web was p0wned. They stopped chanting
the open standards refrain on the day IE took over the lead.

XMPP came up when AIM/ICQ and MSN were dominating the chat market.
Funny that Facebook later won that race by adding a webchat. Its
support for XMPP clients is merely political cosmetics.

In the meantime Google has locked in millions of email users into
their Gmail cloud offering. Even if people later decide to get out
of Gmail again, will all of their friends seriously stop sending 
mail to gmail.com?

So when exactly did an open standard ever get anywhere if there
wasn't a reason for a number of actors to form an alliance against 
some market leaders? I can't think of any such scenario. Ah right.
Whenever some legislator imposed a standard by law.

> standard exists, works, and it brings value to members of a
> community, they will use it.  Whether it takes over the whole world

Community? What is the community? Idealist developers that can talk
their companies into doing something for a while? Independent idealists
that spend their spare time? You chose to compete in a capitalist
market - how does your "community" have any influence in that?
Companies make the market move and they are tied to its rules -
the "community" isn't even a large number of consumers capable of
having some power as consumers, and they aren't organized enough to
have much influence on the actual mass of consumers, either. They
may at best influence media coverage for a bit.

> or not is another story of course, but the whole internet owes its
> existence to IP federation between organisations, just as much of
> the modern Web exists thanks to HTTP API federation between
> organisations.

Sure, the Internet was created in times when it wasn't under the
harsh rule of capitalism yet. I know, because I've felt the shift.
HTTP is not a federation as 99.x% webservers do not interact among 
each other. Some people have done some apps that use HTTP as a 
federation platform, but I don't know of any that are of relevance to 
the market. RSS pings? No. OStatus? No.

> Providing competent tools for building extensible decentralised
> federated platforms like IP and the web is something worth fighting
> for.

Even if it will not affect much, like it hasn't since 1995?

> >And who is the market leader that would motivate Google and Apple to
> >team up with a common standard?
> 
> Perhaps Facebook.  Perhaps the long-tail of next-generation
> WebRTC-backed solutions which happily interoperate via Matrix,
> leaving Google & Apple to really look like dinosaurs who refuse to
> join in the party.

Yes, maybe. Unless they stick to the XMPP which is already doing
the job. Or maybe they come up with something new because they don't
want to put any small company in power of defining the "open" standard.
Again, the companies involved are making the rules and the protocols
they will pick are their playthings. Of course you could just be lucky
by being in the right place at the right time. There is a fringe chance
there.

> Just as even Microsoft eventually ditched X.400 in favour of SMTP
> when it became apparent that SMTP had won the mail protocol war,

Yes, before commerce was on the Internet. Internet just once was able 
to take some folks by surprise and impose itself over ISO standards.

> it's possible to consider that Apple might open up FaceTime if its
> users were voting with their feet and buying other hardware because
> it seemed archaic and over-controlling to be trapped in the FaceTime
> silo.

Yes, if FaceTime popularity decreases they have all reasons to open
up and look for some alliance, but that means somebody else is 
dominating the game and NOT opening up.

> >Who is the regulatory body that
> >would enforce such a standard on them?
> 
> A forward-looking government who wants to avoid monopolies or
> protect citizens from vendor-lock-in might consider enforcing
> interoperability.  Just as telco regulators enforce GSM portability.

Yes, and if they understand the implications for the future of
democracy they should also mandate metadata protection with it
as the law proposal on http://youbroketheinternet.org suggests.

> Otherwise, hopefully, the ecosystem of smaller and newer players
> will make the most of Matrix and showing that consumers /can/ care
> about interoperability if it's clearly linked to improved UX, and
> eventually Google/Apple will reconsider.

I don't see standards improving UX. Since you are forced to decide
upon certain standard behaviours you are giving up flexibility.
Whatever you manage to improve over existing silo apps can be retro-
actively integrated by them - but it is much harder for you to
retroactively integrate cool features the silos have come up with
when you already made some choices and have a codebase spread over
many different places.

> >Get friends with regulatory bodies. I don't think anything else can
> >force companies into playing this game.
> 
> It certainly wouldn't hurt.  But the other body who can force
> companies into playing this game are the end-users. *IF* one can
> find a way to show clear benefit to doing so.  And yes, this is
> hard, but just because everyone else has failed so far doesn't mean
> that we shouldn't try :)

You can lure consumers into focusing on certain features of a product,
but not really into the abstract notion of federation. Especially if
certain promises like being independent and in control of your data
just aren't true at all. Whoever has been trying that won't believe
this lie again. Consumers see their data going to servers and having
the choice between the devil and the beelzebub. So to them there just
isn't a real difference between a silo or a company that says it is
openly federated. It's always folks trying to sell them something.

> Perhaps we'll get there in the end :)

You can spare yourself a whole lot of work if you skip the
dead-end-street called federation.

But hey, I'm still curious if anyone on this list can come up with a
credible scenario for the federation architecture other than as an
alliance against a market leader or as a governmental regulation.


-- 
  E-mail is public! Talk to me in private using Tor.
  torify telnet loupsycedyglgamf.onion		DON'T SEND ME
          irc://loupsycedyglgamf.onion:67/lynX  PRIVATE EMAIL
         http://loupsycedyglgamf.onion/LynX/    OR FACEBOOGLE