Jabber

Another week is has passed and the first evaluation phase slowly approaches. While I already fulfilled my goals (Jingle File Transfer using InBandBytestreams and SOCKS5Bytestreams), I still have a lot of work to do. The first working implementation I did is only so much – working. Barely. Now its time to learn from mistakes I made while I constructed my prototype and find better ways to do it in the next iteration. This is what I was up to in the past week and what will keep me from my usual sleep cycle for the coming week(s).

I spent the past week doing ground work and writing utility classes which will later allow me to send jingle actions in a clean way. The prototype implementation had all constructions of Jingle elements inside of the control flow, which made reading the code very hard. This will change in the next iteration.

While I worked on my implementation(s), I detected some errors in the XEPs involved and created pull requests against the xsf/xeps repository. In other spots I found some unclarities, but unfortunately my questions on the xsf chat were left unanswered. In some cases I found the solution myselves though.

Also I began upstreaming some changes and additions to the Smack repository. Parsers and elements of IBB have already been merged, as well as some more additions to the HashManager (XEP-0300) I created earlier, and some tests and fixes for the existing Jingle framework. Still open are my PR for SOCKS5 parsers and the first parts of the Jingle file transfer package.

I also dedicated a tiny little bit of my spare time to a non-GSoC project around a blog post on how to create an OMEMO capable chat client using Smack in less than 200 lines of code. The source code of the example application can be found in the FSFE’s brand new git repository. Unfortunately I also found a small bug in my OMEMO code that I have to fix sometime in the next weeks (nothing crucial, just some annoying faulty behavior). I plan to spend the coming week working on my Jingle code, so that I have a mostly working framework when the evaluation phase begins. Thats all for now. Happy Hacking

Hello all,
I am Tarun, a senior year student at IIIT Hyderabad, India. This year I was selected as a GSoC student by XMPP Standards Foundation to work on Mediated Information Exchange (MIX), which is intended as a replacement for Multi-User Chat (MUC). I'm being mentored by Tobias and Edwin. Let me begin by explaining why do we need MIX as a replacement for MUC ?

• In the years after MUC was designed, both Publish-Subscribe and Message Archive Management have been developed and it is desirable to reuse these building blocks (e.g., MAM can be used for message history) rather than using the less robust methods defined in Multi-User Chat .
• It is difficult to use MUC for building multimedia applications without undesirable adaptations.
• A number of use cases has emerged in group communication, which are explained here.

I have started by implementing elements (payloads), which will be coming in / send out in the form of XML. Therefore, we need parsers and serializers for each payload to be able to parse the XML and set the private variables for the Element. On the other hand, Serializers serializes an instance (of the Element) to XML format. I am done with some of the elements with their parsers and serializers and unit tests testing the XML's mentioned in the XEP as follows:

• Join Element (and Subscribe): This element will be used by clients and their local servers to join a channel.

<join xmlns='urn:xmpp:mix:0' channel='coven@mix.shakespeare.example'>    <subscribe node='urn:xmpp:mix:nodes:messages'/>    <subscribe node='urn:xmpp:mix:nodes:presence'/>    <subscribe node='urn:xmpp:mix:nodes:participants'/>    <subscribe node='urn:xmpp:mix:nodes:config'/></join>

• Participant Element: This element will be used to store user nick and their real JIDs.

<participant xmlns='urn:xmpp:mix:0'>    <nick>thirdwitch</nick></participant>

• User Preference Element: This element will be used to set user preferences like their JID visibility, private messages, etc.

<user-preference xmlns='urn:xmpp:mix:0'> <x xmlns='jabber:x:data' type='result'>    <field var='FORM_TYPE' type='hidden'>        <value>urn:xmpp:mix:0</value>        </field>        <field var='JID Visibility'>            <value>never</value>        </field>        <field var='Private Messages'>           <value>allow</value>         </field>        <field var='vCard'>          <value>block</value>        </field> </x></user-preference>

• Leave element: In order to leave a channel, a user sends a MIX "leave" command to the channel.

<leave xmlns='urn:xmpp:mix:0' channel=`coven@mix.shakespeare.example`/>That's all for this blog post. This week I'll be completing other elements and service discovery feature. 

Hello,

How was going this week? Well, it took a bit longer to merge code from last PR into Spark's base code, but now there are also buttons for showing certificates (earlier it was shown only after double click on table) and button to upload certificate. Other thing that occurs more complicated that I initially thought, is extracting certificates extensions. Java's X509Certificate class provide methods that return extensions OID (object identifier) for critical and noncritical extensions. It provides also method getExtensionValue( OID ) which returns extensions but encoded. Using some of the Bouncy Castle classes I am able to decode some of this values but they have different structures to which I have to adjust extracted values and map the names of the elements in the structure. Unfortunately some of the values are still enigmatic for me or I can't decode them well yet. On the other hand when I will solve this problems similar work can be done for Openfire to show certificates extensions also there.

 See you next week,Paweł

ejabberd 17.06-beta includes a lot of improvements over the previous 17.04 release. To name the most important ones: new caching system, Riak support for several modules and introduction of Certificate Manager.

Certificate Manager is a feature that has been requested by many organisations, allowing administrators to manage their certificate more easily. From now, starting ejabberd with an invalid certificate will dump a clear entry in ejabberd log file, explaining what’s wrong. Upcoming ACME support will further refine these improvements we’ve worked on early this year to give our users a great experience with certificate management.

The new cache system is also a new component that allows fine tuning of ejabberd performance for either small systems or large scale servers. To use data cache for a supported module, you need to set the module option use_cache. You also have the possibility to define a maximum number of cache entries and/or maximum life time of cached data, so you keep control on your memory use. Example:

modules: mod_roster: use_cache: true cache_size: 10000 cache_life_time: 3600 # 1 hour

The cleanup tasks on all ejabberd API also continue, consider checking against few methods rename.

Some features and fixes are missing from the 17.06 milestone, and will be worked on the next couple of weeks.

Changes API

• Deprecate misc:encode_base64/1 and misc:decode_base64/1
• Rename is_user_exists function to user_exists
• Allow api access on both ipv4 and 6 loopback addresses

Admin

• Refactor ejabberdctl
• Improve ejabberdctl parameters parsing

Configuration

• Validate module options on start_module/2
• Validate new options before module reloading
• Validate second-level options
• Introduce iqdisc global option
• stream_management listen option deprecated, use mod_stream_mgmt
• Check presence of some files during option validation
• Speedup configuration options lookup
• Validate all certfiles on startup
• Only validate certfiles if public_key:short_name_hash/1 is available
• Introduce Certficate Manager

Commands

• Add clear_cache admin command
• Parse correctly presence_broadcast option in change_room_option command
• Describe command arguments and results in mod_muc_admin
• Improve export2sql explanation; remove obsolete and duplicated command
• Fix and document push_roster_all command

Compilation

• Erlang 17.5 or higher is required
• Add --enable-system-deps configure option
• Add --enable-stun and --enable-sip configure options

Core

• Speedup Mnesia tables initialization
• Improve Mnesia tables creation and transformation
• Improve ejabberd_c2s:close()
• ejabberd_c2s: Don’t close session on stream resume
• Speedup loading of translation files
• Fix ejabberd_router:is_my_route/1

Databases

• New sql_connect_timeout option
• New sql_query_timeout option
• Get rid of sql_queries.erl
• Use round-robin algorithm when selecting worker from DB pool
• Add Riak as BOSH RAM backend
• Add Riak as mod_proxy65 RAM backend
• Add Riak as mod_carboncopy RAM backend
• Add Riak as router RAM backend
• Add Riak as session manager RAM backend
• Fix cleaning of Riak route table

Cache

• Implement cache for mod_announce
• Implement cache for mod_private
• Implement cache for mod_privacy/mod_blocking
• Implement cache for mod_last
• Implement cache for mod_vcard and mod_vcard_xupdate
• Implement cache for roster
• Add cache options to the validator
• Use cache for authentication backends
• Use new cache API in mod_shared_roster_ldap
• Use new cache API in ejabberd_oauth
• Use new cache API in mod_mam
• Use new cache API in mod_caps
• Use cache in front of Redis/SQL RAM backends

Modules

• mod_http_upload: Add support for HTTP File Upload 0.3.0
• mod_mam: Added export function
• mod_metrics: Don’t leak with UDP sockets
• mod_metrics: New options ip and port
• mod_muc: Allow a room admin to un/subscribe another JID
• mod_offline: Don’t store messages via a single process
• mod_offline: Make sure only jabber:x:event tag is present in offline event
• mod_register: New option ‘access_remove’ ACL
• mod_stream_mgmt: Preserve stanza count on timeout
• mod_vcard_ldap: Parse ldap_uids like in eldap_utils

Elixir

• Update elixir to v1.4.4

Installer

• Upgrade OTP to 19.3

Feedback

As usual, the release is tagged in the Git source code repository on Github. The source package and binary installers are available at ProcessOne. If you suspect that you’ve found a bug, please search or fill a bug report on Github.

The german interior minister conference recently decided that the best way to fight terrorism is passing new laws that allow the government to demand access to communication from messengers like WhatsApp and co. Very important: Messengers like WhatsApp. Will even free software developers see requests to change their messengers to allow government access to communications in the future? If it comes so far, how are we then still possible to protect our communications?

The answer could be: Build your own messenger. I want to demonstrate, how simple it is to create a very basic messenger that allows you to send and receive end-to-end encrypted text messages via XMPP using Smack. We will use Smacks latest new feature – OMEMO support to create a very simple XMPP based command line chat application that uses state of the art encryption. I assume, that you all know, what XMPP is. If not, please read it up on Wikipedia. Smack is a java library that makes it easy to use XMPP in an application. OMEMO is basically the Signal protocol for XMPP.

So lets hop straight into it.
In my example, I import smack as a gradle dependency. That looks like this:

gradle.build

apply plugin: 'java' apply plugin: 'idea' repositories { mavenCentral() maven { url 'https://oss.sonatype.org/content/repositories/snapshots' } } ext { smackVersion="4.2.1-SNAPSHOT" } dependencies { compile "org.igniterealtime.smack:smack-java7:$smackVersion" compile "org.igniterealtime.smack:smack-omemo-signal:$smackVersion" compile "org.igniterealtime.smack:smack-resolver-dnsjava:$smackVersion" compile "org.igniterealtime.smack:smack-tcp:$smackVersion" } //Pack dependencies into the jar jar { from(configurations.compile.collect { it.isDirectory() ? it : zipTree(it) }) { exclude "META-INF/*.SF" exclude "META-INF/LICENSE" } manifest { attributes( 'Main-Class': 'Messenger' ) } }

Now we can start the main function of our client. We need to create a connection to a server and log in to go online. Lets assume, that the user passes username and password as arguments to our main function. For sake of simplicity, we’ll not catch any errors like wrong number of parameters etc. Also we want to get notified of incoming chat messages and we want to send messages to others.

Messenger.java

public class Messenger { private AbstractXMPPConnection connection; private static Scanner scanner; public static void main(String[] args) throws Exception { String username = args[0]; String password = args[1]; Messenger messenger = new Messenger(username, password); scanner = new Scanner(System.in); while(true) { String input = scanner.nextLine(); if (input.startsWith("/quit")) { break; } if (input.isEmpty()) { continue; } messenger.handleInput(input); } } public Messenger(String username, String password) throws Exception { connection = new XMPPTCPConnection(username, password); connection = connection.connect(); connection.login(); ChatManager.getInstanceFor(connection).addIncomingListener( (from, message, chat) -> System.out.println(from.asBareJid() + ": " + message) ); System.out.println("Logged in"); } private void handleInput(String input) throws Exception { String[] split = input.split(" "); String command = split[0]; switch (command) { case "/say": if (split.length > 3) { String recipient = split[1]; EntityBareJid recipientJid = JidCreate.entityBareFrom(recipient); StringBuilder message = new StringBuilder(); for (int i=2; i<split.length; i++) message.append(split[i]); ChatManager.getInstanceFor(connection).chatWith(recipientJid).send(message); } break; } } }

If we now compile this code and execute it using credentials of an existing account, we can already log in and start chatting with others using the /say command (eg. /say bob@marley.jm Hi Bob!). But our communications are unencrypted right now (aside from tls transport encryption). Lets change that next. We want to use OMEMO encryption to secure our messages, so we utilize Smacks new OmemoManager which handles OMEMO encryption. For that purpose, we need a new private variable which will hold our OmemoManager. Also we make some changes to the constructor.

Messenger.java

private OmemoManager omemoManager; public Messenger(String username, String password) throws Exception { connection = new XMPPTCPConnection(username, password); connection = connection.connect(); connection.login(); //additions begin here SignalOmemoService.acknowledgeLicense(); SignalOmemoService.setup(); //path where keys get stored OmemoConfiguration.setFileBasedOmemoStoreDefaultPath(new File("path")); omemoManager = OmemoManager.getInstanceFor(connection); //Listener for incoming OMEMO messages omemoManager.addOmemoMessageListener(new OmemoMessageListener() { @Override public void onOmemoMessageReceived(String decryptedBody, Message encryptedMessage, Message wrappingMessage, OmemoMessageInformation omemoInformation) { System.out.println("(O) " + encryptedMessage.getFrom() + ": " + decryptedBody); } @Override public void onOmemoKeyTransportReceived(CipherAndAuthTag cipherAndAuthTag, Message message, Message wrappingMessage, OmemoMessageInformation omemoInformation) { //Not needed } }); ChatManager.getInstanceFor(connection).addIncomingListener( (from, message, chat) -> System.out.println(from.asBareJid() + ": " + message) ); omemoManager.initialize(); //additions end here. System.out.println("Logged in"); }

Also we must add two new commands that are needed to control OMEMO. /omemo is similar to /say, but will encrypt the message via OMEMO. /trust is used to trust an identity. Before you can send a message, you have to decide, whether you want to trust or distrust an identity. When you call the trust command, the client will present you with a fingerprint which you have to compare with your chat patner. Only if the fingerprint matches, you should trust it. We add the following two cases to the handleInput’s switch case environment:

Messenger.java

case "/omemo": if (split.length > 2) { String recipient = split[1]; EntityBareJid recipientJid = JidCreate.entityBareFrom(recipient); StringBuilder message = new StringBuilder(); for (int i=2; i<split.length; i++) message.append(split[i]); //encrypt Message encrypted = null; try { encrypted = OmemoManager.getInstanceFor(connection).encrypt(recipientJid, message.toString()); } // In case of undecided devices catch (UndecidedOmemoIdentityException e) { System.out.println("Undecided Identities: "); for (OmemoDevice device : e.getUntrustedDevices()) { System.out.println(device); } } //In case we cannot establish session with some devices catch (CannotEstablishOmemoSessionException e) { encrypted = omemoManager.encryptForExistingSessions(e, message.toString()); } //send if (encrypted != null) { ChatManager.getInstanceFor(connection).chatWith(recipientJid).send(encrypted); } } break; case "/trust": if (split.length == 2) { BareJid contact = JidCreate.bareFrom(split[1]); HashMap<OmemoDevice, OmemoFingerprint> fingerprints = omemoManager.getActiveFingerprints(contact); //Let user decide for (OmemoDevice d : fingerprints.keySet()) { System.out.println("Trust (1), or distrust (2)?"); System.out.println(OmemoKeyUtil.prettyFingerprint(fingerprints.get(d))); int decision = Integer.parseInt(scanner.nextLine()); if (decision == 1) { omemoManager.trustOmemoIdentity(d, fingerprints.get(d)); } else { omemoManager.distrustOmemoIdentity(d, fingerprints.get(d)); } } } break;

Now we can trust contact OMEMO identities using /trust bob@marley.jm and send them encrypted messages using /omemo bob@marley.jm Hi Bob!. When we receive OMEMO messages, they are indicated by a “(O)” in front of the sender.
If we want to go really fancy, we can let our messenger display, whether received messages are encrypted using a trusted key. Unfortunately, there is no convenience method for this available yet, so we have to do a small dirty workaround. We modify the onOmemoMessageReceived method of the OmemoMessageListener like this:

Messenger.java

@Override public void onOmemoMessageReceived(String decryptedBody, Message encryptedMessage, Message wrappingMessage, OmemoMessageInformation omemoInformation) { //Get identityKey of sender IdentityKey senderKey = (IdentityKey) omemoInformation.getSenderIdentityKey().getIdentityKey(); OmemoService<?,IdentityKey,?,?,?,?,?,?,?> service = (OmemoService<?,IdentityKey,?,?,?,?,?,?,?>) OmemoService.getInstance(); //get the fingerprint of the key OmemoFingerprint fingerprint = service.getOmemoStoreBackend().keyUtil().getFingerprint(senderKey); //Lookup trust status boolean trusted = omemoManager.isTrustedOmemoIdentity(omemoInformation.getSenderDevice(), fingerprint); System.out.println("(O) " + (trusted ? "T" : "D") + " " + encryptedMessage.getFrom() + ": " + decryptedBody); }

Now when we receive a message from a trusted identity, there will be a “T” before the message, otherwise there is a “D”.
I hope I could give a brief introduction on how to use Smacks OMEMO support. You now have a basic chat client, that is capable of exchanging multi-end-to-multi-end encrypted messages with other XMPP clients that support OMEMO. All took less than 200 lines of code! Now its up to you to add additional features like support for message carbons, offline messages and co. Spoiler: Its not hard at all
You can find the source code of this tutorial in the FSFE’s git repository. When the government is unable or simply not willing to preserve your privacy, you’ll have to do it yourself. Happy Hacking