Secure Gmail Authentication with X-OAUTH2 SASL

In July 2014 Google has introduced the “More Secure Gmail Authentication feature”.
When Access for less secure apps is set to disabled then Google does not allow login over SASL PLAIN anymore (even on TLS).

During authentication Google replies with the not-authorized error. And they will notify you by email that access to an insecure application was denied.

CLIENT: <auth mechanism="PLAIN" xmlns="urn:ietf:params:xml:ns:xmpp-sasl">AUTH DATA</auth>
SERVER: <failure xmlns="urn:ietf:params:xml:ns:xmpp-sasl">
  <not-authorized />
</failure>

When access for less secure apps is disabled you have to login with the X-OAUTH2 SASL mechanism the Gmail XMPP server offers. Here is a small tutorial how to achieve this with MatriX.

1) API Project

In order to access the Google API you have to setup a project in the Google API Console which provides you the Access credentials. Once you have created your API project you can request your credentials which include the Client Id and Client Secret. During the creation process of the app project we selected “native application”.

2) Request an Access Token

To authenticate with OAuth 2.0 we need to get our Access Token. The OAuth 2.0 is documented at the Google pages under the following 2 links:

There are also several .NET libraries available, e.g DotNetOpenAuth or Google APIs Client Library.

We are not going to use a third party library for now, because this help us also to understand the whole XAuth authentication flow.

We create a new form and put a Webbrowser control on it. When the form loads we have ro navigate to a Url which includes;

  • Client Id of our API project
  • A redirect Uri, we use this urn:ietf:wg:oauth:2.0:oob
  • scopes:
    • https://www.googleapis.com/auth/googletalk
      (required for XMPP authentication)
    • https://www.googleapis.com/auth/userinfo.email
      (optional, we can use this t get the username/email address of the user)

When our form with the webbrowser loads we navigate to this Url with teh following code:

const string CLIENT_SECRET  = "XXXXXXXYYYYYYYYYYYYZZZZZZZZZ";
const string CLIENT_ID      = "yyyyyyyyyyyyy-xxxxxxxxxxxxxxxxx.apps.googleusercontent.com";

private static Uri GetAutenticationUri()
{
    var postData = new List<string>
    {
        "client_id=" + CLIENT_ID,
        "redirect_uri=urn:ietf:wg:oauth:2.0:oob",
        "scope=https://www.googleapis.com/auth/googletalk https://www.googleapis.com/auth/userinfo.email",
        "response_type=code"
    };

    return new Uri("https://accounts.google.com/o/oauth2/auth" + "?" + string.Join("&", postData.ToArray()));
}


private void Form_Load(object sender, EventArgs e)
{
  // navigate to the Uri when the form loads
  webBrowser1.Url = GetAutenticationUri();
}

When you run the code you should see the following: google_xoauth2

3) Authentication

When the user now clicks on “Accept” in the authentication dialog the page in the browser reloads. The new page contains an Authentication code in the title tag. The authentication code is also in the body of the result page. We have seen cases where the code in the title tag was cut off. So it could be more secure to parse it from the body.

private void webBrowser1_DocumentCompleted(object sender, WebBrowserDocumentCompletedEventArgs e)
{
    string docTitle = ((WebBrowser) sender).DocumentTitle.ToLower();
    // when there is "success code=" in the title its the page we are looking for.
    if (docTitle.IndexOf("success code=") > -1)
    {
        webBrowser1.Visible = false; // Hide the webbrowser
        // we can find the auth code in the title tag of the html page
        string authCode = webBrowser1.DocumentTitle.Replace("Success code=", "");
        
        ProcessAuthorizationCode(authCode);
    }
    else if (docTitle.IndexOf("denied error=") > -1)
    { 
        // user denied the app dialog, handle this here.        
    }
}

4) Exchanging the Authentication Code

Now we have the Authentication code, and can use it to request the access token and refresh token over a simple HTTP Post. The result is in JSON, we use Newtonsoft.Json to process it.

public void ProcessAuthorizationCode(string authCode)
{
    var postData = new List<string>
    {
        "code=" + authCode,
        "client_id=" + CLIENT_ID,
        "client_secret=" + CLIENT_SECRET,
        "redirect_uri=urn:ietf:wg:oauth:2.0:oob",
        "grant_type=authorization_code"
    };

    //Posting the Data to Google
    string responseString = Post("https://accounts.google.com/o/oauth2/token", postData);

    //Processing the JSON Response
    JObject jObject = JObject.Parse(responseString);
    
    JToken refreshToken = jObject.SelectToken("refresh_token");
    // Checking for errors.
    if (refreshToken == null)
    {
        //this._ErrorMessage = response;
    }
    else
    {
        RefreshToken = refreshToken.ToString();
        AccessToken = jObject.SelectToken("access_token").ToString();
        // close the browser Windows, we are done
        Close();
    }
}

public static string Post(string url, List<string> postData)
{
    var request = WebRequest.Create(url);

    // Preparing the post data
    var toPost = string.Join("&", postData.ToArray());
    byte[] byteArray = Encoding.UTF8.GetBytes(toPost);

    request.Method = "POST";
    request.ContentType = "application/x-www-form-urlencoded";
    request.ContentLength = byteArray.Length;
    var dataStream = request.GetRequestStream();
    
    dataStream.Write(byteArray, 0, byteArray.Length);
    dataStream.Close();
    try
    {
        // read the response
        var response = request.GetResponse();
        dataStream = response.GetResponseStream();
        var reader = new StreamReader(dataStream);
        string responseString = reader.ReadToEnd();
        
        reader.Close();
        dataStream.Close();
        response.Close();
        
        return responseString;
    }
    catch (Exception)
    {
        return null;
    }
}

5) Getting the username (optional)

The username for XMPP authentication for XOAUTH2 SASL is the users email address. You can either ask the user to enter it in your software. Or you can use your access token to request more user details. This is why we have added the scope https://www.googleapis.com/auth/userinfo.email. This allows us to get the username with a post to the following url:
https://www.googleapis.com/oauth2/v1/userinfo?access_token={your_access_token}

This request returns JSON payload where you could fine the users email address.

6) Authenticate with MartiX

Now we have everything to perform the XOAUTH2 authentication with MatriX. We need to subscribe to the OnBeforeSasl event to manual select the SASL mechanism and pass our access token.

private void xmppClient_OnBeforeSasl(object sender, Matrix.Xmpp.Sasl.SaslEventArgs e)
{
  // make sure we use the correct username for authentication    
  xmppClient.Username = "user12345@gmail.com";
  // disable automatic SASL algrithm selection in MatriX
  e.Auto = false;
  // manual specify the X_OAUTH2 SASL mechanism 
  e.SaslMechanism = Matrix.Xmpp.Sasl.SaslMechanism.X_OAUTH2;
  
  //pass the Access token to MatriX  
  e.SaslProperties = new OAuthProperties()
  {
      AccessToken = "AccessToken"
  };    
}

Unity development with MatriX

A question which comes up very often is the following:

Can I use MatriX with Unity 3d?
Of course you can. Many of our customers are using it with great success in their Unity projects.

Unity is based on Mono. MatriX is designed for cross platform and works on all major .NET platforms, including Mono.

Many Unity developers have some problems to get started with MatriX on Unity.
This post should be a small tutorial and address some of the issues you may be facing during your Unity development.

1) Download the correct MatriX dll (Mono build). MatriX builds are not universal, which means you need to download and reference the MatriX library for Mono.
The Mono build can be found at the latest binary downloads here:
http://www.ag-software.net/download-directory/
when you browse to MatriX => Mono

2) The System.Xml.Linq.dll seems to be not a standard Unity reference. Download it as well and add it to your assets. Otherwise Unity may throw an exception when you run your project.

3) Depending on your Unity version you may get compiler errors on iOS. When you use the latest Unity version this should be no problem. On older Unity versions this problem is related to obfuscation. To resolve this please contact us directly.

4) The iOS stripping feature causes problems. It does not work when reflection gets used. The core Xml serializing engine in MatriX is build on reflection using a factory pattern. You either have to remove stripping from your complete project, or exclude the MatriX classes.
Here is an example of a linker.xml file excluding all MatriX classes:

<linker>
  <assembly fullname="Matrix">
    <type fullname="Matrix.*" preserve="all"/>
  </assembly>
  <assembly fullname="mscorlib">
    <namespace fullname="System.Security.Cryptography" preserve="all" />
  </assembly>
</linker>

5) Some of our users reported that connecting to an XMPP server does not work inside the Unity editor because of some networking problems of the IDE. When the code gets executed or published on a device it works fine. This is a restriction of Unity and not related to MatriX.

If you run into any other problems please contact us directly. We do our best to solve them.

Task-based asynchronous MUC room configuration

The question on how to change the configuration of a Muc room, or how to make a room persistend comes up very often in our forums and support enquiries.

Here is a small example using MatriX and the task based asynchronous pattern.

In MatriX most of the Muc functionality can be accessed with the MucManager component. We assume that you are familiar with the MucManager and have joined or created Muc rooms already.

In this small example we want to make an existing room persistent.
Four steps are required to achieve this:

  1. Request the current room configuration.
  2. Parse the current room configuration in code or render it to a GUI.
  3. Create the xdata result, either in code or from the user input in the GUI.
  4. Submit the new room configuration.

Here is the code:

private async void MakeRoomPersitent(Jid roomJid)
{	
	// Step 1, request the room configuration
	var iq = await mm.RequestRoomConfigurationAsync(roomJid);
	if (iq.Type == IqType.result) // only proceed on result
	{
		// Step 2 and 3, parsing the current config and
		// creating the	result is done in the same loop here.
		var xdata = iq.Query.Element<Data>();
		var xDataResult = new Data
		{
			Type = FormType.submit
		};

		foreach (var field in xdata.GetFields())
		{
			var retField = new Field()
			{
				Type = field.Type, // keep the type
				Var = field.Var // keep the var
			};

			// we are changing the muc#roomconfig_persistentroom only
			// other fields get copied only with the existing values			
			if (field.Var == "muc#roomconfig_persistentroom")
				retField.AddValue(true);
			else
				retField.AddValues(field.GetValues().ToArray());

			xDataResult.AddField(retField);
		}

		// Step 4, submit the changed configuration back to the server (room)
		var submitIq = await mm.SubmitRoomConfigurationAsync(roomJid, xDataResult);
		if (submitIq.Type == IqType.result)
			Debug.WriteLine("success");
		else
			Debug.WriteLine("something went wrong");
	}
}

If you want to render Data froms to UI, and generate responses from the UI input, you can use the sources in the MatriX.Ui project which can we downloaded from here:
http://www.ag-software.net/download-directory/

MatriX Screencast #2 published

The 2nd screencast has been published. This tutorial covers the following:

  • installation of a local Openfire XMPP server
  • configuration of the server
  • creating user accounts
  • starting XMPP client development on the local server

More screencasts and tutorials for other XMPP server software is coming.

NuGet package available

Some of our customers asked for NuGet packages. The roadmap of our development is heavily impacted by our customer requests. We are always receptive to your criticism, ideas and praise

When you prefer NuGet for managing your references then you can start using it for MatriX as well now.

You can find the MatriX NuGet package here:
https://www.nuget.org/packages/MatriX/

MatriX Screencast #1 published

We have published the first MatriX screencast. This small tutorial shows you howto download and install the MatriX XMPP SDK. And howto login with the MiniClient example using your Gmail Id.

You an find the screencasts here. More screencasts are coming soon…

Await MatriX

C# 5.0 brought us the great new Async and Await feature. It helps to avoid bottlenecks when it comes to performance or to improve the responsiveness of UI applications.

Both of them were never a problem in MatriX, because its completely asynchronous internal. We helped GUI developers a lot by handling all multi threading and invoking to GUI threads inside of MatriX.

But the traditional techniques for writing asynchronous code based on callbacks can be complicated, difficult to maintain, hard to debug, error prone, and often ends in the callback hell.

Here is an for requesting a vcard with the new async await API

var viq = new VcardIq { To = "bob@ag-software.net", Type = IqType.get };
var vcardResult = await xmppClient.IqFilter.SendIqAsync(viq);

Here is a more complex example of creating a MUC room containing the following steps.

  • request our reserved nickname
  • create the room with the requested nickname
  • request the room configuration
  • submit the room configuration

With the old async callback based API this simple task requires you to setup multiple callbacks or listen to multiple events. This splits your code in many different functions, or you use many inline lambdas. Error handling gets complicated and the control flow is hard to follow.

So look at the code with the new async await API. It looks like simple synchronous code. Easy to read with simple try/catch error handling, fully asynchronous. The compiler is doing all the complicated work for us.

try
{
    Jid roomJid = "dev@conference.ag-software.de";
    var nick = await mucManager.DiscoverReservedNicknameAsync(roomJid);
    var createResult = await mucManager.CreateRoomAsync(roomJid, nick);
    // check from createResult if the room requires configuration
    if (createResult.Error != null 
        && createResult.Error.Condition == ErrorCondition.ItemNotFound)
    {
        var roomConfig = await mucManager.RequestRoomConfigurationAsync(roomJid);
        var data = ProcessRoomConfig(roomConfig);
        var submitResult = await mucManager.SubmitRoomConfigurationAsync(roomJid, data);  
    }        
}
catch (TimeoutException ex)
{
     // handle exceptions here
}
catch (Exception ex)
{
     // handle exceptions here
}

Many of our customers are tied to older .NET versions and not able to use the latest and greatest features. This is the reason why we will offer separate builds for .NET 4.5 with the async API extensions, and .NET 3.5 builds.

We will upload new .NET 4.5 builds soon, so stay tuned. If you can’t await then contact us directly ;-)

MatriX for WinRT released

Now with the availability of Windows 8 and the first Windows 8 devices we have released the WinRT version of MatriX to the public.

The API and supported features are nearly the same as in all our other MatriX editions.

Start to use the power of XMPP in your WinRT apps now!!

The WinRT version can be downloaded from here:
Download MatriX for WinRT

Web clients with MatriX and SignalR

Persistent realtime connections for ASP.NET in code behind to the web browser were hard to implement in the past. There hasn’t been a ASP.NET component for this for a while.

With SignalR there is an asynchronous realtime signaling library for ASP.NET now that a team at Microsoft is working on. Using SignalR its pretty easy to build real-time web applications.

Using SignalR and MatriX its very easy to write scalable XMPP web applications with a very small amount of JavaScript code.

Let’s get started:

Create a new ASP.NET MVC web application project first. Then install SignalR using the NuGet Package Console with the following command:

install-package SignalR

This will add SignalR with all required references to the new project we just created.

Now we create a Hub for MatriX. Hubs are a higher level framework API to exchange different messages between server (code behind) and client (web browser) bidirectional in realtime.

We First implement the IConnected and IDisconnect interfaces to the Hub. Our Hub has a static Dictionary for all XmppClients instances. Whenever a new client connects to the bidirectional realtime channel of SignalR we create a new XmppClient instance for it, setup the event handlers and add the client to the dictionary. When a browser disconnects from SignalR we disconnect all event handlers and remove the XmppClient from the dictionary.

public class MatrixHub : Hub, IConnected, IDisconnect
{
    private static readonly Dictionary<string, XmppClient> XmppClients = new Dictionary<string, XmppClient>();

    public Task Disconnect()
    {
        if (XmppClients.ContainsKey(Context.ConnectionId))
        {
            var xmppClient = XmppClients[Context.ConnectionId];

            xmppClient.OnReceiveXml -= xmppClient_OnReceiveXml;
            xmppClient.OnSendXml -= xmppClient_OnSendXml;
            xmppClient.OnMessage -= xmppClient_OnMessage;

            XmppClients.Remove(Context.ConnectionId);
        }

        return Clients.leave(Context.ConnectionId, DateTime.Now.ToString());
    }

    public Task Connect()
    {
        if (!XmppClients.ContainsKey(Context.ConnectionId))
        {
            var xmppClient = new XmppClient();

            xmppClient.OnReceiveXml += xmppClient_OnReceiveXml;
            xmppClient.OnSendXml += xmppClient_OnSendXml;       
            xmppClient.OnMessage += xmppClient_OnMessage;
            
            XmppClients.Add(Context.ConnectionId, xmppClient);
        }

        return Clients.joined(Context.ConnectionId, DateTime.Now.ToString());
    }

    public Task Reconnect(IEnumerable<string> groups)
    {
        return Clients.rejoined(Context.ConnectionId, DateTime.Now.ToString());
    }    
}

Now we add a Open and Close function to our Hub which can be executed from the webpage via JavaScript. The JavaScript passes the username, password and XMPP domain as a string to the Open function.

Context.ConnectionId is the Id of the JavaScript client calling this Hub. We use this ids for our XmppClient dictionary and need it to lookup the correct XmppClient instance from the dictionary. After retrieving the XmppClient instance we set Username, Password and XmppDomain. Then we call Open() and MatriX starts to login to the XMPP server with the given credentials.

The same applies to the Close function. We first lookup the correct XmppClient and then call Close on it to close the XMPP stream.

public void Open(String username, String password, String xmppDomain)
{
    XmppClient xmppClient = XmppClients[Context.ConnectionId];
    xmppClient.Username = username;
    xmppClient.Password = password;
    xmppClient.XmppDomain = xmppDomain;

    xmppClient.Open();
}

public void Close()
{
    XmppClient xmppClient = XmppClients[Context.ConnectionId];
    xmppClient.Close();
}

Now we implement the OnSendXml and OnReceiveXml handlers we subscribed to in the Connect Task. This is server side C# code telling the client to call sendXml() and receiveXml() JavaScript functions.

Clients is a dynamic object of SignalR. Clients[Context.ConnectionId] gets the correct client and executes the function on this single client only.

void xmppClient_OnSendXml(object sender, Matrix.TextEventArgs e)
{
    var text = HttpUtility.HtmlEncode(String.Format("Send: {0}", e.Text));
   
    Clients[Context.ConnectionId].sendXml(text);
}

void xmppClient_OnReceiveXml(object sender, Matrix.TextEventArgs e)
{
    var text = HttpUtility.HtmlEncode(String.Format("Recv: {0}", e.Text));
   
    Clients[Context.ConnectionId].receiveXml(text);
}

The last step before we goto our JavaScript is the OnMessage function. SignalR can also send complex objects. So lets create a simple Message class to send the message properties as an object to our JavaScript. The Message class has 2 simple String properties. Body for the Text of the message and From for the full Jid of the sender.

public class Message
{
    public string Body;
    public string From;
}

Whenever MatriX receives a message in the OnMessage handler we raise the callback onMessage in our JavaScript and pass a new instance of the Message object we just created.

void xmppClient_OnMessage(object sender, MessageEventArgs e)
{
    Clients[Context.ConnectionId].onMessage(
           new Message
           {
           		From = e.Message.From,
           		Body = e.Message.Body
           }
    );       
}

The backend and code behind stuff is done now. Now we have to add JavaScript to our web page to setup the SignalR connection.

First we have to include some JavaScript files. We need jquery, the signalR script and the hubs which get created dynamically by SignalR on the server.

<script src="@Url.Content("~/Scripts/jquery-1.6.4.js")" type="text/javascript"></script>
<script src="@Url.Content("~/Scripts/jquery.signalR-0.5.2.js")" type="text/javascript"></script> 
<script src="@Url.Content("~/signalr/hubs")" type="text/javascript"></script>

The following code shows how to create our MatriX Hub in JavaScript and connect it to the SignalR server. matrixHub is defined in the dynamically created JavaScript file we added before.

<script type="text/javascript">
    $(function () {
        var matrixHub = $.connection.matrixHub;
        
		// TODO, callbacks and hub invoker
        
        $.connection.hub.start();
    });
</script>

Now we add the JavaScript code which gets executed when we press the Login and Logout buttons on our wegpage.

The Login button executes the Open function in our C# Hub and passes username, password and XmppDomain. The Logout button executes the Close function without any parameters.

This is all that has to be done to execute a Hub function from JavaScript in realtime.

$("#loginButton").click(function () {
    matrixHub.open(
        $("#txtUsername").val(),
        $("#txtPassword").val(),
        $("#txtXmppDomain").val());
});

$("#logoutButton").click(function () {
    matrixHub.close();
});

The last pieces which are missing are the JavaScript functions we execute form the Hub to send data to the webpage from the server.

The sendXml and readXml callbacks append the Xml debug to a div. The onMessage callbacks appends a incoming message to a div.

// signalR callback for outgoing xml debug
matrixHub.sendXml = function (message) {
    $("#log").append("<span class='log send'>" + message + "</span><br/>");
};

// signalR callback for incoming xml debug
matrixHub.receiveXml = function (message) {
    $("#log").append("<span class='log recv'>" + message + "</span><br/>");
};

matrixHub.onMessage = function (msg) {
    $("#messages").append(
            "<span class='from'>" + msg.From + ":</span>" +
            "<span class='message'>" + msg.Body + "</span>" +
            "</br>"
    );
};

SignalR can also send data to all connected clients or a group of clients, but we don’t need these features.

SignalR handles all the connection stuff for the client and the server automatically for us, and keeps the connection always alive. It chooses the right connection technique for your browser and scales on the server with modern async and await techniques.

Under the hood SignalR is using the same techology as BOSH. On the web server MatriX connects to the XMPP server directly over the default TCP transport. This means you can connect to any XMPP server including Google Talk, Facebook or Windows Live Messenger without using a BOSH proxy from your web page.

The complete example is included in the latest MatriX download.

This post refers to a SinalR version 0.5.x. There were many incompatible API changes in SignalR, so many stuff in this post in not compatible with the current SingalR versions.
The example included in the MatriX download should be always up to date.

MatriX for WinRT

Windows 8 with the new Metro UI is coming soon. We were working hard in the last weeks to port MatriX also to WinRT. Because WinRT is only a subset of the full .NET Framework and introduces many new techniques, namespaces and classes we had to rewrite lots of our code. We are happy to announce that a Beta version of MatriX for WinRT is available now.

Please contact us when you are interested at the beta version.