Thursday 21 May 2009

How to add GZIP compression to a .NET WCF REST POX client

Bandwidth is important! Even these days where everyone has broadband, now we want the same performance on the move, over wireless and 3G networks. Even when clients individually have enough bandwidth, put 100 or 1000 users together in a corporate environment and it's the IT department that may start have the headache instead. Then there's the server side, ultimately all this traffic comes together down a finite number of "pipes" to our servers, so really we still have to consider bandwidth usage at all levels.

We needed to add GZip compression support to a .NET 3.5sp1 client application using WCF for REST POX web services. The responses were sent compressed using GZIP, so the client needed to be able to decompress these, before passing up the stack to the Data Serialisers and application code layers.

The server in this instance is a Java EE web application returning XML, GZipped using a filter servlet. Implementing the server side filter is fairly trivial, however it's companion in .NET on the client side is not.

Since this gave me a little bit of a challenge recently (very little documentation around or good examples) I thought I'd put up a quick blog about it. It's not that there's a lot of code involved, it's understanding the WCF with sufficient clarity to be able to extend it, and debug it when it's not quite right.

One of the issues that crops up straight away is that most examples of extending the WCF for compression are based around the WS.* web service support, the original incarnation of WCF.

However, later versions of the WCF added support for REST/POX web services, and this is where I wanted to add the GZIP compression support. You can not mix WS.* and Rest (aka Web in WCF) web services in the WCF, without creating a runtime error as the WCF channel is instantiated.

So, enough of the preamble already, lets cut to the chase, what's needed?

Well, assuming you have a WebHttpBinding to start with...

WebHttpBinding webHttpBinding = new WebHttpBinding();


Then, in order to add the GZIP compression support, we need to pull apart the binding, into its constituent elements and then reassemble it, adding in the new binding element.

I enlisted the help of a content mapper to force the content to XML:


public class XMLMapper : WebContentTypeMapper
{
public override WebContentFormat GetMessageFormatForContentType(string contentType)
{
return WebContentFormat.Xml; // always
}
}




First, get the binding elements out of the WebHttpBinding, using the CreateBindingElements() function.


BindingElementCollection bec = webHttpBinding.CreateBindingElements();

WebMessageEncodingBindingElement wmbe = bec.Remove();
HttpTransportBindingElement htbe = bec.Remove();

wmbe.ContentTypeMapper = new XMLMapper();

GZipMessageEncodingBindingElement compBindingElement = new GZipMessageEncodingBindingElement(wmbe);

bec.Add(compBindingElement);
bec.Add(htbe);

CustomBinding cb = new CustomBinding(bec);

binding = cb;

HttpRequestMessageProperty httpRequestMessageProperty = new HttpRequestMessageProperty();

httpRequestMessageProperty.Headers.Add(HttpRequestHeader.AcceptEncoding, "gzip");
httpRequestMessageProperty.Headers.Add(HttpRequestHeader.ContentEncoding, "gzip");




Ok, so now binding is a rebuilt binding, based on WebHttpBinding where a GZIP encoder has been used to wrap the WebMessageEncodingBindingElement from the original binding's binding elements.

Note that the HttpTransportBindingElement must be the last binding element in the binding element collection (protocol stack) or a runtime error will be thrown when trying to instantiate the channel.

Now we can create the channel in the normal way:


// create a channel factory
ChannelFactory cf = new ChannelFactory(binding, hostPath);

// create webhttpbehavior for rest / pox get/invoke behaviour support
WebHttpBehavior webHttpBehavior = new WebHttpBehavior();
webHttpBehavior.DefaultOutgoingRequestFormat = WebMessageFormat.Xml;
webHttpBehavior.DefaultOutgoingResponseFormat = WebMessageFormat.Xml;

// add webhttpbehahvior to channelfactory endpoint
cf.Endpoint.Behaviors.Add(webHttpBehavior);

if (ENABLE_COMPRESSION)
{
// add our custom behavior...
cf.Endpoint.Behaviors.Add(new HttpEndpointBehavior());
}

// create our IService channel
channel = cf.CreateChannel();



So far so good, but what about this GZipMessageEncodingBindingElement?

Well, this can be found in the Microsoft WCF samples:

http://msdn.microsoft.com/en-us/library/ms751458.aspx


Detailed steps of integrating the MS WCF GZip example into your code:

http://www.vistax64.com/indigo/113763-wcf-client-j2ee-server-using-gzip.html

Note: I had to make some minor modifications to make my client work correctly, details can be provided if anyone leaves a comment expressing an interest.



Further Resources:

Various threads discussing similar problems:

http://social.msdn.microsoft.com/forums/en-US/wcf/thread/5c67b0da-9e50-4ee1-b7ac-a4733c580980/


http://social.msdn.microsoft.com/forums/en-US/wcf/thread/ddfe06b1-f07c-4da2-a1f1-d06126e4f96e/


http://aspnet.codeplex.com/Release/ProjectReleases.aspx?ReleaseId=24644


Neomax - provide software products to extend the WCF framework, including support for compression:

http://www.noemax.com/products/wcfx/features.html#gzip_compression


Microsoft Windows SDK for Windows 7 and .NET Framework 3.5 SP1: BETA

http://www.microsoft.com/downloads/thankyou.aspx?familyId=a91dc12a-fc94-4027-b67e-46bab7c5226c&displayLang=en


.

Saturday 16 May 2009

Quick notes on OpenSolaris 111a & GlassFish clustering

Some quick notes on upgrading 111 to 111a and preparing for GlassFish v2.1 clustering and load balancing tests:

(Note: This is really a scratch-pad of notes and resources for my own benefit, rather than a structured Blog for others to follow, however some of the resources might be useful for someone, somewhere)

OpenSolaris upgrade problems:

Graphical login failed to start

Simply restarting gdm fixed this problem!

svcadm disable gdm
svcadm enable gdm


Installing GlassFish v2.1 - problems:

Running java -Xmx256m -jar glassfish_xxx.jar reported "Not Enough Space Available"

zfs list showed 8gig space available

check for large files in swap directories:

pfexec du -k /tmp|sort -n
pfexec du -k /etc/svc/volatile|sort -n

Increasing swap space, adding a second swap file:

zfs create -V2G rpool/swap2 ; swap -a /dev/zvol/dsk/rpool/swap2


GlassFish clustering:

Steps for setting up GlassFish clustering:

Setup default clustering profile;

lib/ant/bin/ant -f setup-cluster.xml

Start the DAS;

bin/asadmin start-domain --user admin


Create the node agent;

bin/asadmin create-node-agent

Start the node agent;

bin/asadmin start-node-agent

Create cluster and instances (for node agent) in GlassFish admin console.

Download Load Balancer plug-in according to platform

http://download.java.net/javaee5/external/


Guide to setting up GlassFish clustering:

http://blogs.sun.com/dadelhardt/entry/clustering_web_applications_with_glassfish1



Guide to setting up GlassFish with Load Balancer plugin:

https://glassfish.dev.java.net/javaee5/build/GlassFish_LB_Cluster.html

Further resources:

Clustering with Apache HTTPd:

http://blogs.sun.com/jluehe/entry/supporting_apache_loadbalancer_with_glassfish


Interesting thread about clustering problems:

http://www.nabble.com/Cluster-session-replication-not-working-td20691318.html


GlassFish V2 Clustering presentation:

http://blogs.sun.com/stripathi/resource/Preso/GlassFishv2Clustering.pdf

Sun Clustering with GlassFish v2:

http://developers.sun.com/appserver/reference/techart/glassfishcluster/


Setting up GlassFish SSL support:

http://java.sun.com/mailers/techtips/enterprise/2007/TechTips2_Nov07.html


GlassFish JMX / JConsole:

http://docs.sun.com/app/docs/doc/820-4335/ablwi?a=view


Sun Java web server (for load balancing):

https://cds.sun.com/is-bin/INTERSHOP.enfinity/WFS/CDS-CDS_SMI-Site/en_US/-/USD/ViewProductDetail-Start?ProductRef=SJWS6.1SP5-OTH-G-F@CDS-CDS_SMI