Thursday, 21 May 2009

How to add GZIP compression to a .NET WCF REST POX client

Bandwidth is important! Even these days where everyone has broadband, now we want the same performance on the move, over wireless and 3G networks. Even when clients individually have enough bandwidth, put 100 or 1000 users together in a corporate environment and it's the IT department that may start have the headache instead. Then there's the server side, ultimately all this traffic comes together down a finite number of "pipes" to our servers, so really we still have to consider bandwidth usage at all levels.

We needed to add GZip compression support to a .NET 3.5sp1 client application using WCF for REST POX web services. The responses were sent compressed using GZIP, so the client needed to be able to decompress these, before passing up the stack to the Data Serialisers and application code layers.

The server in this instance is a Java EE web application returning XML, GZipped using a filter servlet. Implementing the server side filter is fairly trivial, however it's companion in .NET on the client side is not.

Since this gave me a little bit of a challenge recently (very little documentation around or good examples) I thought I'd put up a quick blog about it. It's not that there's a lot of code involved, it's understanding the WCF with sufficient clarity to be able to extend it, and debug it when it's not quite right.

One of the issues that crops up straight away is that most examples of extending the WCF for compression are based around the WS.* web service support, the original incarnation of WCF.

However, later versions of the WCF added support for REST/POX web services, and this is where I wanted to add the GZIP compression support. You can not mix WS.* and Rest (aka Web in WCF) web services in the WCF, without creating a runtime error as the WCF channel is instantiated.

So, enough of the preamble already, lets cut to the chase, what's needed?

Well, assuming you have a WebHttpBinding to start with...

WebHttpBinding webHttpBinding = new WebHttpBinding();


Then, in order to add the GZIP compression support, we need to pull apart the binding, into its constituent elements and then reassemble it, adding in the new binding element.

I enlisted the help of a content mapper to force the content to XML:


public class XMLMapper : WebContentTypeMapper
{
public override WebContentFormat GetMessageFormatForContentType(string contentType)
{
return WebContentFormat.Xml; // always
}
}




First, get the binding elements out of the WebHttpBinding, using the CreateBindingElements() function.


BindingElementCollection bec = webHttpBinding.CreateBindingElements();

WebMessageEncodingBindingElement wmbe = bec.Remove();
HttpTransportBindingElement htbe = bec.Remove();

wmbe.ContentTypeMapper = new XMLMapper();

GZipMessageEncodingBindingElement compBindingElement = new GZipMessageEncodingBindingElement(wmbe);

bec.Add(compBindingElement);
bec.Add(htbe);

CustomBinding cb = new CustomBinding(bec);

binding = cb;

HttpRequestMessageProperty httpRequestMessageProperty = new HttpRequestMessageProperty();

httpRequestMessageProperty.Headers.Add(HttpRequestHeader.AcceptEncoding, "gzip");
httpRequestMessageProperty.Headers.Add(HttpRequestHeader.ContentEncoding, "gzip");




Ok, so now binding is a rebuilt binding, based on WebHttpBinding where a GZIP encoder has been used to wrap the WebMessageEncodingBindingElement from the original binding's binding elements.

Note that the HttpTransportBindingElement must be the last binding element in the binding element collection (protocol stack) or a runtime error will be thrown when trying to instantiate the channel.

Now we can create the channel in the normal way:


// create a channel factory
ChannelFactory cf = new ChannelFactory(binding, hostPath);

// create webhttpbehavior for rest / pox get/invoke behaviour support
WebHttpBehavior webHttpBehavior = new WebHttpBehavior();
webHttpBehavior.DefaultOutgoingRequestFormat = WebMessageFormat.Xml;
webHttpBehavior.DefaultOutgoingResponseFormat = WebMessageFormat.Xml;

// add webhttpbehahvior to channelfactory endpoint
cf.Endpoint.Behaviors.Add(webHttpBehavior);

if (ENABLE_COMPRESSION)
{
// add our custom behavior...
cf.Endpoint.Behaviors.Add(new HttpEndpointBehavior());
}

// create our IService channel
channel = cf.CreateChannel();



So far so good, but what about this GZipMessageEncodingBindingElement?

Well, this can be found in the Microsoft WCF samples:

http://msdn.microsoft.com/en-us/library/ms751458.aspx


Detailed steps of integrating the MS WCF GZip example into your code:

http://www.vistax64.com/indigo/113763-wcf-client-j2ee-server-using-gzip.html

Note: I had to make some minor modifications to make my client work correctly, details can be provided if anyone leaves a comment expressing an interest.



Further Resources:

Various threads discussing similar problems:

http://social.msdn.microsoft.com/forums/en-US/wcf/thread/5c67b0da-9e50-4ee1-b7ac-a4733c580980/


http://social.msdn.microsoft.com/forums/en-US/wcf/thread/ddfe06b1-f07c-4da2-a1f1-d06126e4f96e/


http://aspnet.codeplex.com/Release/ProjectReleases.aspx?ReleaseId=24644


Neomax - provide software products to extend the WCF framework, including support for compression:

http://www.noemax.com/products/wcfx/features.html#gzip_compression


Microsoft Windows SDK for Windows 7 and .NET Framework 3.5 SP1: BETA

http://www.microsoft.com/downloads/thankyou.aspx?familyId=a91dc12a-fc94-4027-b67e-46bab7c5226c&displayLang=en


.

Saturday, 16 May 2009

Quick notes on OpenSolaris 111a & GlassFish clustering

Some quick notes on upgrading 111 to 111a and preparing for GlassFish v2.1 clustering and load balancing tests:

(Note: This is really a scratch-pad of notes and resources for my own benefit, rather than a structured Blog for others to follow, however some of the resources might be useful for someone, somewhere)

OpenSolaris upgrade problems:

Graphical login failed to start

Simply restarting gdm fixed this problem!

svcadm disable gdm
svcadm enable gdm


Installing GlassFish v2.1 - problems:

Running java -Xmx256m -jar glassfish_xxx.jar reported "Not Enough Space Available"

zfs list showed 8gig space available

check for large files in swap directories:

pfexec du -k /tmp|sort -n
pfexec du -k /etc/svc/volatile|sort -n

Increasing swap space, adding a second swap file:

zfs create -V2G rpool/swap2 ; swap -a /dev/zvol/dsk/rpool/swap2


GlassFish clustering:

Steps for setting up GlassFish clustering:

Setup default clustering profile;

lib/ant/bin/ant -f setup-cluster.xml

Start the DAS;

bin/asadmin start-domain --user admin


Create the node agent;

bin/asadmin create-node-agent

Start the node agent;

bin/asadmin start-node-agent

Create cluster and instances (for node agent) in GlassFish admin console.

Download Load Balancer plug-in according to platform

http://download.java.net/javaee5/external/


Guide to setting up GlassFish clustering:

http://blogs.sun.com/dadelhardt/entry/clustering_web_applications_with_glassfish1



Guide to setting up GlassFish with Load Balancer plugin:

https://glassfish.dev.java.net/javaee5/build/GlassFish_LB_Cluster.html

Further resources:

Clustering with Apache HTTPd:

http://blogs.sun.com/jluehe/entry/supporting_apache_loadbalancer_with_glassfish


Interesting thread about clustering problems:

http://www.nabble.com/Cluster-session-replication-not-working-td20691318.html


GlassFish V2 Clustering presentation:

http://blogs.sun.com/stripathi/resource/Preso/GlassFishv2Clustering.pdf

Sun Clustering with GlassFish v2:

http://developers.sun.com/appserver/reference/techart/glassfishcluster/


Setting up GlassFish SSL support:

http://java.sun.com/mailers/techtips/enterprise/2007/TechTips2_Nov07.html


GlassFish JMX / JConsole:

http://docs.sun.com/app/docs/doc/820-4335/ablwi?a=view


Sun Java web server (for load balancing):

https://cds.sun.com/is-bin/INTERSHOP.enfinity/WFS/CDS-CDS_SMI-Site/en_US/-/USD/ViewProductDetail-Start?ProductRef=SJWS6.1SP5-OTH-G-F@CDS-CDS_SMI

Wednesday, 8 April 2009

Cloud computing - cloudforce and beyond...

Having recently attended Cloudforce UK - London (07th-04-2009) I though I'd blog a little with some resources and links.

I don't really want to re-invent the wheel and attempt to explain what cloud computing is exactly, hopefully some of the resources and links should cover that. For an overview, the Wikipedia entry is a good start http://en.wikipedia.org/wiki/Cloud_computing

A couple of terms that crop up in this domain are:
  • SaaS - Software as a Service
  • PaaS - Platform as a Service
  • IaaS - Infrastructure as a Service

Cloudforce itself is a worldwide tour of salesforce.com - evangelising and spreading the word about their cloud products and frameworks, customer success stories and break out sessions covering aspects from development to opportunities.

Details on the event http://www.chinwag.com/events/2009/04/cloudforce-london-2009


Salesforce cloud platform:

http://www.salesforce.com/platform/


Main features at a glance:
  • Product portfolio including CRM
  • Application store - the AppExchange
  • Multiple server, massively scalable environment
  • Multi-tenant database optimised for cloud usage
  • Disaster recovery
  • Database, web service, business logic and presentation layers
  • Fast app creation, instant deployment
  • Live update of applications, can copy to sandbox to develop new versions
  • No built in source code control / visioning
  • Eclipse force plug ins available for development integration
  • Presentation framework and components (Visual Force)
  • Strongly types OO language - Apex (an on-demand language)
  • Web service creation API and integration support
  • Integration with Mobile devices - iPhone, Blackberry and Windows mobile based devices
  • Integration with Twitter, Facebook, Google API's supported

Site gallery / reference sites:

http://developer.force.com/sitesgallery


Force.com Applications:

The platform has an App Exchange, a place where cloud based applications can be shared and exchanged http://www.salesforce.com/platform/appexchange/

The Salesforce platform offers a number of off the shelf, customisable applications products, such as Salesforce CRM http://www.salesforce.com/products/


App Development:

Salesforce developer home is here http://developer.force.com/

The platform framework offers some technologies such as Visual Force for GUI components and component creation and Apex for code/page creation - http://wiki.developerforce.com/index.php/Apex

Salesforce blerb about Apex "Force.com Apex Code is a strongly-typed programming language that executes on the Force.com platform. Apex is used to add business logic to applications, to write database triggers, and to program controllers in the user interface layer. It has a tight integration with the database and query language, good web services support, and includes features such as futures and governors for execution in a multi-tenant environment."


Cloudforce take-aways:

As well as the above, the overall event gave an impression of some key takeaway points.

  • The service cloud, service applications now integrating and utilising knowledge from social networking sources such as Twitter, Facebook etc as well as traditional sources such as Phone.
  • Easy of application creation
  • Speed of development
  • Enable business to meet customer demands in terms of cost or time to market that might otherwise be impossible

Some other providers:

Many over companies offer computing in the cloud, such as:



Well, having just signed up for the Google App engine, expect more blogs in the future as I play around and experiment up in the clouds...


Further Resources:

Google AppEngine blogspot - http://googleappengine.blogspot.com/

Sun Cloud Computing Primer (Need a Sun account) - http://www.sun.com/offers/details/cloud_computing_primer.html


.

Thursday, 2 April 2009

.NET and C# - an introduction

Not so much an actual blog today, rather a link to a presentation I recently gave as an introduction to the .NET framework and C# - to a mixed audience of C, C++, Java and web developers.

It's mainly a high level overview of the .NET framework and then an introduction to the C# language, from a programmers perspective, for programmers familiar with other languages like Java and C++, to help get familiar with the basics.


Available in OpenOffice format ODP:
http://www.chillipower.com/Primers/IntroductionToDotNetandCsharpPresentation.odp

Or MS PowerPoint presentation format PPT:
http://www.chillipower.com/Primers/IntroductionToDotNetandCsharpPresentation.ppt


.

Monday, 30 March 2009

Some notes from Sun GlassFish portfolio tech talk, London

Sun GlassFish and GlassFish portfolio - Sun talk London, Regis House 25-03-2009

Some notes that I managed to take whilst at the talk.

www.glassfish.org

Claimed to be the fastest Java EE AS

V2.0 -> reference implementation for Java EE 5
V3.x -> reference implementation for Java EE 6

Java EE standard 10 years old

Metro -> Interop between MS and Java WS.* standards

GlassFish is Open Source under CDDL and GPL 2 license

java.sun.com/javaee

Transparent development

Organic growth into middleware areas

First appearance 2005
V2 in 2007
Current v2.1 in Feb 2009

www.beta.glassfish.java.net:81/maps

6-700 downloads per month, 8 million in the last year / ~28k per day

GlassFish portfolio is a set of related technologies including the AS
  • Enterprise server (including support for SNMP)
  • Web space server (joined with Life Ray - social networking & portal technologies)
  • Web sack (a complete (L)(S)(?)AMP stack)
  • GlassFish ESB (SOA platform) for integration, connectors, adaptors

Java EE + Ruby on Rails, PHP and other frameworks

(note - what is SFA?)

Enterprise version adds:
  • Performance advisor, performance monitor, SNMP, self management and alert manager
  • Update center

Example / reference sites:
  • wotif.com - Glassfish & OpenMQ
  • www.travelmuse.com GlassFish, message queue and mysql (started by using community edition)
  • North american Nationwide Health Information Network - connect (NHIN- Connect)
  • OpenESB
  • ESB.SOA framework
  • Facebook - large mysql based system ~70million users

GlassFish has same levels and structure in terms of support and licensing as MySQL


Web Services: WS.*
  • Metro is core in GlassFish AS
  • Project Metro Web services stack provides high level WS stack with security, reliability etc -> .NET 3 interoperability (incorporates project tango - MS .NET interop)

REST

  • JAX-RS JSR-311 Java API for RESTFul Web Services
  • Annotation based server side API
  • HTTP Centric
  • Server Side only
  • Servlet or SE deployment

Jersey project provides implementation for JSR-311

Performance:

Similar / faster than Weblogic and Websphere


GlassFish V3:
  • OSGi: Apache Felix as default (origins in JSR-8)
  • 1 sec startup
  • 21Mb download
  • admin and update tool downloaded on demand

Add-ons, modules available from update center:
  • EJB 3.1 (preview
  • jRuby on Rails (new WAR packaging required)
  • Grails (also on GFv2)
  • Jersey and Metro (Web services)
  • jMaki (AJAX)

Tools:
  • NetBeans and Eclipse
  • Embedded GlassFish API

MicroKernel approach - modularity - base App server + on demand module loading, extensible and customizable

("Eclipse Con" -> announces GlassFiish bundle available)

Db connection pooling and connection (connector) pooling (JCA)

V3 Prelude
  • available from http://glassfish.org -> using modular (microkernel) v3 ideas with existing EE5 technologies -> this version is supported!
  • v3 prelude is Java EE Web layer only (i.e. servlets, JSPs etc but no Enterprise beans etc)

Compile / Deploy on Change support, dynamic debug loop (netbeans & eclipse), incremental compile of all Java EE artifacts. Auto-deply of all Java EE and static artifacts

Support for: JRuby, Ruby, Groovy, Grails, Python, jython, django, jmaki, JavaScript phobos

Ruby / jRuby and rails can be run on 2.1 today (by packaging up dependancies), on v3 available as OSGi jRuby container


PHP
  • Quercis (Caucho) opensource GPL php 5 implementation in Java
  • War Packaging
  • Java Bridge

V2 -> ee 5
V3 -> prelude with EE5 support
V3 -> early access is EE6 spec (but not supported)


GF Web Stack:
  • Apache HTTPd
  • Sun web server 7 (most scalable web server - now open sourced!) optimized for multi-core CMT (chip-based Multithreaded) systems - 2x scaling vs. Apache & Tomcat
  • lightpd
  • memcached
  • mod_jk, perl, ruby, PHP Ruby, Python, Squid, Tomcat

Current GF web stack is 1.4 (Python 2.5.2, Squid 2.6, Tomcat 5.5.27, memcached 1.2.5, mysql 5.0.67)


GF ESB:
  • JSR 208 JBI (Java business Integration)
  • Plug-able integration into backbone
  • "Metacontainer" - add new containers within the container, extend the AS
  • BPEL, WS-*, XSLT, FTP, LDAP, HTTP, DB service and binding
  • Based on OpnESB -> http://openesb.org / community open-esb.dev.java.net/Components.html
  • Maybe engines and connectors to things like Corba etc

What's next -> http://fuji.dev.java.net -> OpenESB V3 - next gen of SOA

(JBI and SCA stated as "complimentary, not conflicting")


GF: Web Space server (project web synergy)
  • Portlet spec JSR-268 and more (social networking, communities)
  • OpenSSO, identify mechanisms
  • Compelling web UIs

GF enterprise manager:
  • Performance advisor
  • Alerts
  • JDBC pool management - automatically tunes JDBC connection pool to optimize performance etc

Sailfin (Ericson contribution) telco, SIP, VOIP, Instant messaging

dtrace etends into Apache and PHP

Revamped PetStore 4 part tutorial - developer.sun.com

OpenSSO -> new concept "express builds" - milestone interim builds from Open Source community in between main supported releases

WhitePapers:

New White-papers on GlassFish portfolio launch:
  • Comparison with Tomcat
  • Performance optimization
  • GF with identify and MySQL

Training available in Camberly, BSG in London etc

www.javapassion.com

Books:
  • GlassFish
  • Database-driven application development
  • Others available (I didn't get time to note them down)

blogs.sun.com/theaquarium -> god launch pad for developer information

blogs.sun.com/stories -> gives examples and real usage info

Migration tools:

migrate2glassfish.dev.java.net/blogs/blogs.html

built in tools to detect and advise on code usage (for EE standards and vendor specific code dependancies)

--

Examples:

JavaFX mobile - answer to J2ME shortcomings

www.javafx.com -> many tutorials and demos

GlassFish profiles - Developer mode (no clustering etc) but fast start up and change deployment


Clustering:

DAS - Domain Administration server looks after a cluster, deploy to the domain administration server and it deploys to the cluster nodes.

Session replication between cluster nodes, zero down time when switching between nodes

install with -cluster switch

Cluster elements run "node-agents" to talk to the DAS

Extra sections in the admin console -> shows cluster nodes

GlassFish performance monitor - enterprise only (TBC)?

V3 Prelude - web tier only (web apps) but fully functional


Presentation has now been published here: http://uk.sun.com/sunnews/events/2009/mar/glassfish/pdf/UK-GlassFish-Portfolio-Launch-mar09.pdf


.

Sunday, 8 March 2009

Apple iPhone Development - introduction and resources

Apple iPhone Development:

So, it seems like everyone's doing it these days, but that's no excuse for me not to join in is it?...

Objective-C you say, isn't that a bit like Smalltalk?
xCode, what's that exactly?
Cocoa, Nibs? what the?!?

Well there's much to learn, but most of it seems reasonably straightforward. The Apple documentation is good, there are videos to watch and tutorials, most of these resources can be found via the Apple iPhone dev center on the Apple website. From what I've gathered so far, its quite far removed from previous mobile development I've done, such as J2ME and .NET CF.

Armed with a new Apple Mac I'm making some notes as I venture into the unknown.

Firstly, here are a bunch of useful resources:

iPhone Dev center:

http://developer.apple.com/iphone/

iPhone Developer Program:

http://developer.apple.com/iphone/program/

iPhone SDK:

http://developer.apple.com/iphone/download.action?path=/iphone/iphone_sdk_for_iphone_os_2.2.1__9m2621a__final/iphone_sdk_for_iphone_os_2.2.19m2621afinal.dmg


Objective-C:

http://developer.apple.com/iphone/library/documentation/Cocoa/Conceptual/ObjectiveC/ObjC.pdf

Objective-C is an object orientated version of the C language. Specifically for the iPhone, Objective-C 2.0 is used.

Objective-C has been influenced by Smalltalk, yes that's the OO language with all the [] in it.

iPhone development guide:

http://developer.apple.com/iphone/library/documentation/Xcode/Conceptual/iphone_development/iPhone_Development.pdf

First iPhone application:

http://developer.apple.com/iphone/library/documentation/iPhone/Conceptual/iPhone101/iPhone101.pdf

The current latest version of the iPhone OS and iPhone SDK is 2.2.1 (whilst at the time of writing this 3.0 is in beta preview).

The next version, 3.0 is to be officially launched soon, the SDK should be available before the official OS release, in theory. What's in it, what's different I don't know yet, but hopefully it will be largly compatible with the current version, whilst adding some interesting new features into the mix. Rumoured are things like the much awaited support for copy, cut and paste for example.


HTTP and XML, web services:
Interested in hooking up an iPhone application with some web services, some means to make an HTTP request and parse XML is needed, here are a couple of resources on this:


Simple solution using some utility code:

http://blog.atrexis.com/index.cfm/2008/7/28/iPhone--Parse-XML-to-custom-objects

A more advanced solution:

http://code.google.com/p/touchcode/wiki/TouchXML


Useful article on RESTful clients:

http://developer.apple.com/safari/articles/creatingrestfulclients.html


.

Sunday, 1 March 2009

Prime factorization, a comparison between Haskell and Scala

Haskell is a powerful and expressive pure lazy functional programming language with some interesting properties. http://www.haskell.org/ Haskell is of course named after the famous logician Haskell Curry http://en.wikipedia.org/wiki/Haskell_Curry who's name can be found in both the Haskell language and the process of "currying" functions.

By way of example and comparison the task of finding prime numbers is shown in Haskell and then its equivalent is shown in Scala.

We can see the conciseness of Haskell and the power of lazyness in expressions of the form:

[3..]

which creates an infinite list of integers from 3 upwards. This seeming impossibility can be handled in a lazy functional language because the infinite list is not evaluated up front, rather the list is only evaluated when needed so the list is finite at any given point in time.

So, in essence a lazy language is one where an expression is only evaluated when it's needed, unlike an imperative language where it's evaluated when declared.


Haskell: prime factorization example:


divides :: Integer -> Integer -> Bool
divides d n = rem n d == 0

ld :: Integer -> Integer
ld n = ldf 2 n

ldf :: Integer -> Integer -> Integer
ldf k n | divides k n = k
| k^2 > n = n
| otherwise = ldf (k+1) n

factors :: Integer -> [Integer]
factors n | n < 1 = error "Invalid argument: argument must be positive"
| n == 1 = []
| otherwise = p : factors (div n p) where p = ld n


If the above is saved in a file (called say primefactorization) and loaded into Hugs (Hugs is a Haskell interpreter shell based on Haskell 98 - http://www.haskell.org/hugs/) using the :l command as follows:

Main> :l c:\Dev\haskell\primefactorisation


Scala: prime factorization example:


package test

object PrimeFactorisationTest {

def divides(d : Int, n : Int) = {
(n % d) == 0
}

def ld(n : Int) : Int = {
ldf(2, n)
}

def ldf(k : Int, n : Int) : Int = {
if (divides(k, n)) k
else if ((k*k) > n) n
else ldf((k+1), n)
}

def factors(n : Int) : List[Int] = n match {
case 1 => Nil;
case _ => {
val p = ld(n)
p :: factors(n / p)
}
}

def main(args : Array[String]) {
if (args.length == 1)
{
val n = Integer.parseInt(args(0))
println(factors(n))
}
}
}


Results:



Main> factors 12343434
[2,3,79,26041]


Both programs produce the same output and are functionally equivalent - it's also clear to see there's a reasonably similarity in both conciseness and expressiveness of the two languages, where Haskell possibly has the slight edge in some respects such as use of the "where" construct.


Update:

Following on from the excellent comments on the Haskell version from Don Stewart, turning the code into something that can be compiled and executed natively, rather than being interpreted using Hugs, roughly involves the following steps.

Install GHC http://www.haskell.org/ghc/

Rename the file to have a .hs file extension

Add a main that can process the arguments:

main = do
[n] <- getArgs
print (factors (read n))

add import to provide environment functions such as getArgs to the top of the file:

import System.Environment

Compile the file:

$ghc --make primefactorisation.hs

Run the executable:

$ time ./primefactorisation.exe 12343434
[2,3,79,26041]

real 0m0.140s
user 0m0.015s
sys 0m0.015s

and there we have it, a functional executable.

Don Stewart also makes a good point about being able to leave out all the type information.

In particular Haskell uses the powerful Hindley-Milner type inference algorithm to allow minimal (often zero) use of explicit type declarations.

http://c2.com/cgi/wiki?HindleyMilnerTypeInference


.