Monday, 15 June 2009

Scala actors, an introduction with some simple examples

The Scala programming language provides an Actor based framework for handling concurrency.

Scala's Actor model for concurrent programming was originally inspired by the Erlang programming language.

The actor model is a system of asynchronous message passing for concurrent programming, rather than the typical shared data and locks model. The shared data and locks model has some inherent difficulties that make programming in this way difficult and error prone. Major problems with shared data and locks include the well known dead-lock problem, as well as live locks and the general problem of locking shared mutable data without affecting program scalability and performance. Whilst Scala's actor messaging framework is build around asynchronous messaging, synchronous messaging is provided on top of this for convenience.

The Scala's actor model is based around an actor library and the Actor trait, along with some simple syntactical constructs for sending and receiving messages.

The actor library can be included into a project using:

import scala.actors.Actor._

The simplest way to start an actor is to declare an object/class that extends the class Actor and in the body of the actor declare an act() {} method to do the work. The actor x can then be started much like a thread, using x.start(). The act() method is much like the Java Thread's run() method.

A more succinct method is to declare a variable val x = actor {} and put the action definition directly between the braces (no act method required). What happens here is that actor is a helper method on Actor that creates and starts the actor - so there is no need to call x.start() explicitly when using this approach.

Actors can receive messages using the receive {} method. The receive message is passed a partial function to receive the messages. Typically the body of the receive method uses case pattern matching to determine what to do with the message.

A message can be sent to an actor using the ! operation. For example, an Int can be sent to actor x using: x ! 3

All actors get their own thread to call the receive function - this causes scalability problems when there are many actors with receive functions. The way to overcome this in Scala is to use the react() {} method instead. React is similar to receive except it doesn't not need a native thread to run and but must call itself when done to receive more messages - apart from this both are the same in that they both take a partial function.

Scala provides a convenience to avoid having to call act/react within a react method, this is the loop construct. This ends up looking as simple as:


loop {
react {
// message handling body goes here
}
}

Note that react itself does not end end by returning up the call stack in the normal way, all calls to react result in an exception.

Within a receive or react method, an actor can reply to the sender of the received message by messaging the special identifier "sender", which represents the source of the message.


Example 1:

This simple example shows the most basic way to create an actor, by extending Actor, creating an instance and calling start on it.


package scalaactors1

import scala.actors._

class Actor1 extends Actor {

def act {
var x = 1
while (true) {
println("actor1! " + x)
x = x + 1
Thread.sleep(500)
}
}
}

object TestActors1 {

/**
* @param args the command line arguments
*/
def main(args: Array[String]) = {

println(">>main() - starting")

val actor1 = new Actor1()

actor1.start()

println("<<main() - ending")
}
}



Example 2:

This example creates a couple of actors using the val x = actor {} constructs. Each actor will automatically start and run concurrently, outputting a message and sleeping in an infinite loop. If you run this you can observe that the main thread starts and ends, the messages from the two actors are interleaved and run even after the main method has ended.


package scalaactors1

import scala.actors.Actor._

object TestActors2 {

/**
* @param args the command line arguments
*/
def main(args: Array[String]) = {

println(">>main() - starting")


val actor1 = actor {

var x = 1
while (true) {
println("actor1! " + x)
x = x + 1
Thread.sleep(500)
}
}

val actor2 = actor {

var x = 1
while (true) {
println("actor2! " + x)
x = x + 1
Thread.sleep(1000)
}
}

println("<<main() - ending")
}
}



Example 3:

This example is slightly more complex.

Actor kbReaderActor takes input from the keyboard (as Chars) and passes them one by one to the actor kbProcessorActor which prints out the received character and then uses a 3rd actor called sleep to sleep for the specified amount of time before then waking up the kbProcessorActor (to simulate some "work" that takes a finite amount of time to complete). Note that sleep signals the "sender" using a simple case class to represent the message type (with no arguments).



package scalaactors1

import scala.actors.Actor._

case class Wakeup // simple signalling case class


// simple actor example
object TestActors3 {

def main(args: Array[String]) = {

println(">>Starting")

val sleep = actor {

loop {
react {
case x : int => {
println("sleeping for " + x)

java.lang.Thread.sleep(x)

println("waking up sender...")

sender ! Wakeup
}
}
}
}

val keyProcessorActor = actor {

println(">>keyProcessorActor starting")

var key : Option[Char] = null

loop {

react {
case 'q' => exit() // 'q' signifies quit app!
case x : Char => println("keyProcessorActor received '" + x + "'")

sleep ! 1000 // ask the sleep actor to sleep, we will get a Wakeup call when done

// now wait for our wakeup
var r = false

do {
receive {
case Wakeup => r = true
}
} while (!r)
}
}

println("<<keyProcessorActor ending")
}

val kbReaderActor = actor {

println(">>kbReaderActor starting")

var key : Char = ' '

do {

println("kbReaderActor waiting for input")
val i = System.in.read()

key = i.asInstanceOf[Char]

println("Received input " + key)

// message actor1 with the key read from the console
keyProcessorActor ! key

} while (key != 'q')

println("<<kbReaderActor ending")
}

println("<<Application Ending")
}
}



.

Saturday, 6 June 2009

Quantum Entanglement, entanglement with 2 electron singlet

(and now for something completely different... Quantum mechanics and quantum entanglement)

Quantum Entanglement:
Entanglement with 2 element quantum singlet.

Having recently emailed Leonard Susskind (Stanford CA) with a question regarding entanglement of a 3rd element with a 2 element system in the quantum singlet state (an entangled pair).

Leonard Susskind is the Felix Bloch professor of theoretical physics at Stanford University in the field of string theory and quantum field theory http://en.wikipedia.org/wiki/Leonard_Susskind

Susskind is widely regarded as one of the fathers of String Theory.

So naturally, I was somewhat surprised when I actually got a detailed response, and the answer is very interesting and informative too! Like most things in Quantum mechanics, the result is not something that could be guest or anticipated, it's not something that could be expected using classical intuition or experience.


Here is the reply:

--

Let’s begin with the initial state consisting of two electrons in a singlet state |ud-du> .( I should include a factor of one-over-square-root-of-2 to normalize the state but it’s too hard to type in Word so I will leave it to you.) Now add another electron in the up state— |u). Also, in order to keep track of photons, lets indicate that there are no photons present with the notation |0}. The initial state is,

[ |ud-du> |u) ] |0}

Now let’s rearrange the symbols so as to group electrons 2 and 3 together. This is just a notational device. The initial state is

[ |u> |du) - |d>|uu) ] |0}

Now use the trivial identity

|du) = ½ |du-ud) + ½ |du+ud)

to rewrite it

[ |u> |du-ud) ] |0} + ½ [ |u> |du+ud) ] |0}

-[ |d>|uu) ] |0}


Now think of electron 2 and 3 as close one another, while electron 1 is off in the distance. In the first term 2-3 are in a singlet state. Nothing happens to it. In the other two terms the 2-3 system is orthogonal to the singlet. The 2-3 system will interact to emit a photon, and decay to the singlet in those cases, although the photons will be in different states in the two terms. Denoting the presence of the photon by |γ} and |γ’}, the final state will be

[ |u> |du-ud) ] |0} + ½ [ |u> |du-ud) ] |γ}

-[ |d>|du-ud)] |γ’} (note, there is missing sqrt2)

So what do we have in the end? A superposition of states in which the 2-3 system is in the singlet and

  1. electron 1 is up with no photon

  1. electron 1 is up with a photon in state |γ}

  1. electron 1 is down with a photon in state |γ’}.

If there is no photon or a photon in state |γ} then we know that electron 1 is up. If there is a photon in state |γ’} then e-1 must be down.

--

Having such a brilliantly and concise response was fantastic!

I've gone on to calculate the trivial equations to determine the probability in each state, as follows:

Given the probabilities of the 3 possibilities must add up to 1, taking the coefficients as lambda, the probability is given by lambda.lambda* which for real numbers is just lambda^2, so:

[ |u> |du-ud) ] |0} + ½ [ |u> |du-ud) ] |γ}
-[ |d>|du-ud)] |γ’} (note, there is missing sqrt2)

a. electron 1 is up with no photon


| 1 | 2
| ------------| = 1/4 = p1 = 0.25
| 2 sqrt(2) |

b. electron 1 is up with a photon in state |γ}

p2 = 0.25


c. electron 1 is down with a photon in state |γ’}.

| 1 | 2
| ------------| = 1/2 = p3 = 0.5
| sqrt(2) |


Google document:

http://docs.google.com/View?id=df4zskcg_25d8xtc9hp


The Stanford course "Modern Theoretical Physics - Fall 2006" by Leonard Susskind is available on iTunes-U (highly recommended!).


.

Basic Crib sheet for Mercurial hg

Mercurial (hg) is a distributed version control system, with many similarities to Subversion (svn). Being distributed, means people can work offline and still keep their version history local, merging changes with each other by exporting and importing the repository changes periodically.

Generally speaking mercurial is very simple and easy to use, but here are a few commands for quick reference:


Basic crib for Mecurial HG

Clone a repository from a web URL:

hg clone http://www.selenic.com/repo/hello my-hello

Clone from another local repository:

hg clone existing-repos clone-repos

Make a new repository:

hg init

Add files:

hg add

Committing changes:

hg commit (hg ci)

Pull changes from one repository to another:

hg pull

Note: this doesn't update the working directory, to do this update the working directory:

hg update (hg up)

Exporting and import:

hg export -o filename tip

hg import -o filename

Other common commands:

hg log
hg status (hg st)
hg diff
hg revert
hg (-q) tip


HG Editor, the editor for HG comments is determined in the following order:

HGEDITOR environment variable

editor configuration option in [ui] section (in hgrc or passed with --config ui.editor command-line option).

VISUAL environment variable

EDITOR environment variable

vi, if none of the above is set


References:

Mercurial home:

http://www.selenic.com/mercurial/wiki/Mercurial


Mecurial tutorial:

http://www.selenic.com/mercurial/wiki/Tutorial

Quick reference and cheat sheets:

http://www.selenic.com/mercurial/wiki/QuickReferenceCardsAndCheatSheets


.

Thursday, 4 June 2009

GlassFish v2.1 b60e crib sheet / installation notes

Recently, having been working on a project that's migrated from JBoss 4.2.2 to GlassFish v2.1, I needed to hand over a crib sheet / quick guide to people supporting the live system, so as it might be useful in general. Only really covering the basics, but here it is anyway:

GlassFish AS Basic Crib / notes:

Prerequisites:

  • Java JDK 6.x latest

GlassFish AS version V2.1 b60e Final Release

Download jar here http://java.net/download/javaee5/v2.1_branch/promoted/Linux/glassfish-installer-v2.1-b60e-linux.jar

For RHEL derived CentOS, recommended installation would be something like /usr/local/projectname/glassfish


Installation:

java -Xmx256m -jar glassfish-installer-v2.1-b60e-linux.jar

cd glassfish

chmod -R +x lib/ant/bin

lib/ant/bin/ant -f setup.xml

Follow the installer instructions, accepting EULA and any defaults presented...

This will unpack and install the server, creating a default domain called "domain1"

A domain is an administrative "unit" - possibly related to, but not the same as an actual domain.

All domains live in: {glassfish_home}/domains/

We need just one for now, so all our stuff will be in:

{glassfish_home}/domains/domain1/

the installation process will create a default empty domain, domain1 with a default config file, the main configuration file is called domain.xml

e.g. {glassfish_home}/domains/domain1/config.xml

This file may contain some host /installation specific information, as well as stuff specific to our application.


Starting and stopping GlassFish AS:

{glassfish_home}/bin/asadmin start-domain domain1

{glassfish_home}/bin/asadmin stop-domain domain1

If all is good you will see confirmation on the command prompt when started, such as:

Domain domain1 started


Managing GlassFish AS:

By default, the web based management console is available on port :4848

The default admin account is username = admin, password = adminadmin

URL is: http://{hostname}:4848


Web-app deployment options:

There are a few options, the main two are:

Manually, using autodeploy directory

Simply copy .war file to: {glassfish_home}/domains/domain1/autodeploy

Deploying via the admin console:

On main menu on left, click on and expand the Web Applications menu entry

Click on the Deploy button, typically when deploying from a local file, use the "Packaged file to be uploaded to the server" browse button to locate the local war file. Browse for the war file, select it and click Ok

When completed, the web app will appear in the application list (if all is well)

The other settings can be left as defaults. Optionally we can chose to pre-compile JSPs here etc.


Server Logs:

The main server logs appear in the domain under logs, i.e.

{glassfish_home}/domains/domain1/logs/server.log

Logs can also be viewed (and configured, including log rotation) via the admin console, under the Application Server menu entry


Monitoring and management:

JConsole:

GlassFish is a JMX compliant app server (listening by default to port 8686)

Using jconsole, remote connect to: http://{hostname}:8686

Using the same default account: admin/adminadmin

Admin console:

Under the Application Server in the main menu - there are entries for server Monitoring.

This includes a built in chart and call flow logging - which is potentially very useful for diagnostics and monitoring…


HTTPS / SSL:

GlassFish AS will do HTTPS out of the box (using a default Sun Microsystems certificate).

HTTPS is served by default on port :8181


Hopefully in the near future I'll but together something to cover clustering and load balancing too.



.

Wednesday, 3 June 2009

Super quick guide for creating self-signed certificates

Something I've done many times, but every time it's typically been so long in between that I've usually forgotten the specific details, so here is a super quick crib:

Create the private keys:


openssl genrsa -des3 -out server1.key 1024


Create a Certificate Signing Request (CSR):


openssl req -new -key server1.key -out server1.csr


Enter details, example below:


Country Name (2 letter code) [AU]:GB
State or Province Name (full name) [Some-State]:Kent
Locality Name (eg, city) []:Royal Tunbridge Wells
Organization Name (eg, company) [Internet Widgits Pty Ltd]:Acme Co
Organizational Unit Name (eg, section) []:Financial Systems
Common Name (eg, YOUR name) []:www.myexactdomainname.com
Email Address []:louis.botterill@netbuilder.com

Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:
An optional company name []:


Copy the passphrase protected key:


cp server1.key server1.key.org


Export the actual key without pass-phrase protection:


openssl rsa -in server1.key.org -out server1.key


Sign the CSR using the key to create a Certificate (.crt):


openssl x509 -req -days 365 -in server1.csr -signkey server1.key -out server1.crt


Easy huh! :)


.

Thursday, 21 May 2009

How to add GZIP compression to a .NET WCF REST POX client

Bandwidth is important! Even these days where everyone has broadband, now we want the same performance on the move, over wireless and 3G networks. Even when clients individually have enough bandwidth, put 100 or 1000 users together in a corporate environment and it's the IT department that may start have the headache instead. Then there's the server side, ultimately all this traffic comes together down a finite number of "pipes" to our servers, so really we still have to consider bandwidth usage at all levels.

We needed to add GZip compression support to a .NET 3.5sp1 client application using WCF for REST POX web services. The responses were sent compressed using GZIP, so the client needed to be able to decompress these, before passing up the stack to the Data Serialisers and application code layers.

The server in this instance is a Java EE web application returning XML, GZipped using a filter servlet. Implementing the server side filter is fairly trivial, however it's companion in .NET on the client side is not.

Since this gave me a little bit of a challenge recently (very little documentation around or good examples) I thought I'd put up a quick blog about it. It's not that there's a lot of code involved, it's understanding the WCF with sufficient clarity to be able to extend it, and debug it when it's not quite right.

One of the issues that crops up straight away is that most examples of extending the WCF for compression are based around the WS.* web service support, the original incarnation of WCF.

However, later versions of the WCF added support for REST/POX web services, and this is where I wanted to add the GZIP compression support. You can not mix WS.* and Rest (aka Web in WCF) web services in the WCF, without creating a runtime error as the WCF channel is instantiated.

So, enough of the preamble already, lets cut to the chase, what's needed?

Well, assuming you have a WebHttpBinding to start with...

WebHttpBinding webHttpBinding = new WebHttpBinding();


Then, in order to add the GZIP compression support, we need to pull apart the binding, into its constituent elements and then reassemble it, adding in the new binding element.

I enlisted the help of a content mapper to force the content to XML:


public class XMLMapper : WebContentTypeMapper
{
public override WebContentFormat GetMessageFormatForContentType(string contentType)
{
return WebContentFormat.Xml; // always
}
}




First, get the binding elements out of the WebHttpBinding, using the CreateBindingElements() function.


BindingElementCollection bec = webHttpBinding.CreateBindingElements();

WebMessageEncodingBindingElement wmbe = bec.Remove();
HttpTransportBindingElement htbe = bec.Remove();

wmbe.ContentTypeMapper = new XMLMapper();

GZipMessageEncodingBindingElement compBindingElement = new GZipMessageEncodingBindingElement(wmbe);

bec.Add(compBindingElement);
bec.Add(htbe);

CustomBinding cb = new CustomBinding(bec);

binding = cb;

HttpRequestMessageProperty httpRequestMessageProperty = new HttpRequestMessageProperty();

httpRequestMessageProperty.Headers.Add(HttpRequestHeader.AcceptEncoding, "gzip");
httpRequestMessageProperty.Headers.Add(HttpRequestHeader.ContentEncoding, "gzip");




Ok, so now binding is a rebuilt binding, based on WebHttpBinding where a GZIP encoder has been used to wrap the WebMessageEncodingBindingElement from the original binding's binding elements.

Note that the HttpTransportBindingElement must be the last binding element in the binding element collection (protocol stack) or a runtime error will be thrown when trying to instantiate the channel.

Now we can create the channel in the normal way:


// create a channel factory
ChannelFactory cf = new ChannelFactory(binding, hostPath);

// create webhttpbehavior for rest / pox get/invoke behaviour support
WebHttpBehavior webHttpBehavior = new WebHttpBehavior();
webHttpBehavior.DefaultOutgoingRequestFormat = WebMessageFormat.Xml;
webHttpBehavior.DefaultOutgoingResponseFormat = WebMessageFormat.Xml;

// add webhttpbehahvior to channelfactory endpoint
cf.Endpoint.Behaviors.Add(webHttpBehavior);

if (ENABLE_COMPRESSION)
{
// add our custom behavior...
cf.Endpoint.Behaviors.Add(new HttpEndpointBehavior());
}

// create our IService channel
channel = cf.CreateChannel();



So far so good, but what about this GZipMessageEncodingBindingElement?

Well, this can be found in the Microsoft WCF samples:

http://msdn.microsoft.com/en-us/library/ms751458.aspx


Detailed steps of integrating the MS WCF GZip example into your code:

http://www.vistax64.com/indigo/113763-wcf-client-j2ee-server-using-gzip.html

Note: I had to make some minor modifications to make my client work correctly, details can be provided if anyone leaves a comment expressing an interest.



Further Resources:

Various threads discussing similar problems:

http://social.msdn.microsoft.com/forums/en-US/wcf/thread/5c67b0da-9e50-4ee1-b7ac-a4733c580980/


http://social.msdn.microsoft.com/forums/en-US/wcf/thread/ddfe06b1-f07c-4da2-a1f1-d06126e4f96e/


http://aspnet.codeplex.com/Release/ProjectReleases.aspx?ReleaseId=24644


Neomax - provide software products to extend the WCF framework, including support for compression:

http://www.noemax.com/products/wcfx/features.html#gzip_compression


Microsoft Windows SDK for Windows 7 and .NET Framework 3.5 SP1: BETA

http://www.microsoft.com/downloads/thankyou.aspx?familyId=a91dc12a-fc94-4027-b67e-46bab7c5226c&displayLang=en


.

Saturday, 16 May 2009

Quick notes on OpenSolaris 111a & GlassFish clustering

Some quick notes on upgrading 111 to 111a and preparing for GlassFish v2.1 clustering and load balancing tests:

(Note: This is really a scratch-pad of notes and resources for my own benefit, rather than a structured Blog for others to follow, however some of the resources might be useful for someone, somewhere)

OpenSolaris upgrade problems:

Graphical login failed to start

Simply restarting gdm fixed this problem!

svcadm disable gdm
svcadm enable gdm


Installing GlassFish v2.1 - problems:

Running java -Xmx256m -jar glassfish_xxx.jar reported "Not Enough Space Available"

zfs list showed 8gig space available

check for large files in swap directories:

pfexec du -k /tmp|sort -n
pfexec du -k /etc/svc/volatile|sort -n

Increasing swap space, adding a second swap file:

zfs create -V2G rpool/swap2 ; swap -a /dev/zvol/dsk/rpool/swap2


GlassFish clustering:

Steps for setting up GlassFish clustering:

Setup default clustering profile;

lib/ant/bin/ant -f setup-cluster.xml

Start the DAS;

bin/asadmin start-domain --user admin


Create the node agent;

bin/asadmin create-node-agent

Start the node agent;

bin/asadmin start-node-agent

Create cluster and instances (for node agent) in GlassFish admin console.

Download Load Balancer plug-in according to platform

http://download.java.net/javaee5/external/


Guide to setting up GlassFish clustering:

http://blogs.sun.com/dadelhardt/entry/clustering_web_applications_with_glassfish1



Guide to setting up GlassFish with Load Balancer plugin:

https://glassfish.dev.java.net/javaee5/build/GlassFish_LB_Cluster.html

Further resources:

Clustering with Apache HTTPd:

http://blogs.sun.com/jluehe/entry/supporting_apache_loadbalancer_with_glassfish


Interesting thread about clustering problems:

http://www.nabble.com/Cluster-session-replication-not-working-td20691318.html


GlassFish V2 Clustering presentation:

http://blogs.sun.com/stripathi/resource/Preso/GlassFishv2Clustering.pdf

Sun Clustering with GlassFish v2:

http://developers.sun.com/appserver/reference/techart/glassfishcluster/


Setting up GlassFish SSL support:

http://java.sun.com/mailers/techtips/enterprise/2007/TechTips2_Nov07.html


GlassFish JMX / JConsole:

http://docs.sun.com/app/docs/doc/820-4335/ablwi?a=view


Sun Java web server (for load balancing):

https://cds.sun.com/is-bin/INTERSHOP.enfinity/WFS/CDS-CDS_SMI-Site/en_US/-/USD/ViewProductDetail-Start?ProductRef=SJWS6.1SP5-OTH-G-F@CDS-CDS_SMI