Louis Botterill's mostly software and techy Blog

GlassFish v3 and Java EE 6 Sun-Oracle roadshow - key notes

2010-02-03T10:34:00.000-08:00

GlassFish Roadshow 2010 - London 03-02-2010

The following is some notes I took down during the above event.

In brief summary of GlassFish v3 and Java EE 6, here are some of the key takeaways:

GlassFish v3 continues to be developed and supported, as the Java EE 5 & 6 RI app server
GlassFish v3 currently has no clustering, but offers OSGi modularization and extensibility
Supports and takes advantage of new Java EE 6 specifications
Big push on modularity and flexibility in both GlassFish and Java EE 6
Java EE 6 supports annotation based EJBs, RESTful web services
Java EE 6 greatly simplified configuration, optional web-inf.xml etc
Java EE 6 simplied simple class EJBs and improved JPA specification
Ongoing road-map for GlassFish, details TBA later this year

Java EE 6 (Roberto Chinnici):

Released Dec 10 2009

Key new features: New API, Web profiles, Pluggabiliy, dependancy injection

New technologies:

Jax-RS 1.1
Bean validation 1.0
DI 1.0
CDI 1.0
Managed beans 1.0

Closed down gap between EJB and POJO with unification and annotations

CDI works with POJO and EJB classes

Unification of platform types, more uniform programming model

EJB 3.1
JPA 2.0
Servlet 3.0
JSF 2.0
Connectors 1.6
Interceptors 1.1
JAX-WS 2.2
JSR-109 1.3
JSP 2.2
JSR-250 1.1
JACC 1.4
JASPIC 1.1

Key goal in Java EE 6 - flexibility, pruning, extensibility and web profiles

Profiles:

Bundles of technologies targeting specific classes of applications
Decoupled from each other and the platform
Guarantees compatibility of required technologies, individually and in combination - must satisfy all joint requirements

Web Profile:

Modern Java web application stack
First profile to be definite
Mid sized, fully functional (expected 80% web-app coverage), add additional components via extensibility (such as web services API, 3rd party frameworks)

Web profile contents:

Servlet, JSP/EL, JSTL, JSF, Bean Validation, EJB Lite, JPA, JTA, Di, CDI, Managed beans, Interceptors, JSR-250

Pruning:

Goal - to address bloat concerns

2 step process:
Declare components as "Proposed optional"
The make fully optional for the next release

Proposed optional technologies:

JAX-RPC (JAX-WS)
EJB 2.x Entity beans (JPA entity classes)
JAXR (little use)
JSR-88 (deployment, tools API - not used)

* Don't use - use replacements / alternatives

Pluggability / extensibility:

Focus on web tier
Level playing field for 3-rd party libraries and frameworks
Two major extensibility points: Servlets and CDI
Simplify packaging of web-apps
Zero-configuration!

Modular Web applications:

Libraries can contain web-fragment.xml descriptor
web.xml is now optional
Can server resources out of jars with: /META-INF/resources

e.g.
/WEB-INF/lib/catalog.jar
/META-INF/resources/catalog/books.html

e.g. Dojo jar in resources

Web fragments in servlet 3.0:

META-INF.web-fragment.xml

same structure as web.xml, can override in web.xml

Servlet container pluggability:

ServletContainerInitializer interface implements by extensions

@HandlesTypes to declare interest in on or more annotation types

ServletContext now contains method to dynamically register servlets and filters
i.e. ServletContext API has been extended

Registered using META-INF/services

onStartup method gets called with a Set of classes available

** Can only add services at startup, not dynamically once running

Asynchronous HTTP processing:

New programming model for async request processing
e.g. Comet, char, push apps
Opt-in model
Threads are managed by the container

@WebServlet(asyncSupported=true)
public class MyServlet extends HTTPServlet {

}

Goal - Decouple requests from threads

* Changes to the way filters work - thread not attached to the socket, response not written. Filters modified to make safe - option than can be turned on to identify asyncSupported=true
- do this on servlet and any filter involved in the chain

Ideal when waiting for some external resource etc.
Low level API - not very elegant
Idea is that frameworks will use this - see for example Atmosphere framework which is based on annotations - under the hood this async API is used.

JSF 2.0:

Facelet as a standard view declaration language
Composite components
Ajax (declarative and programmatic) e.g. f:ajax tag
Partial state saving (track deltas and sends changes in response, previously all would have been sent even if not changed!)
System events e.g. f:event tag
Resources
Validation e.g. new f:validateBean tag (better integration, validation API. can handle multiple errors, not first one at a time!)

EJB 3.1:

@Singleton beans
@Startup beans (invoked at app startup, works well with Singleton pattern)
@Asynchronous invocations (biggest change, allows non-blocking sync EJB invocations as first class call. Method must be either void or return a Future object)
No interface view (bean impl does not need an interface anymore, now 1 class = 1 EJB!)
Define EJBs directly inside a web app, inside a war file
New API - EJBContainer API works on Java SE, can bootstrap an EJB container in a Java SE application (ideal for testing/development, could be useful in client applications)

Simplified packaging:

EJB class directly into the war file
Previously must built an EJB jar to be included

EJB 3.1 lite:

A subset of EJB 3.1
All types of session beans (stageful, stateless, singleton) - other bean types not supported (timer, entity etc)
Declarative transactions and security
Interceptors
ejb-jar.xml descriptor allows (is optional, probably not useful)

* Class loading, new visibility rules in Java EE spec

Slight differences in class loading rules - if in doubt check the specs

New JNDI Namespaces:

Until now - only java:comp
Now added:
java:module - a module (war, ejb, jar)
java:app - an application
java:global the whole server / cluster

e.g. @Resource(lookup="java:app/CustomerDB") DataSource db;

EJB components have global names now:

e.g. java:global/app1/module2/SomeBeanIcom.acme.Foo

Helps solve problem of remote EJB communication in same app server

Dependency injection (DI):

Combination of DI 1.0 / CDI 1.0
New @Inject annotation
@Inject @LoggedIn User user; (@LoggedIn = "Which one", User = "What")
Beans auto-discovered at start-up
Extensible
Injection metamodel (BeanManager API)
@Resource still available for container resources

* Identified by type and qualifiers (no longer just a string name alone)

* No bean declaration as per Spring declaration, bean discovery at startup
* Injection errors all reported at startup, rather than on use at runtime

Beans can be associated with session - i.e. loggedIn
Beans can be more ephemeral, i.e. for request
Beans can be more long lived, i.e. shopping cart conversation flows
Can add class for beans on the fly, via APIs at runtime

Example of DI annotation:

@Inject
CheckoutHandler(
@LoggedIn User user,
@Reliable @PayBy(CREDIT_CARD)
PaymentProcessor processor,
@Default Cart cart)

* Note that constructor injection is possible
* Note different scopes, PaymentProcessor is probably conversation scope, LoggedIn is session, PaymentProcessor is probably application scope singleton

* Instance and state management handled "for free" by framework/APIs

JAX-RS 1.1:
Already widely adoped
Really a high level HTTP API
Annotation-based programming model
Programmatic API when needed

* think of it as the new HTTP level API (higher level than HTTPServlets, remove low level detail and tedium)

Jax-RS resource class, example:

Identified by the @Path annotation

@Path("widgets./{id}")
@Produces("application/widgets+xml")
public class WidgetResource {
pubic WidgetResource(@PathParam("id") String id { … }

@GET
Widget getWidget() { … }
}

Provides higher level HTTP handling, more declarative, better match with conceptual needs of developer

Bean Validation 1.0:

Integrated with JSF, JPA
Constraints represented by annotations

e.g.

@NotNull
@Size(max=40)
String address;

Fully extensible
@Email
String recipient;

Validation API's for validation directly, create a new validation object etc
Validation of trees of objects is possible (including loops)

JPA 2.0:

Supported for collections of basic types and embeddable objects
JPQL enhancements e.g. CASE WHEN, NULLIF
Pessimistic locking added (annotations added)
Criteria API for dynamic query construction

Criteria API: Uses the canonical metamodel classes

CriteriaBuilder, create criteria
CriterialQuery, typed criteria

Strongly types checking, type parasitised equals checking, compiler errors generated if query does not have the right types etc, so robust and safe query (rather than say String SQL construction directly).

Connectors 1.6 added too (not covered in any detail)

Summary overview:

Improved, more powerful, more flexible, more extensible, easier to use

http://java.sun.com/javaee

--

GlassFish V3 (Alexis Moussine-Pouchkine):

Java EE 6 Reference Implementation (RI)

Geographic download map:

http://maps.glassfish.org/server

Healthy increase in downloads and usage over time

GlassFish V1 first shipped 2006, reusing much from Tomcat

V2.1.1 Nov 2009, V3 (Java EE 6) Dec 10th 2009

GlassFish V3 Open Source CDDL, GPL (with 'classpath exception') licensing

Java EE 5 & 6, enterprise quality - full support is available.

Sub projects:

Jersey (JAX-RS)
Metro (JAX-WS)
Grizzly (NIO)
Atmosphere (Comet)
OpenMQ (JMS)
and scripting jRoR Grails and now Django (python)

Main difference from Tomcat - Grizzly core (rewritten)

Netbeans 6.8 tooling
Support available in Eclipse too

GlassFish development continues
Support contracts through to 2017+ unlimited
Remains the Java EE reference implementation
Now also sold with WebLogic and standalone

Roadmap -> expected soon for remaining year

Don't have to deliver as much standard runtime / frameworks jars as part of the application jar

Netbeans in-place edit of classes, incremental compilation, deploy on save, GF v3 preserves session across redeployments(!)

Session retention:
Deployment option to maintain statefull sessions across re-deployments!

GlassFish v3 key goals:

Modular and dynamic
Modular: Apache Felix (OSGi)
Extensible: HK2 (100k kernel)
Still very fast!

Centralized configuration, modules configured through centralised control

Key Features:

No ejbjar.xml needed, no web.xml, annotation driven, EJB's as single classes

Declarative annotations:

@Stateless annotation for class stateless bean
@Schedule annotation for timers

Eclipse - GlassFish tool bundle for eclipse (contains everything!)

Ultra fast auto-deploy of all Java EE and static artefacts

Maven support: mvn gf:run gf:start gf-deploy

Containers can be added / removed dynamically

New API for EJB testing (EJBContainer):

Example:

EJBContainer c = EJBContainer.createEJBContainer();
Context ice = c.getContext();
SimpleEjb ejb (SImpleEjb)ic.lookup("java:global/sample/SimpleEjb");
ejb.sayHello();

GlassFish "Embedded" - allows all features of GlassFish to be automated

org.glassfish.api.embedded.Server server;
Server.Build builder = new Server.Builder();
server = builder.build();
ContainerBuilder b = server.createConfig(ContainerBuilder.Type.web);
server.addContainer(b);

File archive = new File("hello.war");
server.getDeployer().deply(archive);

i.e. Ship app server inside an application!

OSGi:

GlassFish runs on top of OSGi (Felix by default)
Also runs unmodified on Knopflerfish and Equinox
GlassFish ships with 200+ bundles
Can run without OSGi (static mode based on HK2)
Can use OSGi management tools (CLI or Web)

Any OSGi bundles will run in GlassFish v3
Drop it in glassfish/modules

Servlets can get hold of OSGi bundles (using @Resource DI)

Update centre:

Graphical tool (GlassFish does not need to be started), available in web admin console
CLI version available
Was in v2.x but not from admin console

RESTful admin API:

JAX-RS/Jersey + Grizzly to provide REST interfaces to
Configure runtime (via GET, POST, DELETE)
Invoke commands (restart, stop, deploy, etc)
Monitoring (GET only)
Log rotation etc

e.g. Available from:
localhost:4848/management/domain
localhost:4848/monitoring/domain

Further advantages:

Dynamic language support via modules:
Rails, Grails, Django, Scala/Lift

Comet:
Cometd/Bayeux
Atmosphere

Full support for:
mod_jk
WebDAV, CGI, SSI

OpenMQ 4.4

Web Services Metro 1.4
.NET 3.5 interoperability

v3 Clustering:
Lower priority after Java EE 6 and modularity, so not yet...
Clustering is not built in as per v2
More similar to v1, single instance
Have to take on own clustering (load balancing, deployment)
See roadmap for details...

Doing More with GlassFish (Steve Elliott):

GlassFish v3 - Management and monitoring

Management:

User Friendly, pluggable and extensible for administration
Feature rich Admin console (GUI)
Easy to use Command Line Interface (CLI)
RESTful management and monitoring API
Fully documented AMX API (app server management API)
All management features built on AMX API

OSGi, load on demand = fast initial start up

v3 AdminConsole:
Frame-set removed, now Ajax based pages
Pluggable console (admin panes, trees etc loaded on demand)

Monitoring:

Lightweight prode architecture
Ad hoc monitoring in Production
Client-scripting (JavaScript)
DTrace integration on Solaris (similar to OS probes, uniform tracing experience with MySQL etc)
Extensibility / Pluggability

No overhead when there is no monitoring
Allows Monitoring to be turned on in a production environment with minimal impact
Generate and listen to only interested
Turn on monitoring when needed
BTrace integration
Portable and dynamic

Instrumentation:

Modules expose probes
POJO with annotations
XML

Modules register probe listeners

In code, POJO annotations can be used

@ProbeProvider(providername="glassfish", modulename="web")

ProbeProvider XML configuration also possible

ProbeListeners
JMX exposed

@ManagedAttribute(id="jspcount")

OSGi:

big move to OSGi technology
Big move to more modular development approach

Demands and enforces stronger modularity

OSGi is largely under the covers
Visible to GlassFish developers, but not to GlassFish users

Service based architecture:

Core modules loaded on app startup
Rest loaded on demand

Module Management:
add, remove, update installed modules

OSGi as a container

Web services - Metro : JAX-WS / Jersey : JAX-RS

Metro - SOAP-based web services stack
Built into GlassFish
Works with any servlet 2.5 compliant web container
WebLogic, WebSphere, JBoss, Tomcat
Also standalone
Advances interoperability with .NET 3.x/4.0

Project Tango - focused on interoperability with .NET

JAXB based XML Data Binding (XSD, XPATH)

SOAP Messaging MTOM etc

Bi-directional interoperability with .NET (Java or .NET as client or server)

Standards:

JCP: JAX-WS 2.2 & JAXB 2.2
W3C SPAP 1.1/1.2 WSDL 1.1, WS-Addressing
… etc

JAX-RS:

JAX-RS 1.1 is final and part of EE6

Not a web profile
but included with GlassFish v3 web profile
JCP 311
Spec - JSR 311

http://www.oracle.com/java
Contains links to GlassFish etc

.

Project Euler 12 - Scala Streams, creating triangle numbers

2009-11-01T11:52:00.001-08:00

Just a short blog today on Scala Streams and generating numeric progressions, such as a simple arithmetic series

This is a solution to project Euler problem 12: "What is the value of the first triangle number to have over five hundred divisors?"

This is actually a fairly easy problem to solve, providing you use a sufficiently efficient function to count the divisors!

The method I've adopted for this is particularly simple, but performs sufficiently well to solve this problem as stated within a few seconds.

Triangle Numbers:

Just a quick refresher, the triangle numbers are the sum of integers from 1 to n.

An Arithmetic series is a number series of the form:

a + (a + b) + (a + 2b) + ....

and using this form the Triangle numbers can be written as:

1 + (1 + 1) + (1 + 2) + (1 + 3) + ...

so they clearly are of the form of an Arithmetic series.

The more interesting aspect of this problem was the generation of the Triangle number sequence. Naturally, wanting to use Scala to solve this problem, the idea of using a Stream to evaluate the triangle number sequence lazily on demand is a natural approach.

The implementation I settled on to produce the Stream of triangle numbers was the iterative version as follows:


val trianglesI = Stream.iterate((1L,0L)){
            case (n, sum) => (n + 1L, sum + n) }.map(_._2)

Which is a fairly natural way to solve via the additive definition of the triangle numbers.

What's perhaps more interesting is finding some alternative ways to achieve the same thing, here are some alternatives to the iterative additive stream definition:

Using the multiplicative definition of the Triangle numbers and the map function:


// use a stream map and the multiplicative solution for triangle numbers
val trianglesM = Stream.from(1).map(n => n * (n + 1) / 2)
       
println("trianglesM take(5)")
trianglesM.take(5).foreach(println)

Using "inits" - inits are a concept that can be found in languages such as Haskell:


// using inits (not efficient)
// inits -> inits(Stream.from(1)) is Stream(Stream(), Stream(1), Stream(1, 2), Stream(1, 2, 3), ...)
// and tails -> tails(Stream(1)) = Stream(Stream(1,2,3,...), Stream(2,3,4,...), Stream(3,4,5,...), ...)
def inits[T](xs: Stream[T]) = Stream.from(0).map(xs.take(_))
      
val trianglesInits = inits(Stream.from(1)).tail.map(_.sum)

println("trianglesInits take(5)")
trianglesInits.take(5).foreach(println)

Using Stream.cons and a helper function zipWith that can zip tuples:


// using zipWith
def zipWith[T1,T2,T3](s1: Stream[T1], s2: Stream[T2])(fn: (T1,T2) => T3) = 
  s1.zip(s2).map(Function.tupled(fn))
       
lazy val trianglesZ: Stream[Int] = Stream.cons(1, zipWith(trianglesZ, 
  Stream.from(2))(_ + _))
       
println("trianglesZ take(5)")
trianglesZ.take(5).foreach(println)

So, whilst problem 12 is not too hard, the production of triangle numbers as lazy lists is quite interesting, being that there are several approaches that can be taken!

The whole program solution to problem 12 (and the alternative triangle number streams)


package projecteuler

object Problem12 {

 def main(args : Array[String]) {
 
  println("result = " + problem12(500))
 }
 
 def problem12(target : Int) {
 
  // using Stream.iterate to create a lazy stream of triangle numbers
  val trianglesI = Stream.iterate((1L,0L)){ 
   case (n, sum) => (n + 1L, sum + n) }.map(_._2)
  
  // test it
  //println("trianglesI take(5)")
  //trianglesI.take(5).foreach(println)
  
  // count the divisors (not massively efficient, but sufficient)
  def numDivisors(n : Long) = {
   
   var count = 0
   
   for (i <- 1L to Math.sqrt(n).toInt) {
    if (n % i == 0) {
     count = count + 2
    } 
   }
   count
  }
  
  trianglesI.find(p => numDivisors(p) > target)
  }
 
  def otherTriangleStreams {
   
    // use a stream map and the multiplicative solution for triangle numbers
  val trianglesM = Stream.from(1).map(n => n * (n + 1) / 2)
  
  println("trianglesM take(5)")
  trianglesM.take(5).foreach(println)
  
  // using inits (not efficient)
  // inits -> inits(Stream.from(1)) is Stream(Stream(), Stream(1), Stream(1, 2), Stream(1, 2, 3), ...)
  // tails -> tails(Stream(1)) = Stream(Stream(1,2,3,...), Stream(2,3,4,...), Stream(3,4,5,...), ...)
  def inits[T](xs: Stream[T]) = Stream.from(0).map(xs.take(_))
  
  val trianglesInits = inits(Stream.from(1)).tail.map(_.sum)
  
  println("trianglesInits take(5)")
  trianglesInits.take(5).foreach(println)
  
  // using zipWith
  def zipWith[T1,T2,T3](s1: Stream[T1], s2: Stream[T2])(fn: (T1,T2) => T3) = s1.zip(s2).map(Function.tupled(fn))
  
  lazy val trianglesZ: Stream[Int] = Stream.cons(1, zipWith(trianglesZ, Stream.from(2))(_ + _))
  
  println("trianglesZ take(5)")
  trianglesZ.take(5).foreach(println)
  }
}

Scala & GUIs, a simple maze generator and solver applet

2009-09-27T09:31:00.000-07:00

In this blog we will touch on using Scala for a simple GUI, in the form of a simple 2D Maze Applet.

An example of this applet can be viewed running from here http://www.chillipower.com/examples/maze/maze1.html to give an illustration of what it does. This uses a maze space of 100x100 cells on a regular grid.

The applet illustrates 2 aspects of GUI programming in Scala, the first is how to interact with basic javax.swing APIs directly and the second is how to go one step further and achieve the same effect using the scala.swing wrapper API instead.

The function of the Applet can be broken down into 3 simple steps:

Step 1: Produce a random maze. This uses a modified version of Prim's algorithm, running backwards from the exit until all of the available space has been covered. This will produce a simple path from the source to the exit, which is the path we seek when we try to solve the maze by starting at the start. The standard Prim's algorithm is used to produce a minimum spanning tree from a directed acyclic graph by recursively adding safe edges to the current minimum spanning tree recursively, starting at a source node.

Step 2: Use a regular (recursive) DFS algorithm to search the maze, stopping only when the exit is reached (or all paths are explored fully if there were no reachable exit).

Step 3: Remove all backtrack paths and show the single simple path from start to exit as the solution to the maze.

During maze exploration, the parts of the maze explored are marked with coloured breadcrumbs, red indicates a forward search path and blue a backtracking path (which happens when a path reaches a dead-end).

In step 2, in order to stop immediately when the exit is found (rather than conclude all other possible search branches) the recursive maze solving function throws an exception to escape out of the call stack immediately.

The code given is compatible with the very latest Scala 2.8.x trunk development builds, and with 2.7.6 the latest stable release (some 2.7.6 only versions of functions such as shuffle are commented out in preference to the newer and better 2.8 versions). Note that there are currently some issues with Arrays in 2.8.x causing run time errors with trying to create a multidimensional array. The solution that still works with current 2.8 builds is to use the Array.tabulate function.

i.e. the following works in 2.7.6 but not currently in 2.8:

var cells : Array[Array[Cell]] = new Array[Array[Cell]](WIDTH, HEIGHT)

and there are similar problems with the new Array.ofDims function too. These issues should be resolved in 2.8 final release but recently arrays in scala have been undergoing some re-factoring and so stability in this area has not been too strong.

One interesting thing to note is the use of the new Scala 2.8+ Stream.iterate function to avoid undesirable vars instead of vals, here is an example of that extracted from the code.

A typical iterative piece of code, the while loops is used to iterate over some code block whilst a particular condition is met for a given variable. For this to work the variable n needs to be updated in the loop, hence it can't be a val and has to be a var instead. This is somewhat paradigmatic of iterative imperative programming, but doesn't sit well with the strive for a more functional style.


var n = cells(exit.x)(exit.y)

while (n != null) {

 n.trail = Forward
 n = n.pi
 update
}

What we can do here is make use of the Stream.iterate function to avoid the unsightly var n, as follows:


Stream.iterate(cells(exit.x)(exit.y))(_.pi).takeWhile(_ != null).foreach{
 n => {
   n.trail = Forward
   update
 }
}

Given that this is new in Scala 2.8 and 2.8 isn't released yet, you can augment existing 2.7.6 code with a helper function to aid the transition and improve 2.7 based code in readiness for 2.8. Here is an example of a util class that defines iterate so the same idea can be used in 2.7 based code:


def iterate[T](x:T)(f : T => T) : Stream[T] =
   Stream.cons(x, iterate(f(x))(f))

Another interesting thing to note in the code is the use of a permutation on a list (to create random directions when generating or exploring the maze), here is the code:


def shuffle[T](xs: List[T], r: scala.util.Random) = {
   xs.toStream.zip(Stream.const(r.nextDouble _).map(_())).toList.sort(_._2 < _._2).map(_._1)
}

The code maps the elements with a stream of random real numbers and then permutes by sorting the list, extracting the sorted items to create the random permutation (neat huh!)

The above code is 2.7.x & 2.8+ compatible, but perhaps even better is the following 2.8+ code that makes use of the new Stream.continually function:


def shuffle[T](xs: List[T])(implicit r: scala.util.Random) = {
 xs.zip(Stream.continually(r.nextDouble)).sortWith(_._2 < _._2).map(_._1)
}

In terms of the GUI aspects, the application is an applet, and the example code shows 2 ways of achieving this in Scala. The first, using the swing API directly is as follows:


class MazeApplet2 extends JApplet {

 override def init() {
   ...
 }

 override def start() {
 }
}

Which is familiar to any Java programmer, but the more interesting approach is using the scala.swing API instead, whereby we have:


class MazeApplet extends Applet {

 object ui extends UI with Reactor {

   def init() = {
     ...
   }

   override def start() = {
   }
 }
}

Where the abstract ui attribute (of type UI) is implemented using the Reactor mixin trait (http://www.scala-lang.org/docu/files/api/scala/swing/Reactor.html), to provide the framework for event handling.

One thing that's quite clear is the obviously similarity between the function to generate and the function to solve the maze. Using the DSL-like features of Scala it should be possible to extract the common control structure / algorithm and create a new utility function to provide built-in like support for this, passing in a body function to provide the difference implementation of maze generate vs. solve.

Full code example:


/**
* Scala 2.8+ Maze Applet v0.1 20-09-2009 Louis Botterill
*/
package maze

import scala.swing.Applet
import scala.swing.Panel
import scala.swing.Reactor

import javax.swing._;
import java.awt.Graphics;
import java.awt.Color;


object Direction extends Enumeration {
 type Direction = Value
   val North, South, East, West = Value
}

object Breadcrumb extends Enumeration {
 type Breadcrumb = Value
   val Forward, Backward, Clear = Value
}

import Direction._
import Breadcrumb._

class MazeModel {
 val HEIGHT = 100
 val WIDTH = 100

 // instantiate and initialise the 2d array - there is a direct 2d version but not working currently in scala 2.8.x
 var cells = Array.tabulate(WIDTH)(i => Array.tabulate(HEIGHT)(j => Cell(i, j)))

 // set everything as not visited and with no trail
 def clearVisited() = {

   cells.foreach(_.foreach(c => {
         c.visited = false;
         c.trail = Clear
     }))
 }

 // generate maze
 def generateMaze(update :  => Unit) {

   val exit = new Point(WIDTH - 1, HEIGHT - 1)
   val start = new Point(0, 0)

   // 1. Start at a particular cell, call it the "exit"
   // 2. Mark the current cell as visited, and get a list of all its neighbors.
   //      For each neighbor, starting with a randomly selected neighbor:
   //   1. If that neighbor hasn't been visited,
   //        remove the wall between this cell and that neighbor,
   //        and then recurse with that neighbor as the current cell.

   // cell 0,0 is the start, open the north wall to highlight this
   var s = cells(0)(0)
   s.clear(North);

   // cell width-1,height-1 is the exit, open the south wall to highlight this
   val c = cells(exit.x)(exit.y)
   c.clear(South);

   // recursively process the next cell
   def doNextCell(c : Cell) {

     c.visited = true
     c.trail = Forward

     var dirs = c.getRndDirections()

     while (!(dirs isEmpty)) {
       val dir = dirs.head
       val x = getCell(c, dir)

       x match {
         case Some(n) =>
          
           if (n.visited != true) {
             n.visited = true
             c.clear(dir)
             n.clear(getInv(dir))
             n.trail = Forward
             update
             doNextCell(n)
             n.trail = Clear
             update
           }

         case None =>
       }

       dirs = dirs.tail

       //Thread.sleep(1)
     }
   }

   doNextCell(c)
 }

 // find the maze solution using dfs from the start node until the end
 // node is located
 def solveMaze(update :  => Unit) {
  
   val start = new Point(0, 0)
   val exit = new Point(WIDTH - 1, HEIGHT - 1)

   val s = cells(0)(0)
   s.clear(North);

   def doNextCell(c : Cell) {

     c.visited = true
     c.trail = Forward

     var dirs = c.getDirections()
    
     while (!(dirs isEmpty)) {
       val dir = dirs.head
       val x = getCell(c, dir)

       x match {
         case Some(n) =>

           if (n.visited != true) {
             n.visited = true
             n.pi = c  // set predecessor node
             n.trail = Forward
             update
             if (n.i == exit.x && n.j == exit.y) throw new Exception("Done")
             doNextCell(n)
             n.trail = Backward
             update
           }

         case _ =>
       }

       // take the top item off and use the rest of the list
       dirs = dirs.tail
      
       Thread.sleep(1) // add a little delay so we can watch the dfs explore and find the solution
     }
   }

   doNextCell(s)
 }

 // from the exit recurse over the trail of predecessors,
 // marking with a forward trail
 def showSolution(update :  => Unit) {

   val exit = new Point(WIDTH - 1, HEIGHT - 1)

   // 2.8 avoid var using Stream
   // Stream.iterate(cells(exit.x)(exit.y))(_.pi).takeWhile(_ != null)
   Stream.iterate(cells(exit.x)(exit.y))(_.pi).takeWhile(_ != null).foreach{
     n => {
       n.trail = Forward
       update
     }
   }

   // the following works fine but replaced with the above to avoid unnecessary var
   /*
   var n = cells(exit.x)(exit.y)

   while (n != null) {

     n.trail = Forward
     n = n.pi
     update
   }
   */
 }

 // get the cell if possible based on current cell and the given direction
 def getCell(c : Cell, dir : Direction) : Option[Cell] = dir match {

   case North  => if (c.j > 0) Some(cells(c.i)(c.j-1)) else None
   case South  => if (c.j < east   =""> if (c.i < west   =""> if (c.i > 0) Some(cells(c.i-1)(c.j)) else None
 }

 // find the inverse direction for a given direction
 def getInv(dir : Direction) = dir match {

   case North => Direction.South
   case South => Direction.North
   case East => Direction.West
   case West => Direction.East
 }
}

// a scala.swing.Panel, override paint(Graphics2D) to paint each of the cells
class MazePanel(m : MazeModel) extends Panel {

 override def paint(g : java.awt.Graphics2D) {

   m.cells.foreach(_.foreach(_.draw(g)))
 }
}

// a javax.swing.JPanel, override paint(Graphics) to paint each of the cells
class MazePanel2(m : MazeModel) extends JPanel {

 override def paint(g : Graphics) {

   m.cells.foreach(_.foreach(_.draw(g)))
 }
}

// a scala.swing.Applet based applet using scala swing wrappers
class MazeApplet extends Applet {
 val m = new MazeModel()
 lazy val mp = new MazePanel(m)

 object ui extends UI with Reactor {

   def init() = {
     contents = mp
   }

   override def start() = {
     def update = {
       this.repaint()
     }

     m.generateMaze(update)

     Thread.sleep(2000)

     m.clearVisited

     try {
       m.solveMaze(update);
     }
     catch {
       case e : Exception =>
     }

     Thread.sleep(2000)

     m.clearVisited

     m.showSolution(update)

     mp.repaint()
     this.repaint()
   }
 }
}

// standard javax.swing.JApplet of the maze
class MazeApplet2 extends JApplet {

 val m = new MazeModel()
 lazy val mp = new MazePanel2(m)

 def MazeApplet2 = {
 }

 override def init() {

   this.add(mp)
 }

 override def start() {
   def update = {
     this.repaint()
   }
  
   m.generateMaze(update)

   Thread.sleep(2000)

   m.clearVisited

   try {
     m.solveMaze(update);
   }
   catch {
     case e : Exception =>
   }

   Thread.sleep(2000)

   m.clearVisited

   m.showSolution(update)

   mp.repaint()
   this.invalidate()
   this.repaint()
 }

 override def stop() {
 }

 override def destroy() {
 }
}

// standard Java app of the maze using JFrame
object Maze {

 def main(args: Array[String]) {

   val mazeApplet = new MazeApplet()
   val frame = new JFrame()
   frame.setBounds(50, 50, 750, 750)
   frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);

   frame.add(mazeApplet)

   frame.setVisible(true)
   mazeApplet.init
   mazeApplet.start
 }
}

// represent a Cell of the maze
case class Cell(i: Int, j : Int) {

 val size = 7; // each cell is 7px by 7px
 var north : Boolean = true
 var south : Boolean = true
 var east : Boolean = true
 var west : Boolean = true
 var visited : Boolean = false
 var pi : Cell = null  // predecessor cell
 var trail : Breadcrumb = Clear;

 def draw(g : Graphics) {

   val x = i * size
   val y = j * size

   trail match {
     case Forward => {
       g.setColor(Color.RED);
       g.fillRect(x + 2, y + 2, size - 4, size - 4)
     }
     case Backward => {
       g.setColor(Color.BLUE);
       g.fillRect(x + 2, y + 2, size - 4, size - 4)
     }
     case _ =>
   }

   g.setColor(Color.BLACK);
   if (north) {
     g.drawLine(x, y, x + size, y);
   }
   if (south) {
     g.drawLine(x, y + size, x + size, y + size);
   }
   if (east) {
     g.drawLine(x + size, y, x + size, y + size);
   }
   if (west) {
     g.drawLine(x, y, x, y + size);
   }
 }

 def clear(dir : Direction) = dir match {

   case North => north = false
   case South => south = false
   case East => east = false
   case West => west = false
 }

 def getDirections() : scala.List[Direction] = {

   var d = scala.List[Direction]()

   if (!north) d = North :: d
   if (!south) d = South :: d
   if (!east) d = East :: d
   if (!west) d = West :: d

   d
 }

 def getRndDirections() : scala.List[Direction] = {

   val directions = North :: South :: East :: West :: Nil
   implicit val r = Rand.rand
   Utils.shuffle(directions) //, Rand.rand)
 }
}

// represent a 2d point
case class Point(x : Int, y : Int)

// init a random generator singleton
object Rand {
 val rand : scala.util.Random = new scala.util.Random();
}

object Utils {

 // ref: http://okmij.org/ftp/Haskell/perfect-shuffle.txt
 // scala 2.7+ here's an O(n log n) solution:
 /*
 def shuffle[T](xs: List[T], r: scala.util.Random) = {
   xs.toStream.zip(Stream.const(r.nextDouble _).map(_())).toList.sort(_._2 < nil =""> scala.List(Nil)
   case _ => xs.flatMap(x => permute(xs.filter(_ != x)).map(x :: _))
 }

 // make a stream of x, f(x), f(f(x)), etc.
 // in Haskell this is called "iterate".  it ought to be in the standard library
 // as Stream.iterate.  "unfold" should be more general, but nonetheless I'm
 // going to call this unfold for the moment...
 def unfold[T](x:T)(f:T=>T):Stream[T] =
   Stream.cons(x,unfold(f(x))(f))

 def iterate[T](x:T)(f:T=>T):Stream[T] =
   Stream.cons(x,iterate(f(x))(f))
}

Scala, trees, pattern matching and tree recursion

2009-08-31T11:31:00.000-07:00

Hi! Long time no see...

Well after some outage due to Holiday (road trip around Europe and down to the Swiss Alps!), work commitments and some personal things to deal with, I'm back again.

I thought I'd start with some stuff on Tree structures, whilst continuing on with Scala too. The following code illustrates some good examples of case classes and pattern matching, along with extensive use of Some, None and Option (Scala's improved way of dealing with optional types, rather than relying on null everywhere).

The following code defines a simple recursive binary tree definition as a case class, using option to define the left and right children.


case class Tree[+T](value: T, left: Option[Tree[T]], right: Option[Tree[T]])

The above definition defines a tree recursively, exactly as a tree is defined as a data structure in computer science. Each tree node has a value of type T and an Optional left and right child node (bin tree). The +T makes the Tree type covarient about T, which is a generally useful property of a Tree.

A few helper functions are used that make life a little easier when working with Option types, by extracting the value or providing a safe default when no value exists. These help keep the code a little tidy and manageable.


 /**
  * Helper function: For o return f: A => Int if Some or 0 if None
  *   also could try opt map f getOrElse 0
  */
 def zeroIfNone[A](o: Option[A], f: A => Int): Int = o match {
   case None => 0
   case Some(x) => f(x)
 }

 /**
  * Helper function: Return value of function provided if Some, or 0 otherwise
  */
 def zeroIfNoneVal[A](o: Option[A], n: => Int) = zeroIfNone[A](o, _ => n)

 /**
  * Helper function: Default to provided value if option is None
  */
 def defaultIfNone[A](o: Option[A], f: A => Int, d : Int): Int = o match {
   case None => d
   case Some(x) => f(x)
 }

The example code demonstrates preorder, inorder and postorder recursive tree traversal (which are all simple permutations of a basic depth first search), applying the provided unit function to each node as it is visited.

Recall that the preorder, inorder and postorder functions are defined as follows:

PreOrder - visit parent node, left and then right child node
InOrder - visit left, parent and then right nodes
PostOrder - visit left, right and then parent node

Using recursion and the call stack, depth first search based tree traversal is very simple and concise. The basic DFS inorder traversal looks like:


 /**
  * Perform an inorder tree traversal using the provided Tree[A] => Unit function
  */
 def inOrder[A](t: Option[Tree[A]], f: Tree[A] => Unit) : Unit = t match {

   case None =>
   case Some(x) =>
     if (x.left != None) inOrder(x.left, f)
     f(x)
     if (x.right != None) inOrder(x.right, f)
 }

The breadth first search algorithm is perhaps more interesting. This uses a List as a basic FIFO queue to provide the breadth first search tree traversal:


 /**
  * Breadth first search on a tree, a list is used as a basic fifo queue to
  *   give the bfs traversal
  */
 def breadthFirstSearch[A](t: Tree[A], f: Tree[A] => Unit): Unit = {
   def bfs(ts: List[Tree[A]]): Unit = ts match {
     case Nil => // ignore
     case _ =>
       val children = for{tree <- ts         
         Some(child) <- List(tree.left, tree.right)}         
         yield child      
     ts map f     
     bfs(children)   
   }     
   bfs(List(t))  
 }

The example also shows a treeFold function. Whether this is a useful function as-is is perhaps debatable, but it makes for an interesting example. The code traverses the tree in a DFS inorder traversal and applies the mapping function f : A -> B and then it applies the join function j : B, B -> B to combine the results of the tree map. In this way the function is similar to say flatMap where a list is mapped and then flattened into a single list, or perhaps more closely to foldLeft/foldRight where items in a list a joined into a single resulting value.


 /**
  * Fold up a tree into a single value, using provided map and join functions
  */
 def treeFold[A, B](t: Option[Tree[A]],
                    f: Option[Tree[A]] => Option[B],
                    j: (Option[B], Option[B]) => Option[B]) : Option[B] = t match {

   case None => None
   case Some(x) => j(j(f(t), treeFold(x.left, f, j)), treeFold(x.right, f, j))
 }

Some utility functions are also defined, to return the size and depth of the tree for illustration of how this can be achieved.

In summary Scala can quite elegantly represent Tree data structures, using case classes and operate over them using pattern matching and higher order functions to provide powerful and concise support for trees and the operations that can be performed on them.

Some additional things that could be done here would be to provide an example of a binary search and perhaps insertion, update and delete too. A job for another followup blog perhaps...

The full example code (is complete and self contained) is as follows:


package trees

// Define a recursive Bin Tree structure
case class Tree[+T](value: T, left: Option[Tree[T]], right: Option[Tree[T]])

object TreeObj {

 /**
  * Helper function: For o return f: A => Int if Some or 0 if None
  *   also could try opt map f getOrElse 0
  */
 def zeroIfNone[A](o: Option[A], f: A => Int): Int = o match {
   case None => 0
   case Some(x) => f(x)
 }

 /**
  * Helper function: Return value of function provided if Some, or 0 otherwise
  */
 def zeroIfNoneVal[A](o: Option[A], n: => Int) = zeroIfNone[A](o, _ => n)

 /**
  * Helper function: Default to provided value if option is None
  */
 def defaultIfNone[A](o: Option[A], f: A => Int, d : Int): Int = o match {
   case None => d
   case Some(x) => f(x)
 }

 /**
  * Return the size of the tree
  */
 def size[A](t : Option[Tree[A]]) : Int = t match {

   case None => 0
   case Some(x) =>
     zeroIfNoneVal(x.left, size(x.left)) + zeroIfNoneVal(x.right, size(x.right)) + 1
     //zeroIfNone(x.left, (_:Tree[Any]) => (size(x.left))) +
     //zeroIfNone(x.right, (_:Tree[Any]) => (size(x.right))) + 1
 }

 /**
  * Alternative size using nested function
  */
 def size2[A](t : Option[Tree[A]]) : Int = {

   def sizeB(b : Option[Tree[A]]) : Int = b match {
     case None => 0
     case Some(x) => size(b);
   }

   t match {
     case None => 0
     case Some(x) =>
       sizeB(x.left) + sizeB(x.right) + 1
   }
 }

 /**
  * Return the max depth of the tree
  */
 def depth[A](t : Option[Tree[A]]) : Int = t match {

   case None => 0
   case Some(x) =>
     Math.max(zeroIfNoneVal(x.left, depth(x.left)), zeroIfNoneVal(x.right, depth(x.right))) + 1
 }

 /**
  * Alternative deptch, return the max depth of the tree using nested function
  */
 def depth2[A](t : Option[Tree[A]]) : Int = {

   def depthB(b : Option[Tree[A]]) : Int = b match {
     case None => 0
     case Some(x) => depth(b);
   }

   t match {

     case None => 0
     case Some(x) =>
       Math.max(depthB(x.left), depthB(x.right)) + 1
   }
 }

 /**
  * Breadth first search on a tree, a list is used as a basic fifo queue to
  *   give the bfs traversal
  */
 def breadthFirstSearch[A](t: Tree[A], f: Tree[A] => Unit): Unit = {
   def bfs(ts: List[Tree[A]]): Unit = ts match {
     case Nil => // ignore
     case _ =>
       val children = for{tree <- ts           Some(child) <- List(tree.left, tree.right)}           yield child       ts map f       bfs(children)     }      bfs(List(t))   }    /**    * Sum up the elements of a numeric tree    */   def sumTree[A <: Int](t: Option[Tree[A]]) : Int = t match {      case None => 0
   case Some(x) =>
     sumTree(x.left) + sumTree(x.right) + x.value
 }

 /**
  * Fold up a tree into a single value, using provided map and join functions
  */
 def treeFold[A, B](t: Option[Tree[A]],
                    f: Option[Tree[A]] => Option[B],
                    j: (Option[B], Option[B]) => Option[B]) : Option[B] = t match {

   case None => None
   case Some(x) => j(j(f(t), treeFold(x.left, f, j)), treeFold(x.right, f, j))
 }

 /**
  * Perform an inorder tree traversal using the provided Tree[A] => Unit function
  */
 def inOrder[A](t: Option[Tree[A]], f: Tree[A] => Unit) : Unit = t match {

   case None =>
   case Some(x) =>
     if (x.left != None) inOrder(x.left, f)
     f(x)
     if (x.right != None) inOrder(x.right, f)
 }

 /**
  * Perform a preorder tree traversal using the provided Tree[A] => Unit functio
  */
 def preOrder[A](t: Option[Tree[A]], f: Tree[A] => Unit) : Unit = t match {

   case None =>
   case Some(x) =>
     f(x)
     if (x.left != None) inOrder(x.left, f)
     if (x.right != None) inOrder(x.right, f)
 }

 /**
  * Perform a postorder tree traversal using the provided Tree[A] => Unit functio
  */
 def postOrder[A](t: Option[Tree[A]], f: Tree[A] => Unit) : Unit = t match {

   case None =>
   case Some(x) =>
     if (x.left != None) inOrder(x.left, f)
     if (x.right != None) inOrder(x.right, f)
     f(x)
 }

 /**
  * build tree and run tree functions on it
  */
 def main(args : Array[String]) {

   /* Test Tree structure

               1
              / \
             2   3
            / \ / \
           4  x 5  6
         
   */

   // build a tree
   val myTree = Tree(1,
     Some(Tree(2,
       Some(Tree(4, None, None)),
         None
       )
     ),
     Some(Tree(3,
       Some(Tree(5, None, None)),
         Some(Tree(6, None, None))
       )
     )
   )

   printBfsTree(myTree)

   println("Size = " + size(Some(myTree)))

   println("Depth = " + depth(Some(myTree)))

   println("Sum = " + sumTree(Some(myTree)))
  
   var sum = treeFold[Int, Int](
               Some(myTree),
               (t: Option[Tree[Int]]) => t match { case None => Some(0); case Some(x) => Some(x.value) },
               (x: Option[Int], y: Option[Int]) =>
                 Some(zeroIfNone[Int](x, (x:Int) => x) + zeroIfNone[Int](y, y => y))
             )

   var mult = treeFold[Int, Int](
               Some(myTree),
               (t: Option[Tree[Int]]) => t match { case None => Some(0); case Some(x) => Some(x.value) },
               (x: Option[Int], y: Option[Int]) =>
                 Some(defaultIfNone[Int](x, (x:Int) => x, 1) * defaultIfNone[Int](y, y => y, 1))
              )

   println("Sum Result = " + sum)
   println("Mult Result = " + mult)

   println("InOrder = ")
   inOrder(Some(myTree), (t: Tree[Any]) => println(t.value))

   println("PreOrder = ")
   preOrder(Some(myTree), (t: Tree[Any]) => println(t.value))

   println("PostOrder = ")
   postOrder(Some(myTree), (t: Tree[Any]) => println(t.value))
 }

 /**
  * Print a tree using BFS, passing in print unit function
  */
 def printBfsTree[A](tree: Tree[A]) {

   breadthFirstSearch(tree, (t: Tree[A]) => println(t.value))
 }
}

Notes on Adobe LiveCycle, Flash, Flex and Air

2009-07-09T11:01:00.000-07:00

Having recently attended Adobe LiveCycle training at Adobe Uxbridge, here are a few notes and links from it.

Adobe LiveCycle is an integrated Java based SOA and document management server framework. LiveCycle can be hosted on a JEE application server, including JBoss, WebSphere and WebLogic. LiveCycle requires a Database, mySql, Oracle and MS SQL are supported.

LiveCycle is offers best performance on Microsoft/MS SQLServer platforms - currently.

LiveCycle is compatible with Windows, Solaris, AIX and Linux

LiveCycle is designed to integrate with RIA technologies, such as Flash, Flex and Air.

Key LiveCycle features include:

Forms builder - a PDF based form generation engine for PDF/HTML and Flash presentation
Form Guides - based on PDF but with wizard style Flash presentation
Reader/Reader extensions - Acrobat/Reader document management including digital signatures, revocation, permissions etc
Process Management
Data Services - Web Services, support for RESTful webservices in the next version
Output - print/production services

Flash is the original rich embedded content for multimedia, video, graphics and sound - flash runs as a plugin in the browser (although stand alone players are available too).
Flex builds on Flash and adds a rich component model.
Air is Flex technology that can be run in the browser or as desktop applications transparently.

LiveCycle and Flex based technologies can use ActionScript, a form of ECMA (JavaScript), Flex itself uses an XML format called mxml to define applications and component layout.

Key Tools:

FlexBuilder - IDE for Flex/Air development
LiveCycle Workbench - for LiveCycle development including forms and process management

Main Development WebSite: http://www.adobe.com/devnet/
Main Download home: http://www.adobe.com/downloads/
Flex Downloads: http://www.adobe.com/products/flex/flexdownloads/

Flex Development center: http://www.adobe.com/devnet/flex/?view=home
TourDeFlex: http://www.adobe.com/devnet/flex/tourdeflex/
TourDeLiveLycle: http://www.adobe.com/devnet/livecycle/tourdelivecycle/
LiveCycle Cafe: http://www.adobe.com/devnet/livecycle/cafe/

LiveDocs (online documentation): http://livedocs.adobe.com/
ActionScript 3: http://livedocs.adobe.com/flash/9.0/ActionScriptLangRefV3/
Flex 3 LiveDocs: http://livedocs.adobe.com/flex/3/

Adobe Partners: https://www.adobe.com/cfusion/partnerportal/index.cfm

FlashDevelop: Getting started http://nuigroup.com/forums/viewthread/1689/

.

Scala, the Google App Engine and an iPhone client...

2009-07-05T12:35:00.000-07:00

In this Blog I wanted to explore Scala on the Google App Engine. The Google App Engine has supported Java (as well as Python) for a little while now. There have been a few blogs about running Scala on the App engine and I wanted to further demonstrate this by taking a simple Java Servlet and converting it to a Scala based web application suitable for the App Engine.

The Google App Engine home is here http://code.google.com/appengine/ and contains excellent documentation on how to get stated. I don't want to repeat the documentation, so I'll just point out the essentials relevant to this blog.

The Getting Started guide can be found here: http://code.google.com/appengine/docs/java/gettingstarted/ for Java, and a Python equivalent is available too.

This example is based on the previous blog, the Sieve of Eratosthenes as a simple prime number generator. The request accepts one parameter n whose integer value is the upper limit to generate all the primes until (from 2). The response output is JSON encoded to create a simple web service.

Starting with the very basics, the recommended project directory structure is of the form:


|-src
|---META-INF
|---primes
|-war
|---WEB-INF
|-----classes
|-------META-INF
|-------primes
|-----lib

For any Web App for the Google App Engine, one additional GAE specific file is required alongside the standard web.xml deployment descriptor. This file is called appengine-web.xml and contains the name of the Application, for example:


<appengine-web-app xmlns="http://appengine.google.com/ns/1.0">
  <application>primes-chillipower-com</application>
  <version>1</version>
</appengine-web-app>

In this simple GAE deployment descriptor the main thing is that the name of the application is specified, in this example the application was called 'primes-chillipower-com'.

Applications must be created on the App Engine, this can be done here: http://appengine.google.com/

Note that there are some limitations currently, you are allowed up to a maximum of 10 applications and applications can not be deleted once created!

The simple Java version:

In the simple Java version we can place a simple Servlet in the src directory and build using ant.

Example Java servlet:


package primes;

import java.io.IOException;
import javax.servlet.http.*;

public class PrimesServletJ extends HttpServlet {
public void doGet(HttpServletRequest req, HttpServletResponse resp)
       throws IOException {
    resp.setContentType("text/plain");

int n = 100;

String nStr = req.getParameter("n");

try {

   n = Integer.parseInt(nStr);
}
catch (Exception e) {}

int[] primes = primes(n);

resp.getWriter().println("{ \"n\" : \"" + n + "\",");
resp.getWriter().println("  \"primes\" : [ ");

for (int i = 0; i < primes.length; i++) {
   String sep = (i < primes.length - 1) ? ", " : "";
       resp.getWriter().println(" { \"" + i + "\" : \"" + primes[i] + "\" }" + sep);
   }

resp.getWriter().println(" ] }");
}

public static int[] primes(int n) {

   // initially assume all integers are prime
   boolean[] isPrime = new boolean[n + 1];

   for (int i = 2; i <= n; i++) {
       isPrime[i] = true;
   }

   // mark non-primes <= N using Sieve of Eratosthenes
   for (int i = 2; i*i <= n; i++) {

       // if i is prime, then mark multiples of i as nonprime
       // suffices to consider mutiples i, i+1, ..., N/i
       if (isPrime[i]) {
           for (int j = i; i * j <= n; j++) {
               isPrime[i * j] = false;
           }
       }
   }

   // count primes
   int primeCount = 0;
   for (int i = 2; i <= n; i++) {
       if (isPrime[i]) primeCount++;
   }

   System.out.println("The number of primes <= " + n + " is " + primeCount);

   int[] primes = new int[primeCount];
   int idx = 0;
   for (int i = 0; i <= n; i++) {
       if (isPrime[i] == true) {

           primes[idx] = i;
           idx++;
       }
   }

   return primes;
}

public static void main(String[] args) {

   int n = 100;

   if (args.length > 0 && args[0] != null) {
       n = Integer.parseInt(args[0]);
   }

   int[] primes = primes(n);

   for (int p : primes) {
       System.out.println("Prime: " + p);
   }
}
}

The ant build script goes in the root of the project, to start with I based mine on the example on the Google App Engine documentation here http://code.google.com/appengine/docs/java/tools/ant.html#Uploading_and_Other_AppCfg_Tasks

Using the standard ant script the project can be compiled and even run locally using the SDK.

ant compile

ant runserver

If you start the dev app server locally, it will be accessibly using a URL of the form:

http://localhost:8080/yourappcontextpath

Where the port 8080 is the default and can be changed in the script.

The App Engine SDK comes with appcfg.sh script in the bin directory, which can be used to perform operations such as uploading your app to the App Engine.

../../../../tools/appengine-java-sdk-1.2.1/bin/appcfg.sh update war

The Scala version:

The main tasks in converting this conventional Java Servlet based web app to a Scala based one is to:

Rewrite the Java Servlet in Scala.
Reconfigure the ant build.xml to modify the classpath to include the Scala libraries and to run the Scala scalac compiler to create the Java .class files from the .scala source files.

Since the Servlet API is a Java based API, there will of course be some reference to Java based classes in the Scala Servlet.

Example Scala based Servlet:


package primes

import javax.servlet.http._


class PrimesServlet extends HttpServlet {
override def doGet(request: HttpServletRequest, response: HttpServletResponse) = {

  var n = 100

  try {
    val nStr = request.getParameter("n"); 
    n = Integer.parseInt(nStr);
  } catch {
    case e : Exception => {
      println("Exception e " + e.getMessage())
    }
  }

  val ps = primes take n
  val out = response.getWriter()

  out.println("{ \"n\" : \"" + n + "\", \"primes\" : [")

  var i = 0
  for (p <- ps) {
    val sep = if (i < ps.length - 1) ", " else ""
    out.println(" { \"" + i + "\" : \"" + p + "\" }" + sep)
    i = i + 1
  }

  out.println(" ] } ")
}

def primes = {
 def sieve(is: Stream[Int]): Stream[Int] = {
   val h = is.head
   Stream.cons(h, sieve(is filter (_ % h > 0)))
 }

 sieve(Stream.from(2))
}
}

Example modified and build.xml file:


<project>
<property name="scala.home" location="/Users/chillipower_uk/dev/tools/scala/scala-2.7.5.final/" />
<property name="sdk.dir" location="/Users/chillipower_uk/dev/tools/appengine-java-sdk-1.2.1/" />

<import file="${sdk.dir}/config/user/ant-macros.xml" />

<path id="project.classpath">
<pathelement path="war/WEB-INF/classes" />
<fileset dir="war/WEB-INF/lib">
 <include name="**/*.jar" />
</fileset>
<fileset dir="${sdk.dir}/lib">
 <include name="shared/**/*.jar" />
</fileset>
</path>

<!-- Copy over Scala jars -->
<target name="init">
<property
 name="scala-library.jar"
 value="${scala.home}/lib/scala-library.jar"
  />
<path id="build.classpath">
 <pathelement location="${scala-library.jar}"   />

 <pathelement location="${build.dir}"   />
</path>
<taskdef resource="scala/tools/ant/antlib.xml">
 <classpath>
   <pathelement location="${scala.home}/lib/scala-compiler.jar"   />
   <pathelement location="${scala-library.jar}"   />
 </classpath>
</taskdef>
</target>

<!-- Copy Scala JARs -->
<target name="copyscala"
 description="Copies the Scala JARs to the WAR.">
<copy
   todir="war/WEB-INF/lib"
   flatten="true">
 <fileset dir="${scala.home}/lib">
   <include name="**/scala-library.jar" />
 </fileset>
</copy>
</target>

<!-- Copy App engine jars -->
<target name="copyjars"
 description="Copies the App Engine JARs to the WAR.">
<copy
   todir="war/WEB-INF/lib"
   flatten="true">
 <fileset dir="${sdk.dir}/lib/user">
   <include name="**/*.jar" />
 </fileset>
</copy>
</target>

<target name="compile" depends="copyscala, copyjars, init"
 description="Compiles Java source and copies other source files to the WAR.">
<mkdir dir="war/WEB-INF/classes" />
<copy todir="war/WEB-INF/classes">
 <fileset dir="src">
   <exclude name="**/*.scala" />
<exclude name="**/*.java" />
 </fileset>
</copy>
<scalac
   srcdir="src"
   destdir="war/WEB-INF/classes"
   classpathref="project.classpath"
    />

<!--
<copy todir="war/WEB-INF/classes">
  <fileset dir="src">
    <exclude name="**/*.scala" />
    <exclude name="**/*.java" />
  </fileset>
</copy> -->
<javac
   srcdir="src"
   destdir="war/WEB-INF/classes"
   classpathref="project.classpath"
   debug="on" />

</target>

<target name="datanucleusenhance" depends="compile"
 description="Performs JDO enhancement on compiled data classes.">
<enhance_war war="war" />
</target>

<target name="runserver" depends="datanucleusenhance"
 description="Starts the development server.">
<dev_appserver war="war" />
</target>

<target name="update" depends="datanucleusenhance"
 description="Uploads the application to App Engine.">
<appcfg action="update" war="war" />
</target>

<target name="update_indexes" depends="datanucleusenhance"
 description="Uploads just the datastore index configuration to App Engine.">
<appcfg action="update_indexes" war="war" />
</target>

<target name="rollback" depends="datanucleusenhance"
 description="Rolls back an interrupted application update.">
<appcfg action="rollback" war="war" />
</target>

<target name="request_logs"
 description="Downloads log data from App Engine for the application.">
<appcfg action="request_logs" war="war">
 <options>
   <arg value="--num_days=5"/>
 </options>
 <args>
   <arg value="logs.txt"/>
 </args>
</appcfg>
</target>

</project>

Example request and output:

Request: http://primes-chillipower-com.appspot.com/primes?n=50


{ "n" : "50", "primes" : [ { "0" : "2" }, { "1" : "3" }, { "2" : "5" }, { "3" : "7" }, { "4" : "11" }, { "5" : "13" }, { "6" : "17" }, { "7" : "19" }, { "8" : "23" }, { "9" : "29" }, { "10" : "31" }, { "11" : "37" }, { "12" : "41" }, { "13" : "43" }, { "14" : "47" }, { "15" : "53" }, { "16" : "59" }, { "17" : "61" }, { "18" : "67" }, { "19" : "71" }, { "20" : "73" }, { "21" : "79" }, { "22" : "83" }, { "23" : "89" }, { "24" : "97" }, { "25" : "101" }, { "26" : "103" }, { "27" : "107" }, { "28" : "109" }, { "29" : "113" }, { "30" : "127" }, { "31" : "131" }, { "32" : "137" }, { "33" : "139" }, { "34" : "149" }, { "35" : "151" }, { "36" : "157" }, { "37" : "163" }, { "38" : "167" }, { "39" : "173" }, { "40" : "179" }, { "41" : "181" }, { "42" : "191" }, { "43" : "193" }, { "44" : "197" }, { "45" : "199" }, { "46" : "211" }, { "47" : "223" }, { "48" : "227" }, { "49" : "229" } ] }

The iPhone client:

I'm currently developing using the iPhone 3.1 beta SDK. I'm not going to post all the source code the the simple client, but rather give some essential snippets relevant to the invocation of the simple JSON web service once deployed on the Google App Engine.

To Consume this JSON in an iPhone app, we can use the JSON Objective-C library, which is available on Google code here http://code.google.com/p/json-framework/wiki/FAQ


- (void)loadData {
[super viewDidLoad];

NSString *urlString = [NSString stringWithFormat:@"http://primes-chillipower-com.appspot.com/primes?n=1000"];
NSURL *url = [NSURL URLWithString:urlString];

NSDictionary *primesResponse = (NSDictionary *)[JsonClientHelper fetchJSONValueForURL:url];

if (primesResponse != nil) {

NSArray *primeEntries = [primesResponse objectForKey:@"primes"];

NSMutableArray *primesTemp = [[NSMutableArray alloc] init];
[primesTemp retain];

for (int i = 0; i < [primeEntries count]; i++) {

 NSDictionary *primeEntry = (NSDictionary*)[primeEntries objectAtIndex:i];

 NSString *key = [NSString stringWithFormat:@"%d", i];

 NSString *p = [primeEntry objectForKey:key];

 NSString *msg = [NSString stringWithFormat:@"Prime %d = %@", i, p];

 NSLog(msg);

 [primesTemp addObject:p];
}

primes = [primesTemp copy];

[primesTemp release];
}
}

The above code makes a request to the JSON web service on the App Engine and decodes the response, which once decoded using the JSON library is contained in Dictionaries (NSDictionary) and Arrays (NSArray).

Where JsonClientHelper contains a basic class helper method fetchJSONValueForURL to fetch and decode a JSON response from the supplied URL.


+ (id)fetchJSONValueForURL:(NSURL *)url
{
NSString *jsonString = [[NSString alloc] initWithContentsOfURL:url encoding:NSUTF8StringEncoding error:nil];

id jsonValue = [jsonString JSONValue];

[jsonString release];

return jsonValue;
}

I hope this Blog has helped give some insight into creating a Scala based Web Application for the Google App Engine, as well as the consumption of JSON in an iPhone Objective-C client.

(I appologise for the formatting of the code in this blog, finding it awkward to get the code into the blog without it being mangled in some way!).

Resources:

Handy JSON Lint tool: http://www.jsonlint.com/

Scala Servlet development: http://blogs.oracle.com/aseembajaj/2008/08/scala_servlet_development.html

Scala on the Google App Engine: http://froth-and-java.blogspot.com/2009/04/scala-on-google-appengine.html

The Google App Engine provides support for JPA and JDO based persistence - a comparison of the two can be found here: http://www.jpox.org/docs/persistence_technology.html Further details of the JDO specification can be found here: http://java.sun.com/jdo/index.jsp and the JDO specification here: http://java.sun.com/javaee/technologies/persistence.jsp

Support for the Google App Engine in Netbeans: http://kenai.com/projects/nbappengine/pages/Home

.

Scala, lazy evaluation and the Sieve of Eratosthenes

2009-07-03T10:52:00.000-07:00

This blog posting is primarily concerned with looking at lazy evaluation in functional programing, what it means and how it can be put to use.

Lazy evaluation can be described as an expression that has a value, but that is not evaluated until it's actually needed. Conversely strict refers to the opposite property where all expressions are evaluated up front when declared.

Some functional languages are described as Pure Lazy (no, this is not a derogatory term!), to reflect the fact that all evaluation is performed on demand. Haskell is on such language. Scala has both aspects of strict evaluation and lazy evaluation, as such it couldn't be referred to as 'pure' in either sense.

By means of the simplest example I can think of to illustrate Lazy evaluation, consider the following interaction with the Scala CLI:


scala> lazy val a = b + 1; lazy val b = 1;
a: Int = <lazy>
b: Int = <lazy>

scala> a
res9: Int = 2

scala> b
res10: Int = 1

I think this illustrates lazy evaluation, without defining the 'vals' as lazy the statement lazy val a = b + 1; lazy val b = 1; as val a = b + 1; val b = 1; couldn't be evaluated because b would be undefined when evaluating a!

The examples are based on the Sieve of Eratosthenes. Eratosthenes was an ancient Greek Mathmetician from 276 BC – c. 195 BC. The Sieve of Eratosthenes is a ancient algorithm for finding prime numbers from 2 .. n. Stated simply, this algorithm can be expressed as:

Create a list of all the numbers from 2 .. n.
Starting at p = 2
Strike out all multiples of p up until n from the list
Let p now be the first (lowest) number that has not been struck-off the list
Repeat the last two steps until p^2 > n
All remaining numbers not struck from the list are prime

In Haskell this can be expressed very elegantly and succinctly, as follows:

primes = sieve [2..]
sieve (p : xs) = p : sieve [x | x <− xs, x `mod` p > 0]

Note that [2..] creates an infinite steam of all the integers from 2 to infinity. You might think this is impossible and would create an out of memory error on the machine it ran on, but this is perfectly valid in a pure lazy language because none of the numbers are actually generated until they are requested!

In Scala there's not really a way to represent this quite so succinctly, but it is possibly to create a Lazy stream of numbers using:

var is = Stream from 2

which is essentially equivalent to Haskell's:

[2..]

Syntax.

Another important thing to note and be aware of is that Scala's List class is not a lazy list. Therefore, trying to make the equivalent of the Haskell algorithm in Scala using a List is not going to work.

Scala's Stream class however is Lazy, and so it can form the basic of the equivalent algorithm in Scala. To be specific, Scala's Stream class is strict about head and lazy about tail - which is ok since head contains a finite set of 0|1 elements.

The following Scala code shows two ways to implement the Sieve using Scala Lazy streams.

Example 1:


package primes;

/* Haskell...

primes = sieve [2..]
sieve (p : xs) = p : sieve [x | x <− xs, x `mod` p > 0]
*/
object Primes1 {

def primes : Stream[Int] = {

 var is = Stream from 2

 def sieve(numbers: Stream[Int]): Stream[Int] = {
   Stream.cons(
     numbers.head,
     sieve(for (x <- numbers.tail if x % numbers.head > 0) yield x))
 }

 sieve(is)
}

def main(args : Array[String]) = {

 primes take 100 foreach println
}
}

object Primes2 {
def primes = {
 def sieve(is: Stream[Int]): Stream[Int] = {
   val h = is.head
   Stream.cons(h, sieve(is filter (_ % h > 0)))
 }

 sieve(Stream.from(2))
}

def main(args: Array[String]) {
 println(primes take 100 toList)
}
}

Note the use of the 'take' method to extract the first 100 primes from the stream (so the program terminates!).

This second example shows how we can define a Lazy list like trait and then use it to implement the same, equivalent behavior:

Example 2:


package primes


object PrimesLazyTrait {

sealed trait Lazy[+A] {
 def head: A = this match {
   case Nil => error("head called on empty")
   case Cons(h, _) => h()
 }

 def tail: Lazy[A] = this match {
   case Nil => error("tail on empty")
   case Cons(_, t) => t()
 }

 def filter(p: A => Boolean): Lazy[A] = this match {
   case Nil => Nil
   case Cons(h, t) => if(p(h())) Cons(h, () => t() filter p) else t() filter p
 }

 def foreach(f: A => Unit) {
   this match {
     case Nil =>
     case Cons(h, t) => {
       f(h())
       t() foreach f
     }
   }
 }

 def toList: List[A] = this match {
   case Nil => scala.Nil
   case Cons(h, t) => h() :: t().toList
 }
}

final case class Cons[+A](h: () => A, t: () => Lazy[A]) extends Lazy[A]
case object Nil extends Lazy[Nothing]

def from(n: Int): Lazy[Int] = Cons(() => n, () => from(n + 1))

def primes = {
 def sieve(is: => Lazy[Int]): Lazy[Int] = {
   lazy val h = is.head
   Cons(() => h, () => sieve(is filter (_ % h > 0)))
 }

 sieve(from(2))
}

def main(args: Array[String]) {
 primes foreach println
}
}

In this version we see the Lazy trait provides the necessary methods head, tail, filter, foreach and toList.

These little examples help demonstrate some of the power, elegance and conciseness of lazy functional programming - as well as the beauty of this Ancient Greek algorithm for finding primes.

The basic algorithms complexity is:

O((nlogn)(loglogn))

And has a memory requirement of:

O(n).

Without any optimization.

--

Additional example - e (Euler's number):

Calculating e, a list of converging numbers in the sequence of e

e can be defined as lim n->inf ( 1 + 1/n)^n and therefore expressed directly as a function e of n as follows:

def e(n : Int) = { Math.pow((1.0 + (1.0/n)), n) }

So, how can we turn this into a stream? If we create a stream of numbers from 1 that are mapped using the e function, which can be defined as an anon lambda argument to map:

val es = Stream.from(1).map((n => { Math.pow((1.0 + (1.0/n)), n) }))

Example usage of the e stream:

es take 5 toList

.

Scala actors, an introduction with some simple examples

2009-06-15T14:10:00.000-07:00

The Scala programming language provides an Actor based framework for handling concurrency.

Scala's Actor model for concurrent programming was originally inspired by the Erlang programming language.

The actor model is a system of asynchronous message passing for concurrent programming, rather than the typical shared data and locks model. The shared data and locks model has some inherent difficulties that make programming in this way difficult and error prone. Major problems with shared data and locks include the well known dead-lock problem, as well as live locks and the general problem of locking shared mutable data without affecting program scalability and performance. Whilst Scala's actor messaging framework is build around asynchronous messaging, synchronous messaging is provided on top of this for convenience.

The Scala's actor model is based around an actor library and the Actor trait, along with some simple syntactical constructs for sending and receiving messages.

The actor library can be included into a project using:

import scala.actors.Actor._

The simplest way to start an actor is to declare an object/class that extends the class Actor and in the body of the actor declare an act() {} method to do the work. The actor x can then be started much like a thread, using x.start(). The act() method is much like the Java Thread's run() method.

A more succinct method is to declare a variable val x = actor {} and put the action definition directly between the braces (no act method required). What happens here is that actor is a helper method on Actor that creates and starts the actor - so there is no need to call x.start() explicitly when using this approach.

Actors can receive messages using the receive {} method. The receive message is passed a partial function to receive the messages. Typically the body of the receive method uses case pattern matching to determine what to do with the message.

A message can be sent to an actor using the ! operation. For example, an Int can be sent to actor x using: x ! 3

All actors get their own thread to call the receive function - this causes scalability problems when there are many actors with receive functions. The way to overcome this in Scala is to use the react() {} method instead. React is similar to receive except it doesn't not need a native thread to run and but must call itself when done to receive more messages - apart from this both are the same in that they both take a partial function.

Scala provides a convenience to avoid having to call act/react within a react method, this is the loop construct. This ends up looking as simple as:


loop {
 react {
   // message handling body goes here
 }
}

Note that react itself does not end end by returning up the call stack in the normal way, all calls to react result in an exception.

Within a receive or react method, an actor can reply to the sender of the received message by messaging the special identifier "sender", which represents the source of the message.

Example 1:

This simple example shows the most basic way to create an actor, by extending Actor, creating an instance and calling start on it.


package scalaactors1

import scala.actors._

class Actor1 extends Actor {

 def act {
   var x = 1
   while (true) {
     println("actor1! " + x)
     x = x + 1
     Thread.sleep(500)
   }
 }
}

object TestActors1 {

 /**
  * @param args the command line arguments
  */
 def main(args: Array[String]) = {

   println(">>main() - starting")

   val actor1 = new Actor1()

   actor1.start()

   println("<<main() - ending")
 }
}

Example 2:

This example creates a couple of actors using the val x = actor {} constructs. Each actor will automatically start and run concurrently, outputting a message and sleeping in an infinite loop. If you run this you can observe that the main thread starts and ends, the messages from the two actors are interleaved and run even after the main method has ended.


package scalaactors1

import scala.actors.Actor._

object TestActors2 {

 /**
  * @param args the command line arguments
  */
 def main(args: Array[String]) = {

   println(">>main() - starting")

  
   val actor1 = actor {

     var x = 1
     while (true) {
       println("actor1! " + x)
       x = x + 1
       Thread.sleep(500)
     }
   }

   val actor2 = actor {

     var x = 1
     while (true) {
       println("actor2! " + x)
       x = x + 1
       Thread.sleep(1000)
     }
   }

   println("<<main() - ending")
 }
}

Example 3:

This example is slightly more complex.

Actor kbReaderActor takes input from the keyboard (as Chars) and passes them one by one to the actor kbProcessorActor which prints out the received character and then uses a 3rd actor called sleep to sleep for the specified amount of time before then waking up the kbProcessorActor (to simulate some "work" that takes a finite amount of time to complete). Note that sleep signals the "sender" using a simple case class to represent the message type (with no arguments).


package scalaactors1

import scala.actors.Actor._

case class Wakeup // simple signalling case class


// simple actor example
object TestActors3 {

 def main(args: Array[String]) = {

   println(">>Starting")

   val sleep = actor {

     loop {
       react {
         case x : int => {
           println("sleeping for " + x)
          
           java.lang.Thread.sleep(x)

           println("waking up sender...")

           sender ! Wakeup
         }
       }
     }
   }

   val keyProcessorActor = actor {

     println(">>keyProcessorActor starting")

     var key : Option[Char] = null

     loop {

       react {
         case 'q' => exit() // 'q' signifies quit app!
         case x : Char => println("keyProcessorActor received '" + x + "'")

           sleep ! 1000  // ask the sleep actor to sleep, we will get a Wakeup call when done

           // now wait for our wakeup
           var r = false

           do {
             receive {
               case Wakeup => r = true
             }
           } while (!r)
       }
     }

     println("<<keyProcessorActor ending")
   }

   val kbReaderActor = actor {

     println(">>kbReaderActor starting")

     var key : Char = ' '

     do {

       println("kbReaderActor waiting for input")
       val i = System.in.read()
          
       key = i.asInstanceOf[Char]

       println("Received input " + key)

       // message actor1 with the key read from the console
       keyProcessorActor ! key
          
     } while (key != 'q')

     println("<<kbReaderActor ending")
   }
  
   println("<<Application Ending")
 }
}

Quantum Entanglement, entanglement with 2 electron singlet

2009-06-06T08:50:00.000-07:00

(and now for something completely different... Quantum mechanics and quantum entanglement)

Quantum Entanglement: Entanglement with 2 element quantum singlet.

Having recently emailed Leonard Susskind (Stanford CA) with a question regarding entanglement of a 3rd element with a 2 element system in the quantum singlet state (an entangled pair).

Leonard Susskind is the Felix Bloch professor of theoretical physics at Stanford University in the field of string theory and quantum field theory http://en.wikipedia.org/wiki/Leonard_Susskind

Susskind is widely regarded as one of the fathers of String Theory.

So naturally, I was somewhat surprised when I actually got a detailed response, and the answer is very interesting and informative too! Like most things in Quantum mechanics, the result is not something that could be guest or anticipated, it's not something that could be expected using classical intuition or experience.

Here is the reply:

--

Let’s begin with the initial state consisting of two electrons in a singlet state |ud-du> .( I should include a factor of one-over-square-root-of-2 to normalize the state but it’s too hard to type in Word so I will leave it to you.) Now add another electron in the up state— |u). Also, in order to keep track of photons, lets indicate that there are no photons present with the notation |0}. The initial state is,

[ |ud-du> |u) ] |0}

Now let’s rearrange the symbols so as to group electrons 2 and 3 together. This is just a notational device. The initial state is

[ |u> |du) - |d>|uu) ] |0}

Now use the trivial identity

|du) = ½ |du-ud) + ½ |du+ud)

to rewrite it

=½ [ |u> |du-ud) ] |0} + ½ [ |u> |du+ud) ] |0}

-[ |d>|uu) ] |0}

Now think of electron 2 and 3 as close one another, while electron 1 is off in the distance. In the first term 2-3 are in a singlet state. Nothing happens to it. In the other two terms the 2-3 system is orthogonal to the singlet. The 2-3 system will interact to emit a photon, and decay to the singlet in those cases, although the photons will be in different states in the two terms. Denoting the presence of the photon by |γ} and |γ’}, the final state will be

=½ [ |u> |du-ud) ] |0} + ½ [ |u> |du-ud) ] |γ}

-[ |d>|du-ud)] |γ’} (note, there is missing sqrt2)

So what do we have in the end? A superposition of states in which the 2-3 system is in the singlet and

electron 1 is up with no photon

electron 1 is up with a photon in state |γ}

electron 1 is down with a photon in state |γ’}.

If there is no photon or a photon in state |γ} then we know that electron 1 is up. If there is a photon in state |γ’} then e-1 must be down.

--

Having such a brilliantly and concise response was fantastic!

I've gone on to calculate the trivial equations to determine the probability in each state, as follows:

Given the probabilities of the 3 possibilities must add up to 1, taking the coefficients as lambda, the probability is given by lambda.lambda* which for real numbers is just lambda^2, so:

=½ [ |u> |du-ud) ] |0} + ½ [ |u> |du-ud) ] |γ}
-[ |d>|du-ud)] |γ’} (note, there is missing sqrt2)

a. electron 1 is up with no photon


|      1      | 2
| ------------|      = 1/4 = p1 = 0.25
|  2 sqrt(2)  |

b. electron 1 is up with a photon in state |γ}

p2 = 0.25

c. electron 1 is down with a photon in state |γ’}.


|      1      | 2
| ------------|      = 1/2 = p3 = 0.5
|  sqrt(2)    |

Google document:

http://docs.google.com/View?id=df4zskcg_25d8xtc9hp

The Stanford course "Modern Theoretical Physics - Fall 2006" by Leonard Susskind is available on iTunes-U (highly recommended!).

.

Basic Crib sheet for Mercurial hg

2009-06-06T08:02:00.000-07:00

Mercurial (hg) is a distributed version control system, with many similarities to Subversion (svn). Being distributed, means people can work offline and still keep their version history local, merging changes with each other by exporting and importing the repository changes periodically.

Generally speaking mercurial is very simple and easy to use, but here are a few commands for quick reference:

Basic crib for Mecurial HG

Clone a repository from a web URL:

hg clone http://www.selenic.com/repo/hello my-hello

Clone from another local repository:

hg clone existing-repos clone-repos

Make a new repository:

hg init

Add files:

hg add

Committing changes:

hg commit (hg ci)

Pull changes from one repository to another:

hg pull

Note: this doesn't update the working directory, to do this update the working directory:

hg update (hg up)

Exporting and import:

hg export -o filename tip

hg import -o filename

Other common commands:

hg log
hg status (hg st)
hg diff
hg revert
hg (-q) tip

HG Editor, the editor for HG comments is determined in the following order:

HGEDITOR environment variable

editor configuration option in [ui] section (in hgrc or passed with --config ui.editor command-line option).

VISUAL environment variable

EDITOR environment variable

vi, if none of the above is set

References:

Mercurial home:

http://www.selenic.com/mercurial/wiki/Mercurial

Mecurial tutorial:

http://www.selenic.com/mercurial/wiki/Tutorial

Quick reference and cheat sheets:

http://www.selenic.com/mercurial/wiki/QuickReferenceCardsAndCheatSheets

.

GlassFish v2.1 b60e crib sheet / installation notes

2009-06-04T11:51:00.000-07:00

Recently, having been working on a project that's migrated from JBoss 4.2.2 to GlassFish v2.1, I needed to hand over a crib sheet / quick guide to people supporting the live system, so as it might be useful in general. Only really covering the basics, but here it is anyway:

GlassFish AS Basic Crib / notes:

Prerequisites:

Java JDK 6.x latest

GlassFish AS version V2.1 b60e Final Release

Download jar here http://java.net/download/javaee5/v2.1_branch/promoted/Linux/glassfish-installer-v2.1-b60e-linux.jar

For RHEL derived CentOS, recommended installation would be something like /usr/local/projectname/glassfish

Installation:

java -Xmx256m -jar glassfish-installer-v2.1-b60e-linux.jar

cd glassfish

chmod -R +x lib/ant/bin

lib/ant/bin/ant -f setup.xml

Follow the installer instructions, accepting EULA and any defaults presented...

This will unpack and install the server, creating a default domain called "domain1"

A domain is an administrative "unit" - possibly related to, but not the same as an actual domain.

All domains live in: {glassfish_home}/domains/

We need just one for now, so all our stuff will be in:

{glassfish_home}/domains/domain1/

the installation process will create a default empty domain, domain1 with a default config file, the main configuration file is called domain.xml

e.g. {glassfish_home}/domains/domain1/config.xml

This file may contain some host /installation specific information, as well as stuff specific to our application.

Starting and stopping GlassFish AS:

{glassfish_home}/bin/asadmin start-domain domain1

{glassfish_home}/bin/asadmin stop-domain domain1

If all is good you will see confirmation on the command prompt when started, such as:

Domain domain1 started

Managing GlassFish AS:

By default, the web based management console is available on port :4848

The default admin account is username = admin, password = adminadmin

URL is: http://{hostname}:4848

Web-app deployment options:

There are a few options, the main two are:

Manually, using autodeploy directory

Simply copy .war file to: {glassfish_home}/domains/domain1/autodeploy

Deploying via the admin console:

On main menu on left, click on and expand the Web Applications menu entry

Click on the Deploy button, typically when deploying from a local file, use the "Packaged file to be uploaded to the server" browse button to locate the local war file. Browse for the war file, select it and click Ok

When completed, the web app will appear in the application list (if all is well)

The other settings can be left as defaults. Optionally we can chose to pre-compile JSPs here etc.

Server Logs:

The main server logs appear in the domain under logs, i.e.

{glassfish_home}/domains/domain1/logs/server.log

Logs can also be viewed (and configured, including log rotation) via the admin console, under the Application Server menu entry

Monitoring and management:

JConsole:

GlassFish is a JMX compliant app server (listening by default to port 8686)

Using jconsole, remote connect to: http://{hostname}:8686

Using the same default account: admin/adminadmin

Admin console:

Under the Application Server in the main menu - there are entries for server Monitoring.

This includes a built in chart and call flow logging - which is potentially very useful for diagnostics and monitoring…

HTTPS / SSL:

GlassFish AS will do HTTPS out of the box (using a default Sun Microsystems certificate).

HTTPS is served by default on port :8181

Hopefully in the near future I'll but together something to cover clustering and load balancing too.

Super quick guide for creating self-signed certificates

2009-06-03T11:20:00.000-07:00

Something I've done many times, but every time it's typically been so long in between that I've usually forgotten the specific details, so here is a super quick crib:

Create the private keys:


openssl genrsa -des3 -out server1.key 1024

Create a Certificate Signing Request (CSR):


openssl req -new -key server1.key -out server1.csr

Enter details, example below:


Country Name (2 letter code) [AU]:GB
State or Province Name (full name) [Some-State]:Kent
Locality Name (eg, city) []:Royal Tunbridge Wells
Organization Name (eg, company) [Internet Widgits Pty Ltd]:Acme Co
Organizational Unit Name (eg, section) []:Financial Systems
Common Name (eg, YOUR name) []:www.myexactdomainname.com
Email Address []:louis.botterill@netbuilder.com

Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:
An optional company name []:

Copy the passphrase protected key:


cp server1.key server1.key.org

Export the actual key without pass-phrase protection:


openssl rsa -in server1.key.org -out server1.key

Sign the CSR using the key to create a Certificate (.crt):


openssl x509 -req -days 365 -in server1.csr -signkey server1.key -out server1.crt

Easy huh! :)

.

How to add GZIP compression to a .NET WCF REST POX client

2009-05-21T13:36:00.000-07:00

Bandwidth is important! Even these days where everyone has broadband, now we want the same performance on the move, over wireless and 3G networks. Even when clients individually have enough bandwidth, put 100 or 1000 users together in a corporate environment and it's the IT department that may start have the headache instead. Then there's the server side, ultimately all this traffic comes together down a finite number of "pipes" to our servers, so really we still have to consider bandwidth usage at all levels.

We needed to add GZip compression support to a .NET 3.5sp1 client application using WCF for REST POX web services. The responses were sent compressed using GZIP, so the client needed to be able to decompress these, before passing up the stack to the Data Serialisers and application code layers.

The server in this instance is a Java EE web application returning XML, GZipped using a filter servlet. Implementing the server side filter is fairly trivial, however it's companion in .NET on the client side is not.

Since this gave me a little bit of a challenge recently (very little documentation around or good examples) I thought I'd put up a quick blog about it. It's not that there's a lot of code involved, it's understanding the WCF with sufficient clarity to be able to extend it, and debug it when it's not quite right.

One of the issues that crops up straight away is that most examples of extending the WCF for compression are based around the WS.* web service support, the original incarnation of WCF.

However, later versions of the WCF added support for REST/POX web services, and this is where I wanted to add the GZIP compression support. You can not mix WS.* and Rest (aka Web in WCF) web services in the WCF, without creating a runtime error as the WCF channel is instantiated.

So, enough of the preamble already, lets cut to the chase, what's needed?

Well, assuming you have a WebHttpBinding to start with...

WebHttpBinding webHttpBinding = new WebHttpBinding();

Then, in order to add the GZIP compression support, we need to pull apart the binding, into its constituent elements and then reassemble it, adding in the new binding element.

I enlisted the help of a content mapper to force the content to XML:


public class XMLMapper : WebContentTypeMapper
{
  public override WebContentFormat GetMessageFormatForContentType(string contentType)
  {
      return WebContentFormat.Xml; // always
  }
}

First, get the binding elements out of the WebHttpBinding, using the CreateBindingElements() function.


BindingElementCollection bec = webHttpBinding.CreateBindingElements();

WebMessageEncodingBindingElement wmbe = bec.Remove();
HttpTransportBindingElement htbe = bec.Remove();

wmbe.ContentTypeMapper = new XMLMapper();

GZipMessageEncodingBindingElement compBindingElement = new GZipMessageEncodingBindingElement(wmbe);

bec.Add(compBindingElement);
bec.Add(htbe);

CustomBinding cb = new CustomBinding(bec);

binding = cb;

HttpRequestMessageProperty httpRequestMessageProperty = new HttpRequestMessageProperty();

httpRequestMessageProperty.Headers.Add(HttpRequestHeader.AcceptEncoding, "gzip");
httpRequestMessageProperty.Headers.Add(HttpRequestHeader.ContentEncoding, "gzip");

Ok, so now binding is a rebuilt binding, based on WebHttpBinding where a GZIP encoder has been used to wrap the WebMessageEncodingBindingElement from the original binding's binding elements.

Note that the HttpTransportBindingElement must be the last binding element in the binding element collection (protocol stack) or a runtime error will be thrown when trying to instantiate the channel.

Now we can create the channel in the normal way:


// create a channel factory
ChannelFactory cf = new ChannelFactory(binding, hostPath);
              
// create webhttpbehavior for rest / pox get/invoke behaviour support
WebHttpBehavior webHttpBehavior = new WebHttpBehavior();
webHttpBehavior.DefaultOutgoingRequestFormat = WebMessageFormat.Xml;
webHttpBehavior.DefaultOutgoingResponseFormat = WebMessageFormat.Xml;
              
// add webhttpbehahvior to channelfactory endpoint
cf.Endpoint.Behaviors.Add(webHttpBehavior);

if (ENABLE_COMPRESSION)
{
   // add our custom behavior...
   cf.Endpoint.Behaviors.Add(new HttpEndpointBehavior());
}

// create our IService channel
channel = cf.CreateChannel();

So far so good, but what about this GZipMessageEncodingBindingElement?

Well, this can be found in the Microsoft WCF samples:

http://msdn.microsoft.com/en-us/library/ms751458.aspx

Detailed steps of integrating the MS WCF GZip example into your code:

http://www.vistax64.com/indigo/113763-wcf-client-j2ee-server-using-gzip.html

Note: I had to make some minor modifications to make my client work correctly, details can be provided if anyone leaves a comment expressing an interest.

Further Resources:

Various threads discussing similar problems:

http://social.msdn.microsoft.com/forums/en-US/wcf/thread/5c67b0da-9e50-4ee1-b7ac-a4733c580980/

http://social.msdn.microsoft.com/forums/en-US/wcf/thread/ddfe06b1-f07c-4da2-a1f1-d06126e4f96e/

http://aspnet.codeplex.com/Release/ProjectReleases.aspx?ReleaseId=24644

Neomax - provide software products to extend the WCF framework, including support for compression:

http://www.noemax.com/products/wcfx/features.html#gzip_compression

Microsoft Windows SDK for Windows 7 and .NET Framework 3.5 SP1: BETA

http://www.microsoft.com/downloads/thankyou.aspx?familyId=a91dc12a-fc94-4027-b67e-46bab7c5226c&displayLang=en

Quick notes on OpenSolaris 111a & GlassFish clustering

2009-05-16T15:03:00.000-07:00

Some quick notes on upgrading 111 to 111a and preparing for GlassFish v2.1 clustering and load balancing tests:

(Note: This is really a scratch-pad of notes and resources for my own benefit, rather than a structured Blog for others to follow, however some of the resources might be useful for someone, somewhere)

OpenSolaris upgrade problems:

Graphical login failed to start

Simply restarting gdm fixed this problem!

svcadm disable gdm
svcadm enable gdm

Installing GlassFish v2.1 - problems:

Running java -Xmx256m -jar glassfish_xxx.jar reported "Not Enough Space Available"

zfs list showed 8gig space available

check for large files in swap directories:

pfexec du -k /tmp|sort -n
pfexec du -k /etc/svc/volatile|sort -n

Increasing swap space, adding a second swap file:

zfs create -V2G rpool/swap2 ; swap -a /dev/zvol/dsk/rpool/swap2

GlassFish clustering:

Steps for setting up GlassFish clustering:

Setup default clustering profile;

lib/ant/bin/ant -f setup-cluster.xml

Start the DAS;

bin/asadmin start-domain --user admin

Create the node agent;

bin/asadmin create-node-agent

Start the node agent;

bin/asadmin start-node-agent

Create cluster and instances (for node agent) in GlassFish admin console.

Download Load Balancer plug-in according to platform

http://download.java.net/javaee5/external/

Guide to setting up GlassFish clustering:

http://blogs.sun.com/dadelhardt/entry/clustering_web_applications_with_glassfish1

Guide to setting up GlassFish with Load Balancer plugin:

https://glassfish.dev.java.net/javaee5/build/GlassFish_LB_Cluster.html

Further resources:

Clustering with Apache HTTPd:

http://blogs.sun.com/jluehe/entry/supporting_apache_loadbalancer_with_glassfish

Interesting thread about clustering problems:

http://www.nabble.com/Cluster-session-replication-not-working-td20691318.html

GlassFish V2 Clustering presentation:

http://blogs.sun.com/stripathi/resource/Preso/GlassFishv2Clustering.pdf

Sun Clustering with GlassFish v2:

http://developers.sun.com/appserver/reference/techart/glassfishcluster/

Setting up GlassFish SSL support:

http://java.sun.com/mailers/techtips/enterprise/2007/TechTips2_Nov07.html

GlassFish JMX / JConsole:

http://docs.sun.com/app/docs/doc/820-4335/ablwi?a=view

Sun Java web server (for load balancing):

https://cds.sun.com/is-bin/INTERSHOP.enfinity/WFS/CDS-CDS_SMI-Site/en_US/-/USD/ViewProductDetail-Start?ProductRef=SJWS6.1SP5-OTH-G-F@CDS-CDS_SMI

Cloud computing - cloudforce and beyond...

2009-04-08T11:58:00.000-07:00

Having recently attended Cloudforce UK - London (07th-04-2009) I though I'd blog a little with some resources and links.

I don't really want to re-invent the wheel and attempt to explain what cloud computing is exactly, hopefully some of the resources and links should cover that. For an overview, the Wikipedia entry is a good start http://en.wikipedia.org/wiki/Cloud_computing

A couple of terms that crop up in this domain are:

SaaS - Software as a Service
PaaS - Platform as a Service
IaaS - Infrastructure as a Service

Cloudforce itself is a worldwide tour of salesforce.com - evangelising and spreading the word about their cloud products and frameworks, customer success stories and break out sessions covering aspects from development to opportunities.

Details on the event http://www.chinwag.com/events/2009/04/cloudforce-london-2009

Salesforce cloud platform:

http://www.salesforce.com/platform/

Main features at a glance:

Product portfolio including CRM
Application store - the AppExchange
Multiple server, massively scalable environment
Multi-tenant database optimised for cloud usage
Disaster recovery
Database, web service, business logic and presentation layers
Fast app creation, instant deployment
Live update of applications, can copy to sandbox to develop new versions
No built in source code control / visioning
Eclipse force plug ins available for development integration
Presentation framework and components (Visual Force)
Strongly types OO language - Apex (an on-demand language)
Web service creation API and integration support
Integration with Mobile devices - iPhone, Blackberry and Windows mobile based devices
Integration with Twitter, Facebook, Google API's supported

Site gallery / reference sites:

http://developer.force.com/sitesgallery

Force.com Applications:

The platform has an App Exchange, a place where cloud based applications can be shared and exchanged http://www.salesforce.com/platform/appexchange/

The Salesforce platform offers a number of off the shelf, customisable applications products, such as Salesforce CRM http://www.salesforce.com/products/

App Development:

Salesforce developer home is here http://developer.force.com/

The platform framework offers some technologies such as Visual Force for GUI components and component creation and Apex for code/page creation - http://wiki.developerforce.com/index.php/Apex

Salesforce blerb about Apex "Force.com Apex Code is a strongly-typed programming language that executes on the Force.com platform. Apex is used to add business logic to applications, to write database triggers, and to program controllers in the user interface layer. It has a tight integration with the database and query language, good web services support, and includes features such as futures and governors for execution in a multi-tenant environment."

Cloudforce take-aways:

As well as the above, the overall event gave an impression of some key takeaway points.

The service cloud, service applications now integrating and utilising knowledge from social networking sources such as Twitter, Facebook etc as well as traditional sources such as Phone.
Easy of application creation
Speed of development
Enable business to meet customer demands in terms of cost or time to market that might otherwise be impossible

Some other providers:

Many over companies offer computing in the cloud, such as:

Microsoft - Windows Azure http://en.wikipedia.org/wiki/Azure_Services_Platform
Google - App engine http://code.google.com/appengine/ - recently with support for Java! read more here http://blogoscoped.com/archive/2009-04-08-n87.html
3tera - http://www.3tera.com/
SAP AG - SAP business as a service http://en.wikipedia.org/wiki/SAP_Business_ByDesign
Amazon - EC2 (Elastic Compute Cloud) - http://en.wikipedia.org/wiki/Amazon_Elastic_Compute_Cloud

Well, having just signed up for the Google App engine, expect more blogs in the future as I play around and experiment up in the clouds...

Further Resources:

Google AppEngine blogspot - http://googleappengine.blogspot.com/

Sun Cloud Computing Primer (Need a Sun account) - http://www.sun.com/offers/details/cloud_computing_primer.html

.

.NET and C# - an introduction

2009-04-02T14:18:00.000-07:00

Not so much an actual blog today, rather a link to a presentation I recently gave as an introduction to the .NET framework and C# - to a mixed audience of C, C++, Java and web developers.

It's mainly a high level overview of the .NET framework and then an introduction to the C# language, from a programmers perspective, for programmers familiar with other languages like Java and C++, to help get familiar with the basics.

Available in OpenOffice format ODP:
http://www.chillipower.com/Primers/IntroductionToDotNetandCsharpPresentation.odp

Or MS PowerPoint presentation format PPT:
http://www.chillipower.com/Primers/IntroductionToDotNetandCsharpPresentation.ppt

.

Some notes from Sun GlassFish portfolio tech talk, London

2009-03-30T04:27:00.000-07:00

Sun GlassFish and GlassFish portfolio - Sun talk London, Regis House 25-03-2009

Some notes that I managed to take whilst at the talk.

www.glassfish.org

Claimed to be the fastest Java EE AS

V2.0 -> reference implementation for Java EE 5
V3.x -> reference implementation for Java EE 6

Java EE standard 10 years old

Metro -> Interop between MS and Java WS.* standards

GlassFish is Open Source under CDDL and GPL 2 license

java.sun.com/javaee

Transparent development

Organic growth into middleware areas

First appearance 2005
V2 in 2007
Current v2.1 in Feb 2009

www.beta.glassfish.java.net:81/maps

6-700 downloads per month, 8 million in the last year / ~28k per day

GlassFish portfolio is a set of related technologies including the AS

Enterprise server (including support for SNMP)
Web space server (joined with Life Ray - social networking & portal technologies)
Web sack (a complete (L)(S)(?)AMP stack)
GlassFish ESB (SOA platform) for integration, connectors, adaptors

Java EE + Ruby on Rails, PHP and other frameworks

(note - what is SFA?)

Enterprise version adds:

Performance advisor, performance monitor, SNMP, self management and alert manager
Update center

Example / reference sites:

wotif.com - Glassfish & OpenMQ
www.travelmuse.com GlassFish, message queue and mysql (started by using community edition)
North american Nationwide Health Information Network - connect (NHIN- Connect)
OpenESB
ESB.SOA framework
Facebook - large mysql based system ~70million users

GlassFish has same levels and structure in terms of support and licensing as MySQL

Web Services: WS.*

Metro is core in GlassFish AS
Project Metro Web services stack provides high level WS stack with security, reliability etc -> .NET 3 interoperability (incorporates project tango - MS .NET interop)

REST

JAX-RS JSR-311 Java API for RESTFul Web Services
Annotation based server side API
HTTP Centric
Server Side only
Servlet or SE deployment

Jersey project provides implementation for JSR-311

Performance:

Similar / faster than Weblogic and Websphere

GlassFish V3:

OSGi: Apache Felix as default (origins in JSR-8)
1 sec startup
21Mb download
admin and update tool downloaded on demand

Add-ons, modules available from update center:

EJB 3.1 (preview
jRuby on Rails (new WAR packaging required)
Grails (also on GFv2)
Jersey and Metro (Web services)
jMaki (AJAX)

Tools:

NetBeans and Eclipse
Embedded GlassFish API

MicroKernel approach - modularity - base App server + on demand module loading, extensible and customizable

("Eclipse Con" -> announces GlassFiish bundle available)

Db connection pooling and connection (connector) pooling (JCA)

V3 Prelude

available from http://glassfish.org -> using modular (microkernel) v3 ideas with existing EE5 technologies -> this version is supported!
v3 prelude is Java EE Web layer only (i.e. servlets, JSPs etc but no Enterprise beans etc)

Compile / Deploy on Change support, dynamic debug loop (netbeans & eclipse), incremental compile of all Java EE artifacts. Auto-deply of all Java EE and static artifacts

Support for: JRuby, Ruby, Groovy, Grails, Python, jython, django, jmaki, JavaScript phobos

Ruby / jRuby and rails can be run on 2.1 today (by packaging up dependancies), on v3 available as OSGi jRuby container

PHP

Quercis (Caucho) opensource GPL php 5 implementation in Java
War Packaging
Java Bridge

V2 -> ee 5
V3 -> prelude with EE5 support
V3 -> early access is EE6 spec (but not supported)

GF Web Stack:

Apache HTTPd
Sun web server 7 (most scalable web server - now open sourced!) optimized for multi-core CMT (chip-based Multithreaded) systems - 2x scaling vs. Apache & Tomcat
lightpd
memcached
mod_jk, perl, ruby, PHP Ruby, Python, Squid, Tomcat

Current GF web stack is 1.4 (Python 2.5.2, Squid 2.6, Tomcat 5.5.27, memcached 1.2.5, mysql 5.0.67)

GF ESB:

JSR 208 JBI (Java business Integration)
Plug-able integration into backbone
"Metacontainer" - add new containers within the container, extend the AS
BPEL, WS-*, XSLT, FTP, LDAP, HTTP, DB service and binding
Based on OpnESB -> http://openesb.org / community open-esb.dev.java.net/Components.html
Maybe engines and connectors to things like Corba etc

What's next -> http://fuji.dev.java.net -> OpenESB V3 - next gen of SOA

(JBI and SCA stated as "complimentary, not conflicting")

GF: Web Space server (project web synergy)

Portlet spec JSR-268 and more (social networking, communities)
OpenSSO, identify mechanisms
Compelling web UIs

GF enterprise manager:

Performance advisor
Alerts
JDBC pool management - automatically tunes JDBC connection pool to optimize performance etc

Sailfin (Ericson contribution) telco, SIP, VOIP, Instant messaging

dtrace etends into Apache and PHP

Revamped PetStore 4 part tutorial - developer.sun.com

OpenSSO -> new concept "express builds" - milestone interim builds from Open Source community in between main supported releases

WhitePapers:

New White-papers on GlassFish portfolio launch:

Comparison with Tomcat
Performance optimization
GF with identify and MySQL

Training available in Camberly, BSG in London etc

www.javapassion.com

Books:

GlassFish
Database-driven application development
Others available (I didn't get time to note them down)

blogs.sun.com/theaquarium -> god launch pad for developer information

blogs.sun.com/stories -> gives examples and real usage info

Migration tools:

migrate2glassfish.dev.java.net/blogs/blogs.html

built in tools to detect and advise on code usage (for EE standards and vendor specific code dependancies)

--

Examples:

JavaFX mobile - answer to J2ME shortcomings

www.javafx.com -> many tutorials and demos

GlassFish profiles - Developer mode (no clustering etc) but fast start up and change deployment

Clustering:

DAS - Domain Administration server looks after a cluster, deploy to the domain administration server and it deploys to the cluster nodes.

Session replication between cluster nodes, zero down time when switching between nodes

install with -cluster switch

Cluster elements run "node-agents" to talk to the DAS

Extra sections in the admin console -> shows cluster nodes

GlassFish performance monitor - enterprise only (TBC)?

V3 Prelude - web tier only (web apps) but fully functional

Presentation has now been published here: http://uk.sun.com/sunnews/events/2009/mar/glassfish/pdf/UK-GlassFish-Portfolio-Launch-mar09.pdf

.

Apple iPhone Development - introduction and resources

2009-03-08T01:59:00.000-08:00

Apple iPhone Development:

So, it seems like everyone's doing it these days, but that's no excuse for me not to join in is it?...

Objective-C you say, isn't that a bit like Smalltalk?
xCode, what's that exactly?
Cocoa, Nibs? what the?!?

Well there's much to learn, but most of it seems reasonably straightforward. The Apple documentation is good, there are videos to watch and tutorials, most of these resources can be found via the Apple iPhone dev center on the Apple website. From what I've gathered so far, its quite far removed from previous mobile development I've done, such as J2ME and .NET CF.

Armed with a new Apple Mac I'm making some notes as I venture into the unknown.

Firstly, here are a bunch of useful resources:

iPhone Dev center:

http://developer.apple.com/iphone/

iPhone Developer Program:

http://developer.apple.com/iphone/program/

iPhone SDK:

http://developer.apple.com/iphone/download.action?path=/iphone/iphone_sdk_for_iphone_os_2.2.1__9m2621a__final/iphone_sdk_for_iphone_os_2.2.19m2621afinal.dmg

Objective-C:

http://developer.apple.com/iphone/library/documentation/Cocoa/Conceptual/ObjectiveC/ObjC.pdf

Objective-C is an object orientated version of the C language. Specifically for the iPhone, Objective-C 2.0 is used.

Objective-C has been influenced by Smalltalk, yes that's the OO language with all the [] in it.

iPhone development guide:

http://developer.apple.com/iphone/library/documentation/Xcode/Conceptual/iphone_development/iPhone_Development.pdf

First iPhone application:

http://developer.apple.com/iphone/library/documentation/iPhone/Conceptual/iPhone101/iPhone101.pdf

The current latest version of the iPhone OS and iPhone SDK is 2.2.1 (whilst at the time of writing this 3.0 is in beta preview).

The next version, 3.0 is to be officially launched soon, the SDK should be available before the official OS release, in theory. What's in it, what's different I don't know yet, but hopefully it will be largly compatible with the current version, whilst adding some interesting new features into the mix. Rumoured are things like the much awaited support for copy, cut and paste for example.

HTTP and XML, web services:
Interested in hooking up an iPhone application with some web services, some means to make an HTTP request and parse XML is needed, here are a couple of resources on this:

Simple solution using some utility code:

http://blog.atrexis.com/index.cfm/2008/7/28/iPhone--Parse-XML-to-custom-objects

A more advanced solution:

http://code.google.com/p/touchcode/wiki/TouchXML

Useful article on RESTful clients:

http://developer.apple.com/safari/articles/creatingrestfulclients.html

.

Prime factorization, a comparison between Haskell and Scala

2009-03-01T14:01:00.000-08:00

Haskell is a powerful and expressive pure lazy functional programming language with some interesting properties. http://www.haskell.org/ Haskell is of course named after the famous logician Haskell Curry http://en.wikipedia.org/wiki/Haskell_Curry who's name can be found in both the Haskell language and the process of "currying" functions.

By way of example and comparison the task of finding prime numbers is shown in Haskell and then its equivalent is shown in Scala.

We can see the conciseness of Haskell and the power of lazyness in expressions of the form:

[3..]

which creates an infinite list of integers from 3 upwards. This seeming impossibility can be handled in a lazy functional language because the infinite list is not evaluated up front, rather the list is only evaluated when needed so the list is finite at any given point in time.

So, in essence a lazy language is one where an expression is only evaluated when it's needed, unlike an imperative language where it's evaluated when declared.

Haskell: prime factorization example:


divides :: Integer -> Integer -> Bool
divides d n = rem n d == 0

ld :: Integer -> Integer
ld n = ldf 2 n

ldf :: Integer -> Integer -> Integer
ldf k n | divides k n = k
 | k^2 > n = n
 | otherwise = ldf (k+1) n

factors :: Integer -> [Integer]
factors n | n < 1 = error "Invalid argument: argument must be positive"
   | n == 1 = []
   | otherwise = p : factors (div n p) where p = ld n

If the above is saved in a file (called say primefactorization) and loaded into Hugs (Hugs is a Haskell interpreter shell based on Haskell 98 - http://www.haskell.org/hugs/) using the :l command as follows:

Main> :l c:\Dev\haskell\primefactorisation

Scala: prime factorization example:


package test

object PrimeFactorisationTest {

 def divides(d : Int, n : Int) = {
   (n % d) == 0
 }

 def ld(n : Int) : Int = {
   ldf(2, n)
 }

 def ldf(k : Int, n : Int) : Int = {
   if (divides(k, n)) k
   else if ((k*k) > n) n
   else ldf((k+1), n)
 }

 def factors(n : Int) : List[Int] = n match {
   case 1 => Nil;
   case _ => {
     val p = ld(n)
     p :: factors(n / p)
   }
 }

 def main(args : Array[String]) {
   if (args.length == 1)
   {
     val n = Integer.parseInt(args(0))
     println(factors(n))
   }
 }
}

Results:


Main> factors 12343434
[2,3,79,26041]

Both programs produce the same output and are functionally equivalent - it's also clear to see there's a reasonably similarity in both conciseness and expressiveness of the two languages, where Haskell possibly has the slight edge in some respects such as use of the "where" construct.

Update:

Following on from the excellent comments on the Haskell version from Don Stewart, turning the code into something that can be compiled and executed natively, rather than being interpreted using Hugs, roughly involves the following steps.

Install GHC http://www.haskell.org/ghc/

Rename the file to have a .hs file extension

Add a main that can process the arguments:

main = do
[n] <- getArgs
print (factors (read n))

add import to provide environment functions such as getArgs to the top of the file:

import System.Environment

Compile the file:

$ghc --make primefactorisation.hs

Run the executable:

$ time ./primefactorisation.exe 12343434
[2,3,79,26041]

real 0m0.140s
user 0m0.015s
sys 0m0.015s

and there we have it, a functional executable.

Don Stewart also makes a good point about being able to leave out all the type information.

In particular Haskell uses the powerful Hindley-Milner type inference algorithm to allow minimal (often zero) use of explicit type declarations.

http://c2.com/cgi/wiki?HindleyMilnerTypeInference

.

OpenSolaris, libc problem upgrading to b107

2009-02-23T14:08:00.001-08:00

Just a quick note, upgrading from b106 to 107 left me with a foobared installation. It turns out that libc.so.1 ends up being an incompatible version for the Kernel.

Luckily the fix / work around is fairly trivial :)

On my machine it was roughly as follows:

beadm mount opensolaris-19 /a
rm /a/lib/libc.so.1
rm /a/lib/amd64/libc.so.1
pkg -R /a fix SUNWcsl
beadm unmount opensolaris-19

Reboot into opensolaris-19

Problem fixed!

The bug if filed here: http://defect.opensolaris.org/bz/show_bug.cgi?id=3562

But that's not all, I was then faced with a second problem - which turned out to be unsupported "stuff" left over in the /etc/X11/xorg.conf, in particular an "RgbPath" related config line - so this needed to be removed.

gdm then restarted

svcadm disable gdm
svcadm enable gdm

X11 logs can also be checked for further errors:

/var/log/Xorg.0.log

and viola, a graphical login appears at last...

For further details on b107 issues, see http://blogs.sun.com/alanc/entry/xorg_1_5_3_in

Update: Since these problems with 107, 108 has come out and it seems to address these issues, upgrading from 107 to 108 was completely painless and problem free!

Additional note:

Recently trying to install OpenSolaris 109 under VirtualBox 2.1.4 resulted in a frozen white screen before the graphical login. After some digging around and debugging the problem appeared to be an incompatibility with VirtualBox guest additions (GA).

I believe the problem is captured here (as a major fault).

http://www.virtualbox.org/ticket/3408

The work around / temp solution is to:

Boot into single user command line login (edit boot grub)
Remove VirtualBox GA installation

>pkginfo | grep -i box
>pkgrm SUNWvboxguest

Delete xorg.conf (/etc/X11/)

>rm /etc/X11/xorg.conf

Restart GDM (svcadm restart gdm)

Check logs:

>tail -f /var/log/Xorg.0.log

--

To install VirtualBox Guest Additions, select "Devices -> Install Guest Additions..."

Navigate to the installer resources (usually visible on the guest OS desktop)

>pkgadd -d ./VBoxSolarisAdditions.pkg

.

Scala, implicit conversions

2009-02-22T03:59:00.001-08:00

Today, a small posting on the implicit conversion feature of Scala.

Implicit conversions provide a powerful way to extend classes with new functionality, as if already built into the library or even the language itself. There are similarities with C# extension methods, but Scala's implicit conversion mechanism is more concise, flexible and maintainable.

To define an implicit conversion in Scala, the syntax is:

impict def myImplicitConversionName(n: type) = new MyConversionType(n)

The scenario is that you want to add some new method(s) or functionality to instances of an existing class for which may be final/sealed or where sub-classing is inappropriate or not possible / effective. Implicit conversions provide a very natural way to add functionality and to users it will appear as natural features of the language/library, no special syntax is required.

A good example is provided on the Scala site: http://www.scala-lang.org/node/226

Which is a good example because it shows exactly the reasons why you might want to extend the language or library. In this case, the type int provides a good candidate for extension - to add the ! factorial function as if it was natural function of the type int.

I wanted to expand on this example by providing a couple of examples of the same thing with further explanation.

Example 1: explicitly defining a class that provides the extensions and using it in the implicit def


object extendBuiltins2 {

// the important bit, it says there's an implicit conversion from int to Factorizer(n : int)
implicit def int2fact(n: Int) = new Factorizer(n)

def main(args : Array[String]) = {

  // when the compiler sees this, it will first 'look for' ! method in int,
  // when it doesn't find it, it will look for an implicit conversion on int that provides !
  // which is of course exactly what we've just provided in the Factorizer class
  println("10! = " + (10!))
}

// define ! as a method on a class that can be constructed from int - this class will
// provide the implicit conversion that supplied an implementation of ! on int
class Factorizer(n: Int) {

  def ! = fact(n)
}

// the plain old factorial function
def fact(n: Int): BigInt =
  if (n == 0) 1 else fact(n-1) * n
}

The key line that defines the implicit conversion is this:

implicit def int2fact(n: Int) = new Factorizer(n)

what we are saying is that there is an implicit conversion from type int to type Factorizer(int) - so where we have an int, if ! is called the compiler will look for ! on int and fail to find it. It will then seek an implicit conversion that can convert from int in order to provide this method ! - where it finds our implicit conversion definition that uses the Factorizer class to provide the implementation of !

One reason this is more powerful than extension methods is that a whole class of functionality can be added to an existing class without explicitly extending all the methods individually. This has important benefits when it comes to maintenance, if the definition of the extending class changes, you get those changes automatically, you do not have to modify all the extension methods to match.

Example 2: equivalent to the above example but using an anonymous class for a more concise solution where the explicit definition of a class is of no other use.


/* More concise way of adding ! as a method on int's - no explicit class definition */
object extendBuiltins {

implicit def int2fact(n: Int) = new {

  def fact(n: Int): BigInt =
    if (n == 0) 1 else fact(n-1) * n

  def ! = fact(n);
}

def main(args : Array[String]) = {

  println("10! = " + (10!))
}
}

This achieves the same this as example 1 but does so more concisely with the anonymous class, constructed using the convenient = new {} syntax.

.

Some interesting reading material and resources...

2009-02-20T11:08:00.000-08:00

Some interesting reading material and resources...

Nothing much to see here, I just wanted to capture some links to interesting resources and reading material...

Papers and articles on Lambdas http://library.readscheme.org/page1.html

Functional Java library - http://functionaljava.org/

This library brings some functional programming constructs and library functions to Java, using either Java bgga prototype compiler based on JDK 7 or a more traditional anonymous class approach using standard Java 5. Both are JRE 5 compatible.

The Java Closures BGGA proposal http://javac.info/

.Net functional extension method library http://www.mono-project.com/Rocks

Apple iPhone Dev centre - http://developer.apple.com/iphone/index.action

http://www.iolanguage.com/

.

Currying, some examples and comparisons between C# and Scala

2009-02-18T11:54:00.000-08:00

I'm back again! It's been some 3 weeks since my last blog - I've been in New York for 5 days and busy either side sorting out commitments and work and home.

So, in this posting I'd like to continue on the theme of Lamda and functional programming with some simple examples of Currying in both C# .Net (3.0+) and Scala.

I've tried to keep the examples equivalent as closely as possible but Scala has a few different options when it comes to syntax, so the Scala version is a little longer as it delves into these extra possibilities.

What is currying?

In a nutshell currying is "the process of reducing a function that takes multiple arguments into a series of functions that each take one argument"

Named after Haskell Curry - http://en.wikipedia.org/wiki/Haskell_Curry

A good explanation and definition can be found here http://www.haskell.org/haskellwiki/Currying

C# .Net example using C# 3.0+ syntax for Lamdas:


using System;

static class Program
{
    static void Main()
    {
        Func<int, int, int> add = (x, y) => x + y;

        var curriedAdd = add.Curry();

        var curriedIncrement = add.Curry()(1);
        var curriedDecrement = add.Curry()(-1);

        Console.WriteLine("curriedAdd(10)(11) = " + curriedAdd(10)(11));
        Console.WriteLine("curriedIncrement(41) = " + curriedIncrement(41));
        Console.WriteLine("curriedDecrement(41) = " + curriedDecrement(41));
    }

    // a simple 2 arg curry template function
    static Func<TArg1, Func<TArg2, TResult>> Curry<TArg1, TArg2, TResult>(this Func<TArg1, TArg2, TResult> f)
    {
        return a1 => a2 => f(a1, a2);
    }
}

Results:


curriedAdd(10)(11) = 21
curriedIncrement(41) = 42
curriedDecrement(41) = 40

Notes:

The use of the var keyword for implicit typing
C# does not provide implicit functions/ partial application so a Curry template function for 2 arguments has been provided explicitly.
The Curry method is defined as an extension method so it appears as-if defined on add

Extension Method:

static Func<TArg1, Func<TArg2, TResult>> Curry<TArg1, TArg2, TResult>(this Func<TArg1, TArg2, TResult> f)

Take the above definition of a method, the this keyword in the (this Func...) parameter definition makes the function an extension method, so it can be used as if it were declared on the add function - to illustrate the point, without the extension syntax the Curry function could only be invoked in the regular way as follows:

var curriedAdd = Curry(add);

As an extension method, it's valid to use the original add.Curry() or Curry(add) syntax interchangeably.

Scala example:


package test;

object Currying {

  def main(args : Array[String]) {

    // regular syntax function add
    def add(x : int, y : int) = {

      x + y
    }

    // create curried add using 1 and -1 for increment and decrement by invoking partial function (a:int, _:int):int
    var curriedIncrement = add(1, _ : int)
    var curriedDecrement = add(-1, _ : int)

    // test funtions
    println("add(10, 11) = " + add(10, 11))
    println("curriedIncrement(41) = " + curriedIncrement(41));
    println("curriedDecrement(41) = " + curriedDecrement(41));

    // curried syntax function add
    def add2(x : int)(y : int) = {

      x + y
    }

    // create curried functions using partial functions
    var curriedIncrement2 = add2(1)(_ : int)
    var curriedDecrement2 = add2(-1)(_ : int)

    println("add2(10)(11) = " + add2(10)(11))
    println("curriedIncrement2(41) = " + curriedIncrement2(41));
    println("curriedDecrement2(41) = " + curriedDecrement2(41));

    // special {} syntax
    val res1 = add2(10) {
      11
    }

    println("res1 = " + res1)

    // more complex example
    val x = 1
    val y = 2

    val z = add2(10) {

      add2(20) {

        x + y
      }
    }

    println("z = " + z)
  }
}

Results:

add(10, 11) = 21
curriedIncrement(41) = 42
curriedDecrement(41) = 40
add2(10)(11) = 21
curriedIncrement2(41) = 42
curriedDecrement2(41) = 40
res1 = 21
z = 33

Notes:

Scala supports f(x:int)(y:int) syntax for explicit curried functions
Scala supports implicit currying using partial application, using the placeholder _ syntax
Scala supports curried parameter in {} syntax for functions that appear to extend the language by taking parameter from the body of the call.

Update:

I realised that the way I'd defined the curried add functions in the Scala example is perhaps not as verbose or explicit as it could have been, and therefore perhaps less easy to understand or compare to the C# example. So here is another code snipped that shows a more verbose currying:


object Currying {

  def main(args : Array[String]) {

    def add3(x : int) = {

        (y : int) => {
           x + y
        }
    }

    println("add3(1)(2) = " + add3(1)(2));

    val res = add3(5){

        3
    }

    println("res = " + res)
  }
}

Results:


add3(1)(2) = 3
res = 8

Resourses:

Good explanation of currying in C#.Net http://diditwith.net/2007/08/15/TheArtOfCurrying.aspx

.

Scala, a small example of Euclid's Algorithm

2009-01-23T11:15:00.000-08:00

A small posting today, a simple example of Euclid's algorithm in Scala. This algorithm can be expressed very concisely and simply using recursion. This example demonstrates the use of tuples and map in the Scala language.

If you don't know, Euclid's algorithm is a ancient, simple algorithm for finding the greatest common divisor (GCD) for two integers, without factoring either of the two integers in question.

The algorithm can be expressed recursively thus:

function gcd(x, y)
  if (y == 0) x
  else gcd(y, x mod y)

The particular construction worth noting in this example is the use of tuples.

Tuples are like records, structures or groupings of variables that can be constructed using a (x, y, z) syntax when x, y and z are variables. A tuple can be decomposed by accessing its elements using the t _1 syntax, where t is the tuple and _1 is a method that accesses the first element in tuple. Tuple elements are therefore indexed using a base of 1.

The example defines numbers as a list of tuples, each tuple contains a pair of integers - the two integers to be applied to the GCD algorithm (Euclid's Algorithm).

The list of tuples is then "mapped" using the map function that takes each tuple and calls the function x => ((Int, Int), Int, Boolean), where x will be each tuple in the list in turn.

The tuple x is decomposed into its two integer component parts using the x _1 and x _2 syntax.

The returned tuple ((Int, Int), Int, Boolean) contains the original tuple of integers to be tested, the gcd values and a boolean indicating if the numbers are co-prime. The numbers are co-prime if the GCD is equal to 1.

I've included a few different ways to represent the function used by map, to use and decompose the tuple. The first solution using the tuple argument directly and accesses its members using the _1, _2 syntax directly. The second solution uses a function literal and the _ placeholder syntax on the map argument. The function literal uses the val (a, b) = x; sytax to decompose the tuple into and and b, but this is a bit clumsey/verbose. The 3rd solution is essentially the same as a the 2nd but the function is defined in-line.

The last solution is probably the nicest, it uses the case(a, b) syntax to decompose the tuple into a and b using pattern matching.

Example Code:


package test

object EuclidsAlgorithm {

 def main(arg : Array[String]) = {

   // define a list of tuples (Int, Int), pairs to test for coprimeness
   val numbers = (3, 6) :: (3, 7) :: (4, 8) :: (5, 45) :: (5, 43) :: (1071, 1029) :: (1071, 1023) :: (1071, 1021) :: Nil

   // #1. Working with the tuple directly, using the tuple _1, _2 syntax
   val res1 = numbers map ( x => ((x _1, x _2),  gcd(x _1, x _2), gcd(x _1, x _2) == 1 )  )

   // #2. Defining a function literal
   val f = (x : (Int, Int)) => { val (a, b) = x;  ((a, b), gcd(a, b), gcd(a, b) == 1) }

   // and using it with the _ placeholder syntax for the parameter representing each element of the list
   val res2 = numbers map ( f(_) )

   // #3. Using the #2 above but in-line
   val res3 = numbers map ( x => { val (a, b) = x; ((a, b), gcd(a, b), gcd(a, b) == 1) }  )

   // #3. Using a case match on the tuple to expose its components a and b. This is probably the nicest and most concise way
   val res4 = numbers map ( { case (a, b) => ((a, b), gcd(a, b), gcd(a, b) == 1) } )

   res4.foreach(println)
 }

 // Euclids algorithm, finds the GCD
 def gcd(a : Int, b : Int) : Int = {

   if (b == 0)
     a
   else
     gcd(b, a % b)
 }
}

Results:


((3,6),3,false)
((3,7),1,true)
((4,8),4,false)
((5,45),5,false)
((5,43),1,true)
((1071,1029),21,false)
((1071,1023),3,false)
((1071,1021),1,true)

Scala, insertion sort and pattern matching

2009-01-19T11:27:00.000-08:00

This posting shows insertion sorting in Scala. Insertion sort is not a particularly efficient algorithm but this example code does demonstrate the use of pattern matching on lists using the match keyword and the case x :: y type expression.


val result = values match {

 case List() => List()

 case value :: valuesTail => insert(value, iSort(valuesTail))
}

The above code pattern matched the List values against 2 cases. The first being the empty list, in which case an empty list List() is returned. the second is the pattern value :: valuesTail, which puts the head of the list in value and the tail (the rest of the list) in valuesTail.

Example code:


package test

// Nb: This is not an efficient sorting algorithm but it domostrates
// some aspects of scala for representing algorithms, along with functions as first class objects and currying
object SortingTest {

   def main(args : Array[String]) {

       val list1 = 5 :: 2 :: 3 :: 1 :: 0 :: -1 :: 2 :: Nil

       val matcher = (x : Int, y : Int) => x < y

       val sortedList = insertionSort(list1, matcher)

       println("sortedList = " + sortedList.mkString(", "))

       val revMatcher = (x : Int, y : Int) => x >= y

       val revSortedList = insertionSort(list1, revMatcher)

       println("revSortedList = " + revSortedList.mkString(", "))
   }

   def insertionSort(values : List[Int], matcher : (Int, Int) => Boolean) : List[int] = {

       def iSort(values : List[Int]) : List[int] = {

           val result = values match {

               case List() => List()

               case value :: valuesTail => insert(value, iSort(valuesTail))
           }

           println("iSort called with " + values + ", result = " + result)

           result
       }

       def insert(value : Int, values : List[Int]) : List[Int] = {

           val result = values match {

               // if list is empty return new list with single element in it
               case List() => List(value)

               // otherwise insert into list in order, recursively
               case x :: xTail =>
                   if (matcher(value, x)) {
                       value :: values
                   }
                   else {
                       x :: insert(value, xTail)
                   }
           }
          
           println("insert called with " + value + ", values " + values + ", result = " + result)

           result
       }

       iSort(values)
   }
}

Results:

If you run the above example code the results are as follows:


iSort called with List(), result = List()
insert called with 2, values List(), result = List(2)
iSort called with List(2), result = List(2)
insert called with -1, values List(2), result = List(-1, 2)
iSort called with List(-1, 2), result = List(-1, 2)
insert called with 0, values List(2), result = List(0, 2)
insert called with 0, values List(-1, 2), result = List(-1, 0, 2)
iSort called with List(0, -1, 2), result = List(-1, 0, 2)
insert called with 1, values List(2), result = List(1, 2)
insert called with 1, values List(0, 2), result = List(0, 1, 2)
insert called with 1, values List(-1, 0, 2), result = List(-1, 0, 1, 2)
iSort called with List(1, 0, -1, 2), result = List(-1, 0, 1, 2)
insert called with 3, values List(), result = List(3)
insert called with 3, values List(2), result = List(2, 3)
insert called with 3, values List(1, 2), result = List(1, 2, 3)
insert called with 3, values List(0, 1, 2), result = List(0, 1, 2, 3)
insert called with 3, values List(-1, 0, 1, 2), result = List(-1, 0, 1, 2, 3)
iSort called with List(3, 1, 0, -1, 2), result = List(-1, 0, 1, 2, 3)
insert called with 2, values List(3), result = List(2, 3)
insert called with 2, values List(2, 3), result = List(2, 2, 3)
insert called with 2, values List(1, 2, 3), result = List(1, 2, 2, 3)
insert called with 2, values List(0, 1, 2, 3), result = List(0, 1, 2, 2, 3)
insert called with 2, values List(-1, 0, 1, 2, 3), result = List(-1, 0, 1, 2, 2, 3)
iSort called with List(2, 3, 1, 0, -1, 2), result = List(-1, 0, 1, 2, 2, 3)
insert called with 5, values List(), result = List(5)
insert called with 5, values List(3), result = List(3, 5)
insert called with 5, values List(2, 3), result = List(2, 3, 5)
insert called with 5, values List(2, 2, 3), result = List(2, 2, 3, 5)
insert called with 5, values List(1, 2, 2, 3), result = List(1, 2, 2, 3, 5)
insert called with 5, values List(0, 1, 2, 2, 3), result = List(0, 1, 2, 2, 3, 5)
insert called with 5, values List(-1, 0, 1, 2, 2, 3), result = List(-1, 0, 1, 2, 2, 3, 5)
iSort called with List(5, 2, 3, 1, 0, -1, 2), result = List(-1, 0, 1, 2, 2, 3, 5)
sortedList = -1, 0, 1, 2, 2, 3, 5
iSort called with List(), result = List()
insert called with 2, values List(), result = List(2)
iSort called with List(2), result = List(2)
insert called with -1, values List(), result = List(-1)
insert called with -1, values List(2), result = List(2, -1)
iSort called with List(-1, 2), result = List(2, -1)
insert called with 0, values List(-1), result = List(0, -1)
insert called with 0, values List(2, -1), result = List(2, 0, -1)
iSort called with List(0, -1, 2), result = List(2, 0, -1)
insert called with 1, values List(0, -1), result = List(1, 0, -1)
insert called with 1, values List(2, 0, -1), result = List(2, 1, 0, -1)
iSort called with List(1, 0, -1, 2), result = List(2, 1, 0, -1)
insert called with 3, values List(2, 1, 0, -1), result = List(3, 2, 1, 0, -1)
iSort called with List(3, 1, 0, -1, 2), result = List(3, 2, 1, 0, -1)
insert called with 2, values List(2, 1, 0, -1), result = List(2, 2, 1, 0, -1)
insert called with 2, values List(3, 2, 1, 0, -1), result = List(3, 2, 2, 1, 0, -1)
iSort called with List(2, 3, 1, 0, -1, 2), result = List(3, 2, 2, 1, 0, -1)
insert called with 5, values List(3, 2, 2, 1, 0, -1), result = List(5, 3, 2, 2, 1, 0, -1)
iSort called with List(5, 2, 3, 1, 0, -1, 2), result = List(5, 3, 2, 2, 1, 0, -1)
revSortedList = 5, 3, 2, 2, 1, 0, -1

Scala, example of Diffie-Hellman

2009-01-16T11:12:00.000-08:00

Another in the sequence of postings on Scala by simple example, this one shows an example of Diffie-Hellman to establish a shared secret over a public communication channel without that secret being revealed. Diffie-Hellman itself is prone to man-in-the-middle attacks, this example does not attempt to deal with that problem (normally addressed with authentication of some type).

The example sets up the canonical Alice and Bob, with shared public parameters g (generator) = 5 and p (prime modulus) = a large 128 random prime number.

Alice and Bob each create their secret key [sk] (a large 128 random number) and generate:

y = g^sk mod p

Alice and Bob then swap the result of this calculation.

From this, Alice and Bob both perform y^sk mod p to create a new value, the shared secret key which will be the same for both Bob and Alice even though they start with different random secret keys.

This is because:

(g^x)^y is the same as (g^y)^x

From a Scala perspective, the interesting things to note are:

Ease of use of large integers using the BigInt class as if it were a primitive type.
Constructors to create immutable instances containing g, p and name.
Companion object to provide helper function to wrap probable prime function.

Example code:


package test

import scala.util.Random;


object DiffieHellmanTest {

   val p = DiffieHellman.randomPrime(128)
   val g = 5

   def main(args : Array[String]) {

       // alice and bob initialise with the public parameters for DH, g and p
       val alice = new DiffieHellman(g, p, "Alice")
       val bob = new DiffieHellman(g, p, "Bob")

       // alice and bob generate their own secret keys and public keys
       alice.createSecretKey();
       bob.createSecretKey();

       val a1 = alice.getPublicKey();
       val b1 = bob.getPublicKey();

       println("a1 = " + a1)
       println("b1 = " + b1)

       // alice and bob exchange public keys
       bob.setPeerPublicKey(a1)
       alice.setPeerPublicKey(b1)

       // alice and bob compute their shared secret key
       val alicesk = alice.createSharedKey()
       val bobsk = bob.createSharedKey()

       println("Done, alice sk = " + alicesk + ", bob sk = " + bobsk)
   }
}

object DiffieHellman {

   def randomPrime(n : Int) : BigInt = { // return a random value with n digits

       val rnd = new Random();

       BigInt.probablePrime(n, rnd)
   }
}

class DiffieHellman(val g : Int, p : BigInt, name : String) {

   var secretKey : BigInt = 0
   var peerPublicKey : BigInt = 0


   def createSecretKey() {

       secretKey = random(128)

       println(name + " secretKey = " + secretKey)
   }

   def setPeerPublicKey(x : BigInt) {

       peerPublicKey = x
   }

   def getPublicKey() : BigInt = {

       doExpMod(g, secretKey, p)
   }

   def random(n : Int) : BigInt = { // return a random value with n digits

       val rnd = new Random();

       BigInt(n, rnd)
   }

   def createSharedKey() : BigInt = {

       doExpMod(peerPublicKey, secretKey, p)
   }

   private def doExpMod(x : BigInt) : BigInt = {

       println("doExpMod g = " + g + ", x = " + x + ", p = " + p)
       doExpMod(g, x, p)
   }

   private def doExpMod(g : BigInt, x : BigInt, m : BigInt) : BigInt = {

       g.modPow(x, m)
   }
}

Results:

(Numbers may vary between runs of course!)


Alice secretKey = 197977683133285508684427285955258922592
Bob secretKey = 54690551885126631242925351374829949140
a1 = 4317900055693480003952964194026338337
b1 = 58782279533775365647086521912939284670
Done, alice sk = 275531174494727646906973085281579443284, bob sk = 275531174494727646906973085281579443284

The secret keys for Alice and Bob normally wouldn't be revealed, but they've been output here for debug purposes.

Note that the final shared secret key is the same for both Alice and Bob, which could then be used to establish a secure communication channel using symmetric encryption.

.

Scala, Square roots, Netwon's method, closures and foldLeft

2009-01-11T05:12:00.000-08:00

A simple Scala example. This one computes square roots by successive approximation using Newton's method, defined in a class SquareRoot. A companion object is also used to define the precision constant, the target tolerance we'll use to decide when to stop approximating.

There are two version of the function. squareRootSimple is a simple implementation. The method squareRoot improves on this by refactoring the code to use a closure to capture x, the value for which we are finding the square root. Also, the overloaded average function takes a List and uses list.foldLeft(0.0).(_ + _) which means apply x+y to list element pairs, starting with 0.0, running through all the elements in the list, thus summing all the elements.

Example of the use of foldLeft:

list.foldLeft(0.0).(_ + _)

Which could also be written as:

(start /: list)(_ + _)

/: is the foldLeft operator, equivalent to the .foldLeft method on a list
\: is the foldRight operator, equivalent to the .foldRight method on a list

The fold operator is like a pattern that crops up fairly often in imperative programming, it looks something like this:

B b = startItem

for(final T a : listOfT) {
 b = method(b, a);
}

return b;

Example code:


package test

object SquareRootsTest {

    def main(args : Array[String]) {

        val squareRoot = new SquareRoot()

        assert(((squareRoot.squareRoot(9) - 3) abs) < SquareRoot.precision)
        assert(((squareRoot.squareRoot(64) - 8) abs) < SquareRoot.precision)

        val res1 = squareRoot.squareRoot(842)
        println("square root of 842 = " + res1 + ", " + res1 + " squared = " + res1 * res1)
    }
}


object SquareRoot {

    val precision = 0.000001
}

class SquareRoot {

    def squareRootSimple(x : Double) : Double = (

        squareRootIter(1.0, x)
    )

    def squareRootIter(guess : Double, x : Double) : Double = {

        if (goodEnough(guess, x)) {
          guess
        }
        else {
          squareRootIter(improveGuess(guess, x), x)
        }
    }

    private def improveGuess(guess : Double, x : Double) : Double = {

        val newguess = average(guess, x / guess)
        println("guess for x " + x + " improved to = " + newguess)
        newguess
    }

    private def goodEnough(guess : Double, x : Double) = {

        (square(guess) - x).abs  < SquareRoot.precision
    }

    def squareRoot(x : Double) : Double = {

        def squareRootIter(guess : Double, x : Double) : Double = {

            if (goodEnough(guess)) {
              guess
            }
            else {
              squareRootIter(improveGuess(guess), x)
            }
        }

        def improveGuess(guess : Double) : Double = {

            val newguess = average(List(guess, x / guess))
            println("guess for x " + x + " improved to = " + newguess)
            newguess
        }

        def goodEnough(guess : Double) = {

            (square(guess) - x).abs  < SquareRoot.precision
        }

        squareRootIter(1.0, x)
    }

    private def average(x : Double, y : Double) : Double = {

        (x + y) / 2
    }

    private def average(list : List[Double]) : Double = {

        //println("value = " + list.mkString)
        //(0.0 /: list)(_ + _) / list.size
        list.foldLeft(0.0)(_ + _) / list.size
    }

    private def square(value : Double) : Double = {

        value * value
    }
}

Results:


guess for x 9.0 improved to = 5.0
guess for x 9.0 improved to = 3.4
guess for x 9.0 improved to = 3.023529411764706
guess for x 9.0 improved to = 3.00009155413138
guess for x 9.0 improved to = 3.000000001396984
guess for x 64.0 improved to = 32.5
guess for x 64.0 improved to = 17.234615384615385
guess for x 64.0 improved to = 10.474036101145005
guess for x 64.0 improved to = 8.292191785986859
guess for x 64.0 improved to = 8.005147977880979
guess for x 64.0 improved to = 8.000001655289593
guess for x 64.0 improved to = 8.00000000000017
guess for x 842.0 improved to = 421.5
guess for x 842.0 improved to = 211.7488137603796
guess for x 842.0 improved to = 107.86261164282149
guess for x 842.0 improved to = 57.83441917633403
guess for x 842.0 improved to = 36.19661181949908
guess for x 842.0 improved to = 29.729228773452704
guess for x 842.0 improved to = 29.025762097895374
guess for x 842.0 improved to = 29.017237509256656
guess for x 842.0 improved to = 29.017236257093842
square root of 842 = 29.017236257093842, 29.017236257093842 squared = 842.0000000000015

Source Scheme Code:


(define (sqrt-iter guess x)
(if (good-enough? guess x)
guess
(sqrt-iter (improve guess x)
x)))
A guess is improved by averaging it with the quotient of the radicand and the old guess:
(define (improve guess x)
(average guess (/ x guess)))
where
(define (average x y)
(/ (+ x y) 2))
We also have to say what we mean by ``good enough.'' The following will do for illustration, but it is not
really a very good test. (See exercise 1.7.) The idea is to improve the answer until it is close enough so that
its square differs from the radicand by less than a predetermined tolerance (here 0.001):22
(define (good-enough? guess x)
(< (abs (- (square guess) x)) 0.001))

References:

Blog about fold Left: http://blog.tmorris.net/scalalistfoldleft-for-java-programmers/

.

Scala, for comprehensions

2009-01-10T10:27:00.000-08:00

This post contains some examples of Scala's for comprehension.

A for loop in Scala can be written in the imperative style and will behave rather much the same as a Java for loop, especially the enhanced Java for loop from Java 5.0.

However, this type of simple imperative looping doesn't touch on the power and flexibility of Scala's for comprehension, when used as used in a functional programming context.

In particular, some of the key differences are:

A for comprehension can be a value itself. The value is a product of the yield statement used within the expression body. Since a for comprehension iterates over a list or sequence, the resulting value of a for comprehension is typically a list too.
A for comprehension uses a generator, written in the form item <- listOfItems to define the items to be iterated over
Multiple generators may be used, somewhat analogous to a nested for loop within a for loop in imperative programming
In addition to generators, a guard can be used to filter or restrict the values
The for syntax can use parenthesis with semicolon delimited generators and guards or braces - both are equivalent
The iterations my differ when compared to a seeming equivalent imperative construct, because the implementation is fundamentally different. In particular the list of items to be used is determined using the generator(s) and guard(s) before the body is run, so for example flags used in guards that are modified in the body may not produce the expected result (from an imperative perspective).
Generators can be "projected" using the .project function.

Examples of Scala for comprehension:


package test

object ForComprehensionTest {
    def main(args : Array[String]) {

        // create a couple of simple lists...
        val list1 = "Apples" :: "Oranges" :: "Bananas" :: "Avocados" :: Nil
        val list2 = 11 :: 22 :: 33 :: 44 :: 55 :: 66 :: Nil

        // simple comprehension with one generator and guard
        val res1 = for(item1 <- list1 if item1.startsWith("A"))
            yield item1

        assert(res1 == List("Apples", "Avocados"))
        res1.foreach(println);

        // more complex comprehension, two generators within {brace} syntax with two guards
        val res2 = for {
                item1 <- list1 if item1.startsWith("A")
                item2 <- list2 if item2 % 3 == 0
            }
            yield item2 + " " + item1

        assert(res2 == List("33 Apples", "66 Apples", "33 Avocados", "66 Avocados"))
        res2.foreach(println);

        // as above but using () parens and ; semi-colon syntax instead
        val res3 = for ( item1 <- list1 if item1.startsWith("A");
                        item2 <- list2 if item2 % 3 == 0 )
            yield item2 + " " + item1

        assert(res3 == List("33 Apples", "66 Apples", "33 Avocados", "66 Avocados"))
        res3.foreach(println);

        println("simple guard item == 22")
        for (item <- list2 if item == 22) println(item)

        println("simple comprehension mkString = " + (for (item <- list2 if item % 2 == 0) yield item).mkString)

        // from an imperative / Java perspective you might expect this to stop with item = 22
        // but it continues on because !found is evaluated as always true (irrefutable pattern) when determining what to iterate over
        var found = false
        for (item <- list2 if !found) {

            if (item == 22)
                found = true

            println("without projection, item == 22? item = " + item)
        }

        // with the .projection call, it works more like you might expect from an imperative perspective
        found = false
        for (item <- list2.projection if !found) {

            if (item == 22)
                found = true

            println("with .projection, item==22? item = " + item)
        }

        found = false
        for (item <- list2.projection.takeWhile(item => !found)) {

            if (item == 22)
                found = true

            println(".projection.takeWhile(item => ! found) = " + item)
        }
    }
}

Results:


Apples
Avocados
33 Apples
66 Apples
33 Avocados
66 Avocados
33 Apples
66 Apples
33 Avocados
66 Avocados
simple guard item == 22
22
simple comprehension mkString = 224466
without projection, item == 22? item = 11
without projection, item == 22? item = 22
without projection, item == 22? item = 33
without projection, item == 22? item = 44
without projection, item == 22? item = 55
without projection, item == 22? item = 66
with .projection, item==22? item = 11
with .projection, item==22? item = 22
.projection.takeWhile(item => ! found) = 11
.projection.takeWhile(item => ! found) = 22

Other useful things to know:

In a for comprehension if you want to iterate over a collection you can create a tuple of (element, index) pairs using syntax of the form:


for ((elem, index) <- iter.zipWithIndex) {
    // now we can access element of each iteration along with its associated numerical index!
}

Resources:

A good explanation of for comprehensions can be found here http://creativekarma.com/ee.php/weblog/comments/the_scala_for_comprehension_from_a_java_perspective/

.

OpenSolaris - ZFS and Zones

2009-01-08T11:20:00.000-08:00

Today, just a quick note on setting up an OpenSolaris 10 system local zone.

Firstly, you need to ensure there's a suitable ZFS file system created for the zone, then you define and install the zone itself. The basic steps are quite simple, here's a basic example as a starter:

Note: Global zone means the default zone, the basic machine assuming no zones have been created yet. Local zone is a created zone within the global zone, as described here.

As root on the global zone:

Take a look at available pools:

global# rpool list

On my system, with one disk fully used, I have rpool already, but no other pools.

Creating a new ZFS filesystem under rpool as follows:

global# zfs create /rpool/zones

Now create a new zone configuration, here is a really basic minimal zone config with networking, taken from the zone FAQ.

global# zonecfg -z my-zone
global# zonecfg:my-zone> create
global# zonecfg:my-zone> set zonepath=/zones/zone_roots/my-zone
global# zonecfg:my-zone> add net
global# zonecfg:my-zone:net> set address=10.1.1.1
global# zonecfg:my-zone:net> set physical=hm0
global# zonecfg:my-zone:net> end
global# zonecfg:my-zone> commit
global# zonecfg:my-zone> exit

Install the zone (if this fails, check permissions, shown as *1 below)

global# zoneadm -z my-zone install

Boot the installation!

global# zoneadm -z my-zone boot

Login to the new zone:

global# zlogin -C my-zone

*1 change permissions on the dataset if necessary

global# chown root /rpool/zones/my-zone
global# chmod 700 /rpool/zones/my-zone

A more complete zone configuration, for reference:

create -b
set zonepath=/zones/dev1
set autoboot=true
add inherit-pkg-dir
set dir=/lib
end
add inherit-pkg-dir
set dir=/platform
end
add inherit-pkg-dir
set dir=/sbin
end
add inherit-pkg-dir
set dir=/usr
end
add inherit-pkg-dir
set dir=/opt/apache22
end
add inherit-pkg-dir
set dir=/opt/csw
end
add inherit-pkg-dir
set dir=/opt/mysql5
end
add inherit-pkg-dir
set dir=/opt/pgsql829
end
add inherit-pkg-dir
set dir=/opt/php5
end
add fs
set dir=/opt/SUNWspro
set special=/opt/SUNWspro
set type=lofs
add options ro
end
add net
set address=10.80.117.152
set physical=bge0
end

Resources:

ZFS: http://www.linuxdynasty.org/zfs-howto-howto-create-a-zfs-file-system.html

Zones: http://opensolaris.org/os/community/zones/faq/#sa_create

.

Scala, traits and mix-ins

2009-01-05T13:43:00.000-08:00

Continuing on in the series of Scala blogs, showing Scala features by example, this post attempts to show the operation of Traits. Traits are rather like classes but designed to allow rich feature composition by allowing "mix-in" of required traits dynamically, at run time. Traits achieve what can look like multiple inheritance, without the pitfalls and limitations of multiple inheritance. Conversely then, you might think traits are equivalent to Java's interfaces. Whist similar, the key difference is that traits can (and usually do) contain implementations (method bodies). When a trait method implementation makes a call to super, it is dynamically resolved as the class is linearised.

When traits are combined with classes to create extended behaviour, this process is known as mixin and the traits are said to be mixed-in or mixins.

The example below shows the behaviour of the plain traits class using its putMsg method, and then its behaviour when mixed-in with TraitAddA, TraitAddA and TraitAddB and then with TraitAddA and TraitAddB reversed (ordering of the with parts is significant).

As well as mixin with the new keyword, class TraitABClass shows that traits can be used as mixins with normal classes too, at the point they are declared using the with keyword.

Example Scala file showing the operation of Traits:


package test

object TraitsTest {

  def main(args : Array[String]) {

    val traits : Traits = new Traits()

    traits.putMsg("Hello!")

    println("traits, value = " + traits.toString)

    val traitsA : Traits = new Traits() with TraitAddA

    traitsA.putMsg("Hello!")
    
    println("traitsA, value = " + traitsA.toString)

    val traitsAB : Traits = new Traits() with TraitAddA with TraitAddB

    traitsAB.putMsg("Hello!")

    println("traitsAB, value = " + traitsAB.toString)

    val traitsBA : Traits = new Traits() with TraitAddB with TraitAddA

    traitsBA.putMsg("Hello!")

    println("traitsBA, value = " + traitsBA.toString)

    val traitsABClass : TraitsABClass = new TraitsABClass

    traitsABClass.putMsg("Hello2")

    println("traitsABClass, value = " + traitsABClass.toString)
  }
}

class TraitsABClass extends Traits with TraitAddA with TraitAddB

class Traits {

  var msg : String = ""

  def putMsg(m : String) {

    msg = m
  }

  override def toString = {

    msg
  }
}

trait TraitAddA extends Traits {

  abstract override def putMsg(m : String) = {

    super.putMsg("A" + m)
  }
}

trait TraitAddB extends Traits {

  abstract override def putMsg(m : String) = {

    super.putMsg("B" + m)
  }
}

Program execution results:


traits, value = Hello!
traitsA, value = AHello!
traitsAB, value = ABHello!
traitsBA, value = BAHello!
traitsABClass, value = ABHello2

Modular exponentiation

2009-01-04T10:56:00.000-08:00

For any inquisitive minds, the fast Modular exponentiation used in the last post (Scala example) might at first glance seem odd, perhaps.

What we want to achieve is x = b^n mod m

where b^n is performed first, the result in then operated on by mod m. Mathematically this is what we are doing, this is what we want to achieve.

However, the expmod function computes the value recursively, applying mod at each step of recursion (on each recursion result).

This is in fact a better way to approach the problem, as the intermediate values are kept small (between 0 and m-1) rather than growing exponentially (with potential overflow/performance problems), only to be operated on by modulus to shrink the value back down again!

Here is a simple Scala snippet that compares expMod with fastExp mod m, with some debugging added to show the intermediate values at each stage of recursion.


package test

import java.util.Random;

object ExpTest {

 def main(args : Array[String]) {

   println("fastExpt(4, 7) % 9) = " + fastExpt(4, 7) % 9)
   println("expMod(4, 7, 9) = " + expMod(4, 7, 9))

   println("fastExpt(5, 12) % 6) = " + fastExpt(5, 12) % 6)
   println("expMod(5, 12, 6) = " + expMod(5, 12, 6))
}

 def expMod(base : Int, exp : Int, m : Int) : Int = {

   val res = if (exp == 0)
     1
   else if (even(exp))
     remainder(square(expMod(base, (exp / 2), m)), m)
   else
     remainder(base * expMod(base, exp - 1, m), m)

   println("expMod: b = " + base + ", exp = " + exp + ", m = " + m + ", res = " + res)
   res
 }

 def fastExpt(b : Int,  exp : Int) : Int = {

   val res = if (exp == 0)
     1
   else if (even(exp))
     square(fastExpt(b, exp / 2))
   else
     b * fastExpt(b, exp - 1)

   println("fastExpt: b = " + b + ", exp = " + exp + ", res = " + res)
   res
 }

 def square(n : Int) : Int = {

   n * n
 }

 def even(n : Int) : Boolean = {

   remainder(n, 2) == 0
 }

 def remainder(n : Int, d : Int) : Int = {

     n % d
 }
}

Results are as follows:


fastExpt: b = 4, exp = 0, res = 1
fastExpt: b = 4, exp = 1, res = 4
fastExpt: b = 4, exp = 2, res = 16
fastExpt: b = 4, exp = 3, res = 64
fastExpt: b = 4, exp = 6, res = 4096
fastExpt: b = 4, exp = 7, res = 16384
fastExpt(4, 7) % 9) = 4
expMod: b = 4, exp = 0, m = 9, res = 1
expMod: b = 4, exp = 1, m = 9, res = 4
expMod: b = 4, exp = 2, m = 9, res = 7
expMod: b = 4, exp = 3, m = 9, res = 1
expMod: b = 4, exp = 6, m = 9, res = 1
expMod: b = 4, exp = 7, m = 9, res = 4
expMod(4, 7, 9) = 4
fastExpt: b = 5, exp = 0, res = 1
fastExpt: b = 5, exp = 1, res = 5
fastExpt: b = 5, exp = 2, res = 25
fastExpt: b = 5, exp = 3, res = 125
fastExpt: b = 5, exp = 6, res = 15625
fastExpt: b = 5, exp = 12, res = 244140625
fastExpt(5, 12) % 6) = 1
expMod: b = 5, exp = 0, m = 6, res = 1
expMod: b = 5, exp = 1, m = 6, res = 5
expMod: b = 5, exp = 2, m = 6, res = 1
expMod: b = 5, exp = 3, m = 6, res = 5
expMod: b = 5, exp = 6, m = 6, res = 1
expMod: b = 5, exp = 12, m = 6, res = 1
expMod(5, 12, 6) = 1

Most importantly, the functions yield the same results, but expMod achieves it in a more efficient and scalable way.

Just a small posting, as whilst appearing simple, modular exponentiation underpins much of modern cryptographic methods, and efficient algorithms are important.

.

Scala, Fermat's little theorem

2009-01-03T04:11:00.000-08:00

Today, another Scala posting by way of example, using the interesting Fermat's Little Theorem test for primality. This is a probabilistic test that provides no false negatives (if a number is found not prime, that is certain) but can provide false positives (a number indicated as prime is not certain to be so). The test can be run a number of times with a different base value (a) to gain increasing confidence that the number is prime.

Fermat's Little Theorem: If n is a prime number and a is any positive integer less than n, then a raised to the nth power is congruent to a modulo n.

There are a small set of special numbers that "fool" this test, these are the Carmichael numbers. Further details can be found here http://en.wikipedia.org/wiki/Carmichael_number

Carmichael numbers are extremely rare. There are only 255 Carmichael numbers below 100,000,000. The smallest few are 561, 1105, 1729, 2465, 2821, and 6601.

Because the Carmichael numbers are a special subset of the Fermat prime numbers that satisfy the same condition, they can be known as absolute Fermat pseudoprimes. The Fermat "primes" can be known as Fermat pseudoprimes. If we remove the absolute Fermat pseudoprimes from the Fermat Pseudoprimes then we end up with the absolute primes.

There are revised versions of this test that remove the false positive Carmichael numbers, such as the famous Miller-Rabin test (Miller 1976, Rabin 1980). This starts from an alternate form of Fermat's Little Theorem, which states that if n is a prime number and a is any positive integer less than n, then a raised to the (n - 1)st power is congruent to 1 modulo n.

For this Scala example, I'm only using the original Fermat's Little Theorem. Here is the Scala file, the main contains a number of test cases.


import java.util.Random;

object FermatsLittleTheorem {

 def main(args : Array[String]) {

   val attempts = 10;  // the number of attempts, higher numbers give greater confidence in positive results.

   var n = 17;
   println("result1: n = " + n + ", res = " + fastPrime(n, attempts))  // should be true
   n = 13
   println("result2: n = " + n + ", res = " + fastPrime(n, attempts))  // should be true
   n = 11
   println("result3: n = " + n + ", res = " + fastPrime(n, attempts))  // should be true
   n = 9
   println("result4: n = " + n + ", res = " + fastPrime(n, attempts))  // should be false
 }

 def fastPrime(n : Int, times : Int) : Boolean = {

   if (times == 0)
     true
   else
     if (fermatTest(n)) fastPrime(n, times - 1) else false
 }

 def expmod(base : Int, exp : Int, m : Int) : Int = {

   if (exp == 0)
     1
   else if (even(exp))
     remainder(square(expmod(base, (exp / 2), m)), m)
   else
     remainder(base * expmod(base, exp - 1, m), m)
 }

 def square(n : Int) : Int = {

   n * n
 }

 def even(n : Int) : Boolean = {

   remainder(n, 2) == 0
 }

 def remainder(n : Int, d : Int) : Int = {

     n % d
 }

 def fermatTest(n : Int) : Boolean = {

   def tryIt(a : Int) : Boolean = {

     val res = expmod(a, n, n)
     println("res = " + res + ", a = " + a + ", n = " + n + ", a % n = " + a % n)
     res == a % n
   }

   tryIt(random(n - 1) + 1)    // get a random int between 1 and n
 }

 def random(n : Int) : Int = { // return a random value between 0 and n - 1

   val random = new Random();

   random.nextInt(n);
 }
}

This was refactored from the following Scheme code, as an exercise in Scala:


(define (expmod base exp m)
 (cond ((= exp 0) 1)
   ((even? exp)
    (remainder (square (expmod base (/ exp 2) m))
      m))
   (else
     (remainder (* base (expmod base (- exp 1) m))
      m))))

 (define (even? n)
   (= (remainder n 2) 0))

 (define (fermat-test n)
   (define (try-it a)
     (= (expmod a n n) a))
     (try-it (+ 1 (random (- n 1)))))

 (define (fast-prime? n times)
   (cond ((= times 0) true)
     ((fermat-test n) (fast-prime? n (- times 1)))
   (else false)))

A note of the tail recursiveness of the function fastPrime:

Comment (summary) from PaulP from #scala:

"It's not tail recursive but only for one reason. It's neither final nor private. It just needs to be able to determine it won't be overridden. It'd be tailrec if in an object.

There has also been discussion of a @tailrec annotation so you could get a compiler warning if a tagged function wasn't compiled tailrec as you'd thought."

.

Scala, placeholders and partial functions

2009-01-02T07:22:00.000-08:00

A couple of Scala examples to demonstrate the use of place-holder syntax and partial functions. These two things are related but partial functions goes beyond syntactical conciseness to provide new and powerful constructions.

Substitution:


package test

object Placeholders {

  def main(args : Array[String]) {

    val myNumbers = 1 to 10 //List(1, 2, 3, 4, 5, 6, 7, 8, 9);

    println("Greater than 3")

    var result = myNumbers.filter((x : Int) => x > 3)

    result.foreach(println) // result.foreach((x) => println(x))

    println("Greater than 4")

    result = myNumbers.filter(x => x > 4)

    result.foreach(println)

    println("Greater than 5")

    result = myNumbers.filter(_ > 5) // _ placeholder _ for

    result.foreach(println)
  }
}

Partially applied functions:

Partial functions are functions defined with only some of the arguments (or none), as the following example demonstrates:


package test

object PartiallyAppliedFunctionsTest {

  def main(args : Array[String]) {

    val partiallyAppliedFunctions = new PartiallyAppliedFunctions();

    println("sum 4 arg 1 + 2 + 3 + 4 = " + partiallyAppliedFunctions.getSum4arg()(1, 2, 3, 4))                  // expect 10

    println("partiallyAppliedFunction 3 arg, 1 + 2 + 3 = " + partiallyAppliedFunctions.getSum3arg()(1, 2, 3))   // expect 6

    println("partiallyAppliedFunction 2 arg, 1 + 2 = " + partiallyAppliedFunctions.getSum2arg()(1, 2))          // expect 3
  }
}

class PartiallyAppliedFunctions() {

  private def sum(i1 : Int, i2 : Int, i3 : Int, i4 : Int) = {

    i1 + i2 + i3 + i4
  }

  def getSum4arg() = {

    sum _
  }

  def getSum3arg() : (Int, Int, Int) => Int = {

    sum(0, _ : Int, _ : Int, _ : Int)
  }

  def getSum2arg() : (Int, Int) => Int = {

    sum(0, 0,  _ : Int, _ : Int)
  }
}

The above example shows the function _ syntax to reference a function as a value.

Also, functions getSum3arg() and getSum4arg() show the use of partial functions to re-use an existing function by partially defining and supplying some of the arguments.

.

Scala, example finding prime numbers

2009-01-02T01:15:00.000-08:00

Firstly, Happy New Year! Happy 2009!

Continuing on with Scala, here is a further example, a simple test for prime numbers, also showing re-factoring from Scheme to Scala.

This simple example tests for primality, based on the simple algorithm taken from SICP in the following example given in the programming language Scheme:


package test

object FindPrimes {

  def main(args : Array[String]) {

    for (n <- 1 to 500) {

      val d = smallestDivisor(n)
      println(n + ": lcd = " + d + (if (n == d) " prime!" else " not prime") )
    }
  }

  def isPrime(n : Int) : Boolean = {

    n == smallestDivisor(n)
  }

  def smallestDivisor(n : Int) : Int = {

    findDivisor(n, 2)
  }

  def findDivisor(n : Int, testDivisor : Int) : Int = {

    if (square(testDivisor) > n)
      n
    else if (divides(testDivisor, n))
      testDivisor
    else
      findDivisor(n, testDivisor + 1)
  }

  def square(n : Int) : Int = {

    n * n
  }

  def divides(d : Int, n : Int) : Boolean = {

    (n % d) == 0
  }
}

The above example has a main that can be run. The code tests the first 500 integers for primality using the simple, unoptimised prime test algorithm which looks for the least divisor of n (below root(n)).

It's worth pointing out that the findDivisor function is recursive, but if you look at the way it's constructed, the recursion appears at the end of the construction - and is thus amenable to tail recursion (optimisation).

Note: When testing for Primes in this way, if no divisor d of n is found less than root(n) then the number is prime, thus the search for divisors can end at root(n). This is because for any divisor d of n, there is a corresponding divisor n/d. So the maximum value for either/both occurs at root(n), beyond this, as d increases above root(n), n/d decreases below root(n) symmetrically.

"If d is a divisor of n, then so is n/d. But d and n/d cannot both be greater than n."

In the statement:

println(n + ": lcd = " + d + (if (n == d) " prime!" else " not prime") )

We can see the use of an if expression. In this situation is looks rather like Java's ternary if construct, but Scala's if expressions along with other expressions like for etc are fundamentally different, in that the construct equates to a value, in this example a value of type implicitly determined as String.

However, it's worth noting that the following alteration to the println statement does not output "prime!" text:

println(n + ": lcd = " + d + (if (n == d) " prime!") )

My presumption (will confirm this) is that in the first case, all paths return String, so type inference determines type String. In the second case, the implicit else statement is missing, so not all paths return a String, so type inference probably determines a type of Nothing and the result is lost.

Update: As Jorge Ortiz has kindly pointed out in his comment, this is because the type inference of the Least Upper Bound (LUB). If not all paths return a type, the overall inference is Unit (like void) because this is the only safe inference that covers all cases.

.

Scala, Ant

2008-12-28T06:43:00.000-08:00

An example of building Scala applications using Ant and the Scala Ant tasks:

In the complete Ant build file given below, the key lines are:


<taskdef resource="scala/tools/ant/antlib.xml">
       <classpath>
           <pathelement location="${scala.home}/lib/scala-compiler.jar" />
           <pathelement location="${scala-library.jar}" />
       </classpath>
</taskdef>

Which loads the task definition

scala-library.jar defines Scala tasks for Ant, these are:

scalac
fsc
scaladoc

Note: The Scala library scala-library.jar must be present in the Java class path. The actors and dbc libraries are available as separate JAR files: scala-actors.jarand scala-dbc.jar.

Example build.xml


// LJB 28-12-2008 - basic scala Ant build
<project name="Scala test" default="build" basedir="../">

<property name="base.dir" value="${basedir}" />
<property name="sources.dir" value="${base.dir}/src" />
<property name="build.dir" value="${base.dir}/build//classes.tmp" />

// take scala ant libs from scala installation, assumes SCALA_HOME is set to home of scala in env vars
<property environment="env"/>
<property name="scala.home" value="${env.SCALA_HOME}" />

<target name="init">

  <property name="scala-library.jar" value="${scala.home}/lib/scala-library.jar" />

  <path id="build.classpath">
        <pathelement location="${scala-library.jar}" />
        <!--<pathelement location="${your.path}" />-->
        <pathelement location="${build.dir}"   />
  </path>

  <taskdef resource="scala/tools/ant/antlib.xml">
      <classpath>
          <pathelement location="${scala.home}/lib/scala-compiler.jar" />
          <pathelement location="${scala-library.jar}" />
      </classpath>
  </taskdef>
</target>

<target name="build" depends="init">

  <mkdir dir="${build.dir}"   />

  <scalac srcdir="${sources.dir}"
        destdir="${build.dir}"
        classpathref="build.classpath"
        force="changed">

      <!-- <src location="src" /> -->
      <include name="**/*.scala" />
  </scalac>
</target>

<target name="run" depends="build">
  <java classname="test.Test"
      classpathref="build.classpath">
  </java>
</target>
</project>

Project directory structure:

/build
/build/build.xml
/src
/src/test.scala

Example "hello world" Scala test program


package test

object Test {

def main(args : Array[String]) {

println("Hello world")
}
}

If you type ant run in the /build directory, the application is compiled and run, giving "Hello world"...

References:

http://www.scala-lang.org/node/98

.

Scala - object, singletons, classes and constructors

2008-12-27T12:25:00.000-08:00

Today, a simple example by way of explanation of the basic constructs of OO in Scala.

Things to note:

Scala does not use static methods, in this sense Scala is more purely OO than Java.
The equivalent is achieved using object singletons and companion objects.

The following code example demonstrates:

Use of object singleton and "main" application entry point
Declaration of new objects
Declaration of new class
Class primary constructor
Declaring fields
Declaring alternative constructors
Use of override keyword

Example Code:


package test

// object declares a singleton (like Java static)
object Test {
  // the usual main, takes an Array[String] of arguments
  def main(args : Array[String]) {

      var myClass1 = new MyClass(5, "Louis")  // using primary constructor
      var myClass2 = new MyClass(10);         // using alternate constructor

      // access public field value
      println("myClass1.value1 = " + myClass1.value1)

      // output using overriden toString
      println("myClass1 = " + myClass1)
      println("myClass2 = " + myClass2)
  }
}

// define MyClass and its primary constructor (Int, String)
class MyClass(v1 : Int, s1 : String) {

  // these are fields (public)
  def value1 : Int = v1
  def string1 : String = s1

  // this is an alternate constructor (calls primary constructor)
  def this(v1 : Int) = {
      this(v1, "hello")
  }

  // can override toString like in Java, note no return statement required + the override keyword
  override def toString() : String =
  {
      "MyClass: (" + value1 + ", " + string1 + ")"
  }
}

Scala, an introduction

2008-12-24T01:14:00.000-08:00

Scala - "Scalable Java": a new Language which combines both Object Orientated and Functional programming paradigms.

Scala is an extension of Java and runs on the JVM. Scala draws from languages such as Java, Smalltalk, Erlang and many others.

Scala site: http://www.scala-lang.org/

Downloads: http://www.scala-lang.org/downloads

Installation, using the IzPack Scala installer jar, run: java -jar scala-x-installer.jar

Scala - main features:

Integrates with Java (is an extension of Java), and shares libraries and primitive types
Compiles to Java byte code - runs on JVM (also can be run on .Net)
Imperative and functional support
Object Orientated (classes, objects)
Functional (functions as first class objects, referential transparency)
Statically typed
Type inference (more concise code that looks more like Dynamic/duck typed, but has the benefits of static / strong typing)
Supports closures and function literals
Concurrency model using Erlang Actors system (a 'safer' concurrency model)

Scala can be run stand alone, using the command line interpreter - using the "scala" command.

Scala also has plugin support for NetBeans, Eclipse and InteliJ

Syntax Overview:

Type inference means most type declarations can be omitted (but can still be explicitly declared).
Variables are defined as name : type rather than type name as per Java
Semicolons are often optional
Does not specify or define operators. +-*/ etc are valid method names are defined as such for appropriate types
var and val keywords - val declares a constant type (like Java's final, not reassignable), var declares a variable that can be reassigned. val, like var can still be mutable however, if the type is mutable.
Functions, declared using def keyword. e.g. def myFunction(x: Type1, y: Type2) : ReturnType
Pre and post increment/decrement (i++, ++i, i--, --i) are not supported, use i = i + 1 or i += 1
Recommended indentation is 2 spaces
Imperative for loops not supported, the functional equivalent, a for Expression is used instead
Arryays are indexed using (n) not [n]
Type parameters are specified using [type]. If both type and index are used, the ordering is [type](params)
Scala supports infix as method calls. e.g. 1 + 2 is equivalent to 1.+(2) where + is not a special operator, but a regular method
Applying () to an object (recipient) is equivalent to calling .apply() on it. e.g. myThing(i) is equivalent to myThing.apply(i)
The receiver of an = operator is equivalent to calling .update. e.g. myThing(i) = value is equivalent to myThing.update(i, value)
Method association is by default to the left, unless the method ends with a colon : e.g. 1 + 2 applies + to 1 giving 1.+(2) whereas x :: y applies :: (cons) to y, giving y.::(x)
The List type supports ::: (concat) and :: (cons, prepend), both apply the operation and return the resulting new List
Arrays are always mutable, lists are always immutable. Sets and maps can be either, the default is immutable (from the trait scala.collection.immutable) but the mutable equivalents can be imported from scala.collection.mutable
The -> method provides Implicit Conversion. e.g. x -> y is x.->y which returns a key/value tuple containing x as the key and y as the value

Some simple Functional examples:

Print all command line arguments, using foreach, passing println function:

Verbose:
args.foreach((arg: String) => println(arg))

Type inference on arg:
args.foreach(arg => pringln(arg))

Concise:
args.foreach(println)

The last example is a shorthand that can be applied when a function literal cosists of one statement that takes one argument - known as a Partially Applied Function.

For Expressions:

Replacing imperative for loops, the syntax is of the form:

for (arg <- args)
println(arg)

Note: In the above for expression example, arg is a val (final/const) not a var, and a new val variable is created for each "iteration".

or

for (i <- 1 to 30) ...

Declaring Arrays:

Very concisely:

val myArray = Array("value1", "value2")

defines a string array (type inference) of size 2

This is equivalent to:

val myArray = Array.apply("value1", "value2")

where apply is called on the Array Companion Object

More Verbosely the array could also be declared more traditionally as:

val myArray = new Array[String](2)

or even

val myArray : Array[String] = new Array[String](2)

but the former concise approach is the Scala recommended way, the later hints at the underlying Java roots.

Lists:

Defining new lists is easy with the cons method, as follows:

val myList = 3 :: 2 :: 4 :: 5 :: Nil // Nil is the Empty List, also so is List()

Note: The Nil (empty List) is needed so that working from right to left, the :: method is defined on Nil, which first gives Nil.::(5) returning List(5) which then gives 5.::(4) resulting in List(4, 5), and so on. Without the Nil there is a syntax error as :: (cons) is not defined on 5 (which is of type integer).

Examples:

Factorial ! - Simple factorial, defined in a functional way:

def factorial(value : int) : int = { if (value == 0) 1 else value * factorial(value - 1) }

scala> factorial(10)
res15: int = 3628800

Triangle Numbers:

def tri(x : Int) : Int = { if (x == 0) 0 else x + tri(x-1) }

scala> tri(4)
res14: Int = 10

Scala for scripting:

Scala scripts can be run using scala myscript.scala

Additionally, Scala can be used as a self contained scripting language by invoking the scala command line interpreter with the script, as follows:

On 'nix platforms

#!/bin/sh
exec scala "$0" "$@"
!#
println("Hello world, hello " + args(0))

Don't forget to give the script execute permissions (chmod +x myscipt)

Execute: myscript Louis

A similar thing can be achieved on Windows...

Scala with build systems:

With Ant:

http://www.scala-lang.org/node/98

With Maven:

http://scala-blogs.org/2008/01/maven-for-scala.html

Further Resources:

Books:

Artima book (excellent): http://www.artima.com/shop/programming_in_scala released 17th Nov 2008, by Martin Odersky, Lex Spoon, and Bill Venners.

Source code for book examples: http://booksites.artima.com/programming_in_scala/progInScalaExamples1EdV6.zip

Browse book examples: http://booksites.artima.com/programming_in_scala/examples

OReilly Programming Scala: http://programming-scala.labs.oreilly.com/

Blogs:

http://www.scala-blogs.org/

Blog on Monads: http://james-iry.blogspot.com/2007/09/monads-are-elephants-part-1.html

Articles:

Java refugees: http://www.codecommit.com/blog/scala/scala-for-java-refugees-part-1

Build tools:

SBT: http://code.google.com/p/simple-build-tool/

Interesting Blog about Scala and Lift (using Maven and Jetty).

http://scala-blogs.org/2007/12/dynamic-web-applications-with-lift-and.html

SIDs (software improvement Documents):

http://www.scala-lang.org/sids

Monads:

http://projects.tmorris.net/public/what-does-monad-mean/artifacts/1.1/chunk-html/ar01s04s04.html

http://vimeo.com/8729673

IRC channel: #scala on irc.freenode.net

(Also (unrelated): pastie - http://pastie.org/ is handy for pasting those odd snippets of code around on the internet/IRC).

Interesting blog about Scala type-safe SQL / DB - SQuery - http://szeiger.de/blog/2008/12/21/a-type-safe-database-query-dsl-for-scala/

Updated (13-09-2009): Came across this useful presentation: http://www.slideshare.net/astubbs/scala-language-intro-inspired-by-the-love-game?src=embed

99 Scala problems - an adaption of the 99 Prolog problems into Scala:

http://aperiodic.net/phil/scala/s-99/#p10

.

PCWiz Mac OSX under VMware (2.0)

2008-12-21T14:47:00.000-08:00

(Note - I do not condone the use of illegal / pirated software. If you're going to use Mac OSX, please support Applet and buy a license for the OS.)

Mac OSX can be run under VMware, using a special build such as the "pcwiz" version.

http://pcwizcomputer.com/index.php?Itemid=48&id=76&option=com_content&task=view

Performance is slow, but just about usable.
There's naturally no vmware tools at this time.
Networking can be made to work.

When it comes to the screen resolution, there are no drivers as such to adjust but the screen resolution settings can be modified - see here for further details: http://pcwizcomputer.com/index.php?option=com_content&task=view&id=31&Itemid=45

.

OpenOffice (3.0) / UNO API

2008-12-13T16:08:00.000-08:00

Recently a project needed some reporting enhancements (spreadsheet charts) for which Apache POI didn't provide the necessary support. So, after some exploration, I decided on using OpenOffice with it's API, in this particular instance it was OpenOffice 3.0m9 (latest at the tine of writing) and the UNO Java API.

www.openoffice.org

OpenOffice provides some very powerful tools, including Spreadsheets, Documents, PDF etc along compatibility with MS Office. In addition, OpenOffice is quite powerful at convertion document types and between versions. For example a MS word document could be loaded and then saved as a PDF.

OpenOffice - www.openoffice.org
Forum - http://user.services.openoffice.org/en/forum/
Wiki - http://wiki.services.openoffice.org/wiki/Main_Page (including good developer section)
FAQ - http://wiki.services.openoffice.org/wiki/Uno/FAQ

The OpenOffice suite comes with an SDK and the UNO API, supporting various languages, including Java.

SDK download - http://download.openoffice.org/3.0.0/sdk.html

The SDK/API is based on UNO (Unified Network Objects) which is an alternative/competator to COM, like CORBA etc, and allows the remote (out of process) communication and control of OpenOffice - "UNO is a component model that offers inter-operability between different programming languages, different objects models, different machine architectures, and different processes; either in a LAN or via the Internet."

Another acronym is URE - "Each component lives in a Uno Runtime Environment (URE). A URE is identified by the implementation language (e.g., C++, Java, Perl, ...) and the current process."

See here for more details: http://udk.openoffice.org/common/man/uno.html

When using OpenOffice on the server side to provide reports (in my case, as part of a web application), various SDK jars are needed along with a means to find the OpenOffice installation.

The Java jars needed are:

juh.jar
jurt.jar
ridl.jar
jut.jar
unoil.jar

I chose to include them in my web application WAR file.
This can create problems, because the classpath to the jars will not be where OpenOffice is installed, to work around this the Java classes from the SDK can be loaded and bootstrapped in the class loaded, but this is tricky when using a web app container. For convienience I made use of the useful bootstrapconnector.jar utility library. This library can connect to OpenOffice regardless of the location of the jars, it just needs a path to an OpenOffice installation, which I set as an ENV variable called UNO_PATH.

BootstrapSocketConnector - http://user.services.openoffice.org/en/forum/viewtopic.php?f=44&t=2520

I also made use of the OpenOffice plugin for Netbeans 6.5 to get a quick start developing the code, before moving to the production solution.

Some things to watch out for:

One problem I stumbled into. Installing OpenOffice for the first time on a remote Linux (CentOS) machine over a terminal, when I came to run the web application the program appears to hang. Some investigation with ps showed that soffice was being started, but that I was getting many instances, all apparently hung. I replicated the setup on a different Linux installation, this time one with a desktop / display manager. It was then obvious what was happening, the first time OpenOffice is launched, it prompts the user with a registration wizard! Oops.

So the solution to this is simple, pass an option when launching the OpenOffice process, this can be achieved conveniently using the bootstrapsocketconnect with something like the following:

// Create OOo server with additional -nofirststartwizard option
List oooOptions = OOoServer.getDefaultOOoOptions();
oooOptions.add("-nofirststartwizard");
OOoServer oooServer = new OOoServer(OOO_EXEC_FOLDER, oooOptions);

// Connect to OOo
BootstrapSocketConnector bootstrapSocketConnector = new BootstrapSocketConnector(oooServer);
XComponentContext remoteContext = bootstrapSocketConnector.connect();

JSON, Java and JavaScript

2008-12-07T12:08:00.000-08:00

JSON is so simple there's not a lot to say about it, but I thought it's worth putting up a few links and notes for reference.

JSON is JavaScript Object Notation. Whilst it could be called an Object Notation (a simple one), it's not tied to JavaScript, although it's early origins are from that language (ECMAScript, ECMA-262 3rd Edition). JSON is essentially language independant, like XML. JSON, like XML can represent data structures and data (self describing) but unlike XML, JSON does not support namespaces, schema and other more advanced features. JSONs key strengths are its simpicity and low overhead, terse message size.

JSON looks like this:


{
     "firstName": "John",
     "lastName": "Smith",
     "address": {
         "streetAddress": "21 2nd Street",
         "city": "New York",
         "state": "NY",
         "postalCode": 10021
     },
     "phoneNumbers": [
         "212 555-1234",
         "646 555-4567"
     ]
}

Where { and } contain the object, the properties are listed as "name" : "value" pairs. Objects can be embedded within objects and arrays are denoted using comma separated lists within []

So, special symbols (for escaping) are: {}[]:"

Main Site: http://www.json.org/

Simple JSON for Java: http://www.JSON.org/java/json_simple.zip http://www.json.org /java/simple.txt

Simple JSON for JavaScript: http://www.json.org/js.html

Wikipedia: http://en.wikipedia.org/wiki/JSON

JavaScript:

Basic suppport for converting JSON messages into objects (deserialisation) is to use the often best avoided eval function, as follows:

var myObject = eval('(' + myJSONtext + ')');

where the outer '(' ')' are necessary for the JavaScript language, not JSON itself.

Better (safer) still, various libaries are available, such as the Open Source version from www.json.org in json2.js

Which essentially is used like this:

var myObject = JSON.parse(myJSONtext, reviver);

var myJSONText = JSON.stringify(myObject, replacer);

Java:

Create a JSON string using org.json.simple.JSONObject:

JSONObject obj=new JSONObject();
obj.put("name","foo");
obj.put("num",new Integer(100));
System.out.print(obj);

The toString() on the JSONObject serialises the JSON object (map of name value pairs) to a JSON string.

JSON is simple, by design, so there isn't much more to be said on it.

One of the main things to watch out for is encoding/escaping messages so they are JSON friendly, I'll add more on that soon.

The following link may be useful when working with character escaping: http://www.the-art-of-web.com/javascript/escape/

.

BIOS LBA / Hard disk, apparent failure problem "Blinking cursor of doom"

2008-11-23T10:40:00.000-08:00

A rather interesting low level problem: If you ever get a machine inexplicably stuck after POST with a blinking cursor in the top right and nothing more and want to know why or fix it, please read on!

Whilst there are a number of possible causes (like incorrect BIOS settings) I'm assuming all of those have been checked and ruled out, if so and the problem still persists then the following might be an explanation:

A while back I upgraded my ageing Dell Ispiron 6000 laptop hard drive (80gb) with a 160gb drive. I cloned the original drive to retain everything but made the partition larger to make the extra space available. The machine ran XP Pro SP2 and had been stable for a couple of years, since I first got it (after a battle with a nasty virus that I evenually managed to win a few months previously).

An easy enough job and all was good, for a while. One day the machine needed a reboot, after shutting down it came back with the Blinking cursor of Doom!

This is where, straight after POST (Power On Self Test) the machine totally hangs (before Windows starts up). My first thoughts (after oh crap) where along the lines of, has the drive died, mbr gone wrong, virus trashed the drive, partition table messed up, file system corrupt etc?

After many hours, I'd done a full chkdsk /r, fixmbr, checked the MBR (Master Boot Record) and Partition Table information, made sure the partition was active and so on, I even moved and resized the partition slightly, swapped the drive into another machine (was fine), tried a different drive in the laptop, also fine. Checked my BIOS (Basic Input Output System) for updates and so on - nothing seemed to be wrong - weird!

The MBR is the sector at cylinder 0, head 0, sector 1 - it's launched according to the BIOS boot sequence, if the BIOS finds a valid MBR it's copied to memory and executed as part of the boot sequence of the machine.

I also looked and the Windows MBR code (disassembly) from here http://mirror.href.com/thestarman/asm/mbr/STDMBR.htm to check all paths through the boot loader - all likely paths result in a message of some sort like the archaic "Please insert boot disk" type messages. I just got a blank screen with flashing cursor - how come?

I could see all the files on the disk, the disk and it's contents appears fine - so the question was what on earth was stopping the disk from being bootable!?. It defied any logic, surely if the file system is intact, has no errors, the partition table was fine and everything was set for the system to boot from it, why didn't it?

After much time was spent on it (I really didn't want to have to set everything up again) eventually I decided it would be quicker to re-ghost from the original 80gb drive and get back to where I was 2 or 3 months previously, then copying over any new data from the fine but un-bootable 160gb drive.

I did this and all was well again (but in the back of my mind I was never happy I'd not managed to get to the bottom of it).

My machine was working but I was puzzled. The machine starts up in the System BIOS, it runs various checks and initialises the hardware and subsystems before trying to find a bootable volume. Assuming a bootable hard disk is found, the MBR is executed, the partiton tables are examined and active partition is bootstrapped, the first "file" lauched in a windows 2000/XP/NT platform being the ntldr program responsible for starting windows (and capable of multiboot).

Whilst dissatisfied I'd not understood the problem, as days and weeks passed, the problem faded into a dim and distant memory...

--

Then, about 3 - 4 months later - after updating ITunes and needed to reboot, the same thing stuck again! Now this time, whilst rather disappointed, I knew there had to be an explanation. Maybe it was hardware related, maybe the machine was unreliable - how could I trust it again if it did this every so often?

So, this time around I thought I need to get to the bottom of this - there must be a reason, it's happened twice in a row, and suspiciously in roughly similar time spans (3 months or so) from the original disk clone - this had to be a clue.

So, after chkdsk /r etc just to check everything was ok, I got to work.

The disk was un-bootable, but the file system was fine. It hung immediately after POST, but with the briefest of flashes of the hard disk light. This had to be something low level, before windows, after POST - not much happens, especially knowing the MBR and partition tables were intact and fine too.

I renamed c:\ ntldr, IO.SYS and NTDETECT.COM, the low level bootstrap files for Windows.

After booting, the machine did exactly the same. Ahah this was a clue of sorts, if the disk was bootable but it wasn't getting as far as running ntldr then the fault was prior to that. If MBR and partition table were ok, then the fault was prior to that also, so, the BIOS?

I checked again for BIOS updates, nothing released matched the problem.

Then, after much Googling came the clue. Some BIOSes have an intrinsic 137gb limit.

That's 137gb of addressable disk, which might translate to less taking into account the overhead of the file system and block sizes etc.

Whilst Windows is happy to grow the file system behond this limit, eventually the BIOS will reach this maximum and barf. So, it could explain it....

There was only one way to find out. I resized the partition (made smaller) and created a new one in the new spare space. I copied files over from the original partition and then resized it smaller again, growing the new one until they were about the same size (2 x 80gb partitions rather than 1 x 160gb).

I restart with the downsized partition this time the machine comes up with a more helpful missing ntldr error message - ahah! it's tried to boot from the disk!

Booting up for a final time from the boot disk I renamed the files (ntldr etc) back. Reboot one last time and hey presto, it's working fine. Nothing lost, and this time I only lost a few hours (3 or 4) from my life and most importantly, I got some closure on the problem and an understanding of what happened and how to avoid it forever.

Greatly satisfied and relieve, I thought I'd share this experience in case it helps someone else in a similar position. Like anything, when you know the answer it's easy, but I've worked with computers for years and for a while there I was completely stumped with a problem that seemed, rather un-computer like, to defy all logic!

Whilst I'm entirely happy with two smaller partitions, the longer term solution is known as 48bit LBA, AKA Big LBA - http://en.wikipedia.org/wiki/Advanced_Technology_Attachment

References:

http://en.wikipedia.org/wiki/Master_boot_record
http://en.wikipedia.org/wiki/Booting#Boot_loader
http://en.wikipedia.org/wiki/Windows_NT_Startup_Process

http://mirror.href.com/thestarman/asm/mbr/STDMBR.htm
http://www.ata-atapi.com/hiwmbr.html

http://en.wikipedia.org/wiki/Advanced_Technology_Attachment

.

Applet <-> Javascript integration

2008-11-23T08:42:00.000-08:00

It's been a while since I've done this, but I thought I'd refresh my memory on the options available. Maybe Applets have fallen out of favour a bit and whilst Javascript is popular for Rich, Dynamic Internet Applications, sometimes you need more and one way to do it is to use Applets, or integrate Applets with Javascript.

This example shows some simple ways to achieve Applet/JavaScript integration:

I've included example HTML with JavaScript source and Java source code for an Applet, to demonstrate so ways to achieve JavaScript to Java Applet integration. The simple example code demonstrates both JavaScript calling a Java Applet and a Java Applet calling JavaScript.

The JavaScript function jsShowMsg(String) is called in two different ways from the Java Applet. A single string parameter is passed, and the JavaScript function displays this using the standard alert() function to create a dialog box.

The Applet provides a showMessage(String) public method, which similarly is called by the JavaScript code when the button on the page is pressed. The String parameter is taken from the text field and displayed in the Applet.

HTML Page:

Provides the Applet on the page. Provides the JavaScript function called by the Applet, calls the Applet public method when the button is clicked (takes the String value from the input text box.

--


<HTML>
<HEAD>
 <TITLE>Applet HTML Page</TITLE>
</HEAD>
<BODY>
  <script type="text/javascript">
   
      /**
       * jsShowMsg - demostrates being called by an Applet, being passed a single String argument, the message
       */
      function jsShowMsg(msg)
      {
          alert("Message from applet was: '" + msg + "'");
      }

      /**
       * callApplet - demonstrates calling a public method on the Applet (showMessage(String))
       */
      function callApplet(msg)
      {
          //alert("before calling Applet method with msg=" + msg);

          document.testApplet.showMessage(msg);

          //alert("after calling Applet method");
      }
  </script>

  <H3><HR WIDTH="100%">Applet HTML Page - Applet <-> JavaScript integration example<HR WIDTH="100%"></H3>

  <P>
  <APPLET name="testApplet" codebase="classes" code="com/chillipower/TestApplet.class" width=400 height=150></APPLET>
  </P>

  <input type="text" id="msgTb" name="msgTb" value="A message from JS!" /><input type="button" value="button1" onclick="callApplet(getElementById('msgTb').value);" />

</BODY>
</HTML>

--

Java Applet Class:

Calls the JavaScript function in two different ways when the Applet is started. Provides a showMessage public method that the JavaScript can call.

--


package com.chillipower;

import java.applet.Applet;
import java.awt.Color;
import java.awt.Graphics;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.net.MalformedURLException;
import java.net.URL;

/**
* @author Louis B
*/
public class TestApplet extends Applet
{
  String JS_MESSAGE   = "Hello from Java Applet to JavaScript (using JavaScript alert)";
  String JS_FUNC_NAME = "jsShowMsg";


  /**
   * init - applet init function
   */
  @Override
  public void init()
  {
      setBackground(new Color(0xFAFAFF));
  }

  /**
   * start - applet start function
   */
  @Override
  public void start()
  {
      callJsByProto(JS_FUNC_NAME, JS_MESSAGE + " using showDocument :javascript protocol method");
            
      callJsFunc(JS_FUNC_NAME, JS_MESSAGE + " using JSObject method");
  }

  /**
   * Overide paint with simple text rendering
   *
   * @param g
   */
  @Override
  public void paint(Graphics g)
  {
       g.drawString("The Applet", 20, 20);
  }

  /**
   * Simple function to show a message
   *
   * @param msg
   */
  public void showMessage(String msg)
  {
      Graphics g = getGraphics();
      g.setColor(Color.RED);
      g.drawString("Message from JS: '" + msg + "'", 20, 40);
  }

  /**
   * Call a named JS function, passing a single String parameter
   *
   * @param funcName
   * @param msg
   */
  private void callJsByProto(String funcName, String msg)
  {
      try
      {
          getAppletContext().showDocument
              (new URL("javascript:" + funcName + "(\"" + msg +"\")"));
      }
      catch (MalformedURLException ex)
      {
          System.out.println("ex: " + ex.getMessage());
      }
  }

  /**
   * callJsFunc - call the named JavaScript function, passing the String msg argument
   *
   * @param funcName
   * @param msg
   */
  private void callJsFunc(String funcName, String msg)
  {
      String jscmd = funcName + "('" + msg + "')";    // Buiild the JavaScript command
      String jsresult = null;
      boolean success = false;

      try
      {
          Method getw = null;
          Method eval = null;
          Object jswin = null;
          Class c = Class.forName("netscape.javascript.JSObject"); /* works in IE too */
          Method ms[] = c.getMethods();
      
          for (int i = 0; i < ms.length; i++)
          {
              if (ms[i].getName().compareTo("getWindow") == 0)
              {
                  getw = ms[i];
              }
              else if (ms[i].getName().compareTo("eval") == 0)
              {
                  eval = ms[i];
              }
          }
    
          Object a[] = new Object[1];
          a[0] = this;               /* this is the applet */
          jswin = getw.invoke(c, a); /* this yields the JSObject */
          a[0] = jscmd;
          Object result = eval.invoke(jswin, a);
      
          if (result instanceof String)
          {
              jsresult = (String)result;
          }
          else
          {
              jsresult = result.toString();
              success = true;
          }
      }
      catch (InvocationTargetException e)
      {
          jsresult = e.getTargetException().getMessage();
      }
      catch (Exception e)
      {
          jsresult = e.getMessage();
      }

      if (success)
      {
          System.out.println("JS eval succeeded, result is " + jsresult);
      }
      else
      {
          System.out.println("JS eval failed with error " + jsresult);  
      }
  }
}

Debugging:

1. Using the Java ControlPanel (on 'nix, it is View Settings -> Edit the Java Runtime settings.

-agentlib:jdwp=transport=dt_socket,address=:

This makes the browser plug-in JRE open a debugging socket connection when launching the applet.

2. Open the Netneans project and attach the debugger in the SocketListen mode, using the same settings as above.

Resources:

http://windyroad.org/2006/08/14/reintroducing-javascript-and-hidden-applets-jaha/

http://java.sun.com/javase/6/docs/technotes/guides/plugin/developer_guide/java_js.html

.

Apache httpd 2.x -> Glassfish 2.x mod_jk installation

2008-11-16T08:29:00.000-08:00

Configure Apache Httpd to Glassfish using mod_jk.

Note, for Glassfish 3.x this process has been simplified, I'll probably add a blog entry about that sometime soon.

In this particular scenario, Apache 2.2.3 is running on Centos 5.2 x86_64 and Glassfish 2ur2 is running on OpenSolaris 10 b101, althought the process should be essentiually the same for any Operating system(s), the paths etc might change.

Apache httpd setup:

Get the Apache mod_jk connector.

http://tomcat.apache.org/connectors-doc/

Binary distribution:

http://www.apache.org/dist/tomcat/tomcat-connectors/jk/binaries/

Install into modules directory: e.g. /etc/httpd/modules (RH/Centos)

Configure Apache httpd:

e.g. for RH/Centos in - /etc/httpd/conf.d/

Add ajp worker mapping to httpd.conf (e.g. in /etc/httpd/conf/), in this example using virtual host entries, e.g.


# Virtual host chillipower.com
[VirtualHost 192.168.0.200]

DocumentRoot /var/www/mywebs/chillipower.com

ServerName www.chillipower.com

ServerAlias chillipower.com

DirectoryIndex index.html index.php index.htm index.shtml

JkMount /chillipower*/* worker1

[/VirtualHost]

Note in the above, the url to the glassfish context would be reached using http://mydomain.com/chillipower/ where in this instance the trailing / is important.

Load and configure the module (e.g. place this in /etc/httpd/conf.d/mod_jk.conf)

(all *.conf files are loaded by httpd.conf (before any virtual hosts etc) from the /conf.d directory in typical setups, so we can add a new mod_jk.conf and it will be loaded automatically)



LoadModule jk_module /etc/httpd/modules/mod_jk.so
JkWorkersFile /etc/httpd/conf.d/worker.properties
# Where to put jk logs
JkLogFile /var/log/httpd/mod_jk.log
# Set the jk log level [debug/error/info]
JkLogLevel debug
# Select the log format
JkLogStampFormat "[%a %b %d %H:%M:%S %Y] "
# JkOptions indicate to send SSL KEY SIZE,
JkOptions +ForwardKeySize +ForwardURICompat -ForwardDirectories
# JkRequestLogFormat set the request format
JkRequestLogFormat "%w %V %T"
# Send all jsp requests to GlassFish
JkMount /*.jsp worker1
# Send all glassfish-test requests to GlassFish
JkMount /chillipower/* worker1

Add worker.properties file to configure the ajp worker, /etc/httpd/conf.d/worker.properties


# Define 1 real worker using ajp13
worker.list=worker1
# Set properties for worker1 (ajp13)
worker.worker1.type=ajp13
#worker.worker1.host=localhost.localdomain
worker.worker1.host=192.168.3.101
worker.worker1.port=8009
worker.worker1.lbfactor=50
worker.worker1.cachesize=10
worker.worker1.cache_timeout=600
worker.worker1.socket_keepalive=1
worker.worker1.socket_timeout=300

Glashfish setup:

Download Apace Tomcat 5.5.16 -> http://archive.apache.org/dist/tomcat/tomcat-5/

copy the tomcat-ajp.jar to the glassfish lib folder. e.g.

cp $CATALINA_HOME/server/lib/tomcat-ajp.jar
$GLASSFISH_HOME/lib/.

Also, download and copy the commons logging and commons modeller jar files to $GLASSFISH_HOME/lib

Commons logger -> commons-logging-1.1.1.jar from http://commons.apache.org/downloads/download_logging.cgi

Commons modeler -> commons-modeler-2.0.1.jar from http://commons.apache.org/downloads/download_modeler.cgi

Restart Glassfish...

References:

http://weblogs.java.net/blog/jfarcand/archive/2006/03/running_glassfi_1.html

.

c# code generation added rest describe code gen

2008-11-09T10:17:00.001-08:00

Announcement: c# code generation added to google rest describe code gen api:

At last, the google rest describe code gen api now has a c# code generator. Targeting c# .net 3.5 WCF. This gives an easy route from existing services or WADL to c# client code/framework. I hope to be blogging about it soon, with some details of how it works and any limitations you shoud be aware of...

I'm back! Some notes on SELinux and Apache HTTPD

2008-11-03T13:49:00.000-08:00

Finally! I took some time out, with a big road trip to America and various other activities going on, I'd not had the time to keep the blog going. Anyway, things are back to normal now and after a recent hard disk failure I unexpectedly found myself setting up a new secure web server using CentOS 5.2 x64 and Apache Httpd 2.2.3 and PHP5. Later I'll add in a Java EE application server for more dynamic/web app stuff.

Thought that I'd type up some small notes on a problem I'd come across before, when trying to setup Mercurial HG-WEB in directories (or sym links) to directories outside of the usual /var/www

One of the problems may come from SELinux. Whilst it's tempting to turn it off, for something facing the public on the web - this would be a real last resort!

Security Contexts: - a quick explanation

Files and directories have security contexts. These can be seen using ls, e.g.

ls -aZ

By default Apache Httpd cannot access all files, it can access those with httpd_sys_content_t context types.

For example user home directories come with the context home_root_t which will give a permission denied type error if httpd tries to serve up files from there (or anywhere below!). Another common security context is default_t which will be assigned as the default for new files and folders.

An important difference between mv and cp

cp creates a new context (inherits), wheras mv actually moves it (unmodified). So a common gotcha is to create files in the user home directory and mv them to /var/www/... but this keeps the home context home_root_t - hence this creates the permission denied error.

Some handy commands:

chcon -R -h -t httpd_sys_content_t /test/

change the directory /test/ to httpd_sys_content security context type, recursively but not following sym-links.

i.e. the above would enable /test/ to be served by a documentroot in Httpd

If SELinux is suspected for the permissions problems (as opposed to httpd configuration), the SELinux restrictions will appear in the message log:

tail -f /var/log/messages

SELinux boolean flags:

can be read with

getsebool httpd_enable_cgi

and set with

setsebool httpd_enable_cgi true

Invoke the simple GUI configurator with: system-config-securitylevel

Restore the file context (say after an mv command):

restorecon -R -v ./folderwithwrongcontext/

Additional SE Linux parameters:

Enable Samba to access home dirs:

setsebool -P samba_enable_home_dirs=1

Additional Resources:

Good overview in this link: http://docs.fedoraproject.org/selinux-apache-fc3/sn-simple-setup.html

An interesting query for potential optimisation...?

2008-07-18T02:50:00.000-07:00

Just wanted to capture a query I was working on yesterday which was slightly 'interesting' in that it consists of a couple of layers of left outer joins and the joining with a derived aggregation table - and time permitting I'd like to revisit it because I'm sure it can be improved / optimised (but it is functionally correct).


select
  t1.tid,
  t1.alias as alias,
  t1.asset_label as asset_label,
  t1.terminal_id as terminal_id,
  t1.location_address as terminal_address,
  t1.last_updated as last_connection,
  t1.name as questionnaire_name,
  t1.questionnaire_id as questionnaire_id,
  t1.state as state,
  t1.value as publishing_state,
  response_stats.rcount as rcount,
  response_stats.last_response as last_response
from
  (select 
     t.id as tid,
     t.alias,
     t.asset_label,
     t.terminal_id,
     t.location_address,
     t.last_updated,
     qnr.name,
     qnr.id as questionnaire_id,
     qnr.status as state,
     at.value
   from terminal t
     left outer join
       (delivery_stats d,
        terminal_activity ta,
        questionnaire qnr,
        activity_type at)
     on 
       d.terminal_id = t.id
       and d.terminal_activity_id = ta.id
       and ta.questionnaire_id = qnr.id
       and ta.activity_type = at.id
       and qnr.status = "L"
     where
       t.id in(1, 20, 21, 22, 23)) as t1
left outer join
  (select
     tcq.terminal_id tid,
     tcq.questionnaire_id qid,
     count(*) rcount,
     max(uploaded_date) as last_response 
   from
     tcquestionnaire tcq
   group by
     tcq.terminal_id, tcq.questionnaire_id) as response_stats
on 
  t1.tid = response_stats.tid
  and t1.questionnaire_id = response_stats.qid;

Whilst technically correct - I'm not entirely satisfied of the need for the the top level left outer join which creates a full derived table rather than having a correlated sub-query / nested join that contains only the rows actually relevant to the result set. Something to think about perhaps, on a rainy (very rainy!?!) day :)

.

Update vmware server 2.0 from beta 2 to rc1

2008-07-16T12:03:00.000-07:00

Just some quick notes on this "upgrade" from beta 2 to rc1 - which is more of a reinstall!

Note: pre-release versions forcefully expire (becomes rather apparent when starting a vmware instance) - but the error message is not that clear - "Failed to start, General error" may appear in the logs as the error but is preceded by a more subtle "version expired, there's a new version out" type message - which is the actual problem.

Basic steps:

Stop/suspend all vm's

Stop services
#service vmware stop

[Recommended - move VMs (for safety/paranoia!), simply mv the whole vmware directory]
[Recommended - good time to check system is up-to-date]

Remove/uninstall existing version (based on 2.0 beta2):
#rpm -e VMware-server-e.x.p-84186.x86_64

install rc1
#rpm -ivh VMware-server-2.0.0-101586.x86_64.rpm
or
#rpm -Uvh VMware-server-2.0.0-101586.x86_64.rpm

configure as per normal
#vmware-config.pl

check running via web management console

service vmware stop

move vmware images back

start services again

Note, VM images have to be reloaded into the VM inventory - apart from that everything is entirely straightforward!

Alternative approach from tarball:

#tar xvfz VMware-server-*.tar.gz

#vmware-server-distrib

#vmware-install.pl

Some relevant links:

http://communities.vmware.com/thread/154503?tstart=0
http://communities.vmware.com/thread/154668?tstart=0
http://communities.vmware.com/message/901195;jsessionid=77A29EE9DF1ADEBF37027EB347D638F3
http://www.cyberciti.biz/tips/vmware-on-centos5-rhel5-64-bit-version.html
http://kirkpbm.wordpress.com/category/vmware-server/
http://communities.vmware.com/docs/DOC-4111
http://communities.vmware.com/thread/143674

.

OpenSolaris - upgrade to b93

2008-07-12T14:03:00.000-07:00

More notes on OpenSolaris image-update, to b93 (in my case from b91)

See this thread http://opensolaris.org/jive/thread.jspa?messageID=258127&#258127 discussing the problem and it's solution.

Performing the standard image-update failed with an error of the form:

> pkg: attempt to mount opensolaris-3 failed.
> pkg: image-update cannot be done on live image

The problem is caused by:

2387 libbe.so:beCopy() frees nvlist variables before using them
http://defect.opensolaris.org/bz/show_bug.cgi?id=2387

So, time to find a work around. This wasn't as seamless as perhaps it should have been, this is how I did it:

#beadm list

> my active BE was opensolaris-3, which I was booted in

#export PKG_CLIENT_TIMEOUT=2000

#pfexec pkg refresh

create a temp BE
#pfexec beadm create opensolaris-4

mount it as /mnt
#pfexec mount -F zfs rpool/ROOT/opensolaris-4 /mnt

update it
#pfexec pkg -R /mnt image-update

#pfexec beadm unmount opensolaris-4

#pfexec beadm activate opensolaris-4

reboot into BE opensolaris-4 build 93

Note: b92 was skipped (not released)!

Also, this failed _every time_ when attempting it from desktop terminal (graphical login) - but worked as soon as I did the same thing from an ssh terminal!

Blog for installing / configuring mysql as service using gui http://weblogs.java.net/blog/bleonard/archive/2008/06/opensolaris_200_3.html

Additional notes:

BE_PRINT_ERR=true can be used to output diagnostic error logs from the be process,

e.g.

BE_PRINT_ERR=true beadm create test

.

C# delegates

2008-07-10T06:21:00.000-07:00

A quick note on c# delegate functions:

I found the standard way to define delegates a little clumsy and verbose in some situations, for example when wanting to pass a function to a .Invoke method of a control to add it to its thread work queue. In this situation the delegate's purpose is really just to pass the method to the invoker, there is no other use of the delegate so declaring it at class level is a much wider scope that I would like. What I wanted is a more terse, compact and locally scoped way to achieve the same thing, here it is:

The simple way:

Declare delegate within class, e.g.

public delegate void updateStockView(Stock[] stocks);

use to create a new delegate instance to pass to invoke on a control, e.g.

instrumentDgv1.Invoke(new updateStockView(this.updateStockView), new object[] {instrumentResponse.Stocks});

This approach, whilst perfectly valid feels a little cumbersome. Firstly the delegate is at an uneccessarily high level of scope, secondly, the statment requires a new instance of the delegate to be created, passing the delegate's parameters down as an object array.

A more compact form:

instrumentDgv1.Invoke((MethodInvoker)delegate { this.updateStockView(instrumentResponse.Stocks); });

Here we see, no unnecessary declaration of the delegate type, no creation of the delegate instance, no passing of parameters as an object array - all round much tighter syntax for this type of use.

The key thing to make this work is the MethodInvoker cast. MethodInvoker is from the System.Windows.Forms namespace and is used to cast the existing method to the delegate type without requiring the declaration/instantiation of a new delegate.

.

Netbeans 6.1 Rest JSR-311 RI library problem

2008-07-07T13:20:00.000-07:00

Found a minor problem with Netbeans 6.1 and Rest (Jersey RI) today, thought I'd capture the problem and solution.

Symptom: Can't launch a Rest application using Jersey (JSR-311 RI) libraries - error given is:

HTTP status 500

exception

javax.servlet.ServletException: java.lang.NoSuchMethodError: javax.servlet.jsp.JspFactory.getJspApplicationContext(Ljavax/servlet/ServletContext;)Ljavax/servlet/jsp/JspApplicationContext;
 org.apache.jasper.servlet.JspServlet.service(JspServlet.java:274)
 javax.servlet.http.HttpServlet.service(HttpServlet.java:803)

Because the Jersey (JSR-311 RI) libraries include jsp 2.0 jsp-api-2.0-20040521.jar from Netbeans6.1\enterprise5\modules\ext\rest\ which is an older version of the JSP API.

Fix, copy a newer jsp in or remove the entry from the library definition and add a newer jsp library in.

Example Solution:

Go to tools\libraries
Select the Jersey libary
In the Library classpath, remove the offending jsp-api-2.0-20040521.jar
Now add the newer JSP libary, e.g. Netbeans 6.1 ships with servlet2.5-jsp2.1-api.jar one level up in the Netbeans6.1\enterprise5\modules\ext\ directory.
Remove the Jersey libraries from your project (if already added).
Re-add the Jersey libraries to your project and recompile - problem solved.

Stunnel and wcf 3.5 https testing, an easy dev setup

2008-06-25T03:22:00.001-07:00

Just some quick notes:

Stunnel can act as an https proxy which is very convenient when you need to test / develop against https but don't want to set up or use a proper environment with real certificates etc in the short term.

I wanted to capture some packet traces of WCF3.5 over an https keep alive channel to a Jboss app server serving normal http on the default 8080 - here's what I did:

Download / install Stunnel. The installation process creates a default .pem self signed certificate (sufficient for dev purposes).

Default Stunnel config is in stunnel.conf (or stunnel.cfg on Linux).

For simple https proxying from the standard port 443 to port 8080, the basic condensed configuration in stunnel.conf looks like:

cert = stunnel.pem
socket = l:TCP_NODELAY=1
socket = r:TCP_NODELAY=1
debug = 7
output = stunnel.log
[https]
accept = 443
connect = 8080

where we can see the default certificate is being used and I've enabled logging. HTTPS listening on 443 and connecting through to HTTP on 8080 (happened to be Jboss in this instance).

Simple enough, what next. Running the WCF client application - it will barf over the certificate not being trustworthy (an error of the form):

login failed Could not establish trust relationship for the SSL/TLS secure channel with authority 'localhost'

Overriding .NET WCF 3.5 default certificate policy to allow all certs for testing/dev purposes

For test/dev purposes it's often handy to allow all certificates.

What you can do is override the certificate checking policy to "say yes to everything" (Remember to take this out for product release!)

The code looks something like this:

using System.Net;
using System.Security.Cryptography.X509Certificates;
using System.Net.Security;

...

// ***** WARNING - to be removed for release! *****
ServicePointManager.ServerCertificateValidationCallback +=
delegate(object sender, X509Certificate certificate, X509Chain chain, SslPolicyErrors sslPolicyErrors)
{ return true; };

Which is using an anonymous delegate callback to return true in all cases (true = accept) - which keeps the code nice and compact.

If you're specifically testing keep alive then these additional parameters for stunnel.conf control how the proxy behaves in terms of connection persistence:

session = 300
TIMEOUTclose = 0
TIMEOUTidle = 180

OpenSolaris 2008.05 & update to >= b90

2008-06-20T13:05:00.000-07:00

With the demise of SXDE I thought I'd prepare for life with OpenSolaris...

Currently on build 90, 2008.05 represents build 86.

Installation of OpenSolaris 2008.05 from the Live CD (1 CD) is a breeze, simply boot from the CD (this took some time under VMWare!), have a play then click the install link on the desktop - great!

OS2008.05 uses ZFS and IPS - so this is a good starting point and upgradable using a package management system rather like we've all come to expect from the Linux world :)

Once installed, you can use pkg image-update to udate to the latest build of OpenSolaris, here are some quick notes:

set a timeout to avoid disapointment during the udate process!

#export PKG_CLIENT_TIMEOUT=2000

#pkg refresh

#pfexec pkg image-update

# important - do this before reboot! (or your reboot won't [boot])

http://mail.opensolaris.org/pipermail/indiana-discuss/attachments/20080606/94969481/attachment-0003.txt

Which boils down to:

#beadm list

Check the active on reboot name, e.g. opensolaris-1

#pfexec mount -F zfs rpool/ROOT/opensolaris-1 /mnt

#pfexec /mnt/boot/solaris/bin/update_grub -R /mnt

A guide for OpenSolaris installation under vmware:

http://blogs.sun.com/souvik/entry/getting_started_with_opensolaris_2008

Another good thread: http://opensolaris.org/jive/thread.jspa?threadID=62982&tstart=0

--

Install Java JDK and Netbeans:

http://blogs.sun.com/souvik/entry/getting_started_with_opensolaris_2008

#zcat jdk-6u6-solaris-i586.tar.Z | tar -xf -

#pfexec pkgadd -d . SUNWj6dev SUNWj6cfg SUNWj6man SUNWj6dmo

NB: after installing the 32bit version the 64bit extensions can be installed but don't do this if you want to use applets or web start.

Install FireFox 3:

http://opensolaris.org/jive/thread.jspa?messageID=250311&tstart=0

Installing mysql etc:

#pfexec pkg install SUNWapch22 SUNWmysql5 SUNWapch22m-php5 SUNWphp524-mysql

http://blogs.sun.com/natarajan/entry/how_to_install_apache_php

--

Check platform 32/64 bit:

#isainfo -b

Integrating REST POX web services and .NET WCF

2008-06-02T06:40:00.001-07:00

Notes:

WCF 3.5 supports REST POX services using the web service model.

WebHttpBinding supports cookies for session support etc, e.g:

WebHttpBinding webHttpBinding = new WebHttpBinding();
webHttpBinding.AllowCookies = true;

using (ChannelFactory cf = new ChannelFactory(webHttpBinding, hostPath))
{ ...

Issues:

WebInvoke method POST: post data sent only as JSON or XML - no support for simple form encoded parameters.
General inflexibility when mixing types of data or overloading services make it harder to leverage existing services that support existing AJAX/rich Internet systems.

Blogs, articles and useful resources:

http://bitworking.org/news/125/REST-and-WS

HTTP Programming with WCF and the .NET Framework 3.5 - http://msdn.microsoft.com/en-us/magazine/cc135976.aspx

http://msdn.microsoft.com/en-us/library/system.servicemodel.aspx

http://hyperthink.net/blog/2008/01/18/WCF+Web+Programming+Model+Documentation.aspx

http://msdn.microsoft.com/en-us/library/bb412176.aspx

Rick Strahl's excellent articles:

REST-Based Ajax Services with WCF in .NET 3.5 http://www.code-magazine.com/Article.aspx?quickid=080014

WCF REST Configuration for ASP.net AJAX and plain REST Services http://www.west-wind.com/WebLog/posts/310747.aspx

jQuery AJAX calls to a WCF REST Service http://www.west-wind.com/weblog/posts/324917.aspx

How to consume REST services with WCF - http://blogs.msdn.com/pedram/archive/2008/04/21/how-to-consume-rest-services-with-wcf.aspx

Simple example with DataContract: http://dev.aol.com/node/595

A set of WCF tutorials:

http://dotnet.org.za/hiltong/pages/Windows-Communication-Foundation-_2800_Indigo_2900_-Hello-World-Tutorial.aspx

Interesting problem with XML serialiser quota that you could run into at some point:

http://www.timrayburn.net/2007/10/02/XmlSerializerFormat+Plus+Huge+Schema+Equals+Trouble.aspx

REST WADL - tools and information

2008-05-21T02:21:00.000-07:00

I'm interested to use REST or REST-like web services within a traditional thick client (as well as AJAX web apps) - using WADL to describe them (from existing services) and tools to automatically generate client API code (in a variety of languages).

Standards:

JAX-RS (JSR 311)

Digging around, I come across some interesting articles and blogs, including:

REST Describe and REST Compile:

Thomas Steiner's blog http://blog.tomayac.de/index.php?date=2007-03-12&time=17:11:27&perma=Automatic+Multi+Lang.html describing two tools to be developed.

The REST describe and REST compile project report http://docs.google.com/View?docid=dgdcn6h3_38fz2vn5
A working on-line example of REST describe http://tomayac.de/rest-describe/latest/RestDescribe.html
REST compile/describe screen cast http://youtube.com/user/tomayac
Tech talk video on the tools http://www.youtube.com/watch?v=hZ2EtAEBpq0

Java.Net resources:

Glassfish - Jersey - Java.Net WADL project https://jersey.dev.java.net/what-is-jersey.html
The wadl2java tool https://wadl.dev.java.net/

Other resources and blogs:

http://blogs.msdn.com/dotnetinterop/archive/2008/02/06/wadl-and-wsdl-and-rest-oh-my.aspx

An article on REST WADL http://searchsoa.techtarget.com/tip/0,289483,sid26_gci1265367,00.html

Blog by William Martinez Pomares with interesting follow up discussion http://acoscomp.com/wblog//index.php/a/2007/06/10/wadl_rest_and_wsdl

REST Describe and REST Compile tools:

Trying the on-line REST Describe on an existing web service I'd published previously showed promising results.

The current status of REST Compile is that is supports:

Java
PHP5
Ruby
Python

but .Net is not provided as yet.

DNS troubles on Solaris

2008-05-07T01:37:00.000-07:00

After having a little DNS trouble on Solaris, here are some useful links for future reference:

http://www.sun.com/bigadmin/content/selfheal/smf-quickstart.jsp

http://www.learning-solaris.com/index.php/configuring-a-dns-server/

http://www.softpanorama.org/DNS/dns_clients.shtml

Solaris 10, X fails to start after installing VMWare tools

2008-05-02T05:11:00.000-07:00

Note to self - if this happens restore the pre-installation vmware_drv.so as per this link:

http://communities.vmware.com/thread/93366

and this one too http://knowledgefrontier.blogspot.com/2008/04/opensolaris-snv86-on-vmware-with.html

which is essentially to do this:

mv /usr/X11/lib/modules/drivers/vmware_drv.so.BeforeVMwareToolsInstall /usr/X11/lib/modules/drivers/vmware_drv.so

luckily vmware keeps the old copy, which is handy....

this is dead easy when you know it. When it only shows up as a problem after a reboot, and the time between installing the tools and rebooting is days or weeks with many other changes in-between - working out the cause and effect relationship is not always so obvious :)

Blastwave for easy Apache http 2.2 and mod_jk, on Solaris SXDE 10

2008-05-01T13:02:00.001-07:00

Goal: Install mod_jk for Apache http 2.2.x with minimal effort!

Such that Apache http can be tested as a front for Glassfish 2ur2 (rather than the usual Tomcat) using the same mod_jk connector...

A handy binary of mod_jk for Solaris and the right version of Apache http didn't seem readily available.

Here are some quick notes on Solaris, Blastwave, Apache httpd and mod_jk

Decided to try out the Blastwave software repository for CSW, site is here http://www.blastwave.org/

Rough steps as follows:

Follow http://www.blastwave.org/howto.html

which is essentially:

pkgadd -d http://www.blastwave.org/pkg_get.pkg
/opt/csw/bin/pkg-get -i wget

/opt/csw/bin/pkg-get -i apache2
/opt/csw/bin/pkg-get -i ap2_mod_jk

and optionally:

cp -p /var/pkg-get/admin-fullauto /var/pkg-get/admin

check the exact version installed:

/opt/csw/apache2/sbin/httpd -v

stop existing service installed as part of SXDE:

svcadm disable apache22
svcs apache22

check lib dsos:

/opt/csw/apache2/libexec

start the server manually:

/opt/csw/apache2/sbin/apachectl start
/opt/csw/apache2/sbin/apachectl stop

Todo:

Configure Apache http and mod_jk
Configure Glassfish with libs etc
Test web applications including failure conditions
Set Blastwave installed version as a service

Some quick notes on Db Connection pooling in Glassfish

2008-04-23T01:22:00.000-07:00

To set up a Glassfish connection pooling and JNDI datasource, accessible in your web app, the following things need to be done (tried with Glassfish v2):

Db drivers:

Ensure the relevant JDBC driver is in the Glassfish classpath. I ensured the <glassfish_home>/lib directory contained the latest MySQL J connector. You may need to restart Glassfish (you probably will if you change the classpath).

Configuration:

Connection Pool:

Set up a connection pool in the Glassfish admin interface (under Resources\JDBC\Connection Pools). In my case for MySQL I was happy to use the defaults initially, which includes:

Datasource classname: com.mysql.jdbc.jdbc2.optional.MysqlDataSource
Resource Type: javax.sql.Datasource

Give the pool a sensible name.

Once you have set this and the connection details (host, username, password etc) you can use the Ping feature to check the connectivity. If you get errors regarding classes not found, check your classpath / location of the JDBC library jars - possibly restart Glassfish if you have not already.

Once you have a successful ping, you can move on to configuring the JNDI resource:

JNDI resource:

Under the Resources\JDBC\JDBC Resources set up a JNDI resource to the pool you just configured (by name), give the JNDI resource a name of the form:

jdbc/myLovelyDbJNDIResourceName

ensure the resource is enabled!

Deployment descriptors, configuration:

sun-web.xml - Map the container JNDI to a resource reference in sun-web.xml, e.g. something like:


<resource-ref>
<res-ref-name>myDbResourceName</res-ref-name>
<jndi-name>jdbc/myLovelyDbJNDIResourceName</jndi-name>
</resource-ref>

web.xml - Map the resource as a datasource reference in web.xml


<resource-ref>
<res-ref-name>myDbResourceName</res-ref-name>
<res-type>javax.sql.DataSource</res-type>
<res-auth>Container</res-auth>
<res-sharing-scope>Shareable</res-sharing-scope>
</resource-ref>

Application Code:

In you application code, where you get a connection, you can then use something along the lines of:



Connection connection = null;

try
{
Context env = (Context)new InitialContext().lookup("java:comp/env");

DataSource pool = (DataSource)env.lookup("myDbResourceName");

if (pool == null)
{
 throw new RuntimeException("Unknown DataSource");
}

connection = pool.getConnection();
}
catch(NamingException ne)
{
throw new RuntimeException(ne.getMessage());
}

return connection;

Of course, modify the caught, declared to throw exceptions to suit the application design...

References:

This article is a useful reference point, covering similar information in a slightly more verbose form:

http://thestewscope.wordpress.com/2008/01/08/glassfish-v2-ur1-and-mysql-connection-pool/

This forum post also details the deployment descriptor configuration well:
http://forums.java.net/jive/message.jspa?messageID=264629

Mobile development with .NET CF

2008-04-22T12:47:00.000-07:00

It's been a few days since the last posting, I've been meaning to jot down some notes from the workshop at Microsoft (Reading Uk, 16-April-2008) before anything gets forgotten:

The aim of the workshop was to discuss the use of the Microsoft .Net Compact framework for mobile application development - including performance issues and technology features. This was in the context of both our existing Java mobile applications and choosing suitable technologies for new projects.

Java ME J9 6.1.1 performance evaluation:

To illustrate the performance we'd experienced with out Java ME applications, we demoed our existing Java ME application running on IBM J9 CDC ppro1.1 on an IPAQ. The response was actually very positive. It was suggested that re-implementing it in .Net would not yield any worth while performance gains but it would most likely create a different set of problems to solve.

We ran our app with JIT enabled, noverify and nolinenumbers but it was not compiled as a JXE on this occasion. Focus was on running application performance as perceived through the GUI (user perceptions), we did not delve into profiling information. We did not look into start up times, which are roughly similar for both Java and .Net VM based technologies.

.Net CF:

.Net CF does provide some benefits, in terms of integration with the Windows operating system (integration with the Windows menu bars etc) that would be worth bearing in mind for future projects. Specifically, notification of docking events could prove very useful in some situations. Native code is supported, as external libraries (not within the assembly) but hooks into the OS API are possible by declaring them within the .Net assembly.

Versions / dependencies:

For .Net development some care regarding versions and dependencies should be considered, for example whilst versions of Windows Mobile from 5.0 onwards _should_ support any version of .Net CF, the same can not be guaranteed for the more variable CE, where platform vendors could load a CE image missing key components required for installing .Net CF. In this scenario it would be important to discover this and engage with the customer and platform vendor to negotiate shipment of hardware with the right image components installed.

.Net CF versions:

1.0 -> Old original version, no longer targeted by most development
2.0 -> Good baseline with significant improvements (including performance) over 1.0 (required MS VS 2005 IDE to develop).
3.5 -> Current latest version (required MS VS 2008 Pro IDE to develop).

Windows Mobile versions against their CE base:

WM 2003 'Ozone' -> CE 4.20
WM 5.0 'Magneto' -> based on CE 5 (shipped with .NET v1.0 SP2)
WM 6 .0 'Crossbow' -> based on CE 5.2 (shipped with .NET CF v2.0 SP2 )
WM 6.1 -> CE 5.x ? TBA
WM 7.0 'Photon' -> CE 6.0? TBD

The next major version of Windows Mobile is anticipated in 2009, but no official details were given. General information suggests a planned release of Windows Mobile 7 in the 2nd half of 2009.

CE 6.0 removes the 32 process and 32Mb restrictions:

The 32 process limit raised to 32,768 processes.
The 32 megabyte VM limit raised up to 2GB of private VM is available per process.

Technologies:

Microsoft SQL Server 2005 Compact Edition is available (shipped with WM 6.0). This can be used stand alone, or connected to a server side SQL server. Can be synchronised with server DB over network. Async and synch operation is possible. Loads in process as a dll (therefore doesn't use any precious process slots, but of course it will consume RAM/processor). This is a very cut down DB, things such as triggers and stored procedures are not supported.

Ajax, DOM, Javascript is possible in the browser technology (from WM 6.0 onwards).

XML is supported - recommended to use XML readers not full blown XML DOM for efficiency reasons.

XSLT - it was not clear weather XSLT was supported on the platform, either in the browser or as .Net libraries.

Windows Mobile limitations (until WM 7.0):

32 process limit
32Mb memory limit

Windows Mobile can close applications when it starts to run low on resources - so applications should be developed with this in mind. The OS will pick the lowest application in the Z-order that is inactive. Notification events are sent but if the application doesn't respond gracefully it could receive hard termination!

.Net development tools:

Visual studio is the only supported option (no free express version for CF 3.5 available at this time).

Optimisation tools were not generally recommended - they advocate letting it run 'as-is' but rather to concentrate efforts on getting the design right.

It was admitted that profiling tools were somewhat weak at the moment - but this is an area they are actively addressing.

Code signing:

We were warned that some platform vendors have removed the root CA from the device, which means that the only option for getting the application code signed would be to go through the device vendor (this is something that would need to be highlighted and planed in early in the project).

Testing:

The need for thorough and extended testing was highlighted. Mainly due to the additional use cases that can occur on a mobile platform that are less likely or non-existent on a traditional desktop pc system, for example:

Memory cards being removed
Wireless signal poor / dropped
Flat battery, battery removal
Unexpected key presses from special function keys on the device

.Net configuration:

It's possible to configure the platform .Net CF using a configuration file:

http://msdn2.microsoft.com/en-us/library/bb629366.aspx

Software updates:

Whilst there is no application support for software updates (apart from the M ) there are some software components that can be used as a basis for providing this, the Mobile Client Software Factory and the Mobile Updater.

http://www.microsoft.com/downloads/details.aspx?FamilyID=F9176708-9F57-4C0F-97FB-F9C65A9BBF22&displaylang=en

http://msdn2.microsoft.com/en-us/library/bb629366.aspx

Additional links and info:

Mobile Dev centre http://msdn2.microsoft.com/en-gb/windowsmobile/default.aspx

Application developer consulting service http://www.microsoft.com/uk/adc

Libraries and widgets:

Mobility Marvels: http://msdn2.microsoft.com/en-us/windowsmobile/bb250537.aspx

Mobile Controls: http://www.asp.net/mobile/

Xephyr on Solaris

2008-04-15T13:00:00.001-07:00

Searching for a better Xnest on Solaris, looking for Xephyr as a package or source to build, it turns out Xephyr is already included in SXDE.

location: /usr/X11/bin

simple usage example:

Xephyr :1 -screen 1200x800 -query localhost

This what's new in SXDE 10 guide is quite handy http://dlc.sun.com/pdf/817-0547/817-0547.pdf

Early signs of Xephyr -> ssh -> Xming on Windows seem reasonable.

ProGuard - new 4.2 version:

Evaluated the latest ProGuard 4.2 for our Java ME mobile client and it failed reporting a stack length had gone negative! Have reverted back to 4.1 for now, which seems to be working well. Will report the bug to the team.

Java ME, some J9 options worth remembering:

-Xint -> disable AOT and JIT, interpreted mode only
-noverify -> disable class verification checks (faster loaded on pre-verified code)
-Xnolinenumbers -> turn off line number debug info

.NET CF

2008-04-14T12:24:00.001-07:00

Some quick prep /initial prep / background reading for the workshop with Microsoft at their Reading Campus.

Latest version is currently 3.5 (3.5 RTM 3.5.7283.0 - 2007-11-19)

Overview of 3.5 features http://searchwindevelopment.techtarget.com/news/article/0,289142,sid8_gci1285069,00.html

You can determine the installed CF versions by running /Windows/cgacutil on the device (several versions can be co-installed without issue).

CF 3.5 goes hand in hand with Visual Studio 2008 (whereas the older 2.0 was paired with VS 2005).