Showing posts with label Scala. Show all posts
Showing posts with label Scala. Show all posts

Sunday, 1 November 2009

Project Euler 12 - Scala Streams, creating triangle numbers

Just a short blog today on Scala Streams and generating numeric progressions, such as a simple arithmetic series

This is a solution to project Euler problem 12: "What is the value of the first triangle number to have over five hundred divisors?"

This is actually a fairly easy problem to solve, providing you use a sufficiently efficient function to count the divisors!

The method I've adopted for this is particularly simple, but performs sufficiently well to solve this problem as stated within a few seconds.

Triangle Numbers:

Just a quick refresher, the triangle numbers are the sum of integers from 1 to n.

An Arithmetic series is a number series of the form:

a + (a + b) + (a + 2b) + ....

and using this form the Triangle numbers can be written as:

1 + (1 + 1) + (1 + 2) + (1 + 3) + ...

so they clearly are of the form of an Arithmetic series.


The more interesting aspect of this problem was the generation of the Triangle number sequence. Naturally, wanting to use Scala to solve this problem, the idea of using a Stream to evaluate the triangle number sequence lazily on demand is a natural approach.

The implementation I settled on to produce the Stream of triangle numbers was the iterative version as follows:


val trianglesI = Stream.iterate((1L,0L)){
case (n, sum) => (n + 1L, sum + n) }.map(_._2)


Which is a fairly natural way to solve via the additive definition of the triangle numbers.

What's perhaps more interesting is finding some alternative ways to achieve the same thing, here are some alternatives to the iterative additive stream definition:

Using the multiplicative definition of the Triangle numbers and the map function:



// use a stream map and the multiplicative solution for triangle numbers
val trianglesM = Stream.from(1).map(n => n * (n + 1) / 2)

println("trianglesM take(5)")
trianglesM.take(5).foreach(println)



Using "inits" - inits are a concept that can be found in languages such as Haskell:


// using inits (not efficient)
// inits -> inits(Stream.from(1)) is Stream(Stream(), Stream(1), Stream(1, 2), Stream(1, 2, 3), ...)
// and tails -> tails(Stream(1)) = Stream(Stream(1,2,3,...), Stream(2,3,4,...), Stream(3,4,5,...), ...)
def inits[T](xs: Stream[T]) = Stream.from(0).map(xs.take(_))

val trianglesInits = inits(Stream.from(1)).tail.map(_.sum)

println("trianglesInits take(5)")
trianglesInits.take(5).foreach(println)



Using Stream.cons and a helper function zipWith that can zip tuples:


// using zipWith
def zipWith[T1,T2,T3](s1: Stream[T1], s2: Stream[T2])(fn: (T1,T2) => T3) =
s1.zip(s2).map(Function.tupled(fn))

lazy val trianglesZ: Stream[Int] = Stream.cons(1, zipWith(trianglesZ,
Stream.from(2))(_ + _))

println("trianglesZ take(5)")
trianglesZ.take(5).foreach(println)




So, whilst problem 12 is not too hard, the production of triangle numbers as lazy lists is quite interesting, being that there are several approaches that can be taken!


The whole program solution to problem 12 (and the alternative triangle number streams)


package projecteuler

object Problem12 {

def main(args : Array[String]) {

println("result = " + problem12(500))
}

def problem12(target : Int) {

// using Stream.iterate to create a lazy stream of triangle numbers
val trianglesI = Stream.iterate((1L,0L)){
case (n, sum) => (n + 1L, sum + n) }.map(_._2)

// test it
//println("trianglesI take(5)")
//trianglesI.take(5).foreach(println)

// count the divisors (not massively efficient, but sufficient)
def numDivisors(n : Long) = {

var count = 0

for (i <- 1L to Math.sqrt(n).toInt) {
if (n % i == 0) {
count = count + 2
}
}
count
}

trianglesI.find(p => numDivisors(p) > target)
}

def otherTriangleStreams {

// use a stream map and the multiplicative solution for triangle numbers
val trianglesM = Stream.from(1).map(n => n * (n + 1) / 2)

println("trianglesM take(5)")
trianglesM.take(5).foreach(println)

// using inits (not efficient)
// inits -> inits(Stream.from(1)) is Stream(Stream(), Stream(1), Stream(1, 2), Stream(1, 2, 3), ...)
// tails -> tails(Stream(1)) = Stream(Stream(1,2,3,...), Stream(2,3,4,...), Stream(3,4,5,...), ...)
def inits[T](xs: Stream[T]) = Stream.from(0).map(xs.take(_))

val trianglesInits = inits(Stream.from(1)).tail.map(_.sum)

println("trianglesInits take(5)")
trianglesInits.take(5).foreach(println)

// using zipWith
def zipWith[T1,T2,T3](s1: Stream[T1], s2: Stream[T2])(fn: (T1,T2) => T3) = s1.zip(s2).map(Function.tupled(fn))

lazy val trianglesZ: Stream[Int] = Stream.cons(1, zipWith(trianglesZ, Stream.from(2))(_ + _))

println("trianglesZ take(5)")
trianglesZ.take(5).foreach(println)
}
}



.

Monday, 31 August 2009

Scala, trees, pattern matching and tree recursion

Hi! Long time no see...

Well after some outage due to Holiday (road trip around Europe and down to the Swiss Alps!), work commitments and some personal things to deal with, I'm back again.

I thought I'd start with some stuff on Tree structures, whilst continuing on with Scala too. The following code illustrates some good examples of case classes and pattern matching, along with extensive use of Some, None and Option (Scala's improved way of dealing with optional types, rather than relying on null everywhere).

The following code defines a simple recursive binary tree definition as a case class, using option to define the left and right children.

case class Tree[+T](value: T, left: Option[Tree[T]], right: Option[Tree[T]])
The above definition defines a tree recursively, exactly as a tree is defined as a data structure in computer science. Each tree node has a value of type T and an Optional left and right child node (bin tree). The +T makes the Tree type covarient about T, which is a generally useful property of a Tree.

A few helper functions are used that make life a little easier when working with Option types, by extracting the value or providing a safe default when no value exists. These help keep the code a little tidy and manageable.


/**
* Helper function: For o return f: A => Int if Some or 0 if None
* also could try opt map f getOrElse 0
*/
def zeroIfNone[A](o: Option[A], f: A => Int): Int = o match {
case None => 0
case Some(x) => f(x)
}

/**
* Helper function: Return value of function provided if Some, or 0 otherwise
*/
def zeroIfNoneVal[A](o: Option[A], n: => Int) = zeroIfNone[A](o, _ => n)

/**
* Helper function: Default to provided value if option is None
*/
def defaultIfNone[A](o: Option[A], f: A => Int, d : Int): Int = o match {
case None => d
case Some(x) => f(x)
}


The example code demonstrates preorder, inorder and postorder recursive tree traversal (which are all simple permutations of a basic depth first search), applying the provided unit function to each node as it is visited.

Recall that the preorder, inorder and postorder functions are defined as follows:

  • PreOrder - visit parent node, left and then right child node
  • InOrder - visit left, parent and then right nodes
  • PostOrder - visit left, right and then parent node


Using recursion and the call stack, depth first search based tree traversal is very simple and concise. The basic DFS inorder traversal looks like:


/**
* Perform an inorder tree traversal using the provided Tree[A] => Unit function
*/
def inOrder[A](t: Option[Tree[A]], f: Tree[A] => Unit) : Unit = t match {

case None =>
case Some(x) =>
if (x.left != None) inOrder(x.left, f)
f(x)
if (x.right != None) inOrder(x.right, f)
}


The breadth first search algorithm is perhaps more interesting. This uses a List as a basic FIFO queue to provide the breadth first search tree traversal:


/**
* Breadth first search on a tree, a list is used as a basic fifo queue to
* give the bfs traversal
*/
def breadthFirstSearch[A](t: Tree[A], f: Tree[A] => Unit): Unit = {
def bfs(ts: List[Tree[A]]): Unit = ts match {
case Nil => // ignore
case _ =>
val children = for{tree <- ts
Some(child) <- List(tree.left, tree.right)}
yield child
ts map f
bfs(children)
}
bfs(List(t))
}


The example also shows a treeFold function. Whether this is a useful function as-is is perhaps debatable, but it makes for an interesting example. The code traverses the tree in a DFS inorder traversal and applies the mapping function f : A -> B and then it applies the join function j : B, B -> B to combine the results of the tree map. In this way the function is similar to say flatMap where a list is mapped and then flattened into a single list, or perhaps more closely to foldLeft/foldRight where items in a list a joined into a single resulting value.



/**
* Fold up a tree into a single value, using provided map and join functions
*/
def treeFold[A, B](t: Option[Tree[A]],
f: Option[Tree[A]] => Option[B],
j: (Option[B], Option[B]) => Option[B]) : Option[B] = t match {

case None => None
case Some(x) => j(j(f(t), treeFold(x.left, f, j)), treeFold(x.right, f, j))
}


Some utility functions are also defined, to return the size and depth of the tree for illustration of how this can be achieved.

In summary Scala can quite elegantly represent Tree data structures, using case classes and operate over them using pattern matching and higher order functions to provide powerful and concise support for trees and the operations that can be performed on them.

Some additional things that could be done here would be to provide an example of a binary search and perhaps insertion, update and delete too. A job for another followup blog perhaps...


The full example code (is complete and self contained) is as follows:


package trees

// Define a recursive Bin Tree structure
case class Tree[+T](value: T, left: Option[Tree[T]], right: Option[Tree[T]])

object TreeObj {

/**
* Helper function: For o return f: A => Int if Some or 0 if None
* also could try opt map f getOrElse 0
*/
def zeroIfNone[A](o: Option[A], f: A => Int): Int = o match {
case None => 0
case Some(x) => f(x)
}

/**
* Helper function: Return value of function provided if Some, or 0 otherwise
*/
def zeroIfNoneVal[A](o: Option[A], n: => Int) = zeroIfNone[A](o, _ => n)

/**
* Helper function: Default to provided value if option is None
*/
def defaultIfNone[A](o: Option[A], f: A => Int, d : Int): Int = o match {
case None => d
case Some(x) => f(x)
}

/**
* Return the size of the tree
*/
def size[A](t : Option[Tree[A]]) : Int = t match {

case None => 0
case Some(x) =>
zeroIfNoneVal(x.left, size(x.left)) + zeroIfNoneVal(x.right, size(x.right)) + 1
//zeroIfNone(x.left, (_:Tree[Any]) => (size(x.left))) +
//zeroIfNone(x.right, (_:Tree[Any]) => (size(x.right))) + 1
}

/**
* Alternative size using nested function
*/
def size2[A](t : Option[Tree[A]]) : Int = {

def sizeB(b : Option[Tree[A]]) : Int = b match {
case None => 0
case Some(x) => size(b);
}

t match {
case None => 0
case Some(x) =>
sizeB(x.left) + sizeB(x.right) + 1
}
}

/**
* Return the max depth of the tree
*/
def depth[A](t : Option[Tree[A]]) : Int = t match {

case None => 0
case Some(x) =>
Math.max(zeroIfNoneVal(x.left, depth(x.left)), zeroIfNoneVal(x.right, depth(x.right))) + 1
}

/**
* Alternative deptch, return the max depth of the tree using nested function
*/
def depth2[A](t : Option[Tree[A]]) : Int = {

def depthB(b : Option[Tree[A]]) : Int = b match {
case None => 0
case Some(x) => depth(b);
}

t match {

case None => 0
case Some(x) =>
Math.max(depthB(x.left), depthB(x.right)) + 1
}
}

/**
* Breadth first search on a tree, a list is used as a basic fifo queue to
* give the bfs traversal
*/
def breadthFirstSearch[A](t: Tree[A], f: Tree[A] => Unit): Unit = {
def bfs(ts: List[Tree[A]]): Unit = ts match {
case Nil => // ignore
case _ =>
val children = for{tree <- ts Some(child) <- List(tree.left, tree.right)} yield child ts map f bfs(children) } bfs(List(t)) } /** * Sum up the elements of a numeric tree */ def sumTree[A <: Int](t: Option[Tree[A]]) : Int = t match { case None => 0
case Some(x) =>
sumTree(x.left) + sumTree(x.right) + x.value
}

/**
* Fold up a tree into a single value, using provided map and join functions
*/
def treeFold[A, B](t: Option[Tree[A]],
f: Option[Tree[A]] => Option[B],
j: (Option[B], Option[B]) => Option[B]) : Option[B] = t match {

case None => None
case Some(x) => j(j(f(t), treeFold(x.left, f, j)), treeFold(x.right, f, j))
}

/**
* Perform an inorder tree traversal using the provided Tree[A] => Unit function
*/
def inOrder[A](t: Option[Tree[A]], f: Tree[A] => Unit) : Unit = t match {

case None =>
case Some(x) =>
if (x.left != None) inOrder(x.left, f)
f(x)
if (x.right != None) inOrder(x.right, f)
}

/**
* Perform a preorder tree traversal using the provided Tree[A] => Unit functio
*/
def preOrder[A](t: Option[Tree[A]], f: Tree[A] => Unit) : Unit = t match {

case None =>
case Some(x) =>
f(x)
if (x.left != None) inOrder(x.left, f)
if (x.right != None) inOrder(x.right, f)
}

/**
* Perform a postorder tree traversal using the provided Tree[A] => Unit functio
*/
def postOrder[A](t: Option[Tree[A]], f: Tree[A] => Unit) : Unit = t match {

case None =>
case Some(x) =>
if (x.left != None) inOrder(x.left, f)
if (x.right != None) inOrder(x.right, f)
f(x)
}

/**
* build tree and run tree functions on it
*/
def main(args : Array[String]) {

/* Test Tree structure

1
/ \
2 3
/ \ / \
4 x 5 6

*/

// build a tree
val myTree = Tree(1,
Some(Tree(2,
Some(Tree(4, None, None)),
None
)
),
Some(Tree(3,
Some(Tree(5, None, None)),
Some(Tree(6, None, None))
)
)
)

printBfsTree(myTree)

println("Size = " + size(Some(myTree)))

println("Depth = " + depth(Some(myTree)))

println("Sum = " + sumTree(Some(myTree)))

var sum = treeFold[Int, Int](
Some(myTree),
(t: Option[Tree[Int]]) => t match { case None => Some(0); case Some(x) => Some(x.value) },
(x: Option[Int], y: Option[Int]) =>
Some(zeroIfNone[Int](x, (x:Int) => x) + zeroIfNone[Int](y, y => y))
)

var mult = treeFold[Int, Int](
Some(myTree),
(t: Option[Tree[Int]]) => t match { case None => Some(0); case Some(x) => Some(x.value) },
(x: Option[Int], y: Option[Int]) =>
Some(defaultIfNone[Int](x, (x:Int) => x, 1) * defaultIfNone[Int](y, y => y, 1))
)

println("Sum Result = " + sum)
println("Mult Result = " + mult)

println("InOrder = ")
inOrder(Some(myTree), (t: Tree[Any]) => println(t.value))

println("PreOrder = ")
preOrder(Some(myTree), (t: Tree[Any]) => println(t.value))

println("PostOrder = ")
postOrder(Some(myTree), (t: Tree[Any]) => println(t.value))
}

/**
* Print a tree using BFS, passing in print unit function
*/
def printBfsTree[A](tree: Tree[A]) {

breadthFirstSearch(tree, (t: Tree[A]) => println(t.value))
}
}


.

Sunday, 5 July 2009

Scala, the Google App Engine and an iPhone client...

In this Blog I wanted to explore Scala on the Google App Engine. The Google App Engine has supported Java (as well as Python) for a little while now. There have been a few blogs about running Scala on the App engine and I wanted to further demonstrate this by taking a simple Java Servlet and converting it to a Scala based web application suitable for the App Engine.

The Google App Engine home is here http://code.google.com/appengine/ and contains excellent documentation on how to get stated. I don't want to repeat the documentation, so I'll just point out the essentials relevant to this blog.

The Getting Started guide can be found here: http://code.google.com/appengine/docs/java/gettingstarted/ for Java, and a Python equivalent is available too.

This example is based on the previous blog, the Sieve of Eratosthenes as a simple prime number generator. The request accepts one parameter n whose integer value is the upper limit to generate all the primes until (from 2). The response output is JSON encoded to create a simple web service.


Starting with the very basics, the recommended project directory structure is of the form:


|-src
|---META-INF
|---primes
|-war
|---WEB-INF
|-----classes
|-------META-INF
|-------primes
|-----lib



For any Web App for the Google App Engine, one additional GAE specific file is required alongside the standard web.xml deployment descriptor. This file is called appengine-web.xml and contains the name of the Application, for example:


<appengine-web-app xmlns="http://appengine.google.com/ns/1.0">
<application>primes-chillipower-com</application>
<version>1</version>
</appengine-web-app>


In this simple GAE deployment descriptor the main thing is that the name of the application is specified, in this example the application was called 'primes-chillipower-com'.

Applications must be created on the App Engine, this can be done here: http://appengine.google.com/

Note that there are some limitations currently, you are allowed up to a maximum of 10 applications and applications can not be deleted once created!


The simple Java version:

In the simple Java version we can place a simple Servlet in the src directory and build using ant.


Example Java servlet:


package primes;

import java.io.IOException;
import javax.servlet.http.*;

public class PrimesServletJ extends HttpServlet {
public void doGet(HttpServletRequest req, HttpServletResponse resp)
throws IOException {
resp.setContentType("text/plain");

int n = 100;

String nStr = req.getParameter("n");

try {

n = Integer.parseInt(nStr);
}
catch (Exception e) {}

int[] primes = primes(n);

resp.getWriter().println("{ \"n\" : \"" + n + "\",");
resp.getWriter().println(" \"primes\" : [ ");

for (int i = 0; i < primes.length; i++) {
String sep = (i < primes.length - 1) ? ", " : "";
resp.getWriter().println(" { \"" + i + "\" : \"" + primes[i] + "\" }" + sep);
}

resp.getWriter().println(" ] }");
}

public static int[] primes(int n) {

// initially assume all integers are prime
boolean[] isPrime = new boolean[n + 1];

for (int i = 2; i <= n; i++) {
isPrime[i] = true;
}

// mark non-primes <= N using Sieve of Eratosthenes
for (int i = 2; i*i <= n; i++) {

// if i is prime, then mark multiples of i as nonprime
// suffices to consider mutiples i, i+1, ..., N/i
if (isPrime[i]) {
for (int j = i; i * j <= n; j++) {
isPrime[i * j] = false;
}
}
}

// count primes
int primeCount = 0;
for (int i = 2; i <= n; i++) {
if (isPrime[i]) primeCount++;
}

System.out.println("The number of primes <= " + n + " is " + primeCount);

int[] primes = new int[primeCount];
int idx = 0;
for (int i = 0; i <= n; i++) {
if (isPrime[i] == true) {

primes[idx] = i;
idx++;
}
}

return primes;
}

public static void main(String[] args) {

int n = 100;

if (args.length > 0 && args[0] != null) {
n = Integer.parseInt(args[0]);
}

int[] primes = primes(n);

for (int p : primes) {
System.out.println("Prime: " + p);
}
}
}



The ant build script goes in the root of the project, to start with I based mine on the example on the Google App Engine documentation here http://code.google.com/appengine/docs/java/tools/ant.html#Uploading_and_Other_AppCfg_Tasks

Using the standard ant script the project can be compiled and even run locally using the SDK.

ant compile

ant runserver


If you start the dev app server locally, it will be accessibly using a URL of the form:

http://localhost:8080/yourappcontextpath


Where the port 8080 is the default and can be changed in the script.

The App Engine SDK comes with appcfg.sh script in the bin directory, which can be used to perform operations such as uploading your app to the App Engine.


../../../../tools/appengine-java-sdk-1.2.1/bin/appcfg.sh update war


The Scala version:

The main tasks in converting this conventional Java Servlet based web app to a Scala based one is to:
  • Rewrite the Java Servlet in Scala.
  • Reconfigure the ant build.xml to modify the classpath to include the Scala libraries and to run the Scala scalac compiler to create the Java .class files from the .scala source files.

Since the Servlet API is a Java based API, there will of course be some reference to Java based classes in the Scala Servlet.


Example Scala based Servlet:


package primes

import javax.servlet.http._


class PrimesServlet extends HttpServlet {
override def doGet(request: HttpServletRequest, response: HttpServletResponse) = {

var n = 100

try {
val nStr = request.getParameter("n");
n = Integer.parseInt(nStr);
} catch {
case e : Exception => {
println("Exception e " + e.getMessage())
}
}

val ps = primes take n
val out = response.getWriter()

out.println("{ \"n\" : \"" + n + "\", \"primes\" : [")

var i = 0
for (p <- ps) {
val sep = if (i < ps.length - 1) ", " else ""
out.println(" { \"" + i + "\" : \"" + p + "\" }" + sep)
i = i + 1
}

out.println(" ] } ")
}

def primes = {
def sieve(is: Stream[Int]): Stream[Int] = {
val h = is.head
Stream.cons(h, sieve(is filter (_ % h > 0)))
}

sieve(Stream.from(2))
}
}





Example modified and build.xml file:


<project>
<property name="scala.home" location="/Users/chillipower_uk/dev/tools/scala/scala-2.7.5.final/" />
<property name="sdk.dir" location="/Users/chillipower_uk/dev/tools/appengine-java-sdk-1.2.1/" />

<import file="${sdk.dir}/config/user/ant-macros.xml" />

<path id="project.classpath">
<pathelement path="war/WEB-INF/classes" />
<fileset dir="war/WEB-INF/lib">
<include name="**/*.jar" />
</fileset>
<fileset dir="${sdk.dir}/lib">
<include name="shared/**/*.jar" />
</fileset>
</path>

<!-- Copy over Scala jars -->
<target name="init">
<property
name="scala-library.jar"
value="${scala.home}/lib/scala-library.jar"
/>
<path id="build.classpath">
<pathelement location="${scala-library.jar}" />

<pathelement location="${build.dir}" />
</path>
<taskdef resource="scala/tools/ant/antlib.xml">
<classpath>
<pathelement location="${scala.home}/lib/scala-compiler.jar" />
<pathelement location="${scala-library.jar}" />
</classpath>
</taskdef>
</target>

<!-- Copy Scala JARs -->
<target name="copyscala"
description="Copies the Scala JARs to the WAR.">
<copy
todir="war/WEB-INF/lib"
flatten="true">
<fileset dir="${scala.home}/lib">
<include name="**/scala-library.jar" />
</fileset>
</copy>
</target>

<!-- Copy App engine jars -->
<target name="copyjars"
description="Copies the App Engine JARs to the WAR.">
<copy
todir="war/WEB-INF/lib"
flatten="true">
<fileset dir="${sdk.dir}/lib/user">
<include name="**/*.jar" />
</fileset>
</copy>
</target>

<target name="compile" depends="copyscala, copyjars, init"
description="Compiles Java source and copies other source files to the WAR.">
<mkdir dir="war/WEB-INF/classes" />
<copy todir="war/WEB-INF/classes">
<fileset dir="src">
<exclude name="**/*.scala" />
<exclude name="**/*.java" />
</fileset>
</copy>
<scalac
srcdir="src"
destdir="war/WEB-INF/classes"
classpathref="project.classpath"
/>

<!--
<copy todir="war/WEB-INF/classes">
<fileset dir="src">
<exclude name="**/*.scala" />
<exclude name="**/*.java" />
</fileset>
</copy> -->
<javac
srcdir="src"
destdir="war/WEB-INF/classes"
classpathref="project.classpath"
debug="on" />

</target>

<target name="datanucleusenhance" depends="compile"
description="Performs JDO enhancement on compiled data classes.">
<enhance_war war="war" />
</target>

<target name="runserver" depends="datanucleusenhance"
description="Starts the development server.">
<dev_appserver war="war" />
</target>

<target name="update" depends="datanucleusenhance"
description="Uploads the application to App Engine.">
<appcfg action="update" war="war" />
</target>

<target name="update_indexes" depends="datanucleusenhance"
description="Uploads just the datastore index configuration to App Engine.">
<appcfg action="update_indexes" war="war" />
</target>

<target name="rollback" depends="datanucleusenhance"
description="Rolls back an interrupted application update.">
<appcfg action="rollback" war="war" />
</target>

<target name="request_logs"
description="Downloads log data from App Engine for the application.">
<appcfg action="request_logs" war="war">
<options>
<arg value="--num_days=5"/>
</options>
<args>
<arg value="logs.txt"/>
</args>
</appcfg>
</target>

</project>




Example request and output:

Request: http://primes-chillipower-com.appspot.com/primes?n=50


{ "n" : "50", "primes" : [ { "0" : "2" }, { "1" : "3" }, { "2" : "5" }, { "3" : "7" }, { "4" : "11" }, { "5" : "13" }, { "6" : "17" }, { "7" : "19" }, { "8" : "23" }, { "9" : "29" }, { "10" : "31" }, { "11" : "37" }, { "12" : "41" }, { "13" : "43" }, { "14" : "47" }, { "15" : "53" }, { "16" : "59" }, { "17" : "61" }, { "18" : "67" }, { "19" : "71" }, { "20" : "73" }, { "21" : "79" }, { "22" : "83" }, { "23" : "89" }, { "24" : "97" }, { "25" : "101" }, { "26" : "103" }, { "27" : "107" }, { "28" : "109" }, { "29" : "113" }, { "30" : "127" }, { "31" : "131" }, { "32" : "137" }, { "33" : "139" }, { "34" : "149" }, { "35" : "151" }, { "36" : "157" }, { "37" : "163" }, { "38" : "167" }, { "39" : "173" }, { "40" : "179" }, { "41" : "181" }, { "42" : "191" }, { "43" : "193" }, { "44" : "197" }, { "45" : "199" }, { "46" : "211" }, { "47" : "223" }, { "48" : "227" }, { "49" : "229" } ] }



The iPhone client:

I'm currently developing using the iPhone 3.1 beta SDK. I'm not going to post all the source code the the simple client, but rather give some essential snippets relevant to the invocation of the simple JSON web service once deployed on the Google App Engine.

To Consume this JSON in an iPhone app, we can use the JSON Objective-C library, which is available on Google code here http://code.google.com/p/json-framework/wiki/FAQ



- (void)loadData {
[super viewDidLoad];

NSString *urlString = [NSString stringWithFormat:@"http://primes-chillipower-com.appspot.com/primes?n=1000"];
NSURL *url = [NSURL URLWithString:urlString];

NSDictionary *primesResponse = (NSDictionary *)[JsonClientHelper fetchJSONValueForURL:url];

if (primesResponse != nil) {

NSArray *primeEntries = [primesResponse objectForKey:@"primes"];

NSMutableArray *primesTemp = [[NSMutableArray alloc] init];
[primesTemp retain];

for (int i = 0; i < [primeEntries count]; i++) {

NSDictionary *primeEntry = (NSDictionary*)[primeEntries objectAtIndex:i];

NSString *key = [NSString stringWithFormat:@"%d", i];

NSString *p = [primeEntry objectForKey:key];

NSString *msg = [NSString stringWithFormat:@"Prime %d = %@", i, p];

NSLog(msg);

[primesTemp addObject:p];
}

primes = [primesTemp copy];

[primesTemp release];
}
}




The above code makes a request to the JSON web service on the App Engine and decodes the response, which once decoded using the JSON library is contained in Dictionaries (NSDictionary) and Arrays (NSArray).


Where JsonClientHelper contains a basic class helper method fetchJSONValueForURL to fetch and decode a JSON response from the supplied URL.



+ (id)fetchJSONValueForURL:(NSURL *)url
{
NSString *jsonString = [[NSString alloc] initWithContentsOfURL:url encoding:NSUTF8StringEncoding error:nil];

id jsonValue = [jsonString JSONValue];

[jsonString release];

return jsonValue;
}

I hope this Blog has helped give some insight into creating a Scala based Web Application for the Google App Engine, as well as the consumption of JSON in an iPhone Objective-C client.

(I appologise for the formatting of the code in this blog, finding it awkward to get the code into the blog without it being mangled in some way!).


Resources:

Handy JSON Lint tool: http://www.jsonlint.com/

Scala Servlet development: http://blogs.oracle.com/aseembajaj/2008/08/scala_servlet_development.html

Scala on the Google App Engine: http://froth-and-java.blogspot.com/2009/04/scala-on-google-appengine.html

The Google App Engine provides support for JPA and JDO based persistence - a comparison of the two can be found here: http://www.jpox.org/docs/persistence_technology.html Further details of the JDO specification can be found here: http://java.sun.com/jdo/index.jsp and the JDO specification here: http://java.sun.com/javaee/technologies/persistence.jsp

Support for the Google App Engine in Netbeans: http://kenai.com/projects/nbappengine/pages/Home


.

Friday, 3 July 2009

Scala, lazy evaluation and the Sieve of Eratosthenes

This blog posting is primarily concerned with looking at lazy evaluation in functional programing, what it means and how it can be put to use.

Lazy evaluation can be described as an expression that has a value, but that is not evaluated until it's actually needed. Conversely strict refers to the opposite property where all expressions are evaluated up front when declared.

Some functional languages are described as Pure Lazy (no, this is not a derogatory term!), to reflect the fact that all evaluation is performed on demand. Haskell is on such language. Scala has both aspects of strict evaluation and lazy evaluation, as such it couldn't be referred to as 'pure' in either sense.

By means of the simplest example I can think of to illustrate Lazy evaluation, consider the following interaction with the Scala CLI:


scala> lazy val a = b + 1; lazy val b = 1;
a: Int = <lazy>
b: Int = <lazy>

scala> a
res9: Int = 2

scala> b
res10: Int = 1


I think this illustrates lazy evaluation, without defining the 'vals' as lazy the statement lazy val a = b + 1; lazy val b = 1; as val a = b + 1; val b = 1; couldn't be evaluated because b would be undefined when evaluating a!

The examples are based on the Sieve of Eratosthenes. Eratosthenes was an ancient Greek Mathmetician from 276 BC – c. 195 BC. The Sieve of Eratosthenes is a ancient algorithm for finding prime numbers from 2 .. n. Stated simply, this algorithm can be expressed as:


  • Create a list of all the numbers from 2 .. n.
  • Starting at p = 2
  • Strike out all multiples of p up until n from the list
  • Let p now be the first (lowest) number that has not been struck-off the list
  • Repeat the last two steps until p^2 > n
  • All remaining numbers not struck from the list are prime

In Haskell this can be expressed very elegantly and succinctly, as follows:

primes = sieve [2..]
sieve (p : xs) = p : sieve [x | x <− xs, x `mod` p > 0]


Note that [2..] creates an infinite steam of all the integers from 2 to infinity. You might think this is impossible and would create an out of memory error on the machine it ran on, but this is perfectly valid in a pure lazy language because none of the numbers are actually generated until they are requested!

In Scala there's not really a way to represent this quite so succinctly, but it is possibly to create a Lazy stream of numbers using:

var is = Stream from 2

which is essentially equivalent to Haskell's:

[2..]

Syntax.


Another important thing to note and be aware of is that Scala's List class is not a lazy list. Therefore, trying to make the equivalent of the Haskell algorithm in Scala using a List is not going to work.

Scala's Stream class however is Lazy, and so it can form the basic of the equivalent algorithm in Scala. To be specific, Scala's Stream class is strict about head and lazy about tail - which is ok since head contains a finite set of 0|1 elements.


The following Scala code shows two ways to implement the Sieve using Scala Lazy streams.


Example 1:


package primes;

/* Haskell...

primes = sieve [2..]
sieve (p : xs) = p : sieve [x | x <− xs, x `mod` p > 0]
*/
object Primes1 {

def primes : Stream[Int] = {

var is = Stream from 2

def sieve(numbers: Stream[Int]): Stream[Int] = {
Stream.cons(
numbers.head,
sieve(for (x <- numbers.tail if x % numbers.head > 0) yield x))
}

sieve(is)
}

def main(args : Array[String]) = {

primes take 100 foreach println
}
}

object Primes2 {
def primes = {
def sieve(is: Stream[Int]): Stream[Int] = {
val h = is.head
Stream.cons(h, sieve(is filter (_ % h > 0)))
}

sieve(Stream.from(2))
}

def main(args: Array[String]) {
println(primes take 100 toList)
}
}



Note the use of the 'take' method to extract the first 100 primes from the stream (so the program terminates!).


This second example shows how we can define a Lazy list like trait and then use it to implement the same, equivalent behavior:


Example 2:


package primes


object PrimesLazyTrait {

sealed trait Lazy[+A] {
def head: A = this match {
case Nil => error("head called on empty")
case Cons(h, _) => h()
}

def tail: Lazy[A] = this match {
case Nil => error("tail on empty")
case Cons(_, t) => t()
}

def filter(p: A => Boolean): Lazy[A] = this match {
case Nil => Nil
case Cons(h, t) => if(p(h())) Cons(h, () => t() filter p) else t() filter p
}

def foreach(f: A => Unit) {
this match {
case Nil =>
case Cons(h, t) => {
f(h())
t() foreach f
}
}
}

def toList: List[A] = this match {
case Nil => scala.Nil
case Cons(h, t) => h() :: t().toList
}
}

final case class Cons[+A](h: () => A, t: () => Lazy[A]) extends Lazy[A]
case object Nil extends Lazy[Nothing]

def from(n: Int): Lazy[Int] = Cons(() => n, () => from(n + 1))

def primes = {
def sieve(is: => Lazy[Int]): Lazy[Int] = {
lazy val h = is.head
Cons(() => h, () => sieve(is filter (_ % h > 0)))
}

sieve(from(2))
}

def main(args: Array[String]) {
primes foreach println
}
}




In this version we see the Lazy trait provides the necessary methods head, tail, filter, foreach and toList.


These little examples help demonstrate some of the power, elegance and conciseness of lazy functional programming - as well as the beauty of this Ancient Greek algorithm for finding primes.


The basic algorithms complexity is:

  • O((nlogn)(loglogn))

And has a memory requirement of:

  • O(n).

Without any optimization.

--

Additional example - e (Euler's number):

Calculating e, a list of converging numbers in the sequence of e

e can be defined as lim n->inf ( 1 + 1/n)^n and therefore expressed directly as a function e of n as follows:

def e(n : Int) = { Math.pow((1.0 + (1.0/n)), n) }

So, how can we turn this into a stream? If we create a stream of numbers from 1 that are mapped using the e function, which can be defined as an anon lambda argument to map:

val es = Stream.from(1).map((n => { Math.pow((1.0 + (1.0/n)), n) }))

Example usage of the e stream:

es take 5 toList


.

Monday, 15 June 2009

Scala actors, an introduction with some simple examples

The Scala programming language provides an Actor based framework for handling concurrency.

Scala's Actor model for concurrent programming was originally inspired by the Erlang programming language.

The actor model is a system of asynchronous message passing for concurrent programming, rather than the typical shared data and locks model. The shared data and locks model has some inherent difficulties that make programming in this way difficult and error prone. Major problems with shared data and locks include the well known dead-lock problem, as well as live locks and the general problem of locking shared mutable data without affecting program scalability and performance. Whilst Scala's actor messaging framework is build around asynchronous messaging, synchronous messaging is provided on top of this for convenience.

The Scala's actor model is based around an actor library and the Actor trait, along with some simple syntactical constructs for sending and receiving messages.

The actor library can be included into a project using:

import scala.actors.Actor._

The simplest way to start an actor is to declare an object/class that extends the class Actor and in the body of the actor declare an act() {} method to do the work. The actor x can then be started much like a thread, using x.start(). The act() method is much like the Java Thread's run() method.

A more succinct method is to declare a variable val x = actor {} and put the action definition directly between the braces (no act method required). What happens here is that actor is a helper method on Actor that creates and starts the actor - so there is no need to call x.start() explicitly when using this approach.

Actors can receive messages using the receive {} method. The receive message is passed a partial function to receive the messages. Typically the body of the receive method uses case pattern matching to determine what to do with the message.

A message can be sent to an actor using the ! operation. For example, an Int can be sent to actor x using: x ! 3

All actors get their own thread to call the receive function - this causes scalability problems when there are many actors with receive functions. The way to overcome this in Scala is to use the react() {} method instead. React is similar to receive except it doesn't not need a native thread to run and but must call itself when done to receive more messages - apart from this both are the same in that they both take a partial function.

Scala provides a convenience to avoid having to call act/react within a react method, this is the loop construct. This ends up looking as simple as:


loop {
react {
// message handling body goes here
}
}

Note that react itself does not end end by returning up the call stack in the normal way, all calls to react result in an exception.

Within a receive or react method, an actor can reply to the sender of the received message by messaging the special identifier "sender", which represents the source of the message.


Example 1:

This simple example shows the most basic way to create an actor, by extending Actor, creating an instance and calling start on it.


package scalaactors1

import scala.actors._

class Actor1 extends Actor {

def act {
var x = 1
while (true) {
println("actor1! " + x)
x = x + 1
Thread.sleep(500)
}
}
}

object TestActors1 {

/**
* @param args the command line arguments
*/
def main(args: Array[String]) = {

println(">>main() - starting")

val actor1 = new Actor1()

actor1.start()

println("<<main() - ending")
}
}



Example 2:

This example creates a couple of actors using the val x = actor {} constructs. Each actor will automatically start and run concurrently, outputting a message and sleeping in an infinite loop. If you run this you can observe that the main thread starts and ends, the messages from the two actors are interleaved and run even after the main method has ended.


package scalaactors1

import scala.actors.Actor._

object TestActors2 {

/**
* @param args the command line arguments
*/
def main(args: Array[String]) = {

println(">>main() - starting")


val actor1 = actor {

var x = 1
while (true) {
println("actor1! " + x)
x = x + 1
Thread.sleep(500)
}
}

val actor2 = actor {

var x = 1
while (true) {
println("actor2! " + x)
x = x + 1
Thread.sleep(1000)
}
}

println("<<main() - ending")
}
}



Example 3:

This example is slightly more complex.

Actor kbReaderActor takes input from the keyboard (as Chars) and passes them one by one to the actor kbProcessorActor which prints out the received character and then uses a 3rd actor called sleep to sleep for the specified amount of time before then waking up the kbProcessorActor (to simulate some "work" that takes a finite amount of time to complete). Note that sleep signals the "sender" using a simple case class to represent the message type (with no arguments).



package scalaactors1

import scala.actors.Actor._

case class Wakeup // simple signalling case class


// simple actor example
object TestActors3 {

def main(args: Array[String]) = {

println(">>Starting")

val sleep = actor {

loop {
react {
case x : int => {
println("sleeping for " + x)

java.lang.Thread.sleep(x)

println("waking up sender...")

sender ! Wakeup
}
}
}
}

val keyProcessorActor = actor {

println(">>keyProcessorActor starting")

var key : Option[Char] = null

loop {

react {
case 'q' => exit() // 'q' signifies quit app!
case x : Char => println("keyProcessorActor received '" + x + "'")

sleep ! 1000 // ask the sleep actor to sleep, we will get a Wakeup call when done

// now wait for our wakeup
var r = false

do {
receive {
case Wakeup => r = true
}
} while (!r)
}
}

println("<<keyProcessorActor ending")
}

val kbReaderActor = actor {

println(">>kbReaderActor starting")

var key : Char = ' '

do {

println("kbReaderActor waiting for input")
val i = System.in.read()

key = i.asInstanceOf[Char]

println("Received input " + key)

// message actor1 with the key read from the console
keyProcessorActor ! key

} while (key != 'q')

println("<<kbReaderActor ending")
}

println("<<Application Ending")
}
}



.

Sunday, 1 March 2009

Prime factorization, a comparison between Haskell and Scala

Haskell is a powerful and expressive pure lazy functional programming language with some interesting properties. http://www.haskell.org/ Haskell is of course named after the famous logician Haskell Curry http://en.wikipedia.org/wiki/Haskell_Curry who's name can be found in both the Haskell language and the process of "currying" functions.

By way of example and comparison the task of finding prime numbers is shown in Haskell and then its equivalent is shown in Scala.

We can see the conciseness of Haskell and the power of lazyness in expressions of the form:

[3..]

which creates an infinite list of integers from 3 upwards. This seeming impossibility can be handled in a lazy functional language because the infinite list is not evaluated up front, rather the list is only evaluated when needed so the list is finite at any given point in time.

So, in essence a lazy language is one where an expression is only evaluated when it's needed, unlike an imperative language where it's evaluated when declared.


Haskell: prime factorization example:


divides :: Integer -> Integer -> Bool
divides d n = rem n d == 0

ld :: Integer -> Integer
ld n = ldf 2 n

ldf :: Integer -> Integer -> Integer
ldf k n | divides k n = k
| k^2 > n = n
| otherwise = ldf (k+1) n

factors :: Integer -> [Integer]
factors n | n < 1 = error "Invalid argument: argument must be positive"
| n == 1 = []
| otherwise = p : factors (div n p) where p = ld n


If the above is saved in a file (called say primefactorization) and loaded into Hugs (Hugs is a Haskell interpreter shell based on Haskell 98 - http://www.haskell.org/hugs/) using the :l command as follows:

Main> :l c:\Dev\haskell\primefactorisation


Scala: prime factorization example:


package test

object PrimeFactorisationTest {

def divides(d : Int, n : Int) = {
(n % d) == 0
}

def ld(n : Int) : Int = {
ldf(2, n)
}

def ldf(k : Int, n : Int) : Int = {
if (divides(k, n)) k
else if ((k*k) > n) n
else ldf((k+1), n)
}

def factors(n : Int) : List[Int] = n match {
case 1 => Nil;
case _ => {
val p = ld(n)
p :: factors(n / p)
}
}

def main(args : Array[String]) {
if (args.length == 1)
{
val n = Integer.parseInt(args(0))
println(factors(n))
}
}
}


Results:



Main> factors 12343434
[2,3,79,26041]


Both programs produce the same output and are functionally equivalent - it's also clear to see there's a reasonably similarity in both conciseness and expressiveness of the two languages, where Haskell possibly has the slight edge in some respects such as use of the "where" construct.


Update:

Following on from the excellent comments on the Haskell version from Don Stewart, turning the code into something that can be compiled and executed natively, rather than being interpreted using Hugs, roughly involves the following steps.

Install GHC http://www.haskell.org/ghc/

Rename the file to have a .hs file extension

Add a main that can process the arguments:

main = do
[n] <- getArgs
print (factors (read n))

add import to provide environment functions such as getArgs to the top of the file:

import System.Environment

Compile the file:

$ghc --make primefactorisation.hs

Run the executable:

$ time ./primefactorisation.exe 12343434
[2,3,79,26041]

real 0m0.140s
user 0m0.015s
sys 0m0.015s

and there we have it, a functional executable.

Don Stewart also makes a good point about being able to leave out all the type information.

In particular Haskell uses the powerful Hindley-Milner type inference algorithm to allow minimal (often zero) use of explicit type declarations.

http://c2.com/cgi/wiki?HindleyMilnerTypeInference


.

Sunday, 22 February 2009

Scala, implicit conversions

Today, a small posting on the implicit conversion feature of Scala.

Implicit conversions provide a powerful way to extend classes with new functionality, as if already built into the library or even the language itself. There are similarities with C# extension methods, but Scala's implicit conversion mechanism is more concise, flexible and maintainable.

To define an implicit conversion in Scala, the syntax is:

impict def myImplicitConversionName(n: type) = new MyConversionType(n)

The scenario is that you want to add some new method(s) or functionality to instances of an existing class for which may be final/sealed or where sub-classing is inappropriate or not possible / effective. Implicit conversions provide a very natural way to add functionality and to users it will appear as natural features of the language/library, no special syntax is required.

A good example is provided on the Scala site: http://www.scala-lang.org/node/226

Which is a good example because it shows exactly the reasons why you might want to extend the language or library. In this case, the type int provides a good candidate for extension - to add the ! factorial function as if it was natural function of the type int.

I wanted to expand on this example by providing a couple of examples of the same thing with further explanation.

Example 1: explicitly defining a class that provides the extensions and using it in the implicit def


object extendBuiltins2 {

// the important bit, it says there's an implicit conversion from int to Factorizer(n : int)
implicit def int2fact(n: Int) = new Factorizer(n)

def main(args : Array[String]) = {

// when the compiler sees this, it will first 'look for' ! method in int,
// when it doesn't find it, it will look for an implicit conversion on int that provides !
// which is of course exactly what we've just provided in the Factorizer class
println("10! = " + (10!))
}

// define ! as a method on a class that can be constructed from int - this class will
// provide the implicit conversion that supplied an implementation of ! on int
class Factorizer(n: Int) {

def ! = fact(n)
}

// the plain old factorial function
def fact(n: Int): BigInt =
if (n == 0) 1 else fact(n-1) * n
}


The key line that defines the implicit conversion is this:

implicit def int2fact(n: Int) = new Factorizer(n)

what we are saying is that there is an implicit conversion from type int to type Factorizer(int) - so where we have an int, if ! is called the compiler will look for ! on int and fail to find it. It will then seek an implicit conversion that can convert from int in order to provide this method ! - where it finds our implicit conversion definition that uses the Factorizer class to provide the implementation of !

One reason this is more powerful than extension methods is that a whole class of functionality can be added to an existing class without explicitly extending all the methods individually. This has important benefits when it comes to maintenance, if the definition of the extending class changes, you get those changes automatically, you do not have to modify all the extension methods to match.


Example 2: equivalent to the above example but using an anonymous class for a more concise solution where the explicit definition of a class is of no other use.


/* More concise way of adding ! as a method on int's - no explicit class definition */
object extendBuiltins {

implicit def int2fact(n: Int) = new {

def fact(n: Int): BigInt =
if (n == 0) 1 else fact(n-1) * n

def ! = fact(n);
}

def main(args : Array[String]) = {

println("10! = " + (10!))
}
}


This achieves the same this as example 1 but does so more concisely with the anonymous class, constructed using the convenient = new {} syntax.


.

Wednesday, 18 February 2009

Currying, some examples and comparisons between C# and Scala

I'm back again! It's been some 3 weeks since my last blog - I've been in New York for 5 days and busy either side sorting out commitments and work and home.

So, in this posting I'd like to continue on the theme of Lamda and functional programming with some simple examples of Currying in both C# .Net (3.0+) and Scala.

I've tried to keep the examples equivalent as closely as possible but Scala has a few different options when it comes to syntax, so the Scala version is a little longer as it delves into these extra possibilities.

What is currying?

In a nutshell currying is "the process of reducing a function that takes multiple arguments into a series of functions that each take one argument"

Named after Haskell Curry - http://en.wikipedia.org/wiki/Haskell_Curry

A good explanation and definition can be found here http://www.haskell.org/haskellwiki/Currying

C# .Net example using C# 3.0+ syntax for Lamdas:


using System;

static class Program
{
static void Main()
{
Func<int, int, int> add = (x, y) => x + y;

var curriedAdd = add.Curry();

var curriedIncrement = add.Curry()(1);
var curriedDecrement = add.Curry()(-1);

Console.WriteLine("curriedAdd(10)(11) = " + curriedAdd(10)(11));
Console.WriteLine("curriedIncrement(41) = " + curriedIncrement(41));
Console.WriteLine("curriedDecrement(41) = " + curriedDecrement(41));
}

// a simple 2 arg curry template function
static Func<TArg1, Func<TArg2, TResult>> Curry<TArg1, TArg2, TResult>(this Func<TArg1, TArg2, TResult> f)
{
return a1 => a2 => f(a1, a2);
}
}

Results:

curriedAdd(10)(11) = 21
curriedIncrement(41) = 42
curriedDecrement(41) = 40

Notes:

  • The use of the var keyword for implicit typing
  • C# does not provide implicit functions/ partial application so a Curry template function for 2 arguments has been provided explicitly.
  • The Curry method is defined as an extension method so it appears as-if defined on add

Extension Method:

static Func<TArg1, Func<TArg2, TResult>> Curry<TArg1, TArg2, TResult>(this Func<TArg1, TArg2, TResult> f)

Take the above definition of a method, the this keyword in the (this Func...) parameter definition makes the function an extension method, so it can be used as if it were declared on the add function - to illustrate the point, without the extension syntax the Curry function could only be invoked in the regular way as follows:

var curriedAdd = Curry(add);

As an extension method, it's valid to use the original add.Curry() or Curry(add) syntax interchangeably.


Scala example:

package test;

object Currying {

def main(args : Array[String]) {

// regular syntax function add
def add(x : int, y : int) = {

x + y
}

// create curried add using 1 and -1 for increment and decrement by invoking partial function (a:int, _:int):int
var curriedIncrement = add(1, _ : int)
var curriedDecrement = add(-1, _ : int)

// test funtions
println("add(10, 11) = " + add(10, 11))
println("curriedIncrement(41) = " + curriedIncrement(41));
println("curriedDecrement(41) = " + curriedDecrement(41));

// curried syntax function add
def add2(x : int)(y : int) = {

x + y
}

// create curried functions using partial functions
var curriedIncrement2 = add2(1)(_ : int)
var curriedDecrement2 = add2(-1)(_ : int)

println("add2(10)(11) = " + add2(10)(11))
println("curriedIncrement2(41) = " + curriedIncrement2(41));
println("curriedDecrement2(41) = " + curriedDecrement2(41));

// special {} syntax
val res1 = add2(10) {
11
}

println("res1 = " + res1)

// more complex example
val x = 1
val y = 2

val z = add2(10) {

add2(20) {

x + y
}
}

println("z = " + z)
}
}


Results:
add(10, 11) = 21
curriedIncrement(41) = 42
curriedDecrement(41) = 40
add2(10)(11) = 21
curriedIncrement2(41) = 42
curriedDecrement2(41) = 40
res1 = 21
z = 33

Notes:
  • Scala supports f(x:int)(y:int) syntax for explicit curried functions
  • Scala supports implicit currying using partial application, using the placeholder _ syntax
  • Scala supports curried parameter in {} syntax for functions that appear to extend the language by taking parameter from the body of the call.



Update:


I realised that the way I'd defined the curried add functions in the Scala example is perhaps not as verbose or explicit as it could have been, and therefore perhaps less easy to understand or compare to the C# example. So here is another code snipped that shows a more verbose currying:


object Currying {

def main(args : Array[String]) {

def add3(x : int) = {

(y : int) => {
x + y
}
}

println("add3(1)(2) = " + add3(1)(2));

val res = add3(5){

3
}

println("res = " + res)
}
}


Results:


add3(1)(2) = 3
res = 8


Resourses:

Good explanation of currying in C#.Net http://diditwith.net/2007/08/15/TheArtOfCurrying.aspx


.

Friday, 23 January 2009

Scala, a small example of Euclid's Algorithm

A small posting today, a simple example of Euclid's algorithm in Scala. This algorithm can be expressed very concisely and simply using recursion. This example demonstrates the use of tuples and map in the Scala language.

If you don't know, Euclid's algorithm is a ancient, simple algorithm for finding the greatest common divisor (GCD) for two integers, without factoring either of the two integers in question.

The algorithm can be expressed recursively thus:
function gcd(x, y)
if (y == 0) x
else gcd(y, x mod y)

The particular construction worth noting in this example is the use of tuples.

Tuples are like records, structures or groupings of variables that can be constructed using a (x, y, z) syntax when x, y and z are variables. A tuple can be decomposed by accessing its elements using the t _1 syntax, where t is the tuple and _1 is a method that accesses the first element in tuple. Tuple elements are therefore indexed using a base of 1.

The example defines numbers as a list of tuples, each tuple contains a pair of integers - the two integers to be applied to the GCD algorithm (Euclid's Algorithm).

The list of tuples is then "mapped" using the map function that takes each tuple and calls the function x => ((Int, Int), Int, Boolean), where x will be each tuple in the list in turn.

The tuple x is decomposed into its two integer component parts using the x _1 and x _2 syntax.

The returned tuple ((Int, Int), Int, Boolean) contains the original tuple of integers to be tested, the gcd values and a boolean indicating if the numbers are co-prime. The numbers are co-prime if the GCD is equal to 1.

I've included a few different ways to represent the function used by map, to use and decompose the tuple. The first solution using the tuple argument directly and accesses its members using the _1, _2 syntax directly. The second solution uses a function literal and the _ placeholder syntax on the map argument. The function literal uses the val (a, b) = x; sytax to decompose the tuple into and and b, but this is a bit clumsey/verbose. The 3rd solution is essentially the same as a the 2nd but the function is defined in-line.

The last solution is probably the nicest, it uses the case(a, b) syntax to decompose the tuple into a and b using pattern matching.


Example Code:


package test

object EuclidsAlgorithm {

def main(arg : Array[String]) = {

// define a list of tuples (Int, Int), pairs to test for coprimeness
val numbers = (3, 6) :: (3, 7) :: (4, 8) :: (5, 45) :: (5, 43) :: (1071, 1029) :: (1071, 1023) :: (1071, 1021) :: Nil

// #1. Working with the tuple directly, using the tuple _1, _2 syntax
val res1 = numbers map ( x => ((x _1, x _2), gcd(x _1, x _2), gcd(x _1, x _2) == 1 ) )

// #2. Defining a function literal
val f = (x : (Int, Int)) => { val (a, b) = x; ((a, b), gcd(a, b), gcd(a, b) == 1) }

// and using it with the _ placeholder syntax for the parameter representing each element of the list
val res2 = numbers map ( f(_) )

// #3. Using the #2 above but in-line
val res3 = numbers map ( x => { val (a, b) = x; ((a, b), gcd(a, b), gcd(a, b) == 1) } )

// #3. Using a case match on the tuple to expose its components a and b. This is probably the nicest and most concise way
val res4 = numbers map ( { case (a, b) => ((a, b), gcd(a, b), gcd(a, b) == 1) } )

res4.foreach(println)
}

// Euclids algorithm, finds the GCD
def gcd(a : Int, b : Int) : Int = {

if (b == 0)
a
else
gcd(b, a % b)
}
}



Results:


((3,6),3,false)
((3,7),1,true)
((4,8),4,false)
((5,45),5,false)
((5,43),1,true)
((1071,1029),21,false)
((1071,1023),3,false)
((1071,1021),1,true)



.

Monday, 19 January 2009

Scala, insertion sort and pattern matching

This posting shows insertion sorting in Scala. Insertion sort is not a particularly efficient algorithm but this example code does demonstrate the use of pattern matching on lists using the match keyword and the case x :: y type expression.



val result = values match {

case List() => List()

case value :: valuesTail => insert(value, iSort(valuesTail))
}


The above code pattern matched the List values against 2 cases. The first being the empty list, in which case an empty list List() is returned. the second is the pattern value :: valuesTail, which puts the head of the list in value and the tail (the rest of the list) in valuesTail.


Example code:


package test

// Nb: This is not an efficient sorting algorithm but it domostrates
// some aspects of scala for representing algorithms, along with functions as first class objects and currying
object SortingTest {

def main(args : Array[String]) {

val list1 = 5 :: 2 :: 3 :: 1 :: 0 :: -1 :: 2 :: Nil

val matcher = (x : Int, y : Int) => x < y

val sortedList = insertionSort(list1, matcher)

println("sortedList = " + sortedList.mkString(", "))

val revMatcher = (x : Int, y : Int) => x >= y

val revSortedList = insertionSort(list1, revMatcher)

println("revSortedList = " + revSortedList.mkString(", "))
}

def insertionSort(values : List[Int], matcher : (Int, Int) => Boolean) : List[int] = {

def iSort(values : List[Int]) : List[int] = {

val result = values match {

case List() => List()

case value :: valuesTail => insert(value, iSort(valuesTail))
}

println("iSort called with " + values + ", result = " + result)

result
}

def insert(value : Int, values : List[Int]) : List[Int] = {

val result = values match {

// if list is empty return new list with single element in it
case List() => List(value)

// otherwise insert into list in order, recursively
case x :: xTail =>
if (matcher(value, x)) {
value :: values
}
else {
x :: insert(value, xTail)
}
}

println("insert called with " + value + ", values " + values + ", result = " + result)

result
}

iSort(values)
}
}



Results:

If you run the above example code the results are as follows:


iSort called with List(), result = List()
insert called with 2, values List(), result = List(2)
iSort called with List(2), result = List(2)
insert called with -1, values List(2), result = List(-1, 2)
iSort called with List(-1, 2), result = List(-1, 2)
insert called with 0, values List(2), result = List(0, 2)
insert called with 0, values List(-1, 2), result = List(-1, 0, 2)
iSort called with List(0, -1, 2), result = List(-1, 0, 2)
insert called with 1, values List(2), result = List(1, 2)
insert called with 1, values List(0, 2), result = List(0, 1, 2)
insert called with 1, values List(-1, 0, 2), result = List(-1, 0, 1, 2)
iSort called with List(1, 0, -1, 2), result = List(-1, 0, 1, 2)
insert called with 3, values List(), result = List(3)
insert called with 3, values List(2), result = List(2, 3)
insert called with 3, values List(1, 2), result = List(1, 2, 3)
insert called with 3, values List(0, 1, 2), result = List(0, 1, 2, 3)
insert called with 3, values List(-1, 0, 1, 2), result = List(-1, 0, 1, 2, 3)
iSort called with List(3, 1, 0, -1, 2), result = List(-1, 0, 1, 2, 3)
insert called with 2, values List(3), result = List(2, 3)
insert called with 2, values List(2, 3), result = List(2, 2, 3)
insert called with 2, values List(1, 2, 3), result = List(1, 2, 2, 3)
insert called with 2, values List(0, 1, 2, 3), result = List(0, 1, 2, 2, 3)
insert called with 2, values List(-1, 0, 1, 2, 3), result = List(-1, 0, 1, 2, 2, 3)
iSort called with List(2, 3, 1, 0, -1, 2), result = List(-1, 0, 1, 2, 2, 3)
insert called with 5, values List(), result = List(5)
insert called with 5, values List(3), result = List(3, 5)
insert called with 5, values List(2, 3), result = List(2, 3, 5)
insert called with 5, values List(2, 2, 3), result = List(2, 2, 3, 5)
insert called with 5, values List(1, 2, 2, 3), result = List(1, 2, 2, 3, 5)
insert called with 5, values List(0, 1, 2, 2, 3), result = List(0, 1, 2, 2, 3, 5)
insert called with 5, values List(-1, 0, 1, 2, 2, 3), result = List(-1, 0, 1, 2, 2, 3, 5)
iSort called with List(5, 2, 3, 1, 0, -1, 2), result = List(-1, 0, 1, 2, 2, 3, 5)
sortedList = -1, 0, 1, 2, 2, 3, 5
iSort called with List(), result = List()
insert called with 2, values List(), result = List(2)
iSort called with List(2), result = List(2)
insert called with -1, values List(), result = List(-1)
insert called with -1, values List(2), result = List(2, -1)
iSort called with List(-1, 2), result = List(2, -1)
insert called with 0, values List(-1), result = List(0, -1)
insert called with 0, values List(2, -1), result = List(2, 0, -1)
iSort called with List(0, -1, 2), result = List(2, 0, -1)
insert called with 1, values List(0, -1), result = List(1, 0, -1)
insert called with 1, values List(2, 0, -1), result = List(2, 1, 0, -1)
iSort called with List(1, 0, -1, 2), result = List(2, 1, 0, -1)
insert called with 3, values List(2, 1, 0, -1), result = List(3, 2, 1, 0, -1)
iSort called with List(3, 1, 0, -1, 2), result = List(3, 2, 1, 0, -1)
insert called with 2, values List(2, 1, 0, -1), result = List(2, 2, 1, 0, -1)
insert called with 2, values List(3, 2, 1, 0, -1), result = List(3, 2, 2, 1, 0, -1)
iSort called with List(2, 3, 1, 0, -1, 2), result = List(3, 2, 2, 1, 0, -1)
insert called with 5, values List(3, 2, 2, 1, 0, -1), result = List(5, 3, 2, 2, 1, 0, -1)
iSort called with List(5, 2, 3, 1, 0, -1, 2), result = List(5, 3, 2, 2, 1, 0, -1)
revSortedList = 5, 3, 2, 2, 1, 0, -1



.

Friday, 16 January 2009

Scala, example of Diffie-Hellman

Another in the sequence of postings on Scala by simple example, this one shows an example of Diffie-Hellman to establish a shared secret over a public communication channel without that secret being revealed. Diffie-Hellman itself is prone to man-in-the-middle attacks, this example does not attempt to deal with that problem (normally addressed with authentication of some type).

The example sets up the canonical Alice and Bob, with shared public parameters g (generator) = 5 and p (prime modulus) = a large 128 random prime number.

Alice and Bob each create their secret key [sk] (a large 128 random number) and generate:

y = g^sk mod p

Alice and Bob then swap the result of this calculation.

From this, Alice and Bob both perform y^sk mod p to create a new value, the shared secret key which will be the same for both Bob and Alice even though they start with different random secret keys.

This is because:

(g^x)^y is the same as (g^y)^x

From a Scala perspective, the interesting things to note are:

  • Ease of use of large integers using the BigInt class as if it were a primitive type.
  • Constructors to create immutable instances containing g, p and name.
  • Companion object to provide helper function to wrap probable prime function.


Example code:



package test

import scala.util.Random;


object DiffieHellmanTest {

val p = DiffieHellman.randomPrime(128)
val g = 5

def main(args : Array[String]) {

// alice and bob initialise with the public parameters for DH, g and p
val alice = new DiffieHellman(g, p, "Alice")
val bob = new DiffieHellman(g, p, "Bob")

// alice and bob generate their own secret keys and public keys
alice.createSecretKey();
bob.createSecretKey();

val a1 = alice.getPublicKey();
val b1 = bob.getPublicKey();

println("a1 = " + a1)
println("b1 = " + b1)

// alice and bob exchange public keys
bob.setPeerPublicKey(a1)
alice.setPeerPublicKey(b1)

// alice and bob compute their shared secret key
val alicesk = alice.createSharedKey()
val bobsk = bob.createSharedKey()

println("Done, alice sk = " + alicesk + ", bob sk = " + bobsk)
}
}

object DiffieHellman {

def randomPrime(n : Int) : BigInt = { // return a random value with n digits

val rnd = new Random();

BigInt.probablePrime(n, rnd)
}
}

class DiffieHellman(val g : Int, p : BigInt, name : String) {

var secretKey : BigInt = 0
var peerPublicKey : BigInt = 0


def createSecretKey() {

secretKey = random(128)

println(name + " secretKey = " + secretKey)
}

def setPeerPublicKey(x : BigInt) {

peerPublicKey = x
}

def getPublicKey() : BigInt = {

doExpMod(g, secretKey, p)
}

def random(n : Int) : BigInt = { // return a random value with n digits

val rnd = new Random();

BigInt(n, rnd)
}

def createSharedKey() : BigInt = {

doExpMod(peerPublicKey, secretKey, p)
}

private def doExpMod(x : BigInt) : BigInt = {

println("doExpMod g = " + g + ", x = " + x + ", p = " + p)
doExpMod(g, x, p)
}

private def doExpMod(g : BigInt, x : BigInt, m : BigInt) : BigInt = {

g.modPow(x, m)
}
}



Results:

(Numbers may vary between runs of course!)


Alice secretKey = 197977683133285508684427285955258922592
Bob secretKey = 54690551885126631242925351374829949140
a1 = 4317900055693480003952964194026338337
b1 = 58782279533775365647086521912939284670
Done, alice sk = 275531174494727646906973085281579443284, bob sk = 275531174494727646906973085281579443284



The secret keys for Alice and Bob normally wouldn't be revealed, but they've been output here for debug purposes.

Note that the final shared secret key is the same for both Alice and Bob, which could then be used to establish a secure communication channel using symmetric encryption.


.

Saturday, 10 January 2009

Scala, for comprehensions

This post contains some examples of Scala's for comprehension.

A for loop in Scala can be written in the imperative style and will behave rather much the same as a Java for loop, especially the enhanced Java for loop from Java 5.0.

However, this type of simple imperative looping doesn't touch on the power and flexibility of Scala's for comprehension, when used as used in a functional programming context.

In particular, some of the key differences are:

  • A for comprehension can be a value itself. The value is a product of the yield statement used within the expression body. Since a for comprehension iterates over a list or sequence, the resulting value of a for comprehension is typically a list too.
  • A for comprehension uses a generator, written in the form item <- listOfItems to define the items to be iterated over
  • Multiple generators may be used, somewhat analogous to a nested for loop within a for loop in imperative programming
  • In addition to generators, a guard can be used to filter or restrict the values
  • The for syntax can use parenthesis with semicolon delimited generators and guards or braces - both are equivalent
  • The iterations my differ when compared to a seeming equivalent imperative construct, because the implementation is fundamentally different. In particular the list of items to be used is determined using the generator(s) and guard(s) before the body is run, so for example flags used in guards that are modified in the body may not produce the expected result (from an imperative perspective).
  • Generators can be "projected" using the .project function.

Examples of Scala for comprehension:


package test

object ForComprehensionTest {
def main(args : Array[String]) {

// create a couple of simple lists...
val list1 = "Apples" :: "Oranges" :: "Bananas" :: "Avocados" :: Nil
val list2 = 11 :: 22 :: 33 :: 44 :: 55 :: 66 :: Nil

// simple comprehension with one generator and guard
val res1 = for(item1 <- list1 if item1.startsWith("A"))
yield item1

assert(res1 == List("Apples", "Avocados"))
res1.foreach(println);

// more complex comprehension, two generators within {brace} syntax with two guards
val res2 = for {
item1 <- list1 if item1.startsWith("A")
item2 <- list2 if item2 % 3 == 0
}
yield item2 + " " + item1

assert(res2 == List("33 Apples", "66 Apples", "33 Avocados", "66 Avocados"))
res2.foreach(println);

// as above but using () parens and ; semi-colon syntax instead
val res3 = for ( item1 <- list1 if item1.startsWith("A");
item2 <- list2 if item2 % 3 == 0 )
yield item2 + " " + item1

assert(res3 == List("33 Apples", "66 Apples", "33 Avocados", "66 Avocados"))
res3.foreach(println);

println("simple guard item == 22")
for (item <- list2 if item == 22) println(item)

println("simple comprehension mkString = " + (for (item <- list2 if item % 2 == 0) yield item).mkString)

// from an imperative / Java perspective you might expect this to stop with item = 22
// but it continues on because !found is evaluated as always true (irrefutable pattern) when determining what to iterate over
var found = false
for (item <- list2 if !found) {

if (item == 22)
found = true

println("without projection, item == 22? item = " + item)
}

// with the .projection call, it works more like you might expect from an imperative perspective
found = false
for (item <- list2.projection if !found) {

if (item == 22)
found = true

println("with .projection, item==22? item = " + item)
}

found = false
for (item <- list2.projection.takeWhile(item => !found)) {

if (item == 22)
found = true

println(".projection.takeWhile(item => ! found) = " + item)
}
}
}


Results:


Apples
Avocados
33 Apples
66 Apples
33 Avocados
66 Avocados
33 Apples
66 Apples
33 Avocados
66 Avocados
simple guard item == 22
22
simple comprehension mkString = 224466
without projection, item == 22? item = 11
without projection, item == 22? item = 22
without projection, item == 22? item = 33
without projection, item == 22? item = 44
without projection, item == 22? item = 55
without projection, item == 22? item = 66
with .projection, item==22? item = 11
with .projection, item==22? item = 22
.projection.takeWhile(item => ! found) = 11
.projection.takeWhile(item => ! found) = 22


Other useful things to know:

In a for comprehension if you want to iterate over a collection you can create a tuple of (element, index) pairs using syntax of the form:


for ((elem, index) <- iter.zipWithIndex) {
// now we can access element of each iteration along with its associated numerical index!
}



Resources:

A good explanation of for comprehensions can be found here http://creativekarma.com/ee.php/weblog/comments/the_scala_for_comprehension_from_a_java_perspective/


.

Monday, 5 January 2009

Scala, traits and mix-ins

Continuing on in the series of Scala blogs, showing Scala features by example, this post attempts to show the operation of Traits. Traits are rather like classes but designed to allow rich feature composition by allowing "mix-in" of required traits dynamically, at run time. Traits achieve what can look like multiple inheritance, without the pitfalls and limitations of multiple inheritance. Conversely then, you might think traits are equivalent to Java's interfaces. Whist similar, the key difference is that traits can (and usually do) contain implementations (method bodies). When a trait method implementation makes a call to super, it is dynamically resolved as the class is linearised.

When traits are combined with classes to create extended behaviour, this process is known as mixin and the traits are said to be mixed-in or mixins.

The example below shows the behaviour of the plain traits class using its putMsg method, and then its behaviour when mixed-in with TraitAddA, TraitAddA and TraitAddB and then with TraitAddA and TraitAddB reversed (ordering of the with parts is significant).

As well as mixin with the new keyword, class TraitABClass shows that traits can be used as mixins with normal classes too, at the point they are declared using the with keyword.


Example Scala file showing the operation of Traits:

package test

object TraitsTest {

def main(args : Array[String]) {

val traits : Traits = new Traits()

traits.putMsg("Hello!")

println("traits, value = " + traits.toString)

val traitsA : Traits = new Traits() with TraitAddA

traitsA.putMsg("Hello!")

println("traitsA, value = " + traitsA.toString)

val traitsAB : Traits = new Traits() with TraitAddA with TraitAddB

traitsAB.putMsg("Hello!")

println("traitsAB, value = " + traitsAB.toString)

val traitsBA : Traits = new Traits() with TraitAddB with TraitAddA

traitsBA.putMsg("Hello!")

println("traitsBA, value = " + traitsBA.toString)

val traitsABClass : TraitsABClass = new TraitsABClass

traitsABClass.putMsg("Hello2")

println("traitsABClass, value = " + traitsABClass.toString)
}
}

class TraitsABClass extends Traits with TraitAddA with TraitAddB

class Traits {

var msg : String = ""

def putMsg(m : String) {

msg = m
}

override def toString = {

msg
}
}

trait TraitAddA extends Traits {

abstract override def putMsg(m : String) = {

super.putMsg("A" + m)
}
}

trait TraitAddB extends Traits {

abstract override def putMsg(m : String) = {

super.putMsg("B" + m)
}
}


Program execution results:

traits, value = Hello!
traitsA, value = AHello!
traitsAB, value = ABHello!
traitsBA, value = BAHello!
traitsABClass, value = ABHello2


.

Sunday, 4 January 2009

Modular exponentiation

For any inquisitive minds, the fast Modular exponentiation used in the last post (Scala example) might at first glance seem odd, perhaps.

What we want to achieve is x = b^n mod m

where b^n is performed first, the result in then operated on by mod m. Mathematically this is what we are doing, this is what we want to achieve.

However, the expmod function computes the value recursively, applying mod at each step of recursion (on each recursion result).

This is in fact a better way to approach the problem, as the intermediate values are kept small (between 0 and m-1) rather than growing exponentially (with potential overflow/performance problems), only to be operated on by modulus to shrink the value back down again!

Here is a simple Scala snippet that compares expMod with fastExp mod m, with some debugging added to show the intermediate values at each stage of recursion.

package test

import java.util.Random;

object ExpTest {

def main(args : Array[String]) {

println("fastExpt(4, 7) % 9) = " + fastExpt(4, 7) % 9)
println("expMod(4, 7, 9) = " + expMod(4, 7, 9))

println("fastExpt(5, 12) % 6) = " + fastExpt(5, 12) % 6)
println("expMod(5, 12, 6) = " + expMod(5, 12, 6))
}

def expMod(base : Int, exp : Int, m : Int) : Int = {

val res = if (exp == 0)
1
else if (even(exp))
remainder(square(expMod(base, (exp / 2), m)), m)
else
remainder(base * expMod(base, exp - 1, m), m)

println("expMod: b = " + base + ", exp = " + exp + ", m = " + m + ", res = " + res)
res
}

def fastExpt(b : Int, exp : Int) : Int = {

val res = if (exp == 0)
1
else if (even(exp))
square(fastExpt(b, exp / 2))
else
b * fastExpt(b, exp - 1)

println("fastExpt: b = " + b + ", exp = " + exp + ", res = " + res)
res
}

def square(n : Int) : Int = {

n * n
}

def even(n : Int) : Boolean = {

remainder(n, 2) == 0
}

def remainder(n : Int, d : Int) : Int = {

n % d
}
}


Results are as follows:

fastExpt: b = 4, exp = 0, res = 1
fastExpt: b = 4, exp = 1, res = 4
fastExpt: b = 4, exp = 2, res = 16
fastExpt: b = 4, exp = 3, res = 64
fastExpt: b = 4, exp = 6, res = 4096
fastExpt: b = 4, exp = 7, res = 16384
fastExpt(4, 7) % 9) = 4
expMod: b = 4, exp = 0, m = 9, res = 1
expMod: b = 4, exp = 1, m = 9, res = 4
expMod: b = 4, exp = 2, m = 9, res = 7
expMod: b = 4, exp = 3, m = 9, res = 1
expMod: b = 4, exp = 6, m = 9, res = 1
expMod: b = 4, exp = 7, m = 9, res = 4
expMod(4, 7, 9) = 4
fastExpt: b = 5, exp = 0, res = 1
fastExpt: b = 5, exp = 1, res = 5
fastExpt: b = 5, exp = 2, res = 25
fastExpt: b = 5, exp = 3, res = 125
fastExpt: b = 5, exp = 6, res = 15625
fastExpt: b = 5, exp = 12, res = 244140625
fastExpt(5, 12) % 6) = 1
expMod: b = 5, exp = 0, m = 6, res = 1
expMod: b = 5, exp = 1, m = 6, res = 5
expMod: b = 5, exp = 2, m = 6, res = 1
expMod: b = 5, exp = 3, m = 6, res = 5
expMod: b = 5, exp = 6, m = 6, res = 1
expMod: b = 5, exp = 12, m = 6, res = 1
expMod(5, 12, 6) = 1

Most importantly, the functions yield the same results, but expMod achieves it in a more efficient and scalable way.

Just a small posting, as whilst appearing simple, modular exponentiation underpins much of modern cryptographic methods, and efficient algorithms are important.


.