Best scala questions in December 2010

Good use case for Akka

34 votes

I have heard lots of raving about Akka framework (Java/Scala service platform), but so far have not seen many actual examples of use cases it would be good for. So I would be interested in hearing about things developers have used it succesfully.

Only one limitation: please do not include case of writing a chat server. :-) (why? since this has been overused as an example for lots of similar things)

EDIT: Since there are many good answers, my choice of accepted answer is bit arbitrary; I thought highest voted one seems like a reasonable choice. Thank you everyone for good answers!

I have used it so far in two real projects very successfully. both are in the near real-time traffic information field (traffic as in cars on highways), distributed over several nodes, integrating messages between several parties, reliable backend systems. I'm not at liberty to give specifics on clients yet, when I do get the OK maybe it can be added as a reference.

Akka has really pulled through on those projects, even though we started when it was on version 0.7. (we are using scala by the way)

One of the big advantages is the ease at which you can compose a system out of actors and messages with almost no boilerplating, it scales extremely well without all the complexities of hand-rolled threading and you get asynchronous message passing between objects almost for free.

It is very good in modeling any type of asynchronous message handling. I would prefer to write any type of (web) services system in this style than any other style. (Have you ever tried to write an asynchronous web service (server side) with JAX-WS? that's a lot of plumbing). So I would say any system that does not want to hang on one of its components because everything is implicitly called using synchronous methods, and that one component is locking on something. It is very stable and the let-it-crash + supervisor solution to failure really works well. Everything is easy to setup programmatically and not hard to unit test.

Then there are the excellent add-on modules. The Camel module really plugs in well into Akka and enables such easy development of asynchronous services with configurable endpoints.

I'm very happy with the framework and it is becoming a defacto standard for the connected systems that we build.

Scala collection-like SQL support as in LINQ

10 votes

As far as I understand the only thing LINQ supports, which Scala currently doesn't with its collection library, is the integration with a SQL Database.

As far as I understand LINQ can "accumulate" various operations and can give "the whole" statement to the database when queried to process it there, preventing that a simple SELECT first copies the whole table into data structures of the VM.

If I'm wrong, I would be happy to be corrected.

If not, what is necessary to support the same in Scala?

Wouldn't it possible to write a library which implements the collection interface, but doesn't have any data structures backing it but a String which gets assembled with following collection into the required Database statement?

Or am I completely wrong with my observations?

As the author of ScalaQuery, I don't have much to add to Stilgar's explanation. The part of LINQ which is missing in Scala is indeed the expression trees. That is the reason why ScalaQuery performs all its computations on Column and Table types instead of the basic types of those entities.

You declare a table as a Table object with a projection (tuple) of its columns, e.g.:

class User extends Table[(Int, String)] {
  def id = column[Int]("id", O.PrimaryKey, O.AutoInc)
  def name = column[String]("name")
  def * = id ~ name
}

User.id and User.name are now of type Column[Int] and Column[String] respectively. All computations are performed in the Query monad (which is a more natural representation of database queries than the SQL statements that have to be created from it). Take the following query:

val q = for(u <- User if u.id < 5) yield u.name

After some implicit conversions and desugaring this translates to:

val q:Query[String] =
  Query[User.type](User).filter(u => u.id < ConstColumn[Int](5)).map(u => u.name)

The filter and map methods do not have to inspect their arguments as expression trees in order to build the query, they just run them. As you can see from the types, what looks superficially like "u.id:Int < 5:Int" is actually "u.id:Column[Int] < u.id:Column[Int]". Running this expression results in a query AST like Operator.Relational("<", NamedColumn("user", "id"), ConstColumn(5)). Similarly, the "filter" and "map" methods of the Query monad do not actually perform filtering and mapping but instead build up an AST that describes these operations.

The QueryBuilder then uses this AST to construct the actual SQL statement for the database (with a DBMS-specific syntax).

An alternative approach has been taken by ScalaQL which uses a compiler plugin to work directly with expression trees, ensure that they only contain the language subset which is allowed in database queries, and construct the queries statically.

Tired of non-semantic testing to make up for dynamic typing - suggestions?

9 votes

I used to do a lot of web programming in Rails (PHP before that) before I started studying computer engineering.

Since then, I've done a lot of school work in C, and some personal stuff in Objective-C (Mac stuff). I learnt to love static typing.

But now I'm having to do some professional web development (freelancing) and have picked up Rails once again. I'm finding it really annoying to write non-semantic type-checking tests. I was getting those for free from C and Objective-C compilers. I loved hitting Build and having the system check all my code to see that A can call B, B can call some obscure library C, etc. All I had to do was test the semantics. But with Rails, I'm the compiler. :(

Has anyone treaded this same path? Are my only options for web development ASP.NET MVC with C# and Java + x framework? Looking for some suggestions, or even some sympathy... :P

By the way, I make a specific reference to Rails rather than Ruby because I don't mind Ruby's dynamic nature for simple stuff like scripting or what not. But since Rails depends on so many gems and since one usually adds a number of other gems, the dynamic typing becomes an issue.

Thanks!

edit:

I followed up on pst's suggestion and looked into Scala. In reading the book Programming in Scala, written by the language's creator, Martin Odersky, I came accross this bit of text that in many ways expresses my concerns and a bit more. Very interesting reading.

Taken from page 52 of Martin Odersky's Programming in Scala:

Scala is statically typed

A static type system classifies variables and expressions according to the kinds of values they hold and compute. Scala stands out as a language with a very advanced static type system. Starting from a system of nested class types much like Java’s, it allows you to parameterize types with generics, to combine types using intersections, and to hide details of types using abstract types. These give a strong foundation for building and composing your own types, so that you can design interfaces that are at the same time safe and flexible to use.

If you like dynamic languages such as Perl, Python, Ruby, or Groovy, you might find it a bit strange that Scala’s static type system is listed as one of its strong points. After all, the absence of a static type system has been cited by some as a major advantage of dynamic languages. The most common arguments against static types are that they make programs too verbose, prevent programmers from expressing themselves as they wish, and make impossible certain patterns of dynamic modifications of software systems.

However, often these arguments do not go against the idea of static types in general, but against specific type systems, which are perceived to be too verbose or too inflexible. For instance, Alan Kay, the inventor of the Smalltalk language, once remarked: “I’m not against types, but I don’t know of any type systemsthat aren’t a complete pain, so I still like dynamic typing.”

We hope to convince you in this book that Scala’s type system is far from being a “complete pain.” In fact, it addresses nicely two of the usual concerns about static typing: verbosity is avoided through type inference and flexibility is gained through pattern matching and several new ways to write and compose types. With these impediments out of the way, the classical benefits of static type systems can be better appreciated. Among the most important of these benefits are verifiable properties of program abstractions, safe refactorings, and better documentation.

Verifiable properties

Static type systems can prove the absence of certain run-time errors. For instance, they can prove properties like: booleans are never added to integers; private variables are not accessed from outside their class; functions are applied to the right number of arguments; only strings are ever added to a set of strings.

Other kinds of errors are not detected by today’s static type systems. For instance, they will usually not detect non-terminating functions, array bounds violations, or divisions by zero. They will also not detect that your program does not conform to its specification (assuming there is a spec, that is!). Static type systems have therefore been dismissed by some as not being very useful. The argument goes that since such type systems can only detect simple errors, whereas unit tests provide more extensive coverage, why bother with static types at all?

We believe that these arguments miss the point. Although a static type system certainly cannot replace unit testing, it can reduce the number of unit tests needed by taking care of some properties that would otherwise need to be tested. Likewise, unit testing cannot replace static typing. After all, as Edsger Dijkstra said, testing can only prove the presence of errors, never their absence. So the guarantees that static typing gives may be simple, but they are real guarantees of a form no amount of testing can deliver.

Safe refactorings

A static type system provides a safety net that lets you make changes to a codebase with a high degree of confidence. Consider for instance a refactoring that adds an additional parameter to a method. In a statically typed language you can do the change, re-compile your system and simply fix all lines that cause a type error. Once you have finished with this, you are sure to have found all places that need to be changed. The same holds for many other simple refactorings like changing a method name, or moving methods from one class to another. In all cases a static type check will provide enough assurance that the new system works just like the old.

Documentation

Static types are program documentation that is checked by the compiler for correctness. Unlike a normal comment, a type annotation can never be out of date (at least not if the source file that contains it has recently passed a compiler). Furthermore, compilers and integrated development environments can make use of type annotations to provide better context help. For instance, an integrated development environment can display all the members available for a selection by determining the static type of the expression on which the selection is made and looking up all members of that type.

This is one of my "gripes" about dynamic languages. I want to test for semantics, not type errors ;-) That being said, a good testing framework/setup is really a must in all non-trivial situations and good code-coverage and tested requirements is/are important.

If you do want to go down the static-typing path on the JVM (I have), I would highly recommend looking at Scala. Coming from Ruby, it's far less painful (and actually lots of fun in different ways) than going to Java. You get to "keep" the things you take for granted -- an expression-based syntax, closures, the ability to omit types in many places (not as open as Ruby, but you do get compile-time type checking ;-), everything(*)-is-an-object OO, unified accessor methods, ability to construct DSLs easily, and sugar -- and get the benefits of a statically typed language with local type inference, pattern matching, a relatively rich collection framework, and decent integration with Java (including the numerous web-frameworks, there are some Scala-specific frameworks as well which leverage the Scala language).

C#3.0/4.0 (and .NET3.5+) isn't too shabby either (but avoid C#2.0, which is now hopefully a relic), with the introduction of LINQ/closures, basic type inference and other nice language features I find it "acceptable" for most tasks (take a guess how I would rate Java as a language ;-). However, C# is a CLR-target language (there is/was a .NET Scala port, but I am not sure of the status -- it is not the main target platform though).

Since I have mentioned Scala, I should also mention F# (now an "official" .NET language) which takes the "Funtional with OO" approach being similar to OCaml -- Scala is more of the reverse and I would describe it as "OO with Functional". I have heard arguments for/against F# compared to C# w.r.t the type system, but have no practical experience with F#. You may or may not like the paradigm shift.

Happy coding.

How to track the source line (location) of an XML element?

5 votes

I assume that there is probably no satisfactory answer to this question, but I ask it anyway in case I missed something.

Basically, I want to find out the line in the source document from which a certain XML element originated, given the element instance. I want this only for better diagnostic error messages - the XML is part of a configuration file, and if there is something wrong with it, I want to be able to point the reader of the error message to exactly the right place in the XML document so he can correct the error.

I understand that the standard Scala XML support probably has no built-in feature like this. After all, it would be wasteful to annotate every single NodeSeq instance with such information, and not every XML element even has a source document from which it has been parsed. It seems to me that the standard Scala XML parser throws the line information away, and later on there is no way to retrieve it.

But switching to another XML framework is not an option. Adding another library dependency "only" for the sake of better diagnostic error messages seems inappropriate to me. Also, despite some shortcomings, I really like the built-in pattern matching support for XML.

My only hope is that you can show me a way to alter or subclass the standard Scala XML parser such that the nodes it produces will be annotated with the number of the source line. Maybe a special subclass of NodeSeq can be created for this. Or maybe only Atom can be subclassed because NodeSeq is too dynamic? I don't know.

Anyway, my hopes are close to zero. I don't think there is a place in the parser where we can hook in to change the way nodes are created, and that at that place the line information is available. Still, I wonder why I have not found this question before. Please point me to the original if this is a duplicate.

I had no idea how to do that, but Pangea showed me the way. First, let's create a trait to handle location:

import org.xml.sax.{helpers, Locator, SAXParseException}
trait WithLocation extends helpers.DefaultHandler {
    var locator: org.xml.sax.Locator = _
    def printLocation(msg: String) {
        println("%s at line %d, column %d" format (msg, locator.getLineNumber, locator.getColumnNumber))
    }

    // Get location
    abstract override def setDocumentLocator(locator: Locator) {
        this.locator = locator
        super.setDocumentLocator(locator)
    }

    // Display location messages
    abstract override def warning(e: SAXParseException) {
        printLocation("warning")
        super.warning(e)
    }
    abstract override def error(e: SAXParseException) {
        printLocation("error")
        super.error(e)
    }
    abstract override def fatalError(e: SAXParseException) {
        printLocation("fatal error")
        super.fatalError(e)
    }
}

Next, let's create our own loader overriding XMLLoader's adapter to include our trait:

import scala.xml.{factory, parsing, Elem}
object MyLoader extends factory.XMLLoader[Elem] {
    override def adapter = new parsing.NoBindingFactoryAdapter with WithLocation
}

And that's all there is to it! The object XML adds little to XMLLoader -- basically, the save methods. You might want to look at its source code if you feel the need for a full replacement. But this is only if you want to handle all of this yourself, since Scala already have a trait to produce errors:

object MyLoader extends factory.XMLLoader[Elem] {
    override def adapter = new parsing.NoBindingFactoryAdapter with parsing.ConsoleErrorHandler
}

The ConsoleErrorHandler trait extract its line and number information from the exception, by the way. For our purposes, we need the location outside exceptions too (I'm assuming).

Now, to modify node creation itself, look at the scala.xml.factory.FactoryAdapter abstract methods. I have settled on createNode, but I'm overriding at the NoBindingFactoryAdapter level, because that returns Elem instead of Node, which enables me to add attributes. So:

import org.xml.sax.Locator
import scala.xml._
import parsing.NoBindingFactoryAdapter
trait WithLocation extends NoBindingFactoryAdapter {
    var locator: org.xml.sax.Locator = _

    // Get location
    abstract override def setDocumentLocator(locator: Locator) {
        this.locator = locator
        super.setDocumentLocator(locator)
    }

    abstract override def createNode(pre: String, label: String, attrs: MetaData, scope: NamespaceBinding, children: List[Node]): Elem = (
        super.createNode(pre, label, attrs, scope, children) 
        % Attribute("line", Text(locator.getLineNumber.toString), Null) 
        % Attribute("column", Text(locator.getColumnNumber.toString), Null)
    )
}

object MyLoader extends factory.XMLLoader[Elem] {
    // Keeping ConsoleErrorHandler for good measure
    override def adapter = new parsing.NoBindingFactoryAdapter with parsing.ConsoleErrorHandler with WithLocation
}

Result:

scala> MyLoader.loadString("<a><b/></a>")
res4: scala.xml.Elem = <a line="1" column="12"><b line="1" column="8"></b></a>

Note that it got the last location, the one at the closing tag. That's one thing that can be improved by overriding startElement to keep track of where each element started in a stack, and endElement to pop from this stack into a var used by createNode.

Nice question. I learned a lot! :-)