szeiger.de

I have just created a new scala-2.7 branch for ScalaQuery. My original plan was to target only Scala 2.8 but since I’ve made lots of progress during the last few weeks and I’ve seen increased interest in ScalaQuery, I tried to build it with 2.7.5.

I had to change the semantics of SimpleFunction and SimpleBinaryOperator for 2.7 but I prefer the new version anyway, so it went into the main line. The code on the scala-2.7 branch is currently identical to the master branch, except for the test classes which are different in two regards:

• Although the Scala Language Specification mandates that the part left to the “<-” in a for comprehension is a pattern, the wildcard pattern “_” does not work in 2.7. I have changed it to a dummy variable named “__“.
• The type inferencer in 2.7 cannot infer the correct type for the implicit OptionMapper objects. OptionMapper[_,_,_,_] has four type parameters, the last one being used for the return type of functions which use the mapper, so it is not yet known when looking for an implicit mapper and gets inferred as Nothing. Scala 2.8 apparently knows that this type parameter is undetermined, finds the single matching implicit object for the other three parameters and then fills in the fourth. The type-correct interoperability of option and non-option types in ScalaQuery relies heavily on the improved type inferencer and I don’t see any way of making it work nicely with 2.7. The work-around is to add type annotations to the boolean operators, e.g. a && b might become a.&&[Boolean,Option[Boolean]](b). Yuck!

I’m still focused on Scala 2.8 as a target platform and will probably not spend much time integrating new features into the scala-2.7 branch but contributions are always welcome.

I have recently made a change to ScalaQuery which enables the use of any type for a column without needing specific Column classes and factory methods on Table for it. For this I used an approach which I had not considered earlier and which I wasn’t even sure would work.

Scala (at least in version 2.7) only considers single function calls for implicit conversions. Take the following definitions for example:

class A
class B(a: A)
class C(b: B)

implicit def aToB(a: A) = new B(a)
implicit def bToC(b: B) = new C(b)

val a = new A

def useB(b: B) = ...
def useC(c: C) = ...

You can call useB(a) because the compiler finds the implicit conversion aToB(a) but you cannot call useC(a) — the compiler does not try the chained call bToC(aToB(c)).

But this does not mean that an implicit conversion may not rely on an implicit value! The following works just fine:

trait Column[T]
class ConstColumn[T](value: T, tm: TypeMapper[T]) extends Column[T]

implicit def valueToColumn[T](value: T)(implicit tm: TypeMapper[T]) =
  new ConstColumn(value, tm)

trait TypeMapper[T]
implicit object IntTypeMapper extends TypeMapper[Int]

val c1: Column[_] = 42

The Scala compiler recognizes that valueToColumn[Int] provides the desired conversion from Int to Column[_] and then looks for the required implicit TypeMapper[Int] value, which is available through the implicit IntTypeMapper object.

In ScalaQuery, the real TypeMapper contains only a few methods and there are predefined TypeMappers for most of the basic types used by JDBC. That’s nice because it removes some redundancy from my code base but the better news is that it allows you to write your own TypeMappers. For example, if you wanted to get the java.lang.Integer columns back which I removed in favor of Int and Option[Int], you could add this implicit object to your code:

implicit object IntegerTypeMapper extends TypeMapper[java.lang.Integer] {
  def zero = null
  def sqlType = java.sql.Types.INTEGER
  def setValue(v: java.lang.Integer, p: PositionedParameters) =
    if(v eq null) p.setIntOption(None) else p.setIntOption(Some(v.intValue))
  def setOption(v: Option[java.lang.Integer], p: PositionedParameters) = v match {
    case Some(null) => p.setIntOption(None)
    case Some(i) => p.setIntOption(Some(i.intValue))
    case None => p.setIntOption(None)
  }
  def nextValue(r: PositionedResult) = r.nextIntOption match {
    case None => null
    case Some(i) => java.lang.Integer.valueOf(i)
  }
}

The same should work for any database engine- or domain-specific types.

This is the solution to the exercise from part 5. (more…)

This is the fifth part in a series of articles on formal language processing in Scala. In this part I will introduce some new parser combinators, provide a specification for the Fun1 language and build an interpreter for it. (more…)

Previously, ScalaQuery used nullable Java types for all columns. This is straight-forward for reference types like java.lang.String which are mapped 1:1 to Scala types but not for Scala’s value types like scala.Int or scala.Boolean. Although they are often implemented in terms of Java types like java.lang.Integer and java.lang.Boolean, there is no 1:1 mapping and there are no aliases for those wrapper types in Scala. Most of the time this is not a problem because you can use Scala’s value types everywhere and have the compiler generate optimized code with primitive types where applicable.

Except there is one thing you cannot do with Scala’s value types: They may never be null. The reference types are nullable but that’s probably not a good idea, either. Scala needs nullable reference types for interfacing with Java code but for native Scala APIs, the Option type is the preferred solution. It makes None values explicit and thus prevents NullPointerExceptions at runtime when a null value is used in a place where it shouldn’t be allowed.

SQL databases also use NULL values for various things (e.g. missing values, undefined values, horrid abominations) so this should work nicely with Scala’s Option type. I rejected Options for database results in ScalaQuery at first because most columns are not nullable (so you wouldn’t want to have all columns return an Option instead of just the nullable ones) yet some operations like outer joins need to convert non-nullable columns to nullable ones (which can probably not be typed in Scala's type system, so you'd have to make all columns return an Option).

On the other hand, having nullable java.lang.Integer and other Java wrapper types in a Scala API is rather ugly, so I finally removed them in favor of a more practical yet null-free solution. All columns now use proper Scala types (scala.Int, scala.Predef.String, etc.). NULLs from the database are returned as a default value (0 for Int, an empty string for String) which can be changed with the orElse method, e.g.:

  for(u <- Users)
    yield u.first.orElse("no first name")
        ~ u.last.orElse("no last name")

Note that this value is only used when extracting rows from the result set on the client side. It does not change the handling of NULLs inside the database server!

For nullable types like scala.Predef.String you can use orElse(null) to get the old behaviour back. The argument of orElse() is passed by name, so you can also run an arbitrary block of code or throw an exception when a NULL value is encountered in the result set. There is a convencience method called orFail which does just that. Maybe this should be the default instead of returning a zero value?

The preferred solution for distinguishing between regular values and NULL for any type is to use the ? method to turn a SimpleColumn of type T into one of type Option[T]:

  for(u <- Users) yield u.first.? ~ u.last.?

Currently there are no operations defined on option columns so it does not make sense to declare an option column directly as part of a table. Ideally, columns of types T and Option[T] should be fully interoperable because the database server uses three-valued logic anyway.

I have just pushed the new code to the master branch and updated the stable branch to include the previously redesigned AST.