Sunday, October 11, 2009

Scala Case Statements As Partial Functions

A Scala case statement can be either a Function1 or a PartialFunction depending on the context.

In my previous post I presented a simple Publisher that I used to decouple my Swing actors from their targets. Reader nairb774 pointed out that the standard Scala library includes a Publisher class. In fact, there are two Publisher classes in Scala, scala.collection.mutable.Publisher and scala.swing.Publisher. Although I like my publisher class better, the swing publisher did have one feature that I thought was useful: it accepted as a callback a PartialFunction rather than, as mine did, a Function1. That would mean, I thought, that I could pass in a case statement as a callback.

For example, continuing the Mimprint example from my previous post, if I were only interested in Enabled events published by a particular publisher, rather than explicitly checking this in my callback with an isInstanceOf or a match statement that includes a case _ => clause, I could just use a one-line case statement:
showSingleViewerPublisher.subscribe { case e:Enabled => doSomething() }
My calling code in Publisher would call apply on the PartialFunction callback only if a call to its isDefinedAt method returned true, thus avoiding the MatchError that would occur if I treated it like a Function1 and called its apply method when the value was not Enabled. This seemed like useful functionality, so I decided to add it. I thought it would be easy, but unfortunately it was not.

Consider the following three definitions that assign a case statement to a partial function, full function, or no explicit function type, respectively:
val pfv:PartialFunction[String,Unit] = { case "x" => println("Got x") } val ffv:Function1[String,Unit] = { case "x" => println("Got x") } val nfv = { case "x" => println("Got x") }
For the first line, the variable pfv gets assigned a value which is a PartialFunction representing the case statement. For the second line, you might think that, since PartialFunction extends Function1 and we are assigning the same value to ffv as we did to pfv, that the variable ffv would be assigned a value which is a PartialFunction, just as for the value pfv. This is not the case.

The Scala Language Specification (SLS) explicitly states, in section 8.5, that the type of an anonymous function comprised of one or more case statements must be specified as either a FunctionK or a PartialFunction, and that the value generated by the compiler is different depending on that specified target type. So the value that gets assigned to ffv is a Function1, and ffv.isInstanceOf[PartialFunction[_,_]] evaluates to false. Note that we could assign the value pfv to the variable ffv, in which case ffv would have a value which is a PartialFunction and ffv.isInstanceOf[PartialFunction[_,_]] would evaluate to true.

What happens if you don't specify the type, as in the third line above where we assign the same value to nfv? You might think the compiler could infer the type of the resulting value, but since, as specified in the SLS, the type must be explicitly specified as either a FunctionK or a PartialFunction, our assignment to nfv is actually not a valid statement, and it fails to compile. It would be nice if the error message said something like "You must explicitly specify either a FunctionK or a PartialFunction for a case statement", but instead it gives this relatively unhelpful message:
<console>:4: error: missing parameter type for expanded function ((x0$1) => x0$1 match { case "x" => println("Got x") }) val nfv = { case "x" => println("Got x") } ^
In my case, the situation in which I encountered this message was a little different. Here is an example showing the problem I ran into:
class PF[T] { //partial function type def sub(x:PartialFunction[T,Unit]) = x } class FF[T] { //full function type def sub(x:Function[T,Unit]) = x } class NF[T] { //no unique function type def sub(x:PartialFunction[T,Unit]) = x def sub(x:Function[T,Unit]) = x } val pf = new PF[String] val ff = new FF[String] val nf = new NF[String] pf.sub{ case "x" => println("x") } //works, result is PartialFunction ff.sub{ case "x" => println("x") } //works, result is Function1 nf.sub{ case "x" => println("x") } //fails with compiler error msg
Calling the above method sub with a case statement works when there is only one method of that name, whether it takes a Function1 or a PartialFunction, but although the compiler has no problem compiling the overloaded pair of functions, once they both exist the compiler can no longer unambiguously determine the target type for the case statement, so it delivers that same error message "missing parameter type for expanded function".

In my case I was trying to modify the subscribe method in my Publisher class so that I could pass in either a regular function, such as println(_), or a PartialFunction, in particular an in-line case statement. The three options I tried are essentially classes PF, FF and NF listed above. When I used approach NF I was unable to directly pass in a case statement, but instead would get the compiler error mentioned above. When I used approach PF I could pass in a case statement as a PartialFunction, but I could not pass in a regular function. When I used approach FF I could pass in a regular function, and could pass in and properly deal with a PartialFunction, since it extends Function1, but when I used an in-line case statement it would get compiled as a Function1 rather than a PartialFunction, which would cause execution to fail when a value was passed to that case statement that it did not cover (since it was not a PartialFunction and thus did not have an isDefinedAt method to call first).

I don't like option FF because it would allow code (specifically, an in-line case statement) to compile but then not execute as expected. Options PF and NF are not very useful as is, since neither directly supports both case statements and full functions.

In a mailing list response to someone who was attempting to use option NF in his application, Paul Phillips suggested using option FF with a helper function pf that accepts a PartialFunction and returns the same value, then wrapping any case statements inside a call to that helper function; or, alternatively, assigning the case statement to a val declared as a PartialFunction before passing it to method sub. Unfortunately, if the user forgets to use either of these techniques on a case statement and just passes it directly to method sub in option FF, it will be handled as a Function1 rather than a PartialFunction, so it will compile but not behave as expected.

Paul's suggestion would also work in option NF (and in option PF, although in that case it would be redundant), which would behave much the same as option FF from the user's perspective except that passing a bare case statement to the overloaded method sub would not compile, so we would no longer have the undesirable situation of something that compiles but behaves unexpectedly.

As an alternative to Paul's pf helper function, I could write a helper function ff that takes a Function1 and turns it into a PartialFunction with an isDefinedAt method that always returns true. I would then use this with option PF. This would allow me to directly pass in case statements, but I would have to wrap all regular functions in a call to ff.

I have not yet made any changes to my Publisher class, since I don't particularly like either of the options and I don't currently really need the ability to use in-line case statements. Meanwhile, if I get the compiler error "missing parameter type for expanded function" while trying to use an in-line case statement, at least I now know one more thing to check for.


Ken B said...

You could also just define an implicit to convert any total function into an equivalent partial function:

implicit def toPartialFunction[A,B] ( x: (A)=>B ) = new PartialFunction[A,B]{
def apply(value:A) = x(value)
def isDefinedAt(value:A) = true

Then define your publisher class on a partial function, and let the Scala compiler save you the keyboarding.

Jim McBeath said...

Ken B: That works if the type of the total function is fully defined, but not when the function type is incompletely specified. I want to be able to use the "_" shorthand for my callbacks and not have to fully specify the types of everything; for example, I want to be able to use "println(_)" rather than having to use "(s:String)=>println(s)" or "println(_:String)". "val x:Function1[String,Unit] = println(_)" works because Scala knows that "println(_)" must be of type "Function1[String,Unit]", but "println(_)" does not have enough type information for Scala to match it against an implicit conversion function. If you can make the statement "val x:PartialFunction[String,Unit] = println(_)" compile by using implicit conversions, I would be interested in seeing how that can be done.

Eric said...

Hi Jim,

I've been bitten by this issue recently and trying to revisit it, I'm wondering if one of the way out would be to do this:

class ExtendedFunction[T, S](f: Function[T, S]) {
def applySafely(t: T): Option[S] = try {
} catch { case e: MatchError => None }
implicit def toExtendedFunction[T, S](f: Function[T, S]) = new ExtendedFunction(f: Function[T, S])

class Test[T](val t: T) {
def sub(x:Function[T, Unit]) = x.applySafely(t)
val test = new Test("hello")
test sub { println(_) }
test sub { case s => println("partial " + s) }

A bit ugly but at least from the api user point of view that behaves as expected right?


Eric said...

Hmm, too good to be true. There a nasty situation with what I described. If the function succeeds in applying its case statement but the right hand side of the case statement raises a MatchError, then there is no way to distinguish the two.

Eric said...

I'm sorry, it's like I'm polluting your blog here but I think I may have a solution,...

If I define applySafely as:

def applySafely(a: A): Option[B] = {
try {
} catch {
case e: MatchError => {
if (e.getCause() == null)
throw e

then, this additional check for getCause does the trick. When the function, which we want to use as a partial function, fails because of a second match getCause won't be null. So:

val f2: Function[Int, String] = { case a => a match { case x if x > 0 => x.toString } }

f2.applySafely(0) will throw an exception as expected.

Jim McBeath said...

Eric: In a simple test I tried, getCause was null even when thrown from the nested match expression (which I believe is the correct behavior, since nothing in that code is catching and rethrowing), so I don't think that test works to distinguish the two potential causes of MatchError. But your approach, while I can't say it is elegant, is interesting, and it might be possible to make it work somehow.

Eric said...

You're right, I was actually misled by the fact that my specification is passing as expected:

And I still don't understand why it works (using Scala 2.8.0.RC2).

If you have any idea,... Thanks.

Jim McBeath said...

Eric: your spec test on f2 is passing because you are calling apply rather than applySafely.

Eric said...

OMG, I can't believe I'm that blind. For my defense I was sick in bed yesterday,... So we're left with an applySafely that would swallow *any* MatchError.

Thanks for your patience and your posts!