Lift Without Maven

It is possible to run Lift without Maven. It's actually pretty easy. It's just not supported. I'll tell you how to do it.

• Background
• With Maven
• Without Maven
• Ant Build File

Background

You can tell from my blog posts that I like Scala. I have wanted to try out Liftweb for a while, but was turned off by the apparent requirement of using Maven. I figure that if I do like Lift, it will be enough work to convince my company that they should switch to Scala and Lift without also having to convince them at the same time that they should switch to Maven.

The members of the Lift community seem pleased with Maven and are not interested in supporting a non-Maven workflow. It's not that Lift requires Maven - in fact, David really wants people to know that Lift does not require Maven - but at the moment it's simply not a priority for them to support a non-Maven download. Although you can't really fault a newcomer for thinking that Maven is required, given that the Getting Started page currently states that Maven 2 is a "prerequisite". If you start asking about downloading and using Lift without Maven you tend to get a lot of responses about how great Maven is and how you should really try it. This is perhaps unfortunate, because many people, myself included, are not looking for a discussion about Maven and really want to minimize the number of new tools and workflow changes necessary just to try out a piece of software. Maven may be great, but that's not what I want to hear about right now. So I decided to figure out for myself just what it would take to download, compile and run Liftweb without Maven.

I started by using the standard Maven-based approach to get the files onto my machine, then hacked until I had what I wanted. You can do the same thing (it was actually pretty easy), or if you are having trouble with Maven or you simply don't want to touch it yet, you can use the directions I give below on how to download and set up the Liftweb demo without it. So although the semi-official answer (on the Lift wiki) to how to use Lift without Maven is that you at least need to use Maven to download the initial lift project, I'm here to tell you that you can download, compile and run the demo Lift app without using Maven at all.

With Maven

The download page on the Lift web site is interesting. There is a short paragraph for each of Mac, Windows and Linux downloads. The Mac and Windows paragraphs each include a link to a download package. The Linux paragraph just says, if you have trouble, ask someone on the mailing list.

I scratched my head a bit at the lack of a download link on the Download page, but finally went back to the main page, saw the link to the Getting Started page, opened the HTML document there, and started reading how to install Liftweb.

Step 1 is to run the mvn archetype:generate command, which downloads the Lift templates that Maven uses to build and update Lift. Unfortunately, those Lift templates require version 2.0.9 of Maven, and the newest version available as a standard package on my system (Fedora 9) is Maven 2.0.4. Rather than upgrading my OS, I went to the Maven website, downloaded the latest version of Maven (2.1.0), and installed it in an alternate location on my system. I then updated my $PATH to include that alternate location so that I would be using a new enough version of Maven to work on the Lift demo.

After downloading and installing the updated version of Maven, I was able to run the mvn archetype command listed on the Getting Started page. Since I had not used Maven before on this computer, it downloaded a bunch of files - about 5MB worth. The Getting Started page does say Maven will output "several page's worth of text", so this was not entirely a surprise.

Step 2 is to run the mvn jetty:run command. This command finishes building the demo Liftweb app and starts it using Jetty listening on port 8080. When I ran this second Maven command, it downloaded another big batch of files from the Internet. The Getting Started page says that after the first step, "you've now successfully created your first project", and it just says the second step "should produce more output", implying that there is not much work to be done in that second step. However, the second step took at least 5 times as long and downloaded another 30 MB of files; that was a bit of a surprise. In total Maven took about four minutes and downloaded 35MB of files - 710 files, including 82 jar files - into my new $HOME/.m2 directory (the Maven cache), including older versions of Scala and Ant than I already had on my machine. That's not really all that much time or space for a modern computer, but it was perhaps a bit more than I was expecting would be required for a "hello world" program. And in fact, as you will see below, only a small part of those files are necessary.

Unfortunately, after downloading half the Internet and finishing the build of the Liftweb demo, it failed to run on my system. Ah, but of course: I am already running Tomcat, which is using port 8080, the same port the Liftweb demo was trying to use. No problem, I thought, I will just change the port that the Liftweb demo uses. Unfortunately, my quick attempt was unsuccessful. I searched through all the files for 8080, found it in one place, changed it to 8181, then ran mvn jetty:run again. No luck - after crunching for about ten seconds it still tried to open port 8080. Among the many messages that Maven spit out was the line "Nothing to compile - all classes are up to date", so apparently it did not notice that I had edited one of the source files. Well, no big deal, I just stopped Tomcat for a while and tried it again, and I was able to access the demo web page at localhost:8080.

So getting the Liftweb demo running using the Maven approach was pretty easy:

1. Download and install a new version of Maven.
2. Stop Tomcat on my machine.
3. Run the two maven commands on the Liftweb Getting Started page.
4. Open my web browser to http://localhost:8080 and view the hello world page.

Without Maven

As you can see from the above, getting Lift downloaded and working with Maven was, for me, no problem. But what I really wanted to do was to build a war file that I could drop into my already-running Tomcat. I started with a simple ant script to compile HelloWorld.scala, and kept adding bits and pieces to it until I was able to build a working war file. Below are the details about the final setup I used to create a war file that worked under Tomcat.

I am using a grand total of 11 files:

• Two scala files: Boot.scala and HelloWorld.scala
• Two html files: index.html and templates-hidden/default.html
• Two xml files: web.xml and build.xml
• Five jar files: lift-util.jar, lift-webkit.jar, scala-library.jar, log4j.jar, and commons-fileupload.jar

Of the above files, both scala files, both html files, web.xml and the two lift jar files came unmodified out of the Liftweb demo, the Scala library came from the standard Scala download, the other two jar files were standard downloads from Apache, and the one file I created myself was the ant build file build.xml.

My directory hierarchy looks like this:

liftdemo/
    build/              All generated files go here
    build.xml           Ant build file
    src/
        scala/
            bootstrap/liftweb/Boot.scala
            demo/helloworld/snippet/HelloWorld.scala
        webapp/
            index.html
            web.xml
            templates-hidden/default.html

The five library jar files are stored elsewhere in my system and are copied in to the build directory when the war file is being created. You could create a lib directory in the liftdemo directory and drop the libraries in there if you wanted to. If you used Maven to download Lift, you can pull all of these libraries from your $HOME/.m2 Maven cache, else you can download them from their internet homes if you don't already have them somewhere on your system.

The war file liftdemo.war looks like this:

META-INF/MANIFEST.MF
WEB-INF/
    web.xml
    classes/
        bootstrap/liftweb/
            Boot$$anonfun$1.class
            Boot.class
        demo/helloworld/snippet/HelloWorld.class
    lib/
        commons-fileupload-1.2.1.jar
        lift-util-1.0.jar
        lift-webkit-1.0.jar
        log4j-1.2.14.jar
        scala-library.jar
index.html
templates-hidden/default.html

You can download files directly from the Liftweb Maven repository without using Maven: use these direct download links for the demo sources packed in a jar file, lift-util-1.0.jar, and lift-webkit-1.0.jar. (Note that these are all version 1.0 files, so if you are reading this article well past the posting date there are probably newer versions - poke around the repository to find out what the latest version is.) Unpack the demo source jar to a temporary location, then copy the five files mentioned above into the appropriate places in the hierarchy. Download scala, log4j and commons-fileupload (if you haven't already), drop in a buld.xml file (see below), and modify the properties to fit your environment. At that point you should be able to run a simple ant command to create a war file in the build directory. Copy that war file into the webapps directory, and you should be able to see the demo page in the standard way from your web browser.

After that, you are on your own. Obviously this very simple setup does not handle dependencies like Maven does, so when those dependencies get updated, you will have to manually manage that process. And when you build up your code and start using other features, you will have to manually add additional jar files to build.xml. But for some of us, that's just what we want.

The size of the src directory listed above is under 60KB, the two liftweb jar files are under 3MB together, and the scala and other two libraries are, combined, just over 4MB. The war file weighs in at 6.3MB. The original liftweb demo download includes some tests that I did not include in my war file, so it could be that the tests use a few more of those 82 jar files that Maven downloaded.

The build file below is pretty simple. There are really only two commands necessary to build the webapp: first compile the two scala files, then build the war file. I hope this post has made it clear that you don't need Maven at all to use Lift, and that Lift webapps are satisfactorily small and simple to build and run.

Ant Build File

Here is my sample buid.xml file for building the Liftweb demo without Maven. Remember to review the properties and change them as appropriate for your environment. In particular, you will almost certainly want to change the home.dir property, and probably the jar file locations as well. If you create a lib directory in your liftdemo directory and put the five libraries there, you can simplify the build file by getting rid of home.dir and a number of the properties used to define the locations of the libraries.

<project name="liftdemo" default="war">

    <target name="init" description="Initialize all properties and paths">

 <property name="home.dir" value="/u/jimmc"/>

 <property name="build.dir" value="build"/>
 <property name="class.dir" value="${build.dir}/class"/>
 <property name="scala.src.dir" value="src/scala"/>
 <property name="war.build.dir" value="${build.dir}/war"/>
 <property name="war.lib.dir" value="${war.build.dir}/WEB-INF/lib"/>
 <property name="war.file" value="build/liftdemo.war"/>
 <property name="web.xml" value="src/webapp/web.xml"/>

 <!-- Location of scala files -->
 <property name="scala.dir"
  value="${home.dir}/net/scala/scala-current"/>
 <property name="scala-compiler.jar"
  value="${scala.dir}/lib/scala-compiler.jar"/>
 <property name="scala-library.jar"
  value="${scala.dir}/lib/scala-library.jar"/>

 <!-- Location of Liftweb files -->
 <property name="liftweb.dir" value="${home.dir}/net/liftweb/1.0"/>
 <property name="lift-util.jar"
  value="${liftweb.dir}/lift-util-1.0.jar"/>
 <property name="lift-webkit.jar"
  value="${liftweb.dir}/lift-webkit-1.0.jar"/>

 <!-- A couple of other libraries are required -->
 <property name="log4j.jar"
  value="${home.dir}/net/log4j/log4j-1.2.14.jar"/>
 <property name="fileupload.jar"
       value="${home.dir}/net/fileupload/commons-fileupload-1.2.1.jar"/>

 <path id="scala.classpath">
     <pathelement location="${scala-compiler.jar}"/>
     <pathelement location="${scala-library.jar}"/>
 </path>

 <path id="compile.classpath">
     <path refid="scala.classpath"/>
     <pathelement location="${lift-util.jar}"/>
     <pathelement location="${lift-webkit.jar}"/>
 </path>

 <!-- define the "scalac" ant task -->
 <taskdef resource="scala/tools/ant/antlib.xml">
     <classpath refid="scala.classpath"/>
 </taskdef>
    </target>

    <target name="clean" depends="init">
 <delete dir="${build.dir}"/>
    </target>

    <target name="mkdirs" depends="init">
 <mkdir dir="${build.dir}"/>
 <mkdir dir="${class.dir}"/>
    </target>

    <target name="compile" depends="init,mkdirs"
     description="Compile scala files">
 <scalac
     srcdir="${scala.src.dir}"
     destdir="${class.dir}"
     addparams="-g:vars"
     classpathref="compile.classpath"
     force="changed"
     deprecation="on"
 >
     <include name="**/*.scala"/>
 </scalac>
    </target>

    <target name="war" depends="init,compile">
 <!-- First make an image directory in war.build.dir -->
 <copy todir="${war.build.dir}">
     <fileset dir="src/webapp" includes="**/*.html"/>
 </copy>
 <mkdir dir="${war.build.dir}/WEB-INF"/>

 <copy todir="${war.lib.dir}" file="${scala-library.jar}"/>
 <copy todir="${war.lib.dir}" file="${lift-util.jar}"/>
 <copy todir="${war.lib.dir}" file="${lift-webkit.jar}"/>
 <copy todir="${war.lib.dir}" file="${log4j.jar}"/>
 <copy todir="${war.lib.dir}" file="${fileupload.jar}"/>

 <copy todir="${war.build.dir}/WEB-INF/classes">
     <fileset dir="${class.dir}" includes="**/*.class"/>
 </copy>

 <!-- Put everything from our war directory into a war file -->
 <war
     destfile="${war.file}"
     webxml="${web.xml}"
 >
     <fileset dir="${war.build.dir}" includes="**/*"/>
 </war>
    </target>

</project>

Scala Functions vs Methods

Scala has both functions and methods. Most of the time we can ignore this distinction, but sometimes we have to deal with the fact that they are not quite the same thing.

In my Scala Syntax Primer I mention that I use the terms method and function interchangeably in the discussion. This is a simplification. In many situations, you can ignore the difference between functions and methods and just think of them as the same thing, but occasionally you may run into a situation in which the difference matters. This is analogous to how most of us treat mass and weight. In our daily lives on the surface of planet Earth, we treat them as interchangeable units, with 1Kg being the same as 2.2 pounds. But they are not quite the same thing: when an astronaut walks on the surface of the moon, his mass (kilograms) has not changed but his weight (pounds) is only about one sixth of what it was on Earth.

In contrast to Kg vs pounds, you are rather more likely to come across a situation in which the distinction between Scala functions and methods is important than you are to be walking on the surface of the moon. So when can you ignore the difference between functions and methods, and when do you need to pay attention to it? You can answer that question once you understand the difference.

A Scala method, as in Java, is a part of a class. It has a name, a signature, optionally some annotations, and some bytecode.

A function in Scala is a complete object. There are a series of traits in Scala to represent functions with various numbers of arguments: Function0, Function1, Function2, etc. As an instance of a class that implements one of these traits, a function object has methods. One of these methods is the apply method, which contains the code that implements the body of the function. Scala has special "apply" syntax: if you write a symbol name followed by an argument list in parentheses (or just a pair of parentheses for an empty argument list), Scala converts that into a call to the apply method for the named object. When we create a variable whose value is a function object and we then reference that variable followed by parentheses, that gets converted into a call to the apply method of the function object.

When we treat a method as a function, such as by assigning it to a variable, Scala actually creates a function object whose apply method calls the original method, and that is the object that gets assigned to the variable. Defining a function object and assigning it to an instance variable this way consumes more memory than just defining the functionally equivalent method because of the additional instance variable and the overhead of another object instance for the function. Thus you would not want every method to be a function; but functions give you a great deal of power that is not available with just methods, and in those situations that power is well worth the additional memory used.

Let's look at some details of this mechanism. Create a file test.scala with this in it:

class test {
    def m1(x:Int) = x+3
    val f1 = (x:Int) => x+3
}

Compile that file with scalac and list the resulting files. Scala creates two files: test.class and test$$anonfun$1.class. That strange extra class file is the anonymous class for the function object that Scala created in response to the function expression assigned to f1. If you use more than one function expression in your test class, there will be more than one anonymous class file, even if you write the same function expression over again.

If you run javap on the test class, you will see this:

Compiled from "test.scala"
public class test extends java.lang.Object implements scala.ScalaObject{
    public test();
    public scala.Function1 f1();
    public int m1(int);
    public int $tag()       throws java.rmi.RemoteException;
}

Running javap on the function class test$$anonfun$1 yields this:

Compiled from "test.scala"
public final class test$$anonfun$1 extends java.lang.Object implements scala.Function1,scala.ScalaObject{
    public test$$anonfun$1(test);
    public final java.lang.Object apply(java.lang.Object);
    public final int apply(int);
    public int $tag()       throws java.rmi.RemoteException;
    public scala.Function1 andThen(scala.Function1);
    public scala.Function1 compose(scala.Function1);
    public java.lang.String toString();
}

Because this class implements the Function1 interface we know it is a function of one argument. You can see that it contains a handful of methods, including the apply method.

You can also define a function in terms of an existing method by referencing that method name followed by a space and an underscore. Modify test.scala to add another line that does this:

class test {
    def m1(x:Int) = x+3
    val f1 = (x:Int) => x+3
    val f2 = m1 _
}

The m1 _ syntax tells Scala to treat m1 as a function rather than taking the value generated by a call to that method. Alternatively, you can explicitly declare the type of f2, in which case you don't need to include the trailing underscore:

    val f2 : (Int) => Int = m1

In general, if Scala expects a function type, you can pass it a method name and have it automatically converted to a function. For example, if you are calling a method that accepts a function as one of its parameters, you can supply as that argument a method of the appropriate signature without having to include the trailing underscore.

Back to our test file, now when you compile test.scala there will be two anonymous class files, one for the f1 class and one for the f2 class. You can use either definition for f2, they generate identical class files.

If you use the -c option to javap to get the code of the second anonymous class, you can see the call to the m1 method of the test class in the apply method:

public final int apply(int);
  Code:
   0:   aload_0
   1:   getfield        #17; //Field $outer:Ltest;
   4:   astore_2
   5:   aload_0
   6:   getfield        #17; //Field $outer:Ltest;
   9:   iload_1
   10:  invokevirtual   #51; //Method test.m1:(I)I
   13:  ireturn

Let's fire up the scala interpreter and see how this works. In the following examples, input text is shown in bold and output text in regular weight.

scala> def m1(x:Int) = x+3
m1: (Int)Int

scala> val f1 = (x:Int) => x+3
f1: (Int) => Int = <function>

scala> val f2 = m1 _
f2: (Int) => Int = <function>

scala> m1(2)
res0: Int = 5

scala> f1(2)
res1: Int = 5

scala> f2(2)
res2: Int = 5

Note the difference in the signatures between m1 and f1: the signature (Int)Int is for a method that takes one Int argument and returns an Int value, whereas the signature (Int) => Int is for a function that takes one Int argument and returns an Int value.

At this point we seem to have a method m1 and two functions f1 and f2 that all do the same thing. But f1 and f2 are actually variables that contain an instance of a generated class that implements the Function1 interface, and that object instance has methods that m1 does not have.

scala> f1.toString
res3: java.lang.String = <function>

scala> (f1 andThen f2)(2)
res4: Int = 8

Because m1 is itself a method, unlike f1 you can't call methods on it:

scala> m1.toString
<console>:6: error: missing arguments for method m1 in object $iw;
follow this method with `_' if you want to treat it as a partially applied function
       m1.toString
       ^

Note that each time you separately reference a method as a function Scala will create a separate object.

scala> val f3 = m1 _
f3: (Int) => Int = <function>

scala> f2 == f3
res6: Boolean = false

Even though f2 and f3 both refer to m1, and both do the same thing, they are not considered equal by Scala because function objects inherit the default equality method that compares identity, and these are two different objects. If you want two function values to be equal, you must ensure that they refer to the same instance of function object:

scala> val f4 = f2
f4: (Int) => Int = <function>

scala> f2 == f4
res7: Boolean = true

Here are a few other examples showing that the expression m1 _ is in fact a function object:

scala> m1 _
res8: (Int) => Int = <function>

scala> (m1 _).toString
res9: java.lang.String = <function>

scala> (m1 _).apply(3)
res10: Int = 6

As of Scala version 2.8.0, the expression (m1 _)(3) will also return the same value (there is a bug in previous versions that causes this syntax to give a type mismatch error).

There are some other differences between methods and functions. A method can be type-parameterized, but an anonymous function can not:

scala> def m2[T](x:T) = x.toString.substring(0,4)
m2: [T](T)java.lang.String

scala> m2("abcdefg")
res11: java.lang.String = abcd

scala> m2(1234567)
res12: java.lang.String = 1234

However, if you are willing to define an explicit class for your function, then you can type-parameterize it similarly:

scala> class myfunc[T] extends Function1[T,String] {
     |     def apply(x:T) = x.toString.substring(0,4)
     | }
defined class myfunc

scala> val f5 = new myfunc[String]
f5: myfunc[String] = <function>

scala> f5("abcdefg")
res13: java.lang.String = abcd

scala> val f6 = new myfunc[Int]
f6: myfunc[Int] = <function>

scala> f6(1234567)
res14: java.lang.String = 1234

So go ahead and keep converting pounds to kilograms by dividing by 2.2 (unless you are an astronaut), but when you start mixing functions and methods in Scala, keep in mind that they are not quite the same thing.

Updated 2010-12-08: clarify memory usage in response to comment by jqb.

Fun With Feedback

When I was in high school I participated in an interesting exercise to demonstrate the importance of feedback. You might want to try this out as a party game. It is easy to laugh at others when they do poorly at this seemingly simple exercise, but it is in fact a rather difficult task.

Get a set of children's building blocks, such as a set of hardwood unit blocks, with a variety of shapes and sizes. From the original set of blocks, create two identical sets. Each set should have at least ten blocks in it with at least four different shapes. More variety is better. Having blocks within each set that are similar but not identical, such as two different kinds of triangles, makes it more difficult. Given these two identical sets, anything you can build with one set can be built with the other set.

Arrange two desks or tables, with their chairs, for two people such that there is a level work space in front of each person in which that person can build, and each person can hear the other, but neither can see the other person or the other blocks. On each of the two work spaces, place one of the two sets of blocks you created above. The two work spaces should be as close to identical as is practical to ensure that any structure built on one work space can be built on the other work space. One possible arrangement is to place a large vertical separator in the middle of a table such that it separates the two ends of the table, then have each person sit on an opposite end of the table facing each other. The separator must be large enough to prevent the two people from seeing each other (and stable enough not to fall over). This allows onlookers to stand on the sides of the table and easily see both of the builders and constructions.

Select two test subjects and sit them down in front of their respective work spaces and blocks. Designate one as the master and the other as the apprentice (or one as the speaker and the other as the listener, if you prefer). The master builds a structure with his blocks (any structure of his choice containing a least a designated minimum number of blocks), and instructs the apprentice as to how to build exactly the same structure. The apprentice is not allowed to ask questions or provide any other kind of feedback to the master. The apprentice must build the same structure as the master based solely on the verbal description given by the master. The master, in turn, must provide his verbal description of his structure without any knowledge of what the apprentice is doing.

It turns out that this is surprisingly difficult to do well. In most cases, the apprentice's construction will diverge from the master's construction after only a few blocks, and by the time the master is placing his tenth block and describing it to the apprentice, the apprentice is completely lost and unable to place the block in any location remotely resembling the master's description. At some point along the way, the apprentice becomes aware that something is wrong, but can't do anything about it - and the master, blithely unaware of the apprentice's difficulty, inexorably continues to build and describe something that the apprentice no longer understands. An alert master may be able to tell from the reactions of the viewers that the apprentice is having a problem, but without any feedback will not be able to know quite what the problem is, so will be unable to correct it and will have to forge on.

I watched a number of other pairs who attempted this exercise fail to build structures of any significant size, and I was determined to do a better job. When it was my turn to be master, I very carefully described each block I selected, how I oriented it, and how I placed it next to other previously placed blocks. I was describing everything at an excruciating level of detail, far more than I would in normal circumstances. And, as I learned at the end of my turn, for a while this was working quite well, and my apprentice was able to construct significantly more than had anyone else.

But there was a snag: at one point I selected a small triangular block, which happened to be a right equilateral triangle. When placing it, I instructed my apprentice to orient the triangle by pointing the corner which was a right angle in a certain direction. Unfortunately, math was not a strong point for my apprentice, and for a while she was unable to recall what a right angle was. As a consequence, she did not understand what I meant concerning the orientation of the block, but because I was unaware of her problem and continued my building and describing, she had to guess about how to place the block, and she guessed wrong. Of course, errors cascaded from there because further steps which relied on the placement of that block could not be done properly.

Had she been able to ask me a couple of simple questions, I could easily and quickly have clarified for her what I meant. Without any feedback at all, the failure to understand one small detail meant the game was over.

You can try the same exercise, but this time let the apprentice ask questions. It is amazing how much difference it makes. Without feedback, almost nobody can succeed on this task, and when they do it requires an extraordinary amount of redundancy in the master's descriptions. With verbal feedback, most people can do quite a good job, and much faster than without feedback, since the master has the liberty to be much sloppier in his initial description, confident in the knowledge that the apprentice will ask for more details on any part he does not understand. With visual feedback, virtually every apprentice can quite speedily reproduce the master's design. The speed and accuracy of communication improves dramatically as the feedback bandwidth increases.

I highly recommend this exercise for teaching the value of precise communication (especially when there is no or little feedback available), the power of a high-bandwidth feedback channel (especially when there is any complexity involved) and the recognition that some tasks are far harder than they appear (especially when you have not directly attempted the task yourself).