Debugging ScalaTest tests in NetBeans with the Maven ScalaTest Plugin

In the spirit of creating blog posts whenever there is something important that I've figured out and I should not lose or forget, here I go.

A month or two ago, we moved all of our Java tests at Apprenda to run using the Maven Scalatest plugin . We have been using ScalaTest from day one; however, for a while there we were using SureFire + @RunWith + JUnitRunner annotations to kick off the tests. However, since IntelliJ was quite happy to run ScalaTest tests without anything else (e.g. like the @RunWith and JUnitRunner annotations), we would sometimes end up in a situation where we have some tests that are created during development in the IDE; however, when the build is run using just plain Maven, it would skip over these tests.

So, the as described in the ScalaTest documentation on working w/ the plugin , you first disable the surefire plugin, and then add the ScalaTest Maven plugin to run the tests. That all worked great, and now all of our tests run, regardless of whether we remembered to add the @RunWith annotation.

That presented a problem for me : when we were running the tests with SureFire, NetBeans was more than happy to run and debug the tests. However, when we switched to the Maven ScalaTest plugin - no such luck .. what do to ? I'm totally fine with not seeing the graphical test runner results that NetBeans shows w/ JUnit Tests ; however, where it becomes a problem is when I can't run or debug the individual tests from the IDE. The tests run just fine when you "Test" the whole project (e.g. right click on the project and choose "Test" from the context menu), but if you try to run an individual test (e.g. right click on a test case and choose Test from the context menu), or when you're inside the test case go to the Run -> Test File menu you're out of luck, nothing shows up.

So, getting the single test to run from NetBeans turns out to be pretty simple :
1. In the Project Properties -> Actions panel, choose the "Test File" action, change the properties from "test=${packageClassName}" to "suites=${packageClassName}" . As indicated in the ScalaTest documentation, the way to tell ScalaTest what test to run is using the suites system property, e.g. :

So, that's great, but how do you debug the test ? Looking at the ScalaTest documentation I was a bit confused on how to do this, as there were a few options related to debugging, but which ones to set in order to get this from NetBeans, it wasn't entirely clear.

It turns out the fix was fairly simple :
1. Since by default NetBeans runs surefire:test to run individual tests, the goal that is run needed to be switched to just "test" - this allows the Maven Scalatest plugin to run (as it binds to the test phase and does nothing for surefire:test)
2. I had to add  a couple of additional system properties that ScalaTest needs in order to run the right test, turn on debugging, and connect the NetBeans debugger and the ScalaTest test that is running.

#This teslls scalatest to run the test with this class
suites=${packageClassName}

#These tell ScalaTest to start the new process with debugging enabled
logForkedProcessCommand=true
debugForkedProcess=true
debugArgLine=-Xdebug -Xrunjdwp:transport=dt_socket,server=n,address=${jpda.address}

# This tells netbeans to fire up the debugger and wait for a connection
jpda.listen=true

Here's an example of what it looks like :

Tada - enjoy being able to run and debug individual ScalaTest tests right from NetBeans !

Intro to Programming : Drawing a House

What is this 

To help everyone along in getting started with drawing the Minecraft Creeper, I asked my son to put together a program that draws a house. It uses a lot of similar elements to the Creeper that the students need to draw, and might show a few new things that we didn't cover in class yet.

The Code

   
  clear()
  println("setAnimationDelay() sets how fast the turtle speed is, invisible makes the turtle invisible but the line is still there")
  println("penUp() make the line disapear but the turtle doesn't, setFillColor(Color(0, 0 ,0)) and setPencolor(Color(0, 0, 0,))  changes the color")
  println("repeat(){ } makes the program repeat as many time as you want")
  setAnimationDelay(100)
  invisible()
  penUp()
  left()
  forward(150)
  setFillColor(Color(255, 0, 51))
  setPenColor(Color(255, 0, 51))
  penDown()
  right()
  //makes the square for the house, note the usage of 'repeat'
  repeat(4){
    forward(150)
    right()
  }
  forward(150)
  //this makes the roof
  setFillColor(Color(51, 51, 0))
  setPenColor(Color(51, 51, 0))
  right(60)
  forward(90)
  right(60)
  forward(90)
  penUp()
  //this makes the windows
  setFillColor(Color(51, 153, 255))
  setPenColor(Color(51, 153, 255))
  right(60)
  forward(40)
  right()
  forward(20)
  penDown()
  forward(30)
  right()
  forward(30)
  right()
  forward(30)
  right()
  right()
  penUp()
  forward(80)
  penDown()
  forward(30)
  left()
  forward(30)
  left()
  forward(30)
  left()
  forward(30)
  left()
  penUp()
  left()
  forward(80)
  penDown()
  //this makes the door
  setPenColor(Color(153, 68, 0))
  setFillColor(Color(153, 68, 0))
  forward(60)
  left()
  forward(35)
  left()
  forward(60)
  left()

The Outcome

The finished product looks something like this 

Intro to Programming : The First Class

.. And so it begins... 

This spring, I am teaching an "Intro to Programming" class at the After School Enrichment program at Craig Elementary, so this blog will get some new content with the progress of the class.

The First Class

In the first class, we covered quite a lot of ground : we learned quite a bit about each other, what the students in the class were interested in. As expected, most kids do enjoy video games, like learning how things work, and just plain enjoy working with computers. We briefly chatted about why people program in the first place : to express themselves, to tackle new challenges, and to be able to harness the power of computers for their own inventions.

We covered some of the basic concepts of what a program is : a sequence of instructions that follow some specific grammar rules (just like they do in English). We had a quick introduction to Kojo (our programming environment, more on that later) and the various elements of Kojo's user interface - the command / script area, the drawing area, the menus, the output / error message area. Most importantly, we focused on how to continue learning from the environment - using code completion / prompts to see the syntax of commands that we hadn't used before, reading the command documentation. We also spent some time learning to recover from failure, when something unexpected happens - resetting the zoom level on the drawing area (if one accidentally zoomed out too far), and just bringing up the right windows if one of them suddenly disappeared.

Our first programming exercise really didn't involve a computer : I asked my students to "program me" and command me from one side of the room to the other, while avoiding the obstacles along the way - chairs, large metal poles, backpacks. I did attempt to behave like a computer - I just followed the instructions that were given to me and failed a few times along the way - ran into an office chair, gonked my head against the metal pole. In order to tie up with what came next (which was drawing a square in our programming environment), I then asked the kids to use the same commands that they used to guide me through the room, but this time, to make me walk in a square along the tiled floor using the same limited set of commands - e.g. left, right, forward, back.


So, with the basic commands under our belt, the kids got to programming. We learned how to draw a square. First everyone came up with a short program for a square, and then we experimented with variations on the topic - some of the kids really enjoyed drawing BIG squares and seeing the turtle in action while it was drawing the squares. Then, for the more ambitious students, we tried drawing a triangle (and yes, this is where the geometry skills came shining through - what were the angles in an equilateral triangle, you ask :-) ). The example looks something like this :

The takeaway : for the kids who want a little challenge at home, I asked them to practice drawing a Minecraft Creeper, which very roughly, looks like this - it will give the kids a chance to practice some of the skills that they've already picked up (e.g. drawing squares), but also give a chance to learn a few new things as well (using background and fill colors)

Setting up Kojo

Kojo is a very kid-friendly piece of software which was created specifically to help children learn about programming and have fun along the way. It should run on all modern OS-s (Windows, Linux, MacOS), so as long as you have a desktop computer to install this on (but it will not work on a tablet), your child should be able to continue exploring it on their own. It contains a rich library of examples written in Kojo (which we will lean on in the class) - it even has a game or two that the kids can play with.

Setting up Kojo is pretty straighforward:
1. Dowload Kojo from http://www.kogics.net/kojo-download
2. Run the downloaded .jar file (e.g. by right clicking and choosing "Run" on windows), it will launch the installer - just follow through the installer and you should have an icon on your desktop that you can use to launch the software.

My Scala (and Tapestry 5) experience

Since both a couple of coworkers asked me about Scala in the last few weeks, I thought these might be interesting to read through (something that emerged over the last few days)

•  http://codahale.com/downloads/email-to-donald.txt
•  http://codahale.com/the-rest-of-the-story/

I have obviously not used Scala to the extent that the guy describes, but I can testify at how annoying it is the constant translation between Java and Scala (especially, since my small app, , was using Tapestry 5, which is obviously a Java framework). The conversion between the two was made worse (at least for me, a total Scala newb) by the following:

• Classes with the same names but completely or subtly different usages and intents . The fact that the Scala library used the same class names as the Java library (java.lang.Long and scala.Long) , the default imports use them, and there are some magical conversions that occur between the eponymous types. On a few occasions I was totally baffled about something not working only to find that in the end, the code was getting the wrong type . Thus, in a bunch of my classes, I ended up explicitly having to import the Java classes that the framework expected to work with , e.g. 

    
       import java.util.{List => JList }
       import org.slf4j.Logger
       import scala.collection.JavaConversions._
  
       class FooPage {
  
  
   @Property
   private var catPieces:JList[ArtPiece] = _
  
  
          // explicitly declaring the returning types as the Java types
   def onPassivate():JList[String]= {
                  // and using the Scala provided conversions
    return seqAsJavaList(List(category,subCategory))
   }
  
       }
        
  

  and then explicitly use the java specific types. A similar but different situation exists with java.util.List and the scala List classes (e.g.  scala.collections.immutable.List) although they have the same name they have a completely different purpose (e.g. the Scala list is not necessarily intended to be created, and manipulated like the Java list); the equivalent of the java list is the recommended Scala ListBuffer
• Null handling -  because I was interacting w/ a Java framework, there was an expectation that nulls are OK and at different places, the framework does expect methods to return nulls in order to behave in certain ways. Scala goes for the whole Option pattern (where you aren't supposed to use nulls to make it all better) and has some conversion (that I obviously, don't fully understand) between null and these types. However, because of the interaction w/ the Java framework, I had to learn how to deal with both. It kinda sucked.
• Tapestry 5 and Scala interactions -  because Tapestry 5 pushes the envelope on being a Java framework w/ a whole bunch of annotation processing, class transformations, etc. , in some cases there were clashes between the T5 approach and Scala. In some respects, Tapestry 5 manages to be a respectable and succinct Java framework by adding a whole bunch of metaprogramming features, which when used with Scala make the scala code less attractive, e.g: 
• Page properties that would otherwise be set up as private fields in regular Tapestry 5, now have to be declared as private fields and initialized. If you didn't declare them as private, then T5 would complain (since pages can't have non-private members as it is managed by T5) , e.g. :

    
    class Foo {      
          @Inject 
   var logger:Logger = _
     
   @Inject 
   var pm:PersistenceManager = _
    }   
  

• Sometimes the T5 and Scala approaches seemed to clash in ways that make things complicated. For example, in the persistent objects in the class, I often annotated the private fields w/ @BeanProperty (so that Scala generates proper getters/setters for those fields).

    
    import scala.reflect.BeanProperty
    import javax.jdo.annotations.Persistent;
    class PersistentFoo {      
          @BeanProperty 
   @Persistent
   var title = ""
  
    }   
  

  Yet, when I accidentally did the same for some page properties at weird points the application would start failing (on application reload with Tapestry's live class reloading) until in pages I replaced the approach w/ Tapestry's @Property annotation (although they're supposed to do the same it's quirky w/ BeanProperty)

    
    import org.apache.tapestry5.annotations.Property; 
  
    class FooPage {      
          @Property 
   private var category:String = _
  
    }   
  

When I was working on the app, a few times I had to just stop for a day because I couldn't figure out how to do something massively simple (e.g. how to succinctly join a list of Strings into a comma separated string - stuff that would have taken me 30 seconds to do in Java, 2 seconds in Groovy and the proposed Scala solutions seemed like massive overkill). I originally started wanting to write some tests in Scala for this app, because I thought, "wouldn't it be nice to have something a little more flexible and less verbose than Java", but that still has nice static typing. Later I decided to try the whole Scala+T5 approach, and I have to admit I was pretty mad at myself when I would get stuck .

Obviously, many of my problems described above were due to my own weak Scala-foo (e.g. I had read through at least 2-3 books in order to be brave enough to try this just to learn that until I try things hands on, it doesn't stick too well), and other issues that I had were due to the interaction w/ the specific Java framework that I chose (Tapestry 5). Yet, in some ways, the experience was somewhat disappointing - having worked w/ Groovy for the last few years there is a massive difference in the approaches of the two languages. Where Groovy would often sacrifice some "internal beauty" in order to make a Java developer's life sweet and pleasant, e.g. :

• Joining a list of strings

   
     [1,2,3].join(",");
  

• string formatting using $ inside of strings

   
     "Blah blah $fooVar"
  

• Null safe dereference

   
     foo?.bar
  

... whereas Scala somehow gets stuck in an ideological mode e.g.

• Functional programming purism , or 

     List(1,2,3).reduceLeft(_ + "," + _) 
  

• bizarre nullsafe dereference comments on StackOverflow

  def ?[A](block: => A) =
    try { block } catch {
      case e: NullPointerException if e.getStackTrace()(2).getMethodName == "$qmark" => null
      case e => throw e
    }
  

  ... and usage ...

     val a = ?(b.c.d.e)
  

One part of my setup that worked very  well and I enjoyed quite a bit was the Continuous Compilation and Tapestry's Live Class Reloading. Whereas for prior Tapestry pure-Java projects I had to rely on IDE magic to do some Compile-on-Save so that Tapestry can reload the changed classes, w/ the Scala setup it was much nicer.  I set up a Maven project w/ the Scala Maven plugin , and then kick off the scala:cc goal  to make it compile the changed page classes into my project. Thus, I had a completely IDE-independent setup that gave me a live-reloading experience on-par (and possibly beyond) the reloading experience with Grails.

In the end, after I managed to work through some of the issues described above, it ended up being a pretty reasonable set up and I was able to make pretty decent progress in getting the app out the door (for my wife's birthday). At the same time, I wasn't really able to leverage any cool Scala features that would magically boost my productivity, or make the codebase significantly cleaner or smaller (in some respects, it feels like the Scala based code is more verbose because of all the conversions and casting into Java types). I feel that if I knew more about Scala and was more knowledgeable about Tapestry internals, I might be able to write a Tapestr5 - Scala adapter layer that would plug into some of T5's extensions points to make it work more naturally with Tapestry (e.g. working w/ Scala lists in views, different handling of null values, etc). As a learning experience - I learned a lot, both about things that were interesting and useful (a bit of functional programming, Java/Scala integration), and some things that I really didn't want to know that much about (how Scala and T5 munge the Java classes to make the things tick).

In any event, the advice to people who like to try this kind of integration would be to allow yourself plenty of time for learning and experimentation w/ Scala and not giving up too early ( as I was almost ready to do on a few occasions). Fanf's blog has a few blog entries and a project on GitHub that is  an excellent starting point.