Ad-Hockery functional testing : UI Elements and Selenium experience

This post is a response to Rob's posting on functional testing that just became too long to post in a comment...

-----------
At my job, we've used the UI elements approach pretty extensively and at this time we're moving away from it on a new project.

Conceptually, I've also enjoyed the idea that the location logic is sitting in the same tier as the elements that are being located. The support in S-IDE is also quite nice and impressive (e.g. code completion, documentation), the built-in support for unit testing the locators is quite nice.

Now, the downsides....

• UI Elements development / future : although it is technically a part of Selenium, in practice UI Elements support don't actually evolve in lockstep with it. On a conference call/presentation w/ one of Selenium's maintainers a few months ago, when I asked about UI elements, he indicated that they might be moved out of the core (he didn't seem to be particularly fond of them, I almost had the feeling that he was tempted to just drop the feature). There isn't much of a community using them in the first place, thus if something breaks they can remain broken for a while until someone notices (when that happened to us, the turnaround was pretty quick; although I'm not sure if that is the way it is or because the author of UI elements for Selenium was a former employee).

• The skills gap : the fact that the UI element definitions are defined outside the main "test case authoring" environment (which in our case is Groovy), makes it another distinct skill that team members (often QA folks) need to acquire. When it comes to Javascript, being a user is easy, being an author who can sling Javascript and write nifty unit tests (typically not a QA skillset) inside of the UI elements is not that easy. At least in our case it led to a bunch of not very well maintained UI Elements, w/ most of the Unit tests commented out and the knowledge of writing UI Elements withing the QA team considered almost a Dark Art. As a result of all this

• Development cadence issues : although the usage of UI elements in a test case seems very sexy, the same "development cadence" for UI Elements is kinda slow (this could be my own ignorance as well). Basically, if you wanted to enhance a UI element, you would write some Javascript to set up the UI Element definitions and then close S-IDE and open it up again. If your unit test failed, then you would get a bunch of popups about the failing tests, the UI Elements would be disabled and you're back to editing the javascript file until the unit test passes. Only after that, you can try using the UI element in S-IDE or in your test case. Finally, if you were using selenium-server to run your "real" test cases (like we did), you need to restart selenium server (RC) so that it can pick up the updated UI Elements.

• UI Element organization : this could be an artifact of our own failings w/ UI elements and not necessarily a problem w/ the framework itself; however, the lack of a well defined approach to structuring them has led us into a massive set of pagesets that contains UI elements for a whole bunch of different pages. Thus, when you try to pick up a UI element there isn't a good organizing principle ("Oh, yeah, I'm on the Login page, lemme see what I can locate there") - instead we have something along the lines of ("I'm in feature 'foo', let me see which one of the several hundred UI element in the fooFeaturePageSet might fit").

Although the Page object approach is not a silver bullet it addresses a bunch of the issues above:

• Skillset - the same skills that test authors use to write test cases, they use to write the abstraction (page objects)
• A clear organizing principle - people get it, pages have stuff in them, putting the location logic for an element on a page inside of the corresponding page object makes sense. Then, using that is just a method call
• Deployment / Cadence issues - there is no longer the impedance mismatch of "I want to run this test case, did I re-deploy the UI elements that it depends on?". You just run the test, the testing framework will recompile the modified page objects and the test case runs, no need to do anything w/ Selenium server or anything else.

Anyway, just my 2c on the subject.

My Scala (and Tapestry 5) experience

Since both a couple of coworkers asked me about Scala in the last few weeks, I thought these might be interesting to read through (something that emerged over the last few days)

•  http://codahale.com/downloads/email-to-donald.txt
•  http://codahale.com/the-rest-of-the-story/

I have obviously not used Scala to the extent that the guy describes, but I can testify at how annoying it is the constant translation between Java and Scala (especially, since my small app, , was using Tapestry 5, which is obviously a Java framework). The conversion between the two was made worse (at least for me, a total Scala newb) by the following:

• Classes with the same names but completely or subtly different usages and intents . The fact that the Scala library used the same class names as the Java library (java.lang.Long and scala.Long) , the default imports use them, and there are some magical conversions that occur between the eponymous types. On a few occasions I was totally baffled about something not working only to find that in the end, the code was getting the wrong type . Thus, in a bunch of my classes, I ended up explicitly having to import the Java classes that the framework expected to work with , e.g. 

    
       import java.util.{List => JList }
       import org.slf4j.Logger
       import scala.collection.JavaConversions._
  
       class FooPage {
  
  
   @Property
   private var catPieces:JList[ArtPiece] = _
  
  
          // explicitly declaring the returning types as the Java types
   def onPassivate():JList[String]= {
                  // and using the Scala provided conversions
    return seqAsJavaList(List(category,subCategory))
   }
  
       }
        
  

  and then explicitly use the java specific types. A similar but different situation exists with java.util.List and the scala List classes (e.g.  scala.collections.immutable.List) although they have the same name they have a completely different purpose (e.g. the Scala list is not necessarily intended to be created, and manipulated like the Java list); the equivalent of the java list is the recommended Scala ListBuffer
• Null handling -  because I was interacting w/ a Java framework, there was an expectation that nulls are OK and at different places, the framework does expect methods to return nulls in order to behave in certain ways. Scala goes for the whole Option pattern (where you aren't supposed to use nulls to make it all better) and has some conversion (that I obviously, don't fully understand) between null and these types. However, because of the interaction w/ the Java framework, I had to learn how to deal with both. It kinda sucked.
• Tapestry 5 and Scala interactions -  because Tapestry 5 pushes the envelope on being a Java framework w/ a whole bunch of annotation processing, class transformations, etc. , in some cases there were clashes between the T5 approach and Scala. In some respects, Tapestry 5 manages to be a respectable and succinct Java framework by adding a whole bunch of metaprogramming features, which when used with Scala make the scala code less attractive, e.g: 
• Page properties that would otherwise be set up as private fields in regular Tapestry 5, now have to be declared as private fields and initialized. If you didn't declare them as private, then T5 would complain (since pages can't have non-private members as it is managed by T5) , e.g. :

    
    class Foo {      
          @Inject 
   var logger:Logger = _
     
   @Inject 
   var pm:PersistenceManager = _
    }   
  

• Sometimes the T5 and Scala approaches seemed to clash in ways that make things complicated. For example, in the persistent objects in the class, I often annotated the private fields w/ @BeanProperty (so that Scala generates proper getters/setters for those fields).

    
    import scala.reflect.BeanProperty
    import javax.jdo.annotations.Persistent;
    class PersistentFoo {      
          @BeanProperty 
   @Persistent
   var title = ""
  
    }   
  

  Yet, when I accidentally did the same for some page properties at weird points the application would start failing (on application reload with Tapestry's live class reloading) until in pages I replaced the approach w/ Tapestry's @Property annotation (although they're supposed to do the same it's quirky w/ BeanProperty)

    
    import org.apache.tapestry5.annotations.Property; 
  
    class FooPage {      
          @Property 
   private var category:String = _
  
    }   
  

When I was working on the app, a few times I had to just stop for a day because I couldn't figure out how to do something massively simple (e.g. how to succinctly join a list of Strings into a comma separated string - stuff that would have taken me 30 seconds to do in Java, 2 seconds in Groovy and the proposed Scala solutions seemed like massive overkill). I originally started wanting to write some tests in Scala for this app, because I thought, "wouldn't it be nice to have something a little more flexible and less verbose than Java", but that still has nice static typing. Later I decided to try the whole Scala+T5 approach, and I have to admit I was pretty mad at myself when I would get stuck .

Obviously, many of my problems described above were due to my own weak Scala-foo (e.g. I had read through at least 2-3 books in order to be brave enough to try this just to learn that until I try things hands on, it doesn't stick too well), and other issues that I had were due to the interaction w/ the specific Java framework that I chose (Tapestry 5). Yet, in some ways, the experience was somewhat disappointing - having worked w/ Groovy for the last few years there is a massive difference in the approaches of the two languages. Where Groovy would often sacrifice some "internal beauty" in order to make a Java developer's life sweet and pleasant, e.g. :

• Joining a list of strings

   
     [1,2,3].join(",");
  

• string formatting using $ inside of strings

   
     "Blah blah $fooVar"
  

• Null safe dereference

   
     foo?.bar
  

... whereas Scala somehow gets stuck in an ideological mode e.g.

• Functional programming purism , or 

     List(1,2,3).reduceLeft(_ + "," + _) 
  

• bizarre nullsafe dereference comments on StackOverflow

  def ?[A](block: => A) =
    try { block } catch {
      case e: NullPointerException if e.getStackTrace()(2).getMethodName == "$qmark" => null
      case e => throw e
    }
  

  ... and usage ...

     val a = ?(b.c.d.e)
  

One part of my setup that worked very  well and I enjoyed quite a bit was the Continuous Compilation and Tapestry's Live Class Reloading. Whereas for prior Tapestry pure-Java projects I had to rely on IDE magic to do some Compile-on-Save so that Tapestry can reload the changed classes, w/ the Scala setup it was much nicer.  I set up a Maven project w/ the Scala Maven plugin , and then kick off the scala:cc goal  to make it compile the changed page classes into my project. Thus, I had a completely IDE-independent setup that gave me a live-reloading experience on-par (and possibly beyond) the reloading experience with Grails.

In the end, after I managed to work through some of the issues described above, it ended up being a pretty reasonable set up and I was able to make pretty decent progress in getting the app out the door (for my wife's birthday). At the same time, I wasn't really able to leverage any cool Scala features that would magically boost my productivity, or make the codebase significantly cleaner or smaller (in some respects, it feels like the Scala based code is more verbose because of all the conversions and casting into Java types). I feel that if I knew more about Scala and was more knowledgeable about Tapestry internals, I might be able to write a Tapestr5 - Scala adapter layer that would plug into some of T5's extensions points to make it work more naturally with Tapestry (e.g. working w/ Scala lists in views, different handling of null values, etc). As a learning experience - I learned a lot, both about things that were interesting and useful (a bit of functional programming, Java/Scala integration), and some things that I really didn't want to know that much about (how Scala and T5 munge the Java classes to make the things tick).

In any event, the advice to people who like to try this kind of integration would be to allow yourself plenty of time for learning and experimentation w/ Scala and not giving up too early ( as I was almost ready to do on a few occasions). Fanf's blog has a few blog entries and a project on GitHub that is  an excellent starting point.