MattHicks.com

Programming on the Edge

Web Development Shouldn't Be Hard!

Published by Matt Hicks under on Thursday, November 03, 2011

I've spent most of my career developing in Java for the web. I've used the full range of frameworks: Servlets, JSPs, Struts, JSF, Wicket, and dozens more. Early in my career I did a lot of Swing development. Now I grant there are a lot of inherent problems there as well, but nowhere near the painfulness of web development. At least in Swing you just have to know Java. In the majority of web frameworks you have to know HTML, CSS, JavaScript, whatever your server-side technology is, and then you have to deal with the client-server model and even stateful / stateless decisions. It's such a mess and although frameworks like Vaadin do a phenomenal job of removing much of that pain, the norm in the Java world is still focused towards a complicated web where you have to deal with cross-browser compliance, HTML 5 compliance, and a slew of crappy languages most developers would rather gouge their eyes out than have to stare at all day long.

I've recently been looking at templating frameworks for larger front-facing web sites as larger organizations like where I work have entire departments to create the HTML and design the pages and then the developers have to hook up the dynamic elements. We've been talking about moving more toward Wicket, but as I've been using Wicket more and more I find myself liking it less and less. I dislike the one-to-one mapping that has to take place between the template and the Java code first of all. Secondly, the more complex features are not at all intuitive (at least to me...I'm sure some think it's the most rational thing ever and will readily tell me so). However, I very much like the concept that designers can work completely independently of the developers and can create the markup and even test it with bogus information that is replaced at runtime with production content. To that end I started thinking about what I could see as a better templating system. I like templating frameworks like Mustache, but unfortunately for many cases it's just not powerful enough. After further thought I came up with a similar concept to Wicket that would parse through XHTML content as XML and then provide components that would manipulate those Elements based on the HTML "id" associated. This means that designers assign ids to their elements (like many would already for CSS and JavaScript to access) and then the developer can opt to interact with these elements when the page is being rendered.

For this prototype I opted to use Scala as native XML support made it so much easier. So I started with the idea of a "Component" with the simple concept of Element to Element conversion:

import xml.Elem

trait Component {
  def render(elem: Elem): Elem
}

I then created a couple simple implementations of the Component:

import xml.{Text, Elem}

class Label(textFunction: () => String) extends Component {
  def this(text: String) = this(() => text)

  def render(elem: Elem) = {
    val text = textFunction() match {
      case null => ""
      case s => s
    }
    Elem(elem.prefix, elem.label, elem.attributes, elem.scope, Text(text))
  }
}

import xml.Elem

class Content(content: Elem) extends Component {
  def render(elem: Elem) = Elem(elem.prefix, elem.label, elem.attributes, elem.scope, content)
}

The Label component simply replaces the content of the element with the supplied text (offering function support for more advanced usage) and the Content component even more simply just replaces the content of the element with the passed content.

Now I needed a way to map the Components to ids and render the XML:

import component.Component
import xml.{Node, Elem}
import xml.transform.{RuleTransformer, RewriteRule}
import java.lang.ThreadLocal

object Xemplate {
  private val components = new ThreadLocal[Map[String, Component]]

  def apply(components: Map[String, Component], elem: Elem): Node = {
    this.components.set(components)
    val result = apply(elem)
    this.components.set(null)
    result
  }

  protected[xemplate] def apply(elem: Elem): Node = {
    val transformer = new RuleTransformer(new Rewriter(components.get()))
    transformer(elem)
  }
}

class Rewriter(components: Map[String, Component]) extends RewriteRule {
  override def transform(n: Node) = n match {
    case elem: Elem => convert(elem)
    case node => node
  }

  def convert(elem: Elem) = {
    val id = elem.attribute("id").map(ns => ns.text).getOrElse("")
    components.get(id) match {
      case Some(component) => component.render(elem)
      case None => elem
    }
  }
}

Fortunately Scala has some really powerful features to do exactly this. The RuleTransformer and RewriteRule classes provide the majority of the functionality I need immediately out of the box. I simply take in a Map of id to Component mapping along with the XML to process and when I find an element that has a mapped id I simply replace it with the Component's render result. I called it Xemplate just because I was being lazy and it's XML templating.

At this point I needed a way to simplify bringing this together into an individual page rendering so I created a Page class:

import component.Component
import xml.Elem

class Page(xhtml: Elem) {
  private var components = Map.empty[String, Component]

  def register(id: String, component: Component) = components += id -> component

  def render() = Xemplate(components, xhtml)
}

Now I have ability to instantiate a Page, register Component instances to ids, and then render to get the modified XML. Lets see an example of this in action:

val xhtml = 
  
      
  </head>
  <body id="body">Original content</body>
</html>

val body = <p>Yes, I can simply inject XML!</p>

val page = new Page(xhtml)
page.register("title", new Label("Example Title"))
page.register("body", new Content(body))
val modified = page.render()
println(modified)</pre>
<p>
We simply create a Page instance with our XHTML content, register a Label for the title with "Example Title" as the text, and set a Content component for the body id.  This is the resulting content we should see:
</p>
<pre class="brush: xml;"><html>
  <head id="notfound">
      <title id="title">Example Title
  
  

Yes, I can simply inject XML!

Lets get a little bit more complicated though. One of the most painful things in a framework like this is often cycling over a list of elements and trying to populate them onto the page. In Wicket this is relatively effectively done, but you often end up with a lot of anonymous subclasses layering down so you have a reference to the current item being rendered in the list. I took a different approach to this by creating what I called an EntryHolder that is responsible for holding the current instance being iterated over:

import java.lang.ThreadLocal

class EntryHolder[T] extends Function0[T] {
  val current = new ThreadLocal[T]

  def apply() = current.get()
}

As you can see this is an extremely simple wrapper over ThreadLocal. For the purposes of testing I could have just had a var, but if we ever want to handle a web environment we would presumably have several threads handling requests at the same time and obviously would want to run the risk of transferring one pages' content to someone elses' page.

Now we need to create our Sequence class that allows us to iterate over items:

import xml.Elem
import xemplate.{Xemplate, EntryHolder}

class Sequence[T](items: Seq[T], holder: EntryHolder[T]) extends Component {
  def render(elem: Elem) = {
    val content = for (item <- items) yield {
      holder.current.set(item)
      for (n <- elem.child) yield n match {
        case elem: Elem => Xemplate(elem)
        case node => node
      }
    }
    Elem(elem.prefix, elem.label, elem.attributes, elem.scope, content.flatMap(s => s): _*)
  }
}

Though more complex than Label or Content, it's still relatively simple. We iterate over the items, set the current item to the EntryHolder, and then ask Xemplate to process the child elements so they can be templatized as well.

Lets see an example of this:

val xhtml = 
  
val page = new Page(xhtml) val names = List("Alan", "Brad", "Chris", "Don", "Edward") page.register("names", new Sequence(names, NameHolder)) page.register("name", new Label(NameHolder)) println(page.render())

object NameHolder extends EntryHolder[String]

I simply instantiate my Page with the XHTML content, create a List of names, register "names" to a new Sequence with the names list and NameHolder, then I register "name" to a new Label with NameHolder as the text function. Notice above that EntryHolder extends Function0[T] allowing us to pass it in as a String function to generate the String. What this does is the function is called per iteration on the names list and NameHolder has the current name associated and returns it. This keeps us from having to deal with layering our components for visibility by creating companion objects that are visible for a specific function.

The resulting XHTML should look like this:


      
  • Alan
  • Brad
  • Chris
  • Don
  • Edward

That's about it. Again, where possible I would still recommend using something like Vaadin as it keeps you from dealing with the majority of the evils of the web, but I think this little example demonstrates an extremely simple and fast (performance tests on a 1 meg XML file averaged around 25ms per render) templating system that doesn't have to overcomplicate or obscure development. If anyone is interested in turning this into a full-fledged project feel free to do so and build on it. I would appreciate the credit, but the code above is free to use for anything you see fit.

Why is Scala more difficult than Java?

Published by Matt Hicks under , , , , , on Wednesday, December 08, 2010
I was pretty exclusively a Java developer for twelve years before making the switch to Scala. Yes, it was difficult to learn some of the syntactical differences, but now that I have I don't want to turn back. No, this is not a baited question to get a flame-war going, but an honest attempt to get some real feedback by those that believe Scala is more difficult than Java.

Let me lay some ground-rules before we get this discussion going though. First, complex topics in Scala are complex, there's no debate there, but that doesn't make Scala more difficult than Java, there are just more possibilities to what you can accomplish. I say that the argument cannot be made that Scala is more complex if there is not a side-by-side comparison to Java code that is much simpler to understand. This means we have to leave the advanced concepts of implicit conversions, multiple inheritance, and so much more that Scala can do, but are features that bring it beyond the capabilities of Java. Yes, of course you can make an argument that in switching to a language like Scala you will eventually run into these scenarios, but I am speaking purely of the introductory programmer and the "difficulties" between Scala's syntax versus Java's.

Now, can someone give me a scenario of code in Java that would be more complicated to understand in Scala?

Learning Scala: scala.Either

Published by Matt Hicks under , on Thursday, October 21, 2010
I would consider myself pretty competent with Scala at this point, but there is still so much I have yet to learn. Something I always used to do in Java was to work my way one class at a time through the JavaDocs of the language I am attempting to do with Scala now as well.

I've just run across the Either class in the ScalaDocs and really like the idea.


The idea is that you may have two different return types that may result from one method. Classically that is solved with all sorts of hackery or returning the common superclass to the two and then doing an instanceof check, but in Scala the Either class provides a much more elegant solution:

object Test {
 def main(args: Array[String]): Unit = {
  numOrString("5") match {
   case Left(s) => println("String: " + s)
   case Right(i) => println("Integer: " + i)
  }
 }
 
 def numOrString(s: String) = {
  try {
   Right(s.toInt)
  } catch {
   case exc: NumberFormatException => Left(s)
  }
 }
}

I'm not really sure how practical this example I wrote is, but my method "numOrString" takes in a String and attempts to convert it to an Int. If it can return an Int it does so or it returns a String. By use of the Left and Right subclasses of Either you can define which type of result you are sending back. Finally, with Scala's awesome matching you can easily determine which type of result you got.

The Pitfall of Programmer ADD

Published by Matt Hicks under , , , on Tuesday, July 27, 2010
Programmer ADD: The (often external) push to work on many projects at a time rather than focusing on and accomplishing one task at a time.

I currently work for a small company and unfortunately that means I'm constantly being pulled in many directions. There just aren't enough people to do everything, but lately I've started noticing the effect this has had on the quality of work I do. Since I'm constantly pushing to finish up what I'm working on in order to start on the next project on my plate I often don't take the time really necessary to quality check and test the functionality I've added or changed. This results in work that is not as well thought out as it should be, and not as well executed as I would like.

I can definitely point a finger to my boss for pushing the amount of work I have to accomplish in a given time-frame, but realistically even when I've worked for larger companies this has still been more-or-less true of all development work I've done. As developers we're constantly being pushed for deadlines and getting bugs fixed. This might be great from a management perspective on "work accomplished", but I think this really has a negative impact on the company in the long-term as we aren't really given the time necessary to nurture our projects to write solid and well-tested code, but just enough to get the job done. The result? Code that eventually has to be repaired or updated at twice the effort as it would have taken to do it right in the first place.

This is basically just a rant as I see the quality of my work declining as deadlines push me to do more than I can safely accomplish, but it also raises my curiosity if this is something most other developers deal with as well?

Eclipse Helios (3.6 RC4)

Published by Matt Hicks under , , , , , , , on Monday, June 14, 2010
I've been a user of Eclipse for many years now and when I saw there was a free t-shirt to be had by reviewing the newest release there was only one choice to make. :)

The download is exactly the same as all previous releases but the first thing that struck me was the startup time. Granted I have no plugins installed yet, but Eclipse was up in just a few seconds and that already has me excited. Performance has always been the biggest problem in Eclipse and if they've made major strides in that direction I'm a happy camper.

Granted this is RC4, but immediately after downloading and unzipping I checked for updates and it spend the next 5 minutes downloading a pretty large update to the Eclipse SDK and made me restart when it was done. Again, RC4, so lets just hope the actual releases are up-to-date. :)

Installing MercurialEclipse and the Scala plugin went smoothly and I went to pull down Sgine (http://sgine.googlecode.com) and all was fine except Scala can't seem to build the project correctly. My guess is this is a bug in the Scala plugin with Helios.

After spending a bit more time with it I haven't really seen much difference in 3.6 and 3.5 apart from some performance improvements. However, performance alone is a very worthwhile feature.

Still Alive!

Published by Matt Hicks under , , , , , , , , , , on Saturday, May 01, 2010
Well, this is a great title for this post not only as it's the first blog post I've made here in seven months, but I've also had the song from Portal stuck in my head lately. :)

Looking at my blog it would be easy to classify me as MIA but I've actually been quite busy in the open-source and have even been blogging quite a bit lately, just not here, but I'll get to that in a minute.

First of all, my very basic last post talking about the entry-level learning I was doing with Scala was quite fitting as my last post as very shortly after that post I made a complete shift from Java to Scala programming. With this came the abandoning of lots of open-source projects I've either created (like jSeamless) or been a developer on (like jMonkeyEngine). I believe so strongly in Scala that I've shifted everything I possibly can to Scala.

Second, with the shift I've made to Scala and all the projects I've abandoned as a result, one new project has come to being that, in many ways, embodies much of the vision I've had for several projects I've left behind. That project is called "sgine" (Scala Engine). I've been developing on it for a while now and have made some major strides recently and finally wanted to make a re-appearance on my blog to hopefully start posting here again.

Now, at the beginning of this post I said that I've been blogging elsewhere. If you go to http://www.sgine.org you'll see that it's a blog filled up with posts regarding updates to the engine I've been developing. I'm really excited about this project and though many people have accused me of abandoning Java...and realistically that's pretty much what I've done...I haven't a regret for switching to Scala, it's such an incredibly powerful language and has renewed my vigor in programming. For anyone considering looking into a new language I cannot recommend it enough.

Loops in Scala

Published by Matt Hicks under , , , on Thursday, October 01, 2009
Lately I've been learning the ins and outs of the Scala language and I have to say, having programmed in Java for the past several years, that Scala is scratching where I itch. It is a lot to grasp and mind opening, but all-in-all I see Scala as what Java could have evolved to if they would have just been willing to ditch some backwards compatibility and fix some stuff rather than just adding more and more work-arounds.

Anyway, that's not the point of this post. I wanted to post some cool loop examples I've been playing with in Scala.

In Java we have the standard incrementing for-loop:
for (int i = 0; i < 10; i++) {
...
}

In Scala we have something similar, but I believe more elegant:
for (i <- 0 until 10) {
...
}

However, with Java 1.5 we got the enhanced for-loop when we don't care about indexes:
for (String s : listOfStrings) {
...
}

In Scala we do something similar, but with a more consistent syntax:
for (s <- listOfStrings) {
...
}

Or, we can go a step further with Scala since it's a functional language we can get rid of the for loop entirely and use a friendly method available to us called "foreach" and pass each entry to another method:
listOfStrings.foreach(doSomething);

def doSomething(s:String):Unit = {
...
}

To me this is of significant appeal because we don't end up with a bunch of embedded for-loop blocks in our methods we focus more on modular functionality to deal with the results of the iteration instead.