An intentionally provocative heading, but one I stand behind until someone can prove otherwise (and I welcome just that).  Scribe 1.x was pretty fast (http://www.matthicks.com/2017/01/logging-performance.html), but was not written with performance in mind.  When I came back around and realized just how fast log4j2 is, I could see no reason why a Scala logging solution shouldn't be able to run circles around it.

Note: the benchmarks below are tested using simple file logging to represent the most common production logging scenario.

Log4J Comparison

Now, for people like me that often want to skip all the blustering, explanations, and bravado, I'll cut straight to the result graph of the performance comparison between log4j 2.10.0 and Scribe 2.0.0:

Like I said, log4j2 is fast, but Scribe 2 is just shy of a 50% performance boost on that.  If we look at the over-time graph it provides additional insights:

You can see that log4j does well, but operations increase in cost partway through, while Scribe is smooth and steady for the entire run.

Method and Line Numbers

This is all compelling but practically speaking, 99% of applications won't even notice the difference between 1,500 nanoseconds and 800 nanoseconds per record logging.  Thus far I've been comparing Apples to Oranges though.  In Scribe, the method and line numbers are integrated into the log records by default.  If we add that to the logging (which any reasonable logging framework should have) we see a different story:

Shocking, right?  If you know Java, you'll probably realize why this is.  In Java, the only way to get access to method and line numbers in your logging is by walking the stack trace.  This is a very expensive operation to be doing and slows the logging to a crawl.  This is why it's not on by default, and why you don't see this talked about much in Java logging frameworks.  However, Scala Macros come to the rescue!  In Scribe, we get all of that for free.  The method and line number information are extracted at compile-time so you can see that Scribe is just as fast whether method and line number logging is enabled or not.

Typesafe Scala Logging

In 1.x my focus was the comparison with Scala Logging.  As you can see below, we're not even in the same ballpark:

It's a pretty pathetic comparison.  In Scribe, 1.x Scribe was noticeably faster, but this shows just how inefficient Scala Logging is.

All Loggers

If you'd like to see the side-by-side comparisons of all the competitors:

Though the log4jTrace and scalaLogging results dwarf the performance distinction between log4j and Scribe, if you've seen all the results here, Scribe is the clear performance winner here.  I'll make another post to show why it's also more powerful, but I'll save that for another day.

Sources

I'm not a master with log4j or Scala Logging, so it's possible I'm not giving ideal running circumstances for these loggers to "show their stuff".  The original source code is posted in the repository: https://github.com/outr/scribe/blob/master/benchmarks/src/main/scala/scribe/benchmark/LoggingSpeedBenchmark.scala

The JMH benchmark results were also saved to a JSON file if you want the raw results to make sure I'm not trying to pull a fast one: https://github.com/outr/scribe/blob/master/work/benchmark/2018.01.31.benchmarks.json

Getting Started

To learn more about Scribe and start using it, check it out here: https://github.com/outr/scribe

Please feel free to email me if you find any mistakes in anything I've said to set me straight, or you can publicly shame me by writing a comment on this post.  I welcome criticism and praise, but criticism gives me something to learn from, so it is more appreciated.

I've never been a fan of the setup of logging frameworks as far back as when I was a Java developer.  The hassle and complexity of configuring and managing the logging framework was always a big hassle and would often create serious problems in the application if not done right.  Even today in Scala it doesn't feel much better.  Certainly we have Macros that give some additional compile-time optimizations, but it's amazing how little has changed.

A while back I created a Scala logging framework called Scribe.  Honestly, the primary reason had more to do with giving greater flexibility to control configuration in code over performance or anything else.  However, by not building on top of log4j, logback, slf4j, or JUL, I found that the system was not only far more simplistic and configurable, but it was also faster.

I recently did a comparison between Lightbend's Scala Logging framework to see how it performs.  Without any additional optimizations the results were pretty impressive.  I configured both Scribe and Scala Logging to avoid writing to standard out as that would be the primary bottleneck of performance and simply wrote a custom Writer / Appender that would simply count the log entries:

Over sixty seconds I recorded how many records could be logged.  Scala Logging was able to log 474k records and Scribe was able to log 610k.  Now, obviously this is far more logging than any reasonable application should be doing, but the point was to prove that not only can Scribe keep up with the most popular Scala logging framework (Scala Logging) on top of the most popular Java logging framework (Logback), but it quite a bit faster.

The second thing I measured was memory consumption over the run.  Memory usage is a very important factor with regard to logging as it should have a very small footprint to give maximum allocation to the application itself.  Again, my finding were pretty strongly in favor of Scribe:

While Scala Logging utilized 704kb of memory Scribe utilized on 596kb.  Again, not a substantial difference when we're discussing hundreds of thousands of records being logged, but this is meant to prove that Scribe is the better performer.

Very often I hear that a developer won't use a framework because there aren't enough developers using it.  Obviously this creates a chicken / egg situation as you can't get developers using it because not enough developers use it.

Hopefully this short post will give some additionally credibility to the value of Scribe that people will start comparing features and see just how much it has to offer.  If there's something missing that your existing logging framework has, just create a ticket.

To my relatively small number of followers, it should come as no surprise that my biggest failing is actually getting much visibility for my vast number of open-source projects I write.  To that end I've been doing some research and asking other developers to give me some suggestions on how to get more visibility for me projects.  I've made it a goal for this year to get more visibility for my open-source projects so they will hopefully reach a larger audience.

I decided that I would post the results of my research in a blog post so hopefully others can benefit from this research as well, or at least I can have an easy place to reference back to it for myself.

Speaking / Conferences

The most common suggestion for visibility for a project was speaking about it both in local groups and at conferences.  Several developers said that's how they learned about a useful framework they are using.  This is a good idea and something I'm planning on doing more of this year, even though I really prefer to write code than talk about it, but this is a bit of a longer-term objective than immediate visibility.

Awesome

There are lots of "awesome" lists. Often more than one for each language. This is a no-brainer presuming you can get someone to merge your pull request: https://github.com/bayandin/awesome-awesomeness

Reddit

Apparently there are people that use that site and a common source of information for developers.  I must admit, I haven't been a big user, and this is something I think is going to have to change.  I'll have to get involved in the Scala group and reference my projects there.

Twitter

Yet another thing I am terrible about not keeping up with.  For a very long time I simply had all of my GitHub commits logging to Twitter, but I'm thinking that personalized messages on Twitter with relevant hashtags might be a very good way to get my projects noticed.  Many developers I talked to get most of their news and information from Twitter.

Mailing Lists

Though a bit old-school, this is something a lot of developers still follow and perhaps a good way to get visibility directly into people's inbox of a new framework that should be using.

Based on this research it's clear that my aversion to social media has a direct correlation to my difficulty in getting visibility for my projects.  Shocking right?  I suppose the moral of the story is, if you don't like people, don't expect them to like you either.

I often speak to clients and developers that are pushing the Play Framework as the ideal web framework when developing Scala web applications.  I started considering why Play is the framework that people tend to settle on, especially large companies.  I think there are a few reasons:

1. It's supported by Typesafe *cough*, I mean Lightbend. Certainly Lightbend is the main company supporting Scala and to many people that means you should use whatever technology they are pushing.
2. It has commercial support. This is a bleed-over from point #1, but I think warrants its own point.  Especially for big companies having the ability to get commercial support is very important.  It is unfortunate that truly open-source projects that are entirely community driven are seen as riskier, but typically that is the mentality.
3. Play is actively maintained. It has been around for years and is sure to be around and maintained for many to come. Having a long history seems to give a lot of credibility to frameworks whether they deserve it or not.
4. It's huge! There are a ton of modules and features that Play has available for your various needs and that provides a compelling option to companies that want to grow and expand and may need those nifty features in the future.
5. Easy to get going. The handy-dandy Activator makes creating and managing your application stupid simple (with an emphasis on "stupid").  Especially for people building initial prototypes and evaluating the options available often find this compelling.  Activator helps get them going fast without much fussing with the build tools or framework.
6. Big companies use it so it must be good! LinkedIn and the Guardian use it and certainly others, so it must be a good choice, right?

These are all important and mostly valid points, but for a developer that has created many web frameworks over the years and has built his fair share of web applications most of these points are not as valid as it might seem on the surface.  I'll attempt to address each of them and hopefully give some insight into why I think Play Framework is not the right tool for the job and no matter what you are writing in Scala there are better options available.

1. Lightbend backing the framework is definitely a solid point in favor of the framework, but it doesn't mean it's the best framework for Scala. For example, Lightbend also supports Slick, but there are serious performance problems in that framework which led me to create ScalaRelational.  For an overview of the crazy performance distinctions between the two frameworks see here (this just scratches the surface, there are simple queries that should take milliseconds to complete that take minutes and up to hours to run).
2. Though perhaps one of the most compelling points in favor of Play Framework, commercial support doesn't mean the primary active development on any "supported" framework by Lightbend is being done by Lightbend.  These are primarily open-source frameworks that Lightbend has taken it upon themselves to commercially support and most of the actual development is still being done by the open-source community.
3. Actively maintained is a very good sign in any framework.  Not actively maintained frameworks are incredibly risky unless they have a specific function and aren't currently being maintained because they fulfill that function well and don't need further development.  For a web framework that's certainly not the case.  The Spring Framework for Java is still actively maintained, but it's a monolithic and painful platform that inspired self-loathing and hate.  Many projects start off good or at least good intentioned only to eventually expand themselves into something that has long since lost the focus and benefits of its original intent.  I wouldn't say Play has gotten as bad as Spring, but once you get past the dead-simple getting started tutorial, the nitty-gritty of the framework starts to seep into your bones causing maintenance and maintainability nightmares.
4. Though having support for the various features you may eventually need built as modules to your framework is very nice, it also often results in the framework itself trying to solve too many problems and ending up solving nothing very well.  I try to focus on using frameworks that specifically focused to a specific problem and solve that problem well.  When a framework tries to solve too many problems it starts making compromises and watering down the overall benefits of the framework.
5. Oh Activator, how I loathe thee!  I'm sure it seemed like a good idea to create a shiny wrapper around SBT to help create your project, run it, and even deploy it without being bothered with understanding how building your Scala project actually works.  However, it's when you need to manually do something and you realize you don't have a clue how the stupid thing works that it becomes a maintenance nightmare.  I'm a software developer, and software developers should be smart enough to create a new project from scratch without needing a tool to hold your hand.  If you aren't smart enough to create your build from scratch then you're either an idiot, or the tools are far too complicated.  Instead of dumbing down the complex tools by wrapping it, why not attempt to solve the problem by making the tools themselves less painful to learn?  Yes, I'm talking to you SBT!  What used to be Simple Build Tool has had to change its name to stop the laughter in derision (now Scala Build Tool).  SBT has definitely improved recently, but there is still far too much complexity to manage and maintain.
6. Big companies make stupid decisions constantly.  Web frameworks are one the primary areas in which they make stupid mistakes.  You may argue, "but their site works, so it must have been the right choice", but take a look at Facebook's architectural history and you'll see a good example of sheer stupidity leading down a path of pain and the only way they could stay afloat was due to their popularity they could keep throwing money at it.  Don't use big business as an example of what you should do.  Most companies can't just keep throwing money at a solution until it works, and even ones that can't would be better off spending that money on other things and keep their programmers sane.  I've worked for and more recently have consulted for lots of big businesses and even for the US government (the biggest and most wasteful business in the world).  I try to avoid consulting for big companies when I can now for this very reason.

These points hopefully address at least the primary reasons many people choose Play Framework, but the answers themselves aren't evidence of why it shouldn't be used, they are simply responses to the common reasoning I hear.  My specific reasons why I try to avoid Play are a bit different:

1. Templates with their own dialect and learning curve and injecting code directly within.  This is just wrong on so many levels.  The time it takes to learn Play's specific template dialect is problematic, the injection of code in your templates make things incredibly messy, and don't get me started on the nightmare of trying to refactor code.
2. Play Project Structure. Why? Seriously! There is no logical reason to break from the standard of normal Scala projects "src/main/scala" to the mess that is Play's arbitrary and rigid project structure.  Not only does this further increase the learning curve, but it makes your code very different from every other Scala project.  Again, why?  There is no explicit gain from their mediocre project structure changes, and in the latest version you can override it, but it says it's "experimental" and might cause problems.
3. Routes file.  Again, why?  It's a freakin' text file that compiles to Scala.  It's unfortunate that Scala doesn't have a powerful DSL that would make more sense than writing a text file...oh wait, it does!  I have spent a great deal of time contemplating this and I really have no idea why they would do this.  If we were talking about a configuration file that could be changed in production without rebuilding your project that would be one thing, but the routes file is compiled and built into your project.
4. Performance is bad.  Six months ago I was working on a project and they were experiencing serious performance issues in their web services.  They were using Play and PlayJSON.  After a little bit of testing I found that the major bottleneck was Play.  I quickly swapped out for Spray and uPickle and the performance was then lightning fast, it was fewer lines of code, and the code was much clearer and easier to maintain.  Now, I can't generically defend that Play's performance is always bad, and it's possible in the latest releases that it is no longer an issue at all, but the framework has been around for years and there is just no excuse for serious performance issues like that making it into the code.
5. Distraction from simplicity.  When I work on a project my mantra is, "make it as simple as possible, no more and no less".  Developers working on projects have a tendency to over-complicate things and especially among Scala developers attempt to be clever and end up creating a maintenance nightmare for themselves.  Developers also focus too much on simplicity and leave glaring holes in functionality or paint themselves into a corner that they can't code themselves out of.  A project should be as simple as possible while keeping in mind the overarching goals and a consideration of where the project is going.  Play is not a very flexible framework and when you need to diverge for business needs they make it painful to do so.

Wow, I got through an entire post without writing any code.  I think this is a first.  In summary, yes, I've spewed a lot of hatred towards Play Framework, and though most of it is justified, it is not a horrible framework.  I think there are better frameworks like Scala.js, Spray, Finagle, and possibly even Akka-HTTP now if they've resolved some of the missing features and performance problems.  I know it's a Java framework, but I've also been pretty impressed with Undertow.

Play Framework developers: If you read this, I apologize for the harshness, but hopefully this can be taken as constructive criticism and I would happily recant in a future post.

I've been writing a pretty large mobile application for a client recently and am finally getting close to releasing it, but thought it might be worthwhile to write a little bit about the experience I've had.

The Frameworks

I expected jumping into development for Android and iOS would be fairly straight-forward. Mobile development has been around for quite a while now and there dozens if not hundreds of tools to get the job done.

I began researching frameworks that would allow me to have a shared code-base across Android and iOS and was disturbed at my options:

Option 1: Write Two User Interfaces

Using RoboVM you can deploy your JVM-based application to iOS without any noticeable performance penalty, but you must still create your user interface for each separately.  In many cases this is preferred if you want to maintain compliance for device-specific standards.  However, in my case the client had a very rigid design that they want to see on both devices and writing it twice just seemed painful and wasteful.

Option 2: Phonegap

There is a huge trend for Phonegap and derivatives that let you write your mobile application in HTML and JavaScript and then deploy it to Android and iOS.  In some ways this is a compelling option since you have standards-based tools that you can use to create your user interface and can even potentially reuse some functionality from your web application.  However, on closer inspection the performance is horrible and the UI feels very clunky.  I have yet to see an application written using this methodology that feels like a native application.

Option 3: Write a Game

I'm fairly well known for taking the road less travelled and will write entire libraries to do something "the right way" in my opinion.  After rejecting the above options I realized the only common UI target between Android and iOS is OpenGL ES.  This is the graphical library games are built on and though it might seem like a crazy idea, I created a simple prototype to see if I could write a UI framework on top of it.  Though incredibly painful to write the preliminary foundation, it worked quite well.

Writing an Engine

In an effort to support my app development needs I began to write a framework that provided complete UI functionality using OpenGL ES so I could have a common User Interface.  At this point that framework has been developed out pretty far and I'm extremely happy with the results.

LibGDX

I chose to build on top of LibGDX as it provides some basic building blocks for UI already, although extremely limited and focused toward game development.  I build my framework to add all the niceties for transitions, components, dynamic font loading, etc. to provide for my UI needs.  One of the big wins here is that I am able to rapidly test on the desktop without having to run an emulator or deploy to a device.

PSD to Scala

I love writing Scala code and decided to build this application entirely in Scala. However, the design mockups were created in Photoshop.  This is a common problem converting from design to implementation.  I decided to write a tool that would load the PSD file and allow me to customize options and then export to generate a screen in Scala.  This provided me with the ability to take a design mockup and create a perfect 1 to 1 representation in the final product.  Additionally, this saved me a massive amount of time as I simply needed to integrate the real data with the design then.

Bitmap Font Service

One of the biggest problems with User Interfaces in OpenGL is drawing fonts. The most common way is to use Bitmap Fonts that is a rasterized image representation of all of the characters instead of the vector data you get with a true-type font.  In order to properly support this I created a web service that the mobile app could request fonts from and it would dynamically generate the Bitmap Font on-demand to deliver back to the mobile app. The mobile app could then cache the data for future use.

Scaling for Screens

Another huge problem on mobile devices is that the screen sizes are so wildly different from one device to another.  After lots of consideration I wrote a scaling framework that takes in a virtual width and height that the application is designed for and the content is automatically scaled to properly and intelligently fit to the screen.  For raster images this means some dynamic scaling or multiple size options if pixel perfection is desired.  For fonts it simply means that the font size is scaled before making a request to the Bitmap Font Service so that the font is properly scaled according to the dimensions available to it.  The result is an incredibly clean looking UI at any resolution.  Additionally, since I only have to think about my virtual width and height, layout is incredibly easy.

Next Steps

At this point I now have a pretty powerful engine for mobile application development. One that rivals any framework I have seen to-date.  The question is: what do I do with it now?

I'm considering open-sourcing it to allow anyone to use it, but it's entirely written in Scala, so the audience that would take advantage of it is relatively low.  Another option is to continue to improve it as a closed-source project and focus more on providing OUTR Technologies the capability of creating more powerful mobile applications.

Update

Almost immediately after posting this I received several requests from people that were very interested in using the framework so here you go:

https://github.com/outr/outrgl

It's very rough and needs lots of documentation as it was written entirely for internal use but that will change as I go forward. I'll try to create a G8 project to help create new projects in the near future. 

I've worked with and for a lot of companies over the years and with the ones that actually care about unit testing Mocking (Mockito or some other variation) typically quickly enters the scene. My argument is that in proper coding you should be able to write proper unit tests without any mock objects. I'll go a step further and say that encouraging mocking in your tests actually encourages poor code quality.

What's the purpose of Mocking?

Mocking is extremely common in unit tests, but why?  The basic idea is that you are attempting to test one specific "unit" of functionality, but you don't want calls to the database, third-party calls, or other side-effects creeping into your unit tests. On the surface this seems like a valid use-case.

Why is Mocking bad?

There are many problems related to mocking that I'll try to quickly step through.

Unnecessary Complexity and Decreased Modularity

Though the purpose of mocking is actually to reduce complexity it very often increases the complexity of your unit tests because you often end up with confusing and complicated mocking to avoid running aspects of your code that you don't want executed in your tests.  You often end up pulling your hair out trying to figure out how the internals of the system are supposed to work in order to get the mocks to operate as expected and then all your tests start breaking as soon as that internal logic changes.  You often end up in a scenario where your code works fine because it's written modularly to continue working but your unit tests break because they are essentially implementing the internal functionality via mocks.

Unit tests testing the Mocking framework

In extremely complex unit tests or even simpler unit tests that are managed over a long time or through many developers it becomes more and more confusing what the purpose of the test actually is. I can cite dozens of examples where I've been auditing unit tests only to find out that the unit test does nothing but test the mocking framework instead of the code.

Need for Mocking Represents Bad Code Separation

This is what my entire complaint really boils down to.  If you actually need mocking in your unit tests, your actual code that you're testing is poorly written in the first place.  You should be writing code in proper units to allow individual testing without the need for mocking at all.

For example:


You might have a scenario similar to the code above and you want to test the checkout functionality of your system, but you don't want the payment gateway to be called and you don't want it actually persisting to the database so you might mock those out to avoid those being called. Instead, consider the idea that there are four units of functionality clearly defined in this case:

1. Validating the Order
2. Calling the Payment gateway
3. Persisting the information to the database
4. Creating a Receipt

Herein lies the problem. You have tight coupling in your code of dissimilar things making mocking necessary. If instead you were to refactor your code to look more like this:


You can see now that each unit of functionality has been broken out into its own method that can be individually tested as needed without the necessity of mocking. Further, the code is significantly cleaner and more maintainable as a result of this refactoring. Not only does it make our unit tests cleaner and easier to maintain without mocks, but it also enforces a higher level of code quality through modularity in your code than was there before.

If you ever decided that the checkout method needed to be unit tested you could refactor it so it takes in a PaymentGateway implementation and Persistance implementation as arguments to avoid side-effects and allowing a test instance to be created to fulfill those needs. Though this is a form of mocking, it follows a better paradigm of consistency and modularity than mocking frameworks do.

As you can see, this is an all-around better way to write code and though it is understood that in existing projects mocking still may be necessary, there is no excuse for such to be necessary in new code.

For years media centers, DVRs, and video streaming sites have existed to watch TV and Movies. Over the past several years these services have stagnated. Though there are no shortage of media centers, they all seem to be doing pretty much the exact same thing. For years we have watched and longed for a more featureful interface on our TV to be able to watch the content we're looking for, and for years we have been disappointed.

This is where Nabo TV comes in. We want to bring innovation back to the TV and push the boundaries of what is currently being done on the TV, not just as a product company, but as individuals who enjoy watching media on our TVs.

I just started a Kickstarter to help fund and increase visibility for this endeavor.  Please help me get the word out.

The Problem

I've been asked a lot recently about what template engine I prefer and most people seem shocked when I say that I do my best to avoid them and just generally don't like the idea of templates.  Let me first define what I mean by templates before I get into my explanation so there's no confusion.  When I refer to "Templates" here I'm talking specifically about user-interface templates and primarily HTML templates.  My definition of a template in this context is a mechanism for allowing dynamic content to be incorporated into design.

The Good

To your average designer or developer I'm sure the immediate response is, "But templates can save so much time!" and they are correct.  Templates are definitely an improvement over developers generating UIs dynamically.  Templates exist for the specific need of collaboration between designers and developers.  This has always been a big problem as designers generally don't know the programming languages the developers are using and the developers don't have the creative design skills to be creating user interfaces.  The solution provided by templates is a dumbing down of the dynamic contents (developer side) that allows designers and developers to interact via templates so both can continue to work effectively and collaboratively.

The Bad

Here are the primary points I have against templates:

1. Templates are a dumbing down of your programming language by definition. You cannot accomplish everything you can in your language of choice in a template.  This leads to multiple layers of abstraction and increases the complexity of your application.
2. Templates often try to solve #1 by incorporating WAY too much of the language and thus become increasingly complex and painful for designers to utilize.
3. Templates are yet another "language" you have to learn and maintain.
4. Templates never find the right balance. Templates either focus too much on keeping it simple and thus create severe pain for developers for their lack of capabilities or they incorporate way too much and become impossible for designers to understand.
5. Templates generally require (though not always) a server-side technology to process and populate data into.  This means your designers can't work on the user interface without requiring a server to run it on. This not only makes set up for a designer much more complicated, but it decreases their ability to make quick changes and see the results.

 The Awesome

Hopefully I've effectively shown the problems with Templates, but I have yet to offer a better alternative.  Yes, I believe one does exist and I've been developing this way for over a year and it is being utilized effectively with several production sites (ex. http://www.projectspeaker.com).  The idea is to allow designers to work with HTML and create representations of the content for the site independently of the developers.  The developers then manipulate the content in code before delivery.  Hyperscala was an ideal framework to support this type of development since it allows interaction with HTML, CSS, and JavaScript in a type-safe infrastructure so you can manipulate the DOM without impacting the designer's work.  For an example of this take a look at my post comparing Hyperscala to Play Framework (http://www.matthicks.com/2013/01/hyperscala-why-not-play.html).

I have been negligent giving proper support to Hyperscala's public appearance and have spent the past several months working on the API itself. However, today I finally released a very basic web site at hyperscala.org:

The site is incredibly basic right now and not all that pretty but it is written in 100% Hyperscala and has links to the source code on Github on every page along with working examples.

Feel free to ping me if you have any problems with the site or something is missing that would help make the site better.

I've been pretty busy the past few weeks with clients and haven't had much time to blog. Last week, however, I gave a presentation to the OKC JUG (Java Users Group) about Scala. As anyone that actually reads my blog must know, Scala is my primary language and I absolutely love it. This week since I don't have a lot of time to delve into anything complex I'll simply share my presentation slides comparing Scala and Java code:

Case class scala from Matt Hicks

It is amazing how much more simplistic tasks can be accomplished in Scala versus Java.

This article is continuing in the series on exploring Hyperscala. If you have not already done so, I would highly recommend reading the following previous posts as many topics discussed here build on concepts previously discussed:

• Hyperscala: An Introduction
• Hyperscala: Getting Started
• Hyperscala: Chat Example

The Play Framework is perhaps the most popular web framework among Scala developers today. I have been asked quite a few times what is wrong with Play that made me decide against using it and instead write my own web framework. To be fair, Play is a pretty awesome framework and is a great choice for web applications. However, there are a few problems I have with it that led me to choose against using it moving forward:

Templating Language

Though Play does a far better job of this than pretty much anyone else I've seen, you're still dealing with special syntax in your HTML templates for accessing your Scala code. This is incredibly problematic when there is a separation of designer and programmer. Life becomes quickly more difficult when the programmer injects all the Play-specific code into the HTML file that came from the designer and then the designer needs to make a change. Not only does it become difficult for the designer to even preview what the content should look like now, but it is highly likely that the designer will mess something up when they are trying to make changes to the HTML. This was something I spent a lot of time considering and trying to come up with a better solution for in Hyperscala. I have worked at a lot of large companies that have teams of designers and teams of developers and it is incredibly painful bridging that gap. In Hyperscala there are actually several solutions to this problem, but the best one for this specific scenario, I believe, is DynamicContent. We've discussed this in a previous post and demonstrated it in yet another post. The idea is keeping the HTML clean and pure and the developer simply loads in only what they need to manipulate by the HTML 'id'. In my opinion, this is the simplest solution and a much cleaner separation between the designer and the developer.

Modularity

Yes, there is module support in Play, but it is confusing at best and extremely limited at worst. There is an inherent problem in any framework that doesn't have complete control over the HTML and Play simply does not. The way Play works is very similar to JSPs where content is injected but the HTML is never parsed or comprehended. This leads to several problems when you have modules that need to introduce something like jQuery but shouldn't load it multiple times and want to avoid an issue where another module is trying to load a different version. These are complicated issues that need to be dealt with in large web applications. In Hyperscala the Module system is incredibly powerful and solves all of these problems incredibly well. Not only that, but a Module in Hyperscala is dead simple to write and even more simple to use.

Complicated Setup

Any framework that needs its own console in order to create an application is too complicated. Yes, I agree that it's cool that the Play framework has its handy-dandy console utility to create your application for you, setup your IDE, and probably even brush your teeth for you. However, I very much dislike "magic" that happens and is unexplained. I prefer to understand what is going on under the hood and though you can do this in Play, the decision was made as a default route to hide this from the developer. I prefer to work with frameworks in which you add a Maven dependency and then start writing some code. I don't want configuration files. I don't want a bunch of boilerplate code that is generated on my behalf. I want to instantiate something and run it. This is exactly what Hyperscala does. Once you include your Maven dependencies for Hyperscala you need only create an implementation of Website and add a Webpage to be up and running. In Play you have a minimum expectation of an Application.scala, index.scala.html, application.conf, and routes file.

In an effort to compare Hyperscala with Play practically I decided to take an example from Play and re-write it in Hyperscala. My hope was for a simple 'Hello World' example to keep it simple and straight-forward, but what Play considers their 'Hello World' example is quite a bit more than that. It will have to do.

First lets take a look at the Application.scala file:

This is fairly simple. The idea is that this page asks you for your name to display, the number of times to repeat it, and a color. When you hit submit, it validates the form (based on the validations specified near the top of the page) and then either displays errors or writes out the name you specified the number of times you specified in the color you specified. Notice here that the form validation process is hard-coded, so no special handling is necessary. I point this out now because in Hyperscala there is no "default" for how validation should be handled so it takes a little more code (but not much).

Next we take a look at the HTML files defined for the display.

hello.scala.html index.scala.html main.scala.html

It's fairly clean and I'm sure once you get used to the Play syntax it becomes easier to grasp. However, this would be quite difficult for a designer to work with. I'm sure there is a better way to represent the page for designers, but then you obviously lose out on much of the power of Play.

Now lets take a look at how this same functionality would look in Hyperscala. Like I've said several times now, there are many ways to accomplish the same task in Hyperscala and this is just one way.

First lets look at the HTML, since this is what we really should start developing from. We can focus on getting the design exactly how we want it before we even think about Hyperscala:

play_hello_world.html

This is our basic HTML for the page. Notice there is absolutely nothing specific to Hyperscala or non-HTML in the file at all.

play_hello_world_configuration.html

I've extracted the 'configure' page out into this snippet because we're not only going to replicate Play's Hello World example, but we're going to leverage the Realtime module of Hyperscala to avoid doing a POST and thus keep you on the same page for the entire experience. I could have replicated the form posting utilizing FormSupport mixin, but hopefully this will work as a good example of how easy real-time communication is in Hyperscala. Again, notice that there's nothing special about this HTML. It can easily be previewed in the browser and edited by a designer.

Now, lets take a look at our Scala code:

PlayHelloWorldPage.scala PlayHelloWorldConfiguration

Not much to see that hasn't already been explained in past examples but there are a few things of note. First, notice the call to Realtime.connectStandard(). This, as the scaladocs says, connects all inputs, textareas, and selects to fire change events and buttons to fire click events to the server. We can do this manually, but this saves us a few lines of code. The other thing of note here is the adding of validation. The 'addValidation' method is received as an implicit conversion on FormFields (input, textarea, and select) by importing org.hyperscala.ui.validation._. We utilize some built-in validations and use the ClassValidationsHandler to apply the error class to the outer container and set the message to the error container when validation fails on a field. Like I said before, how validation works in Hyperscala is not built-in, it's part of the UI sub-project and defines some convenience functionality of how error might be handled, but opens the door to supporting validation errors any way you see fit.

In conclusion, I would argue that while in this example the lines of code may be equivalent, the benefits should be seen particularly for larger applications and for interaction with designers. I do not want this to come off as me bashing Play Framework. I have nothing but respect for the framework and the developers that created and use it. My purpose in this post is merely to compare and contrast the architectural choices in one framework against another and to hopefully better explain why I believe Hyperscala to be a better framework in many situations.

Source code referenced for the Play Hello World example came from Play samples on github:

• https://github.com/playframework/Play20/tree/master/samples/scala/helloworld

Source code referenced for the Hyperscala comparison came from Hyperscala examples on github:

• https://github.com/darkfrog26/hyperscala/blob/master/examples/src/main/scala/org/hyperscala/examples/comparison/PlayHelloWorldPage.scala
• https://github.com/darkfrog26/hyperscala/blob/master/examples/src/main/resources/play_hello_world.html
• https://github.com/darkfrog26/hyperscala/blob/master/examples/src/main/resources/play_hello_world_configuration.html

Up to this point we've talked about the high-level features of Hyperscala and have gone through a simple Hello World example, but today we're going to write a real application to show a fairly simple real-world web application.

The real-world application we're going to write today is a chat example. This will utilize real-time messaging, cross-session interaction, abstraction from the user-interface through DynamicContent, and much more.

We will be building upon concepts we've already discussed in the past two posts, so if you haven't already read them I would recommend doing that first. I'm going to skip over the project setup and even the website configuration and focus solely on the webpage for the chat in this post.

The first thing we need to do is to create our webpage class and for the purposes of this tutorial we'll call it ChatExample. As we can remember from previous discussion the basic setup for a webpage simply requires extending from org.hyperscala.web.site.Webpage:

Pretty simple so far. Next we are going to leverage existing HTML for the user-interface rather than writing it all in Hyperscala. This is generally the preferred route when you have a disconnect from the designer and the developer and allows for a very clean separation of duties.

The first HTML file is chat.html. It defines the form and layout of the page:

This should be put in the src/main/resources path in your project. Now, DynamicContent requires the HTML to be presented to it as a String, so we need to load and cache that HTML file:

That will load in the HTML file and store it as a String for usage in each page instance. Now lets create a DynamicContent instance in our page to load that HTML for usage and add it to the body of the page:

Now that we have the DynamicContent loaded we can extract any HTML elements we want to modify or listen to:

If you look back at the HTML file you'll see that the Strings being supplied reference the HTML elements by id for lookup and loading. After these elements are loaded we can then modify them or even introspect the existing data from the HTML. Technically we could just load the entire HTML file and parse it as a Hyperscala structure, but that would be a waste of resources and add unnecessary complexity. By using DynamicContent we can reference only the things we care about in existing HTML and the design can change over time without any necessity of the Scala code changing (so long as those elements remain of the same type and id).

Next we need to add some real-time support to these elements so we can listen for changes and clicks:

The require call is necessary to make sure we have support for Realtime messaging. This is a Module that provides two-way communication between the server and client preferring WebSockets and falling back to long-polling AJAX. The other three lines simply connect JavaScriptEvent to the client-side corresponding JavaScript events. This will intercept client-side events of the specified type and fire them on the server as well. Note that this is simply making that occur, we now need to actually listen for the changes we're interested in:

The above code listens for changes to chatName and updates the internal nickname. This will allow you to distinguish between who is messaging. Second, we add a ClickEvent listener to the submit button to send the message. Now lets see the body of these methods:

This method is rather simple. It calls off to our companion object calling its sendMessage method passing your nickname and the message you typed. Next it clears the value of the TextArea. Finally, it uses jQuery to request focus back to the TextArea. We'll cover the ChatExample.sendMessage method in a minute.

First we add a nickname property to our page instance so we can reference the currently accepted nickname for this user. The updateNickname method first checks to see if the chatName has anything entered. If it doesn't then it assigns the name 'guest'. Next it calls off to ChatExample.generateNick with the nickname to verify that an existing nickname isn't already present. Finally, we assign the chatName input's value to be the newly assigned nickname.

Now we need a HTML representation of each chat entry:

We need to load the HTML again, so we add another val representing the HTML as a String in our companion object below Main:

Now we create a custom class representing an individual chat entry:

We can simply instantiate this and add it to our messages div for each message. Notice that this is very similar to our previous use of DynamicContent except this time we are extending it instead of simply instantiating it. Also, notice the use of reId = true during load of the name and body divs. This is necessary when using an item multiple times on a page as you'll end up with duplicate HTML ids in the code if you don't. When we specify reId = true that will simply assign a new unique id to the element at the time it loads it so each time it will be different.

Lets look at the rest of the content of our companion object now:

There's quite a lot going on in the above code although hopefully the majority is fairly easily understood. First we see we're keeping a history of all the messages so when a new person joins they get to see what's already been said. The instances method gives us a peek into the powerful session framework to look up all of the ChatExample pages across all sessions on the site. The generateNick method takes the supplied nickname and checks to see if it's already in use. If it is then it will add an increment to the end until an available nickname is found. Finally, sendMessage broadcasts the message to all pages adding a ChatEntry in their context. It is necessary to mention context at this point. For many of the underlying features of Hyperscala events and other information are localized to a page so in order to invoke changes across multiple pages we must work within that pages' context. As you can see it is incredibly easy to contextualize to the page by simply calling the context method and passing a function to be invoked within the context.

We have covered a very broad scope of functionality in this post but hopefully it was all clear and understandable. The end result should look like this:

All of the source referenced in this tutorial can be found in the GitHub repository:

• ChatExample.scala
• chat.html
• chat_entry.html

Last week I did an introduction to Hyperscala and briefly outlined some really cool things it can do. This week I want to slow down a bit and take you through the basics of getting your first application up and running with Hyperscala.

Requirements: Since there is a broad number of IDEs / editors used for working in Scala I won't make any presumptions in this tutorial regarding an IDE. However, I will quickly say that I use IntelliJ and absolutely love it for Scala development. That being said, the only requirements necessary to begin this tutorial is SBT and your editor of choice.

First we need to create our directory structure:

hello-hyperscala/
src/
main/
scala/
hello/
HelloPage.scala
HelloSite.scala
build.sbt

Now lets create our build.sbt file in the hello-hyperscala folder with the following contents:

This is a fairly simplistic build.sbt file with just a few specifics to note. First, we need the Typesafe Repository because one of Hyperscala's dependencies uses Akka Actors. The external repo will not be required once we upgrade to 2.10 but because of some known bugs in 2.10 that undertaking is currently on hold. The second thing to notice is the hyperscala-web dependency. The web sub-project of Hyperscala provides the functionality to create a Website and Webpages. There are several sub-projects in Hyperscala (core, html, javascript, svg, web, ui, examples, and site) but web is the highest level we need and depends on the rest of the functionality we want to use right now.

Next we need to create our actual webpage:

You'll name this HelloPage.scala and put in the hello directory as reflected in the structure above. It's fairly simple what we're doing here. We're extending from Webpage that defines the basic HTML structure (html, head, body) and we can then add content on to it. We set the page title and add "Hello World!" to the body of the page.

Next we need to create a Website. The website is responsible for routing URLs to pages, managing access to the session, and much more. So here's our very basic website:

You'll name this HelloSite.scala and put it in the hello directory with HelloPage.scala. We simply define the val page that represents the URI /hello.html to point to a new HelloPage. The second argument is a function, so every request creates a new instance of HelloPage. We talked about Scope in the previous article but for the purposes of our do-nothing page we just have it create the page, render it, and then die when a request comes in. By default a WebpageResource will automatically add itself to the Website so there's no need to explicitly add it. Lastly, the createSession is responsible for creating a new Session that is used throughout the website across all pages.

Now that we've got all of our code written we simply need to run it. Issue the following command:

You should end up with some output like the following:

Notice the "bound to *:8080" signifies that it is wildcard bound to all IP addresses and on the port of 8080. Now just open up your browser and hit http://localhost:8080/hello.html and you should be greeted with "Hello World!".

It has been well over a year since my last post. In 2011 I went to work for Overstock and moved to Utah. Life got busy and I worked during the day and when I had time I programmed on Sgine at night (http://www.sgine.org).  In February of 2012 I left Overstock and moved back to Oklahoma to start my own company writing software for businesses around the world (http://www.outr.com).  Life got busier. :)  Now, here it is, 2013 and I've resolved to blog at least once a week on the many amazing things I get to do.

Today I want to introduce you to Hyperscala (https://github.com/darkfrog26/hyperscala).  It's a web framework I've been working on for a few months in Scala to provide type-safe HTML, CSS, SVG, and JavaScript to developers.  The first question out of my mouth when I hear another person talking about another web framework is, "Why another web framework?" and I've asked myself that question many times both before creating and since creating it. Each time I come up with the same answer, and one that by the end of this introduction I hope you all will agree: It was necessary.

First, as any good introduction should, I'll show you a little Hello World example:

Obviously this is a very simplistic example and simply sets the page title and adds a String to the body of the page. If you'd like a far more detailed example of the bare-metal type-safe HTML writing capabilities I would recommend looking at the TodoMVC example.

This is all well and good if there are no designers and you prefer working in Scala to HTML and CSS. Unfortunately, this is not always the case and the goal of Hyperscala is not to make presumptions on how it will be used, so we try to accommodate everyone.

So, what do you do when you have existing HTML that needs to be brought into your project?

The first option is to convert it into Scala using the handy-dandy code conversion utility in Hyperscala. The code in the TodoMVC above was generated using this neat little tool. This works great when you have a snippet or large amount of HTML that needs to be converted into code to work with. For more information about this tool take a look HTMLToScala.scala. You can run the main method to get a nice visual file selector and tell it where to output the resulting Scala source file. This route works great until you need a designer to make a change to your HTML and they are unwilling to edit your Scala source files. ;)

This brings us to the second option. This is sort of a hybrid approach and I believe really gives you the best of both worlds in the case that you have externally maintained HTML and CSS that needs dynamic content integration. I created the DynamicContent trait and utilization of this trait allows you to dynamically import HTML and load only the elements you want to work with. See the following HTML snippet:

In this example we care about the two inputs and the button but we don't care about the rest of the content. So, utilizing DynamicContent we create SimpleDynamicForm:

This is a bit more elaborate than is necessary, but shows some of the power of PowerScala Properties in the mix with Hyperscala. Look specifically at the line where the button is loaded:

DynamicContent finds the button by its id in the dynamic.html file and loads it as a Hyperscala tag.Button and any changes I make to the loaded button are reflected at render time. I could have simply loaded the name and age inputs as well, but using bind I cross-bind the value of the input to their reflected field names in the Person class. This means that when I change the value of the inputs it updates the 'person' Property to reflect the change. DynamicContent.bind introduces two other features of Hyperscala we need to briefly discuss: Modules and Realtime.

Modules provide the ability to define shared functionality between applications, components, features, etc. For example, a big problem in web site development is avoiding multiple includes of jQuery. In Hyperscala there is a built-in module for jQuery that any tag or page that needs to make use of can simply 'require' in their code. For example:

The above code requires jQuery be present by referencing the jQuery Interface and passing a default implementation of jQuery182. This will inject jQuery182 if no other implementation of jQuery is specifically provided before render of the page. The require method may be invoked with just an Interface (and an exception will be thrown at render time if no implementation has been provided), an Interface and a default Module (as seen above), or just a Module (at render time the module will be used if it is the highest version number provided for the same module name).

Modules are extremely easy to write. For example, the jQuery182 module looks like this:

Every Module needs a name and version. If they implement an Interface they can override the implements method to provide a list of those they implement (see we are implementing jQuery interface). The init method is invoked the very first time the module is used within a Website. This allows one-time actions to be taken like registering a JavaScript resource to a path as seen above. The load method is invoked at first render of every page that requires this module. This gives the module the ability to interact with the page prior to rendering to the client. As you can see in the jQuery182 module we simply add a tag.Script instance to the head of the page.

Realtime is actually a Module like we just discussed, but an extremely powerful one. This allows two-way communication between the client and server via WebSockets falling back to AJAX polling. It is incredibly easy to use as seen below:

That's all that's necessary in order to allow cross-communication to a loaded webpage. Now any changes to the HTML after page load will automatically synchronize to the browser. Further, you can begin listening to JavaScript events on the server by specifying what events you want to be sent down. For example:

This is an extremely simple example of hooking up a button that sends click events to the server. Notice on line 12 the use of "event.click". All HTML tags expose JavaScript events via the "event" object. In this case we are assigning the result of calling JavaScriptEvent() that interacts with Realtime to send the event down to the server. We could do something like this instead if we wanted just basic JavaScript:

Notice on line 14 we add a synchronous listener to handle a ClickEvent (the JavaScript event that is thrown when clicked). When clicked we simply log that the button has been clicked to the server-side console.

This has been a pretty long post, but barely scratches the surface of what Hyperscala can do. My goal is to spend the next few weeks blogging about the features and capabilities of Hyperscala and get into some even more powerful features of the framework. Again, the biggest advantage of Hyperscala is that at its core it is simply a webpage rendering framework. Everything else is just icing on the cake. This means anyone that has other needs for the features of Hyperscala can simply build their own abstraction layer on top to provide whatever they want.

I was pretty exclusively a Java developer for twelve years before making the switch to Scala. Yes, it was difficult to learn some of the syntactical differences, but now that I have I don't want to turn back. No, this is not a baited question to get a flame-war going, but an honest attempt to get some real feedback by those that believe Scala is more difficult than Java.

Let me lay some ground-rules before we get this discussion going though. First, complex topics in Scala are complex, there's no debate there, but that doesn't make Scala more difficult than Java, there are just more possibilities to what you can accomplish. I say that the argument cannot be made that Scala is more complex if there is not a side-by-side comparison to Java code that is much simpler to understand. This means we have to leave the advanced concepts of implicit conversions, multiple inheritance, and so much more that Scala can do, but are features that bring it beyond the capabilities of Java. Yes, of course you can make an argument that in switching to a language like Scala you will eventually run into these scenarios, but I am speaking purely of the introductory programmer and the "difficulties" between Scala's syntax versus Java's.

Now, can someone give me a scenario of code in Java that would be more complicated to understand in Scala?

I would consider myself pretty competent with Scala at this point, but there is still so much I have yet to learn. Something I always used to do in Java was to work my way one class at a time through the JavaDocs of the language I am attempting to do with Scala now as well.

I've just run across the Either class in the ScalaDocs and really like the idea.


The idea is that you may have two different return types that may result from one method. Classically that is solved with all sorts of hackery or returning the common superclass to the two and then doing an instanceof check, but in Scala the Either class provides a much more elegant solution:

object Test {
 def main(args: Array[String]): Unit = {
  numOrString("5") match {
   case Left(s) => println("String: " + s)
   case Right(i) => println("Integer: " + i)
  }
 }
 
 def numOrString(s: String) = {
  try {
   Right(s.toInt)
  } catch {
   case exc: NumberFormatException => Left(s)
  }
 }
}

I'm not really sure how practical this example I wrote is, but my method "numOrString" takes in a String and attempts to convert it to an Int. If it can return an Int it does so or it returns a String. By use of the Left and Right subclasses of Either you can define which type of result you are sending back. Finally, with Scala's awesome matching you can easily determine which type of result you got.