Wrapping up little bits of Scala evangelism

Thus far I’ve tried to argue that Scala is a functional programming language that is also very pragmatic for real-world work. I think any Java developer who’s looking for something more sophisticated but doesn’t want to give up their development environment or their current toolchain should take a very serious, hard look at what Scala has to offer. JetBrains, creators of the popular IntelliJ IDEA, have shown interest in Scala with good reason; a few of their employees post to the Scala mailing list on a regular basis.

Here are some links to other interesting code examples.

• Here is an example of using a GUI library in the works by EPFL. It wraps around Swing and makes it much easier to use. It’s only in the early stages but it looks promising. It certainly makes Swing easier to use.
• David Pollak has been writing some interesting extended examples in Scala and you should have a look.
• The Scala wiki has an impressive example of simplifying JDBC with implicit definitions. Implicit definitions allow you to consume Java libraries (which tend to be akward to use, a state of affairs still preferable to what you find if you’re using Lisp/OCaml/Haskell, where the libraries you need don’t exist and probably never will unless you write them yourself) on much more amiable terms without having to extend them and thereby getting stuck in a rigid class hierarchy unfamiliar to a Java programmer who has used the library in question before. Implicit definitions are quite useful when used properly.

With that, I will take it as a given that Scala is a legitimate, useful language that any person who cares about producing brief, readable, high-performance code should definitely consider. The entirety of Java-land is the Scala hacker’s oyster, and he picks and plucks his favorite libraries at will, writing tiny bits of view code to simplify away the ugliness and verbosity, writing code often comparable to Erlang, Haskell, or OCaml in brevity. It is up to the Java community as well as functional programmers whether they want to be open to new ways of doing things that will help them and their customers or not.

Scala is definitely a little rough around the edges, still, being so young, of course. Probably its biggest flaw at the moment is documentation. But that is getting better on a rapid basis. The Scala team has shown that they are extremely open to user feedback and they are making rapid, regular progress towards improving both the language and documentation. They have gotten amazingly far, considering that Scala is only about 5 years old, and have only had a 1.0 release for a few years.

If all this isn’t enough to convince you about the potential merits of Scala, probably nothing will, and therefore I will cease preaching to either the converted or the unconvertible.

When bad web development causes real-world financial problems

I intend for this blog to be technical and consist of solid technical content, but I’m still a little freaked out over what just happened when I tried to pay tuition fees online. They must have changed the interface somehow, because it wasn’t displaying properly in Firefox – everything was broken and garbled up. When I changed how I was going to pay, it didn’t reflect the change (even when, e.g., I hit F5 to refresh). After a little while I got the idea to try again in Internet Explorer and it worked just fine. If someone waited until the 11th hour to pay and didn’t know what to do in situations like this, it could cause someone’s classes to get canceled, and that’s not cool.

When real dollars and important events in people’s lives are involved, do not fuck around.

Planned: polite HTTP retrieval, JDBC made easy, and massively scalable systems à la Erlang using thread-based actors in the new scala.actors package screw it, a lot of the examples are outdated and don’t work, and the library is terribly documented.

Example: Using Jakarta Commons HttpClient for multithreaded HTTP retrieval

Public service announcement: The following code is not usable for real-world HTTP retrieval. If you write a spider or a news aggregator, please use gzip compression and conditional GETs. We now return you to our regularly scheduled blogging already in progress.

The Apache people have an HTTP client designed to deal with the shortcomings of the stuff you can find in the java.net packages. They have an example program in their svn repository of retrieving a given list of URLs in multithreaded fashion. Here’s a slightly simpler Scala adaptation which simultaneously retrieves the webpages for Google, Yahoo, and Sun:

import org.apache.commons.httpclient._
import org.apache.commons.httpclient.methods._

object mtget extends Application {
  // a list of uris to retrieve -- make a thread for each and start it
  List("http://www.google.com/", "http://www.yahoo.com/",
       "http://www.sun.com/") map (x => new Getter(x)) foreach (x => x start)

  class Getter(uri: String) extends Thread {
    val m = new GetMethod(uri)
    override def run() =
      try {
	m setFollowRedirects true
        (new HttpClient(new MultiThreadedHttpConnectionManager)) executeMethod m
        Console.println(m.getResponseBody.length + "b from " + uri)
      } catch {
	case e: Exception => Console.println("error: " + e)
      } finally {
        m.releaseConnection
        Console.println("connection released")
      }
  }
}

While my example is slightly simpler there are parts in the original Java program I don’t have to write or can write in a briefer fashion. For instance, code like this:

        Getter[] threads = new Getter[urisToGet.length];
        for (int i = 0; i < threads.length; i++) {
            threads[i] = new Getter(urisToGet[i]);
        }
        for (int j = 0; j < threads.length; j++) {
            threads[j].start();
        }

would become urisToGet map (x => new Getter(x)) foreach (x => x start) in Scala; there’s no use, after all, in creating variables if you’re not going to use them. I also don’t need to write boilerplate code for class variable declaration and a constructor that just initializes the class variables to the constructor parameters due to how you declare classes in Scala. Notice in the Scala code that I used a style of invoking methods comparable to Smalltalk. Instead of writing client.executeMethod(method), you could write client executeMethod method, whatever you prefer. It’s just syntactic sugar.

Also notice that I have fewer variables and less mutability. Removing unnecessary mutability (e.g., by declaring our GetMethod object to be an immutable val instead of a mutable var) reduces the number of places things can go wrong. The programming style Scala encourages, then, leads to fewer bugs. Also, we can have inner classes rather than having to declare things as static. Finally, the type inference system in Scala’s compiler lets us write code free of redundant type declarations.

The point of this post is that Scala lets you do real-world stuff in a pretty brief manner, and it compares favorably to Java. Scala is not a mere academic curiosity useless for getting real work done. Scala is useful for real-world programming, today. Say goodbye to for(int i = 0; i < N; i++); say hello to type inference and beautiful, brief code.

Example: Using Java libraries to manipulate regular expressions

I love learning about new programming languages, but I always get burned by the small communities that surround them. Polished libraries for database interaction, XML processing, network protocols, and other real-world stuff often aren’t there, and assembling the tools you need to do a simple application may be hours or days of work.

Fuck that. Really.

So here’s an idea: piggyback on a language/environment that already has lots of libraries in it, and make it easy to use those. That is the strategy Scala takes.

So, consider the example of wanting to do simple simple stuff with regular expressions. You can read a bit about how that’s normally done at Sun’s website. Have a look at that code; the basic idea is to use two classes, java.util.regex.Pattern and java.util.regex.Matcher. Get the pattern and string to match, and then loop over the results. Here’s the code to do that in Scala.

import java.util.regex.{Pattern, Matcher}

object regextest extends Application {
  while(true) {
    Console.println("Enter regex")
    val p = Pattern.compile(Console.readLine)

    Console.println("Enter string to search")
    val m = p.matcher(Console.readLine)

    while (m.find())
      Console.println(m.group() + "; " + m.start() + ", " + m.end())
    Console.println("End of resultsn")
  }
}

On the first line, we see that we can import multiple classes from the same package using a shorthand bracket and comma notation. Next, we instantiate a console application which frees us from having to even put in a main method. The rest of the code is isomorphic to the Java code. See, I told you Scala could be similar to Java code.

Here’s a sample of compiling and running that.

> scalac regextest.scala

> scala regextest
Enter regex
[a-zA-Z]+
Enter string to search
these are some words
these; 0, 5
are; 6, 9
some; 10, 14
words; 15, 20
End of results

Enter regex
[0-9]+
Enter string to search
these are numbers: 42, 1000, 5
42; 19, 21
1000; 23, 27
5; 29, 30
End of results
Enter regex ^C
Terminate batch job (Y/N)? y

Not too bad, huh? This suggests a general way of using Java libraries.

1. Find a Java library you want to use.
2. Find sample code that makes use of it.
3. Isomorphically transcribe it to Scala as we did above.
4. After verifying that it works correctly, replace lousy Java idioms with briefer Scala idioms.

The entirety of the massive Java ecosystem is at our disposal, which we can exploit easily and without losing development momentum. That is what separates Scala from Haskell, Lisp, O’Caml, and other functional languages.

What is Scala, and why you should care

I intend to start writing about Scala, a fairly new (circa 2003) programming language for the Java Virtual Machine. It tries to unify object-oriented and functional programming; it is multi-paradigm. Why trying to unify those two is worth doing would take a while to discuss, but the practical result is that you can do object-oriented programming when you want to but you’re always free to use any functional constructs that you like, such as map, filter, and the other usual suspects or their analogues that you might be familiar with from Ruby, Smalltalk, Haskell/OCaml, Lisp or what have you. And actually if you like any of those languages (or if you just use Java! Scala is similar to Java in a lot of ways, really, gimme a chance!) you should take a look at Scala. Scala is quite stable and worth using, so I’ll try to discuss a little bit more about it below. I’ll make the disclaimer now that I’m still learning myself but hopefully you’ll consider what I have to say just the same.

Scala runs on the Java Virtual Machine, so it can transparently use Java code, classes, and libraries. No foreign function interface gunk, no nothing. It’s effortless, and I’ll show an example in the next post of using java.util.regex to do simple regular expression stuff.

For now, let’s answer a simple question: what does Hello World look like in Scala? Well, here’s how it looks:

object HelloWorld {
  def main(args: Array[String]) =
    Console.println("Hello, world!")
}

OK, that’s not too bad. How do you install Scala? Well, first you’ll need either the JRE or the JDK installed and set up; head over to Sun’s website and git ‘er done. If you can compile HelloWorld.java for Java you should be OK. Alright, go ahead and download Scala for your platform (I’m on Windows but it should work on anything the JVM runs on). So, after you add the Scala bin directory to your path, you can compile and run the above Scala code by doing:

> scalac HelloWorld.scala
> scala HelloWorld
Hello, world!
>

You don’t have to use CamelCase, by the way. You could call everything helloworld and Scala just won’t give a damn.

Well, hokai, you’ve seen how to do a Hello World in Scala. It’s a little smaller than Java’s, huh? I’ll reprint it again just for convenience.

object HelloWorld {
  def main(args: Array[String]) =
    Console.println("Hello, world!")
}

So, the object keyword gives us a singleton (yeah, that pattern becomes invisible in Scala). We don’t need to declare anything static ’cause, well, just ’cause for now. The main method takes an array of strings as a parameter, just like in Java. Console is another convenience object that has stuff for reading from and printing to the console. In Scala, you declare things like var: type rather than type var like in C, C++, Java and the like. Hopefully it doesn’t look too foreign.

But here’s the cool thing: in larger programs, Scala can infer the types of a lot of stuff (it has type inference in the finest tradition of Haskell and O’Caml). Like, when you add two integers, it knows the result is an integer; and actually in our humble Hello World program Scala inferred that your program returned a unit, Scala’s version of void (it’s like O’Caml’s unit, if you’ve used that). So when you declare a variable in a function, you probably won’t have to supply type information. That winds up saving you a lot of typing. So you get brevity while still being static and typesafe. Oh, yeah, Scala is statically typed. But you get a REPL! But when you compile to native code, it’s still pretty fast. And you don’t have to give up using Eclipse/Ant/Maven/other Java tools you might be familiar with… Oh dear, I’m getting ahead of myself. Well, this is probably a good stopping point before I start rambling about all the neato crap in Scala, so stay tuned…

Ah christ, I can’t resist. You might head over to Scala’s website, but I think it’s pretty confusing and ugly, so try to not let it scare you away. Look in a subdirectory of your Scala installation (scala_dir/misc/scala-tool-support, where scala_dir is your Scala installation directory) for editor support (IntelliJ, Vim, Emacs support and so on), and there’s also a directory that has helpful documents which are at this point flawed, so don’t let that scare you either; I’m your helpful guide on the road to Scala. OK, hopefully you think Scala is worth spending a little more time on. Till next time.

Agenda

This will be about my experiments in particle swarm optimization, grammatical evolution, evolutionary computation in general, and also my interests in functional programming languages.

First up will be about Scala, a functional programming language I’ve been learning about.