My last article JavaScript for people who are in a hurry was yet another attempt to share my appreciation for this language with the world. JavaScript isn't perfect by any stretch of the imagination, but it's a much nicer language than most people give it credit for. Also, cross-browser issues aside, JavaScript is great fun to program in. The problem is that JavaScript is so different from other mainstream languages, that most people shy away from learning it properly when they hear about things like prototypial inheritance. While with the last article, I tried to cram as much information into as little text as possible, with this new article series I'm going for a more relaxed approach. This is the first article and it's about higher-order functions and closures - again. A lot of people didn't fully understand the example in the previous article and the whole thing even spawned a very interesting discussion on reddit. So here's an attempt at a better explanation. Enjoy, let me know what you think and feel free to ask questions.

Higher-order functions revisited

Let's say you're programming in a language that does not support higher-order functions and you're implementing a method that receives an unsorted array, sorts it in an ascending order and then returns it.

1  class SortUtils {
2      public static Object[] sort(Object[] values) {
3          // Magic happens
4          return sortedValues;
5      }
6  }
Here's the signature of said method in Java. Now if you look at it more closely, you'll realize that the method accepts an array of objects and returns an array of objects (I didn't throw generics into the mix to keep it simple) which essentially means that the method can sort arrays of any reference type. This sounds good in theory, but what about this scenario:

1  Integer a = 42;
2  String b = "Cellar door";
3  Object[] values = new Object[] {a, b};
4  Object[] sorted = SortUtils.sort(values);
Since there is no implicit ordering relation for Integer- and String objects in Java, we somehow need to tell our sort-Method how to compare these two types. In the absence of "true" higher-order functions, we could do something like this:

1  class MyComparer {
2      public int compare(Object a, Object b) {
3          // Return a number that indicates how a and b relate to each other
4          return value;
5      }
6  }
7  
8  Object[] sorted = SortUtils.sort(values, new MyComparer());
Here we have altered our sort method to accept two parameters: the array to be sorted and a comparer object. The comparer object provides a single method compare that defines an explicit ordering relation for the types in our array. Our sort method is still nice and generic, because everything it needs to know to sort the array is passed into it by the caller. But what exactly are we passing to our sort method? The array we want the method to sort is obviously data, but what about the comparer object? It doesn't have any state so is it still data? What the comparer really is is a piece of executable code that you can pass around like an object. This is where we stop making a distinction between code and data: code is data. It's no more special than an integer or a string.
What we just did is we have turned our sort-method into a higher-order function. But this is Java and Java doesn't do higher-order functions, right? Actually that's just semantics. What Java certainly doesn't have is a convenient syntax for "just functions" and especially no convenient way of creating them on the fly. There's anonymous inner classes, but they're still pretty verbose.
JavaScript on the other hand doesn't have classes, but it does have anonymous functions. So what would the above example look like in JavaScript?

1  sort(values, function(a, b) { /* Magic happens */ return value; });
Pretty nifty, huh? In all honesty though, anonymous functions are really just syntactic sugar. They're no more powerful than single-method objects, they're just more convenient to work with.

Because JavaScript doesn't have classes, functions (both anonymous and "named") are of special significance to the language. They serve as constructors for objects, they play the role of methods and they're involved in a whole bunch of cool tricks that make the language so interesting (more about that in a later article).

Closures

In the previous section we have established that higher-order functions are nothing special. As this would be a pretty lame conclusion of this article, let's talk about closures. Before we talk about closures however, let's talk a little bit about scoping. In most curly-brace languages, every block of code (delimited by opening and closing curly braces) introduces a new scope. Program code can access both variables defined in its own scope and in scopes further up the scope chain as this example demonstrates:

 1  void scope(int a) {
 2      int b = 2;
 3      {
 4          int c = 3;
 5          do {
 6              int d = 4;
 7              System.out.println(a + "," + b + "," + c + "," + d);
 8          } while (false);
 9      }
10  }
11  
12  scope(1);
As you would expect, this prints the string "1,2,3,4". JavaScript behaves exactly the same way, except it does not support block-level scoping prior to JavaScript 1.7. What that means is best demonstrated with an example:

 1  function scope(a) {
 2      var b = 2;
 3      {
 4          var c = 3;
 5          do {
 6              var d = 4;
 7              alert(a + "," + b + "," + c + "," + d);
 8          } while (false);
 9      }
10      alert(d);
11  }
12  
13  scope(1);
This will alert "1,2,3,4" as before, but then it will show another alert box with the value "4" in it. A Java compiler on the other hand would give you a compile error on line 10, as, according to Java scoping rules, the variable d wouldn't be in scope anymore.
Now let's do something a little bit more interesting:

1  function scope(a) {
2      var b = 2;
3      return function(c) {
4          var d = 4;
5          alert(a + "," + b + "," + c + "," + d);
6      };
7  }
8  
9  scope(1)(3);
Here we define a function inside another function. If you remember the first section of this article, you'll realize that this is the equivalent of instantiating a one-method class and therefore nothing to be scared of. However, there is something slightly odd about this code: the inner function uses variables and parameters defined in the outer function. Odd maybe, but perfectly in line with the simple scoping rule we established earlier, remember? "Program code can access both variables defined in its own scope and in scopes further up the scope chain." For the inner function, the outer function is simply one level up in the scope chain. The reason why this seems so odd is because when we think about method invocation, we think about stack frames. That's just an implementation detail though. If you look at the code above more abstractly, it makes perfect sense that the inner function can access the variables of the outer function. You should also not be bothered by the fact that the local variables of the outer function are preserved beyond their natural lifespan because they've been captured by the inner function. Essentially what's happening is that the JavaScript VM makes sure that your code works as expected, even if it has to jump through a bunch of hoops to make it happen.
The concept whereby local variables and parameters of an outer function are captured by an inner function is called a "closure". Closures are a powerful concept, but when you think about it, they're not really that special either. A closure is a function bound to a specific environment. If that doesn't sound extremely familiar, take a look at this code:

 1  class Rectangle {
 2      int width;
 3      int height;
 4      
 5      Rectangle(int width, int height) {
 6          this.width = width;
 7          this.height = height;
 8      }
 9      
10      int area() {
11          return width * height;
12      }
13  }
14  
15  Rectangle r = new Rectangle(2, 4);
16  int area = r.area();
The area-method can access the object's two attributes width and height despite the fact that they're defined outside of it's local scope. This works because when you invoke the area-method on an object, a reference to that object is implicitly passed to the method in a magic variable called this. Exactly the same thing happens when a function closes over a bunch of variables and parameters (except it happens more transparently).

Using higher-order functions and closures

Maybe now you're asking yourself: that's all very nice, but what's the point of all this? Higher-order functions and closures are essentially just another tool in your toolbox and they can be used to solve certain types of problems more elegantly. In JavaScript however, higher-order functions and closures are absolutely vital. In the next article, I'll be talking about how JavaScript does OOP and closures actually play a critical role in that. To not get ahead of myself though, let's look at some other uses of higher-order functions and closures:

 1  function filter(values, f) {
 2      var out = [];
 3      for (var i = 0; i < values.length; i++) {
 4          if (f(values[i])) {
 5              out.push(values[i]);
 6          }
 7      }
 8      return out;
 9  }
10  
11  alert(filter([1, 2, 3, 4, 5], function(value) { return value % 2 == 0; }));
Filter (along with map and reduce) is a traditional higher-order function. It takes an array of values and a function and returns a new array containing only the values of the original array for which the specified function f returned true. In our particular example, we use filter to retrieve all even numbers from the array. Because we can pass code around as data, filter can be used to filter the array by any criterion that can be expressed in the underlying language. Higher-order functions therefore promote code-reuse which generally is a good thing (unless you're paid per line of code).

Closures on the other hand are great for things like event-driven- or asynchronous programming. An example:

 1  function loadAsync(target, url) {
 2      var request = new Request(url);
 3      request.oncomplete = function(response) {
 4          target.value = response.text;
 5      };
 6      request.start();
 7  }
 8  
 9  loadAsync(textField1, "?content=1");
10  loadAsync(textField2, "?content=2");
Here, a document is requested from a server using an asynchronous HTTP request. Once the request returns with a response, a callback function is invoked (line 3). This callback function then displays the response in a UI control. Using closures, we can define a single function loadAsync that wires up a particular request to a particular target control without explicitly holding any state. The callback function knows which control to update because it has captured the parameter target of the outer function loadAsync. While the same could be achieved without closures, the use of closures allows for greater genericity and makes the code much more readable. In fact, you could almost forget that the code is asynchronous at all because the lines between the code that is running sequentially an the code that is called at an unspecified point in time are so blurry. If JavaScript supported real multi-threading, this could be problematic (variables held in a closure could be shared between two functions running on two separate threads). Since it doesn't, the code above is perfectly safe.

Conclusions and outlook

Higher-order functions and closures are nothing special. You probably have been using them forever, you just haven't realized it yet. Nonetheless, they're extremely powerful tools to have in your toolbox and because JavaScript has a bunch of syntactic sugar to make their use more convenient, it's much more pleasant to program in a "functional style" in JavaScript than in certain other imperative languages.

In the next chapter of this series, we'll take a closer look at how JavaScript does OOP and how closures play into it.

As always, if you have any questions, feel free to ask!