Archive for category Kotlin

Android, Rx and Kotlin: part 2

I haven’t been quite honest with you my previous post: the code I showed in the article doesn’t exactly result in the short video of the application at the top of the article.

If you run the code as is, you will notice something very irritating (and unacceptable in any application): whenever the app is pretending to make a network call, the entire user interface freezes for a second. You can’t type anything and the loading icon stops spinning. This is the classic symptom of blocking the main thread. You will remember that I am simulating network calls by simply sleeping for a little while, and obviously, if you do this on the main thread, you will freeze your UI.

By default, Rx runs everything on your current thread, which is the main thread in Android: the thread that is in charge of updating your user interface. Android is exactly like most graphical toolkits: you should only use the main thread to update your UI but anything else you do (network or file system access, computations, database updates, etc…) needs to be done on a background thread. Rx has a very good solution to this problem.


Until recently, AsyncTask was the recommended way of performing this kind of task: by creating and executing an AsyncTask, you can run your code in two locations, one that will be run in a background thread (doInBackground()) and once that task completes, code that will run on the main thread (onPostExecute()).

AsyncTask has a troubled past and it has evolved quite a bit over the many revisions of the Android API: first it was single threaded, then it became multithreaded and more recently, it’s running in the background on one thread in an attempt to provide both parallelism and sequencing at the same time. If you need more information about AsyncTask, this article explains how it evolved.

This is not the only issue with AsyncTask: it’s also fairly challenging to get its behavior right while going through configuration changes or the possibility of your activity being paused or destroyed while the task is still running.

Rx offers a few solutions to some of these problems, but not all.

Threading and Rx

Rx offers two methods to control your threading model: subscribeOn() and observeOn().

In a nutshell, observeOn() defines what thread your observer will run on (this is where you usually do the work) and subscribeOn() defines the thread where your operators will run (map(), filter(), etc…).

The parameter you give to these methods is a Scheduler, an Rx abstraction that encapsulates a thread. Rx defines a few standard ones:

  • Schedulers.computation(): When you are calculating something.
  • When you are doing I/O (network, file system, database access, …).
  • And a few others I won’t get into here.

Additionally, RxAndroid defines the more Android-specific AndroidSchedulers.mainThread(), which is self explanatory.

A typical piece of code on Android is to run a few tasks in the background (network access, expensive computation, database update, etc…) and based on the result of that action, you update your UI. The way to implement this with Rx is straightforward: you subscribe on whichever background thread is more appropriate for your actions and you observe on the main thread:

trait Server {
    fun findUser(name: String) : Observable<JsonObject>

data class User(val id: String, val name: String)

fun p(s: String) {
    println("[${Thread.currentThread().getName()}] ${s}")

    .flatMap {
    	p("Calling server.findUser");
    .map{jo ->
        p("Mapping to a User object")
    .subscribe{ u -> p("User: ${u}a") }

We start with a string (which could come from an EditText and we specify that we’ll be subscribing from the I/O thread. Then we call the server with that name (on the I/O thread), turn the JSON response into a User object and we print that object:

[IoThreadScheduler-1] Calling server.findUser
[IoThreadScheduler-1] Mapping to a User object
[Main] User: User(id=123, name=cedric)

Note that even though you can specify multiple subscribeOn, all the subscriptions will happen on the first scheduler (subsequent subscribeOn will be ignored). I’m not sure if this is by design or just an oversight, but it’s not really a problem in practice. If you ever want to subscribe on multiple schedulers, you can always make this happen in the body of your subscription itself (for example, in the example above, if the server call was actually using Retrofit, you would see it’s using its own thread pool to make that call).

And that’s about all there is to get started with thread management with Rx on Android. As you can see, structuring your code this way makes the intent and thread handling extremely clear and easy to trace through, much more so than with AsyncTask.

With the growing number of Android libraries adding support for Rx, it’s becoming even more trivial to use these libraries within this framework and combine them in straightforward yet powerful ways. You can see in the examples I used in this post and the previous one how Rx makes it trivial to combine network calls and GUI updates simply by the fact that Retrofit returns Observables. You should also take a look at SQLBrite, which wraps SQLiteOpenHelper in Observables to offer you similar flexibility but for database access.

Read part 1, part 3.

Kotlin Eclipse plug-in

JetBrains just released the first alpha version of the Kotlin Eclipse plug-in. The update site is:

You will need Luna to run it. It’s a bit rough around the edges, as you would expect from an Alpha, but it supports basic features:

  • Problems view (I miss this so much on IDEA).
  • Debugger.
  • A few quick fixes (e.g. “Replace ‘get’ with index operator).
  • Testing.
  • Auto imports.
  • Navigate to symbol.

Hopefully the Eclipse plug-in will soon catch up to IDEA’s already awesome support for Kotlin.

Android, Rx and Kotlin: a case study

There are countless introductions to Rx and quite a few discussing Rx on Android, so instead of writing another one, I decided to come up with a simple activity and go through the entire exercise of implementing it using Rx and Kotlin. I’m sure you’ll be able to follow these article easily even if you’re not familiar with Kotlin because the syntax is remarkably similar to what you would write if you were using Java 8 (just with fewer semi colons).

The activity

It’s shown at the top of this article. It’s pretty simple while covering some important basics and, as we’ll find out along the way, the devil is in the details. Here is a simple functional specification of this activity:

  • The user types a nick name in an EditText. The activity asks the server if such a name is known and if it is, it returns a user record.
  • At this point, the user can “Add” this person as a friend, which triggers another call to the server to verify that the addition is allowed. The server returns a simple “ok” or “error” if that person can’t be added as a friend.

A few additional details:

  • We want to display a progress bar during all server calls.
  • Since we don’t want to bother the server too much, we only start sending requests for names that have three or more characters (we’ll add to this condition later).

Without Rx

Implementing this activity with “regular” Android practices is straightforward:

  • Add a listener to the EditText so we get notified each time a character is typed.
  • Send a network request with an AsyncTask.
  • When the result comes, update the UI (on the main thread and not in whatever thread AsyncTask ended up running).
  • Enable the “Add friend” button and listen to it, calling the server if pressed to verify that the friend can be added (one more AsyncTask + main thread post tap dance).

This implementation is straightforward but it’s very scattered. You will very likely end up with a lot of empty methods added just to satisfy listener requirements, storing state in fields to communicate between the various asynchronous tasks and making sure your threading model is sound. In other words, a lot of boiler plate, a messy mix of headless and graphical logic and graphical update code spread a bit everywhere.

There is a better way.

Kotlin and Android

I found Kotlin to be a very good match for Android even in the early days but recently, the Kotlin team has really cranked up their support for Android and adding specific functionalities for the platform in their tooling, which makes Kotlin even more of a perfect match for Android.

Kotlin M11 was released about a week ago and it added a functionality that makes the appeal of Kotlin absolutely irresistible for Android: automatically bound resources. Here is how it works.

Suppose you define the following View in your layout activity_search.xml:

<Button ...

All you need to do is add a special kind of import to your source:


and the identifier addFriendButton becomes magically available everywhere in your source, with the proper type. This basically obsoletes ButterKnife/KotterKnife (well, not quite, there’s still OnClick which is pretty nice. Besides, Jake tells me he has something in the works). And if you press Ctrl-b on such an identifier, Android Studio takes you directly to the layout file where that View is defined. Very neat.

The server

For the purpose of this article, I’m just mocking the server. Here is its definition:

trait Server {
    fun findUser(name: String) : Observable<JsonObject>
    fun addFriend(user: User) : Observable<JsonObject>

If you think this definition looks a lot like a Retrofit service interface, it’s because it is. If you’re not using Retrofit, you should. Right now, I’ll be mocking this server by returning a set of hardcoded answers and also making each call sleep one second to simulate latency (and so we can see the progress bar on the screen). Note that each call on this interface returns an Observable, so they fit right in with our Rx implementation.

In this example, I hardcoded the server to know about two friends (“cedric” and “jon”) but only “cedric” can be added as a friend.

The Rx mindset

Switching to the Rx mindset requires you to start thinking in terms of event sources (observables) and listeners (subscribers). If this idea doesn’t sound that novel, it’s because it’s not. This model was already being advocated in the book “Design Patterns” in 1994 and even in the early versions of Java twenty years ago (and no doubt you can find traces of it in the literature before that). However, Rx introduces new concepts in this idea that we’ll explore in this series.

So let’s rethink our activity in Rx terms: what are event sources (I’ll use the name “observable” from now on) in this activity?

I can count four observables:

  1. First, we have the EditText: whenever a new character is typed, it emits an event that contains the entire text typed so far. We can emit a new name once we have more than three characters.
  2. Next is the name observable, which calls the server and emits the JsonObject it receives in response.
  3. Next in the chain, we have a “user” Observable, which maps the JsonObject into a User instance with the name and id of that person.
  4. Finally, the “Add friend” button is another observable: if the user presses that button, we make another call to the server with the User we have and we update our UI based on the results.

There are various ways we can break this problem into observables and the final result depends on how you want to handle various tasks, which observables you want to reuse, the threading model you prefer, etc…


The “EditText” Observable

Let’s start with our EditText. Its implementation is pretty straightforward:

    .doOnNext { e: OnTextChangeEvent ->
    .map { e: OnTextChangeEvent -> e.text().toString() }
    .filter { s: String -> s.length() >= 3 }
    .subscribe { s: String -> mNameObservable.onNext(s) }

Let’s go through each line in turn:

  • WidgetObservable turns an ordinary View into an Observable. It’s a wrapper coming from RxAndroid that calls all subscribers with the full text whenever a new character is typed.
  • Next we reset the UI to its default state.
  • The map line converts an OnTextChangeEvent into a string. It’s a purely convenience step, so that the next operators in the chain can deal with the context of the EditText as a String instead of having to repeatedly extract it from the event (e.text().toString() everywhere).
  • Next, if the text is less than three characters long, we stop right there.
  • Finally, we subscribe to the value we have obtained if we have reached this far: a string that’s at least three characters long. We pass this string to our name observable.

The “name” Observable

val mNameObservable: BehaviorSubject<String> = BehaviorSubject.create()

A BehaviorSubject is a special kind of Observable that you can send events to after its creation. I like to think of it as an event bus, except that it’s focused on a very specific kind of events (as opposed to an event bus that is used to post pretty much anything to). Using a Subject here allows me to create that Observable early and only post new events to it as they come in, which is what we did with the snippet of code above.

Let’s see how we use that Observable now, by simply subscribing to it, which means receiving names that have three or more characters in them. All we do is call the server and pass the result to the “user” observable:

    .subscribe{ s: String ->
        mServer.findUser(s).subscribe { jo: JsonObject ->

We’re not quite done, though: we actually have another subscriber to that Observable:

    .subscribe { s: String ->

We need to let the user know we just issued a network call, so we show the progress bar (this is a suboptimal implementation, this logic should be done at the server level, but we’ll save this for later).

Note that I’m intentionally hiding a very important part of this logic in order to stay focused on the topic, and this also explains why we have two separate subscriptions. I explain why at the end of this article.

The “User” Observable

Next, we have the User Observable, which gets notified when the server sends us a response to the query “Does the user named ‘foo’ exist?”:

    .map { jo: JsonObject ->
        if (mServer.isOk(jo)) {
            User(jo.get("id").getAsString(), jo.get("name").getAsString())
        } else {
    .subscribe { user: User? ->
        addFriendButton.setEnabled(user != null)
        mUser = user

This Observable does two things: it updates our UI and it maps the JsonObject response to our data class User. If the call was a success, we assign this value to the field mUser.

The “Add friend” Observable

Finally, we have the “Add friend” button, which only gets enabled once we have a valid User. If the user presses that button, we issue another call to the server to request that person to be added as a friend and then we update the UI depending on the response:

    .subscribe { e: OnClickEvent ->
            .subscribe { jo: JsonObject ->
                val toastText: String
                if (mServer.isOk(jo)) {
                    toastText = "Friend added id: " + jo.get("id").getAsString()
                } else {
                    toastText = "ERROR: Friend not added"
                Toast.makeText(this, toastText, Toast.LENGTH_LONG).show();

Stepping back

This is a very different implementation of how you would write the code with regular Android calls, but in the end, it’s not just compact but it also divides our entire logic in four very distinct components that interact with each other in very clear ways. This is the macro level. At the micro level, these four components are not just self contained, they are also highly configurable thanks to operators, operations you can insert between your Observable and your Subscriber and which transform the data in ways that are easier for you to handle. I only have one such example of this in the code above (transforming an OnTextChangeEvent into a String) but you get the idea.

Another benefit that should be immediately obvious to you even if you don’t buy into the whole Rx paradigm shift yet is that thanks to Rx, we now have a universal language for observables and observers. I’m sure that right now, your Android code base contains a lot of such interfaces, all with subtly different method names and definitions and all needing some adapaters to be inserted before they can talk to each other. If ever you felt the need to write an interface with a method called "onSomethingHappened()“, Rx will be an immediate improvement.


I have barely touched upon operators and this topic in itself is worthy of an entire book but I’d like to spend a few minutes to give a quick example of the power of operators.

Going back to our activity, remember that as soon as the user types more than three characters, we send a query to the network for each character typed. This is a bit wasteful: a lot of phone users are very fast typists and if they type letters in quick succession, we could be sparing our server some extra work. For example, how about we define that we only call the server when the user stopped typing?

How do we define “stopped typing”? Let’s say that we’ll decide that the user stopped typing if two keypresses are separated by more than 500ms. This way, a quick typing of “cedric” will result in just one server call instead of four without this function. How do we go about implementing this?

Again, the traditional approach would mean that each time our text change listener is invoked, we compare the current time with the timestamp of the last character typed and if it exceeds a certain value, we trigger the event.

As it turns out, observables have an operator that does this for us called debounce(), so our code becomes:

    .doOnNext {(e: OnTextChangeEvent) ->
    .map { e: OnTextChangeEvent -> e.text().toString() }
    .filter { s: String -> s.length() >= 3 }
    .debounce(500, TimeUnit.MILLISECONDS)
    .subscribe { s: String -> mNameObservable.onNext(s) }

I know what you’re thinking: “You just picked a problem that could be solved by an operator that already exists”. Well, kind of, but that’s not my point. First of all, this is not a made up problem and I’m sure a lot of developers who write user interface code will agree that this is pretty common.

However, my point is more general than this: the debounce() operator has nothing to do with Android. It is defined on Observable, which is the base library we are using. It’s an operator that’s generally useful to have on any source that emits events. These events might be graphical in nature (as is the case here) or of any other kind, such as a stream of network requests, coordinates of a mouse cursor or capturing data from a geolocation sensor. debounce() represents the general need for getting rid of redundancies in streams.

Not only do we get to reuse an existing implementation without having to rewrite it ourselves, we preserve the locality of our implementation (with the traditional approach, you would probably have polluted your class with a couple of fields) and we maintain the composition abilities of our observable computations. For example, do you need to make sure that the user is not adding themselves before calling the server? Easy:

    // ...
    .filter { s: String -> s.length() >= 3 }
    .filter { s: String -> s != myName }

Wrapping up

At this point, you should probably take a look at the full source so you can step back and see how all these pieces fit together. It’s obvious to me that the view of decomposing your problem in terms of observables that emit values that subscribers observe is extremely powerful and leads to cleaner, more self contained and composable code. The non-Rx version of this class feels incredibly messy to me (a lot more fields holding state, methods added just to satisfy slightly incompatible listener interfaces and a total lack of composability).

Having said that, our work here is not over:

  • The threading model is all wrong. I intentionally omitted any thread consideration in this article in order to stay on topic but the code as it is right now is extremely suboptimal. It works but you shouldn’t ship it in its current form. In the next installment, I will show how Rx makes it trivial to get Android’s multithreaded model right.
  • There is a lot of UI code peppered through the business logic. It’s messy, it obfuscates how our UI works and it’s hard to maintain. We’ll have to fix that as well.

Stay tuned!

Update: Discussion on reddit.

Read part 2.

Swift, Apple’s new language

Apple just announced a new language called Swift. I took a look at the language manual, here is a quick overview:

  • Statically typed with type inference.
  • Generics.
  • Closures.
  • No exceptions.
  • Extension methods.
  • Properties (syntax similar to C#), including lazy properties with the "@lazy" annotation.
  • Functions, methods and type (static) methods.
  • Support for observers (with "willSet" and "didSet"). Interesting to see the observer pattern baked in a language although I’m more partial to event buses for this kind of thing.
  • Enums.
  • Classes and structures (structures have restrictions regarding inheritance and other things).
  • For and while loops (statements, not expressions).
  • "mutating" keyword.
  • Named parameters.
  • Deinitializers (finalizers).
  • Protocols (interfaces).
  • Optional chaining with "a?.b?.c" and forced dereference with "!."“.
  • Convenient “assign and test”: "if let person = findPerson() ...".
  • Type casting with "is", down casting with "as?" (combines nicely with the "let" syntax. Ceylon does it right too).

Very interesting overall, and a clear step up from Objective C. From the feature set, I would say the language that Swift has the most overlap with is Kotlin, which is great news for Apple developers.

Update: discussion on reddit.

No silver bullet: reloaded

I had to read the recent article “Functional Programming Basics”, by Robert Martin, several times to make sure that I wasn’t missing any deeper point, but every time, I just became more and more concerned by the sloppiness of the arguments presented.

Here are a few thoughts.

First, it’s almost certainly true that functional programming is the next big thing.

We’ll see about that. This has been predicted for several decades and it still hasn’t happened. Some say that it will never happen, others say it’s already happened and more moderate people just observe that quite a few core concepts of functional programming have been leaking into mainstream languages for a while, so in some way, functional programming is slowly percolating through the industry.

Functional programming is programming without assignment statements.

This is a pretty silly statement since functional programming languages have assignments. Maybe the emphasis should be more on referential transparency or perhaps “variables can never be assigned more than once” would be a more accurate description of one of the important tenets of functional programming.

Let’s look at a functional program for the squares of integers. We’ll use the language Clojure for this, though the concepts we’re going to explore work the same in any functional language.

(take 25 (squares-of (integers)))

I find the use of Clojure (or more generally, Lisp) to illustrate his points a bit puzzling because while the question of whether functional programming will play a big part in the software industry’s future is still up for debate, it seems pretty clear to me that 1) object orientation and 2) static typing are two concepts that have clearly established themselves as powerhouse ideas that have an established track record and a bright future ahead of them. Scala or Kotlin would have been a better choice, especially they are not that much more verbose on such a trivial example:

// Kotlin { it * it }.take(25)

Read that sentence again; because I did something important there. I took the three separate definitions of the functions that I gave you in the preceding paragraph and combined them into a single sentence. That’s called: (are you ready for the buzzword?) Referential Transparency. [cue: Fanfare, glitter, ticker tape, cheering crowds]

This is where I start being a bit confused by Robert’s point, because if anything, the code he just showed is illustrating a lot more composition than referential transparency.

If you are trying to sell functional programming as a big thing in order to prop up your consultant business, you should probably skip the trivial stuff and start with the hard problems right away, such as preserving referential transparency and immutability, demonstrating the power of composition while accessing a database, returning JSON data from a servlet and have another servlet modify that same database. Do this with a functional programming language and with snippets of code that show clear advantages over today’s Java and C# based solutions and you have yourself a market.

Honestly, we programmers can barely get two Java threads to cooperate.

Not really. Hundreds of thousands of Java programmers write multithreaded code every day and they are doing just fine, because the intricacies of parallelism are abstracted away from them by the frameworks they rely on to get their job done. I’m not talking just about complex Java EE or Spring-based containers but even the simple Tomcat or Jetty servers. More advanced programmers dabble with the amazing java.util.concurrent classes but even those never really have to think much about the hard problems that heavy paralellism involve, because domain gurus such as Brian Goetz and Doug Lea have already done all the hard work for them.

I continue to be wholly unconvinced by the claim that “the new era of multi cores is going to wreak havoc on our code bases unless we adopt a functional programming language”. This claim has two sides, both equally bogus.

As I mentioned above, the fact that we are now running our programs on multiple cores instead of multiple CPU’s brings very little practical difference for anyone writing multithreaded code. What processor/core/green thread/lightweight process your thread will be riding should have zero impact on how you structure your code. If your code was multithread safe on “n CPU’s / 1 core”, it will be equally multithread safe on “1 CPU / n cores” or even “n CPU’s / m cores”. The only major difference is that it now becomes possible to test for actual parallelism (not just concurrency) on inexpensive developer boxes.

The second part of the claim, that only functional programming languages can save us from the impending multicore apocalypse, is equally dubious but I’ll save this for another post, although I’ll point out that traditional OO and imperative programming has shown an amazing power of adaptation and versatility for several decades now, so whatever the next big paradigm will be, it will have to bring significant improvements to the table.

More importantly, if you could peer into the computer’s memory and look at the memory locations used by my program, you’d find that those locations would be initialized as the program first used them, but then they would retain their values, unchanged, throughout the rest of the execution of the program. In other words, no new values would be assigned to those locations.

This is a bizarre claim that shows that Bob only has a passing familiarity with the internals of how a compiler or a VM works. Just because the observable effects of a program are consistent with immutable structures does not mean that the underlying memory is not being shifted around by the OS or by the abstraction layer standing between the source code and the CPU.

Moreover, the claim that immutable structures automatically lead to easy concurrency is not just naïve but plain incorrect: there are a lot of algorithms (such as merge sort) that are inherently not parallelizable.

Overall, this is a fairly unconvincing article that seems to miss most of the well recognized benefits of functional programming (composability remains the aspect that has the highest impact on my code on a daily basis) while touting advantages that range from poorly understood concepts to plain incorrect claims.

A note on null pointers

It’s the second time in a few weeks that I have read something along the lines of:

In a year at foursquare, I’ve seen exactly one null pointer exception.

To me, this kind of statement is similar to

Ever since I switched to BASIC, I haven’t seen a single stack trace.

and it shows a fundamental misunderstanding of what a programming error is. Similarly, hearing Tony Hoare say that introducing null was a “one billion dollar mistake” makes me really question if he understands the fundamental idea behind crashes.

Null pointer exceptions, stack traces, core dumps, guru meditations, etc… are usually all the manifestation of a simple phenomenon: unexpected values. Somehow, your code expected a value but received another one. This is a programming error, also known as a bug.

If you are using a language that supports monadic Option/Maybe types, by definition, you will not be seeing any null pointer exceptions, but this doesn’t mean that you have solved the “unexpected value” problem.

In languages such as Java or C, null pointers translate into crashes that are hard to miss and usually easy to diagnose. What would be the equivalent of this in Scala?

The answer is easy: receiving a None when you expected a Some. Because of the way you thread monads through your functions, such a bug will not trigger any exceptions nor any crash: your None value will happily make its way through monadic transforms and will simply result in a no-op every time it’s being mapped. Of course, your program will yield an unexpected value that you will take notice of at some point, and then you will have to go through the painful process of retracing all the Options that your transformations have gone through to find out which one returned None when it should have returned a Some.

At this point, you should probably ask yourself: if this piece of code should never return a None, why return an Option at all? In such cases, you are better off unboxing your Option and returning its raw value. Of course, the downside of this is that you might have to lift your value again if the following computations happen in the Option monad as well.

As you can see, Options come with their own trade offs and hard design decisions as well. No such thing as a free lunch.

Which leads us to the next big misconception about this class: Option doesn’t solve null pointer exceptions nor unexpected values, it just saves you from having to test against null. And actually, doing so (e.g. pattern matching your option against Some/None) is usually considered bad practice, even though it’s sometimes necessary for Java interoperability reasons.

Personally, I favor the way Groovy, Fantom and Kotlin address the problem over having to lift all my values into Option. For example, here is how Kotlin lets you ignore null values encountered along a chain of invocations:


If either of these accesses returns null, the result of this expression will be null. No need to put your values into monadic boxes nor mapping through them, just use the standard composition operator you are familiar with if you are using a C family language.

By all means, do use Option whenever you can if you are programming in Scala (and you will probably realize that you can’t use it as much as you would like because Java libraries, and some Scala ones, are just not implemented to take Option parameters), but don’t listen to people who tell you that because you are no longer seeing any null pointer exceptions, your code is safer. It’s not. You still have to fix your bugs.

Erasure vs reification

Java 5 was released in 2004 and introduced generics to the language using a mechanism called “erasure”. In the following couple of years, a lot of discussions took place comparing this approach to its counterpart, usually referred to as “reified generics”. The discussions then tapered down for a few years, until recently when two languages brought this topic back on the scene by supporting reified generics. These two languages are Gosu (created and used in production by GuideWire) and Kotlin (under development, created by JetBrains).

There is a lot of literature available on the subject of erasure and reified generics, but I thought I would take a few minutes to summarize the current state of the world and share a few thoughts on the pros and cons of each approach.

Let’s start with a few concrete examples. Here are snippets of code which do not compile under an erasure system but which would work fine with reified generics. If you are not familiar with the issues involved, here is a short rule of thumb that should allow you to understand what is going on: whenever you see a generic type, replaced it with Object (since that’s exactly what’s happening behind the scenes):


public class Test<K, V> {
  public void f(K k) {

  public void f(V v) {
} name clash: f(K) and f(V) have the same erasure
  public void f(K k) {
              ^ name clash: f(V) and f(K) have the same erasure
  public void f(V v) {

The workaround here is simple: rename your methods.


public class Test {
  public <T> void f() {
    Object t;
    if (t instanceof List<T>) { ... }
} illegal generic type for instanceof
    if (t instanceof List<T>) {}

There is no easy workaround for this limitation, you will probably want to be more specific about the generic type (e.g. adding an upper bound) or ask yourself if you really need to know the generic type T or if the knowledge that t is an object of type List is sufficient.


public class Test {
  public <T> void f() {
    T t = new T();
} unexpected type
found   : type parameter T 
required: class
    T t = new T();

This case is also a bit tricky but since it is, in my experience, more common than the others, I’ll spend a little more time discussing it.

As mentioned above, the virtual machine has no knowledge about the type T, which it only sees as an Object, so it won’t allow you to create an instance of it.

This is a good thing.

From a type standpoint, you know nothing about the T so you shouldn’t be able to instantiate it with the amount of knowledge you have. For all you know, T could be an abstract class, an interface or a class with no default constructor. In this case, type erasure keeps you honest by limiting what you can do on T to the operations that it knows about.

If you need to manipulate T, you will have to enable this through the type system. Do you need a new instance? Pass an additional Factory<T>. Do you need to call query()? on it? Create a type that contains this method and constrain T with it.

In 2006, Neal Gafter came up with a clever idea to make it possible to instantiate such erased types. He dubbed the technique “super type tokens”, and like most good ideas, it’s extremely simple: you force the creation of an anonymous class that contains the generic type, which you can then retrieve by a clever use of the introspection API.

Here is an example:

abstract class TypeReference<T> {}
public class TT {
  public static <T> void f(TypeReference<T> t) {
    ParameterizedType pt = (ParameterizedType)

  public static void main(String[] args) {
    TT.f(new TypeReference<String>() {});

This will print "class java.lang.String" on the console, even though the method printing it is completely generic. The trick is on the highighted line: notice the empty braces, which create an anonymous instance of TypeReference, setting the stage for the introspection code in the method to retrieve the generic type.

Shortly thereafter, Bob Lee picked up this feature, fleshed it out and included it in Guice under the name TypeLiteral. Scala supports a similar feature called Manifest.

Ever since Java 5 came out, and despite my initial fears, I can’t say that I have been bothered much by the absence of type information in Java generics, and I am tempted to generalize this observation to the general Java population. Erasure turns out to have quite a few advantages and as it turns out, reified generics come with their own set of issues. Here are some of them.

The main problem is that reified generics would be incompatible with the current collections. In binary form, for sure, and probably in source form as well (we would want to distinguish between collections making used of reified types from their older counterpart). Rewriting would probably be mostly a matter of copy/pasting, except for the parts that make use of introspection and which would need to be adjusted. The generated byte code would also contain more information. For example, the following test:

o instanceof List<String>

would now test that the object is an instance of List but also that its elements are of type String. That’s quite a bit more work.

The extra type information also impacts the interoperability between languages within the JVM but also outside of it. For example, Scala recently announced some progress on its .Net compiler, which contains the following caveat:

The key limitation for the moment is that Scala programs cannot use libraries in .Net that are compiled using CLR generics, such as the .Net collections.

This is just a consequence of the fact C# has reified generics, and bytecode containing this supplemental type information requires more work to be parsed and converted in a form suitable for the client, as opposed to a simple List type.

So, where does this leave us?

Erasure has proven to work quite well for Java, and actually for quite a few other languages as well. Besides the two languages that I named above, there are only two other popular languages that support reified generics: C# and C++. All the others use erasure of some sort, and overall, it’s hard to argue that either approach brings a significant improvement in ease of use.

All in all, I am pretty happy with erasure and I’m hoping that the future versions of Java will choose to prioritize different features that are more urgently needed, such as closures or an improved module system.

Oh, and happy Java 7 day everyone!

Five reasons why you should rejoice about Kotlin

As you probably saw by now, JetBrains just announced that they are working on a brand new statically typed JVM language called Kotlin. I am planning to write a post to evaluate how Kotlin compares to the other existing languages, but first, I’d like to take a slightly different angle and try to answer a question I have already seen asked several times: what’s the point?

We already have quite a few JVM languages, do we need any more?

Here are a few reasons that come to mind.

1) Coolness

New languages are exciting! They really are. I’m always looking forward to learning new languages. The more foreign they are, the more curious I am, but the languages I really look forward to discovering are the ones that are close to what I already know but not identical, just to find out what they did differently that I didn’t think of.

Allow me to make a small digression in order to clarify my point.

Some time ago, I started learning Japanese and it turned out to be the hardest and, more importantly, the most foreign natural language I ever studied. Everything is different from what I’m used to in Japanese. It’s not just that the grammar, the syntax and the alphabets are odd, it’s that they came up with things that I didn’t even think would make any sense. For example, in English (and many other languages), numbers are pretty straightforward and unique: one bag, two cars, three tickets, etc… Now, did it ever occur to you that a language could allow several words to mean “one”, and “two”, and “three”, etc…? And that these words are actually not arbitrary, their usage follows some very specific rules.

What could these rules be? Well, in Japanese, what governs the word that you pick is… the shape of the object that you are counting. That’s right, you will use a different way to count if the object is long, flat, a liquid or a building. Mind-boggling, isn’t it?

Here is another quick example: in Russian, each verb exists in two different forms, which I’ll call A and B to simplify. Russian doesn’t have a future tense, so when you want to speak at the present tense, you’ll conjugate verb A in the present, and when you want the future, you will use the B form… in the present tense. It’s not just that you need to learn two verbs per verb, you also need to know which one is which if you want to get your tenses right. Oh and these forms also have different meanings when you conjugate them in past tenses.

End of digression.

The reason why I am mentioning this is because this kind of construct bends your mind, and this goes for natural languages as much as programming languages. It’s tremendously exciting to read new syntaxes this way. For that reason alone, the arrival of new languages should be applauded and welcome.

Kotlin comes with a few interesting syntactic innovations of its own, which I’ll try to cover in a separate post, but for now, I’d like to come back to my original point, which was to give you reasons why you should be excited about Kotlin, so let’s keep going down the list.

2) IDE support

None of the existing JVM languages (Groovy, Scala, Fantom, Gosu, Ceylon) have really focused much on the tooling aspect. IDE plug-ins exist for each of them, all with varying quality, but they are all an afterthought, and they suffer from this oversight. The plug-ins are very slow to mature, they have to keep up with the internals of a compiler that’s always evolving and which, very often, doesn’t have much regards for the tools built on top of it. It’s a painful and frustrating process for tool creators and tool users alike.

With Kotlin, we have good reasons to think that the IDE support will be top notch. JetBrains is basically announcing that they are building the compiler and the IDEA support in lockstep, which is a great way to guarantee that the language will work tremendously well inside IDEA, but also that other tools should integrate nicely with it as well (I’m rooting for a speedy Eclipse plug-in, obviously).

3) Reified generics

This is a pretty big deal. Not so much for the functionality (I’ll get back to this in the next paragraph) but because this is probably the very first time that we see a JVM language with true support for reified generics. This innovation needs to be saluted.

Correction from the comments: Gosu has reified generics

Having said that, I don’t feel extremely excited by this feature because overall, I think that reified generics come at too high a price. I’ll try to dedicate a full blog post to this topic alone, because it deserves a more thorough treatment.

4) Commercial support

JetBrains has a very clear financial interest in seeing Kotlin succeed. It’s not just that they are a commercial entity that can put money behind the development of the language, it’s also that the success of the language would most likely mean that they will sell more IDEA licenses, and this can also turn into an additional revenue stream derived from whatever other tools they might come up with that would be part of the Kotlin ecosystem.

This kind of commercial support for a language was completely unheard of in the JVM world for fifteen years, and suddenly, we have two instances of it (Typesafe and now JetBrains). This is a good sign for the JVM community.

5) Still no Java successors

Finally, the simple truth is that we still haven’t found any credible Java successor.

Java still reigns supreme and is showing no sign of giving away any mindshare. Pulling numbers out of thin air, I would say that out of all the code currently running on the JVM today, maybe 94% of it is in Java, 3% is in Groovy and 1% is in Scala. This 94% figure needs to go down, but so far, no language has stepped up to whittle it down significantly.

What will it take? Obviously, nothing that the current candidates are offering (closures, modularity, functional features, more concise syntax, etc…) has been enough to move that needle. Something is still missing.

Could the missing piece be “stellar IDE support” or “reified generics”? We don’t know yet because no contender offers any of these features, but Kotlin will, so we will soon know. Either way, I am predicting that we will keep seeing new JVM languages pop up at a regular pace until one finally claims the prize. And this should be cause for rejoicing for everyone interested in the JVM ecosystem.

So let’s cheer for Kotlin and wish JetBrains the best of luck with their endeavor. I can’t wait to see what will come out of this.

Oh, and secretly, I am rooting for Eclipse to start working on their own JVM language too, obviously.

Link to the discussion on Google+