Author: rzwitserloot

Using multiple cores, the basics

The problem with multicore code

Having an application use more than one CPU during its execution introduces an explosion of execution paths. Instead of a simple “first Java executes the first line of this method, then, it executes the second line of this method, all the way to the end,” it becomes: “An arbitrarily chosen thread chooses an arbitrary number of instructions to execute, then the VM will make an arbitrary choice as to whether or not to make any changes to fields this thread performed visible to other threads unless you’ve done a good job on reading the Java memory model to ensure propagation of these changes.”
The former is easily understood. The latter is (largely) untestable and unfollowable, and therefore, it’s easy to write code that looks good, passes all tests, runs great on your machine… and bombs in production five days later, in a way that is utterly mystifying; no exception pointing at the problem. The problem won’t be reproducible (do the same thing, click the same buttons, feed in the same data… and now it works fine. And then tomorrow when that important customer logs in, it’ll fail again on you). Finding a bug like this is literally on the order of 100 to 1000 times harder to find, so avoiding even one such bug is ‘worth’ having 100 others.
Nevertheless, using all the cores in your CPU is:

  • Practical, in that just running it all on one can be unacceptably wasteful for performance, but more importantly
  • Inevitable, because you don’t write every line of code in your app yourself (you use libraries; the core java.* libraries at the very least), and THOSE will sometimes use threads.

So, what is a programmer to do?
You may feel like the only right answer is for the programmer to completely understand threading and learn how to write bug free code (given that tests can’t really find these bugs anyway, and even if you so happen to run into one, they are hard to reproduce and its hard to figure out which line(s) of code are faulty when you do find them). This means reading up on the entire Thread API and reading the Java Memory Model front-to-back (If thread 1 writes some data to some field, and sometime later thread 2 reads this field, it may or may not see the change. The JMM defines when Java is, and more importantly is not,  required to have one thread ‘see’ updates of another.
But that’s a fool’s errand. You cannot guarantee writing bug free code.
Instead, anytime you have a need to have your code run on multiple cores, try to push towards going big or going small; in both cases, you NEVER call ANY method on Thread (possibly you do some Thread.sleep in the ‘going big’ style), you have no need whatsoever of synchronized, no state is shared between threads, and you don’t create new threads (some framework / library does this for you).

Going Big

Write or use a framework that runs your code for you and which takes care of multithreading. For example, a web framework is such a thing: You start it, and then it runs your code (and it takes care of creating whatever threads are necessary). Generally in these cases you do not write public static void main, or you do, but all that your main does is fire up the framework while you configure it or pass to it a bunch of ‘handlers’, and then the framework does the hard work for you.
Such frameworks are really good at setting up threading. This way each handler can be in its own little world; if a thread shares no data with any other, reasoning about threads is much, much simpler. Take webservers: Any given web handler simply does not get to interact with other handlers. It’s not like you can ask the web framework for a list of WebHandler objects that you can then call methods on.
There is often still a need to have 2 handlers interact. However, you should do this interaction by using tightly controlled communication channels where the issue of how concurrent operations interact is either irrelevant, or very well specified. There are 2 usual ways to go:

  • Databases. Databases define how concurrently running operations go via transactions. So, use transactions, make your database calls, and the database takes care of it. Communications-via-database is very common in web frameworks. You can use JDBI if you want to write SQL directly, or use Hibernate if you just want to store objects persistently.
  • Message queues, such as RabbitMQ. The idea behind these is that one handler will tell the message queue framework that it is interesting in all ‘foo’ events, and the other will tell the message queue framework: Here’s a ‘foo’ event, please deliver it to all handlers that said they’d like to know about it. All communication between handlers goes via the message queue.

Editor’s note: the projects mentioned here are very far from being your only choices. JDBI and Hibernate are good, but note that there are projects like myBatis, jOOQ, and others for SQL-ish libraries, and Hibernate is only one JPA implementation among many. Likewise for messaging: RabbitMQ is an AMQP 0.9 server, but you don’t use the JMS standard to talk to RabbitMQ; you do use JMS when using ActiveMQ, HornetQ, or any other of … quite a few messaging platforms. This is not saying that RabbitMQ or Hibernate, et al, are bad – your Editor uses them daily – but remember that your choices are not limited.

Going Small

Reduce the code that needs to run multithreaded to the smallest possible thing it can be, and then write a single stream/collection based operation that uses threading to apply this operation to a great many inputs concurrently. The fork/join framework is the usual go-to here.
Imagine you are writing a bitcoin miner. This operation can be described as follows:
GIVEN: A list of a few million randomly generated codes to try, and exactly 1 input block.
TASK: Write the hash into that 1 input block, hash it, and see if the hash ends up having the appropriate amount of 0s at the very end.
You can do this by going small: Write a trivial method (it won’t be larger than half a screen’s worth, common in this ‘go small’ model) which injects the code into the block, hashes it, and returns an empty string if the hash is not suitable, and if you hit the jackpot, returns the block. Then tell the framework you only want the non-empty returned values and voila, you’ve written a parallel bitcoin miner without ever having to touch java.lang.Thread.

Libraries

In practice, especially for the ‘going big’ route, you may need to have a cache or some such that should be shared between whatever threads your web framework is making for you, but try to find libraries for this, too.1 There are some tricks to interacting with such libraries. Generally, you have to go ‘atomic’. For example, you can use the various collections in the java.util.concurrent package, but, the only guarantees it can make is that a single method call does the right thing. It cannot guarantee that a series of calls does the right thing. So, don’t do this:

if (concurrentMap.containsKey(myKey)) {
    String v = concurrentMap.get(myKey);
    operateOn(v);
}

Instead, do this:

String v = concurrentMap.get(myKey);
if (v != null) {
    operateOn(v);
}

The bad example can cause a problem: What if, in between the call concurrentMap.containsKey and the call to concurrentMap.get, some other thread removes the entry from the map? You’ll now call operateOn on a null value, not what was intended. Problematically, if this can happen, it will happen, but only very very rarely. Tests are unlikely to catch this problem.
Don’t do this:

String v = myCache.get(myKey);
if (v == null) {
    v = doExpensiveCalculationOfValue(myKey);
    myCache.put(myKey, v);
}

Instead, do this:

myCache.computeIfAbsent(myKey, k -> doExpensiveCalculationOfValue(k));

The reason to use computeIfAbsent here is because the first snippet can lead to running doExpensiveCalculationOfValue more than once. In the first snippet, imagine two (or more) threads get to the code with the same key about the same time. They’ll both find that there is no associated value (yet), so they both call doExpensibeCalculationOfValue and then they both set it. With computeIfAbsent, provided you use a properly concurrent map, such as java.util.concurrent.ConcurrentHashMap for example, doExpensiveCalculationOfValue is only ever called once, guaranteed.

Lessons

  • Use frameworks, either large: Web frameworks, or small: fork/join.
  • Thread-to-thread communication uses abstractions like message queues or databases. Don’t share state, don’t touch any fields from more than one thread.
  • If you must have direct thread-to-thread interop, use libraries with collections types that are designed for this, such as java.util.concurrent and guava‘s CacheBuilder stuff. When interacting with these collections, know that consecutive method calls to them have no guarantees of internal consistency, so reduce it to one call. Be aware of methods like java.util.Map‘s computeIfAbsent to enable this.

Footnotes

1 Editor’s note: caches are nontrivial in and of themselves. Go for “transactional caches” if you can – or just trade the speed that a cache would give you for the reliability that using a system of record gives you. Figure out what you need and do what fulfills that, before thinking that a cache is magic performance sauce – and note that the whole point of this article is that “magic performance sauce” usually isn’t what it says it is.

Non-Blocking vs. blocking I/O: Go with blocking.

You might have heard or read a various things on the Internet extolling the virtues of writing your applications to be ‘non blocking’. Java offers various libraries that help with this endeavour; Undertow is a java web server that lets you write your servlet handlers non-blocking, for example. XNIO is a more general I/O library that lets you write non-blocking code in simpler fashion.
If you take nothing else away from this article, at the very least use one of those libraries to do your non-blocking work.
But the real point is: don’t write non-blocking code.
That’s right: Don’t write non-blocking code. It’s not worth it. You’re using Java, it ships with a garbage collector. Going non-blocking is like manually cleaning up your garbage: in theory it’s faster, in practice it’s more annoying, more error prone, and performance-wise usually either irrelevant or actually slower.
Said differently: Programming for non-blocking is a lot harder than you think it is, and the performance benefits are far less than you think they are.

What does ‘non-blocking’ mean?

Let’s say you wish to read from a file, so you create a FileInputStream and proceed to call the read() method on it. What happens now? Well, the CPU needs to send a signal that travels to various associated chips on the motherboard, to eventually end up at the disk controller, which will then query some cells in the solid state array and returns this to you. Heaven forbid if you still have a spinning disk, in which case we might have to wait for something as archaic as a motor to spin some metal around and a pick-up-needle like thing to swing in there to read some bits! Even with very fast hardware this process takes ages, at least, as far as the CPU is concerned.

From Your Editor: Want to know more about how your computer actually works and why? A good book to read is Charles Petzold’s Code.

So what happens? Well, the CPU will just… wait. In parlance, the thread executing the read() call ‘blocks’, that is, it’ll stop executing while the disk system fetches the requested bytes, and the CPU is going to go do some other stuff (perhaps fill the audio buffer with the next bit of the music you’re playing in the background, do some compilation, et cetera), and if there’s nothing else to do, it’ll idle for a bit and save power.
Once the disk system has fetched the bytes and returned them to the CPU, the CPU will pick back up where it left off and your read() call returns.
That’s how ‘blocking’ works. Blocking is relevant for lots of things, but it’s almost always going to come up when talking to other systems: Networks, Disks, a database, et cetera: Input/Output, or I/O.
There’s another way to do it, though: ‘non-blocking’ code. (Yes, the quotes are intentional.) In non-blocking, things work a little differently. Instead of the thread freezing when there is no data yet, a non-blocking read() returns the bytes that are available, and execution continues. If you didn’t get it all, well, call it some more, or later. In practice, there are 2 separate strategies to make this work correctly:

  1. The functional/closures/lambda way: You ask for some data, and you provide some code that is to be executed once the data is fetched, and you leave the job of gathering this data (waiting for the disk, for example), and calling this code, to whatever framework you’re using, and it might use non-blocking I/O under the hood.

  2. The multiplexing way: You are a thing that responds to many requests (imagine a web server, designed to handle thousands of simultaneously incoming requests), and if there isn’t yet enough data to respond to some incoming request, you simply… go work on some other request. You round-robin your way through all the requests, handling whatever data there is.

It’s not as fast as you think it’ll be

Generally, ‘non blocking is faster!’ is a usually incorrect oversimplification that is based on the idea that switching threads is very slow. Let’s compare a webserver handling 100 simultaneous requests written with blocking I/O in mind, versus a webserver that is written non-blocking style. The blocking one obviously needs 100 threads to execute simultaneously; most of them will be asleep, waiting for data to either arrive or be sent out to the clients they are handling.
The blocking webserver will be switching active threads a lot. Contrast this to the non-blocking webserver: It might well run on only a single core: The one thread this webserver has will sleep as long as all 100 connections are waiting for I/O, and if even a single one has something to do, the thread awakes, does whatever job is needed for all connections that have data available, and only goes back to sleep when all 100 connections are awaiting I/O.

From Your Editor: The nonblocking approach just described is what Python and Node use nearly exclusively, by the way.

Thus, the theory goes: Switching threads is slow, especially compared to.. simply.. not doing that.
But this is an oversimplified state of affairs: In actual practice, modern kernels are really good at taking care of the bookkeeping that threads require. You can create many thousands of threads in Java and it’ll be fine. Furthermore, your one non-blocking thread model is also switching contexts: Every time it jumps to handling another connection, it needs to look at a completely different chunk of memory. This will usually invalidate the caches on your CPU and loading a new page into cache takes on the order of magnitude of 600 or so CPU operations. The CPU just sits there doing nothing while a new page is loaded into cache. This cost is similar to (and takes a similar duration as) thread switches. You gained nothing.
For more on why you’re not going to see a meaningful performance boost (in fact, why you’ll probably get less performance), check out Paul Thyma’s Presentation “Thousands of Threads and Blocking I/O: The Old Way to Write Java Servers Is New Again (and Way Better)“. Thyma is the operator of mailinator.com; he knows a thing or two about handling a lot of simultaneous traffic.

It’s really, really complicated

The problem with non-blocking is two-fold: First of all, modern CPUs definitely have more than one core, so a web server that has a single thread handling all connections in a non-blocking fashion is actually much slower: All cores but one are idling. You can easily solve this problem by having as many ‘handler threads’ all doing non-blocking operations, as you have cores. But this does mean you get none of the benefits of having a single thread. That is, you do not get to ignore synchronization issues unless you’re willing to pay a huge performance fine (bugs that occur because of the order in which threads execute, so called ‘race conditions’ – these bugs are notoriously hard to find and test for).
Secondly: The point of non-blocking is that you do not block: Whenever you are executing in a non-blocking context it is a bug to call any method or do anything that DOES block. You can NOT talk to a database in a non-blocking handler, because that will, or might, block. You can’t ping a server. Something as simple and innocuous as writing a log might block. The problem is, if you do something that does block, you won’t notice until much later when your server seems to fall over even at a fairly light load. You won’t get a log message or an exception if you mess up and block in a non-blocking context. Your server just stops being able to handle more than a handful of requests (equal to the # of cores you have) in a timely fashion for a while. Whoops! Your big server designed to handle millions of users can’t handle more than 8 people connecting at once because it’s waiting for a log line to be flushed to disk!

From Your Editor: Of course, you could write all of those operations in a nonblocking fashion as well, which is what Python and Node.JS have to do – and there’s a good reason why such things lead to a condition known as “callback hell.” It can be done, obviously; they do it. It’s also incredibly ugly and error-prone; it’s a good example of the “cure” being worse than the “disease.”

This really is a big problem: As a rule, most Java libraries simply do not mention whether or not they block – you can’t rely on documentation and you can’t rely on exceptions either.
Trying to program in this ‘you cannot block!’ world is incredibly complicated.
For a deeper dive into the nuttiness of that programming model, read “What color is your function?” by Bob Nystrom, a language designer on the Dart team.

Soo.. completely useless, huh? Why does it exist, then?

Well, non-blocking probably exists because, like the idea of the tongue map, it’s just a very widespread myth that non-blocking automatically means it’s all going to run significantly faster.
It’s not completely useless, though. The biggest benefit to writing your code non-blocking style, is that you gain full control of buffer sizes. Normally, in the blocking I/O model, most of the state of handling your I/O is stuck in the stack someplace: If you ask an inputstream to read a JSON block, for example, and only half of the data is readily available, half of that data is processed by your thread into the data structures you’re reading that JSON into, and then the thread freezes. That data is now stuck in bits of heap memory, and the rest is in this thread’s stack memory. Contrast to a non-blocking model where you’ll have made a ByteBuffer explicitly and most of the data is in there.
CPU stack, ByteBuffer: Potato, potato: It’s all RAM. The difference is: stacks are at least 1MB, and the same size for each and every thread in the VM. ByteBuffers are fully under your control. You can make them smaller than 1MB, you can dynamically update the size, and you can have different buffer sizes for different connections, for example.
Non-blocking code tends to have a smaller memory footprint because of this, if the coder’s aware and takes advantage of the possibility, and that’s the one saving grace it does offer.
It is up to you to figure out if that benefit is worth the hardship and performance hit of going with the non-blocking model. Generally, RAM is very cheap. I’d wager my bets on the blocking model. After all, we don’t write all our software in hand tuned machine code either!

How to use Scanner

The java.util.Scanner class built into java can be used to read inputs. It can for example be used to set up an interactive command line session, where you prompt the user for stuff and your program acts on what the user types in. You can also point it at a file or network connection if you like.

TL;DR: Do not use nextLine()

Phew, glad we got that out there. If you don’t want to read the rest, just never call that method. Instead, to read a full line of text, you call:

scanner.useDelimiter("\r?\n");
String entireLine = scanner.next();

Creating scanners

To create a scanner, just call its constructor, and pass in the data you want to read from. You have lots of options:

import java.util.Scanner;
import java.io.InputStream;
import java.nio.file.Paths;
import java.nio.charset.StandardCharsets;
Scanner in = new Scanner(System.in);
Scanner in = new Scanner(Paths.get("/path/to/file"), StandardCharsets.UTF_8);
Scanner in = new Scanner("input to read");
try (InputStream raw = MyClass.class.getResource("listOfStates.txt")) {
  Scanner in = new Scanner(raw, StandardCharsets.UTF_8);
}

The above code shows 4 different things you can read with scanner:

  1. “Standard input” – what the user types into the command line.
  2. Files. You can also pass a java.io.File object if you still use the old file API.
  3. A string containing data you want to read. The string is read directly; you can’t pass a file path (see the second example for reading files).
  4. Input streams, byte channels, and Readables.

Note that in all cases where you’re dealing with byte input, you can opt not to explicitly pass along a character encoding, but don’t do this! – this means you get the platform default and you have no idea what that’s going to be. That is a good way to write code that works on your computer but fails elsewhere. When in doubt, pass in StandardCharsets.UTF_8 just like in the examples above.

Reading data from a scanner

To read data from a scanner, first decide what you want to read. Then, call the right next method:

A whole number

Call nextInt(), nextLong(), or nextBigInteger(). If you don’t know what these are, call nextInt().

A number in hexadecimal

Call nextInt(0x10), or nextLong(0x10).

A number with fractional elements (i.e., numbers after a decimal point)

Call nextDouble(). However, be aware that how people type floating point numbers depends on where they live. You may want to call useLocale() first to explicitly set the style.

A boolean value

Call nextBoolean()

A single word

Call next()

A whole sentence

Ah, well, that’s tricky. Do not call nextLine(), that doesn’t do what you might think based on its name. Instead, read the next section.

Tokens?

The way Scanner works is by first reading a so-called ‘token’, and then converting this into the representation you asked for. So, if you call nextInt(), a single token is read and this token is then parsed as an integer. If it isn’t an integer, for example because the user typed hello instead of 85, an InputMismatchException is thrown. Catch this exception if you wish to respond to faulty inputs (I advise not calling hasNextInt() and friends either; but that’s for another article).
By default, ‘read the next token’ is done by reading until end-of-input, or any whitespace (tabs, spaces, enters, anything goes). This means that ‘read the next entire line’ just isn’t a thing scanner can do: That would involve reading multiple tokens.
The trick is to turn ‘the entire line’ into a single token. To do that, just tell Scanner to delimit on something else! The delimiter is what separates tokens. So, to read an entire line, first tell scanner that tokens are separated only by newlines, then just read the next token:

scanner.useDelimiter("\r?\n");
String entireLine = scanner.next();

This sets the delimiter to be the newline character, optionally preceded by the carriage return character (windows formatted text files tend to put that in front of their newlines, other OSes don’t).
If you want to return to whitespace-separated tokens, you just call scanner.reset(), though, note, that also resets useLocale.

So what does nextLine do?

nextLine() is unlike all the other next methods. It doesn’t read tokens. Instead, it just keeps reading characters up to the next newline character and returns what it read. It completely ignores the delimiter. This is problematic, because the next method family reads up to the delimiter but doesn’t ‘consume’ it. Thus, if you write this code:

Scanner scanner = new Scanner(System.in);
System.out.print("Enter your age: ");
int age = scanner.nextInt();
System.out.println("Enter your name: ");
String name = scanner.nextLine();

It won’t work: In the user types 5 and then hits enter, well, the 5 goes to nextInt() and that lone enter is all that nextLine is going to read. But if the user types 5 Jane Doe then you’ll end up with age = 5 and name = "Jane Doe". There’s no real way to try to fix this (you can try to call nextLine() twice but that breaks it for those who type spaces instead of enter); just don’t use nextLine, and change the delimiter instead.

Byte Order Marks (BOM)

The so-called Byte Order Mark is a special unicode character that has no visual representation. The point of it, is to start your text data with this pseudocharacter; it serves as a way to identify “Endianness” – that the text is encoded with UTF-16 (Little Endian), or UTF-16 (Big Endian), or UTF-8.
Java handles it kinda weirdly; this post describes how it works.
The BOM is the bytes: 0xFE 0xFF. That means:

Encoding First bytes in the stream
UTF-16, Little Endian FF FE
UTF-16, Big Endian FE FF
UTF-8 EF BB BF

You can use these to identify streams.
In Java, the BOM is left in the stream data. So, if you for example have a UTF-8 text file that starts with a BOM, and you read it into a String, your string starts with the BOM character. It doesn’t show up when you print it, but it still ‘counts’, in the sense that the .length() call on your string counts the BOM as 1 character, and a string that starts with a BOM is not equal to one that doesn’t start with it, even if they are otherwise the same. You probably want to filter it out!!
The only exception is the special encoding UTF-16. This encoding will, if it’s there, consume the BOM and use it to configure itself as Little Endian or Big Endian. If there is no BOM, it defaults to big endian. Note that this ‘consume the BOM’ behaviour does not apply to the encodings UTF-16LE and UTF-16BE. They read the BOM as normal.
NB: Esoteric note: The unicode character 0xFF 0xFE is intentionally defined as not valid, so that the BOM can be used unambiguously as indicating the endianness of a UTF-16 stream. However, in java, reading this special invalid character does not throw an exception. You can therefore read the byte data: FF FE 41 00, which is the string "A" encoded with a BOM as UTF-16 Little Endian using the encoding UTF-16BE. This produces garbage, but does not throw an exception.

Deep Dive: All about Exceptions

Overview

Exceptions are a mechanism for java code to signal extraordinary conditions (such as virtual machine errors like running out of memory, code bugs such as passing a negative number to a method that doesn’t accept negative numbers, alternative ways a method can exit, such as an attempt to read from a network socket aborting because the other end severed the connection, and even intentional control flow such as aborting a thread ASAP). They are similar to signals, if you are familiar with that concept.

Basic mechanism

Exceptions can be thrown anywhere in Java code.
When an exception is thrown, the currently executing line of code is aborted, and the JVM checks if the currently executing line of code is inside of a try block that has an associated catch block that catches the exception (this is known as the exception handler block). If an associated exception handler block exists, execution continues in that block, and the act of throwing the exception is effectively equivalent to break whatever loops you need to break and then go to the exception handler immediately.
If there is no exception handler, the entire method is aborted immediately (the method exits immediately), and the exception ‘bubbles up’ to the next method in the call stack (the method that called the method that just aborted). The process then repeats: Either the caller of the method that threw the exception has an exception handler for it, in which case execution continues there, or it does not, and this method, too, is aborted and now the method that called the method that called the method that threw the exception gets its chance.
This process continues all the way up to the JVM if no code on the call stack has an appropriate exception handler. At the top level, the thread itself will stop execution and echo the exception class name, associated message, stack trace, etc to System.err.

Throwing exceptions

To throw an exception, you use the throw statement, like so:

throw new IllegalArgumentException("foo should be positive");

When throwing an exception, you should usually add a convenient message explaining the situation in plain English, without ending the sentence in a full stop. You don’t have to create a new instance of an exception, but 99+% of the time you throw an exception, you create a new one like in the example. The aim of throwing an exception is usually to abort this method and signal to the caller that the method has finished executing with an unexpected result. In this sense, throw is a lot like return.

Catching exceptions

To catch an exception, you use the try/catch block, like so:

try {
    someMethodThatMightThrowAnException();
} catch (IllegalArgumentException e) {
    // handle the issue here.
}

You can add multiple catch blocks to a single try, each catching a different exception type. If an exception does occur in the body of the try block, the first exception handler whose catch line lists an exception type that matches the thrown exception ‘wins’ and code will resume execution there.
If you’re using Java 1.7 or newer (as you should be), you can also list multiple exception types for a single catch block. For example:

try {
    someMethodThatMightThrowAnException();
} catch (IllegalArgumentException | NullPointerException e) {
    // handle either of those exceptions here.
} catch (IOException e) {
    // And handle I/O issues here.
}

The exception type hierarchy

All things that you can throw or catch must either be the type java.lang.Throwable or a subtype of it. There are 4 major types you should be aware of:

java.lang.Error

A subclass of Throwable, Error is used to communicate serious, generally unrecoverable problems with the hardware or JVM itself: the system is out of memory, the jar file containing your classes is corrupt, or you’ve got a method that endlessly calls itself, and the stack has overflowed.
You should generally never throw any Errors yourself, with the possible exception of InternalError which you can throw if a JVM guarantee fails to hold (for example, the JVM specification guarantees that UTF-8 is an available character encoding. If it isn’t, you can throw InternalError).
You should subclass Error and throw it if an invariant is broken that can only be explained by a corrupt installation or failing hardware, such as when you read a data file via .getResourceAsStream that’s packed along with your code, and it’s not there. That’s as catastrophic as a missing class file, which are also handled with errors.
Error, and all subclasses of Error, are so-called ‘unchecked exceptions’ (see below). You rarely catch them.
Note that some errors may not have reliable stack traces, and some (most notable OutOfMemoryError) may indicate the JVM has become unstable.
The usual strategy to handle an Error is to just let the JVM crash. Most Errors thrown in java applications result in the application crashing entirely, and this is intentional, because the problem cannot be solved.

java.lang.Exception

The only other subclass of Throwable in the java.* namespace, this one indicates a more or less ‘expectable’ problem. Anything from unexpected user input to failing network connections to invalid SQL statements – they are generally handled by throwing some subclass of j.l.Exception. You should never throw Exception itself (always a more suitable subclass, and if no such suitable subclass exists, make your own), and, like j.l.Error, don’t catch it unless you are some sort of app server and you’re the final line of defense against a failing servlet, applet, event handler, etc.

Editor’s Note: This is a Java-centric concept; other JVM-based languages like Scala are far less restrictive in how exceptions are handled. Which approach is better for you depends very much on what you’re comfortable with. Using Java’s more restrictive approach is hardly ever a bad thing, but some other languages’ communities look down on it.

java.lang.RuntimeException

This is a subclass of java.lang.Exception and covers a mix of code bugs and unexpected and generally unfixable problems.
Examples (all mentioned exceptions in this list are subclasses of RuntimeException):

  • Whenever you write x.foo in java, and x is null, the line throws a NullPointerException.
  • Many methods will throw IllegalArgumentException if you for example pass a negative number to a method that expected only positive numbers.
  • If you try to refer to an array element that is outside of the bounds of that array, an ArrayIndexOutOfBoundsException is thrown. For example: int[] x = new int[10]; print(x[11]);

Like Error, RuntimeException and all subclasses of it are so-called ‘unchecked’.
The usual strategy to handle RuntimeExceptions depends on the exception; for exceptions that signify that you programmed a bug, the appropriate resolution is to just crash the application; the stack trace serves as debugging tool. There are also many runtime exceptions that occurr because of invalid user input (the user entered a word where they were supposed to enter a number) or some other problems for which you can write a resolution; you should catch those.

java.lang.Throwable

This gets us back to Throwable itself. You can subclass it, but you should only do that when you use exception as control flow, which is not something you should think about until you have outgrown the need for articles such as these. Until you know a lot better, don’t ever subclass Throwable itself.

Editor’s note, because it bears repeating: You can subclass Throwable, but you should only do that when you use exception as control flow, which is not something you should think about until you have outgrown the need for articles such as these. Until you know a lot better, don’t ever subclass Throwable itself.

Checked vs. Unchecked

As the previous paragraph explained, java.lang.Error and java.lang.RuntimeException, and all subclasses of those two, are known as the ‘unchecked exceptions’. The rest are checked. Those 2 classes are special and listed by name in the Java language specification. You can’t make more unchecked exceptions without subclassing Error, RuntimeException, or some other class that is itself a subclass of one of these two.
Checked exceptions MUST be handled in one of two ways:

  • You wrap the code that is declared to possibly throw one in a try/catch block which catches the checked exception in question, -OR-
  • The method with the code that is declared to possibly throw one, has a ‘throws’ clause that explicitly declares that it throws this exception.

In other words, if you’d like to write throw new IOException("Network connection lost"), then your method signature has to look like: public void myMethod() throws IOException. (IOException is a checked exception). Now code that calls myMethod inherits the requirements to handle it somehow: Either wrap the call to myMethod() in a try/catch block, or also put throws IOException in the method declaration line.
For unchecked exceptions, you have no such requirements. As such, you can throw them without adding that exception to the throws clause. In other words, all methods in all of Java, whether they are specified to do so or not, implicitly act like they have throws Error, RuntimeException tacked onto them. You can add throws NullPointerException to your method declaration if you like; this is meaningless (as NullPointerException is a subtype of RuntimeException and thus unchecked) but some programmers do this as a way to document their code.

Features of Throwable

Stack traces

Whenever any throwable object is created (with new, such as new FileNotFoundException()), the call stack (the chain of method invocations) is stored in the object you just made. A call stack looks something like this:

java.lang.FileNotFoundException: /foo/bar/baz (No such file or directory)
    at java.io.FileInputStream.open(Native Method)
    at java.io.FileInputStream.<init>(FileInputStream.java:138)
    at java.io.FileInputStream.</init><init>(FileInputStream.java:93)
    at Example.main(Example:8)

The actual exception was thrown (with a throw statement) inside the top listed method. That method was invoked by the constructor of FileInputStream, on line 138, that constructor was invoked by another FileInputStream constructor, and that constructor was invoked by our example code.
You can programatically browse the call stack using the .getStackTrace() method of Throwable. Default handlers of execution frameworks will print them to logs or the screen so you can review the problem.
ADVANCED TOPIC: If you wish to ‘save’ a stack trace (for example, because in a different thread at a later time, a problem will surface that was initially caused by the current invocation, but we don’t know yet if that will happen, so we wish to save the stack trace for now and possibly refer to it much later if failure does eventually happen), just create a new exception without actually throwing it (yet).

Cause

Often the right approach to handling an exception is to restate the problem in your own exception type, and with a more appropriate message. It helps to staple the original cause to this brand new exception. This is called the ’cause’ and can be attached via the initCause() method. Also, most exception types have a constructor that takes the cause if you have one. The cause will be logged/printed along with the new exception.
Make sure you add the appropriate cause if there is one; it really helps debug issues and figure out what went wrong.

Which exception should I throw?

Before we start, be aware that Java has been around for a long time and believes strongly in not changing already released code. As such, you can find many examples of methods throwing the ‘wrong’ kind of exception in java.* and popular libraries. That’s not ‘proof’ that their style is accepted convention or a good idea.

  • A method must list each checked exception that it might possibly throw, and calling code must then handle it somehow (either with a try/catch block, or by adding a throws clause to the method, passing on the burden of handling the issue to all callers of this method). This is very annoying if the method is ever invoked such that the checked exception couldn’t possibly occur (in the sense of: Having to write code that is literally pointless, therefore impossible to test, and aggravating the author of the code calling yours). Therefore, for all problems that can only occur for invalid input or due to conditions entirely under control of the caller, ALWAYS use unchecked exceptions, that is: subtypes of RuntimeException.

  • The type of the exception is the most important tool that you can give to callers of your code to handle the issue. Therefore, the more likely it is that you expect callers of your method to try/catch your exception, the more important it is to throw an appropriate type. Often, this means you should make your own type. Don’t shy away from this; making your own type (writing public class MyOwnException extends Exception {... in MyOwnException.java) is usually a good idea. When in doubt, make your own exceptions.

  • When different issues can come up but they are related in some way, especially if you expect that a caller might just want to handle all these issues in the exact same manner, you should build an exception hierarchy: One overarching type, with various subtypes of that type for more details. For example, java.io.FileNotFoundException is a subtype of java.io.IOException, which is a subtype of java.lang.Exception. If you call any method that may fail due to a file not existing, then FileNotFoundException may occur. You can catch this condition with catch (FileNotFoundException e) {...}, but if you don’t write such a catch block but you do catch all I/O problems with catch (IOException e) {...}, then all file not found problems will also be handled by this more general catch block. This is good API design: You give your caller the flexibility to handle issues as specifically, or as generally, as is appropriate for them. Make hierarchies of exception types if that is applicable.

  • The more likely a problem signaled via an exception can be handled in some appropriate way (other than ‘just crash the application or abort the web request’), the more you should lean towards a checked exception. The more likely a problem cannot be feasibly handled other than by aborting the entire operation, the more you should lean towards an unchecked exception. After all, a checked exception enforces the caller to deal with it, which is just pointless boilerplate if it is highly unlikely that the programmer that calls your method can do anything useful if the problem does occur.

Examples:

  • To signal that the caller provided invalid input arguments, use RuntimeException subtypes. A few common ones already exist:
    • NullPointerException is appropriate if an argument is null and that is not valid input. The message of the exception should be the name of the argument: if (foo == null) throw new NullPointerException("foo");
    • IndexOutOfBoundsException is appropriate when an index is passed that is out of the valid range.
    • IllegalStateException is appropriate if the object is in a state such that this method call isn’t valid at all. For example, a List class has been ‘locked’ and later a caller attempts to add another object to it.
    • UnsupportedOperationException is appropriate if the object is in a state such that the call isn’t valid, and the object has always been, and will always be, in this state. For example, the add method of a list that is designed to be immutable (all objects that are in it are added during its construction and the list will never change) should throw UnsupportedOperationException.
    • Various other types exist, but if you can’t find anything more appropriate, there’s always IllegalArgumentException. Because these are almost always bugs, you don’t need to make a specific subtype (under the rule: “the less likely it is that callers intend to catch it, the less need there is to make a subclass for it”).
  • To signal I/O errors, be it because of disk failure, network connection failure, or something more exotic, like failure to communicate with a device plugged into a serial port, throw java.io.IOException. You rarely throw it yourself though; you use APIs that talk to networks (such as HttpServletResponse, or Socket) and the API will throw the IOException for you.

A few examples where common libraries actually got it wrong:

  • NumberFormatException is thrown by Integer.parseInt(), Long.parseLong(), etcetera; these methods parse text input for a number and return it. The exception is thrown if the text passed to the method is not, in fact, a number. This error is both to be expected (users can make mistakes; if talking to another software product, it might be buggy or on some different version), and often handleable, and yet NumberFormatException is a runtime exception. I/O issues are in fact somewhat less recoverable and somewhat less expectable, and yet those are checked exceptions, which is inconsistent.

  • Various methods in the JDK take a string that represents the character encoding. For example, when turning an array of bytes into a string, you should pass in the name of the text encoding that it used. Certain encoding types such as UTF-8 are guaranteed by the Java virtual machine specification to always be available. Still, this code: new String(someByteArray, "UTF-8") is specced to throw UnsupportedEncodingException, which is a checked exception. That exception couldn’t possibly occur unless your JVM is corrupt (at which point you have far bigger problems). Fortunately this has been ‘solved’ in later versions by way of a method specifically designed for calling with known-valid charset names (new String(someByteArray, StandardCharsets.UTF_8)).

    • @SneakyThrows

    Project Lombok is a compiler plugin that adds the ability to ‘ignore’ the rule that you must either try/catch, or declare that you ‘throws’ a checked exception, by annotating your method with @SneakyThrows(IOException.class) for example. This is particularly useful for working around unwieldy APIs, such as the UnsupportedEncodingException example listed above.

    Editor’s note: I heartily endorse the use of Lombok, whose author contributed this article. I rarely work on any Java projects without it now.

    The ‘default’ exception handler

    Some IDEs and many examples use the following default implementation for an exception handler:

    try {
    // Code that throws some exception
} catch (IOException e) {
    e.printStackTrace();
}

    This is a really bad default! You miss the message and the type of exception, and, more importantly, the code will just continue running immediately following the catch block. Most likely another exception will occur soon (given that clearly something is wrong, and in addition a bunch of your code, namely everything in the try block from the place where the exception occurred and onwards) also did not run. If that exception is handled similarly, yet another exception will occur soon. You’ll be faced with a cavalcade of stack traces, most of which are complete red herrings. Your app is also now completely broken, as code continues to execute even though your method has failed and its state is most likely no longer valid.
    Don’t do this!
    The single best default exception handler looks like this:

    try {
    // Code that throws some exception
} catch (IOException e) {
    throw new RuntimeException("XXTODO: Uncaught", e);
}

    The exception text makes no bones about it: You’ve intentionally decided not to worry about this exception right now. Your method will also abort instead of continuing to run in an invalid state, and you can throw RuntimeException without having to add a throws clause.

    How to make your own

    If no exception exists that exactly describes the issue you are attempting to signal to your method’s callers, or it is unchecked when you want it to be checked or vice versa, you have to make your own. Fortunately, it is very easy to do this. For example, to make a new unchecked exception, you would write:

    public class MyException extends RuntimeException {
    public MyException(String msg) {
        super(msg);
    }
    public MyException(String msg, Throwable cause) {
        super(msg, cause);
    }
}

    Use a more descriptive name than MyException, of course. You can extend any existing class. If there’s no existing class that seems to make sense, extend Error, Exception, or RuntimeException depending on what kind of exception you want to create.

    The finally construct

    Sometimes you want code to execute to close resources or bring your object back to a valid state, and you want this code to run even if the main body of your method exits via an exception. The finally construct can help here. It’s part of the try syntax, and looks like:

    try {
    // code that might throw an exception
} catch (NumberFormatException e) {
    // Runs if NumberFormatException occurs in body.
} finally {
    // Runs always
}

    In the above example, the code in the finally block is always executed. If the main body (or the catch handler) uses the return statement, your finally code runs right before your method returns. If the try block just gets to the end naturally without an exception occurring, execution jumps to the finally block. If a NumberFormatException occurs, execution jumps to your NumberFormatException catch handler, and after that handler has completed (or if that handler itself throws an exception), your finally block is executed. If some other exception occurs in the try body, then code execution first jumps to your finally block, and once that finishes, the exception is actually ‘thrown’ (execution jumps to the caller and the exception is raised there).
    If your finally block also explicitly exits the method (either via a return statement, or a throw statement), it overrides the explicit method exit that caused your finally block to run. Because that gets very confusing, you should not use return or throw in a finally block.
    A try block needs to have either a catch block or a finally block, or it wouldn’t do anything. However, you don’t need both; just try {} finally {} is valid, for example.
    NB: If a method never exits, for example because it loops endlessly or it has deadlocked, then the finally block would never execute. Also, if the JVM is shut down, either normally (with System.exit(0) for example), or forcibly (killed by the OS, or someone trips over a power cable), your finally blocks don’t run either.

    Automatic Resource Management (‘ARM’)

    Closing resources is a common pattern: Many classes, such as for example java.io.FileInputStream, are specified to require the caller of the constructor to eventually call close() on the stream. Failure to do so means the VM will leak resources, and, eventually, it can’t open any more files and the only solution is to restart the VM. Just calling close() when you are done is not sufficient because exceptions exist: You need to use a finally block. Because this is such a common pattern, there’s an easier way to do it, which is legal Java starting with Java v1.7:

    try (FileInputStream in = new FileInputStream(path)) {
    // code goes here; it can access 'in'
}

    In the above snippet, no matter how execution exits the try block (via return statement, by running to the end of it, or via an exception), the resource will be closed.
    Another option is to use project lombok:

    @lombok.Cleanup FileInputStream in = new FileInputStream(path);
// code goes here

    Here, ‘in’ will be closed when it goes out of scope, for example at the end of the method, no matter how it exits. Lombok’s cleanup works from java v1.6 and up. For more information, see the lombok feature page on @Cleanup.

    How ‘checkedness’ is javac only.

    The concept of checked exceptions are twofold:

    • You cannot catch a checked exception unless at least one thing in the associated try block is declared to throw that exception.
    • If any line is declared to throw a checked exception, then this line must exist either inside of a try block with an associated catch handler for this checked exception, or, the method it is in must have declared this checked exception in its throws line.

    However, these 2 rules are applied only by the Java compiler (javac). The actual JVM treats all exceptions as unchecked; bytecode that throws a checked exception without declaring it does so / catching it, will run just fine, and, at the bytecode level, you can have a catch handler for a checked exception that can’t actually occur and the JVM will run it. This is why other languages that also compile to class files but which don’t have checked exceptions can work at all, and it’s also what makes lombok’s @SneakyThrows tick.

    First steps: Structure of your first source file

    This is what a class file ought to look like when you take your first steps writing java programs:

    public class YourAppNameHere {
  public static void main(String[] args) {
    new YourAppNameHere().go();
  }
  private String iAmAField;
  void go() {
    System.out.println("Your code goes here!");
  }
  void someOtherMethod() {
    // You can call this method from 'go' with: someOtherMethod()
    iAmAField = "Hello, World!";
  }
}

    In particular, copy/paste the main() method verbatim and don’t touch it. If you need access to the command line arguments, pass the args parameter directly to your go method, and update the go method to take String[] args as parameter.

    Editor’s note: Another way of handling arguments is to use JCommander, which itself encourages the kind of object design in this post.

    Why should you do it this way? The short answer is: Because main() has to be static, but the idea of static is advanced java. You will and should learn about it in time, but trying to explain it to you (so that you don’t make mistakes because you don’t understand what that means) as you take your first steps is a bit like teaching you about the fuel gauge during your first drive. It’s just not that interesting, and you don’t need to know about it to start taking your first steps. It’ll become much more obvious once you’ve done some exercises. This main() method will get you out of static immediately so that you can ignore its existence until the time comes to learn about what it’s for.

    How to access static resources in Java

    Your Java app may need access to data that you intend to ship directly with your code and which will not change without a version update. Examples might include a list of countries to show in a country dropdown, or icon images for a web or Swing application. The proper mechanism to access these files is:

    YourClass.class.getResource("file.txt");
// or
YourClass.class.getResourceAsStream("file.txt");

    This mechanism is built into java and will allow you to store your data files in the same place you store your class files (so, generally, on the classpath, and usually in a jar file). getResource returns a URL instance; many APIs that require a complete resource will accept these. If you wish to read the resource yourself, use getResourceAsStream, which creates an InputStream. (remember to always close it with a try/finally block or @Cleanup!)

    Example code

    Here’s some example code to read an image icon for Swing:

    package example1;
import javax.swing.*;
import java.net.URL;
public class GetResourceExample {
    public static JLabel createPrintIcon() {
        URL printIconUrl =
                GetResourceExample.class.getResource("icons/printer63.png");
        ImageIcon printIcon =
                new ImageIcon(printIconUrl, "Print document");
        return new JLabel(printIcon);
    }
    public static void main(String... args) {
        JFrame frame = new JFrame("GetResourceExample");
        frame.setDefaultCloseOperation(WindowConstants.EXIT_ON_CLOSE);
        frame.add(createPrintIcon());
        frame.pack();
        frame.setVisible(true);
    }
}

    In the above example, take the place GetResourceExample.class is located , and make sure a subdirectory called icons is also located there, which contains the printer63.png file. GetResourceExample.class is in a directory somewhere on your classpath, or in a jar. printer63.png should be in the same place.
    Here’s an example to read a list of all states in the US:

    package example2;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
public class PlainJavaStates {
    private static final List<String> states;
    static {
        List<String> out = new ArrayList<>();
        try (InputStream in =
                     PlainJavaStates.class.getResourceAsStream("states.txt")) {
            BufferedReader br =
                    new BufferedReader(new InputStreamReader(in, "UTF-8"));
            for (String line = br.readLine();
                 line != null;
                 line = br.readLine()) {
                if (line.isEmpty() || line.startsWith("#")) continue;
                out.add(line);
            }
        } catch (IOException e) {
            // RuntimeException is fine; the 'states'
            // file not existing is as likely as your States.class
            // file not existing; your app can crash
            // in the face of corrupt executables, which is what's happened
            // if states.txt isn't here.
            throw new RuntimeException("states.txt cannot be loaded.", e);
        }
        states = Collections.unmodifiableList(out);
    }
    public static List<String> getStates() {
        return states;
    }
    public static void main(String[] args) {
        System.out.printf("Number of states: %d%n ", getStates().size());
    }
}

    Or, if you use Guava, this becomes even simpler:

    package example2;
import com.google.common.base.Charsets;
import com.google.common.collect.ImmutableList;
import com.google.common.io.Resources;
import java.io.IOException;
public class GuavaStates {
    private static final ImmutableList<String> states;
    static {
        try {
            states = ImmutableList.copyOf(
                    Resources.readLines(
                            GuavaStates.class.getResource("states.txt"),
                            Charsets.UTF_8));
        } catch (IOException e) {
            // RuntimeException is fine; the 'states'
            // file not existing is as likely as your States.class
            // file not existing; your app can crash
            // in the face of corrupt executables, which is what's happened
            // if states.txt isn't here.
            throw new RuntimeException("states.txt cannot be loaded.", e);
        }
    }
    public static ImmutableList<String> getStates() {
        return states;
    }
    public static void main(String[] args) {
        System.out.printf("Number of states: %d%n ", getStates().size());
    }
}

    How do those paths work?

    Files are resolved relative to the location of the class file you use as context. In the ‘states’ example, states.txt is supposed to be located in the exact same place in the classpath as PlainJavaStates.class. You can also start this path with a slash to resolve relative to the root of the relevant classpath entry. For example, if your jar file looks like:

    /META-INF/MANIFEST.MF
/example2/PlainJavaStates.class
/example2/states.txt
/example2/foo/bar.txt
/icons/printer63.png

    Then you can get to these files in any of these ways:

    PlainJavaStates.class.getResource("states.txt");
PlainJavaStates.class.getResource("foo/bar.txt");
PlainJavaStates.class.getResource("/example2/states.txt");
PlainJavaStates.class.getResource("/example2/foo/bar.txt");
PlainJavaStates.class.getResource("/icons/printer63.png");

    Why not files?

    You could use FileInputStream to just read files from somewhere, but then you would need to know exactly where your files were. The JVM starts in the ‘working directory’, which may not be the same place your jars or classpath is, and at any rate, this does not work if you ever decide to modularize your application. It is possible to figure out where your jar file is and read from there, but that’s even more effort and easy to do wrong. Why bother?

    Why not the other ways to use .getResource / .getResourceAsStream

    You may have read about one of these alternate forms:

    getClass().getResource(...);
getClass().getResourceAsStream(...);

    This is not a good idea; getClass() is dynamic and resolves to the actual class of the instance. If your class is extended by some class that lives in another jar, that means all of a sudden your resources will no longer be found. You may also have read about this alternate form:

    ClassLoader.getSystemClassLoader().getResource(...);

    or:

    getClass().getClassLoader().getResource(...);
MyClass.class.getClassLoader().getResource(...);

    Do not use those forms either; the Java specification states that the class loader of certain classes can be null. That would mean a NullPointerException will occur. Also, the above forms are needlessly wordier. Just use this one form, there is no reason to use anything else:

    MyClass.class.getResource(...);
MyClass.class.getResourceAsStream(...);

    How do I get my resource files into my jars?

    Eclipse will automatically copy any non-java files that are located in the same place your source files are, to the binary directory, and thus they’ll be found by your app when running it inside your IDE. Eclipse will also package these files along with your compiled code if you tell Eclipse to make jars.
    With Maven or Gradle (or any build system that follows Maven’s source directory structure), put your resources in the src/main/resources directory, using the same path structure as in your src/main/java directory. Your build system will ensure your resources and your class files end up together in the same jar file, and that it’ll work out when you tell your build system to run your application.
    With Ant or Ant+Ivy, you have to explicitly add your resources. You can use the <copy> task for this, or just list them as an <include> in your <jar> task.
    Lastly, you could simply put the place that holds your resource files on the classpath when you start the JVM.

    If you’re interested in seeing a working example of the project layout, this site has a zipped maven project available.