How to execute a group of tasks

Someone on the channel today asked about how to run a series of tasks, with a terminal condition of sorts. Basically, they wanted to run a number of tasks, and know when they were done.
The question was originally built around having a List of results (List<integer>, maybe?), counting the number of results until the number of results matched the number of tasks.
The channel suggested using an ExecutorService, particularly the shutdown() and awaitTermination() methods, with another suggestion being invokeAll() – which seemed particularly apt because the person asking didn’t want to have to build a new ExecutorService over and over again (which the shutdown() call would necessitate, although the cost of doing this shouldn’t be very high.)
Here’s an example with Java 8, of using the invokeAll() to execute a series of tasks; the task is made-up (it basically rolls three six-sided dice to determine a histogram of results, as a callback to Dungeons and Dragons). It has one dependency, on Apache’s commons-lang3.

package org.javachannel.examples.executors;
import org.apache.commons.lang3.StringUtils;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.locks.LockSupport;
import java.util.stream.IntStream;
class Worker implements Callable {
    int roll() {
        LockSupport.parkNanos((long) (Math.random() * 10000000));
        return (int) (Math.random() * 6) + 1;
    }
    @Override
    public Integer call() throws Exception {
        return roll() + roll() + roll();
    }
}
public class Main {
    public static void main(String[] args) throws InterruptedException {
        List workerList = new ArrayList<>();
        IntStream.range(0, 100000).forEach(i -> workerList.add(new Worker()));
        ExecutorService service = Executors.newCachedThreadPool();
        int[] count = new int[19];
        service.invokeAll(workerList).stream().map(f -> {
            try {
                return f.get();
            } catch (Throwable e) {
                return 0;
            }
        }).forEach(m -> count[m]++);
        IntStream.rangeClosed(3, 18).forEach(r -> {
            System.out.printf("%02d %s%n", r, StringUtils.repeat("*", count[r]/100));
        });
    }
}

This code creates a list of tasks, populates it, and then submits all of them in one call (service.invokeAll(workerList)), then uses the Stream API to populate the array of results. It looks a little more complicated than it should because of the lambda to pull the int out of the Future</integer><integer>, which is what return value for call() becomes when it’s submitted to the Executor.
It’s not a perfect example; surely others can come up with better examples. Feel free to submit them!

Returning void type from a method call

Someone on ##java asked about how to return a void value. This is a terrible question, but let’s look at it anyway, just because.
Here’s some example code they asked about, translated:

void foo(boolean bar) {
  if(bar) {
    return baz();
  }
}
void baz() {
  // do stuff here
}

The intent was to call baz() as a replacement for foo()‘s execution; the one with the question didn’t include code after the if() block, but further questions indicated that the remaining part of foo() was to be avoided.
There’s a lot of monstrosity here. The worst is “return baz();“, which won’t compile, and shouldn’t compile; void‘s whole purpose is to avoid being put on the call stack, so “return void” makes no sense.
One way to rewrite this code is obvious:

void foo(boolean bar) {
  if(bar) {
    baz();
    return;
  }
  // extra code goes here,
  // not to be executed if bar is true
}
void baz() {
  // do stuff here
}

This calls baz() and terminates the method execution immediately after. You could also do something else that’s obvious:

void foo(boolean bar) {
  if(bar) {
    baz();
  } else {
    // extra code goes here
  }
}

This has the advantage of a single termination point for the method. From “Code Complete:”

17.1 return

Minimize the number of returns in each routine. It’s harder to understand a routine if, reading it at the bottom, you’re unaware of the possibility that it returned somewhere above.
Use a return when it enhances readability. In certain routines, once you know the answer, you want to return it to the calling routine immediately. If the routine is defined in such a way that it doesn’t require any cleanup, not returning immediately means that you have to write more code.
(Content copied shamelessly from an excellent answer on StackOverflow.)

Initializing arrays of arrays: avoid fill()

A long, long time ago, on a blog far, far away, this horror was posted by someone who really should have known better (and who has since removed this section from the original post):

BTW: Ricky Clarkson pointed out that I missed a golden opportunity to use Arrays.fill() to fill in the “multiple dimensions.” Even here, you get to loop some, but here’s some grin-worthy code:

int[][][] arr = new int[10][10][10];
Arrays.fill(arr[0][0], 5);
Arrays.fill(arr[0], arr[0][0]);
Arrays.fill(arr, arr[0]);

Hey, it works. It’s retarded, but works. 🙂

This… doesn’t work. It actually copies references, so altering any of the deep references changes all of the deep references. You can see this in action with this code:

import java.util.Arrays;
public class ArrayFillBad {
    public static void main(String[] args) {
        int[][] arr = new int[2][4];
        Arrays.fill(arr[0], 5);
        Arrays.fill(arr, arr[0]);
        display(arr);
        arr[1][2] = 1024;
        display(arr);
    }
    private static void display(int[][] arr) {
        for (int[] a1 : arr) {
            String sep = "";
            for (int a2 : a1) {
                System.out.print(sep + a2);
                sep = ", ";
            }
            System.out.println();
        }
        System.out.println();
    }
}

When run, this offers this output:

5, 5, 5, 5
5, 5, 5, 5
5, 5, 1024, 5
5, 5, 1024, 5

See the two new values? Only one is supposed to have been changed.
So what’s happening here? We’re not actually copying array contents, we’re copying array references – so we have multiple copies of one array, copied across each deep reference.
Chances are very good that this is not what you wanted. We can do better than that (where “better” means “actually works,” as opposed to “causes you to fail code reviews.”)
Here’s working code; it’s “uglier” but ugly trumps broken:

public class ArrayFillGood {
    public static void main(String[] args) {
        int[][] arr = new int[2][4];
        for (int[] anArr : arr) {
            Arrays.fill(anArr, 8);
        }
        display(arr);
        arr[1][2] = 6;
        display(arr);
    }
private static void display(int[][] arr) {
        for (int[] a1 : arr) {
            String sep = "";
            for (int a2 : a1) {
                System.out.print(sep + a2);
                sep = ", ";
            }
            System.out.println();
        }
        System.out.println();
    }
}

How to access static resources in Java

Your Java app may need access to data that you intend to ship directly with your code and which will not change without a version update. Examples might include a list of countries to show in a country dropdown, or icon images for a web or Swing application. The proper mechanism to access these files is:

YourClass.class.getResource("file.txt");
// or
YourClass.class.getResourceAsStream("file.txt");

This mechanism is built into java and will allow you to store your data files in the same place you store your class files (so, generally, on the classpath, and usually in a jar file). getResource returns a URL instance; many APIs that require a complete resource will accept these. If you wish to read the resource yourself, use getResourceAsStream, which creates an InputStream. (remember to always close it with a try/finally block or @Cleanup!)

Example code

Here’s some example code to read an image icon for Swing:

package example1;
import javax.swing.*;
import java.net.URL;
public class GetResourceExample {
    public static JLabel createPrintIcon() {
        URL printIconUrl =
                GetResourceExample.class.getResource("icons/printer63.png");
        ImageIcon printIcon =
                new ImageIcon(printIconUrl, "Print document");
        return new JLabel(printIcon);
    }
    public static void main(String... args) {
        JFrame frame = new JFrame("GetResourceExample");
        frame.setDefaultCloseOperation(WindowConstants.EXIT_ON_CLOSE);
        frame.add(createPrintIcon());
        frame.pack();
        frame.setVisible(true);
    }
}

In the above example, take the place GetResourceExample.class is located , and make sure a subdirectory called icons is also located there, which contains the printer63.png file. GetResourceExample.class is in a directory somewhere on your classpath, or in a jar. printer63.png should be in the same place.
Here’s an example to read a list of all states in the US:

package example2;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
public class PlainJavaStates {
    private static final List<String> states;
    static {
        List<String> out = new ArrayList<>();
        try (InputStream in =
                     PlainJavaStates.class.getResourceAsStream("states.txt")) {
            BufferedReader br =
                    new BufferedReader(new InputStreamReader(in, "UTF-8"));
            for (String line = br.readLine();
                 line != null;
                 line = br.readLine()) {
                if (line.isEmpty() || line.startsWith("#")) continue;
                out.add(line);
            }
        } catch (IOException e) {
            // RuntimeException is fine; the 'states'
            // file not existing is as likely as your States.class
            // file not existing; your app can crash
            // in the face of corrupt executables, which is what's happened
            // if states.txt isn't here.
            throw new RuntimeException("states.txt cannot be loaded.", e);
        }
        states = Collections.unmodifiableList(out);
    }
    public static List<String> getStates() {
        return states;
    }
    public static void main(String[] args) {
        System.out.printf("Number of states: %d%n ", getStates().size());
    }
}

Or, if you use Guava, this becomes even simpler:

package example2;
import com.google.common.base.Charsets;
import com.google.common.collect.ImmutableList;
import com.google.common.io.Resources;
import java.io.IOException;
public class GuavaStates {
    private static final ImmutableList<String> states;
    static {
        try {
            states = ImmutableList.copyOf(
                    Resources.readLines(
                            GuavaStates.class.getResource("states.txt"),
                            Charsets.UTF_8));
        } catch (IOException e) {
            // RuntimeException is fine; the 'states'
            // file not existing is as likely as your States.class
            // file not existing; your app can crash
            // in the face of corrupt executables, which is what's happened
            // if states.txt isn't here.
            throw new RuntimeException("states.txt cannot be loaded.", e);
        }
    }
    public static ImmutableList<String> getStates() {
        return states;
    }
    public static void main(String[] args) {
        System.out.printf("Number of states: %d%n ", getStates().size());
    }
}

How do those paths work?

Files are resolved relative to the location of the class file you use as context. In the ‘states’ example, states.txt is supposed to be located in the exact same place in the classpath as PlainJavaStates.class. You can also start this path with a slash to resolve relative to the root of the relevant classpath entry. For example, if your jar file looks like:

/META-INF/MANIFEST.MF
/example2/PlainJavaStates.class
/example2/states.txt
/example2/foo/bar.txt
/icons/printer63.png

Then you can get to these files in any of these ways:

PlainJavaStates.class.getResource("states.txt");
PlainJavaStates.class.getResource("foo/bar.txt");
PlainJavaStates.class.getResource("/example2/states.txt");
PlainJavaStates.class.getResource("/example2/foo/bar.txt");
PlainJavaStates.class.getResource("/icons/printer63.png");

Why not files?

You could use FileInputStream to just read files from somewhere, but then you would need to know exactly where your files were. The JVM starts in the ‘working directory’, which may not be the same place your jars or classpath is, and at any rate, this does not work if you ever decide to modularize your application. It is possible to figure out where your jar file is and read from there, but that’s even more effort and easy to do wrong. Why bother?

Why not the other ways to use .getResource / .getResourceAsStream

You may have read about one of these alternate forms:

getClass().getResource(...);
getClass().getResourceAsStream(...);

This is not a good idea; getClass() is dynamic and resolves to the actual class of the instance. If your class is extended by some class that lives in another jar, that means all of a sudden your resources will no longer be found. You may also have read about this alternate form:

ClassLoader.getSystemClassLoader().getResource(...);

or:

getClass().getClassLoader().getResource(...);
MyClass.class.getClassLoader().getResource(...);

Do not use those forms either; the Java specification states that the class loader of certain classes can be null. That would mean a NullPointerException will occur. Also, the above forms are needlessly wordier. Just use this one form, there is no reason to use anything else:

MyClass.class.getResource(...);
MyClass.class.getResourceAsStream(...);

How do I get my resource files into my jars?

Eclipse will automatically copy any non-java files that are located in the same place your source files are, to the binary directory, and thus they’ll be found by your app when running it inside your IDE. Eclipse will also package these files along with your compiled code if you tell Eclipse to make jars.
With Maven or Gradle (or any build system that follows Maven’s source directory structure), put your resources in the src/main/resources directory, using the same path structure as in your src/main/java directory. Your build system will ensure your resources and your class files end up together in the same jar file, and that it’ll work out when you tell your build system to run your application.
With Ant or Ant+Ivy, you have to explicitly add your resources. You can use the <copy> task for this, or just list them as an <include> in your <jar> task.
Lastly, you could simply put the place that holds your resource files on the classpath when you start the JVM.

If you’re interested in seeing a working example of the project layout, this site has a zipped maven project available.

Combining two functions into one

Every now and then, you may want to apply one function to only some of the elements in your stream.
This is not so much an example of manipulating streams but rather of combining two functions into one with a predicated dictating which function to apply. In concrete terms, we will create a stream of the integers in the range 0 through 10 (non-inclusive), multiply the even numbers by two and add one, then finally display the result.
The first function, or IntUnaryOperator – as it takes and returns one int – (slightly contrived use of andThen and compose) – looks like this:

IntUnaryOperator f = IntUnaryOperator.identity()
   .andThen(i -> i + 1)
   .compose(i -> 2 * i);

The second function is just the identity:

IntUnaryOperator g = IntUnaryOperator.identity();

The predicate, or just a function taking an int and returning a boolean, was to do something for the even integers:

IntPredicate p = (i -> i % 2 == 0);

One way of combining the two functions is close to the lines of “if the value is this, do that to the value and return it, else do something else to the value and return it:”

IntUnaryOperator m = (i -> (p.test(i)
   ? f.applyAsInt(i)
   : g.applyAsInt(i)));

So now we can map the integers by

IntStream.range(0, 10)
   .map(m)
   .forEach(System.out::println);

However, instead of creating a function taking and returning an integer we could create a function taking an integer and returning a function (which then take and return an integer):

IntFunction<IntUnaryOperator> mm = (i -> p.test(i) ? f : g);

This looks a bit nicer, but applying it to the stream gets messy:

IntStream.range(0, 10)
    .map(i -> mm.apply(i).applyAsInt(i))
    .forEach(System.out::println);

It might make be more sensible to just skip the intermediate step and write:

IntStream.range(0, 10)
    .map(i -> (p.test(i) ? f : g)
    .applyAsInt(i))
 .forEach(System.out::println);

.. which is also a way to get around the problem of prepending or appending the functions f or g:

IntStream.range(0, 10)
    .map(i -> (p.test(i) ? f : g.andThen(j -> j + 100))
       .applyAsInt(i))
 .forEach(System.out::println);

Caveat lector: There are probably much cleaner and more sensible ways to do this. Feel free to suggest alternatives!

Collapsing multiple ints into a long

Every now and then, you may want to collapse multiple values into a long, to use as a key for a Map, for example.
The following code is an example of collapsing three ints into one long. Obvious care should be taken here: three ints don’t actually fit into one long, so the excess bits are just discarded. If we do this, we can afford to use the lowest 20 bits, plus the sign bit, leaving us with a data range of 1,048,576 to -1,048,575.
For each int we take the lower 20 bits:

  input & 0xfffff

…and the sign bit, which we move to position 21:

  (input & Integer.MIN_VALUE) >>> 11

and then combine the two:

  (input & 0xfffff | (input & Integer.MIN_VALUE) >>> 11)

Now, we repeat the process for all our inputs and put them in our long, shifting two of them by 21 and 42 respectively so they don’t overlap:

  long l = 0;
  l |= (long)(input1 & 0xfffff | (input1 & Integer.MIN_VALUE) >>> 11);
  l |= (long)(input2 & 0xfffff | (input2 & Integer.MIN_VALUE) >>> 11) << 21;
  l |= (long)(input3 & 0xfffff | (input3 & Integer.MIN_VALUE) >>> 11) << 42;

If you were to combined a higher amount of things with less bits, this loop version will probably be more concise, courtesy of Maldivia:

  public static long pack(int i1, int i2, int i3) {
    int[] data = new int[] { i1, i2, i3 };
    long l = 0;
    for (int i = 0; i < 3; i++) {
       l |= (long)(data[i] & 0xfffff | (data[i] & Integer.MIN_VALUE) >>> 11) << (i * 21);
    }
    return l;
  }

And to unpack the long back into pieces:

  public static int[] unpack(long l) {
    int[] res = new int[3];
    for (int i = 0; i < 3; i++) {
       res[i] = (int) (l & 0xfffff);
       if ((l & 0x100000) != 0) {
         res[i] |= 0xfff00000;
       }
       l >>>= 21;
    }
    return res;
  }