Expected Exceptions from JUnit

Use JUnit’s expected exceptions sparingly, from the JOOQ blog, says not to use JUnit‘s “expected exceptions” feature. (Or, more accurately, to use it “sparingly.”) It’s good advice, but it’s also largely limited to JUnit (with one main exception).
What they’re addressing is the transition from this pattern:

@Test
public void testValueOfIntInvalid() {
  try {
    ubyte((UByte.MIN_VALUE) - 1);
    fail();
  }
  catch (NumberFormatException e) {}
}

To this pattern:

@Test(expected = NumberFormatException.class)
public void testValueOfShortInvalidCase1() {
  ubyte((short) ((UByte.MIN_VALUE) - 1));
}

They have four reasons that this doesn’t actually get you anything:

  • We’re not really gaining anything in terms of number of lines of code
  • We’ll have to refactor it back anyway
  • The single method call is not the unit
  • Annotations are always a bad choice for control flow structuring

Of these, “The single method call is not the unit” is by far the strongest point.

We’re not really gaining anything in terms of number of lines of code

It’s a matter of opinion, but we generally are gaining something in terms of the code. The refactored test has one line of actual code: ubyte((short) ((UByte.MIN_VALUE) - 1));. Everything around it is window dressing. The original test has the try/catch code, which adds a lot of noise – and discards the “error condition,” which is the actual metric for success (it fails the test if there’s no error.)
Is this a big deal? … No, it’s not. But it’s also not a point worth making.

We’ll have to refactor it back anyway

Here’s where JUnit itself messes things up. The authors actually have a really good point, if you’re locked into JUnit:

In the annotation-driven approach, all I can do is test for the exception type. I cannot make any assumptions about the exception message for instance, in case I do want to add further tests, later on. Consider this:

try {
    ubyte((UByte.MIN_VALUE) - 1);
    fail("Reason for failing");
}
catch (NumberFormatException e) {
    assertEquals("some message", e.getMessage());
    assertNull(e.getCause());
    ...
}

Ah, JUnit. JOOQ is making the assertion (see what I did there?) that they can’t examine Reason for failing or some message with the annotation.
They’re right… if you’re locked into JUnit. TestNG can, of course, look at the message and validate it. If the message doesn’t fit the regex, then the test fails, just as expected (and desired).

@Test(expectedExceptions={NumberFormatException.class},
      expectedExceptionsMessageRegExp="Reason for failing")
public void testThings() {
  ...

The single method call is not the unit

Here’s what JOOQ says:

The unit test was called testValueOfIntInvalid(). So, the semantic “unit” being tested is that of the UByte type’s valueOf() behaviour in the event of invalid input in general. Not for a single value, such as UByte.MIN_VALUE - 1.

They’re running yet again into a limitation of JUnit. The actual point they’re making is correct, but TestNG provides a @DataProvider mechanism, where you’d actually give the test a signature of testValueOfIntInvalid(int) and pass in a set of integers. That means you can test a range and corner cases, with a single appropriate test.
People have written a data-provider-like feature for JUnit (see junit-dataprovider) – which only highlights how JUnit is affecting the choices made by the JOOQ team.

Annotations are always a bad choice for control flow structuring

Hard to argue with this one: Java has excellent control flow, even if it’s verbose at times. However, the annotation is direct enough (with TestNG, at least) that you aren’t really violating control flow: you’re saying “this is a test, it has these execution rules, and this is how you measure success.” With JUnit, you don’t have the same control.

JUnit.next

According to JOOQ, the next generation of JUnit (junit-lambda) fixes things a little, by offering a lambda solution to exceptions:

Exception e = expectThrows(NumberFormatException.class, () ->
  ubyte((UByte.MIN_VALUE) - 1));
assertEquals("abc", e.getMessage());

That’s admirable, but they’re still behind TestNG by a few generations – and there’s still no data factory mechanism.

In Conclusion

Their closing statement is actually really good:

Annotations were abused for a lot of logic, mostly in the JavaEE and Spring environments, which were all too eager to move XML configuration back into Java code. This has gone the wrong way, and the example provided here clearly shows that there is almost always a better way to write out control flow logic explicitly both using object orientation or functional programming, than by using annotations.

It’s worth noting that the statement was actually driven by a lack of features in JUnit (when compared to TestNG) but the point remains.

Interesting Links, 20 Jan 2016

Sorry for the change of date format, folks. Some interesting links that came up include:

  • Don’t tell me what to make, tell me how to make it, by David Whiting, talks about using Java 8’s Supplier to help clean up construction of objects. Actually pretty useful stuff – it’s a common pattern to use no-argument constructors with mutable objects, and Supplier<t> might help us use immutable objects more often.
  • Arrays of Wisdom of the Ancients, by Alexei Shipilev, takes on the question of Collection.toArray(new T[0]) or Collection.toArray(new T[size]) in a long, well-written (and well-reasoned) post. It takes a long time to get to the point (Collection.toArray(new T[0]) wins, and has for “five years now”) but it also walks through the reasoning for the conclusion, and is fascinating reading.
  • The Java Security Manager: Why and How, by Nicolas Fränkel, shows a use of reflection to change a reference to a Character to a reference to a String and talks a bit about how to set up your security manager to be useful in addressing potentially malicious code.
  • Testing persistence with Arquillian on TomEE describes an end-to-end test for persistence with Arquillian. This is different from an embedded unit test for persistence in that Arquillian can actually start up an application server (TomEE in this case) and run the test there – which serves as a better integration test than a custom persistence configuration for the testing scope.
  • Neo4J published Graph Databases for Beginners: ACID vs. BASE Explained, explaining the two consistency models for transactions typically found in storage systems. ACID is typically trumpeted as a strength for relational databases, and stands for “Atomic, Consistent, Isolated, and Durable” transactions, and BASE stands for “Basic Availability, Soft-State, and Eventually Consistent” transactions. BASE is a strength of NoSQL datastores.
  • Jihed Amine Maaref posted Setup a docker environment for Liferay with MySQL inside a VirtualBox VM, which walks through a Docker configuration. It’s not so much that Liferay (an open source portal) and MySQL are fascinating (although they’re useful, I guess, especially Liferay), but the walkthrough is a good read, including explanations of how and why along the way.
  • ByteBuddy 1.0 has been released. ByteBuddy is a code generation library for Java – which can be used to build mocks, interceptors, or just plain classes out of thin air. Cheesy name and logo, but it seems worthwhile.

If you have interesting links that you’ve run across, please send them my way! It’s worth noting that Petri Kainulainen noted some of the same links as being interesting – check out Petri’s stuff for more focus on testing.

Using SQL's "IN" in JDBC

In SQL, the IN operator is used to restrict columns to one of a set of values. Using IN in JDBC, though, is sometimes problematic because of the way different databases handle prepared statements.
With JDBC, prepared statements use ? to serve as markers for values in a SQL statement. Thus, you might see:

PreparedStatement ps=connection.prepareStatement("SELECT * FROM FOO WHERE BAR = ?");

This serves to help prevent SQL injection attacks; assigning a value of "'' or 1==1'" would check that actual value against BAR rather than return all rows.
Exploits of a Mom
The parameter number of each ? is an index, starting from 1, so to set the value against which to compare BAR we might see:

ps.setParameter(1, "BAZ");

The IN operator in SQL allows selection from a set of values. Thus, we might see:

SELECT * FROM FOO WHERE BAR IN ('BAZ', 'QUUX', 'CORGE')

If BAR is one of BAZ, QUUX, or CORGE, then the row matches the query and will be returned.
It would make sense to see a PreparedStatement declared as:

PreparedStatement ps=connection.prepareStatement("SELECT * FROM FOO WHERE BAR IN (?)");

However, this doesn’t work. (It gives you only one element to use for the IN selector.) You have two choices: you can write SQL against your specific database, or you can generate custom SQL for the query.
Let’s look at the most general form (the SQL customization) first, since that’s going to be supported best. We are assuming a simple table, created with:

create table if not exists information (id identity primary key, info integer)

Note that we’re presuming H2 at this stage. In PostgreSQL, an equivalent statement would be create table if not exists information (id serial primary key, info integer). With MySQL… oh, who cares, nobody should use MySQL.

Given an array of data to use for the IN clause of Integer[] data = {3, 4, 6, 11};, we can construct a viable (and general) SQL query like this:

StringJoiner joiner = new StringJoiner(
  ",",
  "select * from information where info in (",
  ")");
for (Object ignored : data) {
  joiner.add("?");
}
String query = joiner.toString();
try (PreparedStatement ps = conn.prepareStatement(query)) {
  for (int c = 0; c < data.length; c++) {
    ps.setObject(c + 1, data[c]);
  }
  try (ResultSet rs = ps.executeQuery()) {
    showResults(rs);
  }
}

This code isn’t complicated, although it looks like a lot for what it does. It first creates a SQL statement with a placeholder for every element in the data array, then sets each placeholder to the corresponding value in data, and then runs the query. The SQL has to be regenerated for every case where data has a different length. (We could potentially reuse the statement if data always has the same length.)
You can also generalize this, depending on your database. It requires custom SQL, though, and the code to use the SQL differs by database as well.

H2

For H2, we can use the ARRAY_CONTAINS function. Our SQL statement will look like "select * from information where array_contains(?, info)", and the code to use this statement looks like this:

try (PreparedStatement ps = conn.prepareStatement(query)) {
  ps.setObject(1,data);
  try (ResultSet rs = ps.executeQuery()) {
    showResults(rs);
  }
}

H2 can use setObject() and use that as the input for the ARRAY_CONTAINS function; this way, we have one placeholder and we don’t have to generate custom SQL for every different size of the input array.

PostgreSQL

In PostgreSQL, we can use the ANY function. Our SQL looks like "select * from information where info = ANY(?)". Our code to use the statement:

try (PreparedStatement ps = conn.prepareStatement(query)) {
  Array array=conn.createArrayOf("INTEGER", data);
  ps.setArray(1, array);
  try (ResultSet rs = ps.executeQuery()) {
    showResults(rs);
  }
}

MySQL

Nobody should use MySQL.

This is offered somewhat tongue-in-cheek, for a few reasons: one is that I genuinely dislike MySQL, another is that the SQL technique offered here probably isn’t needed very often in the first place (so doing an exhaustive solution is overkill), and the third is ironic: this site is hosted in WordPress, and uses MySQL as the backend database. Irony ftw, right?

Conclusion

We’ve shown a few possibilities for restricting the results of queries, using a general-purpose restriction (IN, with custom SQL generated for every query, still protected from SQL injection attacks), and custom SQL queries for both H2 and PostgreSQL. These are definitely not the only possibilities; feel free to show how you’d do it, or discuss potential optimizations. Some sample code for these examples can be found at https://github.com/jottinger/jdbc_contains – note that some of the code might require modification for each database, and the project doesn’t describe how to create the PostgreSQL database. (The project was written largely to prove the mechanisms described here, and wasn’t meant to be a one-size-fits-all solution.)

Some interesting links for and from ##java, Jan 15 2016

Some links relevant to Java, as of Jan 15 2016:

  • A git-flow cheatsheet. git-flow is a set of command-line scripts that implement Vincent Driessen‘s branching model for git projects, such as commands to complete feature requests, and the like.
  • Yahoo has released a giant dataset for machine learning.
  • jQuery has reached 3.0 beta. jQuery may not be the framework du jour for implementing a Javascript front end for Java web applications (Angular.js is, based on anecdata), but it’s still relevant. Lots of little changes, but one big one is that IE 6-8 support is being dropped. Die, Internet Exploder. Die.
  • Azul Systems has published “Four Reasons why Java is still #1“. In short, they are:
    • Practicality
    • Backwards Compatibility
    • Scalability/Performance/Reliability
    • Freshness, largely centering around all of the changes introduced in Java 8.
  • The Hibernate blog published “JPA test case templates“, documenting a set of templates (go figure) to help replicate bugs in tests. Templates exist for the native Hibernate APIs and, obviously, JPA as well. This would be useful for all kinds of test cases – if you have a problem with JPA or Hibernate, consider using one of these to show the problem, even if you’re not using Hibernate – you should be able to change the JPA provider fairly easily if you’re avoiding Hibernate.
  • A set of Java EE 7 sample projects built by the “WASdev” — whom I don’t know, but it looks like they’re the people who gave us Liberty – an Java EE 7 application server built with technology from IBM. Seems like useful stuff, though, and if it’s written properly it’s portable to other Java EE 7 containers.

Using karma on ##java

Ah, karma:

me> ~karma
javabot> All living beings have actions (karma) as their own,
         their inheritance, their congenital cause, their
         kinsman, their refuge. It is karma that differentiates
         beings into low and high states.
me> ... what?

Karma is normally a way of referring to the kind of person you are. It’s the sum of your actions in Hinduism and Buddhism.
IRC isn’t Hinduism or Buddhism, but karma in ##java is a way of showing what people think of you; positive karma (and lots of it) means you do a lot of good things, and negative karma (and lots of that) show that people don’t care for you much.
In ##java, there are a few operations related to karma. You can:

  • Add or subtract karma.
  • Query for karma.
  • Complain about there being no third point.

Adding or subtracting karma

The grammar for changing karma is pretty simple:

~ TARGET ( ++ | -- ) COMMENT

Thus, you might see:

someone> ~ eclipse -- for never working the way I want it to

This means that the user someone is decrementing Eclipse’ karma, which – given Eclipse’ karma of -673 as of this writing – happens a lot.
The text that follows the operation (the ++ or --) isn’t significant – it’s just a way to allow people to say why they’re changing karma, without the burden of an extra line.
There are a few problems with the syntax: note that changing C++’ karma is rather hard, because it looks like an increment of C’s karma…

Note that you can’t positively affect your own karma! Doing so is lame, and attempting to do so lowers your karma… let others sing your praises.

Querying for karma

The grammar for querying karma is also simple:

~ karma TARGET

This will tell you what TARGET‘s present karma score is, much as you’d expect. You can tell if nobody’s ever tried to change something’s karma, based on whether the score is 0 or “neutral” – neutral karma means there are no karma changes for the subject of the query.

Conclusion

Javabot is, of course, open source; if you have suggestions or improvements, feel free to contribute. File issues! Fork the project and submit pull requests; most of the code in Javabot is fairly simple, and we all benefit when things in the bot get smarter.

Eclipse Neon will require Java 8

Excellent news from the Eclipse Foundation: the next version of Eclipse, called “Neon,” will require the Java 8 runtime. The announcement in email is underwhelming, but the actual project plan is a little more explicit and informative:

In general, the 4.6 release of the Eclipse Project is developed on Java SE 8 VMs. As such, the Eclipse SDK as a whole is targeted at all modern, desktop Java VMs. Most functionality is available for Java SE 8 level development everywhere, and extended development capabilities are made available on the VMs that support them.

Frank Vogella, in Eclipse Neon (Eclipse 4.6) will require a Java 8 runtime, says this:

Several Eclipse projects like m2e and Jetty have already moved to Java 8. This moves allows us in the Eclipse platform to use the improved Java 8 API to modernize and optimize our code base and will hopefully make the Eclipse project even more interesting for potential Eclipse contributors.
After all, who wants to work in his unpaid time with an outdated Java version?

From Apache: Log4J 1 end-of-lifed; update to Log4J 2 recommended

On 5 Aug 2015, the Apache Foundation announced that Log4J 1, the popular logging framework, has been end-of-lifed, meaning that no future releases are expected and maintenance and support have been discontinued.
Migration to Log4J 2 is enabled through the use of a Log4J 1.X bridge, as described in a short FAQ about the EOL of Log4J 1:

Q: Is there a way to quickly migrate from Log4j 1.x to Log4j 2.x?
A: Yes, you can use the Log4j 1.x bridge http://logging.apache.org/log4j/2.x/log4j-1.2-api/index.html You must use this bridge without Log4j 1 jars in your application’s classpath. The bridge forwards all requests to Log4j 1 to Log4j 2.

Which logging framework do you prefer, and why?

A set of testing tools

Petri Kainulainen posted “12 Tools That I Use for Writing Unit and Integration Tests,” which does a pretty good job of describing a set of testing tools and approaches, including solutions in the following categories:

  • Running Tests
  • Mock and Stub frameworks
  • Writing Assertions
  • Testing Data Access
  • Testing Spring

It’s not comprehensive (nor does it claim to be), with no mention of things like TestNG, Arquillian, Liquibase or Flyway, or testing CDI in general (see Arquillian), but that doesn’t mean it’s not a good start on an interesting idea. What tools would you suggest for testing?

Link: Jenkins over HTTPS with JNLP slaves

A user in ##java recently ran into a problem where he needed to connect Jenkins slaves to a master, using SSL – and it didn’t work, thanks to security. He helpfully wrote up how using correct policy files got it working.
Jenkins is a continuous integration server – basically an application that runs your builds when changes occur in your source tree. Slaves allow the build to take place on different hardware configurations, as well as providing horizontal scalability to your builds.
Good stuff, and thanks!

PriorityQueue and mutable item behavior

From today, in Java: a user asked if changing the value of an item in a PriorityQueue would still result in the queue being correctly ordered. (He explained that he was doing this to prioritize based on counters, which makes sense. It’s worth noting that this blog entry is not about solving his problem, but about the behavior of PriorityQueue.)
Getting items out of a PriorityQueue requires peek() and/or poll(). Let’s take a look at some simple usage; first, let’s look at an object that represents something with a priority:

class Container implements Comparable {
  int x;
  public Container(int x) { this.x = x; }
  public int getX() { return x; }
  public void setX(int x) { this.x = x; }
  @Override
  public String toString() { return "Container{" + "x=" + x + '}'; }
  @Override
  public int compareTo(Container o) {
    return Integer.compare(getX(), o.getX());
  }
}

Let’s use this from another class. First, a utility method that builds a Queue, returning one element from the Queue for us to mutate later:

private static Container buildQueue(Queue q) {
  q.clear();
  q.add(new Container(4));
  q.add(new Container(2));
  q.add(new Container(6));
  Container c = new Container(5);
  q.add(c);
  return c;
}

As stated, we need to use peek() or poll() to get results out in correct order. If we use normal iteration, our order starts off correctly, but fails immediately. Here’s the code:

buildQueue(q);
for(Container c1:q) {
  System.out.println(c1);
}

And the output is:

Container{x=2}
Container{x=4}
Container{x=6}
Container{x=5}

The first element is in the right place – we expect 2 as the highest priority – but the others are in insertion order, not actual priority order.
If, however, we use poll(), we get different results. The code:

buildQueue(q);
while((c=q.poll())!=null) {
  System.out.println(c);
}

And the output:

Container{x=2}
Container{x=4}
Container{x=5}
Container{x=6}

What poll() does is simple: it removes the head of the Queue – the item with priority – and then it sifts the queue, with the result that the Queue‘s order is adjusted. Depending on the nature of the queue, this may put it in proper order all the way through the queue, but this is not required — the actual result is that the Queue is more ordered, with the new head of the Queue (the result of the next poll() or peek()) being set properly.

If you want to see the actual code for this process, grepcode has it: see PriorityQueue.poll(). Internally, “sift” is the actual term used.

Mutating the Elements in the Queue

What happens if we mutate the priority, then?
As it turns out, the behavior of the Queue doesn’t change much – except that the results of the last sifting of the Queue are maintained. This is actually a big deal. poll() doesn’t sift the Queue before returning the head of the queue. Therefore, if this is our code:

Container container = buildQueue(q);
System.out.println("1: "+q);
container.setX(3);
System.out.println("2: "+q);
while((c=q.poll())!=null) {
  System.out.println(c);
}
1: [Container{x=2}, Container{x=4}, Container{x=6}, Container{x=5}]
2: [Container{x=2}, Container{x=4}, Container{x=6}, Container{x=3}]
Container{x=2}
Container{x=3}
Container{x=4}
Container{x=6}

Note what happens: we modify the last element in the Queue to be 3, and the Queue maintains its order, but pulling the results out via poll() gives us the results as we expect. But if we change the priority of the last element such that it should be first… let’s see what that looks like.

container = buildQueue(q);
System.out.println("1: "+q);
container.setX(1);
System.out.println("2: "+q);
while((c=q.poll())!=null) {
  System.out.println(c);
}

Our output is:

1: [Container{x=2}, Container{x=4}, Container{x=6}, Container{x=5}]
2: [Container{x=2}, Container{x=4}, Container{x=6}, Container{x=1}]
Container{x=2}
Container{x=1}
Container{x=4}
Container{x=6}

Summary

As stated, PriorityQueue sifts its values after pulling the top element from the Queue. It’d be unwise to expect a Queue to work properly with mutable items in any case – much as using mutable keys in a Map would be unwise – but as you can see, mutating the priorities can work but can also fail if you happen to modify the priorities such that the element at the head of the list should change.