ZeroTurnaround's Developer Productivity Report

ZeroTurnaround has issued their latest developer productivity survey – if enough people take it, they’re giving money to Devoxx4Kids.
It’s an interesting survey, done very well – probably the best execution of such a survey as I’ve ever seen (and I’ve seen a lot). Check it out, let’s get Devoxx4Kids that donation (and/or join ourselves).

Interesting Links, 1 Feb 2016

Editor’s note before we get to the links: I’ve been trying to keep the links’ length down to a manageable four or five, so the frequency’s been higher than I might otherwise have desired (roughly every two or three days). I’m going to try a longer list of interesting links, and move the frequency down – we’ll see if that’s useful and palatable. The problem is that ##java (and the Java community overall) just has a lot of interesting, relevant content that’s worth keeping! (Keep it up, guys. “Too much interesting stuff” is a good problem to have.)

  • WildFly 10 has been released. This is the latest version of the open source (community-supported) JBoss Application Server; it’s fully Java EE 7, and requires Java 8. Very cool stuff, congratulations to the WildFly team.
  • Neo4J has released milestone 1 of their object/graph mapping library’s second version. (Read it with me: “Neo4J has released OGM 2.0 m1.” Much simpler that way.) It sounds promising, especially since they seem to have straightened out the connection process to Neo4J instances such that both embedded and remote instances have similar capabilities.
  • Implied Readability” uses readability as a term to address transitive dependencies in Java 9, more or less, and shows how a module can export visibility of a dependency to other modules. As stated: Java 9, not Java 8, so it’s a new feature – but it looks a little like how OSGi exports visibility rules. It might be really relevant as time goes on and Java 9 gets closer. (It’s based off of information in “Programming with Modularity and Project Jigsaw. A Tutorial Using the Latest Early Access Build” published on InfoQ, so there may be more interesting stuff in that article.)
  • How we accidentally doubled our JDBC traffic with Hibernate” discusses an obvious issue (doubling JDBC traffic!), found when Hibernate logging was set to WARN – because Hibernate then re-executes every query in order to show the warnings associated with the query in question. The warnings can be useful, to be sure, but be wary!
  • PNG encoding from Java’s ImageIO can be slow, according to one ##java op. He said that he used ObjectPlanet‘s PngEncoder and got much better performance.
  • As an update to the process by which one can examine request headers (mentioned in “Interesting Links, 22 Jan 2016“) a ##java user mentioned RequestBin, which allows you to build a URL and issue requests against it, to examine the actual traffic data.
  • Moving to a Plugin-Free Web – by Oracle – says it point blank: “Oracle plans to deprecate the Java browser plugin in JDK 9. This technology will be removed from the Oracle JDK and JRE in a future Java SE release.” This makes sense – browsers are ignoring the java plugin (and should). If you’re still doing applets, stop. Oracle has spoken, and the technology is going away. (Now if we could get Oracle to get rid of Vector somehow…)
  • ZeroTurnaround – who gave the world JRebel – posted Java 8 Streams cheat sheet, which offers a one-page example of a lot of useful, relevant information around streams for handy, quick reference. I looked for a DZone Refcard on streams for a point of comparison, but they didn’t have one that I saw on first scan – which is surprising, since the Refcardz are actually done really well, in general.

Interesting Links, 27 Jan 2016

  • Mutation Testing: Watching the Watchmen talks about the need for mutation testing, in addition to addressing the desire for unit testing. His point is: unit testing is good – do it. But it’s often not enough, and he promises to go more into mutation testing as an enhancement in future posts.
  • JOOQ has published If Java Were Designed Today: The Synchronizable Interface, discussing what could have been designed as opposed to what we got with the synchronized keyword – a Synchronizable interface, that could have been used for specific situations a lot like we use Iterable in more recent Java coding.
  • The Double Colon Operator in Java 8 discusses, of all things, the double colon operator in Java 8 (surprise!), which is a shorthand way to refer to a method reference. You find this used very often when working with streams, because the shorthand is so useful.
  • RedPen 1.4 has been released (back in October 2015, but I didn’t know about the project until it showed up on homebrew.) RedPen is a proofreading tool for technical documents, NOT for programming, and as such won’t be very useful for most readers of JavaChannel.org, but it’s still pretty interesting. For some reason, it complained a lot about the content of this post…

Interesting Links, 25 Jan 2016

  • From ##java itself, Hot deploy to Jboss from Intellij using deployment scanner documents a problem someone had while trying to deploy from IDEA into JBoss, and discusses how to get it working. Of particular note: “The Intellij deployment options for ‘upload external changes’ (turn it on!) and ‘preserve file timestamps’ (turn it off!) have given me particular grief in the past; if you find that Intellij doesn’t seem to be picking up changes and uploading them, look at the former.”
  • J2ObjC 1.0 has been released. From the project website: “J2ObjC is an open-source command-line tool from Google that translates Java source code to Objective-C for the iOS (iPhone/iPad) platform. This tool enables Java source to be part of an iOS application’s build, as no editing of the generated files is necessary.” The UI for each platform is still written in code for that platform, but this might make porting code to iOS easier for Java coders.
  • A user on ##java posted a link to a free app he’d written for iOS devices: Trident. It’s an app that interacts with Gitlab (and Github, in early preview form), with a primary focus on discussing issues, pull requests, and files from a mobile device. This isn’t an endorsement (your author doesn’t have an iOS device) but it might be worth watching.
  • We’ve seen a lot of JavaFX questions lately – mostly they’re from a small cluster of users (therefore: a few JavaFX users, each with a lot of questions, and yes, this is anecdata.) Gluon maintains SceneBuilder, which, well, builds scenes for JavaFX, and just had a new release, version 8.1.0 of Scene Builder.
  • Feature Flags are a way to release features to specific users. Turns out there’s a hub dedicated to the method, including a reference to multiple libraries that enable limited releases in wide-area deployments. (The hub itself looks like it’s provided by the authors of Launch Darkly, but that’s okay.) It’s not a new idea, I think, but it’s written up pretty well here and it’s nice to have a hub for the capability.

Interesting Links, 22 January 2016

  • Every so often you need to see exactly what headers your HTTP-based application is working with. One way to look at the HTTP headers is to use an HTTP proxy: possibilities in this space include Fiddler, Charles Proxy, and Paros Proxy (with Paros’ website being the Sourceforge project site, which isn’t very confidence-inspiring). There are certainly others (feel free to send them to me, if you’ve used one you like!). You can also use Wireshark if you need to examine all network traffic, but for HTTP Wireshark’s a bit of overkill.
  • Picking correct Java executors for background tasks, by ZeroTurnaround (makers of JRebel), goes into how (and why) you would choose specific Executors. Good reading, spoken as one who more or less uses Executors.newFixedThreadPool(int) by default.
  • Use JUnit’s expected exceptions sparingly, from the JOOQ blog, talks about a feature I love from TestNG: the ability to expect exceptions from a test, as opposed to a try/catch block that uses fail() to fail the test if an exception isn’t thrown. The JOOQ people argue against using expected exceptions, and if you’re using JUnit, they’re right. TestNG, of course, has a better alternative as usual.
  • Auditing with JPA, Hibernate, and Spring Data JPA discusses writing audit data with basic JPA (which looks like a pain, relying on JPA lifecycle events) and both Hibernate Envers and Spring Data JPA. Spring Data JPA and JPA have difficulty auditing removals; Spring Data JPA has integration with Spring Security; Hibernate Envers is probably the least intrusive, but doesn’t have Spring Security integration.

Expected Exceptions from JUnit

Use JUnit’s expected exceptions sparingly, from the JOOQ blog, says not to use JUnit‘s “expected exceptions” feature. (Or, more accurately, to use it “sparingly.”) It’s good advice, but it’s also largely limited to JUnit (with one main exception).
What they’re addressing is the transition from this pattern:

@Test
public void testValueOfIntInvalid() {
  try {
    ubyte((UByte.MIN_VALUE) - 1);
    fail();
  }
  catch (NumberFormatException e) {}
}

To this pattern:

@Test(expected = NumberFormatException.class)
public void testValueOfShortInvalidCase1() {
  ubyte((short) ((UByte.MIN_VALUE) - 1));
}

They have four reasons that this doesn’t actually get you anything:

  • We’re not really gaining anything in terms of number of lines of code
  • We’ll have to refactor it back anyway
  • The single method call is not the unit
  • Annotations are always a bad choice for control flow structuring

Of these, “The single method call is not the unit” is by far the strongest point.

We’re not really gaining anything in terms of number of lines of code

It’s a matter of opinion, but we generally are gaining something in terms of the code. The refactored test has one line of actual code: ubyte((short) ((UByte.MIN_VALUE) - 1));. Everything around it is window dressing. The original test has the try/catch code, which adds a lot of noise – and discards the “error condition,” which is the actual metric for success (it fails the test if there’s no error.)
Is this a big deal? … No, it’s not. But it’s also not a point worth making.

We’ll have to refactor it back anyway

Here’s where JUnit itself messes things up. The authors actually have a really good point, if you’re locked into JUnit:

In the annotation-driven approach, all I can do is test for the exception type. I cannot make any assumptions about the exception message for instance, in case I do want to add further tests, later on. Consider this:

try {
    ubyte((UByte.MIN_VALUE) - 1);
    fail("Reason for failing");
}
catch (NumberFormatException e) {
    assertEquals("some message", e.getMessage());
    assertNull(e.getCause());
    ...
}

Ah, JUnit. JOOQ is making the assertion (see what I did there?) that they can’t examine Reason for failing or some message with the annotation.
They’re right… if you’re locked into JUnit. TestNG can, of course, look at the message and validate it. If the message doesn’t fit the regex, then the test fails, just as expected (and desired).

@Test(expectedExceptions={NumberFormatException.class},
      expectedExceptionsMessageRegExp="Reason for failing")
public void testThings() {
  ...

The single method call is not the unit

Here’s what JOOQ says:

The unit test was called testValueOfIntInvalid(). So, the semantic “unit” being tested is that of the UByte type’s valueOf() behaviour in the event of invalid input in general. Not for a single value, such as UByte.MIN_VALUE - 1.

They’re running yet again into a limitation of JUnit. The actual point they’re making is correct, but TestNG provides a @DataProvider mechanism, where you’d actually give the test a signature of testValueOfIntInvalid(int) and pass in a set of integers. That means you can test a range and corner cases, with a single appropriate test.
People have written a data-provider-like feature for JUnit (see junit-dataprovider) – which only highlights how JUnit is affecting the choices made by the JOOQ team.

Annotations are always a bad choice for control flow structuring

Hard to argue with this one: Java has excellent control flow, even if it’s verbose at times. However, the annotation is direct enough (with TestNG, at least) that you aren’t really violating control flow: you’re saying “this is a test, it has these execution rules, and this is how you measure success.” With JUnit, you don’t have the same control.

JUnit.next

According to JOOQ, the next generation of JUnit (junit-lambda) fixes things a little, by offering a lambda solution to exceptions:

Exception e = expectThrows(NumberFormatException.class, () ->
  ubyte((UByte.MIN_VALUE) - 1));
assertEquals("abc", e.getMessage());

That’s admirable, but they’re still behind TestNG by a few generations – and there’s still no data factory mechanism.

In Conclusion

Their closing statement is actually really good:

Annotations were abused for a lot of logic, mostly in the JavaEE and Spring environments, which were all too eager to move XML configuration back into Java code. This has gone the wrong way, and the example provided here clearly shows that there is almost always a better way to write out control flow logic explicitly both using object orientation or functional programming, than by using annotations.

It’s worth noting that the statement was actually driven by a lack of features in JUnit (when compared to TestNG) but the point remains.

Interesting Links, 20 Jan 2016

Sorry for the change of date format, folks. Some interesting links that came up include:

  • Don’t tell me what to make, tell me how to make it, by David Whiting, talks about using Java 8’s Supplier to help clean up construction of objects. Actually pretty useful stuff – it’s a common pattern to use no-argument constructors with mutable objects, and Supplier<t> might help us use immutable objects more often.
  • Arrays of Wisdom of the Ancients, by Alexei Shipilev, takes on the question of Collection.toArray(new T[0]) or Collection.toArray(new T[size]) in a long, well-written (and well-reasoned) post. It takes a long time to get to the point (Collection.toArray(new T[0]) wins, and has for “five years now”) but it also walks through the reasoning for the conclusion, and is fascinating reading.
  • The Java Security Manager: Why and How, by Nicolas Fränkel, shows a use of reflection to change a reference to a Character to a reference to a String and talks a bit about how to set up your security manager to be useful in addressing potentially malicious code.
  • Testing persistence with Arquillian on TomEE describes an end-to-end test for persistence with Arquillian. This is different from an embedded unit test for persistence in that Arquillian can actually start up an application server (TomEE in this case) and run the test there – which serves as a better integration test than a custom persistence configuration for the testing scope.
  • Neo4J published Graph Databases for Beginners: ACID vs. BASE Explained, explaining the two consistency models for transactions typically found in storage systems. ACID is typically trumpeted as a strength for relational databases, and stands for “Atomic, Consistent, Isolated, and Durable” transactions, and BASE stands for “Basic Availability, Soft-State, and Eventually Consistent” transactions. BASE is a strength of NoSQL datastores.
  • Jihed Amine Maaref posted Setup a docker environment for Liferay with MySQL inside a VirtualBox VM, which walks through a Docker configuration. It’s not so much that Liferay (an open source portal) and MySQL are fascinating (although they’re useful, I guess, especially Liferay), but the walkthrough is a good read, including explanations of how and why along the way.
  • ByteBuddy 1.0 has been released. ByteBuddy is a code generation library for Java – which can be used to build mocks, interceptors, or just plain classes out of thin air. Cheesy name and logo, but it seems worthwhile.

If you have interesting links that you’ve run across, please send them my way! It’s worth noting that Petri Kainulainen noted some of the same links as being interesting – check out Petri’s stuff for more focus on testing.

Using SQL's "IN" in JDBC

In SQL, the IN operator is used to restrict columns to one of a set of values. Using IN in JDBC, though, is sometimes problematic because of the way different databases handle prepared statements.
With JDBC, prepared statements use ? to serve as markers for values in a SQL statement. Thus, you might see:

PreparedStatement ps=connection.prepareStatement("SELECT * FROM FOO WHERE BAR = ?");

This serves to help prevent SQL injection attacks; assigning a value of "'' or 1==1'" would check that actual value against BAR rather than return all rows.
Exploits of a Mom
The parameter number of each ? is an index, starting from 1, so to set the value against which to compare BAR we might see:

ps.setParameter(1, "BAZ");

The IN operator in SQL allows selection from a set of values. Thus, we might see:

SELECT * FROM FOO WHERE BAR IN ('BAZ', 'QUUX', 'CORGE')

If BAR is one of BAZ, QUUX, or CORGE, then the row matches the query and will be returned.
It would make sense to see a PreparedStatement declared as:

PreparedStatement ps=connection.prepareStatement("SELECT * FROM FOO WHERE BAR IN (?)");

However, this doesn’t work. (It gives you only one element to use for the IN selector.) You have two choices: you can write SQL against your specific database, or you can generate custom SQL for the query.
Let’s look at the most general form (the SQL customization) first, since that’s going to be supported best. We are assuming a simple table, created with:

create table if not exists information (id identity primary key, info integer)

Note that we’re presuming H2 at this stage. In PostgreSQL, an equivalent statement would be create table if not exists information (id serial primary key, info integer). With MySQL… oh, who cares, nobody should use MySQL.

Given an array of data to use for the IN clause of Integer[] data = {3, 4, 6, 11};, we can construct a viable (and general) SQL query like this:

StringJoiner joiner = new StringJoiner(
  ",",
  "select * from information where info in (",
  ")");
for (Object ignored : data) {
  joiner.add("?");
}
String query = joiner.toString();
try (PreparedStatement ps = conn.prepareStatement(query)) {
  for (int c = 0; c < data.length; c++) {
    ps.setObject(c + 1, data[c]);
  }
  try (ResultSet rs = ps.executeQuery()) {
    showResults(rs);
  }
}

This code isn’t complicated, although it looks like a lot for what it does. It first creates a SQL statement with a placeholder for every element in the data array, then sets each placeholder to the corresponding value in data, and then runs the query. The SQL has to be regenerated for every case where data has a different length. (We could potentially reuse the statement if data always has the same length.)
You can also generalize this, depending on your database. It requires custom SQL, though, and the code to use the SQL differs by database as well.

H2

For H2, we can use the ARRAY_CONTAINS function. Our SQL statement will look like "select * from information where array_contains(?, info)", and the code to use this statement looks like this:

try (PreparedStatement ps = conn.prepareStatement(query)) {
  ps.setObject(1,data);
  try (ResultSet rs = ps.executeQuery()) {
    showResults(rs);
  }
}

H2 can use setObject() and use that as the input for the ARRAY_CONTAINS function; this way, we have one placeholder and we don’t have to generate custom SQL for every different size of the input array.

PostgreSQL

In PostgreSQL, we can use the ANY function. Our SQL looks like "select * from information where info = ANY(?)". Our code to use the statement:

try (PreparedStatement ps = conn.prepareStatement(query)) {
  Array array=conn.createArrayOf("INTEGER", data);
  ps.setArray(1, array);
  try (ResultSet rs = ps.executeQuery()) {
    showResults(rs);
  }
}

MySQL

Nobody should use MySQL.

This is offered somewhat tongue-in-cheek, for a few reasons: one is that I genuinely dislike MySQL, another is that the SQL technique offered here probably isn’t needed very often in the first place (so doing an exhaustive solution is overkill), and the third is ironic: this site is hosted in WordPress, and uses MySQL as the backend database. Irony ftw, right?

Conclusion

We’ve shown a few possibilities for restricting the results of queries, using a general-purpose restriction (IN, with custom SQL generated for every query, still protected from SQL injection attacks), and custom SQL queries for both H2 and PostgreSQL. These are definitely not the only possibilities; feel free to show how you’d do it, or discuss potential optimizations. Some sample code for these examples can be found at https://github.com/jottinger/jdbc_contains – note that some of the code might require modification for each database, and the project doesn’t describe how to create the PostgreSQL database. (The project was written largely to prove the mechanisms described here, and wasn’t meant to be a one-size-fits-all solution.)

Some interesting links for and from ##java, Jan 15 2016

Some links relevant to Java, as of Jan 15 2016:

  • A git-flow cheatsheet. git-flow is a set of command-line scripts that implement Vincent Driessen‘s branching model for git projects, such as commands to complete feature requests, and the like.
  • Yahoo has released a giant dataset for machine learning.
  • jQuery has reached 3.0 beta. jQuery may not be the framework du jour for implementing a Javascript front end for Java web applications (Angular.js is, based on anecdata), but it’s still relevant. Lots of little changes, but one big one is that IE 6-8 support is being dropped. Die, Internet Exploder. Die.
  • Azul Systems has published “Four Reasons why Java is still #1“. In short, they are:
    • Practicality
    • Backwards Compatibility
    • Scalability/Performance/Reliability
    • Freshness, largely centering around all of the changes introduced in Java 8.
  • The Hibernate blog published “JPA test case templates“, documenting a set of templates (go figure) to help replicate bugs in tests. Templates exist for the native Hibernate APIs and, obviously, JPA as well. This would be useful for all kinds of test cases – if you have a problem with JPA or Hibernate, consider using one of these to show the problem, even if you’re not using Hibernate – you should be able to change the JPA provider fairly easily if you’re avoiding Hibernate.
  • A set of Java EE 7 sample projects built by the “WASdev” — whom I don’t know, but it looks like they’re the people who gave us Liberty – an Java EE 7 application server built with technology from IBM. Seems like useful stuff, though, and if it’s written properly it’s portable to other Java EE 7 containers.

Using karma on ##java

Ah, karma:

me> ~karma
javabot> All living beings have actions (karma) as their own,
         their inheritance, their congenital cause, their
         kinsman, their refuge. It is karma that differentiates
         beings into low and high states.
me> ... what?

Karma is normally a way of referring to the kind of person you are. It’s the sum of your actions in Hinduism and Buddhism.
IRC isn’t Hinduism or Buddhism, but karma in ##java is a way of showing what people think of you; positive karma (and lots of it) means you do a lot of good things, and negative karma (and lots of that) show that people don’t care for you much.
In ##java, there are a few operations related to karma. You can:

  • Add or subtract karma.
  • Query for karma.
  • Complain about there being no third point.

Adding or subtracting karma

The grammar for changing karma is pretty simple:

~ TARGET ( ++ | -- ) COMMENT

Thus, you might see:

someone> ~ eclipse -- for never working the way I want it to

This means that the user someone is decrementing Eclipse’ karma, which – given Eclipse’ karma of -673 as of this writing – happens a lot.
The text that follows the operation (the ++ or --) isn’t significant – it’s just a way to allow people to say why they’re changing karma, without the burden of an extra line.
There are a few problems with the syntax: note that changing C++’ karma is rather hard, because it looks like an increment of C’s karma…

Note that you can’t positively affect your own karma! Doing so is lame, and attempting to do so lowers your karma… let others sing your praises.

Querying for karma

The grammar for querying karma is also simple:

~ karma TARGET

This will tell you what TARGET‘s present karma score is, much as you’d expect. You can tell if nobody’s ever tried to change something’s karma, based on whether the score is 0 or “neutral” – neutral karma means there are no karma changes for the subject of the query.

Conclusion

Javabot is, of course, open source; if you have suggestions or improvements, feel free to contribute. File issues! Fork the project and submit pull requests; most of the code in Javabot is fairly simple, and we all benefit when things in the bot get smarter.