Hibernate Ignores Entity Listeners

I switched a codebase from EclipseLink as JPA provider to Hibernate and found out that with Hibernate you can not use inheritance in event listener classes. The JPA specs do not go into much detail on this topic so JPA providers are bound to handle this different from one another.

I separated my entity listeners from my entities even though it is possible to add methods annotated with e.g. @PostPersist directly in the entity class. The entity listeners themselves were less than trivial but only slightly so: I created a generic base class for all entity listeners and specialized subclasses for each entity.

public class EntityListenerBase<T> {
	@PostPersist
	public void postPersisted(T object) {
		SomeEventBus.sendEvent(createPostPersistEvent(object));
	}
	
	protected abstract Event createPostPersistEvent(T object);
}

public class ThingEntityListener extends EntityListenerBase<Thing> {
	@Override
	protected Event createPostPersistEvent(Thing thing) {
		return new ThingAdded(thing);
	}
}

Adding the entity listeners to the entity is straight-forward: Just add the @EntityListeners annotation to the entity and point it to the listener.

@Entity
@EntityListeners(ThingEntityListener.class)
public class ThingEntity implements Thing {
	…
}

EclipseLink had no trouble at all using event listeners like these. The problems started after I switched the JPA provider to Hibernate: at some point I realized that apparently the app was not generating any events anymore in response to persistence events. It was like the entity listeners weren’t even there!

After a longer debugging session into how Hibernate manages its event listeners I discovered that my previous approach for the event listeners was no longer viable, for two reasons.

The first reason is that Hibernate insists on the annotated event methods being declared directly in the entity listener class. Methods inherited from super classes are not scanned for event annotations! This is different from the behaviour when methods in the entity class itself are annotated; in that case, super classes annotated with @MappedSuperclass would be used to locate event listener methods as well. However, this is not the case for entity listener classes.

Upon moving the postPersisted method down to the subclass I stumbled upon the second reason: for every event annotation Hibernate only allows a single method in the entity listener class to be annotated with it. Now, because the original postPersisted method has a generic parameter and the subclass has a fixed type parameter, the compiler creates two versions of the method, one with Object as the parameter type, and one with the actual type I want, Thing. One of the methods is marked as synthetic but both get the annotation, and Hibernate doesn’t like that. Not. One. Bit.

So, the solution is simple, even if it means a little bit of code duplication over all the entity listeners: just remove the type hierarchy from the event listeners.

public class ThingEntityListener {
	@PostPersist
	public void postPersist(Thing thing) {
		SomeEventBus.sendEvent(new ThingAdded(thing));
	}
}

One could argue that this is actually even better; less class hierarchy, and all relevant listener methods are right there in the class, increasing visibility and reducing complexity.

WebMvcConfigurationSupport Mangles ISO8601 Timestamps in Spring Boot

I was working on a test in my Spring-Boot app and noticed that the timestamps in JSON output were sometimes formatted incorrectly. Luckily I was able to identify the issue and fix it.

During an integration test I noticed that an expected JSON output of "2019-08-26T08:22:21Z" was actually 1566807741.000000000 even though in other tests in my application the automatic conversion from java.time.Instant to an ISO8601-formatted string was working just fine. As is the default for Spring-Boot, Jackson’s ObjectMapper was used for the conversion. So what could make Jackson work correctly in one case but not another?

First I thought that I did something wrong with configuring so I did a lot of googling in order to find out what I was doing wrong. I added jackson-datatype-jsr310 to my project’s dependencies, I disabled Jackson’s “write dates as timestamps” serializer feature in four different ways but the error would persist. I dug into how the ObjectMapper used by Spring was created; maybe the feature was not disabled correctly? No, it was disabled just fine but Jackson still would not format the Instant properly.

During my research on how to create your own ObjectMapper and have Spring use it, I stumbled upon this question. It had nothing to do with my immediate problem but it mentioned that when using a @Configuration class extending WebMvcConfigurationSupport in conjunction with @EnableWebMvc Spring-Boot’s auto-configuration would be disabled. A disabled auto-configuration could explain an ObjectMapper that can not format Instants properly!

As it turns out, I had a SwaggerConfiguration class that extended WebMvcConfigurationSupport. Following the hints in the StackOverflow post I replaced the superclass by WebMvcConfigurerAdapter only to discover that it was deprecated. Fixing that was easy, though, as the interface WebMvcConfigurer could (and should!) be used instead. And this made my integration test work…

…although at this point I am still not 100% sure why. I am not using @EnableWebMvc anywhere (just @SpringBootApplication) so I do not yet know why the SwaggerConfiguration class extending WebMvcConfigurationSupport would disable Spring-Boot’s auto-configuration mechanisms. Spring’s usually excellent documentation is rather unclear as to what happens if you only do one of those things.

I also don’t know why the Jackson mapper in the working test had its “write dates as timestamps” serializer feature disabled but the one in the other test had not. From my understanding it must have been disabled so that Jackson’s jackson-datatype-jsr310 module was used—it was already a part of my dependencies without me knowing so explicitely adding it to my project’s build file did not actually change anything.

And even though my mental model of some things involving Spring has been improved by getting this to work I will not investigate this any further. My integration tests work and that shall be good enough for me, for today.

Help! My Java Locale is Wrong in JDK11!

We discovered this issue when a user from the Netherlands reported that, upon upgrading to Java 11, his application now reported that the week started on Sunday. This is incorrect for the Netherlands, but the real issue was the change of behavior.

With JDK11, Java now loads locales differently. https://www.oracle.com/technetwork/java/javase/documentation/java11locales-5069639.html#providers details the loading behavior for JDK11, and how it supersedes previous versions. Notably, CLDR, or classloaded entries, now go before COMPAT, or things included with the JDK. This can lead to incorrect behavior with complex classpaths.

To supersede this setting, the java.locale.providers system setting is used. It’s a comma-separated list of values, where the possible values are the providers in the blog post above. The default value is CLDR,COMPAT,HOST,JRE where JRE is also COMPAT. The value that I would have expected is COMPAT,HOST,CLDR, where it uses the built-in locales first (from JDK9!), then the host locales, then the classloader locales. Setting -Djava.locale.providers=COMPAT will cause the JDK11 locale loader to act like the JDK9 locale loader.

Netty is not a web framework

Recently, I have seen more and more web frameworks adopt Netty as their backend (the part that handles the actual HTTP requests). Examples are Spring WebFlux (the spring reactive web stack), Vert.x and Ktor.

What is Netty?

Netty calls itself “an asynchronous event-driven network application framework”. Let’s go over this word for word.

  • Netty is used for network applications: Instead of the “normal” java.io APIs (Socket, InputStream, …) you can use the Netty framework to build network applications.
  • Netty is a framework, not a library. It is fairly opinionated for things like connection management and expects you to use its pipeline model (more on that later) to build your application. This is in contrast to a library like XNIO that aims to simplify and complement the existing Java nio APIs.
  • Netty is asynchronous by design. When processing data from a network connection, you do not do so by calling blocking read methods like you would with InputStream, you instead get chunks of data asynchronously (in a “callback” you register). This is one of the main selling points – it allows you to serve many connections with few threads, which can help performance, and makes for a very convenient programming model.
  • Netty is event-driven. An idiomatic Netty application has a pipeline of “transformations” it applies on incoming and outgoing data. For example, you might first get incoming chunks of bytes, buffer them until a packet boundary and then send the full packet further downstream. The next handler in the pipeline would then parse the packet, and the handler after that finally runs the code that actually handles the packet.

In addition, Netty offers many implementations of common protocols out-of-the-box that you can basically mix-and-match as you wish. This goes from simple handlers such as DelimiterBasedFrameDecoder used for splitting incoming data on delimiters such as newline, to advanced handlers for full protocols like TLS or HTTP 1.1/2.

All this makes Netty one of the best (if not the best) general-purpose network application frameworks of any language out there. Nowhere else do you get network application development that is so convenient and powerful. I have used Netty in many projects over the years, and have contributed to it, and it remains my number one recommendation for anyone writing code that deals with custom protocols.

So why is Netty not appropriate for the Web?

Netty and HTTP

Netty includes handlers for parsing HTTP messages, both versions 1.1 and HTTP 2. It even supports SPDY if you wish to go back a few years! Netty got HTTP 2 support very early while other frameworks were still catching up, which may have contributed to its newfound popularity as a web framework.

I will not dispute Netty HTTP support can be very convenient. Netty’s design allows you to do things using its built-in HTTP handlers that would be impossible or difficult with traditional web frameworks. Is your HTTP/2 connection somehow TLS-less and wrapped in a custom encryption scheme? Does the remote server use upgrade mechanisms that are anything but to spec? Netty can implement your protocol anyway!

However, there are two disadvantages that Netty has that you will not get around easily.

Netty is not optimized as a HTTP server. Netty’s programming model brings with it certain static overhead – per-connection memory use is pretty high compared to raw Java NIO and its buffer management is convenient but not free. “Real” web frameworks can do more here: They can minimize parsing overhead, tune their buffer management to use patterns commonly seen for web servers and drop the niceties that Netty offers internally in favor of performance. When it’s not actually you doing the protocol parsing, the convenience of Netty’s internals don’t matter anymore.

Netty is not binary compatible, even across minor versions. This came as a surprise to me a few years back, but it makes sense: Netty offers very detailed views of its internals and evolution of the API would be hard without breaking binary compatibility at some point. Now this is not necessarily a problem for most Netty users that use Netty directly from their application. However, if you try to use two libraries on the same classpath at the same time that both use Netty (maybe Vert.x and the Java mongo driver) you could be in for a rude awakening in the future. You will either need to recompile or even patch one of the two. An alternative is relocation but that is inconvenient for reasons I will not discuss here. The incompatibility may become an even larger problem in the future when Netty 5 releases (if it ever does, it’s been in development for years).

So what’s the alternative?

Use Undertow. Undertow is a relatively new web server and the backend in Wildfly. Undertow was designed from the ground up as an asynchronous server, same as Netty. However, it is actually designed as a HTTP server and uses NIO directly instead of more expensive wrappers. This makes Undertow faster than Netty HTTP and appropriate for asynchronous applications. On top of that it is binary compatible with previous versions.

When you are building a “vanilla” HTTP server or client – one that does not mess with the HTTP protocol in non-standard ways – or you develop a web framework, use Undertow. There are few advantages Netty offers for the purely HTTP use case, and you are less likely to regret picking Undertow in the future.

Jackson-databind and Default Typing Vulnerabilities

Today, GitHub sent out security notices to owners of projects using old jackson-databind versions (older than 2.8.11.1 and 2.9.5). These notices pertain to this issue. I have talked about its relevance before on IRC, but since it is getting more attention now, I will describe it here again.

The Problem

The “bug” comes from using the so-called default typing. This feature allows a user to deserialize subclasses (or even Object) without specifying the full possible type hierarchy. Consider this model:

interface I {
}
@Data
class A implements I {
    private int i;
}
@Data
class B implements I {
    private boolean b;
}

Now, if we wish to serialize the interface I, you usually need to specify some sort of type info. This is typically done through an annotation on the interface:

@JsonTypeInfo(use = JsonTypeInfo.Id.NAME)
@JsonSubTypes({@JsonSubTypes.Type(value = A.class, name = "A"), @JsonSubTypes.Type(value = B.class, name = "B")})
interface I {
}

If we now serialize an object of type I, we get a result like this:

new ObjectMapper().writerFor(I.class).writeValueAsString(new A())
{"@type":"A","i":0}

Deserializing this JSON works as expected.

Default typing

This annotation-based registration is fine for small use cases, but can get cumbersome if the types are in different modules, or there are just a lot of them, or it would be bad style to reference them from the parent class. Normal Java serialization does not have this problem (it just carries the actual dynamic class name with it), so this could be a barrier for adoption of Jackson for previous Java serialization users.

The problem here is that we want people to migrate from Java serialization. There are lots of reasons, most of them compelling.

Enter default typing. With default typing, we don’t need the type info annotations at all:

new ObjectMapper().enableDefaultTyping().writerFor(I.class).writeValueAsString(new A())
["net.hawo.tv.tvui17.A",{"i":0}]

(Ignore the fact that this is now suddenly an array – this is one of the ways jackson may include type info)
The idea is simple: Include the full class name in the json, and you can simply get the proper class at runtime! Sounds good, right?

The Vulnerability

Well, turns out this is not that good of an idea. Java serialization does a very similar thing (though it still requires the named class to be serializable, which jackson doesn’t) and this has lead to what feels like a third of all serious security vulnerabilities in Java applications, ever. The problem lies with the fact that Java classpaths are often massive, and allowing any class on that classpath to be deserialized at will can be disastrous since it exposes a huge attack surface. If you can get any class on the classpath to execute code when deserialized with jackson, you have successfully achieved remote code execution.
This is exactly what happened with Jackson. Some classes that were common on user classpaths could be deserialized to execute arbitrary code. The fix for this issue is basically a blacklist of a few of these classes that could be exploited. Blacklists are not a solution, though, and since this first list, the list has been amended several times. The maintainers are playing whack-a-mole here, and in my opinion it is a waste of everyone’s time to be adding all exploitable classes to this list.

The Solution

Our experience with this same issue in the Java serialization world tells us not to deserialize untrusted data. Luckily, Jackson is much more secure than Java serialization – if you don’t use default typing. The only acceptable solution to this issue in the long run is: do not use default typing to deserialize untrusted data. Default typing is rarely necessary or even a good idea.
Unfortunately, online resources saying this are sparse. Default typing is an “easy” solution, and many people simply do not have the security awareness to see the issue with it – they will stumble over stackoverflow answers such as this one and simply enable default typing to easily serialize Object fields. The documentation of Jackson also doesn’t highlight this as much as it should.
Two alternatives to default typing exist in Jackson:

  • Normal, annotation-based typing as shown above. This allows you either to use the full name as with default typing, or even specify your own name for greater compatibility (you can later change the class name without affecting the serialized representation). This is the “standard” solution, and the appropriate one for most use cases.
  • Should you not know the possible subtypes of a class you wish to deserialize in advance, you can use the rich registerSubtypes API to dynamically add the types you desire. These types could be detected through an existing module system you are already using, or using something like SPI.

All in all, I am a bit dissatisfied with the attention this issue has gotten. The issue is a security vulnerability by design, and anyone using default typing should have been aware of it. Luckily, default typing is not on by default. I do not have statistics but I would be surprised if many people used it or knew of its existence in the first place, and so I find the attention GitHub has given this a bit over the top – the biggest thing these notifications will spread is uncertainty about Jackson, so this article was an attempt at clearing up what it’s actually all about.

OpenJDK 11 on Fedora

Interesting in running the current version of Java (and its compiler) on your Fedora instance? It’s easy:

sudo dnf install java-11-openjdk-devel
sudo alternatives --config java # select Java 11
sudo alternatives --config javac # if necessary, select java 11

Note that this is still labeled as the “early access” version as of this writing – but the early access version is virtually identical to the released version.

The JavaChannel Podcast, Episode XVII (17)

The XVIIth podcast is live! … one wonders why the Roman Empire didn’t last much longer, doesn’t one. One also wonders how long one can refer to oneself in the whatever-person this is. (It’s… second-person? No, folks, it’s “third person.” English grammer fumbling and lesson over.)
In this podcast, we chase a few things: the discussion topic centers back around on Python vs. Java, and why one might encounter one over the other in certain situations.
We also jump down the gullets of a lot of varied topics, only some of which are Java, but most are at least sort-of relevant:

  • Expanded switch statements in java! … Stuff we can look forward to in Java 12. Java’s looking more and more like Kotlin all the time.
  • Hibernate with Kotlin – powered by Spring Boot describes, of all things, using Kotlin, Hibernate, and Spring Boot together. (The summary has a good laundry list of things to pay attention to.)
  • Another Java Enhancement Proposal: the Java Thread Sanitizer. It would provide a dynamic data race detector implementation for Java and native JNI code utilizing ThreadSanitizer in OpenJDK with interpreter and C1 support… and while it’s great that tools like this might exist, it’s fantastic that they’re apparently rarely needed if they show up twenty years after Java’s release.
  • The Left Hand of Equals is not strictly java related, but still interesting. It mostly centers around what it means for one object to be equal to another. I wonder if the channel blog should have recommended LINKS as well as books….
  • TomEE: Running with Systemd is a pretty fair walkthrough, and should be applicable to other app servers as well… that is, if appservers are still relevant in the world where microservice containers do be existin, mon.
  • https://brilliant.org/ is a kinda neat site for math and logic skills. If you use it, don’t forget to take the question surveys, because that’s how it’ll improve.
  • How to Be a Person Who Can Make a Damn Decision is the first of at least two annoying links for Andrew – this one says it’s “how to be a person who can” but actually mostly documents people being those kinds of people. We also have the resurgence of the podcast drinking game; take a shot whenever game theory is mentioned. However, the article doesn’t really have a lot of takeaways, apart from pointing out that the ability to make a decision quickly is probably a worthwhile skill to have.
  • OpenPDF 1.2.1 has been released. Joe didn’t even know about this library. No idea how useful it is; this release doesn’t look like a big one from the surface, but still: the more libraries out there, the merrier, right? (Unless they’re logging libraries.)
  • 7 Scientific Benefits of Reading Printed Books is the second annoying link for Andrew. It goes over some, uh, tenuous reasons print books are worth reading, some of which were taken exception to. Joe thought it was worth thinking about when e-books are ALL the rage for programming topics…
  • Other tangential topics:
    • https://hmijailblog.blogspot.com/2018/08/you-could-have-invented-lmax-disruptor.html I hated reading this, so I stopped.
    • https://perens.com/2018/08/22/new-intel-microcode-license-restriction-is-not-acceptable/ Apparently Intel was saying not to publish benchmarks, which is kinda gross. However, worth noting is that after the initial scraping of this article, Intel backed down and changed the police. Way to go, Bruce Perens!

The JavaChannel Podcast, vol 16

It’s only been six months, so it’s finally time for a new podcast. This one doesn’t even pretend to go over the mountains of killer content from ##java since the last podcast – it focuses on some of the more recent links, and that’s it. Well, apart from talking about the Java ecosystem a bit, especially in contrast with Python, an upstart language that’s making a lot of headway lately thanks to a giant upsurge in data science applications.


(A bit of irony: the very first paragraph in the podcast says it’s only been “four months” when it’s actually been six. Yikes.)


But there are some interesting links, and here are the ones the podcast focused on!



This was written with the new editor plugin for WordPress, called “Gutenberg.” It’s a lot like Medium.com’s editor. It’s effective for writing… unless you have any actual features you want in the text.


Gradle Properties

Gradle supports properties in builds just like Maven does, but the actual documentation and examples are harder to come by. This article will show you something that actually works.
In Maven, you typically have a dependencyManagement section that declares the dependencies with versions; I typically put the versions in a properties block so that they’re all located in a nice, handy, easy-to-manage place. See https://github.com/jottinger/ml/blob/master/pom.xml for a simple example; it’s not consistent with the property usage, but the idea should be clear. I have my versions all coalesced into an easy place to update them, and the changes propagate through the entire project.
Gradle examples rarely do anything like this. Therefore, for your edification, here’s a simple example:
gradle.properties:

aws_sdk_version=1.11.377
kotlin_version=1.2.50

build.gradle:

buildscript {
  repositories {
    mavenCentral()
  }
  dependencies {
    classpath "org.jetbrains.kotlin:kotlin-gradle-plugin:$kotlin_version"
  }
}
apply plugin: 'java'
apply plugin: 'kotlin'
repositories {
    mavenCentral()
    jcenter()
}
dependencies {
    compile "org.jetbrains.kotlin:kotlin-stdlib-jdk8"
    compile "com.amazonaws:aws-java-sdk:$aws_sdk_version"
}
tasks.withType(org.jetbrains.kotlin.gradle.tasks.KotlinCompile).all {
    kotlinOptions {
        jvmTarget = "1.8"
        javaParameters = true
    }
}

You can use a constraints block in the dependencies to do the same thing as dependencyManagement in Maven; that would look like this:

dependencies {
    constraints {
        implementation "com.amazonaws:aws-java-sdk:$aws_sdk_version"
    }
}

This would allow submodules (or this module) to use a simpler dependency declaration, leaving off the version, just like Maven can do:

dependencies {
    compile "org.jetbrains.kotlin:kotlin-stdlib-jdk8"
    compile "com.amazonaws:aws-java-sdk"
}

If you don’t want to use gradle.properties, that’s doable too (as pointed out by channel member matsurago) – you can put it in the buildscript block, as so:

buildscript {
  ext.aws_sdk_version="1.11.377"
}