Month: February 2018

JavaChannel’s Interesting Links podcast, episode 15

Welcome to the fifteenth ##java podcast. Your hosts are Joseph Ottinger, dreamreal on the IRC channel, and Andrew Lombardi (kinabalu on IRC) from Mystic Coders.
As always, this podcast is basically interesting content pulled from various sources, and funneled through the ##java IRC channel on freenode. You can find the show notes at the channel’s website, at javachannel.org; you can find all of the podcasts using the tag (or even “category”) “podcast”, and each podcast is tagged with its own identifier, too, so you can find this one by searching for the tag “podcast-15”.
Discussion today centered around specifications: any good ones out there? What would good specifications look like?

  1. A few podcasts ago (podcast 11) we mentioned “resilience4j” – well, there’s also retry4j on github. It’s another library that offers retry semantics. retry4j’s semantics are really clean; maybe it’s me (dreamreal), but it seems more in line with what I’d expect a retry library to look like.

  2. From Youtube, we have a video showing how to use Graal in the JVM. Graal exposes JVMTI internals – which means that theoretically you could write a replacement for the just-in-time compiler, and change some fundamental ways of how Java works. Twitter is using Graal today, and the presenter says that it works fantastically; it’s also slotted to become standard in the next few revisions of Java, so we’ll have it probably next month or so.

  3. From the channel blog itself, user yawkat posted a way to cache HTTP results regardless of response with OkHttp. Why is this important? Well, if you’re testing a crawler, it’s handy to not slam the target server – even on results that should give things like “not found.” It’s written in Kotlin; the Java code would be trivial (and maybe we should post it as well just for the sake of example) but it’s still pretty handy.

  4. There’s also a Java Enhancement Proposal to allow launching single-source-code files like “java File.java”, and it automatically invokes javac; also with support for shebang, so we could be looking at Java scripting before too long. Some people already do this with scripts; they have something that pipes Java source into a temporary file, compiles it, and runs it, but this would be more standardized.

  5. We’ve seen a few posts about Hibernate on the channel lately, including some about projections (where an object isn’t represented as such in the data model but is constructed on-the-fly). Vlad Mihalcea actually had an appropriate post about caching such projections.

  6. We also have seen references lately to Project Sumatra – GPU enablement for the JVM, which seems abandoned – and then there’s this from the valhalla developer’s list: there IS a GPU exploitation thing being worked on. Who knows, we might see highly-GPU-enabled JVMs or libraries innate for Java before long.

  7. DZone has an article called “Java Phasers Made Simple” – basically a set of limited-scope locks. It’s really well done although quite long; hard for it to go into detail without being fairly long, though, so that’s okay. Worthwhile read.

Caching All OkHttp Responses

When (integration) testing scrapers you need to strike a balance between “get as much input data as possible to cover parsing edge-cases” and “don’t DoS the backend, please”. Caching can help with that, and gives you a nice performance boost during testing on top.
The OkHttp http client library contains a cache built-in but by default it follows HTTP server caching headers. It also allows you to set a particular request to always use the cache, and error if the request isn’t present yet, but that is too harsh (we do want to fetch the first time after all). Turns out you can do both:

val cachedHttpClient = OkHttpClient.Builder()
        .cache(Cache(File(".download-cache/okhttp"), 512 * 1024 * 1024))
        .addInterceptor { chain ->
            chain.proceed(
                    chain.request().newBuilder()
                            .cacheControl(CacheControl.Builder()
                                    .maxStale(Integer.MAX_VALUE, TimeUnit.SECONDS)
                                    .build())
                            .build()
            )
        }
        .addNetworkInterceptor { chain ->
            log.info("Fetching {}", chain.request().toString())
            chain.proceed(chain.request())
        }
        .build()!!

This is kotlin code, but you get the point.

JavaChannel’s Interesting Links podcast, episode 14

Welcome to the fourteenth ##java podcast. Your hosts, as usual, are Joseph Ottinger, dreamreal on the IRC channel, and Andrew Lombardi from Mystic Coders (kinabalu on the channel), and it’s Wednesday, February 7, 2018.
As always, this podcast is basically interesting content pulled from various sources, and funneled through the ##java IRC channel on freenode. You can find the show notes at the channel’s website, at javachannel.org; you can find all of the podcasts using the tag (or even “category”) “podcast”, and each podcast is tagged with its own identifier, too, so you can find this one by searching for the tag “podcast-14”.

  1. javan-warty-pig is a fuzzer for Java. Basically, a fuzzer generates lots of potential inputs for a test; for example, if you were going to write a test to parse a number, well, you’d generate inputs like empty text, or “this is a test” or various numbers, and you’d expect that your tests would validate errors or demonstrate compliance to number conversion (this is a way of saying “it would parse the numbers.”) A Fuzzer generates this sort of thing largely randomly, and is a good way of really stressing the inputs for the methods; the fuzzer has no regard for boundary conditions, so it’s usually a good way of making sure you’ve covered cases. The question, therefore, becomes: would you use a fuzzer, or HAVE you used a fuzzer? Do you even see the applicability of such a tool? There’s no doubt that it can be useful, but potential doesn’t mean that the potential will be leveraged.

  2. Simon Levermann, mentioned last week for having released pwhash, wrote up an article for the channel blog detailing its use and reason for existing. Thank you, Simon!

  3. Scala is in a complex fight to overthrow Java, from DZone. Is the author willing to share the drugs they’re on? Scala’s been getting a ton of public notice lately – it’s like the Scala advocates finally figured out that everything Scala brought to the table, Kotlin does better, and with far less toxicity. If kotlin wanted to take aim at Scala, there’d be no contest – Kotlin would win immediately, unless “used in Spark and Kafka” were among the criteria for deciding a winner. It’s a fair criterion, though, honestly; Spark and Kafka are in fairly wide deployment. But Scala is incidental for them, and chances are that their developers would really rather have used something a lot more kind to them, like Kotlin, rather than Scala.

  4. More of the joys of having a super-rapid release cycle in Java: according to a post on the openjdk mailing list, bugs marked as critical are basically being ignored because the java 9 project is being shuttered. It’s apparently on to Java 10. This is going to take some getting used to. It’s good to have the new features, I guess, after wondering for years if Java would get things like lambdas, multiline strings, and so forth, but the rapid abandonment of releases before we even have a chance to see widespread adoption of the runtimes is… strange.

  5. James Ward posted Open Sourcing “Get You a License”, about a tool that allows you to pull up licenses for an entire github organization – and issue pull requests that automatically add a license for the various projects that need one. Brilliant idea. Laziness is the brother of invention, that’s what Uncle Grandpa always said.

Modular password hashing with pwhash

When building web applications, we usually also have to store user authentication data. When doing so, there’s generally two choices: Either we use an external authentication provider like OAuth, or we store passwords for our users in our database. This blog post focuses on how to do the latter correctly.

Password hashing basics

Before we dive into the specifics of the pwhash library, we briefly discuss the basics of what we need to be wary of when storing passwords. The first, and most important point is that we do not actually ever store our users’ passwords. Instead, we run them through a cryptographic hash function. This way, in case our database ever gets compromised, the attacker doesn’t immediately know all our users’ passwords. But this is not considered enough any more. Because if we just put our users’ passwords through a simple hash function, like SHA-3, two users that pick the same password will end up with the same hash value. Knowing this is of course useful to an attacker, because now they can attack multiple passwords at the same time, if they get the user data.

Editor’s note: so choose passwords likely to be unique… and unguessable even to people who know you. Please.

Enter salts

To mitigate this problem, we use so-called salts. For each password, we generate a random salt, and prepend (or append) it to the password, before passing it through the hash function. This leaves us with
sha3(firstsalt:password) = 0bee3940e2d74f5155e73a9e90ea75b5d06407db85527f62acefa97af2d59f69589b276630e20a1ff8b0b781a372ae17db88b9f782acf7ed0022ab4c2fc766df
sha3(secondsalt:password) = edaa1e8b31e2d2943d689928a5ba1c503bb3220ddc164d6feff9e178dd18a4727b1905400252ef071d28b370c0b9727420cd0e2011109ca3e8934a4082d2bf9e
As we can see, this yields two completely different hashes even though two users used the same (admittedly very bad) password. The salt can be stored alongside the hash in the database. An attacker now has to break each password individually, instead of attacking them all at the same time. Great, right? Surely, now we’re done and can get to the actual library? Not quite yet. The next problem we have is that modern graphics cards are really fast at calculating these kind of hashes. For example, an Nvidia GTX 1080 calculates around 800 million SHA-3 hashes per second.

Dedicated password hashing functions

To get rid of this problem, we do something we usually don’t want to do in computing: We make things intentionally slow. There are several functions that can be used for this. An older idea is simply applying many rounds of the same hash function over and over again. An example of this approach is the PBKDF2 (Password-Based-Key-Derivation-Function). But since GPUs are really fast these days, we also want functions that are harder to compute on GPUs. Functions that use a lot of memory, and a lot of branches are generally very hard to compute on graphics cards. An example of this is argon2, a function which was specifically designed for hashing passwords, and won the Password Hashing Competition in 2015. The question, then, is how do we create a re-usable approach that allows us to stay current with always-current password hashing requirements?

The pwhash library

While there are already Java implementations and bindings for argon2 and other password hashing functions such as bcrypt, what is still missing in the Java ecosystem is a library that unifies them under a single interface. Functions like argon2 come with different versions, and a lot of adjustable parameters. So if we hardcode those parameters into our application in 2018, the parameters are probably going to be outdated (and inefficient or insecure) in five or ten years. So ideally, we want our library to handle this for us. We change our parameters in one place, and the library automatically takes care of upgrading both new and existing password hashes every time a user logs in or signs up. In addition to this, it would also be convenient to not just switch between parameters of a single algorithm, but also to switch the algorithm to something newer and better altogether.
That’s the goal of the pwhash library.

The HashStrategy interface

To offer all this, the core functionality of pwhash is the HashStrategy interface. It offers 3 methods:

  1. String hash(String password)
  2. boolean verify(String password, String hash)
  3. boolean needsRehash(String password, String hash)

This interface is largely inspired by the PHP APIs for password hashing, which offers the functions password_hash, password_verify and password_needs_rehash. In the author’s opinion, this is one of the better things in the PHP core library, and hence I decided to port the functionality to Java in a bit more Object-Oriented style.
The first two methods are relatively straightforward: String hash(myPassword) produces a hash, alongside with all its parameters, which can later be passed to boolean verify(myPassword, storedHash) in order to verify if the password actually matches the given hash.
The third method is the magic that lets us upgrade our hashes without needing to write new code every time. In the first step, it checks whether the given password actually verifies against the hash, in order to make sure that we don’t accidentally overwrite hashes when the user supplied an incorrect password. In the next step, the parameters used for the current HashStrategy (which are generally supplied by the constructor, or some factory method) are compared with the ones stored alongside the password hash. If they match, false is returned, and no further action needs to be taken. If they do not match, the method returns true instead. Now, we call hash(myPassword) again, and store the newly generated hash in the database.
This way, we only have to write code once for our login, and every time we change our parameters, they are automatically updated every time users log in. However, for a single implementation, like the Argon2Strategy, this only handles changes in parameters. If somehow a weakness is discovered, we still do not have a good way to migrate away from Argon2 to a different algorithm.

The MigrationStrategy class

Migration between two different algorithms is handled by the MigrationStrategy class, which implements the above interface, and in its constructor, accepts two other strategies: One to migrate from, and another to migrate to. Its implementation for String hash(String) simply calls the hash functions of the latter strategy, since we want all new hashes to be performed with the new algorithm.
Its implementation of boolean verify(String, String) first attempts to verify against the old strategy, and if that fails, against the new strategy. If neither succeed, the password was incorrect. If either succeeds, the password was correct, and true can be returned.
Again, the boolean needsRehash(String, String) method is a little bit more complicated. It first attempts to verify the given password against the old strategy. If this succeeds, the password is obviously in the old format, and needs to be rehashed, and we immediately return true. In the second step, we attempt to verify it with the new strategy. If this is also unsuccessful, we return false as we don’t want to rehash if the user supplied an incorrect password. If the verification succeeds, we return whatever newStrategy.needsRehash(String, String) returns.
This way, we can adjust parameters on our new strategy while some passwords are still hashed with the old strategy, and we receive the expected results.
Note that it is not necessary to use this class when migrating parameters inside one implementation. It is only used when switching between different classes.

Chaining MigrationStrategy

Fun fact about MigrationStrategy: Since it is also an implementation of the HashStrategy interface, you can even nest it in another MigrationStrategy like so:

MigrationStrategy one = new MigrationStrategy(veryOldStrategy, somewhatBetterStrategy);
MigrationStrategy two = new MigrationStrategy(one, reallyGoodStrategy);

This might not seem useful at first, but it is useful if you have a lot of users. You may want to switch to yet another strategy at some point, while not all your users are migrated to the intermediate strategy yet. Chaining the migrations like this allows you to successfully verify against all three strategies, still allowing logins for even your oldest users who haven’t logged in in a while, and still migrating all passwords to the newest possible algorithm.

Examples

Examples for the usage of the pwhash library are hosted in the repository on GitHub.
Currently, there are two examples. One uses only the Argon2Strategy and upgrades parameters within that implementation. The second example uses the MigrationStrategy to show code that migrates users from a legacy Pbkdf2Strategy to a more modern Argon2Strategy.
The best thing about these examples: The code that actually authenticates users is exactly the same in both examples. The only thing that is different is the HashStrategy that is plugged in. This is the main selling point of the pwhash library: You write your authentication code once, and when you want to upgrade, all you have to do is plug in a new strategy.

Where to get it

pwhash is available on Maven Central. For new applications, it is recommended to only use the -core artifact. Support for PBKDF2 is mainly targeted at legacy code bases wanting to migrate away from PBKDF2. JavaDoc is available online on the project homepage. JAR archives are also available on GitHub, but it is strongly recommended you use maven instead.