MockWebServer + HTTPS

Sep 11, 2021
android security testing development
6 min read

Recently I ran into a fairly niche configuration problem with MockWebServer, where I needed to be able to connect to the server via https. What follows is some notes on the problem and the solution we used - hopefully somebody else finds this information useful and I’m sure there are other solutions which would have worked just as well.

Context

Over the last few weeks, one of the things I’ve been focusing on at Cuvva is enabling the team to write tests which are broader in scope - i.e. instrumentation testing. Historically, we’ve not invested in this but we’re at a point where this would be highly useful, because these types of tests provide a couple of serious benefits:

Acceptance testing - given a well-spec’d acceptance document, it should be easy for us to write an instrumentation test which proves that the behaviour of the screen is correct.
Regression testing - we currently manually smoke test large sections of the app on release builds. Automating some of these would save us lots of time and improve our confidence in releases.
Providing a harness - despite having done a tonne of work over the last two years, we still have a few core pages which contain scary legacy code. Being able to verify their behaviour in a test makes refactoring them much less scary.
More of the stack covered - unit tests are great, and proving that your function which adds two numbers together is correct is brilliant. But unit testing doesn’t guarantee that these otherwise tested functions work together in the way that you expect.

Once setup, these tests would run on our new Flank sharding setup, executing on Firebase Test Lab.

Approach

Part of the enabling work for this was getting MockWebServer up and running so that we could intercept and respond to all network traffic under test.

This approach isn’t particularly common as far as I’m aware - a more usual way of doing it is to mock out some other layer further up the stack - whether that is your networking layer, repositories or usecases. But for various reasons it would have been pretty painful for us to do this. For some legacy screens this method simply isn’t possible, and the only way we can have any control is by providing pre-determined responses.

Instead we chose to utilize MockWebServer to allow us to provide all of the inputs to an instrumentation test with a minimum amount of changes to our existing codebase.

This approach provides a number of benefits; chiefly we wanted to be able to store JSON responses containing all sorts of edge cases and test how various screens react to that, rather than hitting our test backend where we don’t have control over the state. In doing so we get to test the entire stack quite exhaustively, which was very appealing.

The Problem

Initially I started mocking responses for some of our endpoints but rapidly realised that there was a bit of an issue - our app (and indeed almost every modern Android app) expects secure communication with the server using TLS. MockWebServer by default works in plaintext, serving responses from http://localhost/.

You could override your debug configuration in your network security config to allow plaintext communication, and that would work fine. But we’d also told OkHttp explicitly to use TLS:

@Provides
@Singleton
internal fun provideOkHttp(
    interceptors: Set<@JvmSuppressWildcards Interceptor>,
): OkHttpClient = OkHttpClient.Builder()
    .apply { interceptors().addAll(interceptors) }
    .connectionSpecs(listOf(ConnectionSpec.MODERN_TLS))
    // Other stuff
    .build()

And I didn’t really want to have some if (BuildConfig.DEBUG) ... logic in our Dagger module which configures OkHttp. So my preference ended up being to enabling https traffic from MockWebServer.

Getting set up

Step 0 - you’ll need the okhttp-tls artifact. If you’re using version catalogs because you’re mega trendy, it’ll look like this:

squareup-okhttp-tls = { module = "com.squareup.okhttp3:okhttp-tls", version.ref = "okHttp" }

There’s more documentation on the TLS artifact here which goes into depth about a few of the concepts I’m discussing in this article.

Next - you’ll need to generate a certificate: these are in .pem format. The easiest thing to do that I found was to simply generate one in code and print the output - normally this isn’t something you’d do, but because these are strictly for tests it’s not a huge deal if someone gets a hold of the private key.

Don’t forget to set a duration time! Without this, the default is 24 hours - this could result in a nasty surprise when tests start failing on your build server the day after you’ve merged them.

val localhost = InetAddress.getByName("localhost").getCanonicalHostName()
val localhostCertificate = HeldCertificate.Builder()
    .addSubjectAlternativeName(localhost)
    .duration(10 * 365, TimeUnit.DAYS)
    .build()
// Print public key
println(localhostCertificate.certificatePem())
// Print private key
println(localhostCertificate.privateKeyPkcs8Pem())

Your output will look something like this:

Next, passing these to OkHttp is easy enough. You’ll want to store the entire output - from BEGIN CERTIFICATE to END PRIVATE KEY - in a .pem file, placed in your test resources:

val localhostCertificate =
    HeldCertificate.decode("instrumentation_cert.pem".loadString())

val serverCertificates = HandshakeCertificates.Builder()
    .heldCertificate(localhostCertificate)
    .build()

server.useHttps(
    serverCertificates.sslSocketFactory(),
    tunnelProxy = false,
)

Where loadString simply reads a file from test resources:

fun String.loadString(): String =
    object {}.javaClass.getResource("/$this").readText()

The resulting HandshakeCertificates object provides an SslSocketFactory, which you pass to OkHttp. Super easy.

One last gotcha

What we’ve done here is we’ve created a self-signed certificate. This is counter to how certificates normally work and modern networking stacks won’t like it.

Typically, when OkHttp connects to a server, it reads the certificate provided and finds that it’s been signed by someone else. It then works its way up the chain until it finds a root Certificate Authority - this is a self-signed certificate too, but an authoritative one which is internationally recognised. By working up this chain, OkHttp can verify that the server is who it says it is.

google.com’s certificate, showing a chain from a root CA.

Here you can see the chain for google.com, showing their certificate, an intermediary CA, and the root CA - which also happens to be issued by Google Trust Services.

Anyway, the point is that no-one which anyone recognises has verified your freshly-minted self-signed certificate, so you have to tell Android that you trust it. This is easily done in your network security config:

<?xml version="1.0" encoding="utf-8"?>
<network-security-config>
    <debug-overrides>
        <trust-anchors>
            <certificates src="@raw/debug_cert"/>
        </trust-anchors>
    </debug-overrides>
</network-security-config>

Make sure that you only add your test certificate inside debug-overrides - this will ensure that your network security config on release builds isn’t affected.

If you really want to, you can provide a separate network security config file and place it under your debug/src folder so that you can guarantee that your app doesn’t ship with this certificate.

It’s worth pointing out that in this particular case, debug_cert is just the public key (i.e. the BEGIN CERTIFICATE/END CERTIFICATE block), stored in plaintext this time.

This should now work fine, and you should be able to run tests hitting MockWebServer securely without any issues.

Cleaning up

Lastly, we don’t want to repeat ourselves in every test, and we sure don’t want to extend some awful BaseTest class - so create a test rule, like a good citizen:

public class InstrumentationTestRule : ExternalResource() {

    val server = MockWebServer()

    override fun before() {
        val localhostCertificate =
            HeldCertificate.decode("instrumentation_cert.pem".loadString())

        val serverCertificates = HandshakeCertificates.Builder()
            .heldCertificate(localhostCertificate)
            .build()

        server.useHttps(
            serverCertificates.sslSocketFactory(),
            tunnelProxy = false,
        )

        server.start()
    }

    override fun after() {
        server.shutdown()
    }
}

Rounding off

If you made it this far, thanks for reading. This won’t apply to many people but hopefully for the few that have this problem, this article serves as a useful reference.