Functional test that uses drupalLogin() does not work on nginx+php-fpm on 8.5.x

I have a very simple functional test that checks if we can create a user and log in:

/**
 * Test login.
 */
public function testLogin() {
  $web_user = $this->drupalCreateUser();
  $this->drupalLogin($web_user);
}

This worked fine on 8.4.6 but after upgrading to 8.5.1 this fails with the following message:

1) Drupal\Tests\login_example\Functional\LoginExampleLoginTest::testLogin
User <em class="placeholder">sud6Y9W6</em> successfully logged in.
Failed asserting that false is true.

/var/www/html/docroot/core/tests/Drupal/KernelTests/AssertLegacyTrait.php:31
/var/www/html/docroot/core/tests/Drupal/Tests/BrowserTestBase.php:783
/var/www/html/docroot/modules/custom/login_example/tests/src/Functional/LoginExampleLoginTest.php:26

Another note of importance is that we're running this in a Docker setup with php:7.1-fpm and nginx:1.12. I tried the same code on php:7.1-apache and it worked there. This makes me unsure if it's nginx, php-fpm or Drupal that is the issue. But seeing as this worked on 8.4.x and not 8.5.x I'm guessing Drupal.

I have tracked it down to the GoutteDriver not having the cookie for the session ID, but not sure on how to debug it further.
To illustrate the problem and help others debug I have set up a repository with the example code:
https://bitbucket.org/johnzzon/login-example-nginx

Sincerely grateful for any help on this matter!