After upgrading from Drupal 7, no users are able to log into the site until clearing cookies (or presumably, waiting 3 weeks until the cookie expires).
Before and after the upgrade, the site is hosted on https and with www.
Unfortunately, since I cleared my cookies I don't have the Drupal 7 cookie anymore, but comparing with other sites, it looks like the cookie name / value are the same format across D7/D8, both are HttpOnly and Secure, path is the same. However, the D8 cookie includes the www. in the cookie domain while the D7 one does not.
Drupal 7 cookie: .example.com
Drupal 8 cookie: .www.example.com
I also tried to login using the one-time login created with drush uli, but even this wouldn't work.
Proposed Solution:
Unset or ignore any cookies that do not correspond to an active session.