Private tempstore should store a value that persists with the user's session.
However, if the user is anonymous, then this is broken, because saving to tempstore doesn't start a session.
You can see the problem by running this code (eg in Devel's exec PHP form). Each execution causes a new row in key_value_expire:
\Drupal::service('user.private_tempstore')->get('test')->set('test', 'test value!');The expected behaviour is that after the first row is inserted, subsequent executions just update it.