I'm using drupal restricted to family and friends, so I've switches after some preparation from public to private files, as I also wanted to restrict downloads to authenticated users.
While the public stored files got only downloaded again after changes, the private stored files get downloaded again and again at every request. The browser cache is completely bypassed.
I noticed this extremely as I've let run a colorbox slideshow and it still downloaded the pictures again after the end reached and it went back to start. Using public files caused to use the browser cache, as expected. But not for private files.