Using CacheableResponseInterface to better determine which responses should be cached/cacheable

Problem/Motivation

Blocked on #2463009: Introduce CacheableResponseInterface: consolidate ways of setting X-Drupal-Cache-Tags/Contexts headers.

Quoting Fabianx in #2463009-12: Introduce CacheableResponseInterface: consolidate ways of setting X-Drupal-Cache-Tags/Contexts headers:

[…]
We use a check on instanceof CacheableResponseInterface for _any_ cache related data we set in FinishResponseSubscriber as final headers on the response.

That means that if you do:

<?php
return new Response('Hi');
?>

you are yourself responsible for everything, but the default Response is uncacheable (Cache-Control: private, no-cache, no-store, etc.) so this is not a security issue.

This essentially makes Drupal's cache, etc. system opt-in instead of mandatory, which should be fine and was a goal of the WSCCI initiative in the first place.

[…]

Proposed resolution

So, talking concretely, this issue wants to do what Fabianx said in #2463009-27: Introduce CacheableResponseInterface: consolidate ways of setting X-Drupal-Cache-Tags/Contexts headers:

a) Only do any Cache-Control headers if it is a cacheable response, this truly lets someone opt out of core's caching architecture, etc.
b) Have page cache honor Cache-Control header OR check for cacheable response as well.

Critical because

Quoting catch in #2463009-18: Introduce CacheableResponseInterface: consolidate ways of setting X-Drupal-Cache-Tags/Contexts headers:

Discussed this with Fabianx in irc.

If we do #2472321: Move rest.module routes to /api, then we'll be sending Vary: accept in the case that a route has two possible formats.

In that case, the page cache needs to handle it.

Previously we had content negotiation up-front for page caching, but we don't want to do that on every page cache request since it's not needed.

However, if we use cache contexts, we can set a request_format cache context, whitelist this in the page cache (currently it strips all cache contexts), then rely on cache contexts (i.e. cache redirect) to get the correct item from cache.

That has no runtime impact on any other page cache responses (or not much more than an if statement anyway), and will work accurately for the ones that need it.

I think it's likely that we can't do this in a patch release since it'll be quite a large internal refactor of the page cache - i.e. it might break it until you run rebuild.php or similar, so bumping to critical, but adding triage tag to discuss with other committers. The impact of not doing this is we'd have no internal page caching at all for /api/ routes.

Remaining tasks

Do it.

User interface changes

None.

API changes

TBD