Error when a template is not found via an "include" is confusing

Problem/Motivation

This is the error you get if a template was not found on the filesystem via an include statement in a twig template:

Twig_Error_Loader: Template "footer.html.twig" is not defined (Drupal\Core\Template\Loader\ThemeRegistryLoader: Unable to find template "footer.html.twig" in the Drupal theme registry.) in "themes/havanasquare/templates/pages/page.html.twig" at line 3.

The problem is that the file could not be found on the filesystem, not the "Drupal theme registry".

Proposed resolution

Help out our themers and explained what really happend.

Remaining tasks

User interface changes

Original Report

Hi,
I'm having issues with including files in my theme through: {% include 'filename.html.twig' %}

I have my page.html.twig template and I want to break out the header and footer of the template into separate files for easy updating. Simple enough.

I create my new file 'header.html.twig' in an 'includes' folder in the same directory as my 'page.html.twig' and in the page template I have {% include 'includes/header.html.twig' %}. When I save and refresh the page I get a white screen with this:

The website encountered an unexpected error. Please try again later.

Twig_Error_Loader: Template "footer.html.twig" is not defined (Drupal\Core\Template\Loader\ThemeRegistryLoader: Unable to find template "footer.html.twig" in the Drupal theme registry.) in "themes/havanasquare/templates/pages/page.html.twig" at line 3. in Twig_Loader_Chain->getCacheKey() (line 115 of vendor/twig/twig/lib/Twig/Loader/Chain.php).
Drupal\Core\Template\TwigEnvironment->getTemplateClass('footer.html.twig', NULL) (Line: 378)
Twig_Environment->loadTemplate('footer.html.twig', NULL) (Line: 286)
Twig_Template->loadTemplate('footer.html.twig', 'themes/havanasquare/templates/pages/page.html.twig', 3) (Line: 47)
__TwigTemplate_fea998e363960c4f3282a328eb416af5281fbd1ab4e1a4a6d5313635516d3b1a->doDisplay(Array, Array) (Line: 381)
Twig_Template->displayWithErrorHandling(Array, Array) (Line: 355)
Twig_Template->display(Array) (Line: 366)
Twig_Template->render(Array) (Line: 64)
twig_render_template('themes/havanasquare/templates/pages/page.html.twig', Array) (Line: 384)
Drupal\Core\Theme\ThemeManager->render('page', Array) (Line: 435)
Drupal\Core\Render\Renderer->doRender(Array, ) (Line: 195)
Drupal\Core\Render\Renderer->render(Array) (Line: 468)
Drupal\Core\Template\TwigExtension->escapeFilter(Object, Array, 'html', NULL, 1) (Line: 87)
__TwigTemplate_bc4fcddd7ff4eda4647e5ce2941a77f75b23b0d918da1ce4d489ab83bcbfc449->doDisplay(Array, Array) (Line: 381)
Twig_Template->displayWithErrorHandling(Array, Array) (Line: 355)
Twig_Template->display(Array) (Line: 366)
Twig_Template->render(Array) (Line: 64)
twig_render_template('themes/havanasquare/templates/html.html.twig', Array) (Line: 384)
Drupal\Core\Theme\ThemeManager->render('html', Array) (Line: 435)
Drupal\Core\Render\Renderer->doRender(Array, ) (Line: 195)
Drupal\Core\Render\Renderer->render(Array) (Line: 147)
Drupal\Core\Render\MainContent\HtmlRenderer->Drupal\Core\Render\MainContent\{closure}() (Line: 574)
Drupal\Core\Render\Renderer->executeInRenderContext(Object, Object) (Line: 148)
Drupal\Core\Render\MainContent\HtmlRenderer->renderResponse(Array, Object, Object) (Line: 90)
Drupal\Core\EventSubscriber\MainContentViewSubscriber->onViewRenderArray(Object, 'kernel.view', Object) (Line: 111)
Drupal\Component\EventDispatcher\ContainerAwareEventDispatcher->dispatch('kernel.view', Object) (Line: 144)
Symfony\Component\HttpKernel\HttpKernel->handleRaw(Object, 1) (Line: 62)
Symfony\Component\HttpKernel\HttpKernel->handle(Object, 1, 1) (Line: 57)
Drupal\Core\StackMiddleware\Session->handle(Object, 1, 1) (Line: 47)
Drupal\Core\StackMiddleware\KernelPreHandle->handle(Object, 1, 1) (Line: 99)
Drupal\page_cache\StackMiddleware\PageCache->pass(Object, 1, 1) (Line: 78)
Drupal\page_cache\StackMiddleware\PageCache->handle(Object, 1, 1) (Line: 47)
Drupal\Core\StackMiddleware\ReverseProxyMiddleware->handle(Object, 1, 1) (Line: 50)
Drupal\Core\StackMiddleware\NegotiationMiddleware->handle(Object, 1, 1) (Line: 23)
Stack\StackedHttpKernel->handle(Object, 1, 1) (Line: 652)
Drupal\Core\DrupalKernel->handle(Object) (Line: 19)

Every article I have found said that should work. I have also tried putting a direct path {% include directory ~ '/includes/header.html.twig' %} and other attempts with no luck. Been stuck on this for a while.

Any help would be appreciated!