[meta] Drupal and PSR-0/PSR-4 Class Loading

Quick summary

It was decided (Dries, #111) to use PSR-4 for module-provided classes, instead of PSR-0.
#2083547: PSR-4: Putting it all together has a technical implementation that works.
This desperately needs reviews, so we can have PSR-4 as soon as possible.

Longer summary

Problem: Deeply nested PSR-0 directories

Currently, module-provided classes in Drupal 8 live in PSR-0 directories under lib/.
It was argued that this brings a poor DX, because of
- the deeply nested directory structure, e.g. "core/modules/system/lib/Drupal/system/Foo.php".
- the repetition of the module name in the directory path, which feels confusing.

DX aspects that were mentioned (Pro and Con):
- Learning curve: There are controversial opinions in this thread about whether PSR-0 or PSR-4 is harder or easier to learn.
- Consistency: It was argued it is weird to have PSR-4 in modules, but PSR-0 in core classes.
- Consistency: It was argued it is weird if a namespaced class name does not match 1:1 to a filesystem path.
- Productivity: Number of actions needed to perform a task (e.g. to browse in directories).
- Visual clutter: Redundant directory levels showing up in error logs, stack traces, command prompt, git diff, etc. They cost screen space, and they can be distracting.
- Distraction: The risk to have your "flow" interrupted due to any of the above.

How you are affected depends on the tools you are using:
- Command line: Additional TAB-TAB to drill down the directories, additional "../../" to go up again. Visual clutter in error logs, stack traces, git diff/status, patches, the command prompt, the command itself (if it contains a file path), git diff, error logs, etc.
- IDE (e.g. PhpStorm): Visual clutter in file browser, search results etc. Most of the productivity cost can be avoided with shortcuts etc.
- Filesystem explorer (Finder, Nautilus, Nemo): Overall more clicking, and visual clutter. Longer "breadcrumb".
- Web repository explorers (github, drupalcode.org, dreditor patch review): More clicking, more pages to visit, longer breadcrumbs, longer urls.
- Error reports shown in the browser: Longer file paths.

There is a controversy in the comments about the actual DX impact.

Interoperability?

It has been argued that the realistic scenario for "interoperability" is not to use a Drupal module outside of Drupal, but to have the reusable parts in a Composer package, and only use the module as an "adapter" to the Drupal world. So "interoperability" in Drupal modules would not matter that much.

This being said: This point is going to be irrelevant, because PHP-FIG and Composer are going to support PSR-4. Composer even wants to make it the preferred way to register namespaces, giving it priority over PSR-0.
Most projects that used to register their namespaces as PSR-0 will be able to register them as PSR-4, even without moving any files around.
See below, "PSR-4 in the PHP community".

Decision: PSR-4 for modules

In this issue it was decided (Dries, #111) to use the upcoming/proposed PSR-4 for module-provided classes, instead of PSR-0, to get rid of the additional directory levels.
https://github.com/php-fig/fig-standards/blob/master/proposed/psr-4-auto...
This proposal introduces two changes to PSR-0:

• Remove redundant directory levels for classes provided by modules. For us, this will eliminate the "Drupal/$modulename/", so we get paths like core/modules/system/lib/Foo.php.
• Underscores in the class name no longer have a special meaning. This does not practically affect us, because none of our classes has an underscore after the last namespace separator.

PSR-4 in the PHP community

PSR-4 is being voted on NOW (Tue, Nov 19, 2013) in the PHP Framework Interoperability Group (php-fig):
https://groups.google.com/forum/#!topic/php-fig/L8oCDQCzDcs
A previous vote a few months ago was mostly successful (a lot of yes votes), but was stopped by the editor to improve the wording.

The Composer maintainers have been supportive of PSR-4, and a PR was started even before approval by the PHP-FIG.
The reason it is not finished is some refactoring that was planned to do beforehand.
(the refactoring plan is probably being postponed now, so PSR-4 won't have to wait for it any more)
https://github.com/composer/composer/pull/2121

Technical plan

The plan is to do this in separate steps/patches:

1. An intermediate solution to support PSR-4 in modules alongside PSR-0.
   #2083547: PSR-4: Putting it all together
   This means
   • Class loading:
     Introduce a Drupal-native copy of the PSR-4 class loader proposed for Composer. But make it as easy as possible to switch back to using Composer's class loader once it is ready.
     This class loader will be wired up both with the PSR-0 and the PSR-4 mappings for module namespaces.
   • Port AnnotatedClassDiscovery (and plugin managers) to PSR-4 alongside PSR-0.
   • Port discovery of module test classes (web test) to PSR-4, while still supporting PSR-0.
   • Introduce a script that can move class files in core/modules to their new PSR-4 location.
2. Run the script to move all module-provided class files, and commit the result.
   This should happen at a planned time (point release) to not make too many people angry.
3. Allow some time for contrib to catch up.
4. Remove support for PSR-0 in modules.

Originally proposed alternatives

Both quicksketch and webchick have asked me about this before. I'm sure others have voiced concerns too. This is a write up of what our current PSR-0 system uses, and possible alternatives...

PSR-0?

First I'd like to cover what PSR-0 is. It's pretty much saying that a class's namespace matches its directory structure. PSR-0 is not a class loader, or any implementation, it's just a practice. MyStuff\MyClass should be at MyStuff/MyClass.php, somewhere. Is it the right thing for us? Read on...

1. Current System

Drupal 8 currently uses PSR-0 on each module's lib directory. This means that Drupal\mymodule\MyClass is loaded at mymodule/lib/Drupal/mymodule/MyClass.php. It's PSR-0 as the namespace matches to the directory structure (1:1). Simple. Easy. /lib is registered with Drupal/mymodule.

Pros

• Biggest bang for our buck when we started the PSR-0 Initiative
• Already a huge amount of class loaders available
• Most PHP projects out there already are using this approach

Cons

• Developers have to make three nested directories when making a module (mymodule/lib/Drupal/mymodule)

2. Move away from PSR-0 (Re-Invent the Wheel)

If we were to move away from PSR-0, we could have Drupal\mymodule\MyClass registered at mymodule/lib/MyClass.php.

Pros

• Contrib developers only have to make one directory (lib) instead of three (lib/Drupal/mymodule).

Cons

• No longer PSR-0, since the namespace does not match the directory structure
• Since it's not PSR-0, we'd break any class loader that expects PSR-0
• We'd have to implement our own versions of all the classloaders that use PSR-0

3. Remove the lib directory

Move lib up a directory so that Drupal\mymodule\MyClass is loaded at modules/mymodule/Drupal/mymodule/MyClass.php.

Pros

• One less sub-directory to create
• Easiest

Cons

• Doesn't actualy gain us anything

4. Register the namespace to the directory itself

Move the .module files to where Drupal\mymodule resides.

Pros

• Module classes now live in the same directory as the module
• Is still PSR-0 so we wouldn't need to re-implement the loaders
• Symfony uses this approach with their components by using Composer's target-dir, but unfortunately Drupal 8 won't get this far

Cons

• We don't have control over where users install modules
• Classes in same directory as functional code?
• Broken namespaces when the module has a dash in the name

5. Remove both "lib" and "Drupal"

Remove the "Drupal" prefix in the namespace and move lib up a directory. Register just the module name as the namespace instead of Drupal\my_module. This means that my_module\MyClass is loaded at modules/my_module/my_module/MyClass.php. Could even take it one step further and PascalCase the module names. MyModule/MyClass is found at modules/my_module/MyModule/MyClass.php.

Pros

• Might make it a bit more familiar for people
• Still PSR-0
• PascalCasing the module name makes for pretty namespaces

Cons

• Might conflict with third-party namespaces (symfony module conflicting with Symfony namespace)

6. Remove "lib" and "Drupal" namespace prefix, and register the module's parent directory

Register the module's parent directory as the namespace for the module. This would make it so that my_module/MyClass is found at modules/my_module/MyClass.php.

Pros

• One less directory than #5

Cons

• Result in broken namespaces when the module has dashes in them
• Module name must match namespace
• Requires one module per folder, by design

7. Package-Oriented Autoloader

Beau Simensen pointed me to Package-Oriented Autoloader proposal in the PHP-FIG, which is pretty much PSR-0 without the leading vendor prefix being required in the file system. Allows adding code in src instead of requiring it to be at the top level.

Very nice, and definitely would love to see the proposal grow.

Conclusion

All in all, lib/Drupal/my_module was what we went with because the solution was already done for us. PSR-0 has complete documentation, a plethora of loaders already written for us, and it's all working and tested. Most of the PHP world that deals with loading classes also use it and it was the most bang for our buck. I'd really love for people to understand why we went with PSR-0, and what the benefits/cons to that are.

But, it's always good to see different solutions and alternative methods! There are a couple of different contrib classes that bring this to Drupal 7 and below too: ClassLoader, xautoload, etc. This isn't anything that limits us in Drupal 8 either, we can port the xautoload to Drupal 8 and have alternative means of class loading in contrib. The sky is the limit! Is there a #8 solution that you'd like to see?