CLI entry point in Drupal Core (Dex)

dex module:command --options

This is a proposal for a common entry point for CLI commands relying on a booted Drupal install.

This issue comes out of many years of maintaining a slim console, and building the console work for the SM project. Now the hope is the skeleton can move to core.

Summary

I'd like to propose a new slim console initiative. All about getting a minimal viable console entrypoint which first and foremost an easy way for Drupal modules to introduce CLI commands.

This is a continuation of the discussions at the decade old #2242947: Integrate Symfony Console component to natively support command line operations. I've read through the remarks and address points raised below.

Scope/Proposal

• To add a single entry point of entry, for which users can run commands.
• For module developers to be able to add commands in a consistent manner.
• Only commands defined by enabled modules/extensions.

Exclusions/Out of scope

Mostly since its been proven these things have caused other issues to stall.

• Anything supporting a non-booted site. This could be addressed in the future. Projects like Drush can continue to work, and can even make use of the new commands.
• Conversion of existing core scripts.
• Adding commands to core itself.
• A base/abstract command. The Symfony Console class is recommended.

Addressing the original Symfony Console POC

The discussion at #2242947: Integrate Symfony Console component to natively support command line operations is certainly valuable, however is long in the tooth at 10 years old. Discussions indicate (Comments #129 / #131 / #193 / #207) a preference to close out in favour of a newer issue which will provide commands for core. Notably, that issue also has stalled.

This issue is designed to replace #2242947: Integrate Symfony Console component to natively support command line operations in its entirely. With respect and acknowledgement of the discussion there.

Addressing the Issue summary of the original Symfony Console POC

This CLI integration should be viewed as a separate application which ships with Drupal. This might include going so far as to turn it into a subtree split similar to Drupal components.

Since this proposal is so small, the main systems are included. If others, including Drush, want to recycle commands. They certainly can since the command loader system is reusable.

Some commands will need to run without an installed Drupal, or even without a booted kernel. Different boot levels in functional vs. kernel vs. unit tests is a useful metaphor for what is needed.

The proposal here is simpler, requiring a fully booted install.

We use a separate service discovery container as a place to resolve dependencies and commands. This is independent of the Drupal container because it's not a Drupal, it's a console application that happens to use some Drupal as needed.

This proposal would use the same container, and thusly any and all services a booted Drupal install would have access to.

We use symfony/finder to locate extensions and find service definition files

This proposal does not need any new extension/module handling. The closest thing to this would be the proposed plugin-directory auto service registration.

Discussions about core commands

#2242947: Integrate Symfony Console component to natively support command line operations had plenty of discussion about how and where core would include commands, eventually leading to #3422359: Directory based automatic service creation. I'd like to draw the line and not consider any core commands, other than test-only commands. The proposal here is first and foremost a benefit for contrib. Anything more can be handled in the future.

Proposed name

Not going to bikeshed too much with a name, so putting it out there:

dex

Rationale

• Origin story, at least for now, its for Drupal EXtensions (modules, themes, profiles), as in: it requires a booted Drupal install. "Drupal" itself is more ideal, however this namespace is taken, for now. And makes sense to reserve that name for an all-encompassing version supporting non-booted installs.
• It's easy to type, and memorable.
• Its Searchable (Googleable), much like drush, and unlike drupal.

Technical proposal

Core implementation

• Provide an entry point bin file, which is then wrapped by symfony/runtime.
  • Includes just enough information to synthesise HTTP requests / base URL. (Like Drush's uri option)
  • Brings in command loader, and legacy commands (uses configure() instead of Attributes).
• composer.json changes:
  • Add symfony/runtime , so we dont need to deal with low level / bootstrap / autoloader.
  • Adds bin file so it can be placed in vendor/bin, available in PATH
• Add a console.command_loader service to core; the registry of commands.
• Adds a compiler pass to core.
  • Which scans src/Command/ directories in enabled extensions for commands, and creates tagged command services from them.
  • Adds tagged command services to the console.command_loader service.
• Associated tests.

Command Loader

The main meat of registering commands to the command loader to be the same, or as close to, how Symfony does it in \Symfony\Component\Console\DependencyInjection\AddConsoleCommandPass (Code)

Runtime

The proposal here relies on symfony/runtime, which we've also discussed at #3313404: Use symfony/runtime for less bespoke bootstrap/compatibility with varied runtime environments. I dont think we need to block on either of these issues. In fact we can preempt the runtime issue entirely, since we are providing a real use case with real motivation. It may be used in the future for non-cli purposes. Interestingly, there seems to be some community interest in symfony/runtime, as after 18 months, the runtime issue has 40 followers already. Despite little discussion between 4 people.

The maturity of symfony/runtime proves we can implement this idea without needing to invent new things for Drupal.

Testing is simplified, the surface of our testing can focus on the integration since we are not maintaining significant amounts of new code.

Expected UX

Since runtime sets up linking to a projects'bin directory, if you have it in your PATH, then the expected UX is:

dex my:command --blah --foo=bar

Expected DX

• Developers create a class in src/Command/. Such that namespace is Drupal\mymodule\Command
• Class extends \Symfony\Component\Console\Command\Command, as required by AddConsoleCommandPass.
• Class has attribute #[\Symfony\Component\Console\Attribute\AsCommand]
• The class must implement only one method: protected function execute(), featuring the necessary custom logic for the command.
• The command has access to the container for autowiring.
• Discovery is as simple as a container reset (cache clear).
• There is no requirement for a services.yml entry, though advanced 1% use cases can do so.

The expected code is concise:

declare(strict_types=1);

namespace Drupal\dex_test\Command;

use Drupal\Component\Datetime\TimeInterface;
use Symfony\Component\Console\Attribute\AsCommand;
use Symfony\Component\Console\Command\Command;
use Symfony\Component\Console\Input\InputInterface;
use Symfony\Component\Console\Output\OutputInterface;
use Symfony\Component\Console\Style\SymfonyStyle;

#[AsCommand(name: 'example:command', description: 'An example command.')]
final class DexExampleCommand extends Command {

  public function __construct(
    private readonly TimeInterface $dateTime,
  ) {
    parent::__construct();
  }

  protected function execute(InputInterface $input, OutputInterface $output): int {
    $io = new SymfonyStyle($input, $output);

    $now = new \DateTimeImmutable('@' . $this->dateTime->getRequestTime());
    $io->note('The current time is ' . $now->format('r'));

    return static::SUCCESS;
  }

}

Drush, Core Commands, Future Non-Booted version

Drush may continue to exist for now as it provides use cases outside of a booted Drupal installation, including code generation, install, etc. In fact Drush may choose to provide a layer to register the commands we're adding here, to Drush itself.

Perhaps a future all-encompassing Drush replacement would utilise the drupal namespace and cleanly replace dex.

#3089277: Provide core CLI commands for the most common features of drush is a good place for working towards core commands or non bootstrap commands. Which would not block or be postponed on the issue here.

MR notes

• My vision for this issue is not too much larger than the companion MR, please take a look.
• Very basic env var pieces are included in the Dex command in order to resolve issues with not being in web requests.
• File system auto discovery is limited to Drupal extensions, not vendor. Though if a Drupal extension wants to register a command located in vendor, they can via manually tagged service definitions instead of #[AsCommand]. vendor doesn't participate in autodiscovery for modules, either.
• Dependencies are provided by autowiring. If aliases don’t exist for dependencies, they can be defined in the custom project, create a patch in the service-defining project, or ultimately can fall back to defining a command in services.yml

Other notes

• Existing commands, such as those from Config Split or Tome, those commands get registered automatically.
• This wouldn’t be a dev only console. Commands can opt to make themselves hidden using compiler-passes, or env-based conditionals, if they want to prevent execution on production.
• If this issue progresses to commit, please also considering crediting those with significant participation in #2242947: Integrate Symfony Console component to natively support command line operations.

Contrib Project

I have created an intentionally short lived project that gives contrib all its needs for command execution. It contains just the pieces needed to evaluate the UX/DX of the MR for this issue. And to avoid the inability to have runtime set things up when used as a patch, since patches can’t modify composer.json until after Composer does its thing. Unless of course you apply the patch to a branch of some kind and require it.

https://www.drupal.org/project/dex_console