Quantcast
Channel: Issues for Drupal core
Viewing all 294899 articles
Browse latest View live

Serialize file content (base64) to support REST GET/POST/PATCH on file entity

$
0
0

Problem/Motivation

File entities and file/image fields do not currently work with the new REST GET/POST/PATCH/PUT operations.

Proposed resolution

The plan is to include file contents in base64-encoded form when serializing file entities (e.g. for GET requests). File/image field serialization should behave like for entity reference fields in general, providing a URL for the referenced file. Upon deserializing (POST requests), the content will be decoded and put in a new file on the server.

Read about how to POST aka create entities https://www.drupal.org/node/2098511

More documentation about REST: https://www.drupal.org/documentation/modules/rest

Remaining tasks

Review

User interface changes

API changes

Original report by @moshe weitzman


[policy] Decide what the Drupal Public API should be and how to manage it

$
0
0

Updated: Comment #12

Problem/Motivation

There has been, in the past, no way to distinguish between functions, classes, methods, etc. that absolutely cannot change within a Drupal major version, and those that are allowed to change, except:
- Functions starting with _ can change
- Private class methods can change

It would be useful to have a concept of a stable, public, API for Drupal. We would allow things that are not part of this API to change at certain times, and require that things that are part of this API not change.

Proposed resolution

At least broadly, follow http://symfony.com/doc/current/contributing/code/bc.html

This means using the following two tags in core, for both classes/interfaces and individual methods:

@api means 'guaranteed backwards compatibility' - always safe to use the class/method or implement it.

@internal means 'likely to change' - don't use this, or be prepared for upheaval if you do.

Not tagged means we assess the impact vs. disruption of any backwards compatibility breaks, only make them when necessary, and ensure there are clear instructions to update. This means that code that is not tagged is in roughly the same category as 7.x or 8.x beta code now.

Remaining tasks

Agree on whether @api vs. @internal vs. untagged is a good distinction.
Decide what to do about ArrayPIs (mainly form and render structures, but really anything (route definitions etc.) that get passed to alter hooks.
Initially open issues to tag classes/methods as @internal - since this tag can't be applied to code once 8.0.0 is released.

User interface changes

None.

API changes

None. This is a policy, not an API change.

Original report by Crell

In order to make it easier to extend core without breaking APIs, we need to clearly define what our APIs are that we don't break. :-) In practice, there are different levels of "API-ness".

Our traditional approach here has been "a line that begins with 'function' can never change without it being an API break (unless the function begins with _, but then we still don't change it just in case)". That is an inadequate standard.

A common technique is to tag "for reals APIs that we're going to promise won't change" with an @api doc tag. We should do the same. That means deciding how we determine what should get tagged that way.

Standards we should NOT use:

* All public methods (this is needlessly redundant and over-promising)
* All protected methods, for child classes (this is needlessly redundant and over-promising)
* All methods that are part of an interface (again, needlessly redundant)

So, what methods, classes, and interfaces should we document as @api meaning "we promise this won't break unless we have to for security" as opposed to simply part of an interface which means "this is the interface and we're pretty sure it won't break, and we'll try really hard not to break it"?

Discuss.

Allow comments to be attached to entities using a string primary key

$
0
0

According to https://www.drupal.org/node/2100015: This means you can add commenting to any entity type by adding a comments field.

We're building a custom entity type, using a custom storage controller (in essence a REST backend), we can define the comment type and select our custom entity, but when trying to add the comment 'field' it does not show in the field.

Either the CR is wrong in stating that it can be added to any entity type while it isn't, or CR is right and there's something wrong with the code.

Wrong font for Taxonomy field widget label and description in Bartik

$
0
0

In Bartik theme, when creating content with a taxonomy field, the title of the vocabulary is displayed in a different font than other titles than the Title, Body, etc.

Vocabulary: Document minimal requirements

$
0
0

In D6, you check a box to associate a taxonomy term with a content type. In D8 that association is a field that gets added with the content type. Do we intend to make that association in a migration?

Review the hook_help for Internal page cache module

$
0
0

Problem/Motivation

The Internal Page Cache module has been committed as a new module. Some of its functionality might be described in the System module help.

Proposed resolution

Check whether the system module help now includes anything that is handled by Internal Page Cache instead. If so both need to be rewritten.

Remaining tasks

User interface changes

API changes

Views::getApplicableViews() initializes displays during route rebuilding etc.

$
0
0

Problem/Motivation

Steps to reproduce:

1. Load admin/modules
2. drush cr
3. Enable a module
4. Look at profiling data for the form POST and the next page render - see two screenshots attached.

Views::getApplicableViews() runs for route rebuilding, menu link plugins, and menu local task plugins. All of these are global registries/caches that completely block normal page rendering for any other request (i.e. not a single themed request can be served until all items have finished building).

With the standard profile they are not great for performance, and they get significantly worse once you have more views enabled on a site.

Note this is currently worse in HEAD due to #2495073: Views feed display plugin has to get all views data on init - but it makes sense to fix both issues independently of each other, however profiling will see more modest improvements depending on which gets fixed first.

Proposed resolution

Rather than initializing displays, check the config directly instead. This also avoids initializing the plugins which is itself expensive.

Remaining tasks

Get that working and profile before/after to see if it sufficiently fixes the issue.

User interface changes

None.

API changes

None.

views_theme() gets 19 types of plugin definition, only needs five

$
0
0

Problem/Motivation

@todo

Proposed resolution

Only get the plugin definitions for plugins that can legitimately provide theming information.

Remaining tasks

See if this still passes tests.

User interface changes

None.

API changes

Would formalize that particular types of plugins can't provide theme information.

xhprof screenshots with before/after attached. You can see the many less plugin definitions retrieved, however the overall numbers don't seem to improve that much.


[meta-8] Submitting the modules page exceeds 64M with standard profile

$
0
0

Problem/Motivation

Opening this as a child of #2493035: Discover any memory limit issues we have.

When submitting the modules page (no xdebug or xhprof), I see 70M of memory usage. We've just set core's minimum memory limit to 64M after lots of discussion (See #2289201-66: [Meta] Make drupal install and run within reasonable php memory limits so we can reset the memory requirements to lower levels, so opening this as a critical bug.

Proposed resolution

The immediate worst offender I found was field_help(). This takes 800ms wall time on my machine, and 8mb of memory.

In ModuleListForm::buildRow() we check if the module has a hook_help() for the main help path - by executing hook_help() with that path. For field_help() this means listing every field type, widget and formatter - just to render a link. Needs an issue - we can probably move the actual listing to a dedicated route then link to it?

Other bad offenders:

views_theme() - loads lots of plugins. Not a new problem particularly - theme registry rebuilds have always been bad, but grrr. Worth an issue in case we can come up with something.

fileld_system_info_alter() 3.8M - should already have an issue.
filter_system_info_alter() 3M - should already have an issue.

Remaining tasks

Get the child issues RTBC and committed, then check the memory usage again.

User interface changes

Not here.

API changes

Not here.

Migrations need to use the configuration entity dependency system

Add DateFormatter::formatDiff() as a non-buggy alternative to DateFormatter::formatInterval() when the start and end of the interval are known

$
0
0

Problem/Motivation

Currently, the only function that exists in Drupal Core to format date intervals is DateFormatter::formatInterval(). The input to this function is a number of seconds.

This is fine for small intervals, up to a couple of weeks. However, if the interval goes over 30 days, it is not possible to say how many "months" is really being represented, even though the function will return a formatted string containing "months" and "years" -- so it is not accurate.

The reason is that a "month" is not a fixed number of seconds -- for example, it's 30 days between April 10 and May 10, but 31 between May 10 and June 10, but both of those should be formatted as "1 month". Similarly, a "year" is not a fixed number of seconds, because some years are 365 days and some are 366.

Another problem is that the formatInterval() method currently defines a "month" as 30 days and a year as 365 days, which means that there are not 12 months exactly in a year. This leads to "interesting" sequences, such as #2430529: "Member for" date format has a funny way of describing 11 years, and the strange jumps between 11 months, 12 months, and 1 year that you can see around this page in Tracker: https://www.drupal.org/user/124982/track?page=101 (if you don't see it there, try going to a page or two older until you get to the 1 year mark).

Proposed resolution

The only way to make an accurate formatting of a time interval that is longer than a month or so is to know what the start and end date/times are for the interval, so that you can tell whether, for instance, a given number of seconds is the difference between April 10 and May 11 (1 month 1 day) or May 10 and May 11 (1 month) -- these are the same number of seconds, but the output should be different.

The proposed fix is to make a new function that has inputs for the start and end as date/times, rather than a single input of the number of seconds in the difference between them. Specifically, the proposal is to introduce DateFormatter::formatDiff(), which should be used when the start and end date of the interval are known (as opposed to cases where you just know the difference). The function can use the DateTime and DateInterval class provided by PHP to do the work of formatting.

Most (but not all) uses of the formatInterval() function in Core are really representing a certain date/time as "x time ago" or "x time hence" (comparing now and a given date), so most of them can use this new function. However, there are a few uses in Core that are using formatInterval() that do not have two dates that are being compared. For these cases, thin wrappers for "time ago" and "time hence", using the current Requests REQUEST_TIME as the second date, can be used to make the cases explicit. There are discussions on #445414: format_interval doesn't format monthly intervals correctly about whether that function needs to be improved to fix the "12 months is not a year" problem.

Remaining tasks

- Add the new function.
- Add unit test coverage for the new function.
- Replace all occurrences in core where the interval is the difference between two dates, to use the new function.
- Leave occurrences in core where the interval is not a difference between two dates alone - for instance when choosing how often to run cron or something like that and formatInterval() is used to present a list of options, it cannot be changed to the new formatDiff() function.
- Replace instances with the thin wrappers when the second date is implicitly the current Request's REQUEST_TIME.
- Expand test coverage to cover the thin wrappers

- If needed, add web test coverage for the changes in core. However, existing uses of formatItnterval() already have tests; they do not cover cases where the date intervals are greater than 30 days, but since we will have unit test coverage for the new function, that should be OK.
- Review

User interface changes

Formatted "x time ago" and "x time hence" dates should be more accurate. They are currently inaccurate if they are more than 30 days ago/hence.

API changes

A new function DateFormatter::formatDiff() is introduced, which can be used to calculate and format a datetime interval correctly. No changes to the existing API, because there are several places in Core where only an interval (not a start/end date) is provided (like formatting options for choosing how often to run Cron), and you cannot use the new function for those cases, so we need the old function still. Two think wrappers, DateFormatter::formatTimeSince() and DateFormatter::formatTimeUntil(), are also added when the second date is implicitly the time of the current request.

Beta phase evaluation

Reference: https://www.drupal.org/core/beta-changes
Issue categoryBug because date formatting is not accurate as-is for "time ago" formatting more than a month in the past.
Issue priorityNormal: the current implementation is wrong since it doesn't take a start and end date into account, it sometimes returns unexpected results, and core uses this in several places. Not really major though since it only affects time intervals greater than a month, which is a lot of them (when it's used for comment times etc.) but it's not that crucial to get them exactly right and there aren't many cases where they're really pathological.
Prioritized changesPrioritized: The main goal of this issue is fixing a bug in the formatting of date intervals when a start and end date are known and the interval is greater than a month.
DisruptionNone. New API functions are added. Old one remains. New ones are used in Core where appropriate; old one is still needed for a few cases so it is retained.

Configuration translation UI does not support plural sources/targets

$
0
0

Problem/Motivation

Following the introduction to plural sources in #2453761: Views numeric formatter's plural formatting setting incompatible with many languages, config translation needs to support exposing a user interface to display singular/plural sources and plural targets. It does support translating those now so long as users carefully insert EXT (\x03) manually between the variants, which is a no-go in terms of UX.

Proposed resolution

Implement UI similar to TranslateEditForm::buildForm().

Remaining tasks

Implement. Review.

User interface changes

Singular/plural sources would show properly, targets would provide multiple input fields appropriate for the language.

BeforeAfter

API changes

The form element API within config translation may need to change.

Regression test coverage: integration test for an uncacheable menu link that depends on session data

$
0
0

Problem/Motivation

Until #1805054: Cache localized, access filtered, URL resolved, and rendered menu trees landed, the Masquerade module was impossible to get to work correctly in Drupal 8. The Masquerade menu links contain a CSRF token and have a dynamic (uncacheable) menu link text (e.g. Stop masquerading as ). Due to the menu block's incorrect caching, this link wasn't updated dynamically as it should be.

Proposed resolution

Add an integration test covering this particular case, the problem was fixed in #1805054: Cache localized, access filtered, URL resolved, and rendered menu trees, but more integration tests covering complex use cases are of course welcome :)

Remaining tasks

None.

User interface changes

None.

API changes

None.

Original report by andypost

Problem/Motivation

Currently blocks with menus are cached by default, varying only by a few hardcoded cache contexts. This is completely wrong.

@andypost is porting the Masquerade module to Drupal 8. The Masquerade menu links contain a CSRF token and have a dynamic (uncacheable) menu link text (e.g. Stop masquerading as ). Due to the menu block's incorrect caching, this link isn't updated dynamically as it should be.

Proposed resolution

Do not cache menu blocks. Menu blocks will be made cacheable in #1805054: Cache localized, access filtered, URL resolved, and rendered menu trees.

Remaining tasks

None.

User interface changes

None.

API changes

None.

Beta phase evaluation

Reference: https://www.drupal.org/core/beta-changes
Issue categoryTask because it now just adds test coverage.
Issue priorityNormal, because while its important its not majorly important.
Unfrozen changesUnfrozen because it only changes tests.

Ensure #markup is XSS escaped in Renderer::doRender()

$
0
0

Problem/Motivation

Follow-up to #1825952: Turn on twig autoescape by default

If drupal_render produces #markup it is always safe. If, however, the caller sets #markup it is presumed to be safe to make #1825952: Turn on twig autoescape by default reviewable.

Proposed resolution

  1. Run all #markup through SafeMarkup::checkAdminXss.
  2. Add SafeMarkup::replace - identical to str_replace - but if all elements are safe, mark the resultant string safe too
  3. Stop double rendering rest (xml/json) resources

Remaining tasks

Reviews

User interface changes

None

API changes

All #markup goes via SafeMarkup::checkAdminXss
New SafeMarkup::replace

Add condition to ignore PHP 7 on rest requirements check


Remove any un-used external dependencies

$
0
0

There were various points along the way during D8's development that we felt optimistic about our ability to utilize external libraries to solve problems. In general, the workflow was to commit the library first, then file a second issue to actually use it.

We need to check through all of our various external dependencies (Composer, assets, etc.) to ensure we're actually using all of them now. If not, rip 'em out.

Table generated by hand on 2015-05-28:

PHP libraries

LibraryNamespaceUsage
sdboyer/gliphGliph\GraphUnused (though apparently used by https://github.com/larowlan/default_content)
symfony/class-loaderSymfony\Component\ClassLoaderUsed by DrupalKernel for the APC class loader
symfony/consoleSymfony\Component\ConsoleUnused (currently), but expected to be used by #2447573: [meta] Make sure 8.x - 8.x hook_update_N() testing is possible. Also referenced from various symfony/http-kernel tests.
symfony/css-selectorSymfony\Component\CssSelectorUsed by a couple of core tests, as well as behat/mink and symfony/dom-crawler.
symfony/dependency-injectionSymfony\Component\DependencyInjectionYes. :P
symfony/event-dispatcherSymfony\Component\EventDispatcherYes. :P
symfony/http-foundationSymfony\Component\HttpFoundationYes. :P
symfony/http-kernelSymfony\Component\HttpKernelYes. :P
symfony/routingSymfony\Component\HttpKernelYes. :P
symfony/serializerSymfony\Component\SerializerUsed by Serialization module (part of D8's web services).
symfony/validatorSymfony\Component\ValidatorUsed all over the place, via the TypedData system
symfony/processSymfony\Component\ProcessBorderline? Looks like this was initially added in #1784774: Remove Assetic component from core as a requirement of Assetic (which has since been removed). It currently is used, but only in SimpleTest module to prevent executing core's PHPUnit on windows (#1936010: phpUnit fails on Windows).
symfony/yamlSymfony\Component\YamlYes. :P
twig/twigYes. :P
doctrine/commonPresumably a requirement for below...
doctrine/annotationsPHP class annotations
guzzlehttp/guzzleGuzzleHttpUsed by Update Manager, Statistics, Aggregator, HAL.
symfony-cmf/routingYes. :P
easyrdf/easyrdfUsed by RDF module. (Shock!)
phpunit/phpunitYes. :P
zendframework/zend-feedAggregator module
mikey179/vfsStreamorg\bovigo\vfsUsed by a few tests: core/modules/user/tests/src/Unit/PermissionHandlerTest.php, core/tests/Drupal/Tests/Component/Discovery/YamlDiscoveryTest.php, core/tests/Drupal/Tests/Component/DrupalComponentTest.php, core/tests/Drupal/Tests/Component/PhpStorage/PhpStorageTestBase.php, core/tests/Drupal/Tests/Core/File/FileSystemTest.php
stack/builderStackUsed by core/lib/Drupal/Core/DependencyInjection/Compiler/StackedKernelPass.php.
egulias/email-validatorEgulias\EmailValidatorUsed by the core email action, contact module edit form, and update manager settings form for email notifications.
behat/minkBehat\MinkUnused. Used by core/modules/simpletest/src/BrowserTestBase.php, but itself is unused (outside of its test). We are hoping to use it in #2232271: [Meta] Use Behat for validation testing.
behat/mink-goutte-driverBehat\MinkSee above. Requirement for behat/mink.
fabpot/goutteGoutteSee above. Requirement for behat/mink.
masterminds/html5Masterminds\HTML5Borderline. Only seems to be used by core/modules/views/src/Tests/Plugin/StyleTestBase.php. Was added in #2429363: Add HTML5-lib to Drupal 8 core for the filter system and for the testing system for both that and the filter system, but #1333730: [Meta] PHP DOM (libxml2) only understands XHTML4, misinterprets HTML5, but D8 must cope with HTML5 is still active.

JS/CSS Libraries

According to nod_ in #2:

We could remove jquery.cookie library once admin_compact_mode uses localstorage (can't find the issue now… will update when I get it). The rest of the JS stuff is required.

Core profiling scenarios

$
0
0

Opening this to try to get a basic list of core profiling scenarios. This gives people new to profiling have somewhere to start looking, and we can also eventually have an install profile with default content to look at.

The idea of this list is not to simulate a 'real' site, but to look at both fastest, slowest and 'average' core scenarios that are easy to reproduce.

"Install a module"
1. Install standard
2. Log in as user/1
3. Go to admin/modules
4. drush cr
5. Select 'actions' module and submit the form.
6. Look at profiling data for the last two requests - i.e. the form submit and the subsequent rendering of the form.

"Front page as an anonymous user with page caching disabled and warm caches"
1. Install standard (stay logged out).
2. drush pm-uninstall page_cache
3. Refresh front page twice
4. Refresh once more - check profiling data for this request

"Front page as an anonymous user with page caching enabled and warm caches"
1. Install standard (stay logged out)
2. Refresh front page twice
3. Refresh once more - check profiling data for this request

node/1 cool and cooler cache:

1. Install standard and log in as user 1
2. Create two article nodes.
3. drush pm-uninstall page_cache
4. Log out
5. drush cr
6. Visit front page
7. Visit node/1
8. Visit node/2

Profiling data for 6 gives you cold cache performance
Profiling data for 7 gives you 'very cool' cache performance (anything route/entity-type specific is cold)
Profiling data for 8 gives you 'cool' cache performance - anything entity/path specific is cold

[meta] Resolve known performance regressions in Drupal 8

$
0
0

During the 8.x cycle we've introduced several known performance regressions compared to Drupal 7, which we need to resolve before release so that Drupal 8 isn't slower than Drupal 7.

This doesn't mean every single regression needs to be individually optimized away - in some cases it might be necessary to do that, or trying to micro-optimize it won't be worth the extra effort.

However there are places where we've introduced something with the intention that it will allow us to make performance or scalability enhancements elsewhere (like blocks as sub-requests for example), and introducing the performance regression without getting the nice performance feature at the end of it puts us in a not very happy place.

Opening this as a meta-issue to try to track those regressions as they get committed, along with the issues that are attempting to resolve them - making this a critical task since I'm not prepared to release Drupal 8 with obvious performance regressions compared to Drupal 7, it was bad enough from 6 to 7 and we shouldn't do that again.

Criteria for critical performance issues

A performance issue is critical by itself if some of the following are true:

  • There is concrete performance issue identified by profiling (MySQL, PHP or browser equivalent) and a viable plan to resolve it
  • It can't be committed to a patch-level version (8.0.0 => 8.0.1)
  • Over ~100ms or more savings with cold caches and would have to be deferred to a minor version
  • Over ~10ms savings with warm caches and would have to be deferred to 9.x
  • Over ~1ms or more with the internal page cache and would have to be deferred to 9.x
  • Gets measurably worse with lots of contrib modules or large data sets (e.g. non-indexed queries) and would have to be deferred to a minor version
  • Other specific issues at branch maintainer discretion

Working spreadsheet

For up-to-date information on work being done on performance and caching, see the Drupal 8 performance issue spreadsheet.

High priority issues

Several new core APIs have lost optimizations from Drupal 7 and earlier where multiple objects could be loaded with a single request from the database/cache (i.e. compare CMI to variable_init()), the following issues attempt to add some kind of multiple load/pre-loading/CacheCollector support to those systems:

Routes: #2058845: Pre-load non-admin routes (related: menu tree caching #1805054: Cache localized, access filtered, URL resolved, and rendered menu trees)
Configuration objects: #1880766: Preload configuration objects
State lookup: #1786490: Add caching to the state system
Plugin discovery caching: #2114319: Lots of cache requests from plugin discovery
Block context discovery: #2354889: Make block context faster

As a generic performance improvement, there is also a focus on enabling render caching by default for all entities, this is being tracked in the D8 cacheability tag:

Other general performance issues

Use the Performance tag to find any other performance issues. Filtering by 'critical' or 'major' should find the most serious ones.

Original issues where some of the more serious regressions were originally committed

This is an incomplete list, but it would be useful to track specific commits that made performance worse - feel free to add them here. However a lot of performance issues are introduced by new APIs and only become measurable as core is converted to the new API and backwards compatibility layers removed...
#1571632: Convert regional settings to configuration system
#1290694: Provide consistency for attributes and classes arrays provided by template_preprocess()
#1599108: Allow modules to register services and subscriber services (events)
#1535868: Convert all blocks into plugins
#916388: Convert menu links into entities
#636454: Cache tag support (the minimum lifetime removal and also #1848968: Too many checksum tag queries executed by the cache backend)
#1786490: Add caching to the state system
#1272870: No semantics for nested comments / bad for screen-readers
#1696640: Implement API to unify entity properties and fields
#2102777: Allow theme_links to use routes as well as href

[META] Support PHP 7

$
0
0

Problem/Motivation

PHP 7 is likely to be released sometime this year and Drupal 8 is incompatible with PHP 7 in several significant ways. This is critical because:

  • PHP 7 will be released late this year, deprecating all 5.x versions.
  • PHP 5.4 is already in security-fix only, and 5.5 will be as well.
  • We expect at least two years of D8 being the primary stable supported release, plus an LTS after that, and active support for 5.6 will even end before that.
  • Drupal 8 is MUCH faster on PHP 7, with performance gains that would qualify for the Critical performance issue criteria.
  • Drupal 8 is incompatible with PHP 7 in ways that will require BC breaks to significant public APIs, and we have no way of preserving BC while also supporting PHP 7, making it infeasible to add PHP 7 support before Drupal 9 if we don't make these key changes before releasing Drupal 8.

See:

Proposed resolution

Fix any obvious things that have been accepted into PHP 7 and are already broken in Drupal 8.

Remaining tasks

Fix the child issues:

Fixed

Menu rebuild is broken

$
0
0

Problem/Motivation

If you have menu links on multiple levels they will fall to the first level on cache rebuild. This happens only for the menu links that are defined to link for a url. Issue doesn't exist for menu links linking for a node where autocomplete functionality has been used.

Proposed resolution

This seems to be caused by this piece of code in \Drupal\Core\Menu\MenuTreeStorage:

protected function saveRecursive($id, &$children, &$links) {
    if (!empty($links[$id]['parent']) && empty($links[$links[$id]['parent']])) {
      // Invalid parent ID, so remove it.
      $links[$id]['parent'] = '';
    }
}

Remaining tasks

-

User interface changes

-

API changes

-

Viewing all 294899 articles
Browse latest View live


<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>