Sometimes other modules makes markup that rises javascript to "double-trigger" the contextual links popup event. Which leads that the contextual links popup does not appear. For example it is happening when using Panels (https://www.drupal.org/node/2743353).
The simple patch below fixes the problem.