This is related to #592800: Dependent modules are still installed when required modules return errors in hook_requirements('install'), which appears to have been fixed.
Here's what happens:
1. Enable module A in the module admin
2. Module A depends on module B: get taken to the 'do you want to enable module B?' confirmation
3. Module A's hook_requirements() then reports a problem.
4. The user is returned to the module admin page, with the error message from hook_requirements(). Module A is not enabled, however module B *is*.
The expected result would be to return the system to how it was before the attempted operation was begun.