I've tried enabling the config_translation module and now I can translate field labels for node types and terms.
But I have a custom entity and the fields I have added in the field UI (those starting with field_) are not translatable.
Ironically, the base fields in my custom entity are translatable.