Yes you can but at least a little prep is needed. The door needs to be clean. A good way of doing this is to use a liquid sander/deglosser, I like Paso but this might not be available in your area. For the primer use an oil base that will dry quickly.
Adhesion will be improved if you can do a light sanding.