Based on specific user actions, the article photo captions can "survive" article deletion, and prevent images to be purged from the system.
This happens when users remove a photocaption element from an article, without deleting (spiking) it, as described in Add or remove article elements.
As this is a global configurable action, you can change the way it works and avoid orphan elements. See Global variable for spiking removed article elements for instructions.
See also
Getting rid of orphan article elements after articles were purged