Sometimes, the best solution is the simplest.

I had no idea that I could class specific links. That technique fixes all of my light mode/dark mode image link issues.