Thats a really complicated (yet simple) questions.
First it depends on your current aerobic / anaerobic endurance. If these are at the elite level, any apnea training will likely take away from them.. Just like rock-climbing or serious jui-jitsu would take away from endurance training.
If you're apnea training is 'casual', then probably not.
In theory: Apnea training is 'bad' for aerobic fitness (opposite demands) but 'good' for anaerobic fitness (similar low O2 demands)..
Again, it's all relative..
If you're in 'ok' endurance shape (what ever that means), apnea won't really affect it at all (good or bad) in my opinion.
--
It depends a lot on the seriousness of the two..
When I'm in peak freediving shape, everything else suffers.. Why? Becuase I'm focusing on apnea, and neglecting non-apnea. In the 'off-season' I build back my general fitness and my apnea suffers..
The the simple, complicated answer is.. If you take apnea really serious, it will mess with your endurance. If you do it casually and improve just for fun as a side thing.. it wont.