I too have found that, and find staying down shallower than about 4-5m difficult without being overweighted/swamped at the surface. I would say I'm neutral around 6m.
Which isn't too bad for people who head for that kind of depth most dives. For someone like myself who stays pretty shallow all of the time, it's a pain.
This summer I'm going to experiment with having an additional hand weight (kettlebell style) to hold on to on the sea bed. Would be interested in hearing if this has been successful for anybody else.