But there are also other things to consider. Like, you loose boyancy, and if you misjudge how far you are to the surface and how much you breath out, you might end up 1 meter below the surface thinking "hey, where's the air??" or be forced to make an extra hard kick at the end which MAY kick you into a SWB (i am talking about max-dives, when you are close to your limits at the end of the dive.. i agree that on "normal" dives you probably have so much error-margin that exhale or not exhale makes little difference)
I do this sometimes (i admit, i do not always practice what i preach.. bad me) when doing dynamics with much weights (>3 kg to be aligned in the water), and when i hit the wall i exhale, turn around, and go up all the same time... but sometimes i forget how much weight i have on and sinks when exhaling.. it's only 1-2 meters deep, so no problem, just embarrasing/annoying to misjudge the surface and be another 2 seconds without air (and empty lungs) when you are already tired from the dynamic..
In my opinion it is a matter of judgement. Sure, you might add a meter on your depth, by adding a slightly increased risk for SWB. If you know what you are doing and know your body, then exhale.. But if you are clueless (as stated in the first post) then you better keep the extra safety and use some other technique to add that one meter (cardio training or mental training for example).