What Irishdiver said.
A very wise and experienced diver once said something like "Short surface intervals are a deal with the devil" I once played rescuer for a diver that ignored that wisdom. BO is your danger, longterm damage is not. However, serial dives with too short surface intervals will lead you to BO.
That said, if you are diving fairly easy, shallow, and short and especially half lung, less that twice the dive time is fine. For intense dives, its insufficient. I'm doing serial dives, dive half lung and, for more intense dives, find 2.5-3 times dive time(sometimes more) is more comfortable. For short, easy and fairly shallow, 1.5 times dive time, or even less, seems fine for me.
Its hard to sit on the surface waiting, but you will find that longer surface intervals pay off in longer, more comfortable dives.