My own experience (probably relatively limited compared to other here - except that I feel the cold a lot...) leads me to suspect the most important thing for warmth is the fit - i.e. wearing something that (as you also mentioned) doesn't let in the water through its edges - as well as not allowing water through seams(/gaps/holes), of course.
When I did my first freedive course I was using the school's own suits (6mm). Also similar for a couple of other freedive trips I did afterwards. In all of those cases I was cold after being in the water for an hour or so - even shivering and sometimes fingers going white (despite the gloves). What I started to notice during these dives was that when I moved around, I could regularly feel the cold water trickling in a bit through various places (especially head, neck & back - despite attached hood).
By the time I did the L2 course I had my own fitted Polosub 5.5mm, with open-cell inner & outer lining, and that made such a huge difference (even though the water was actually a touch cooler than for the L1 course).
Also, I was freediving a little while back using my cheapo 2mm wetsuit (typical surfing type thing), which I thought would be fine for the Med in late Sept. But I feel the cold pretty quickly, and it wasn't enough...
However, I also had my long-sleeve 2mm zone3 top with me, which has lined inner & smoothskin outer (with glued & taped seams - did that myself). It's a little on the tight side, TBH, which is why I'd chosen to use the wetsuit instead. Wearing that top under the wetsuit made a huge difference, of course, but I did also try without the wetsuit over it (i.e. just that top, plus cheap 1.5mm dive&sail bottom) and I was still fine with that - much warmer than the wetsuit.
In conclusion, then, my custom fitted Polosub has open-cell inner & lined outer, while my (tight-fitting) long-sleeve top has lined inner & smoothskin outer (and glued+taped seams). I find either of those to be way (way) better when compared to something of apparently similar (or greater) thickness that doesn't fit quite so well.
Hope that helps!