Having had many dives with sambas and several blackouts, including one samba on a recreational dive in 2002, I can say first hand what happens in these situations.
Even for someone with great experience with hypoxia, you will never realize that you are in trouble until you reach a shallow depth. If you really overdid it, you may realize you are in trouble at a depth of around 15m, but usually you'll realize it around 10m or less. Once you realize that you are in trouble, you have about 1 second of useful consciousness, which is not enough to do any task. Once that one second elapses, the only thing you will do is continue kicking towards the surface, in a virtually unconscious state.
Even if taking the mask off had an oxygen saving effect, this oxygen saving effect would have be IN EFFECT for at least 10-20 seconds of dive time to actually have a noticeably positive effect. But, you need to SUBTRACT the energy of actually removing the mask (and holding it hard clenched in one hand, which is what you would likely do). Not to mention the total loss of velocity caused by the drag of doing it. In my opinion it would be detrimental.
I did a 'dumb' solo freedive in 1998 to 31m with a big weight belt and bifins. On the ascent, I realized I was in trouble around 15m, and my last clear thought was 'I should ditch my weight belt.' The next thing I knew, I was at the surface and my vision was gradually returning. I was still wearing the weight belt.
By 2002, I was much smarter. After having already set a WR and having had blackouts at the end of dives, I realized what I should really be thinking when in trouble. I did a shallow dive on a wreck (23m) for a very short total dive time (1'30"), but my recovery had been far too short from the previous dive. On the ascent I suddenly realized I was in trouble around 15m. I knew I had one second of useful consciousness. I used that one second to tell myself one thing: Remember to breathe upon surfacing!! I reached the surface in a virtually unconscious state, but I took a deep breath, and had a big samba. My buddy was 30m away pointing in a different direction. If I had tried to ditch my weight belt, I would probably be dead.
In my mind, the only time it is conceivable to ditch the weight belt with benefit (or remove the mask), would be if you got stuck in something on the bottom (wreck, fishing line, etc.), and you overstayed your bottom time and realized you had been on the bottom for WAY too long. Even before beginning the ascent, you decide that you must ditch your belt and remove your mask. You don't lose any momentum or cause any drag, because you aren't even moving yet.