The feeling may be misleading. Much more interesting would be seeing and analyzing dive profiles in different conditions. But if you look at the
viscosity table, you can see that the difference between fresh and salt water is quite small (5%) and practically constant over wide range of temperatures. In contrary, already difference of 1-2°C makes for the same difference in viscosity. So from this point of view, if you feel a difference in the glide, it is much more likely that it is due to the temperature difference than due to the salinity.
You would have identical viscosity at sea water of 22.15°C and at freshwater of 20.00°C. Now, the surface temperature is little important, it is more the temperature/depth profile which may significantly change, but here I'd tell that more typical is that the temperature changes faster in lakes. But that's quite individual and often it may be the opposite case.
So yes, if you dive in a warm freshwater lake with a deep thermocline, the viscosity will be lesser (and the sink phase faster) than in cold ocean. But in typical case of a lake with quickly dropping water temperature with depth versus a warm ocean, the viscosity will be actually much lesser (and hence the sink phase faster) in the ocean.
But again, there are so many other factors, that it is hard making any prediction without carefully measuring and evaluating all the conditions: not only salt vs. fresh water density and viscosity, the temperature/depth profiles, and different buoyancy changes, but also vertical and horizontal currents, different turbulence, variable salinity with depth, variable impurities in fresh water having influence on the viscosity too, and perhaps more.
Otherwise, I agree with Eric, that the high viscosity would be rather an advantage on the ascent (an likely in the first meters after immersion too).