I've never had it happen, but my ruling would be Yes. You lose whatever ability you gained from the increase.
I also track the abilities generated at start-up according to which level of Bloodline...