[Clielldub]

I don't think you are doing anything wrong here. Your handicap is steadily decreasing as you play more golf, which is normal. I'd just keep playing golf and let the other people think what they want. As long as you know your not a sandbagger that's really all that matters.