Kerala's Left-led government today decided to remove Governor Arif Mohammad Khan as Chancellor of universities in a sharply escalating confrontation that has been building up for days.