Gavin Barwell, who served as chief of staff to Theresa May and now sits in the House of Lords, said there was a “strong case for change”.