The British dancer, 78, joined the series as a judge during its very first season back in 2005 and has decided to leave to spend more time with his family in the UK.