Charles has been a controversial royal over the years, from his affair while married to Princess Diana, and alleged persistent political interference, to occasional gaffes and scandals involving aides.