“I had genuinely thought I would do one Bond movie, then it would be over. But by then we knew we had a hit on our hands,” actor says