“I can’t be one of those people. I can’t just go on TV and say what John meant to me. It was just too deep. It’s just too much,” he remembered in a recent interview