Australia, the United States, the United Kingdom and Canada will "pay the price" for a diplomatic boycott of the Beijing 2022 Winter Olympic Games, a spokesperson for the Chinese government says.