“When people say horrible things about Rob right now, I find it, quite frankly, infuriating and distasteful,” the actor said.