“The situation with his father and brother is still very difficult for him,” an unidentified source told People.