While the decision to remove Partha Chatterjee as minister will be taken by CM Mamata Banerjee, the party may also take a decision on removing him from the party's Secretary General post.