I have the same issue, only one comment was outlawed and the tag still there, even if I disabled the moderation. I think that orange pill could scary and block people from posting interesting content. Maybe @k00b or @ek can do something in this regard?

Another thing that I noticed is that comments outlawed by admins or bot aren't included in the orange pill.