An abusive sometimes angry critique of the victim's actions or work. Such a painful and embarrassing evaluation that it feels like the authority is trying to tear a new asshole for you. (Assholes being an embarrassing body area.)
Secondarily, tear a new asshole may refer to total domination by an opponent similar to a gay rape where the attacker didn't even bother to find to the actual asshole.
The drunken male marines performance on the whore house obstacle course was so pitiful that their sergeant had to tear a new asshole for all of them with a rusty pocket knife. But he praised the females for being righteous Ho's on the beach.