But when he stabs Karin at the LOI he does cross the threshold of Villainy. At that point he loses what defenses he had before. And on top of that he also attacked B to help a terrorist organization. Then much much later he tells Naruto that his plan to FORCE every single Shinobi village to unite under him as their supreme leader. Essentially fixing the system by stifling the freedom of everyone in the universe whether they like it or not. That is not a Heroic thing to do, that is a villainous thing to do. It's essentially World Domination.