In his book, Putting the Truth to Work, Prof. Dan Doriani explains the differences between four classes of legalism:
Class-one legalists are auto-soterists; they declare what one must do in order to obtain God's favor or salvation. The rich young ruler was a class-one legalist.
Class-two legalists declare what good deeds or spiritual disciplines one must perform to retain God's favor and salvation.
Class-three legalists love the law so much they create new laws, laws not found in Scripture, and require submission to them. The Pharisees, who build fences around the law, were class-three legalists.
Class-four legalists avoid these gross errors, but they so accentuate obedience to the law of God that other ideas shrivel up. They reason, 'God has redeemed us at the cost of his Son's life. Now he demands our service in return. He has given us his Spirit and a new nature and has stated his will. With these resources, we obey his law in gratitude for our redemption. This is our duty to God.' In an important way this is true, but class-four legalists dwell on the law of God until they forget the love of God. Worshiping, delighting in, communing with, and conforming to God are forgotten.
(I found the above on my friend Jason's blog)
I think I was a class four legalist for most of my life. The breakthrough in my life occurred when I discovered that the only way to please God, to live a holy life, was to do absolutely nothing at all. And I was freed.