I agree with you that as soon as you are conscientious, careful and pay attention then you are fine and the system works for you. But the numbers don't lie and what you are saying does not apply to majority of americans: Credit card debt explodes at 2nd fastest pace on record.
Now specifically for credit score, here are the factors that go into it: .
Different factors have different "weight" in the final score, the weight depends on which company do you use for the score. Here are the rough weights: