It's a problem because third parties and counter parties can view your transaction history and follow it going forward. Not only bad for security and personal safety, but bad for fungibility in general.

Another side affect is that targeted mining censorship becomes possible, while if Bitcoin were private, to censor a single transaction they would have to censor them all indefinitely. Much more difficult and expensive to do.