Who decides which words are "curse words" and which aren't?

I find it so strange that we police the manner in which another person moves their facial muscles to make a certain sound. Thousands of possible sounds we can make with our mouths, and for some reason a few of them are considered "foul".