Probably because evaluating the value of a promissory note is a very hard thing to do -- it can be risky.

If I know you well, maybe a promissory note suffices. I once lent my brother some money with such an arrangement.

The whole complicated banking system thing also probably exists to diffuse the risk. One loan to one individual is highly vulnerable to bad luck -- if something goes wrong, you get 100% of the downside.

With a third party sitting in the middle, any particular loan going bad doesn't spell doom. It seems similar to how insurance works.

I agree that banks do shitty things and get fat sucking off the teat of gov't, but to me this is a problem of over regulation, too big to fail, fdic, and Cantillon effects rather than a root problem of fractional reserve banking.