I remember seeing a project in Make magazine, or on Adafruit, or Hackaday (one of those), for a modular cell phone that I thought was a great idea. DIY cell phone is certainly doable. Pine phone, as someone else mentioned, is an option which I really hope gains more traction. (but I doubt it will ever become more popular than it already is)
I personally cast my vote for just not using cell phones quite so much. I still rely on my good, old-fashioned, chunky, clunky desktop for all my really important stuff; bills, taxes, passwords, anything with encryption, etc... And that is made all the easier as my eyesight deteriorates (thanks, "smart" phone) and I can't see the tiny little screen.
Desktop PC was my first #1 as a kid. Then when I was eventually able to afford a laptop, after getting over the initial oo-la-la feeling of the fancy new gadget, I found myself still preferring the PC. My phone could never replace that in any truly satisfying way.
That's true for almost all use-cases, too. Whether for gaming, business, hobby, or casual browsing, nothing beats modular, and customizable. Until we find a widely adopted solution like that, I'd say the sleek, fancy, big-brother cell phone is here to stay.