At least for serious non-fiction books, I think academic engagement is a big part of it.

If you want to be taken seriously, you need to fill in all the details and cover all the gaps. Otherwise you open yourself up to attacks. So a lot of the "filler", so to speak, is covering all your bases and making sure you address issues from multiple angles to minimize your attack surface.

But in terms of just getting your big picture idea out to a layperson, yeah usually the introductory chapter is enough to make your point.