I'm not sure the premise is right. Most of the libertarians I know, or am familiar with, are married.

Setting that aside, The Righteous Mind might shed some light on the issue. Libertarians, who are almost entirely men, have different moral sensibilities than everyone else and likely aren't good at relating to non-libertarian sensibilities.

Conservative women would be able to understand their libertarian husbands, but progressive women likely couldn't, so there might just be fewer potential matches for libertarian men.