I suspect this is only a theme among investors that have never been founders or even operators in a serious way (outside of other money moving schemes). Are you talking to investors that were founders?

at what point does holding someone's technical background against them, without clear justification, become a form of prejudice?

That's literally the definition of prejudice.