@denlillaapan mentioned here that he'd be interested in seeing a writeup on what's known as the "Secretary Problem". Turns out, it's been discussed on SN before, and it's often sold as some "mathematical rule for life decisions".... but after studying up on it I think it's a bit sillier and less applicable than most people realize. So here's my writeup on the subject.

The Secretary Problem, as it's known, is a puzzle in probabilistic optimization theory. Understanding the setup will help us understand how useful it is in real life. The setup goes like this:

• You want to hire the best secretary out of N possible candidates.
• You don't know their quality levels until you interview them.
• You can only interview candidates one at a time.
• Once you interview a candidate, you gain sufficient information to rank them among all previously interviewed candidates, though you still don't know the rankings of the un-interviewed candidates.
• [Unrealistic Assumption #1] After you interview a candidate, you must immediately hire them or not... you can't wait to interview more candidates.
• [Unrealistic Assumption #2] If you reject them, you can't go back to them again later.
• [Unrealistic Assumption #3] The only measure of success is hiring the absolute best out of the N candidates. Hiring the second best vs. hiring the third best are considered equal failures.

Note that since your only goal is to hire the absolute best, you'd only ever hire an interviewee if they're the best you've seen so far. The basic tradeoff is this:

• If I hire them now, I risk the possibility that the best one would've come later.
• If I reject them now, I risk the possibility that they were actually the best.

The math is a bit tricky, but it turns out that the optimal decision rule is (approximately) to always reject the first 37% of candidates, then hire the first candidate afterwards who ranks better than all the previous.

The idea behind the solution is that it's too early to hire the first 37% you interview. The probability that the best candidate is among the remaining applicants is still too high.

In popular media

The Secretary Problem has been used to describe search processes where you have to go through candidates one by one, like dating, house hunting, and job interviews. It has achieved popular attention because of the seemingly ruthless rule of automatically rejecting the first 37%. Especially when applied to dating, it would suggest that you should never marry your first few boyfriends/girlfriends, because the probability that the best match is yet to come is still too high.

Critiques

The problem with applying this to real life is that some of the assumptions are unrealistic. The worst culprit is Unrealistic Assumption #3. The 37% rule is only optimal if you literally do not care about anything other than finding the perfect match. The problem is set up in such a way that ending up with 2nd best is equally bad as ending up with the worst candidate. There is no such scenario in real life where this is true, so the 37% rule is already not very applicable.

Unrealistic Assumptions #1 and #2 may be true in dating. You often can't go back to a bf/gf that you've previously broken up with, and it's usually tough to explore multiple relationships at the same time. However, in job interviews and house hunting, there are usually no such restrictions... there's just some risk that if you wait too long, the candidate is no longer available. But that's a far cry from saying you have to immediately accept or reject a candidate after interviewing it.

One other thing I'll add is this. The problem setup assumes you know what N is. That is, you now exactly how many candidates there are going to be. In most real life scenarios, like dating, you don't know that. If you break up with your current partner, you don't know for sure another candidate is even going to arrive.

Oh, and one more unrealistic thing is that the information you get when you interview is purely ordinal, meaning it lets you rank the candidates, but it doesn't tell you an actual measure of the quality of whether this person can do the job and how well. In most realistic settings, we care less about ranking amongst the candidates, and more about some absolute level of quality that we're looking for.

Conclusion

It's a fun statistical puzzle, but the applicability to real life is way overstated. One of those things where people hear a counterintuitive brutal strategy---"auTOmAtiCaLLy rEJECt thE FIrsT 37% oF APPLICanTs!"---and make much hay of it.... but yeah honestly you shouldn't do that in real life.