Ir's simple: only those you give them your email or you alrwady have them as personal contacts. You create a base whitelist in your email server config and slowly you build it.

Keep in mind: spam is also coming even from gmail itself. Exampke: you sent an email from your personal domain to a gmail address. Gmail automatically sell that info to data brokers. Your petsonal domain is automatically in hands of data brokers.

Another method is to give free email accounts to your family, friends, close business partners, from your own email server. That way you keep everuthing under control.