This is just my personal opinion, but I have two criticisms:
  1. It rests on faulty assumptions about the possibility of moral/ethical neutrality which can be somehow determined scientifically. In practice, it is a tool for moving power into the hands of those with "expert" credentials to determine what is good policy and what is not.
  2. It is too easily used as a veneer for people to hide their selfish behavior / selfish intentions behind public acts of charity.
It reminds me of Jesus's saying in the book of Matthew:
Beware of practicing your righteousness before other people in order to be seen by them, for then you will have no reward from your Father who is in heaven. Thus, when you give to the needy, sound no trumpet before you, as the hypocrites do in the synagogues and in the streets, that they may be praised by others.
Oftentimes, the veneer of virtue that charity imparts can actually lead people to worser behavior in their private lives. I've heard this first hand from some friends who work in the non-profit/NGO space. (Though this is more a criticism of any kind of non-profit/charity/activism, and not necessarily a criticism of effective altruism per se.)