A larger UTXO set does push up the minimum chainstate a pruned node needs to keep around, and more UTXOs make it (slightly) slower to load UTXOs from disk, while also requiring the UTXO cache to be flushed slightly more often. It is more of a graceful degradation, though.