We need way more merchants, services, and exchanges onboard.
Beyond that I think the biggest challenges are channel 'liquidity' for onboarding users and finding a way for LSPs to help allow users to receive payments offline in a non-custodial manner that is also somewhat private.