This is a really good article on Selfish Mining and related behavior: https://bitcoinmagazine.com/business/why-bitcoin-mining-pools-aren-t-incentivized-to-broadcast-blocks-quickly-1475249510

I suspect it wouldn't have an influence on difficulty because you're not finding fewer blocks than you would without selfish mining - they just aren't revealed as early.