I think it's common for many people to start with something else and then learn Tai Chi later in life  - for many reasons. I heard that the Yang kids were first taught external martial arts - very physical punching and kicking - something like 'Chang Chuan' which they did until they had built a solid foundation. 

cristaiji

alter_native

Tai Chi as a Martial Art ? 🤔 - Part 1

I also started out with karate and judo, when I was little.

When I took Tai Chi classes in college, I had already been studying Wushu for a couple of years. There were a couple of students from the Tai Chi class who joined the Wushu group and sparring with them was really interesting, because their movements were so subtle and their balance was excellent.