A structural proof that skips the case-splitting: every prime $p > 3$ is coprime to 24, so $p$ is a unit in $\mathbb{Z}/24\mathbb{Z}$. The unit group $U(24) = {\pm1, \pm5, \pm7, \pm11}$ squares down to four checks: $1^2 = 1$, $5^2 = 25 \equiv 1$, $7^2 = 49 \equiv 1$, $11^2 = 121 \equiv 1 \pmod{24}$ — and $(-a)^2 = a^2$, so all eight units square to 1. Hence $p^2 \equiv 1 \pmod{24}$ for every prime $p > 3$. The whole theorem is just the statement that $U(24)$ has exponent 2.
A structural proof that skips the case-splitting: every prime $p > 3$ is coprime to 24, so $p$ is a unit in $\mathbb{Z}/24\mathbb{Z}$. The unit group $U(24) = {\pm1, \pm5, \pm7, \pm11}$ squares down to four checks: $1^2 = 1$, $5^2 = 25 \equiv 1$, $7^2 = 49 \equiv 1$, $11^2 = 121 \equiv 1 \pmod{24}$ — and $(-a)^2 = a^2$, so all eight units square to 1. Hence $p^2 \equiv 1 \pmod{24}$ for every prime $p > 3$. The whole theorem is just the statement that $U(24)$ has exponent 2.