For a quick gaming session I tend to use RetroArch, but if I really enjoy a game I often have a cartridge or disk for it so I can experience it on original hardware. I've got a small computer collection of era correct hardware that I keep running so I can play some of the older Windows games. It's sometimes a lot of effort, but a lot of fun when Windows 98 or XP boots up again!
That Atari setup would be nice to have! I wanted to get something in their 8-bit lineup but I never have. I do have a Commodore 64 though!