Lots of AI products will ex-filtrate your data. This one seems to be accidental though.

These types of attacks have been shown as possible in many different prominent applications, such as Microsoft Copilot, Google Bard, etc.