The mathematicians who participated had to sign a nondisclosure agreement requiring them to communicate solely via the messaging app Signal. Other forms of contact, such as traditional e-mail, could potentially be scanned by an LLM and inadvertently train it, thereby contaminating the dataset.

I find myself prompting LLMs with ever-sloppier grammar and punctuation that I never would have used before (even incomplete sentences that don't make sense)

“When an inscrutable technology becomes an invisible technology, we would be wise to be concerned. At that point, the technology's assumptions and intentions have infiltrated our own desires and actions. We no longer know whether the software is aiding us or controlling us. We're behind the wheel, but we can't be sure who's driving.”

“If we’re not careful, the automation of mental labor, by changing the nature and focus of intellectual endeavor, may end up eroding one of the foundations of culture itself: our desire to understand the world. Predictive algorithms may be supernaturally skilled at discovering correlations, but they’re indifferent to the underlying causes of traits and phenomena. Yet it’s the deciphering of causation—the meticulous untangling of how and why things work the way they do—that extends the reach of human understanding and ultimately gives meaning to our search for knowledge. If we come to see automated calculations of probability as sufficient for our professional and social purposes, we risk losing or at least weakening our desire and motivation to seek explanations, to venture down the circuitous paths that lead toward wisdom and wonder. Why bother, if a computer can spit out “the answer” in a millisecond or two? In his 1947 essay “Rationalism in Politics,” the British philosopher Michael Oakeshott provided a vivid description of the modern rationalist: “His mind has no atmosphere, no changes of season and temperature; his intellectual processes, so far as possible, are insulated from all external influence and go on in the void.” The rationalist has no concern for culture or history; he neither cultivates nor displays a personal perspective. His thinking is notable only for “the rapidity with which he reduces the tangle and variety of experience” into “a formula.”