GenAI as a System of Cognitive Flow and Power

Text written by Eliott, no GenAI use. Only the images were created using a GenAI tool (ChatGPT). 

I’m an assistant professor of CS: I’m an AI researcher, teach AI and RL (Reinforcement Learning), and am currently serving on two AI committees (one geared towards pedagogy). 

I rarely spend more than three hours of my day without getting in touch with the “AI” word. Sure, AI is a big umbrella term that is often underrecognized and currently used interchangeably with GenAI (it pains me, but I learned to accept it). 

Since the “GenAI” word occupies my sentences quite often, I thought I should share how I interpret it. Below, I present my current interpretation of “GenAI” systems and, as my philosophy hat constantly makes me question everything, that interpretation is subject to change. 

It is common to connect GenAI to a tool and restrain related interpretations to utilitarian uses of tools like ClaudeAI, Gemini, or ChatGPT. However, my strong background in philosophy and cognitive science prevents me from seeing those just as “tools”, but rather as systems of cognitive flow and power. 

Yes, I’m very excited about GenAI tools, and yes, I use them often. Still, depending on a user’s interaction patterns (mine included): 

1. Our ideas, problem-solving templates, and worldview can become fuel for the infrastructure underneath those tools. Cultural embeddedness and identities can be apprehended and modeled. 
2. All of those, together, can help create a vision, a cognitive map of the world, and how to influence people and players in a way that is hard to beat. 
3. Those are just a few of the elements that can provide intelligence about a country’s way of thinking, solving problems, vulnerabilities, and taboos, ultimately posing geopolitical unfolding. 
4. Considering a widespread use of these systems and possible uniformization and sanitization caused by continuous user-tool interaction, I fear producing exploitable users, especially the ones who grow using these technologies, which can be mapped back to geopolitical implications (in the sense of people weakening their diversity of thought, e.g., uniformizing writing and problem-solving, leading to easier-to-map vulnerabilities and users more subject to exploitation techniques, such as manipulation.) 

In the end, I invite people to be more curious about both, a) cognitive shaping (yes, Skinner as intuition fits well here) and b) how cognitive flow/leakage may travel and be uncovered/repackaged via GenAI systems (which may be worsened depending on the tools you use). Note: If you are an RL person and forgot to read the RL book, chapter 14, now may be a good time to do so. 

Ultimately, my way of dealing with these concerns currently translates into staying informed and designing a course for students and interested teaching colleagues, both within a main umbrella of a Humanities Conversation I launched this spring: Untraining Predictability: A Humanistic Approach to GenAI.  

If you reference or adapt this framing of GenAI systems, please credit Fernanda Eliott, AI Aura.