"Reflexive AI usage is now a baseline expectation at Shopify" - Shopify's CEO Tobi Lütke

Understanding AI capabilities

• What AI is good at:
  • Creating new projects from zero to working state blazing fast.
  • Writing documentations that are comprehensive and look good.
• What AI struggles with:
  • Making sure the software it produces actually work.
  • Saying no or questioning requests with wrong assumptions or logic.
  • Creating repositories and implementing code that is secure.

What I have tried so far

• Autocompletion
  • The least intrusive of all approaches
  • Speeds up development without sacrificing control over the code produced
  • A few examples of tools where this is available: VS Code, Cursor, JetBrains products, etc.
• Composer-driven development agent
  • A middle ground of autocompletion and autonomous development agents
  • The agent plans and applies changes across multiple files to satisfy a request by the user speeding up the development process tremendously
  • Most of the problems with AI coding start here because the agents are so fast to produce code the developer will feel the desire of accepting all changes proposed without fully understanding and vetting the code
  • A few examples of tools where this is available: Cursor, Replit, etc.
• Autonomous development agent
  • Also known as vibe coding where the person cooperating with AI relinquishes all responsibility over the code
  • This approach is similar to the one above where the user will provide the model with prompts to make changes to the application being built, the difference is that there is no code review or approval of the changes
  • Tools like Replit will take care of the deployment, databases, authentication, websockets, etc, so the user never have to do any development/operation low level activity

Ideas worth trying

• Locked spec, but lose implementation
  • One approach I've been experimenting with composer-driven development agent is to provide a specification (unit test, gherkin, etc) and to tell the AI it has to make the test pass without changing it, so the AI has freedom to change the behavior as long as the other specs still pass and until the behavior in hand passes the test
• Reflexive AI usage
  • It refers to how people (or organizations) use AI while being aware of how that AI is affecting their thinking, behavior, or decisions, and then adjusting their usage based on that awareness. In practical terms, it means:
    • Not blindly trusting AI outputs. You question the response, double-check facts, and consider bias or limitations.
    • Recognizing how AI shapes your behavior. For example, if you're relying on AI for writing, you might notice you're starting to lose your edge in crafting sentences or arguments. Reflexive use means catching that and adjusting.
    • Monitoring overdependence. You notice if you're using AI as a crutch instead of a tool, and you deliberately shift back to using your own expertise when needed.

Conclusion