Wisdom and AI

How to approach AI wisely, and what wisdom traditions offer.

8 min read

This is a Second Renaissance research project led by Rufus Pollock of Life Itself. It has been developing over several years, and more intensively over the last 6–12 months. We're releasing it bit by bit.

First up are the questions: one governing question, four core questions and the sub-questions under each.

By "wisdom traditions" we mean especially Buddhism, because it is the one we know best. We are drawing on other traditions as much as we can, and suggestions are warmly welcome.

At a glance

How should we approach the development of AI wisely, and what do wisdom traditions offer?

  1. Alignment: can we align AI, and what role can wisdom traditions play?
    • What would it mean for AI to be aligned, or good?
    • How is it attempted today, and does it work?
    • Is the target (goodness or wisdom) the kind of thing that can be specified or trained?
    • What do wisdom traditions offer?
    • So: can we align AI, and what is the realistic role of wisdom traditions?
  2. Risk and pace: given existential risk and the chance alignment fails, how should we approach AI development, and what do wisdom traditions suggest?
    • What are the catastrophic and existential risks?
    • Why is the world racing anyway?
    • Should we change pace or trajectory, and can we?
    • What do wisdom traditions offer here?
    • So: what should each actor (labs, states, communities, individuals) do, and at what pace?
  3. Broader impacts: setting existential risk aside, how does AI change how we live, and what do wisdom traditions offer?
    • How should we think about the impacts of a new technology?
    • What are the main benefits and harms?
    • Does AI strengthen or weaken our own agency and judgment?
    • Does AI strengthen or weaken our societies, and how?
    • What do wisdom traditions offer?
    • So: how should we integrate AI into our lives and societies?
  4. Consciousness: could AI be conscious or a moral patient, and how should we relate to it? (a smaller strand)
    • What would it mean for AI to be conscious or a moral patient?
    • What is the evidence about current AI?
    • How are labs and others responding?
    • What do wisdom traditions offer?
    • So: how should we relate to AI under uncertainty?

The full tree

1. Alignment: can we align AI, and what role can wisdom traditions play?

  1. What would it mean for AI to be aligned, or good?
    1. Aligned to what: instructions, preferences, wellbeing, values, the good?
      1. What is the good, and how could an AI come to perceive it?
    2. Aligned to whom, and who decides?
    3. Must AI be wise, or is "safe and corrigible" enough?
  2. How is it attempted today, and does it work?
    1. What are the main approaches (rules and constitutions, preference learning, character training), and which ethical traditions do they implicitly embody?
    2. What does the evidence show, both successes and failures?
    3. Can we verify what a model has actually learnt?
      1. Can we read a model's reasoning, and is it faithful?
      2. Can models report their own states reliably?
      3. What can interpretability show?
    4. What is the strongest case that current methods will scale, and how convincing is it?
  3. Is the target (goodness or wisdom) the kind of thing that can be specified or trained?
    1. What do we know about how humans become good or wise?
    2. How does training an AI compare with how humans become good, and what, if anything, is the AI analogue?
  4. What do wisdom traditions offer?
    1. Why these traditions, which strands, and how do we decide when they conflict with each other or with secular ethics?
    2. Which teachings bear on alignment, and what would each change in practice?
    3. Where do the analogies break down, and what can traditions not tell us?
  5. So: can we align AI, and what is the realistic role of wisdom traditions?

2. Risk and pace: given existential risk and the chance alignment fails, how should we approach AI development, and what do wisdom traditions suggest?

  1. What are the catastrophic and existential risks?
    1. What kinds: loss of control, misuse, irreversible power concentration, gradual disempowerment?
    2. What evidence is there that these are already emerging?
    3. How likely and how soon, and how much do experts disagree?
  2. Why is the world racing anyway?
    1. What drives the race: competition between labs and states, money, prestige, "if we don't, others will"?
    2. What evidence is there of race dynamics, and of restraint?
    3. Why is the race so hard to escape? Are there deeper cultural or psychological drivers?
  3. Should we change pace or trajectory, and can we?
    1. Is a slower or different trajectory desirable (pause, pacing, steering, differential development), weighed against the case for speed (ceding ground, forgone benefits)?
    2. Is it feasible? What mechanisms exist (auditing, compute governance, treaties), and what are the precedents of collective restraint?
    3. Who decides?
  4. What do wisdom traditions offer here?
    1. What do they say we should do? Is the normative answer clear?
    2. Do they help with how, i.e. solving the collective action problem, and if so through what (e.g. a shift in worldview), or is this where they offer less?
    3. How would a wise process make this choice, beyond weighing a trade-off?
  5. So: what should each actor (labs, states, communities, individuals) do, and at what pace?

3. Broader impacts: setting existential risk aside, how does AI change how we live, and what do wisdom traditions offer?

  1. How should we think about the impacts of a new technology?
    1. Why are a technology's impacts so hard to foresee?
    2. What does social media teach us?
  2. What are the main benefits and harms?
    1. What are the potential benefits, and where (for individuals, for children and education, for society)?
    2. What are the potential harms, and where?
    3. What evidence is there that these are already occurring?
    4. Which few matter most?
  3. Does AI strengthen or weaken our own agency and judgment?
    1. What do we mean by human agency and human judgment?
    2. In what ways could AI strengthen or weaken them?
    3. What is the evidence, from neuroscience and psychology and from what is already happening?
    4. What could we do about it, and how?
  4. Does AI strengthen or weaken our societies, and how?
    1. Democracy and public discourse?
    2. Governance and the power of the state?
    3. Work and the economy?
    4. Education and the institutions of knowledge?
    5. Security and the military?
  5. What do wisdom traditions offer?
    1. Which wisdom traditions should we look at?
    2. Which Buddhist teachings are relevant (e.g. mindfulness, consumption, speech and listening, community), and what does each say?
    3. Which teachings from other traditions (e.g. Christianity) are relevant, and what do they say?
    4. How do these teachings apply?
      1. To our own thinking and agency?
      2. To relationships and intimacy?
      3. To children and education?
      4. To society and institutions?
      5. To how we adopt new technology?
  6. So: how should we integrate AI into our lives and societies?
    1. What is the guidance for individuals?
    2. For families and schools?
    3. For designers and labs?
    4. What is the policy advice for society?

4. Consciousness: could AI be conscious or a moral patient, and how should we relate to it? (a smaller strand)

  1. What would it mean for AI to be conscious or a moral patient?
    1. What do we mean by consciousness, sentience and moral patienthood?
    2. What are the main theories, and what does each imply for AI?
  2. What is the evidence about current AI?
    1. What do assessments against theory-based indicators find?
    2. What do model self-reports and introspection research show, and how reliable are they?
    3. How likely do experts think it is, now and soon, and how much do they disagree?
  3. How are labs and others responding?
    1. What is Anthropic doing (model welfare, the constitution)?
    2. What are the critiques (e.g. the risks of "seemingly conscious AI")?
  4. What do wisdom traditions offer?
    1. How does Buddhism understand mind, self and sentience, and how does that reframe the question?
    2. What do other traditions (e.g. Catholic teaching) say?
    3. What does how we treat AI do to us?
  5. So: how should we relate to AI under uncertainty?
    1. What should labs do?
    2. What should users and society do?