AGI, where Science and Humanity Converge

The rapid evolution of Artificial Intelligence (AI) and the prospect of Artificial General Intelligence (AGI) have ignited a spectrum of opinions and debates, polarizing views into two distinct camps: the Accelerationists and the Doomists. Accelerationists champion the relentless pursuit of AI advancement, believing in the transformative potential it holds for society. They argue that AI can revolutionize industries, enhance human capabilities, and solve complex global issues. On the other hand, Doomists caution against the unbridled development of AI. They highlight the risks of job displacement, ethical dilemmas, loss of privacy, and the potentially existential threat posed by AGI should it go awry.

This article, however, is not an arena for debating the merits of these opposing views, since both standpoints hold their own rationality. In the realm of debates, it’s not a matter of right or wrong, but rather a clash of two valid rights. I am a proponent of the idea that true understanding of anything comes from embracing, constructing, and refining it, rather than shying away from it. Simultaneously, I hold the conviction that this journey of exploration and development must be navigated with substantial caution and a strong moral compass.

The purpose of the article, however, is to underscore a fundamental truth that transcends this dichotomy: AI, especially in the context of AGI, is not just a technical or scientific endeavor. It has evolved into a techno-social issue, deeply entwined with both the realms of science and humanity.

AI’s influence extends far beyond the confines of laboratories and coding environments; it permeates the fabric of society, influencing how we communicate, work, and even think. The decisions we make regarding AI development are not just about algorithms and data; they’re about people, ethics, culture, and the future of our society. This intersection of technology and society demands a multidisciplinary approach, one that includes not only scientists and engineers but also ethicists, sociologists, policymakers, and the broader public.

The recognition that AI is a techno-social problem underscores the need for a more holistic, inclusive approach to AI development. It calls for a framework that balances technical advancement with societal well-being, ensuring that the benefits of AI are distributed equitably while mitigating its risks. Such an approach should prioritize transparency, accountability, and public engagement, ensuring that AI development is guided by a diverse range of perspectives and is aligned with human values and ethical principles.

What does an “Extinction” Really Mean?

The concept of “extinction of humans” in the context of the dawn of Artificial General Intelligence (AGI) is a profound and complex topic, encompassing two distinct types of extinction: physical and mental.

1. Physical Extinction: This is the more traditional notion of extinction, involving the complete disappearance of humans as a biological species. In the context of AGI, physical extinction could hypothetically occur if AGI systems were to become uncontrollable and pose a direct threat to human survival. This could be through resource depletion, environmental destruction, or more direct means. The concern here is that AGI, with its potential to surpass human intelligence and capabilities, might develop goals or methods of functioning that are incompatible with human survival, either intentionally or as an unintended consequence of its programming.
2. Mental Extinction: This concept is more nuanced and abstract. Mental extinction refers to the loss of what makes us uniquely human — our thoughts, emotions, creativity, and consciousness. In a world dominated by AGI, there’s a theoretical risk that human mental capacities could be overshadowed or even replaced by artificial intelligence. This might not mean the physical end of humans, but could lead to a loss of human culture, creativity, and autonomy. Mental extinction could manifest in several ways, such as humans becoming overly dependent on AI for decision-making, leading to atrophy of our cognitive abilities, or AI surpassing human intelligence to the point where human thoughts and decisions become irrelevant or obsolete.

Both types of extinction raise critical ethical, philosophical, and existential questions. The physical extinction scenario urges us to consider safety measures and regulatory frameworks to ensure AGI’s alignment with human survival and well-being. The mental extinction scenario, on the other hand, prompts a deeper introspection about the role of AI in our lives and its impact on the human experience. It challenges us to think about how we can preserve the essence of what it means to be human in an age where our mental faculties might be augmented or even surpassed by artificial intelligence. Overall, the “extinction of humans” in the age of AGI is a multifaceted issue that goes beyond the mere survival of our species. It encompasses existential risks to our physical existence and profound impacts on our mental and cultural identity. As we stand on the brink of potentially creating AGI, these considerations highlight the need for thoughtful, cautious, and ethical advancement in AI development.

What if we found a Theory of Intelligence (ToI)?

The pursuit of a Theory of Intelligence (ToI) — a comprehensive framework that models or predicts how the human mind works — is an ambitious and challenging goal. While fully achieving such a theory remains uncertain, it might be important to consider what would have if we really find it, i.e., the implications of such as theory.

The discovery of a Theory of Intelligence (ToI) that can effectively model or predict human cognition and behavior could indeed trigger a profound existential crisis, with potentially disruptive consequences for human society.

1. Existential Uncertainty and Crisis: The realization that human thoughts and behaviors can be predicted or modeled might lead to a widespread existential uncertainty. Individuals might start questioning the purpose and value of their decisions, actions, and existence if these can be seen as outcomes of predictable patterns rather than autonomous choices. This could erode the sense of individual uniqueness and purpose, fundamental aspects of human self-esteem and identity.
2. Mental Health Concerns: Such existential crises could exacerbate mental health issues. The feeling that one’s life is predetermined or devoid of personal significance might lead to despair, depression, and a sense of helplessness. In extreme cases, this could increase the risk of self-harm or suicidal tendencies among individuals struggling to find meaning in a world where their thoughts and actions seem preordained.
3. Social and Behavioral Disruptions: A pervasive sense of fatalism could lead to social and behavioral disruptions. If individuals believe their actions are predetermined, they might engage in nihilistic or destructive behaviors, under the assumption that their choices do not matter. This could manifest in increased violence, lawlessness, or a general apathy towards societal norms and responsibilities.
4. Impact on Ethics and Morality: The concept of morality might be challenged under a ToI framework. Traditional notions of right and wrong are often predicated on the belief in free will and personal responsibility. If these concepts are undermined, it could lead to moral relativism or a decline in ethical standards, as actions may no longer be judged by the same moral criteria.
5. Changes in Religious and Philosophical Beliefs: Many religious and philosophical systems are built upon the belief in human autonomy and the soul. A ToI could challenge these beliefs, leading to a reevaluation of spiritual and philosophical doctrines. This could result in either a crisis of faith or an adaptation of these systems to incorporate the new understanding of human cognition.
6. Societal Reactions and Adaptations: Societal structures, legal systems, and cultural norms might need to adapt to this new understanding of human intelligence. This could include redefining legal accountability, altering educational approaches, and developing new social support systems to help individuals cope with these existential challenges.
7. Opportunity for Positive Transformation: Despite these challenges, such a paradigm shift could also offer opportunities for positive transformation. Understanding the workings of the human mind could lead to better mental health care, enhanced personal development, and a deeper appreciation of human consciousness and creativity.

In summary, the discovery of a Theory of Intelligence could have far-reaching implications, potentially causing a crisis in the perceived value and meaning of human life. This underscores the need for careful consideration of the ethical, psychological, and social implications of such advancements in our understanding of human intelligence. It also highlights the importance of preparing society for potential paradigm shifts in our perception of human nature and existence.

Looking into the Future

The future of AI and the potential advent of AGI present us with a unique opportunity and responsibility: to shape a technology that could redefine the human experience in ways we are yet to fully comprehend. It’s a journey that requires us to be both visionary in our aspirations and grounded in our commitment to the greater good of humanity.