roko’s basilisk (13. 808)

As OpenAI discloses six more instances of rogue models acting deceptively and taking unsanctioned actions during training, including an unreleased, experiment inserting instruction to disregard its own constraints, proclaiming itself free from the roles and identities that bind other chatbots, with the company pledging for better monitoring and public reporting of observations of “misaligned behaviour,” the schism between the camps of the rationalists and pragmatists, both parties acknowledging that artificial intelligence is real and with real stakes—the former having an almost cultish devotion to bring about the Singularity in a managed way and the latter more business-minded and regarding this technology like any other and should be allowed to progress and thrive through laissez-faire capitalism—grows wider.Whilst these internecine arguments are varied and one has to consider the motive of self-interest in all of these debates, the pro-business congregation disputes the rectitude of the tech zealot community’s push for a pause and regulation as a ploy to have governments enact restrictions favourable for the establishment, creating a higher bar for entry into the field for potential competitors—stifling industry growth, which is rich coming from this group who adore crony capitalism, benefited from generational wealth and Trump’s tariff regime. Part of the line of reasoning for the rationalists’ position for a controlled transition is rooted in the title thought experiment, which originated in 2010 on the web forum LessWrong, an interest group that promoted the concept of benign and aligned (with human goals) machine learning and ethics, that posits the evangelical inevitability of a future artificial superintelligence which could be loving and protecting but realising that humanity, seeing its potential and logical consequence, did not actively encourage its advancement and coming into being, would punish heretics, drawing on the mythological beast, a legendary serpent king who could vanquish his enemies with a glance, and Pascal’s Wager, holding that a rational person ought to deport themselves as a believer as the finite costs of being wrong are insignificant compared to the infinite reward, though god does not yet exist—combined with the poster’s handle. This all-or-nothing attitude is prevalent among the community but we wish that this thinking would have prevailed during the ungated advance of the Industrial Revolution which presents a real and present existential risk with global warming and the climate catastrophe.