From 628046738b0e4f410c639dd4844925ff044c79d2fb14b0e42722f1bee733f1ad Mon Sep 17 00:00:00 2001 From: Nicholas Johnson Date: Mon, 27 May 2024 00:00:00 +0000 Subject: Fix tons of links --- content/entry/companies-are-being-reckless-with-ai.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) (limited to 'content/entry/companies-are-being-reckless-with-ai.md') diff --git a/content/entry/companies-are-being-reckless-with-ai.md b/content/entry/companies-are-being-reckless-with-ai.md index f212a3a..1186985 100644 --- a/content/entry/companies-are-being-reckless-with-ai.md +++ b/content/entry/companies-are-being-reckless-with-ai.md @@ -6,11 +6,11 @@ draft: false --- Today I want to talk about a problem with the incentive structure inside which powerful AIs are being developed. -Companies are incentivized to be the first to develop new technology since they'll benefit from the [first-mover advantage](https://www.wikipedia.org/wiki/First-mover_advantage). The problem is that it's cheaper and easier to develop powerful unsafe AI than powerful safe(r) AI. So **companies are economically incentivized to neglect AI safety**. I have been thinking about this issue for a while and now we have a very concrete example of it. +Companies are incentivized to be the first to develop new technology since they'll benefit from the [first-mover advantage](https://en.wikipedia.org/wiki/First-mover_advantage). The problem is that it's cheaper and easier to develop powerful unsafe AI than powerful safe(r) AI. So **companies are economically incentivized to neglect AI safety**. I have been thinking about this issue for a while and now we have a very concrete example of it. Microsoft's Bing Chat was released very quickly, almost certainly to prevent competitors from releasing something similar first. Microsoft didn't share its training methodology for Bing Chat, so all we can do is speculate. But there are strong indicators that Bing Chat wasn't trained using Reinforcement Learning from Human Feedback (RLHF) despite RLHF seeming to yield safer AI than other methods at the moment. -Microsoft had clear monetary incentives not to use RLHF since it takes more time and money to implement over other techniques. So naturally, Bing Chat was less aligned and apparently less safe than [OpenAI's ChatGPT](https://openai.com/blog/chatgpt), which is itself already [misaligned](https://www.wikipedia.org/wiki/AI_alignment) and thus unsafe. I think Microsoft has since improved Bing Chat's alignment, but their past actions still set a very dangerous precedent. +Microsoft had clear monetary incentives not to use RLHF since it takes more time and money to implement over other techniques. So naturally, Bing Chat was less aligned and apparently less safe than [OpenAI's ChatGPT](https://openai.com/blog/chatgpt), which is itself already [misaligned](https://en.wikipedia.org/wiki/AI_alignment) and thus unsafe. I think Microsoft has since improved Bing Chat's alignment, but their past actions still set a very dangerous precedent. Companies continuing to develop increasingly powerful AI disregarding safety poses an existential threat to humanity. They can't be allowed to continue. @@ -20,4 +20,4 @@ But there is no reacting to artificial general intelligence. It's going to be sm I'm aware that neither of those options is easy to pull off, but it's hard to see an alternative. -Note: The same logic could apply to nations if there is indeed an [AI arms race](https://www.wikipedia.org/wiki/Artificial_intelligence_arms_race). +Note: The same logic could apply to nations if there is indeed an [AI arms race](https://en.wikipedia.org/wiki/Artificial_intelligence_arms_race). -- cgit v1.2.3