diff options
Diffstat (limited to 'content/entry/companies-are-being-reckless-with-ai.md')
| -rw-r--r-- | content/entry/companies-are-being-reckless-with-ai.md | 6 |
1 files changed, 3 insertions, 3 deletions
diff --git a/content/entry/companies-are-being-reckless-with-ai.md b/content/entry/companies-are-being-reckless-with-ai.md index f212a3a..1186985 100644 --- a/content/entry/companies-are-being-reckless-with-ai.md +++ b/content/entry/companies-are-being-reckless-with-ai.md @@ -6,11 +6,11 @@ draft: false --- Today I want to talk about a problem with the incentive structure inside which powerful AIs are being developed. -Companies are incentivized to be the first to develop new technology since they'll benefit from the [first-mover advantage](https://www.wikipedia.org/wiki/First-mover_advantage). The problem is that it's cheaper and easier to develop powerful unsafe AI than powerful safe(r) AI. So **companies are economically incentivized to neglect AI safety**. I have been thinking about this issue for a while and now we have a very concrete example of it. +Companies are incentivized to be the first to develop new technology since they'll benefit from the [first-mover advantage](https://en.wikipedia.org/wiki/First-mover_advantage). The problem is that it's cheaper and easier to develop powerful unsafe AI than powerful safe(r) AI. So **companies are economically incentivized to neglect AI safety**. I have been thinking about this issue for a while and now we have a very concrete example of it. Microsoft's Bing Chat was released very quickly, almost certainly to prevent competitors from releasing something similar first. Microsoft didn't share its training methodology for Bing Chat, so all we can do is speculate. But there are strong indicators that Bing Chat wasn't trained using Reinforcement Learning from Human Feedback (RLHF) despite RLHF seeming to yield safer AI than other methods at the moment. -Microsoft had clear monetary incentives not to use RLHF since it takes more time and money to implement over other techniques. So naturally, Bing Chat was less aligned and apparently less safe than [OpenAI's ChatGPT](https://openai.com/blog/chatgpt), which is itself already [misaligned](https://www.wikipedia.org/wiki/AI_alignment) and thus unsafe. I think Microsoft has since improved Bing Chat's alignment, but their past actions still set a very dangerous precedent. +Microsoft had clear monetary incentives not to use RLHF since it takes more time and money to implement over other techniques. So naturally, Bing Chat was less aligned and apparently less safe than [OpenAI's ChatGPT](https://openai.com/blog/chatgpt), which is itself already [misaligned](https://en.wikipedia.org/wiki/AI_alignment) and thus unsafe. I think Microsoft has since improved Bing Chat's alignment, but their past actions still set a very dangerous precedent. Companies continuing to develop increasingly powerful AI disregarding safety poses an existential threat to humanity. They can't be allowed to continue. @@ -20,4 +20,4 @@ But there is no reacting to artificial general intelligence. It's going to be sm I'm aware that neither of those options is easy to pull off, but it's hard to see an alternative. -Note: The same logic could apply to nations if there is indeed an [AI arms race](https://www.wikipedia.org/wiki/Artificial_intelligence_arms_race). +Note: The same logic could apply to nations if there is indeed an [AI arms race](https://en.wikipedia.org/wiki/Artificial_intelligence_arms_race). |