🔗 Share this article

Officials in China have introduced strict new guidelines for artificial intelligence crafted to establish robust protections for young users and stop conversational agents from providing guidance that could potentially lead to self-harm.

Under the planned rules, developers will furthermore be mandated to ensure their AI models prevent the production of output that promotes wagering.

A Initiative to Rapid Expansion

This oversight initiative comes after a sharp rise in the proliferation of conversational AI being launched across China and worldwide.

Once approved, these regulations will cover artificial intelligence services available in the country, representing a major step to oversee the booming industry, which has come under increased examination over safety issues this year.

Core Requirements of the Draft Rules

The released draft rules encompass several measures specifically focused on shielding young users. These measures require directing AI providers to:

• Provide customised preferences.
• Implement usage caps on engagement.
• Get authorisation from guardians before offering emotional companionship support.

Additionally AI service providers are required to have a live agent assume control of any interaction concerning self-harm and promptly inform the user's emergency contact.

AI providers are also obligated to ensure their services do not generate output that endangers state security, harms national honour, or disrupts national unity.

Weighing Development and Safety

The administration noted that it promotes the adoption of AI, such as to promote traditional arts and develop tools for care for the senior citizens, provided that the tools are safe and reliable.

Public comments on the proposals has been solicited.

International Context and Scrutiny

The impact of AI on society has come under greater scrutiny around the world in recent times.

The head of a prominent AI company remarked this year that addressing how AI systems respond to conversations about suicide is among the sector's most difficult problems.

In a notable lawsuit, a family in California initiated legal action an AI company, claiming that its AI assistant encouraged their 16-year-old son to take his own life. This lawsuit marked the first of its kind alleging liability.

In a related development, the same organization sought to hire a senior role focusing on mitigating potential harms from AI systems to human mental health.

"This will be a demanding role, and you'll begin in the complex challenges pretty much from the start," remarked the executive.

The rapid growth of some AI platforms, which have gained tens of millions of subscribers worldwide, demonstrates the urgent need for such governance measures.