The company will build a suite of products for customers to evaluate the dangers of their AI products.
Scale AI, an artificial intelligence company that helps other firms train and label their own AI software, said on Wednesday it would launch a new AI safety lab intended to address emerging concerns over the responsible development of artificial intelligence.
Called the Safety, Evaluations, and Analysis Lab, or SEAL, the unit will build a suite of products for Scale’s customers to use to evaluate possible risks posed in their AI deployments, as well as implement red team methods that seek to find faults and biases in software before it is launched. The lab’s work will include developing automated rating systems based on large language models and conducting research about potential AI harms.
The lab is led by former Google researcher Summer Yue, who was previously a lead for Google’s Bard chatbot. In her role at Google, Yue worked on RLHF, which stands for “reinforcement learning from human feedback,” a technique for training AI that involves human guidance and machine learning.
The initiative comes after the explosion of generative AI services like OpenAI’s ChatGPT and Google’s Bard and a backdraft of intense debate over AI safety. Last week, President Joe Biden issued an executive order on artificial intelligence, placing guardrails on companies developing the technology. One provision, for example, requires companies to develop models that could pose national security risks to disclose to the government. In the UK, meanwhile, Prime Minister Rishi Sunak held an AI safety summit in Bletchley Park where political and business leaders, including US Vice President Kamala Harris and X’s Elon Musk gathered to discuss the new wave of technology.
“Every enterprise and government deploying an LLM application is facing the need to adopt and comply with the forthcoming standards and regulations that will be put in place,” Scale said in a blog post announcing SEAL. (The company declined to make Yue available for an interview.)
Scale, co-founded in 2016 by 26-year-old CEO Alexandr Wang, got its start by helping companies to label data for autonomous driving before broadening its focus to include logistics, ecommerce, and defense. Scale also runs Remotasks, which recruits freelancers worldwide as data labelers. (Disclosure: Wang is slated to become a member of Forbes’ board when a planned sale of the company is completed.)
“To advance our knowledge and understanding of large language models (LLMs)–and therefore advance their capabilities–we must understand their weaknesses as much as their strengths,” Wang wrote in September after Scale signed onto a voluntary commitment from the Biden administration on AI safety and security. “It is imperative for the tech industry to take proactive measures like these given the rapid pace of technological development.”
