AI Roundup - Monday, January 29th 2024
New embedding models and API updates
The introduction of a new highly efficient embedding model has brought about a significant upgrade in performance compared to its predecessor. This new model, released in December 2022, outperforms the previous model in several aspects. On the commonly used benchmark for multi-language retrieval (MIRACL), the average score has increased from 31.4% to 44.0%. Similarly, on the benchmark for English tasks (MTEB), the average score has increased from 61.0% to 62.3%. In addition to improved performance, the new model also comes at a reduced price. With a price per 1k tokens of $0.00002, it is 5X cheaper than the previous model, which was priced at $0.0001 per 1k tokens. Although the newer model is recommended, the previous generation model will not be deprecated, and customers are still welcome to use it. However, the new model, with its larger embedding size of up to 3072 dimensions, is described as the best performing model yet. On MIRACL, its average score has increased from 31.4% to an impressive 54.9%, and on MTEB, the average score has increased from 61.0% to 64.6%. This makes it a compelling choice for customers seeking optimal performance.
Democratic inputs to AI grant program: lessons learned and implementation plans
OpenAI received nearly 1,000 applications from 113 countries for their AI Grant program. A joint committee of OpenAI employees and external experts in democratic governance selected the final 10 teams. These teams come from 12 different countries and have expertise in various fields, including law, journalism, peace-building, machine learning, and social science research. Throughout the program, the teams received hands-on support and guidance. OpenAI encouraged the teams to document their processes using "process cards" and "run reports", which facilitated collaboration and faster iteration. In September, OpenAI organized a Demo Day where the teams showcased their concepts to each other, OpenAI staff, and researchers from other AI labs and academia. The projects covered different aspects of participatory engagement, such as novel video deliberation interfaces, crowdsourced audits of AI models, representation guarantees, and mapping beliefs to fine-tune model behavior. AI itself played a role in the projects, providing customized chat interfaces, voice-to-text transcription, and data synthesis. OpenAI is sharing the code created by the teams and providing brief summaries of their work.
How OpenAI is approaching 2024 worldwide elections
In order to protect the integrity of elections, Google is committed to ensuring that their technology is not used in any way that could undermine the democratic process. They recognize that collaboration from all stakeholders is crucial in achieving this goal. Google's tools are designed to empower individuals and solve complex problems, but they also come with their own set of challenges. As they prepare for elections in 2024, Google's approach involves elevating accurate voting information, enforcing policies, and enhancing transparency. They have dedicated a cross-functional team to election work, comprised of experts from various fields. Some key initiatives they are investing in include improving platform safety, addressing potential abuse, and leveraging threat intelligence. Google is committed to evolving their approach as they gain more insights into how their tools are used.