Meta launched its first AI model from Meta Superintelligence Labs under Alexandr Wang. Called the Muse Spark, the AI model powers the Meta AI assistant in the Meta AI app and meta.ai. Hours after the launch, some users criticized the new Meta AI model. One such user François Chollet @fchollet shared a post on X calling Muse Spark a “disappointment” and “overoptimized for public benchmark numbers at the detriment of everything else”. Wang, who Meta hired as part of $14 billion investment into Scale AI last year to head Superintelligence Labs, defended the AI model saying that the company has been upfront about the model not performing well on ARC AGI 2.Criticizing Meta’s frontier Muse Spark AI model, Chollet wrote: “The new model from Meta is already looking like a disappointment: overoptimized for public benchmark numbers at the detriment of everything else. Knowing how to evaluate models in a way that correlates with actual usefulness is a core competency for AI labs, and any new lab is unlikely to be successful without first figuring that out.”To this, Meta AI chief Alexandr Wang replied:“We are always open to feedback and welcome any perspective on weaknesses you’ve noticed in the model from using it. We are quite upfront that our model does not perform well on ARC AGI 2 for example, and published those results for the community to understand. That might reflect some areas of improvement of the model that we could focus on in the future.In general, we have been pleasantly surprised by users’ feedback on the models in areas like visual coding, writing style, and reasoning queries.”
What is Meta Muse Spark
Meta’s Muse Spark is the company’s first model in its new Muse series. The AI model can help users with everyday tasks such as calculating the calories in a meal from a photo or superimposing an image of a mug on a shelf to see how it looks.Initially, the AI model will power the Meta AI chatbot and will be available on Meta AI app and website. In the coming weeks, it will replace the existing Llama models powering chatbots on WhatsApp, Instagram, Facebook and Meta’s collection of smart glasses. The Meta AI chatbot will remain free for users, but the company is reportedly considering implementing subscription fees in the future.“This initial model is small and fast by design, yet capable enough to reason through complex questions in science, math, and health. It is a powerful foundation, and the next generation is already in development,” the company said in a blog post.However, Meta did not disclose the model’s size, a key measure typically used to compare an AI system’s computing power with rivals.
Leave a Reply