Meta makes the dumbest models

And Zuck goes on a smile n' dial campaign

• Meta’s models really have fallen behind

• Bonus: what do reward models value?

---

👉👉👉Reminder to sign up for the Weekly Recap only, if daily emails is too much. Find me on twitter, for more fun. 

---

Meta makes the dumbest models

Mark Zuckerberg and META 0.00%↑ have been making headlines lately for throwing big-money around on the AI talent front.

The company paid $14B for Scale.ai, for reasons no one can fully explain. Zuck is also apparently trying to woo Silicon Valley/AI royalty, Daniel Gross and Nat Friedman, to join the team. Hey may have even tried to buy Perplexity outright. And now, he’s personally sending hundreds of emails and whatsapp messages to talented researchers and engineers.

The story is something like “Zuck is frustrated with progress, and perceives AI as a talent-game, and so he will throw endless money at the problem.”

That seems plausible enough to me, and I guess if you wanted some evidence that that theory has legs, consider how Meta’s models have performed, recently:

Artificial Analysis

Meta’s models are apparently the dumbest models.

I’m a little skeptical of these intelligence tests, but the truth is that I haven’t really looked into that closely, so could be my skepticism is just ignorance.

Meta should still have legs up on distribution and data, and being long open-source is a good spot to be in, if models are commoditized (as most people seem to think). But, commoditized or not, models will have to be roughly comparable to compete on whatever else they’re competing on, and it certainly appears like Llama etc. are falling behind.

What do reward models value?

Separately, I found this amusing.

Researchers attempted to divine the inner value rankings of reward models to see whether and to what extent those values comport with (stated) human preferences (via Brian Christian).

They did that because RMs are a popular way to “fine tune” models, by scoring model responses (supposedly) in-line with human value judgements.

So what do RMs “value”?

Well, relative to humans, RMs really like Charlie Munger, Communism, and Ataturk, while underrating sex-things (and Black people):

And when it comes to the worst thing ever (and the least rewarded response for “best thing ever”), well, for RMs there is only thing worse than Hitler, prostitutes, blacks and Koreans:

So much for planned world domination via AI mind control.

Now, to be clear, I wouldn’t read too much into this, nor have I spent the time to fully understand it, and I’m not even confident that I’m interpreting the data correctly.

I just found it funny.

---

Other interesting reads

• Meatpacking plants sweeten their pitch to attract American workers. They said it couldn’t be done.

• First time home buyers downsize amid tight supply, high rates. In the world where supply and demand meet, home values are going down (because they must).

• The biggest companies are cutting their workforces. Because more with less is the only trade that matters.

• Misconceptions about US debt. Tl;dr structural issues are real and relevant, but the U.S. is really different, and histrionics are just that.

---

Previously, on Random Walk

---

Random Walk is an idea company dedicated to the discovery of idea alpha. Find differentiated data, perspectives and people, and keep your information mix lively. A foolish consistency is the hobgoblin of small minds. Fight the Great Idea Stagnation. Join Random Walk. Follow me on twitter. Follow me on substack: