Transcript 0:00 Did you see Ox Alpha? This model made Twitter spend an unreasonable amount of time trying to figure out who built it, and after all those hours, we might have an answer. 0:11 It is very probably connected to ZAI, also known as Zhipu AI, and the GLM family, except that it's not the most interesting part of that story. 0:20 Because while we were treating Ox Alpha as a mystery to solve, whoever made Ox Alpha was watching several hundred thousand developers take an unreleased model and throw real production work at it, long context, tool use, serving infrastructure under load. 0:36 They discovered weird failures, and after that, they published benchmark trajectories and VOO comparisons. People even reversed engineering the tokenizer for free. 0:44 So we thought we were investigating the model, but from the other side of the API, the model was investigating us. [upbeat music] Attention Span. Welcome to Attention Span by Turing Post. 0:59 I'm Ksenia, and today I want to talk about Ox Alpha with no hype involved, what we know, how we know it, how good this thing really is, and the unpaid work all of us have been doing for an anonymous lab. Let's go. 1:13 So what is Ox Alpha? It appeared on OpenRouter and OpenCode four days ago on August 20 under the model ID stealth-ox-alpha. 1:22 The provider namespace is stealth because there is no company name to put there, at least for now. Both platforms describe it as a reasoning model for coding, sustained agentic work, and production workloads. 1:34 It accepts text, images, and video. That is important. It has a context window of just over one million tokens, generates up to 130,000 tokens, calls tools, and costs zero dollars in and zero dollars out. 1:48 OpenRouter is explicit that it only routes the requests. The system was developed and operated by a third party that chose to remain anonymous during the preview. 1:58 OpenCode's launch note said that this model is free for one week from August 20 approximately to August 27th, so if you are watching this and thinking about testing this model, the window is closing. 2:11 But you might think twice if you should throw your energy and provide even more results to the anonymous lab after you watch this episode. So how does it usually work on the internet? 2:23 A mysterious model is already super interesting. A mysterious model that might be frontier level running free with no rate limits worth speaking of is absolutely irresistible. 2:35 So people immediately began asking the model who it was, which is kinda useless. 2:41 A model can tell you it's Claude, Gemini, GLM, OpenAI, or Joe Mama for that matter, because self-identification is just another generated answer. Much cleverer tests look at things the model does without meaning to. 2:55 So how do you fingerprint a model? Start with the tokenizer. Every language model has to turn text into tokens before it can process anything, and different tokenizer vocabularies break the syntax differently. 3:08 So you feed several APIs deliberately strange strings, Chinese characters, Russian words, code syntax, weird emojis, unusual Unicode, whatever you want, and you look at how many input tokens each API reports back. 3:23 You never need access to the tokenizer itself. You only need the count. And if Ox Alpha says one weird string is 17 tokens, another is 34, another is 11, and you repeat that 95 times, you have a signature. 3:37 One investigation ran exactly that and found Ox Alpha matched the released GLM vocabulary on, wait for it, all 95 probes, 95 out of 95. That tells us something very specific but still a little bit limited. 3:53 It does not prove who trained the model because anyone can reuse a tokenizer, but it does make GLM a far better hypothesis than Gemini or Claude. 4:02 Then researchers started sending malformed requests, and this is my favorite part because they stopped interrogating the model and started annoying the API. 4:12 So when you send an API something slightly wrong, the model often never sees it. The serving software rejects the request first, and serving software has habits. 4:22 Ox Alpha returned the same validation behavior as ZAI-hosted GLM models, including a distinctive error code and an internal Java class path matching Zhipu's API structure. 4:34 The clever control was checking a different provider hosting the same GLM weights, same model family, different server infrastructure, and any error that showed up on Ox Alpha and on ZAI but not on the other host had to be a fingerprint of the company's software rather than the model. 4:52 Then video. A multimodal system does not consume a movie frame by frame at full resolution. It samples frames, resizes them, converts the visual information into tokens, and every system makes those choices differently. 5:07 In one community test, four videos produced identical visual token counts on Ox Alpha and on GLM-5V Turbo. So tokenizer looks like GLM, plumbing looks like ZAI, video accounting looks like GLM. Oh my God. What is that? 5:23 I should be fair about the noise, though, because for about a day, the consensus wobbled hard. Serious people were posting that this was Gemini. 5:32 One outlet revised its position to suggest an unreleased Microsoft My model. 5:37 There were also people at Google posting cryptic things and collecting millions of views, which is either a coincidence or a very cheap way to warm up your own launch. 5:46 That gap is actually the whole lesson of this section. Vibes moved the story back and forth for the whole day, but the token counts did not move at all. It's still the same. 5:57 And here is what makes the GLM answer even a better guess. Earlier this year, ZAI did something close to this with GLM-5. They put it on OpenRouter anonymously under the name, wait for it- Pony Alpha. 6:13 The team said afterwards that the idea was simple: strip the brand and see how people judge the model when they cannot see who made it. And people guessed widely. Is it Claude, Grok, DeepSeek, GLM? It was Pony Alpha. 6:25 It was GLM. Now we see Ox Alpha not saying anything. But it's not one precedent in the bigger picture. It is a pattern. 6:33 Ox Alpha is actually the fifth anonymous model to appear on OpenRouter in about six months, and all four before were eventually claimed by Chinese labs. 6:43 Zhipu's GLM 5, Xiaomi's Mimo V2 Pro, and GroupLink 2.6 Flash, and Mate 1's Longcat 2.0. Four for four. Now maybe most likely five for five. 6:56 And there is one more piece that I think is the strongest of all because it explains why this model exists in the first place. Zhipu shipped GLM 5.3 as a text-only model. Ox Alpha reads video. 7:08 So this is not vaguely some unreleased GLM. 7:11 The evidence points at the unified multimodal flagship that ZAI has not announced, and now it's being heavily tested in public at zero cost by everyone who wants to, which brings me to the part I care about. 7:25 With Pony Alpha, people were mostly judging outputs. Prompt in, answer out, opinion formed. 7:33 With Ox Alpha, people are handing the model repositories, terminals, test suites, screenshots, and coding sessions that run for dozens of hundreds of steps. 7:44 That changes what the lab can learn, and I want to be precise about how. Benchmarks usually measure the result. Arenas measure the preference. Do I want this answer or that answer? 7:58 Agents expose the process, where the model chokes and stop working, how it recovers from a tool error, whether it holds its shape across a long session, which part of the harness breaks first. 8:11 It's impossible to buy that. You can barely simulate it internally. It's too much work. And here, with this type of testing, it arrives for free in four days from people like you and me. 8:23 OpenCode's dashboard gives a tangible feeling to that. As of August 24th, it was showing roughly twenty-four trillion tokens processed across almost eight million sessions with a 93% cache hit rate. 8:37 This 93% rate is important because it tells you what kind of work this is. Coding agents resend large parts of the same context on every step as they read files, run tools, and continue a task. 8:51 A 93% hit rate means the traffic is overwhelmingly long agent loops rather than one-off questions. 8:58 And without caching, serving those sessions would get expensive very fast, making it free widen the test in a way pricing never would. 9:08 So developers tried it on languages, repositories, tools, and workflows they would not have paid to experiment with. How smart is that? 9:17 So the provider gets to see whether people want the model with no brand attached, where the infrastructure holds under real demand, and where the model starts to come apart inside long runs. 9:28 Then the community volunteers a second layer on top. Researchers publish full benchmark runs with patches, test results, and complete trajectories. Others probe the tokenizer, the API errors, the multimodal behavior. 9:42 Somebody built a leaderboard entry before the lab wrote a model card. That is what I mean by you are the eval. I don't only mean the private repository becomes training data. We will get to the data terms in a minute. 9:55 The experiment is valuable even if not a single token is ever trained on. The feedback alone maps where the model works and where the system around it breaks. 10:04 And by the time an official launch happens, a rough public version of the model card already exists, written by you. So how actually good is this model? 10:13 Stripped of the mystery, Ox Alpha is very good, and it's not the best coding model in the world. The cleanest evidence is DeepSWE because it measures long repository-level software work rather than coding trivia. 10:28 Current leaders sit around 74% for Claude Opus 5, 73% for GPT 5.6 Sol, and roughly 69% for GLM 5.3 and Kimi K3. 10:40 Ox Alpha has two published full 113 task community runs, 58.4% with 66 of 113 tasks solved. 10:51 Another run had 64.6% with 73 out of 113 tasks solved under a different setup. The famous 80% That went around the internet came from 10 handpicked tasks, not 113, but 10. 11:08 Against Code Fable 5 at 65 and GPT 5.6 Solo at 52 on the same 10. Pretty good result. 11:15 So I would place OxAlpha in the strong frontier adjacent tier, clearly capable of serious repository-level engineering, clearly not beating the top group. 11:26 But I want to go back to the 58.4% because it undersells the actual work of the model. In that full run with 113 tasks, 80% of all tasks passed at least 90% of their fail to pass tests. 11:42 On most of its failures, OxAlpha got the engineering mostly right. It found the correct region of the code base, understood the request feature, implemented the bulk of it, and then missed an age case. 11:54 I want you to picture two models to understand it better. Model A solves 60 tasks perfectly and completely misunderstands the other 40. 12:03 Model B solves 60 tasks perfectly and gets 35 of the remaining 40 almost entirely correct. You see the difference, right? Same pass rate, wildly different models. 12:15 OxAlpha is Model B, which is exactly why developers come away saying, "This thing is excellent," while the binary score reads 58. There are two weaker parts of this model is agent reliability and speed. 12:29 In that same run, 11 of 113 tasks were lost because the model stopped emitting valid tool calls, not because it could not reason about the software. 12:38 That was nearly 10 percentage points evaporating because the model failed to keep talking to the harness correctly. And as I said, it's slow. 12:47 OpenRouter is showing around 24 output tokens per second with median time to first token somewhere in the four to six second range, depending on when you look. 12:56 It's fine for an agent grinding for a long task, but it's not a fast interactive model. The data terms deserve some attention too, because somehow they do not agree with each other on OpenCode and OpenRouter. 13:10 OpenCode's Zen documentation says the OxAlpha provider follows a zero retention policy and excludes user data from training. 13:17 The OpenRouter OxAlpha page says the provider retains prompts and completions while excluding them from training. OpenRouter's general stealth program agreement is broader still. 13:28 It says stealth models are made available free to collect user content for training and improvement, and it grants OpenRouter the right to share that content with the anonymous provider for those purposes. 13:39 This may describe different routes, and the model-specific language may be intended to override the general program terms. Nobody has really clarified which controls. 13:49 For a developer with proprietary code in the buffer, that ambiguity is still a big question. And the practical rule here is simple. 13:58 Public repositories, sanitized experiments, and throwaway spikes are reasonable material for a free preview. 14:04 But private source, customer data, credentials, security-sensitive infrastructure, and unreleased research stay out until the operator and the controlling terms are named. And notice the trade we're being offered. 14:17 The same anonymity that removes brand bias from our evaluation also removes our ability to do any diligence at all. 14:25 We get to judge the model without knowing its reputation, and we are asked to trust the operator without knowing its name. And here's a very interesting part I'm leading you to. 14:37 That trade is only available because of where the model showed up, and this is the thing I really want to end on. So let's discuss this launch in reverse. 14:49 The normal sequence is a lab announces a model, shows selected benchmarks, opens access, and then testing begins, or it hands early access to a few influencers, few people who, uh, have large followings, and lets social media handle the rest. 15:05 But it's not enough testing. OxAlpha inverted that. People got the model first and had to construct the story themselves. They spent hours of testing it. 15:15 That produces far more varied testing than any launch post would have. It also exposed something I think is becoming the real story, where a model is discovered and evaluated. 15:26 That's starting to carry as much weight as who built it. And the market has already priced this in. Last week, we've seen an incredible sell. 15:36 Five days before OxAlpha appeared, Stripe confirmed it was acquiring OpenRouter, reported price around seven and a half billion dollars against the $1.3 billion valuation three months earlier. Wow. 15:49 OpenRouter now moves more than 10 trillion tokens a day across 400-plus models for over 10 million developers. 15:56 Then OxAlpha launched on that platform, and Stripe's CEO posted publicly that it was impressive while the acquisition was still closing. So if you look at that, look at what the routing layer now does. 16:08 It decides which models are easy to reach. It decides which get a free tier. It publishes the usage dashboards that shape what people believe about a model's popularity. 16:20 It also hosts the anonymous previews and writes the terms that govern your data during them. So much power. 16:27 Meanwhile, OpenCode sits closer to the actual coding work and increasingly holds its own opinion about which model provider combinations perform best in practice. No OpenRouter, no OpenCode trained anything. 16:42 None of them did. But both of them are now positioned to influence which models the industry takes seriously. And one of them just sold for seven and a half billion dollars for exactly that reason. 16:53 So we are going to pick up right there on Friday. What happens when the platform around the model starts deciding which models win? And for now, I'll leave you with a question I keep going back and forth on myself. 17:07 Would you run an anonymous model on real work if the access platform promised zero retention? Or does the missing name on the other end stop you cold? Thank you for watching Attention Spend by Turing Post. 17:19 There is a lot of going on this week, and I'm still thinking about this anonymous launch and what does it bring us to with the platforms that decides which model wins. We'll discuss it later. I'll see you next week. 17:30 [outro music]