Normally I'd say "it's never too late!" but clearly would diverge and require an entirely new project, maintaining two bases for the same thing, etc.

Good to see you alive and kicking. Happy holidays

Probably because some apps aren't NSFW apps but have it (Reddit)

lol I love how OpenAI just straight up doesn't compare their model to others on these release pages. Basically telling us they know Gemini and Opus are better but they don't want to draw attention to it

Not sure why they don't compare with others, but they are actually leading on the benchmarks they published. See here (bottom) for a chart comparing to other models: https://marginlab.ai/blog/swe-bench-deep-dive/

It's like apple, they just don't want users or anyone to even be thinking of their competitors, the competition doesn't exist, it's not relevant.

is swe-bench saturated? or they switch to swe-bench pro because...?

At least on swe-rebench it does pretty well: https://swe-rebench.com/

This was the one thing I scanned for. No comparison against Opus. See ya.

Though this Codex version isnt on the leaderboard, GPT-5.2-Medium already seems to be a bit better than Opus 4.5: https://swe-rebench.com/

Is that your website or something? You keep promoting it

No, I am not affiliated with the website, I just want to see more discussions based on uncontaminated benchmarks and feel that people rely too much on benchmarks that companies can conduct themselves. If that is the case, I don't feel I can trust them. For general LLM capabilities, for example, I would also tend to rely on dubesor [1] rather than artificial analysis or similar leaderboards.

[1] https://dubesor.de/benchtable

This is a bigger problem than just Instacart. The prices are much harder to revert after they've been established (as we've seen since 2020).

I'm reading these articles in Addy's voice. It's quite wonderful

This is a whole bunch of patting themselves on the back.

Let me know when Gemini 3 Pro and Opus 4.5 are compared against it.

How did you run a 123B model locally? Or did you do this on a GPU host somewhere? If so, what spec was it?

I haven't run the 123B one locally yet. I used Mistral's own API models for this.

Wonder why Gemini 3 Pro and Sonnet 4.5 are on this comparison but Opus 4.5 is not?

Based on the prompt you wrote, this woulnd't be a "Hallucination"

And as I write this critique of your HN title, I see you have edited it since I last refreshed. I'm guessing a few others have already echoed this sentiment a few times.

Indeed so: https://news.ycombinator.com/item?id=46216933, https://news.ycombinator.com/item?id=46213179

I never knew "Parisian" was how to refer to "one from Paris"

Learn something new every day.

To be clear, I just didn't think anyone would refer to someone from Paris specifically (rather than, "French").

I mean, a lot of places you would add "-ite" but I'm guessing that would be a less-than-ideal suffix for this particular city lol

As someone who knows a lot of New Yorkers and Texans, it's definitely curious that people would refer to themselves as from a city versus from the country itself.