Things get better as the technology gets more mature. It's a promising start for sure.

:( wish it was free

I run a crowd sourced website to collect data on the best and cheapest hardware setup for local LLM here: https://inferbench.com/

Source code: https://github.com/BinSquare/inferbench

Cool site, I noticed the 3090 is on there twice.

https://inferbench.com/gpu/NVIDIA%20GeForce%20RTX%203090

https://inferbench.com/gpu/NVIDIA%20RTX%203090

Oh nice catch, I'll fix that

---

Edit: Fixed

Nice! Though for older hardware it would be nice if the price reflected the current second hand market (harder to get data for, I know). Eg. Nvidia RTX 3070 ranks as second best GPU in tok/s/$ even at the MSRP of $499. But you can get one for half that now.

Great idea - I've added it by manually browsing ebay for that initial data.

So it's just a static value in this hardware list: https://github.com/BinSquare/inferbench/blob/main/src/lib/ha...

Let me know if you know of a better way, or contribute :D

It seems like verification might need to be improved a bit? I looked at Mistral-Large-123B. Someone is claiming 12 tokens/sec on a single RTX 3090 at FP16.

Perhaps some filter could cut out submissions that don't really make sense?

I am not working on web or server stuff.

I'm building a better primitive for infrastructure via microvm's (think virtual machine but fast and easy to use).

I am about to launch a complete rewrite of this: https://github.com/BinSquare/ERA

I run a keywords research tool, it scans posts across social media sites like bluesky, mastodon, hackernews, etc.

KeywordsPal.com

It's actually super interesting the technical aspects to scan 50k posts a day for as cheap as possible. I write about it here: https://keywordspal.com/blog/building-multi-platform-content...

I also built it as a result of being unsatisfied with f5bot

What were your issues with f5bot? I tried to sign up and Supabase auth went to spam btw.

Rohit was the driving force behind Alexa before he went on to AGI.

With this change in leadership, I'm not confident the ship is going the right direction.

You really think that Rohit was driving Alexa or AGI in the right direction?

I think the nova foundational models + bedrock + ml services are in the right direction.

The same way you learn to trust other dev's to do work.

I see ai as a tool, not a peer. I trust a peer when we aligned on the requirements of the project and where we want to go.

So the answer to how we get confidence in a workflow of agents to develop, to review and test without a human verifying the implementation?

I personally don't see me getting there.

added!

Woah, what are you using for the isolation?

It’s not OS-level sandboxing or containers.

Each sub-agent is executed as a separate CLI invocation (e.g. Cursor CLI or Claude Code), which means it gets a fresh model context window. The isolation is purely at the LLM context level, not process or filesystem isolation.

The main agent passes only minimal inputs (file paths, task instructions), gets a concise result back, and keeps its own context clean.

neat - but the link seems to be broken btw