@marcinteodoru: Full setup is in my profile. This is Colibri, and it can run absolutely massive AI models locally by using your SSD, RAM and VRAM together. We’re talking 744 BILLION to 2.8 TRILLION parameters on consumer hardware. It’s still slow right now. But if this gets faster, local AI gets very interesting very quickly. And NVIDIA might want to pay attention. 👀

BobZ :

getting it to work at 1 token / 5 minutes isn't the hard part. never was.

Ten Ten :

ok but howany tokens per sec?

Eric the Shellback :

Colibri is great, but for my set up, Im gonna try qwFNfer. Looking to get Qwen3.8-Flash-Next on a Dell 7060 i7-8700, 40gb RAM, A2000 (12gb) GPU, Crucial P310 1tb 2280 nvme ssd. Crossing my fingers for a minimum of 10 tok/s. Im currently using Ornith1.5:9b with 35 tok/s. Big difference between the two...

user9762258332577858437 :

1 token per month

Administrator :

1 token every 10 seconds

Farook Mohammed :

People are building systems for this exact purpose, some are pulling off things on the software side, others are building on the hardware side.

feelurdreams :

so we can now crack any crypto wallet?

Nathy Mosquera :

You do understand that it remains a hardware limitation issue? This is putting a moped engine in a tow truck. And then hopes the moped engine will be more powerfull. Even normal ram is not efficient (slow). Cpu power is also slow. Gpu hardware is very efficient but also expensive. So even Vram is not the issue, even a rtx 2060 with 1 terabyte vram will be slow for this model, even a 5090. I hope you understand.

culturecrush :

But how fast is it

Frontiersman :

SmiteVils :

I tried Strata. It works well

Patrick2018 :

Runs at 1 token per day 😏

anthony_s2000 :

lol ramdrive.sys enters the chat.

Dr.Papa :

Dude r u serius ??? XD you will get max 1t/s it cannot be optimized… read how inference works

HyperionX :

lol 10 tokens/s

loudr0ar :

So is it a stand alone app? Or will it run in lmstudio ?

TMM :

I built something similar in 1995 I have a386dx PC with 2MB Ram and was unable to run Aladdin game because it required 4 MB ram, so I built a driver that would provide Ram on disk, the game ran and was 1 frame per second, I was very happy 😊

chunky fresh :

I am waiting for the Nvidia arm cpu computer

Ev1L Un1c0rN :

1 token per month?

sifudanferrera :

AI

⠀ ⠀ ⠀ ⠀ ⠀ ⠀ :

AI

🌍 Deans World 🌎 :

How much space do you need?

Shannon Squatch :

AI

🎤❄️Holger❄️🎤 :

Yes, but it's very time consuming

MonoluminAI :

That is unusable 😓 Is it works yes… but so slow that it’s Mabe good only for students…