I am sure he is quite happy to have people disagree with him on HN. He usually wears it as a badge of pride. We might even have a follow-up in a couple of days about how these techy weirdos lost the plot.
This just proves that you can cram pretty much anything into the client/server dichotomy if you just define "client", "server" and "request" broad enough. Similarly, I remember how desperately people tried to argue that Emacs follows the "Unix philosophy" as long as your LISP functions are doing just one thing, and do them well.
I don't know what you would gain from these things. Emacs follows the idea of LISP machines, I think that much anybody can agree on. From there, Emacs can be or do pretty much whatever you want. It's excellent in communicating with CLI tools - you can call that client/server if you want, but I wouldn't know what you'd gain from that definition. The reality is that Emacs has gone through a lot of fads and hypes over its decades of existence, and each time, it has taken something along the way. Heck, there's a whole semantic parsing engine buried within (CEDET), which nowadays is pretty much unused, because then LSPs came along, and now we have agents (which Emacs btw is a pretty decent frontend for).
Unix philosophy clearly favors and privileges certain methods of composition and interoperation. C-calls, program invocation, files and sockets/pipes. Calls inside various vms like lisp, java, python or wasm are not among those.
Emacs is a more usable interface to all those methods of composition than, say, a terminal emulator. Take dired for example; it presents the verbatim output of the ls command but allows me to interact with it in a much smoother way to rename, copy, and run arbitrary shell commands on them.
Why is it acceptable to compose simple modular tools in a bash script but not in an Emacs script?
> Except that, for more than two years, the encryption key remained resident in memory across suspend, leaving it there for the taking by anyone who seized the still-powered laptop.
I don't get it. Obviously, the laptop is locked when it resumes, how is that key "for the taking by anyone"? I'm not saying it is impossible to read out RAM from a locked laptop, but surely not by "anyone".
There are attacks that allow dumping RAM if the device is powered on though and you have physical access. Depending on config it may be very easy (just plug in a dumper over Thunderbolt on USB C and do direct memory access) or hard (freeze and swap physical RAM to an unlocked machine).. but the idea was defense-in-depth here; a well configured device should both be hard to dump RAM on and it should not give encryption keys if an attacker succeeds.
Anyone with physical access. I think it is understandable from the phrase.
There is a common misconception about how lock-screens in general work - they usually just prevents using the current hardware and software as it is to access the current OS. But the disk encryption is the main thing that prevents modification and other kind of access to actual data. And if the disk encryption key is lying in the memory, then effectively, the disk encryption is bypassed if someone can access the machine physically and assuming that there are no sufficient tampering protections in place for that machine.
Anyone with physical access, significant tools, and experience. The FBI has people who can pull data out of memory after freezing the RAM but the average laptop thief doesn’t so how serious this is depends significantly on your threat model. If you’re not a major criminal, bitcoin whale, or intelligence target this is almost certainly academic.
While that is true, the fact is that encryption is technically useless for anyone who is not constantly powering off and on, which is surely a bunch of people. That this is not widely understood is IMO a problem. And excellent reason for Debian's feature.
PS: Downvoting is not a rebuttal. Disk encryption is not effective security on a suspended (sleeping) system. That is a fact.
You’re being downvoted because you’re taking a narrow edge case and saying it invalidates all usage. Disk encryption is not “technically useless” when it works for 99.999+% of the people using it who aren’t targeted by professional attackers. Most people expect it to protect them against an unskilled thief or when they resell the device, and it works for that.
People in those high risk categories already need more than disk encryption anyway, so this isn’t even the critical piece for them! (Consider how likely it is that I would have the resources and access to freeze and extract your RAM but not, say, the ability to record you entering your password using the same access. Yes, you can come up with scenarios where that makes a difference but it really underscores that you have many additional things to worry about if this is your threat model.)
Yes I understand all that. I don't have secure boot, so I'm not protected against the evil-maid attack either. I know that too.
It's why I used the word "technically", by which I meant "in the absolute". What word do you propose instead? Encryption that can be worked around by anyone with skills and commonly available equipment is technically useless. It offers some protection (the thief will probably reboot or unplug) but ultimately it's null and void.
>Most people expect
This would need to be sourced. I say most people expect "encryption" to be as secure as the encryption password. In the case of an unattended sleeping computer using Linux with FDE enabled and the screen locked, it's not. I'm not sure most people know that. I believe things are different on, say, iOS. All of this was the rationale for Debian's (buggy) feature.
I would talk about threat models. There’s a bunch of detail to cover but the short of it is “you’re protected when you resell the device or replace a drive” or “you’re safe from a laptop thief in a coffeeshop but not the Mossad”. How you get there is the step in the conversation after you decide which of those concerns you.
> Anyone with physical access. I think it is understandable from the phrase.
Sorry, I'm probably dense, I still don't get it. You steal a laptop, you open it, the screen is locked with a password/fingerprint whatever. How do you read out the RAM from that laptop?
Several options. One is you restart and boot from a live system where you are root, and then dump all memory. This is described in the paper with the witty title "Lest We Remember: Cold Boot Attacks on Encryption Keys":
Other options: DMA attacks. Also you never know what the Intel Management Engine hidden in your computer is doing. It's running a version of Minix you don't have any control over, and it has full access to memory.
Well, when they announced it (7 months ago) I got laughed out of the room when I said this will be at least 1k$ because of the RAM crisis, and people quoted famous Youtuber "Moores Law is Dead" that this thing has a 300$ BOM and will be 600$ max, probably just 450$...
I can understand the joy of running things yourself, and can also see the privacy aspect. However, I pay ~3$ per 1M/tokens for that model on Openrouter, and it's not even quantized. A refurbished 3090 and a 5080 will set you back well over 2k, not to mention the electricity to run them...
> I pay ~3$ per 1M/tokens for that model on Openrouter
I think the thing is, there's an unspoken "for now" at the end of that sentence and people running this locally are hedging against that "for now". Some people prefer to feel that they own the means rather than rent the means, even if the one they own is worse than the one they can rent. Especially with today's Fable news and the harsh realisation that the "for now" is dependent on very many unpredictable factors, where the one you have locally costs you capital today and a relatively predictable run-rate (made more predictable with on-prem solar for example), but should otherwise work predictably forever.
I'm not saying that you're wrong to do what you're doing, just that many people have their own lines in the sand where renting vs buying makes sense, and it doesn't only boil down to a rational (or irrational) financial decision.
You're treating open weight inference providers the same as proprietary ones. They're fundamentally different business models. Proprietary companies have an incentive to subsidize actual inference and training costs in order to gain market share. The few dozen or so companies selling Qwen models by the token on openrouter are in a commodities market.
If suddenly the CCP declared a total digital embargo on Alibaba's Qwen models or even if for some reason all of mainland China (and Singapore) was completely unreachable from the rest of the world, the dozen or so companies selling Qwen by the token elsewhere in the world could continue business as usual.
I don’t know anything about the open weight host business model. Do we know for certain that the folks selling inference by the token are really selling them in an upfront and profitable way? No subsidies from harvesting the info, to sell to the model trainers or anything like that?
Or subsidies from hopeful investors sweet-talked into not understanding the commodity nature of the business they are investing in. But that does not change much about the general assessment.
Chances are the typical story goes founders start fully believing that they would succeed with their own innovation but slip down a gradient towards commodity provider without really noticing themselves.
I was thinking of user-side regulations as well, not only provider-side ones. I could imagine a world where a government rules that you may not use LLMs for anything, which would be much easier to get around if you have local means.
I've spent the past week trying to scheme a way to get affordable local inference of something useful (Qwen3.6-36B-A3B) for ~$500 and have come to the conclusion that it simply isn't viable. A pair of power-restricted P100s in a workstation gets close but the workstations themselves are expensive and rare as hen's teeth (not to mention loud and large). I think early '27 will be when things open up as the hardware market unclenches and further strides are made in small capable models.
I'm running Qwen3.6-35B-A3B on a very ordinary desktop PC (32GB DDR5, 8GB Radeon 6600XT) and getting a useful 15-20 tok/sec out of it. The MoE architecture and auto offloading from system to VRAM is just fantastic. Unsloth Q4_K_XL.
The Qwen3.6-27B is unbearably slow as it doesn't fit in VRAM, though, i think the MoE is very easy to run.
It is also extremely nice that you can just `apt install llama.cpp libggml0-backend-vulkan` now too.
I wonder what parent poster means with „useful” and what he actually tried? Feels like he was just comparing some benchmarks.
Yesterday I downloaded Gemma4-26B with Ollama on quite rusty desktop with 1070 8gb and 32gb of ram and Core i5-9400.
I drop photo of my water meter and tell it to read the value and serial number. It was far from instant but it was also easily under 3 minutes and result was correct.
Earlier like in February I was trying the same photo with Gemma3 on the same hardware and results were bad.
> I drop photo of my water meter and tell it to read the value and serial number. It was far from instant but it was also easily under 3 minutes and result was correct.
"Useful" as in "has a use that isn't just for show". It takes me two seconds to read a photo of a water meter. Having an LLM read it for me in 3 minutes isn't useful. Similarly small models are capable of tool use (e.g. web searches) but their synthesis leaves much to be desired. As an example I'd ask some small models to find examples of products with specific characteristics and they'd come back with only one or two because they discounted other possibilities incorrectly by reasoning themselves out of it.
> Feels like he was just comparing some benchmarks.
I don’t get agent to read the meter for me - I can do that when I take the photo.
I send the photo to a bot that ingests photos from me and stores readings for me with date and time so later I can ask „what was last reading” or what was the usage between x and y dates”, without me having to make a perfect photo, without me having to dabble with OpenCV.
Even if it takes 30mins it is still useful for me.
An R9700 is $1350 and can get 100 TPS running Qwen3.6-35B-A3B Q5 with 130k context window (with room to spare) with a bit of fine tuning llamacpp-vulkan, but llamacpp's repository instability and lack of real versioning frustrates me.
In terms of electricity, if you aren't using it, even with all the vram loaded, at most your wasting about 30 watts or so.
Prompt processing a large uncached context is annoying, which is why I forced a lower context window, but I don't know if it's any worse in performance than the cloud models I've used.
There's a niceness, to me, knowing I don't have to rent it anymore. If you rent it, the terms can change regularly.
I use local models to explore, hosted models to refine. I somewhat envy those who can sustain local models (q8 120b+) running as a hobby.... for me, the practical path is a better SearXNG setup and knowing my routes forward.
It’s a personal hobby project why should we care this is how someone chooses to spend their free time and money? Lots of hobbies are expensive and pointless if you think of commercially available offerings. That’s why it’s a hobby and not a small business
I think it's important to be able to do both so you can stay in control of the price to value created relationship.
In last year, some people were publishing aider /ollama/open router [1] and now thankfully people are publishing all around about pi/qwen/llama.cpp/openrouter. It's widespread.
Any sane crypto miner undervolted and underclocked their GPUs for efficiency's sake; if anything, they went through less wear than, say, regular gaming.
Openrouter doesn't give you access to the models internals, i.e. complete control of logprobs, sampler stack, any PeFTs.
Openrouter fking sucks and I don't know why people here act like it's so great. Stop using it if you care about local AI and accept that the cost you'll pay for tokens is higher than you will when consumed via any cloud. That's the price for privacy, control, and better quality via inference time optimizations that otherwise aren't available.
> Openrouter doesn't give you access to the models internals, i.e. complete control of logprobs, sampler stack, any PeFTs.
Openrouter gives you access to whatever the inference provider gives. They're just the middleman. Many providers give logprobs if you ask, it's in their API. And yeah, no Peft or Lora, but that's an entirely different product. And some of the inference providers do that directly.
> Openrouter fking sucks and I don't know why people here act like it's so great. Stop using it if you care about local AI
But the whole point of openrouter is that you can run models by the token and you don't have to care about local AI? Sounds like you're more upset that people aren't making the same calculation on privacy and local control vs cost and ease of use.
That engineer went on to create Brave, a browser that pays you Monopoly money for watching ads, injected affiliate links, installed their commercial VPN without asking, and leaked DNS traffic when using Tor in its "privacy" mode. I'd say Mozilla dodged a bullet there.
What does Firefox pay you for piping your keystrokes off to Google? BAT is a reward for your attention; far better arrangement and exchange than what has existed up to this point. It's not perfect, but what's your solution?
> "injected affiliate links"
You seem to be a little free and loose with _facts_. Rather than exchange your data for revenue, Brave explores revenue streams which won't keep us up at night. One such consideration was affiliate links. We had a couple (quite literally a couple/few), that would appear when you typed certain crypto-related keywords into the address bar. When suggestions were offered, so too would be our affiliate option.
This solution presented a means by which users could support Brave without involving their data. Unfortunately, a UI/UX bug caused the affiliate option to appear even for a fully-qualified domain, which meant a user who quickly typed a URL for which we offered an affiliate link and mashed Enter, could unintentionally have selected the affiliate option. That isn't _injection_.
The issue was identified pretty quickly, and a patch was sent out. Guess how much Brave made from the buggy behavior before it was patched? I'll help you: $0.
This one is actually somewhat true. We did indeed ship an inert service for some Windows users. The goal was to have the VPN option be immediately available to users who wished to purchase it, as a means of supporting Brave. Details are in the GitHub issue: https://github.com/brave/brave-browser/issues/33726.
> "and leaked DNS traffic when using Tor in its 'privacy' mode."
Oh, this is one of my favorites. It's a classic story with depth, misdirection, unexpected side-effects of decisions made years in between, and more! This one is the type of thing I would have expected to read about in _Joel on Software_ many years ago.
So, we shipped a browser with a "privacy" mode, much like everybody else. But, we weren't fans of the common approach used by Chrome, Firefox, Edge, and others. Their approach doesn't really make you _incognito_, or _private_; it just creates an ephemeral account locally and basically does some file-system cleanup. We wanted something stronger!
As fans of the Tor project, we opted to bake-in support for Tor as an optional enhancement to private tabs. This would give you one extra, super-thick layer of incognito-ness. Tor Private Tabs were shipped back in mid 2018, and the next couple of years were pretty awesome. Brave users who enabled optional Tor support enjoyed a superior experience to that found in other popular browsers.
Years later—as the tracker wars waged on—some data-harvesters got the idea that they could evade detection by way of CNAMEs, giving them first-party privileges. So in late 2020, Brave shipped CNAME decloaking, unmasking more trackers than Mystery Inc., and dramatically expanding the privacy moat.
But the story wouldn't be all that exciting if it didn't have a twist, right!? Brave's new CNAME-decloaking didn't consider the Tor scenario, and performed DNS lookups outside of an existing proxy!
While the combination of these features didn't make Brave as porous as ordinary "incognito mode", it did punch an embarrassing hole in the Tor boundary: page traffic still went through Tor, but CNAME adblocking DNS lookups accidentally went out through the user's normal DNS path.
For that narrow slice of activity, Brave drifted uncomfortably close to what Mozilla calls "private browsing": https://support.mozilla.org/en-US/kb/common-myths-about-priv... ("Private browsing [in Firefox] doesn't hide your activity from your ISP, mask your IP address or location, or stop websites from identifying or tracking you…)
I wouldn't say these are "basic bugs". The first is specific to using 'rrsync', and the second is when using the rsync daemon, and I can't remember when I last saw a system using that one (yes, I'm aware there are still use cases for using the rsync protocol, but I would consider it pretty obscure nowadays).
You could argue that he should've bumped the version more and should've done a longer beta test, but on the other hand, these were mostly security fixes, and I can understand he wanted to get them out there rather sooner than later (also "doing a beta test" is easier said than done - how do you get people to run a test version of rsync?).
Nice post and technically impressive work. I agree we need to understand the build pipeline and be able to do things locally. However, depending on your electricity cost, it might not make sense financially. These old servers are not energy efficient at all (I'm guessing that old Xeon server will easily pull 200W on load), and that model is currently at 0.1$/0.3$ per 1M tokens (with 76 tps and 262k context) in Openrouter (also, these servers are LOUD).
EDIT: I stand corrected, 200W is apparently way too high of an estimate. I used to run a bunch of old Xeon servers and they slurped watts like crazy, but I can't remember which ones exactly those were.
OK, then you're in luck. I had a bunch of old 1U rack servers and even in the next room it was too annoying to run them (they had a bunch of 40mm fans which always ran at full speed, because in a server room, no one can hear you scream).
Could it just be really bad cooling? Looking at 9800X3D, it seems like it's running in a similar range wrt TDP unless you really push the 9800X3D. I'm comparing with desktop cpu's because that's what my workload is. cpu governor is set to performance (no schedutil). No audible change in fan speed during heavy compilation or gaming (very silent humming), and i don't have any fans beside cheap intake, cpu and exhaust fans (1 each) + an excessive amount of dust.
These servers had no fan control whatsoever, they always ran full blast. That's not untypical for rack servers, because as written: they are designed for server rooms, and you're supposed to wear ear protection there anyway... Yes, I could've modified them, but I ditched them because running them simply made no sense (especially the high idle power consumption was ridiculous).
Yeah, 1u is gonna do that. Get something that can accommodate a big tower air cooler such as the Hyper 212 and your airflow will be quieter than the disks.
I don't run it anymore but my old server was a dual xeon (with two of those coolers crammed in) and I rarely heard a peep out of it.
Only when you remove it from the original server or enable low fan mode (if available). Most 1U/2U cases will happily blow at full speed well over 90db.
You likely need to replace the flow-through server chassis system with an active "normal" cooler to achieve a bit of silence.
85W might be about right. My old server CPU is in the same ballpark and compiling kernels it reached about 90w in power usage. If you want to keep it running: idle is not very low power unless you have one of the "low power" L versions, keep that in mind.
Get a 4U case, many options if you want to combine it with a NAS. Not hard to cool and keep somewhat quiet. If you can store it in a closet or something that helps too.
Well, you can use it for lots of other things as well.
Compared to the cloud you can probably save up to buy a new server every month. And don't underestimate the gains of having something to experiment on and play with.
These servers are loud if you're trying to fit them into a 1U or 2U, which requires high speed fans to generate the necessary static pressure to push air through the case. I run a similar setup in a 4U case with slow 120mm fans and it's fine.
Well, if Apple killed it, Lenovo killed it even more. I recently was looking for a laptop for a student. The Lenovo E14 Gen7 is 800 Euros here in Germany (where prices are always higher, the MacBook Neo is 700 Euros), it has 16GB of RAM, 1TB SSD, a 2.8k IPS display, a Intel Ultra5 12core CPU, and it has a repairability score of 9/10 from ifixit. Framework doesn't even come close to that package.
Same thought, as an owner of a similar Lenovo, that's top bang for the buck. Also, matte screen and hinge that opens 180 degrees is something the Neo and most Macs doesn't have.
Though I assume the Apple clientele is always different than those shopping for PCs, and doesn't care about specs, they just want MacOS and the Apple ecosystem, most likely they already have an iPhone or are planning to get one anyway so then a Macbook is the only thing on their radar. Those people aren't really shopping for PCs anyway unless they need some Windows/Linux exclusive apps like CAD/CAE.
But if you want to run linux and game then that Lenovo would be a good deal.
Similar to the Framework, it has its own niche clientele who values the company motto, tinkering and repairability aspects way more than the value proposition. Most likely they run Linux too.
It is funny how Mac OS is a draw for some, when it is the main reason I don't use a Mac. Their hardware is excellent, but when I've tried using a Mac as my main machine, my productivity suffered. The only part of the Apple ecosystem I wish I could get on Windows is iMessage, and maybe FaceTime.
> The only part of the Apple ecosystem I wish I could get on Windows is iMessage, and maybe FaceTime.
It annoys me that these are such a draw. There are a dozen other viable messaging and video call apps, but there's always someone who feels like spending two minutes to install and activate one is a major imposition.
I like Apple hardware. I like the Apple integration. I like the hardware quality. I LOVE the silence of the M series machines.
But for me you’re right. More than anything, I’m not giving up Mac OS. Despite Tahoe, which I do severely dislike, I’m still far happier using it daily than Windows or Linux.
Until that changes, or the hardware gets bad enough (it’s going in the other direction), I’m not leaving. I don’t even look at other options for my real computers.
“Toy” computers that I want to throw Linux or BSD or something on just to play with, yeah of course. But not what I want to use all day every day.
Framework is definitely premium-priced, but I don't think most people are cross-shopping the Framework 12 (a 12" convertible tablet) and the Thinkpad E14 (a 14" dedicated laptop).
No such model exists. The Framework 13 comes closest, but a 13" screen and a premium shell would compete more directly with the Thinkpad X13.
Direct price comparisons get tricky because different buyers care about different details. I really like the Thinkpad's Trackpoint, for example, but I also like the Framework's 3:2 aspect ratio. I'd have a hard time choosing.
Dammit. I got an IdeaPad of similar price in december 2024. It didn't have one of the fancier displays from the era but still a decent option, it has 16Gb and I thought I'd try a Ryzen mobile thing that time. Wish I'd gone for the Thinkpad E series had I known about it then : that lower-end IdeaPad feels like trash.
SSD IO is sluggish, fans always spin when plugged in, audio crackles if I so much as scroll a page while a youtube video is playing, the keyboard might be the worst I've touched in many, many years, the 3.5mm audio jack wore out into intermittent connectivity within a couple of months. At least the display still looks good.
Went through the windows optimization motions with it too. My x230 with an i5 still has lower and more stable DPC latency and has remained my DJ laptop.
