It’s not a rust thing. Custom markup is usually implemented in rust frameworks using macros. If mixing macros and logic becomes a problem, we can change how the macros are implemented.
Yes, I saw that's it's via macros, but my point is that it reminded me of some web development practices from a much earlier era, and why we moved towards doing templates and separating app logic from presentation
The trend has been to move back to this style in general (see most modern JS frameworks). The main point is less about combining code and view, but breaking up the page into many small components and pushing data loading down into where it is needed vs up front.
Doing that enables a bunch of performance tricks like starting to steam the page before data i loaded and concurrently render components.
If you have to split your page into many small components, a separate view file for each becomes tedious.
20% slower than the M5 is still a great result. I'm still using an M1 macbook pro, and it's more than fast enough for editing film, browsing the internet and writing code.
> I am confident a human and about 5 minutes with chrome debugger would yield better results
It really depends which human. I've worked with very few engineers who were good at this sort of optimisation work. A depressingly large percentage of people who make websites for a living don't really understand how http requests are really processed, or how to read and use the chrome profiler and benchmarking tools.
Claude isn't as good at optimisation work as someone who really knows what they're doing and goes deep on a problem. But I'm optimistic that it will help plug a capability gap in teams which don't have this sort of expertise on hand.
(That said, the chance that people actually learn this stuff is going to also go down if people get used to outsourcing this work to claude.)
The job of making a website is a large part something that should be given to people who have majors in art, English, and similar things. Well, there is some value in an engineer to build the whole site, but that's more of a framework task. The vast majority of the content is not an engineering thing to do. It is a job of people who are experts in language, art, journalism, those types of things that have very little to do with engineering.
In short, if most of the people working on your website could do those things you named, you have a problem that you have hired the wrong people for the job.
I think it's genius to use this element of computer security in narrative fiction. But I think they still had comms across the ship. The cylons could still clone the captain's voice and have random crew members perform arbitrary tasks by directing people to do so over the ship's radio.
Most of the in-ship comms were 'old school' (in our terms) analogue devices. They actually had to pick up speakers. Internal speakers were wired, not radio.
Sometimes I wonder if filmmakers have the right of it. They need to show what's going on in the computer to a distracted audience in a heartbeat. When they pull that off on screen, I wonder if us software devs should be taking notes.
Isn't that what dashboards are for? They may not look like simplistic LEDs, but being able to "see" what the automation is doing is something everyone that's ever made automation wants to do. It starts with making logs before migrating to live dashboards.
Its basically that staring at the log patterns in graph form or system stats you get a pretty good idea of what 'normal' looks like and anomalies stick out pretty quickly.
Though having a person interpretingthe degradation progress and relaying it to the rest of the team could be a usable scene.
A guy I went to high school with used to do that for a living. I have no idea what they call it, but it's basically theatrical UI, where the screen has to act and usually deliver a bunch of exposition.
I can't recall where I saw it, there's at least one company working on this issue specifically for nursing staff. Everything beeps in hospitals, but there's no rhyme or reason to it, so room call responses are delayed, serious alarms can be missed, etc.
IIRC, the company's idea is to create a harmonious sound bed when everything's fine, and make different kinds of "out of the norm" alerts use a variety of sounds so that nursing staff get more info about what's wrong before finding the thing thats sounding an alert and looking at its screen.
This company's goal is to ... change an aspect of hundreds of different million-dollar highly regulated machines made by dozens of different multi-billion-dollar worldwide companies? And somehow make money doing this? Good luck, I guess.
> It is unthinkable to me that anyone believes there is such a thing as computer security after so many years of nonstop hacks and leaks.
Of course there is. For example, SeL4’s security and reliability proofs still hold in the world of LLMs. The problem is that most software isn’t written on that firm foundation. Instead, most software is made by people with the philosophy of “if it looks like it works, ship it”. You don’t get secure software by working like that, because security vulnerabilities aren’t visible.
We - humans - know how to write secure software. Just like we know how to make safe aeroplanes. The problem isn’t that we lack the capability to make secure computers. The problem is we don’t have a culture of security. Secure software is - somehow - niche. And as such, it’s much more expensive. And nobody wants to pay.
I'd argue the management has a point there; without a pressure to ship, nothing would ever get released, because computer security has not yet understood the basic concepts that every non-computer security work does:
- nothing is, can be, or even should be 100% secure; the optimal rate of security incidents in society is not 0 (with apologies to 'patio11)
- security is a simultaneous trade-off against costs and usability, and those two other factors are more important:
-- security is achieved primarily through raising costs for attackers to beyond profitability, and reducing impact of such attacks (due to "not in isolation from the world" below, this also mostly translates to costs)
-- if "properly secured" (in the current cybersecurity sense) product/service cannot fulfill its function anymore, then you may just as well not make it; either way, no point in paying you for security work
- security isn't done in isolation from other systems and the world at large; "if this happens we'll go straight to filing crime report with the police" is perfectly legitimate security measure (even if it works somewhat less well on the Internet); similarly, "this is secured by us having insured against it" is also a valid solution to some security problems
This is all fine and good and the sorts of conversations we were having on tradeoffs like any other engineering org. Theres always been pressure to ship, but now its that various pockets of AI Believers have popped up, egged on by a manic management, containing such beliefs that the code no longer matters, that it’s possible to move fast and fix it up later, human review isn’t important anymore, and that lines of code is a valuable metric.
Those running projects with these beliefs are sputtering and producing impressive PoCs that struggle to make it into production - either through underestimating the amount of detail needed to scale, or often throwing away good practices in favor of letting LLMs handle tradeoffs that later make changes slow to a crawl.
Theres plenty of good ways to utilize LLMs to speed things up, but so many teams got so incentivized by management to move fast at any cost that they’ve thrown out “load-bearing” good practices for software. That bet hasn't been paying off the way they’d hoped. It’s now clear that they thought they’d be able to massively downsize the engineering orgs. Massive token spend is giving very little RoI and now like other companies they’re trying to rein in the biggest spenders who are often not producing value.
As much as you're right, the attitude in most software development is not "lets make this as secure as reasonable," it is "lets make this barely functional and then move onto the next thing."
It's not as if best practices aren't well documented, or as if CVEs don't come out every day, or as if the information is somehow unavailable to even the most junior devs to take basic security measures.
Not all hacks are caused by pure negligence, laziness or stupidity, but most of them are. Even a little effort goes a long way.
My grandfather spent a couple decades as a builder, ran a construction crew. Whatever the project was, he wanted to know everyone he hired personally was going to reinforce and report to him anything they had the slightest doubt about. "Always hammer in an extra nail" was basically his motto.
What we do ain't that different. The difference is that when an apartment building collapses, it's bigger news than when a govenrment database does.
The incentives from society are all about it. What company has ever faced serious consequences for hacks or data leaks? A cheap fine is just an unlucky cost of business.
I like to think behind every Dev anxious to ship half baked software sits an omniscient middle manager with a vague idea of what the product was supposed to do, maybe
Guess you don’t remember the days of ssl on login pages, ssl strip, exfiltering data via JavaScript prototype pollution, and a million other things like that.
Only just when we started to have a resemblance of security we got agile and startups breaking things (making rubbish software to capture a few bucks faster) and now vibe coding and llm assisted hacking.
The point of my, arguably rant, is that there is nothing new under the sun.
It should. We went through couple of cycles of "things are bad, inmates are running the asylum" before, and nothing of consequence happened. The world still goes on.
It's not a guarantee this time will be the same - but it should temper the worry somewhat.
I thought security clearances are a dime a dozen, and "all FBI employees" list is full of administrative work and basically 80% mirrored on LinkedIn? (Yes, the remaining 20% - or however much - leaking is a problem.)
Nothing new under the sun, it's the banality of evil all over again my dude. We never left scary waters, but they do seem to be growing more agitated. Is this the storm before the storm?
No because it's a problem of human communication. Taking humans out of the loop creates other social problems that have been thoroughly documented in cyberpunk mythology, not actual a solution, just trading one big problem with multiple little ones.
Noted, but I think it's important to comment seriously because this is an important topic, sorry I didn't acknowledge the sarcasm, should have opened with something like "I for one welcome our AGI overlords"
PS. Likewise, apologies, perhaps I was at fault for style: There was an underlying more serious point ...
... that, it is indeed a serious problem, that folks shouldn't be having their souls (or anything else for that matter) crushed, and that it indeed would appear to be an issue of conflicting incentives vs. management.-
I believe the technical term is “Move fast, and break things.” MVP is a huge disaster. I can see it working for applications that don’t process PID, but only an idiot ships data handling software before it’s been dragged through a lot of testing. I tested my app for two years, before finalizing, and an LLM still found a couple of holes (minor ones, but ones I missed).
After the DOGE debacle, I suspect that all the previously really secure stuff, is now out there, too. In fact, I wouldn’t be surprised if some of these leaks, came from that.
The issue is that in consumer and enterprise software, move fast-and-break-things outcompetes secure-by-default every time. Critical infrastructure needs to have a different set of priorities, but it’s very hard because the expertise is so thin on the ground. Why would anyone with the expertise to make these calls bang their head against the wall trying to educate bureaucrats about these things for $150k a year when they can easily make multiples of that in big software companies that don’t own that level of risk.
The incentives have to change. Any breach regarding PID should have fines as a percentage of revenue of the company. Any breach intentionally covered up and found out later by a third party should mean jail time for the C level. Yes, I know it is hard to make such laws "foolproof". And yes, in the current political and economical climate it will not happen anyway.
The real problem is that even for companies that wish to pay more and wait more for secure-by-default can't easily tell the difference.
The only solution I can come up with is some form of certification or paid code review from a third party. I know that at least for Windows prior to 7 Microsoft actually allowed some parties to come in and check the code/checksum on an air-gaped computer. We somehow moved to "trust more" in the last decade, and now we can trust nobody
I've yet to see any form of certification or paid code review I'd be willing to bet critical infrastructure on. And working in safety critical software, that's not for lack of trying. Good review is usually harder than building a working system and the asymmetry of offense and defense applies to anything you miss.
Not really. It’s likely that the dumped (and compromised) data might contain things like keys and URLs that could be used to pry open other sites. Blackhats have become really good at following breadcrumb trails, and using “innocuous” clues to ascertain much more dangerous access.
LLMs have been a huge force multiplier. Here.
If that data got out (which probably happened within hours of the data being dumped to insecure storage), then it’s probably already been analyzed and used to leverage access.
> Secure software is - somehow - niche. And as such, it’s much more expensive.
Its not expensive because its niche, its expensive because its hard. Its a lot easier to learn a bit of html and javascript and knock up some web projects then it is to become proficient at all the things necessary to be good at security. Generally it also takes consulting with maths experts who have spent their life studying cryptography and as such command a decent wage.
There's very little, if any, incentive to do anything more securely than absolutely necessary. It is only a problem, when it is a problem and is treated that way in nearly every org I've worked in.
I agree that security isn't bottlenecked by our ability to learn. We generally fail at the grit needed to do it day after day, especially when the benefits aren't immediately visible.
"Its not expensive because its niche, its expensive because its hard."
I'd counter and suggest it is expensive because almost nobody gives a damn about the first 3 layers in the OSI model, and have pushed most security responsibility up to layers 4-7. That's roughly half of your attack surface still exposed.
The bank has the best doors, the best locks, and the best cameras, and it is patrolled by a guard who props the doors open to so he doesn't have to keep fooling with the locks and points the cameras the other way to extend his smoke break. SeL4 would be another system used by humans.
It's always possible to break a perfect system by moving an additional layer of abstraction outward, and attacking one of the assumptions upon which it's built. Some of our era's highest security systems - game consoles - have been broken by undervolting them until the logic failed.
It's often possible. But not all systems are vulnerable to undervoltage attacks. For example, I don't think the iphone secure enclave is vulnerable to this.
And good security uses "defence in depth". Multiple layers which each individually need to be compromised to break the whole thing. To hack chrome, you need a vulnerability in the renderer or VM. Then you also need a sandbox escape, and a way to use that to attack the browser's parent process. This is much harder to do.
I'd stick with always. Defense vs offense in anything reasonably complex suffers from one issue that simply cannot be overcome. To defend, you need to defend against every single possible imaginable attack, from now until forever. To attack, you need to find a single attack that works. And on a practical level all systems need to be accessible by somebody, yet that somebody is himself also now a part of your security structure and is never going to be 100% reliable, both in terms of corruption and incompetence.
> not all systems are vulnerable to undervoltage attacks. For example, I don't think the iphone secure enclave is vulnerable to this.
Then there's decapping / depotting, a world of different types of microscopy - some destructive some not, directed EM attacks, etc.
> And good security uses "defence in depth"
And automation has enabled "offense in depth"
> To hack chrome, you need a vulnerability in the renderer or VM. Then you also need a sandbox escape, and a way to use that to attack the browser's parent process.
Or you just phish the user into installing your exploit. There's always another layer. Always a potential exploit. Because ultimately the same properties of the universe which permit computation within a closed system allow for predictably observing and influencing it. The expense and hassle of doing so are widely variable, of course.
> There's always another layer. Always a potential exploit.
So what? Most attackers aren't nation state adversaries. They're some kid in Wyoming messing around with deepseek. We live in a world where most exploits happen because someone was running an unpatched, 8 year old copy of wordpress. Because they put their insecure mongodb instance on the open internet. Because they used admin / "12345" as the username and password. We don't need to make hacks physically impossible for a nation state adversary. Just really, really difficult and expensive to pull off.
Honestly. If people talked about physical security like they talk about computer security, you'd have people telling you that, because walls can be physically smashed through, they don't bother locking the front door to their house.
> Honestly. If people talked about physical security like they talk about computer security, you'd have people telling you that, because walls can be physically smashed through, they don't bother locking the front door to their house.
Sure, but also if there are a dozen bikes locked to the rack all costing $1000 and a $500 bike just sitting there completely unlocked, the average thief is probably gonna take the cheaper unlocked one and ride away.
It’s not hard to get into a garage but it’s really easy to steal a lawnmower if you leave the door open all night. I wouldn’t call that thief honest but even the minor deterrent of closing the garage was enough to make you not the target.
How many locked doors have easily breakable glass windows right next to them, nevermind breakable walls? What we need is a red-team waiver, and an AI model, and a budget, and say something like: your site has to be unhacked by the HackerAI3000 bot after 2 days on a 5090 Nvidia GPU.
I once worked on AUD 450M banking project, the root password was kept in a kickstart file and unchanged, root SSH was allowed. The bank didn't care until I told the external security auditor who included it as part of their report.
Decades ago, I worked in a bank in an old building. The door had a card reader for access. You boop your card and the door opened. People would hold the door open for each other all the time out of politeness, even when they didn't know each other. Security told us not to do that, but it's hard to convince people to stop being polite.
I had a laptop stolen from my desk in a place like that once. (Not a bank - but similar door-card reader system). This guy came in in the middle of the day, wearing overalls. He confidently walked through the door after someone, like he belonged there. He walked up to my desk, swiped my laptop and just strolled out.
At the bank, they've replaced the door with mechanical gates and a security guard. The gates - physically - only let one person to walk through at a time. You can't hold a gate open any more. And the security guards stop anyone who tries.
Is it 100% foolproof? No. But it's way more secure. It would have stopped that laptop thief.
There's this pernicious, defeatist attitude that if you can't make a system 100% secure, so you shouldn't try. That's misguided. Most systems can be made orders of magnitude more secure than they are today. It just takes a bit of care and work.
Fine. Make the universe work for it. The whole system becomes more resilient as a result.
Look at our immune system. Incredibly complex and clever, and able to keep us alive in the face of all sorts of pathogens. It exists because of this cat and mouse game, played over millions of years.
There's people in the highlands of PNG who regularly eat each other. Of course, many are thought to have died due to prion diseases. But now these tribespeople seem to have become largely immune to prion disease. Incredible.
It explicitly mentions Kuru as "prion disease", it ignores that the mortuary practice of eating various parts of respected dead has long passed in time .. although the lingering effect on the woman and children that ate portions of the brain in the 1970s, early 1980s, is still residual in a very few.
From an evolutionary PoV a perfect defence is overkill - with two separate defences against prion diseases in that region it's only the rare variation that causes any issue - and that rarely occurs before a new generation is birthed - ie. 'perfect' from the PoV of the selfish genes.
The claim is that people in PNG are "largely immune" to these diseases, where kuru shows that they are not, and the "cure" was a stop in the practice, not some evolutionary upgrade
They are largely immune to Kuru - the number of people that could contract it (ie. the number who ate brain) was significantly larger than the number of people who actually did contract it.
The resistance came about via two separate "evolutionary upgrade"(s).
It was specifically endemic to the Fore, not so much to the Yate and Usurufa, and not particularly at all to other highland people in the general region.
There's a twofer that skittled the Fore, a ~1900 mutation that created a new form of infectious prion proteins, and a local variation that saw less uptake in the Fore of a resistant prion protein (alongside other resistant prion protein).
So, over the highlands region, there was general resistance thanks to several evolved variations, in one specific locale (the Fore) there was insufficient resistance to the mutation that hit a peak of 200 deaths / annum for about three years(?) in the late 50s.
I can't speak to "the literature", I just had a lot of conversations with the people on the ground (Mike Alpers, etc), on again / off again, since the mid 1960s.
What seems obvious to you doesn't seem obvious to me. I'm not a malicious or incompetent enemy. But you will need to explain your perspective for me to understand it.
By the way, there are countless ways to account for humans. There are entire branches of engineering devoted to this. If you don't want someone to leave the bank with a pen customers use for signing checks, you just chain it to the desk. If you don't want the installer to forget to put the pen-chain in, make a photo of the chain part of the checklist required to get paid. If you want to... etc.
The idea is that you determine an acceptable level of risk, then secure to that level. Maybe the acceptable level of risk chosen by companies is wrong. Maybe we need to increase that risk exposure via heavier fines and regulations. Maybe the cost of reducing that risk is too high already. Maybe we need to fund that. Maybe it's too confusing and we need to research better standard practices. I dunno. But this is not some unsolvable problem.
Ask anyone seriously involved in security - whether computer science related, or in general.
A thought experiment:
Think about the most important secrets a country can have - now think how they are still discovered by competing countries, enemies, etc.
As long as there are humans in the loop there is a known weakness.
We know about many famous cases of leaks - like the USSR stealing notes from the manhatten project. But I bet there are thousands of secrets which remain secret. We just don't actually know about them, because, y'know, they're kept secret.
This simply isn’t the case. There are plenty of secrets that every military keeps just fine. See how long we’ve gone without anyone ripping off the F-22? That’s not even a new airframe anymore.
This is EXACTLY what I was talking about. The tech that makes the F-22 an unmitigated terror of the skies is of utmost importance, so it’s still kept secret. The barriers around that information are obnoxious, but effective. What blood type a subsection of your military has is of much less importance. That’s why that data was stolen (See: OPM hack) and our best weapons remain secret.
SeL4 or proof assistant are not panacea. They do not help if assumptions about the task are wrong. And correctly formulating the task in real world is very messy.
So physical security is just as important. I really like how ARINC serial bus on planes work. One can have a reader that is physically incapable of sending anything to the writer. This allows to connect entertainment systems to flight data sensors safely.
In Airbus this system is replaced with Ethernet switches that in software ensures separation of traffic. The software was proven mathematically. But I am skeptical that it is absolutely bulletproof as a client under malicious control can influence Ethernet signaling and may exploit hardware bugs.
> SeL4 or proof assistant are not panacea. They do not help if assumptions about the task are wrong. And correctly formulating the task in real world is very messy.
Nobody said sel4 was a panacea.
My claim is that doing this kind of computer security is possible. It's just expensive and inconvenient. We know how to make computers a lot more secure than they are today. The limiting factor isn't humanity's knowledge. The limit is that barely anyone wants to pay the bill.
> My claim is that doing this kind of computer security is possible.
But your evidence does not support that claim. SeL4 has proven that it is possible to design a secure microkernel and prove its security guarantees. It does not prove that you can build entire systems (filesystem+database+web server+browser) on top of that kernel while maintaining the same security guarantees.
I'm all for improving the state of computer security, and I'd love for capability systems like SeL4 to become more prevalent. But it's only a microkernel, and it's by no means certain that the PeopleSoft vulnerability exploited here required a kernel-level compromise.
I don’t expect every piece of software written to be proven correct like SeL4. But I don’t think we don’t need to do that to get big improvements in the security of a lot of systems. Honestly the biggest insight I take from sel4 is that we can get improvements in security by breaking up a large program into isolated pieces. Give each piece as few permissions as possible, so a compromise of one part doesn’t lead to a whole system compromise. And give them a way to talk. This has big benefits for reliability - since you can fail and restart individual processes. And it has benefits for security, since a system compromise should require an attack of multiple systems simultaneously. And it has benefits for debuggability, since you can add tracing at the comms layer or isolate modules for testing.
Wasm does this. Erlang does this. SeL4 does this. Chrome is built this way. The windows driver model is moving this way. And so on. You want a solid core to build around - which is what SeL4 and beam try to be. Then it’s up to us to use those primitives and build good software. Combine that with a memory safe language (rust, go, c#, etc) to protect against buffer overruns and use after frees. And a picture starts to form of how you can build software that is a lot more secure by default.
I don’t think perfect security is worth the cost for many companies. But so many security leaks happen because of amateur hour somewhere. Bugs happen - I get that. But a single bug in a C++ program shouldn’t immediately lead to RCE with system level privileges. This stuff isn’t rocket science.
You mean something is theoretically possible, but in practice only works at small scale and is otherwise effectively impossible. You only have so many resources for all those big topics.
And after you spent all the world's resources on the "perfect", formally bug-free software, you get hacked via social engineering or malicious insider.
> The limit is that barely anyone wants to pay the bill
Do you even have any experience with how most companies work? SMEs barely have the cashflow to cover their daily expenses, let alone suddenly pay thousands for regular professional security audits and overhauls of their code. This is why security is an afterthought.
If restaurants can't make sure their food is safe to eat, they shouldn't be allowed to be in business.
If builders can't build houses to code, and the buildings fall down, they shouldn't be allowed to stay in business.
If civil engineers build bridges that fail. Or doctors hurt patients. Or police officers shoot innocent people, they shouldn't keep their jobs.
Software engineers are no different. If you collect my user data and it's at high risk of leaking on the dark web, either clean up your act or close shop.
The main issue is that market evolution will always surfaces the most cost-efficient entities within the ecosystem pressures.
That means designing the ecosystem pressures is crucial: things more meaningful than just pure capitalist private-profit logic must by enforced by thoughtful regulation or otherwise the ecosystem converges for private-profit of a small sliver of individuals (billionaires) to the detriment of all other ecosystem members (99% of the world population).
This holds for anything broader than pure private gain, may it be security, social fairness or ecological topics. Sole monetary-value optimization for private gain must be properly constrained or else it results in pure predatory capitalism that implodes society from within, may it be through leaky security, poisoned environments or social unrest.
Yeah this is an interesting argument with respect to why we keep burning carbon. For both software and systems security and rapid climate change, we know the solution. But we haven’t been able to sway the incentives and the system keeps churning out bad results.
> ...there is such a thing as computer security...
> Of course there is. (...continues to talk about software security)
A bit over-optimistic when looking at hardware vulneraribilties like Spectre/Meltdown.
Secure software isn't worth much when it runs on vulnerable hardware (at least Spectre/Meltdown could be worked around in software though, but at a cost.
One time? lscpu on my system lists two dozen CPU vulnerabilities known to the kernel. I stopped tracking them a long time ago when it felt like every week brought another information leak or bypass.
A piece of software Lockheed made gave its outputs in US Customary units (not following their specification), and NASA expected it in SI units (as their specification expected)
It seems like it should have been checked by Lockheed not NASA since supposedly NASA provided a specification, that specified the units, and paid for the software no?
> "The output from the SM_FORCES application code as required by a MSOP Project Software Interface Specification (SIS) was to be in metric units of Newtonseconds (N-s)"
(MSOP = Mars Surveyor Operations Program).
One of the recommendations was
> "Conduct software audit for specification compliance on all data transferred between JPL and Lockheed Martin Astronautics"
So yes, NASA should have checked the provided software more thoroughly, but also Lockheed should have actually followed the spec they were given. I doubt the SIS is available online to check any harder
No, if NASA chose not to triple check everything before sending something into space then they are as much to blame as the supplier. Blindly assuming something works like it should without testing it makes an ass out of u and me.
Very fitting for the AI age as well.
"Yeah I just put the specs of the project you're paying me to do (in NASA's case, probably 100% public funds) but Claude the gimp missed the units because all previous training data use Stones and Yards as units, you should have verified it yourself! I just prompt!"
Should have been checked by both. Lockheed should have checked because of contractual obligations; NASA should have checked because of minimal engineering practices (bugs happen in all your dependencies).
They happen and some are put there on purpose too. In this case it's not a "bug" in your dependencies, and yes NASA since it's using public funding should have been more careful, but ultimately it's Lockheed that was paid to do something no?
Lockheed's job is to follow the customer's specifications.
NASA's job is to check to make sure what they paid for is what they received.
I do this every single day as a quality inspector here. I don't know why a bunch of highly-degreed engineers can't do a simple job that a person with oonly a GED does without fail.
Actually no, it's Lockheed's that should have had a test-suite and conformance-suite for it, but probably only has a C-Suite. If the program is non-trivial proving its correctness can be very expensive to prove, both money and time wise, and something that is done contrary to the spec is obviously on the one executing the spec and being paid for it, which probably wasn't cheap and probably these expenses add to the "NASA only burns money..." narrative.
Yeah it's about costs. I care about a lot of things, but the causes I actually give money to is a much shorter list. If personal data was radioactive, and leaking it cost companies real money, then more money would get spent on security. (and insurance, and lawyers.)
Yeah I've long said we should treat data leaks like food safety. The only way we'll see software security improve is if there were serious fines and/or jail time for leaking user data due to negligence.
> SeL4’s security and reliability proofs still hold in the world of LLMs
Oftentimes the weakest link is the human operator who has direct access to those systems, not the computer system itself. Good old social engineering, in other words.
Even though one could use LLMs for social engineering, come to think of it, like re-enacting the movie "Her" involving a modern AI as Scarlett Johansson and an engineer working for the water utility as the romantic target.
mind that just because SeL4 micro-kernel is proven secure, there's no guarantee that all leyers on top of it are. There's a presentation (https://www.youtube.com/watch?v=LhUwwsVq5E4) that's pretty much about "so we gave sel4 to students, and NONE of them managed to build utilities on top safely - now we have to provide our own Core Platform implementation to be sure"
But most important of all, the highest level - human operators - are not provable secure anyway. Any castle gate can be opened from inside - so why have the gate anyway? Security by absence is absolute
<< And as such, it’s much more expensive. And nobody wants to pay.
Eh. If only it was that simple. I mean, yes, money is always a factor, but not nearly as big of a factor as 'my convenience outweighs pretty much everything ( until it causes sufficient amount of havoc.. and even then.. )'. You can see it in just about everything. It is not just the money. It is the convenience that drives most of the unsecure behavior.
Cybersecurity is a fragile system in Taleb's fragile-robust-antifragile framework. That is, as t->infinity, the probability of a hack approaches 1, because you only need to slip up once, and there is a small probability of the system's maintainers slipping up each day.
Lol, just because a tool exists that in theory can be used to potentially do part of what it takes, doesn’t mean anyone will use it that way consistently or reliably.
the actual answer to this is accreditation. you will see safety standards rise across the board if engineers risk losing their license for shipping unsafe features.
sel4's guarantees break if you have DMA, e.g., from your NIC. It doesn't help with timing attacks. It doesn't cover your network stack. AFAIK, no one has taken up the mantle from Project Everest, so you'll need to write a verified TLS library. Once you've built all that, you can start thinking about your database/application/whatever. Then of course you'll have to verify all your dev's machines and scripts to ensure nobody is misusing a credential that can get stolen.
"So what?" you say. "Making a heavier-than-air metal tube take off and land millions of times per year without a catastrophe is also hard, and we no longer expect most or even many of those tubes to blow up or fall down."
Mother nature is not spending $$$ using AI and HI adversarially trying to find the exact combination of atoms that will cause your device to fail.
> sel4's guarantees break if you have DMA, e.g., from your NIC.
Modern CPUs support IOMMU. If you set that up, your NIC can only DMA to virtual addresses, managed by the operating system.
> It doesn't help with timing attacks. It doesn't cover your network stack
It does help with all this stuff, because your network stack and whatever else can be split off into isolated processes which talk over capabilities. Compromises in those processes are of course terrible. But they don't automatically allow kernel level takeover of the whole machine like on windows / linux.
> The problem isn’t that we lack the capability to make secure computers.
Depends on the "we". "We" have the capability to make secure computers like how "we" have the capability to make EUV lithography machines. There exists a relatively small number of people and organizations in the world who can do so. Microsoft does not have that capability. Google does not have that capability. Linux does not have that capability. Amazon does not have that capability. Apple does not have that capability. Cisco does not have that capability. IBM does not have that capability. etc. All of those organizations have tried for literal decades, thumped their chests about how they have awesome security year after year, and yet have totally and utterly failed despite their best efforts.
Acquiring the capability to do so is difficult and challenging and requires years to invent if you start right this very second and know what you need to do, which these organizations emphatically do not. We need security at scale and fast. The only way forward is to scale up working solutions rather than letting the bozos who put us in this spot fail at scale with yet another promise that this time for sure they will solve the problem they have repeatedly failed at for decades.
> Microsoft does not have that capability. Google does not have that capability. Linux does not have that capability. Amazon does not have that capability. Apple does not have that capability. Cisco does not have that capability. IBM does not have that capability. etc
This is all by choice. They could easily have that capsbility, very unlike EUV.
Part of the argument (aiui) is that there is no way that culture could ever change to producing secure systems. There are too many weaknesses embedded in the organizational structure.
Yeah, just like a lot of organizational change, one approach would probably be to put together a small Skunkworks-type team to nail it on one specific product. Existence proofs have, in my experience, a pretty powerful effect on the naysayers.
Oh, Microsoft can just figure out how to make unhackable systems, they just choose not to. They spend all of those billions of dollars per year on security and spent all of those decades on failed attempts as a prank.
You really think that if they could have they would not have, even just for bragging rights? Or are we going with that it is some kind of task demanding enormous expenditure even though the organizations that have made secure systems are infinitesimally small in comparison?
Microsoft has spent orders of magnitude more money and time than the organizations that have succeeded and the result of their efforts is Windows. That says everything you need to know about their capabilitys.
Multiple literal trillion dollars organizations have spent literal decades failing at it. You are really underselling the capability gap.
If windows was reimplemented as a capability based microkernel like sel4, it would be far more secure. Run drivers in their own isolated processes. Do interprocess communication between them via capabilities and shared memory. Remove all ambient authority from programs. All the programs a user launches stop automatically inheriting all of that user's permissions.
There's no secret knowledge required to do this. The SeL4 team has written extensive documentation of how they did it. They also opensourced their kernel implementation, with correctness proofs for the whole thing.
The reason windows hasn't done it is the cost. You'd have to rewrite half of the NT kernel and refactor everything else. All existing windows drivers would need to be rewritten. If you forced windows userland use a capability based system, you'd essentially be inventing a new way to write windows programs. You'd need to document that, and write a compatibility layer for legacy programs. And solve some UX problems. It would be terribly inconvenient for everyone. Oh, and some programs would run slower as a result.
They could do it if they wanted to. But microsoft just doesn't care about security as much as they care about performance and compatibility. Linux is the same.
The big irony is that security would be a lot cheaper for microsoft if they designed the NT kernel to be more like sel4. Microsoft has to spend millions on security every year because any tiny bug in the kernel (including in drivers) might result in the whole OS being compromised. In a microkernel, a buggy driver is nowhere near as dangerous.
Your knowledge is out of date: Windows has supported capabilities from the start of NT and does run a lot drivers in isolated processes, along with most OS services. It has done for years. It has the most sophisticated IPC framework of any OS (DCOM) which is integrated with the operating system kernel's security frameworks, and used pervasively both internally and by apps. And Microsoft has created a new way to write Windows programs in which apps are expected to advertise the capabilities they need (see WinRT and MSIX). This is also over 15 years old.
macOS does the same but with more developer adoption. Apps don't have the ambient capabilities of the user and must advertise what they need via entitlements embedded in the binaries, or get permission just in time.
Which is all very good, and modern platforms are much more secure than they once were. Yet "capabilities" as a silver bullet are academic overpromises. This article I wrote is more about language/runtime level capabilities but OS capabilities are not much better.
SeL4 isn't secure because of One Weird Trick that others would adopt if only if they could be made to care enough, it's "secure" because it hardly does anything, which is why nobody uses it and why it has no impact on real world computer security.
The hard part of desktop security is not changing the operating system. The hard part is getting app developers to care. Most security features added to operating systems are ignored by developers, which is why Apple forces you to adopt some of them as the price of admission to the app store. If they didn't nobody would use them, as can be seen for apps distributed outside of the app store. The reason is security is a market for lemons. Nobody can see the result of security investments so it's irrational to invest. SeL4 has no solution.
> Your knowledge is out of date: Windows has supported capabilities from the start of NT and does run a lot drivers in isolated processes, along with most OS services.
Thanks! I quickly googled this point before posting earlier to make sure I was still right. Gemini helpfully told me that yes indeed, drivers in windows run in the kernel's main process. Thanks, AI.
> This article I wrote is more about language/runtime level capabilities but OS capabilities are not much better.
I think I responded to this article at the time. I still find this article somewhat confusing and unconvincing. For example, you conflate Java's SecurityManager with capability systems, even though it seems more like an permission based access control system. Then you point out many of its weaknesses. To what end? What conclusion about capability systems am I supposed to draw from a criticism of this quite different security model?
A capability is not a permission flag. Unlike your example, a good capability system would generally pass all HTTP requests to a given endpoint through a single capability object. You wouldn't need different caps for each HTTP method like SecurityManager apparently requires. It's like file handles. You don't create several different file handles to interact with the same file, one for reading, one for writing and so on. We just open the file once, with whatever options are needed. Then the file descriptor can be passed into any function which needs to access that file. And whatever code receives the file descriptor doesn't know if they're talking to an actual file, or some in-memory object or something else. Just like a virtual object.
You also say this:
> File descriptors are a kind of capability provided by the kernel, but a rather odd and inflexible kind. They aren’t a great example of object capabilities.
Huh? File descriptors are often treated as the canonical example of object capabilities. This comment makes me wonder if we're even talking about the same thing. At the risk of being indelicate, are you sure you know what capabilities are? Can you give a definition of object capabilities which doesn't describe file descriptors?
The point about god objects lands. I also agree that trying to retrofit a language like java to make modules unable to share memory is difficult. But many aspects of language design work like this. Consider garbage collectors. Before GC languages existed, I could write the same article talking about the difficulties of hacking a GC into C. But that wouldn't teach me anything about how well a GC would work in a language like Java or Ruby.
Anyway, the main advantage of capabilities is the ability to split programs out into sub-modules such that a compromise or bug in one part of the system doesn't lead to the entire system failing. We can argue about whether bringing this into the language runtime is a good idea. But I feel pretty confident that this sort of separation is a good idea at the systems level, helping with security and reliability. We can look at Chrome, SeL4, Erlang and - apparently - windows for examples. Even if they don't all think of this as a capability based problem.
The SecurityManager exists in complement with Java capabilities. You could, in theory, use the Java language with a different standard library and implement a pure object capability system, as the language rules allow for that (if you disable reflection). But it would have bad ergonomics and still end up with a SecurityManager equivalent because a DSL for permissions is more convenient than doing it all in code.
Consider the most common task the SecurityManager was deployed for: stopping plugins calling System.exit() by accident. One might say, the right to exit the process should be an object capability. OK. But then where does that object come from? Java programs start at main() and it doesn't receive an object.
You'd need a new design where you pass in a god object to main(), which in turn has properties giving access to a ProcessExiter interface or something similar, and then any code that genuinely needs to exit the process would need to request it in the function arguments, threading it down the stack. You'd get an explosion of types. A simple permissions DSL is much easier to write and reason about, and it gets out of the way when you don't want sandboxing.
Why would you want all HTTP requests to flow through a single capability object? I think it's pretty common to want to let code do GETs but not POSTs. You end up wanting pretty fine grained permissions in a lot of real scenarios.
File descriptors are poor object capabilities because the interface they implement is fixed by the OS, except then there's a weird ioctl escape hatch that isn't properly typed, reflectable, wrappable or interposable. To see what can go wrong with this, consider a recent fix to the Codex sandbox on macOS:
The sandbox forbids writing to a file descriptor except, oops, someone at Apple forgot about the F_TRANSFEREXTENTS ioctl which is still allowed on a read only fd. It should be possible to do what is expected here and just pass in a read only fd where all you can do is call read() and maybe seek(), or perhaps pass in an fd where a specific ioctl is the only thing you can do, but POSIX has no concept of this.
A good example of an object capability system would be Mojo, which I describe in the essay. You can create objects representing capabilities and pass them between sandboxes, in an unforgeable way.
We live in a golden era of prototyping so if you wanted to make a language where everything is a capability passed into main(), you could. The code doesn't have to be executable, you could just mock out some realistic programs and see how the code feels. My guess is you'd need a lot of language features to hide the explicit object capabilities away for ergonomic reasons and it'd end up feeling a lot like a SecurityManager based system.
> But it would have bad ergonomics and still end up with a SecurityManager equivalent because a DSL for permissions is more convenient than doing it all in code.
This might be the core contention. I don't know if using actual capabilities in a language would have problematically bad ergonomics. You'd probably be passing more arguments to functions. But haskell seems to manage ok despite needing to pass IO to functions that need it. Capabilities seem similarly inconvenient. I think I'd need to see it tried. I agree - I might need to try it myself.
> Consider the most common task the SecurityManager was deployed for: stopping plugins calling System.exit() by accident. One might say, the right to exit the process should be an object capability.
I don't think this is a great example. Caps are generally for resources outside of your program or module scope. A program already has the capability to exit, so that wouldn't be something you would pass in from outside of the program.
> Java programs start at main() and it doesn't receive an object.
I agree that retrofitting caps into an existing language like java would be difficult and inconvenient. Passing a "god cap" to main() is the easy part! The hard part is just how much of the standard library implicitly depends on ambient authority. I've thought about doing this in rust, and concluded that I'd probably need to fork rust's std library.
> You'd get an explosion of types.
I've never heard of that stopping java programmers before.
The way SeL4 handles this is to have a generic call() interface for capabilities. It's very simple, and it would work fine in this example.
> Why would you want all HTTP requests to flow through a single capability object?
Capabilities are a combination of resource + access rights over that resource. If I wanted to give a module access to a REST endpoint, I'd make a cap representing that endpoint. The resource is the URL base (eg "example.com/foo/bar"). And I'd also specify access rights (eg only HEAD+GET, or HEAD+GET+POST or whatever makes sense). Then pass that object around to any modules which need access. I'd even keep the URL prefix private in the capability object. The capability object would only expose methods for http_get(), http_head(), http_post() and so on. This design would be more or less impossible to misuse. And it would be super handy for unit testing and dev environments.
It's not "one capability for everything" and it's not "a million fine-grained access rights". You want one cap per semantic resource, just like one fd per open file. If you want to refine the granted permissions, just reimplement the same interface with a different implementation of http_get() and friends. (Or, simpler: just wrap your existing RESTEndpoint class with another class which adds your extra checks).
> except then there's a weird ioctl escape hatch that isn't properly typed,
This is a flaw of the unix syscall API. In comparison, SeL4 only has 9 syscalls (plus 2 for debugging). The syscalls just let you call capabilities, and have your capabilities be called by other processes. And yield(). That's all the syscalls on sel4.
Because everything runs through that same API, it's trivial to stub out or replace capabilities provided by different components. Eg, any program can reimplement the filesystem API if it wants to. No need for FUSE, or special loopback mounting or anything like that. Because the filesystem is just a userland process which doesn't have access to the kernel's memory, there are no ioctls that you can accidentally forget to sandbox. The only special thing about the filesystem is that it holds a capability to do raw IO on the block device. (And that cap, in turn, is provided by another userland process.)
> My guess is you'd need a lot of language features to hide the explicit object capabilities away for ergonomic reasons
Yeah, I think that's our big disagreement. You seem to think that hiding object capabilities would be a necessary design choice. I think using caps directly would be much more ergonomic than a SecurityManager style design because custom caps can just be implemented in normal code. And caps are better because they encapsulate a resource, not just access control rights.
> > But it would have bad ergonomics and still end up with a SecurityManager equivalent because a DSL for permissions is more convenient than doing it all in code.
>
> This might be the core contention. I don't know if using actual capabilities in a language would have problematically bad ergonomics. You'd probably be passing more arguments to functions. But haskell seems to manage ok despite needing to pass IO to functions that need it. Capabilities seem similarly inconvenient. I think I'd need to see it tried. I agree - I might need to try it myself.
One of the common objections I hear to Bluefin is "isn't it too inconvenient to pass around capabilities everywhere?". Perhaps surprisingly, no, I haven't found it remotely inconvenient. I find it liberating, actually.
Spot on. And it's gonna get a lot worse now that "but my slop machine's code is run through a really large number of tests, like enormously many, I don't need to understand the code!" is apparently not an embarrassing way to do development anymore.
It'd be much better if this test was done in low light. The tiny sensors in iphones work fine when there's plenty of light around. The real test is how well they do when the you aren't sitting in the sun.
> But iPhone + zoom + crop is better in some ways than my 50mm 1.2 with bokeh.
Well, yeah. f1.2 is a nice gimmick but opening up so wide rarely translates into better photos. Don't get me wrong - you can get some beautiful portraits with that lens. But in practice, my 24-70 f2.8 zoom lens gets way more use than a 50mm prime.
There's a reason high end phones have multiple cameras on them. It's because not every photo wants to be taken at the same focal length.
I'd much rather a mirrorless camera with a 24-70 than an iphone. A mirrorless camera with just a 50mm prime will take some beautiful photos. And some photos you won't be able to capture at all.
This is my feeling too. It looks great - I think I might pick one up. But I’m going to give apple a few years to iron out the bugs first. Version 3 of the duo will be a much better product than version 1.
reply