Somehow I feel like the biggest post-apocalyptic problem will be the loss of home equity due to uninsured damage causing a collapse of financial markets.
Really nice read. I'll add that it's effective to scope design docs up and down as needed, both in terms of how big your project is and how big your company is. A 50-person startup doesn't need a full design doc with multiple approvers. But a one-pager explaining what you're doing and having some documentation is really helpful even with an engineering team of 5. I'll also still write design docs for code changes that only touch a handful of files, if I feel it's important enough. The process of writing in a concise and precise way for other people to consume also has the positive side effect of making things more clear for yourself, too.
> Interfaces section
Only real criticism I have here, is I would not include any code in a design doc, unless it is really really vitally important. I've seen a lot of design docs (especially in the LLM age, written by more junior staff) which are effectively just an English summary of code. The point of a design doc is (generally) not to explain that you're going to have a WidgetManager and WidgetFactory class and what specific properties and methods they have, it's to explain how the widget creation workflow works and maybe you have the WidgetManager/WidgetFactory in an architecture diagram. As a general rule, if you're starting to include actual code, you've gone too low-level, IMO.
> Not all design decisions are equally important. Some choices are more permanent than others.
At AWS one of the corporate culture memes was calling a decision either a "one-way door" or "two-way door". Just asking yourself the question, if we had to walk this back, is it truly irreversible or just an inconvenience? Turns out most day-to-day decisions are two-way doors, particularly engineering ones. Even if the choice made is wrong and a bunch of work needs to be done to switch back, it's still preferable to what a lot of companies end up in, which is decision paralysis where every change needs approval from multiple committees, resulting in months or years before work can start. Note that this doesn't meaningfully increase the odds the decisions made are the right ones; it just delays the implementation and diffuses responsibility if the wrong decision was made.
> A from-scratch rewrite would never work, and even if you manage to write new code in Rails, you’re still maintaining code in two wildly different languages.
Orthogonal to the article, but this line of thinking (including the link to the classic 2000 "Things You Should Never Do, Part I" article[0]) may be worth reviewing in the post-LLM world; for all their flaws, LLMs are spectacular at language-to-language translation, and we already have one major project released[1] that shows porting a relatively large and mature project from one language to another is possible. Not to say that it's the best use of your time, or that you shouldn't do your due diligence and pick the right language up front, or even that the original 2000 article was about a language-to-language rewrite (the Netscape rewrite was an architectural redesign).
Thanks for reading and for the thoughtful feedback!
> Only real criticism I have here, is I would not include any code in a design doc, unless it is really really vitally important.;
Yeah, that's fair. If I were setting guidelines for a large org, I'd maybe discourage code snippets in design docs, as it's hard to know when is too much. At my last company, the dev team was just 3-4 people, and I found it helpful to have little snippets in design docs especially when we're talking about semantics of a new library or how to migrate existing code to a new system.
> Orthogonal to the article, but this line of thinking (including the link to the classic 2000 "Things You Should Never Do, Part I" article[0]) may be worth reviewing in the post-LLM world; for all their flaws, LLMs are spectacular at language-to-language translation, and we already have one major project released[1] that shows porting a relatively large and mature project from one language to another is possible.
Yeah, I agree this could change with LLMs, but I think Joel is still correct up to today. Bun is an interesting case because it's friendliest possible conditions for an LLM rewrite (self-contained inputs and outputs, easy to test old implementation and new implementation side by side, huge test corpus w/ third-party tests). I haven't followed it closely, but it seems like the jury's still kind of out as to whether the rewrite was a good idea.
> I never understood why anybody would pay for Slack when its based on IRC and IRC is free, but whatever.
Comparing IRC vs Slack maybe made sense when Slack was very young, and it was like buying a Honda Civic vs a Ferrari, where one is obviously nicer but the other still accomplished the same thing with less frills.
These days it's the choice between buying a Honda Civic vs a Boeing 747. They both get you from one place to another (communicating between employees) but that's where the similarities end now. Slack has way too many API integrations, historical search, features for everything from compliance message archiving to embedding images to dumb GIF searches that people seem to like. The fact that it has irc-like channels and chatting isn't really the point of the platform anymore.
Yes you could code this all yourself using IRC bots, but at that point IRC is no longer free, it's costing you time. If you get to that point, Slack does it better and cheaper.
I guess I can see it both ways. Most people who claim to be software developers have never written software. They likely write some JSX, type a few commands, and pat themselves on the back.
For those of us who do write software an API integration is typically one HTTP request and I wouldn’t pay a subscription price per headcount for something that takes 10 minutes to build and test one time.
> Simple example: feature X is rarely used (thus we should get rid of it)
Oh man, I remember a very specific example of this: many years ago now, Google Chrome pushed an update that got rid of the option on the menu bar for "Close Tabs to the Right". I remember looking into the Google issue tracker where people were complaining, and some PM provided a "data-driven" justification: when people opened the context menu, they only clicked on the "Close Tabs to the Right" option 1-2% of the time.
It's a great example of why data without context can give you the wrong answer. Of course the option is used relatively very rarely; you only need to clear out your tabs every once in a while, compared to creating new ones or managing tab groups! But it's still an essential task. It's like saying filing your taxes isn't important because you only need to do it 0.2% days of the year.
I do all the time because the feature exists. I would treat it like a sliding line in my tabs bar. When I'm done with a tab for now it goes to the right, and eventually everything to the right gets closed once I haven't used it in a while
I do. I open a search page, middle click a bunch of links, then want to close all of the ones to the right because I found one that told me what I needed and don't need the rest of them.
It's useful when starting a thread of research of research as new tabs will be opened to the right (Ctrl+Clicking a link). So after it's done, close to the right can be quite useful.
Because every other company has also replaced their 6 warehouse workers with a fleet of robots and 1 technician, and that technician's skills are more rare and specialized than the warehouse workers, so you're competing with every other warehouse-owning company to hire that technician, so that technician can demand a higher salary. In other words, capitalism.
"Humans use <new technology> to do evil things" is a tale as old as time, though. An AI didn't decide to fly into a warzone and start shooting people on its own.
Of course, but that's not my point. AI is different than past technologies because it's much more difficult to deterministically control, and we just saw that in a couple of wild cases (Hugging Face et al).
My point about the drones is we are starting to put AI in more systems that can affect (and blow up) the physical world, so it shouldn't be hard to imagine how a misaligned AI can do more terrifying damage than just take a German Wiki down.
> Can someone please explain to me how an LLM is going to "destroy humanity"? Even if the claims of ChatGPT, Claude et al. are true that their super scary agents were able to escape a sandbox and start hacking other servers (which I feel is at least 50% likely to just be marketing bullshit), how is an AI going to affect anything in the real world?
It's a great question. I've yet to see any research into how a humanity-destroying LLM defends itself against a curious toddler who pulls the power cord out of the wall.
The Internet virtually indestructible against people smarter than a toddler. It's already impossible to turn off. We would need to prevent the models from getting smarter.
> The Internet virtually indestructible against people smarter than a toddler. It's already impossible to turn off.
This isn't true at all. The internet has already been turned off in localized areas during major events, not to mention the (unintentional?) cutting of submarine cables. You could say "use satellite internet" but then you're just moving the point of failure to Starlink et al, who could also cut off the internet if they choose.
The internet is a lot more centralized than it appears.
Huh? Not really, internet shutdowns are a documented and real thing. Suggested reading (all of which refer to the internet as being shut down or blacked out): [0] [1] [2]
The fact that you need to connect to the internet through a service provider plus mostly-centralized control of infrastructure like DNS or BGP means shutting down the internet is a real thing that has been done several times. It doesn't happen in the developed world because their service economies are so highly dependent on internet connectivity, but if e.g. the US government had to contain a theoretical rogue escaped AI, they could absolutely get ISPs to metaphorically unplug the network cable and take down the internet as we know it.
I'm saying if you gave me full admin credentials to every ISP, BGP router, DNS provider, etc., then yes, I could effectively destroy the internet in an afternoon, and LLMs wouldn't be able to do a damn thing about it.
> I've yet to see any research into how a humanity-destroying LLM defends itself against a curious toddler who pulls the power cord out of the wall.
by acquiring multiple power cords. the world has plenty of hosting providers. if it manages to copy itself, then we'd need to shut down every computer in the world, not just one. there's plenty of ai compute on the web now, and there will be even more. and as it gets cheaper, the less security will be around it.
Go on please, prove you can go right now at the Texas data-center owned by OpenAI and unplug one single server for 1 minute. Surely you are much more capable than one curious toddler.
I'm a bit confused, when did we escalate from a curious toddler to a ballistic missile strike? Why didn't Iran send a curious toddler? It's much cheaper.
The use of the phrase "curious toddler" is called a metaphor[0], which is a figure of speech[1] used to make writing more interesting.
I apologize for the confusion that caused you to interpret this as a literal, physical, toddler. To restate the point with more straightforward language: it is very easy to disrupt the operation of large language models, because they require computer hardware in stable environments with a large amount of electrical power, and function within an operating system. I have not seen any evidence LLMs are immune to things such as: `kill -9`, `sudo reboot now`, physically unplugging the machine(s) on which they are running, or in the case of the AWS data center, collateral damage during a real-world conflict.
We are talking about a humans vs AI scenario. There are probably thousands of humans in the world right now that could do exactly that right now if they wanted to (datacenter and OpenAI employees, power grid workers).
Are you assuming that the humans are all unanimously aligned against the AI in this scenario? That sounds like the hard part. How does that happen? Does the AI announce its evil plans like a movie villain?
Until then, it's the job of the power grid workers to keep the power running, and shutting down the datacenters would just get them fired.
As crazy as it sounds, I'm not going to commit a felony to prove that disrupting a data centre's operations for a sufficiently motivated individual is a fairly easy task.
> If we pursued the "Most Qualified" in its truest form, then ~95%[1] of the jobs in this country would be handed over to foreigners. [1] The USA is less than 5% of the worlds population.
This would be true if you're talking about randomly picking people from around the world to live in the US for their formative years and go through the US education system (at which point they barely count as "foreigners"). This isn't reality, though, and if you're talking about hiring a worker today, a large swath of that 95% won't speak English, or won't be literate, or will have never received a formal education, all of which are disqualifying for pretty much every job in the US.
> Kind of a bummer that 95% of software engineering is dedicated to creating a padded cell to make sure no one makes a whoopsie instead of doing creative and interesting things.
s/software engineering/running a business.
I'd much prefer that the people building the bridges and planes I use, or the staff at the hospital I visit, have procedures in place to prevent them from "making a whoopsie" than the space to be creative or interesting. Even if they enjoy their job less.
The good news with software engineering is that unlike building a plane or running a hospital, nobody is stopping you from making your own projects and being creative and interesting as you want!
Yea, it's too bad that "making whoopsies" has become such a common occurrence, treated so casually, in what is made out to be a serious engineering discipline.
I mean, all engineering disciplines seemingly build in for margin of error / tolerances. e.g. Bridge, etc. are often overbuilt in case of faults in materials or expected circumstances.
I do agree our profession grapples with complexity in bad ways though.
You should check your home insurance contract, for instance... It likely would not cover an ICBM strike.
reply