> LLMs are vectorial databases with losses that index statistically filled data
Yes, and that statistically filled data is insanely useful. It remains true that it's a relatively poorly understood how this can be applied in various scenarios and what processes are needed to ensure robust results (or quantify the uncertainty).
What is it insanely useful for? (Besides convincing investors to sink more money into LLM-related companies? Because that is the one thing it does seem to truly be good at.)
LLMs generate text output that appears to be useful, but regularly is not. They're alleged to be a substantial boost to writing code, but that verdict seems to be in dispute. They can generate custom mediocre prose at scale, but that seems to be of ultimately limited utility (although it may be a godsend for propagandists).
We're coming up on the 4th anniversary of ChatGPT's release. And while I get that revolutionary technologies can take a while to mature, the Wright Brothers and Goddard weren't preaching imminent societal transformation by the end to the decade from the rooftops, either. (And that's before we get into the how they got there - getting to ignore laws and steal whatever they wanted might be insanely useful to a lot of people.)
With good input (prompts, specs...) LLMs can generate code that is often correct, faster than a human could generate equivalent code. Even when there are bugs, it is still "useful" from purely a time savings perspective. If you don't like the results, you can iterate rapidly.
Yes, you can use it to generate crap. I find Claude especially bad at writing like a normal person.
LLMs are not being promoted as "this can help write code faster/better/cheaper" (for the sake of argument presuming that they really can), they're being promoted as "Cortana" (. And their underlying economics are likewise premised on "Cortana" (for a huge amount of money). Which is going to be a disaster if/when they don't deliver.
Yes it is. I find them empirically not useful. You may not wish to debate it, but the fact remains that there are a great many people who are not convinced of their usefulness.
LLMs are very good at writing code. The reality is that they are able to write code faster, at higher quality and with fewer bugs, with correct prompting. They are also really good at code analysis, penetration testing and discovery, and adjacent computer science disciplines.
No they are not perfect, nor do they produce the best code. But the undeniable reality is that any good engineer will produce more code, at higher quality, using an LLM.
So, that’s not really up for debate. The debatable part is if all that code is a good idea or has as much value as we think. The conversation has long moved passed “can LLMs write code?”. Yes, they can, very well, particularly if they’re steered by trained engineers.
We have good engineers that gave us curl, ffmpeg, the 4.4BSD, tmux, vim and emacs, x11,… To this day, no one can show any particular important software that comes from LLM assistance at scale. It’s all slop.
Well it's been barely a year, but basically all commercial software, and most open-source software, is engineered with AI assistance. It's not all slop, you just aren't looking around. Ask any engineer at any company - are they using LLMs to assist in writing code? All of them will say yes. Source: I'm a software engineer working on commercial software at a company, I also talk to many engineers at other companies.
Agreed. Despite the many claims of how awesome LLMs are for productivity, we have yet to see that supposed productivity produce fruit. Moreover, I dispute the claims of productivity: in my own usage I find them to be at best neutral, or even a drain on productivity. In my opinion, there is to date zero evidence of the supposedly insane utility.
To meet the threshold of "insanely useful"? The Sagan standard is, "Extraordinary claims require extraordinary evidence."
I can see that some people can make some use of them. (This is true of almost everything.) Whether or not that usefulness is worthwhile overall, whether it is a net good, or even ethical is a different question. But insanely useful?
Computers are insanely useful. So are engines. Water. Sunlight. Electricity. Grain and bread. Writing. Printing. And I don't feel bad making those sorts of comparisons, because that's the level of impact LLMs' advocates are promising. But it's not what we have.
What I would consider sufficient evidence for insanely useful? Reliably replace a human in prolonged, arbitrary, detailed interaction, without any inhuman screwups.
crack growth in metals is driven by microscopic flaws that cause high-intensity but very localized stress concentrations. Over time even low stress levels cause these flaws to grow to the point where they start causing strength problems. Even in traditional aluminum machined parts, increased surface roughness can have a large impact on fatigue life.
3D printed parts are chock-full of these microscopic flaws, porosity, and have horrible surface roughness (most parts you see in production are post-machined to improve the finish). Additionally, the repeated heating-cooling of the layers as they are deposited builds up residual stress in the part. All just due to the nature of how they are manufactured.
Is there a known source of internal flaws/porosity in an otherwise solid part? Presumably laser melting produces a puddle which shouldn't allow for internal pores, as long as it isn't printed too fast (or solidifies too fast, which is why I think most chambers are internally heated to near melting temp).
Re: surface roughness, I can understand that the powder grain size creates a sort of minimal structure size, and can in principle be the start of a crack if a surface grain gets knocked loose. Is that the sort of thing you mean? I can see that for any internal or external surfaces, and a rocket engine combustion is certainly applying a lot of heat and pressure on surface grains. Can this be alleviated by smaller grain sizes, or is there some limit there?
Re: repeated heating/cooling and internal stresses, this strikes me as just requiring standard post-printing stages like tempering to alleviate internal stresses.
I can't speak to the proximal cause of the roughness and porosity, but if you've ever held a raw printed aluminum part in your hand it is immediately apparent.
That said there are processes to deal with porosity like Hot Isostatic Pressure (HIP) treatment that basically crushes all the voids with immense pressure. This does come at the cost of dimensional accuracy though (HIP will compress the part somewhat).
Similarly, annealing a 3D-print to relieve residual stress does work, but it also will cause warping as those stresses are relieved. Again, sacrificing dimensional accuracy. Frontier AM companies have ways to compensate for all of these effects but it's a trial and error process for each part essentially.
At this point you're now stacking multiple processes on each other just to try to get to near-billet properties. Calibration Trials > Print > HIP > Anneal > Machine. The cost adds up quickly. It can be justified especially in non-fatigue-critical applications but it's no free lunch
SLS printers lay a layer height worth of powdered metal and fuse it with a laser engraver. Thee bed lowers one layer and the process is repeated. They don't bring materials used like inconel to full melting temperature, only do what it takes for the metal sand to clump together. That's one source of pores.
I assume you can just anneal or print then recast in sand or whatever, maybe even lostwax with Al as wax, but the point is that porosity in a print itself is inevitable with current powder based tech.
3D printed metal is now as strong as machined metal, assuming an identical alloy. The process has been pretty well perfected.
The strength loss comes from the fact that not all alloys are 3d-printing friendly, so you often have to compromise and you end up with a less than ideal alloy for your application.
Sure, but I mean what's the technical reason a material isn't it 3D printing friendly? Are we talking grain structure here? Is it something that can be at least partly mitigated by some post-printing heat treatments, like tempering?
Some alloys don’t like to be melted. If an alloy has a large solidification range, certain areas can partially solidify without the liquid part keeping up to fill in the gaps so to speak. This leads to solidification cracking / hot tearing. This is a simplification and only one possible cause, but there are literal books written about this kind of thing (I like Solidification by Dantzig and Rappaz). This is also why you see things like friction stir welding for rocket bodies. No melting means no solidification means no solidification issues.
I've found the opposite on my R9700 (n-max=7, no other speculative decoding like ngram-mod, which I found slows it down). I think it depends whether your workload and system are bandwidth limited or compute limited. I see draft acceptance around 0.55, so 0.55 * 7 = 3.8 tokens per pass, which on my bandwidth-limited card takes me from 30tps to a peak of 80tps on llama.cpp (MTP peaked at ~65tps). I'm also running a Qwen fine tune whose speculative execution is better than the base model.
Strix Halo has lower compute than the R9700 but the RAM is also slower, so not sure what would be the ultimate limiting factor.
I run llama.cpp with -ctv=8, -ctk=q4. Vulkan has better throughput if you're doing single-stream decode, but ROCm has better throughput if you have "--parallel 2" or higher. If supporting parallelism, unified kv cache should be off, especially with Vulkan.
Of course, some of these may be specific to my card so try variations for your hardware. Hermes can concoct a test suite and run some tests for different llama.cpp parameter permutations to find something optimal.
Resistance training should be a form of stretching. If you're taking the loaded muscle through full ranges of motion, as you should be, then that also means ranges where the muscles are fully stretched. Unfortunately many people cut the range of motion short so they can add more load, which is counterproductive for both flexibility, injury risk and progress.
We're already doing those experiments. Tailor made mRNA vaccines targeted to one's own specific cancer mutations can be bought right now. The techniques are getting more sophisticated and more targeted every year. Our ability to predict what happens at these levels is improving by leaps and bounds too thanks to AI like AlphaFold. Anyone can be reasonably confident that no AI or teen can do this now or in the next few years, but are you really so confident what might be possible in 10 years?
I worked in immuno-oncology: cancer vaccines work in melanoma where many things work because of the neoantigen abundance, they've generally been very mixed in efficacy. And even BionTech's BNT111 failed in melanoma! We have hope for these approaches but the reality of this stuff is way more nuanced than you think it is.
Alphafold can't reliably predict thermal energy landscapes or make functional predictions - and how could it? It wasn't trained on anything that could capture structure - function relationships.
Again, most people just have no idea how hard - fundamentally hard - molecular physics is to predict, and how necessary experiments are for any development of biological systems.
> We have hope for these approaches but the reality of this stuff is way more nuanced than you think it is.
Sure, everything has more nuance. The point is this stuff is available now; this isn't some future sci-fi, it's only going to get better, it's not the only research on gene targeting, and AI is starting to help with this research. By the time AGI is actually here, consider the breadth of knowledge and capabilities that will be at its disposal.
> Alphafold can't reliably predict thermal energy landscapes or make functional predictions - and how could it? It wasn't trained on anything that could capture structure - function relationships.
If your point is that the only reason an AI like AlphaFold can't make functional predictions is that we haven't trained an AI to do that, then unless you're arguing we can't or won't ever do that, I'm not sure how that's supposed to be an objection to the argument that AI will be able to make use of this information without doing all of the experiments people seem to think would be necessary.
Like I said, we're already going to be doing these experiments because it's useful to us, and we will train AIs to make these predictions, again, because it's useful to us. Stop imagining what an AGI has access to now, and start thinking what it will have access to with the inevitable march of progress that we're already on.
Edit: and of course, this doesn't even take into account the fact that an AI could acquire resources to pay people to do this research. The internet provides ample opportunities like this now.
We're doing that science right now. We're using AI to do it. By the time AGI truly spreads, we'll have a lot of the knowledge you say will be needed. This is simply not as far fetched as you seem to think. Next few years, unlikely. 10-15 years? Increasingly possible. 20+ years, I'd hazard even likely.
> The said user (Tristan Buckmaster) didn't solve the millennium problem. He didn't really accuse that OpenAI stole his research either. The beef came from the fact OpenAI asked him to remove another mathematician, who works for Anthropic, from the credit.
Not quite accurate, Buckmaster was taking an approach that nobody else was, and this new proof uses this same approach just weeks after he saved those results to OpenAI workspaces. He asked OpenAI if they used chat logs for training the new model, and they did not confirm or deny.
Asking to remove his collaborator is also totally over the line though.
Wow, this sentence is doing a lot of work in that tweet: "While unlikely, we cannot rule out that de-identified data derived from their usage of our products helped improve our models."
reply