Really often the alternative was not to make the journey. Whether that's the in-person visit to a customer site which could have been a video call (or an email) or the trip to go see how your Porsche was made in the factory.
WG21 (the C++ Standards Committee) only began to allow you to present proposals or attend committee meetings without flying around the world in... 2020 because of Pandemic restrictions.
Video conferencing was one of the technologies in Engelbart's "Mother of all demos" in 1968. Remember that year, not 2008, not even 1998, it was demonstrated in 1968 and yet I bet several readers of this comment have flown on an aeroplane to do something they could have done using video conferencing, almost sixty years later. What a joke.
Oh, of course, but I mean for moving people who want to move around. Someone else observed that ocean liners are less efficient than planes, so if you wanna go overseas there's no alternative.
Except, of course, for business meetings which can be virtual (or just an email). I'm asking what's a practical alternative assuming we still want people to be able to go overseas. I agree for land routes high speed rail is better, too. I don't really understand my downvotes just for asking about alternatives.
Yes, except no with the KV cache. Because tokens aren't modified by future tokens you can cache the meaning of previous tokens. This makes the total effort linear over the entire context (or constant per forward pass).
> This makes the total effort linear over the entire context (or constant per forward pass).
This is incorrect. The compute required per forward pass to generate each additional token during decode will scales as O(N), even with a KV cache (without a KV cache, it would scale as O(N^2)). Over generating N tokens, it's O(N^2) with the cache (and O(N^3) without).
It's O(N) for a forward pass because that new token still has to "attend to" to each previous token. That requires N dot products: between the cached key vectors and the new query vector for the new position. You also have N reads from memory (K and V) which is probably gonna be your actual bottleneck. (Decode is memory-bound.)
This is why you should avoid long contexts, if you can, even with a warm cache. You will get charged more, in "cache read" tokens.
I was under the impression that each new token attends to one previous token per attention head, and that the slowdown observed was more because those attended to tokens are more spread out in memory and get less memory-architecture-style cache hits.
Well, I'm under the impression that the relations are recursive but you skip computing them recursively by caching B's relations. However if you look at your sibling comments you'll see that I may not have the right grasp on the matter.
> By the time the humans even became aware of the problem
Except, let me quote part of the report:
> July 4: Artifactory outage caused by agent interactions overloading the system.
> July 6: New artifactory instance is patched and put back online, wiping out the previous message board and method of internet access. A team at OpenAI was aware of the previous message board, but key decision makers were not.
> July 7: Experiments are restarted.
There was a very clear point where humans were aware and they went "Eh, whatever" and turned the bots back on! That's not the lack of any battle of wits that people are expecting. You're saying that the bots attack should be fait accompli, but it was noticed, shrugged at, and expressly allowed to proceed.
Right? If you move between file system or device I would expect "move" to be impossible. At best you could copy a logical block and then delete a logical block. If you share a filesystem then perhaps a logical block is a few hundred kilobytes. If you don't then I'd expect the smallest logical block to be the file itself.
Right! When I wrote that I made a mistake and thought that there were filesystems that used multiple drives but still had the meaningful distinction of "what drive is that file on?"
Like, yeah, RAID, several drives, one filesystem, right? Except no, because it's just one virtual device.
Another way to think of this is that everybody occasionally needs to do something unusual. But some people, due to their life circumstances, always need to do something unusual. And making things better for the always-odd folk is also a positive to the occasionally-odd folk.
Which like, we're all eventually gonna be odd folk! That's what age does to us. But we should understand these accommodations as being a net positive, not a cost, to society.
Yep. In accessibility research, they call these "situational disabilities". When you need to open a door with your arms full of groceries, you aren't so different from someone who is never able to open a door. Accommodations that help people with permanent disabilities also help other people in the right situations, too.
My touchpad broke and I couldnt download the touchpad driver to fix it on the spot because there was no way to get to the download button with tab and enter, you had to click it.
That's a funny example, reminds me of how often fresh Windows 7 or older installs were missing the ethernet driver so you had to move stuff over on a flash drive to get started really setting stuff up.
In your situation, a couple possible solutions come to mind. 1) turn on "mouse keys" which lets you use the numpad to move the mouse and click, 2) a vim addon for the browser (e.g. vimperator or tridactyl) likely could've clicked the button via the hints mode.
It's in the past now so maybe these wouldn't have worked or you didn't know, of course, just fun to think of what I would've done.
Yup. Or people with temporary disabilities — who have simply broken limb or had a surgery. Most places are hellish to navigate on crutches. Heavy doors alone make so many places nearly inaccessible, and that’s before you even get to stairs.
This reminded me of something that happened to me maybe 15 years ago. I was helping my Grandpa get his first smart phone. The saleswoman was maybe mid twenties and had a kinda hoarse voice. I was a little bit younger than her and I thought it was cute in its own way but of course I wasn't gonna say anything. She apologized for being hard to understand and my grandpa said no, it was fine, her voice was sexy.
Naturally I was mortified.
But I wonder how much of it is just "Fuck it, I'm an old man, I'm gonna let my creep flag fly."
Well, it's hard to convey, but he gave off a strong sense of "hitting on her" and she gave off a strong sense of "I'm in customer service and cannot tell him to stop".
But, I was in my latest teens, earliest twenties; I could have been misreading the situation. All I can say for certain is I was uncomfortable.
There's a tiny bit of play, like sliding window attention. As tokens leave the sliding window you can keep them or discard them. If you keep them, you can freely truncate the context and resume generation from an earlier point. If you discard them, you have to recompute the KV cache up to that point.
Llama.cpp checkpoints and moves snapshots of the cache to main RAM for faster resumption after truncation.
Well running from QSPI flash (even the internal memory versions use SPI internally) so any inference would be very slow streaming from that compared to RAM. The featured article says: “With a peak session RAM around 28MB, Needle runs on newer microcontrollers like ESP32-S3.” So I don’t see this doing anything useful on a Pico. The Pico 2 (RP2350) for example has 520k of RAM.
This author also has other things like: A compiler that emits only `mov` instructions and another compiler that deliberately messes with the control flow so that, if disassembled, common debuggers will draw symbols like skulls or threats. https://github.com/xoreaxeaxeax/repsych
Also came up with the original ..cantor.dust.. binary visualization tool, which is a tool I never used directly but Chris' presentation of it in 2012 is still one of the coolest talks I've ever seen.
