It's really important to get people to recognize that concurrency can happen on a single core or a single task-switching thread. You don't necessarily need to split off parallelism to explain that, but it helps.
And it's worth talking about how you can have a single task run in a parallel way, for varying strictness of 'single'.
Coroutines and SIMD are far enough apart that their execution models should have different words.
Apple used to be great when it comes to text, but lately their policy has been to use the worst possible software implementation and hope there's enough pixels to make it work.
In other words I bet that subpixel-aware or even just pixel-shape-aware text rendering could avoid that fringing but they don't want to bother.
That's usually in hindsight? And I said it wasn't obviously insane to them, not that it would pencil out in the end.
Can also just be an asymmetric bet - relatively small outlay with low chance of working, but massive upside if it does, such that it's still positive expected value.
But mostly, I think all these armchair quarterbacks should not have very strong opinions about something that they almost certainly don't know much about compared to the people who build these. The person I responded to cited a random blog that ran some numbers, and includes such gems as "In space, a broken fan or a fried motherboard is a crisis." Just... lol
There are lots of motivations other than "this could be a profitable project". Something can be completely insane in a monetary way while not being insane in other ways. A pilot project being built is pretty weak evidence of larger practicality.
There are significant differences in feasibility and cost between putting something in low Earth orbit (or even very low orbit like Starlink) and putting it in any other orbit around anything else - like easily 2-5x depending on where you're going to put it. I'll leave it to the Kerbal Space Program veterans to explain the mechanics that also make it much more difficult to actually launch and deploy successfully unless you're happy with just having the DC arbitrarily orbit around the sun (and drift year-by-year).
Once you move outside low Earth orbit (i.e. outside the magnetosphere) you also have to deal with much more radiation so using regular commercially available hardware for the DC is no longer an option so you're looking at another 10-100x increase in cost for the chips alone (and at worse performance because radiation-hardened chips are literally generations behind at this point). You'll also need signficantly more (very heavy) shielding and that also adds to your launch costs. Of course OceanGate is a good reminder that billionaires don't always concern themselves with such trivialities as using equipment and materials actually suited for the task if it means they get an opportunity to brag about how they're disrupting the market by jerry-rigging mission critical components.
Speaking in terms of feasibility again, availability isn't the only problem you'd be facing. Obviously latency is going to be horrendous because of the incredible distances involved but really there's just literally no infrastructure you can piggyback off of. You could potentially achieve data rates of up to tens of Gbit/s even from L4/L5 (stable orbits 1 AU away) using something like NASA's DCOS but this requires optical ground stations or LEO relays, bringing the weakest link much more within reach again.
Even if you somehow manage to develop sufficiently ruggedized variants of reasonably modern chips, add the necessary shielding and develop and build out the necessary communication infrastructure from scratch (remember that we haven't even considered how we generate enough power to run the actual DC itself) and are also willing to accept the disgusting latency and packet loss ratio, the elephant in the room remains that DCs on the ground require constant maintenance to operate and replace failing parts. DCs in space can only ever be disposable - once any redundancies run out, they're just very expensive space junk.
But you have a point: putting it in space is inherently ridiculously expensive. Getting it to L4/L5 is only insignificantly more expensive than getting it to L1/L2 but assuming you can fit the entire DC in a single Falcon Heavy (so you don't also have to worry about assembling the entire thing in space across multiple launches) you're limited to a payload of 25 t and the launch itself already costs you $120 million. Of those 25 t only 7 t or so can be spent on actual compute so your space DC would probably be constrained to the equivalent of about 500 GPUs (assuming this is for AI because everything is for AI these days). Most of the weight budget and the actual cost however would be eaten by all the structural and infrastructural parts required - in terms of cost for compute a space DC that fits in a single Falcon Heavy is easily an order of magnitude more expensive than one built in the US today and those don't need to be replaced from scratch when a significant chunk of the initial parts have degraded.
TL;DR: Building a DC in space is easily 10x as expensive, requires technology that hasn't been developed yet, requires you to also build all infrastructure needed to actually operate and connect to it, and in the end just gives you a high latency low performance Beowulf cluster that is rapidly converting itself into an expensive piece of space junk if random debris doesn't kill it first.
Then why attribute the work to MiMo? People mostly only attribute the model when they've outsourced the understanding along with the coding. If I understand the code I don't bother to mention the tools I used to create it.
That's still an algorithm that underwent immense amounts of tuning. The tuning here on gzip is very simple, very few parameters, and generic. There is no reason to reject it.
Also I don't know about calling GPT-2 "really tiny". You can get coherent responses out of 5-10M parameters.
They wanted a basic explanation in response to that.
Or to put it differently, I would take the sentence "He asked for a purpose and a use-case not an explanation." and replace the word "not" with a comma.
And it's worth talking about how you can have a single task run in a parallel way, for varying strictness of 'single'.
Coroutines and SIMD are far enough apart that their execution models should have different words.
reply