Discrete event simulation frameworks, like SimPy, are a great mathematical way to simulate the environment if wanting to discover better methods of any methodoical staging flow. Elevator scheduling is one example that can be simulated.

https://simpy.readthedocs.io/en/latest/

The leverage is the trick in the whole equation. During dotcom days the leverage was fine until the cash flow ran out. Then bankruptcies started and the bottom fell out. It appears this is happening again but perhaps the business model will just evolve. Although right now it feels more of the same.

I'm not a sysadmin but I often hear about server failures due to hardware just wearing out. Is there an expected shelf life of the hardware? I would expect for sure on the storage devices. So if that were the case then why isn't there some sore of daemon or system software service that tracks the expected life of the hardware? I would expect that the software would start showing warnings of imminent useful life of devices to expire.

I'm running my own home server on an old 2014 Mac mini, 24/7 basically. It both has a SSD and a HDD (old "Fusion" disk type of machine, but I use both parts separately, the SSD part for the OS, the HDD part for the data.)

A few days ago the HDD had its 100,000 Power_On_Hours anniversary. :) According to SMART this is way past the expected/ designed life time of the disk (the normalised value reports "1" since a few years when the disk hit about 42,000 Power_On_Hours)

So you never know. It either works or it doesn't. You have backups for the worst case.

Not unlike a combustion engine, cycles can do more damage than hours.

My main desktop machine with assorted components is from 2014, also with an SSD and HDD. Each component has lasted all this time somehow. The total uptime of each drive is 11.5 years (101,023 hours) with 159 power cycles. The SSD is 256 GB and has 71 TB written, it's my main drive. It has 46% lifetime remaining, I hope everything lasts another 10+ years.

My SSD only reports as 121 GB. Dunno why it's such an odd size.

With 21,647,615 MiB (~20.64 TiB) written. Reporting as 99 (down from initial 100?) in the normalised value, but the normalised value hasn't changed since I first checked in 2015, so maybe it doesn't work? The wear leveling is at 180 (down from initial 200?), and the threshold at 100, so still ~80 to go until expected EOL. I'm losing between 1 and 2 points per year at the current rate.

I'm also hoping for another 10 years at least. *knock-on-wood*

Reminds me of the old forum posts where people show off their uptimes.

Uptime is fine, until you restart. And poof

There is for hard drives - SMART - the problem is, unless you're an enterprise (and let's all stop and laugh because we KNOW that most "enterprises" run at least some hardware until the only person who even knows it exists is long in the grave) you're not going to do preventative replacement.

And especially for home use, where nien 9s is more realistic than nine 9s, you're better off having a solid backup strategy and a source of spare parts.

My main "tank" server is 16 years old this year, and has blown out one motherboard - it was a bit annoying to be "offline" for a few days as a replacement wandered in from eBay, but it worked. If it happened today I might import the zpool into another device temporarily - or permanently, who knows? This R510 probably takes more power than a datacenter.

I run a k8s cluster in my homelab, with Longhorn providing storage. Except for some extra-large volumes for bulk storage (just movies in Jellyfin right now), everything is replicated three ways. I've had nodes fail to boot up after a power outage (turned out to be a dead CMOS battery) and procrastinated bringing them back up because everything just kept working.

That’s the biggest danger with hot fail over at homelab setting. It keeps working so there’s no reason to fix it.

My primary home zfs server is from 2010, running 24x7 since then. It has a 4-way mirror zfs pool for data. Over the years I have replaced two of the four drives, so two are still original from 2010. Everything else on the machine is from 2010.

You can get unlucky and have components fail quickly, but usually if they last, they'll last for a long time.

But trying to run a server from an SD card like in the article, seems silly. False economy, just get a good SSD.

For drive health monitoring there is SMART, but IME while it does monitor drive usage and lifetime, it rarely predicts catastrophic failure since that happens suddenly. Both drives that failed in this system went from healthy (according to SMART) to unresponsive overnight.

I run a fairly large homelab (probably ~10 nodes when including OPNSense routers, backup NAS at our cottage, etc.), and hardware failures have been incredibly rare, especially compared to what it used to be a couple of decades ago (I had quite a few motherboards fail due to blown capacitors due to the stolen electrolyte formula scandal).

Hardware failures tend to follow a bathtub curve: there are more frequent failures early on due to manufacturing defects, then they become rare until you reach the end of the natural lifespan of the equipment. That is shorter for some components (spinning rust HDDs, fans, CMOS batteries, power supplies in noisy environments), but extremely long for others.

Life expectancy really depends on how the hardware gets used. I run a small fleet of servers that are heavily CPU / GPU loaded, and the most frequent hardware failure I see is power supplies, with RAM as #2 and spinning HDDs as #3. Each of these (on enterprise-grade gear) has some amount of reporting / monitoring, but it's rarely the case that I get any advance warning before a failure. The RAM is probably the nicest, as it sometimes starts with correctable ECC warnings. Nearly all of the PSU failures have been sudden deaths, with the only saving grace being N+1 redundant supplies usually keeping the victim server alive.

> So if that were the case then why isn't there some sore of daemon or system software service that tracks the expected life of the hardware?

Some linux distribution will regularly check SMART data from disk and warn you if some threshold has passed (like number of corrected errors, spin up/down, total hours etc).

That said, if we talk about consumer hardware in a home lab or the like, of the disks that I bought in the past 15 years, only one broke down, and it did fairly quickly so my suggestion is to use redundant storage, make backup and change the disk when it breaks down. (Enterprise disk and usage is another can of worms of course).

Hardware wearing out is not common at the individual component level. It only becomes common when you are running a large number of servers, those individual small probabilities add up.

There are plenty of sensors and systems that can "predict" failure and in some cases try and work round them.

Do most people actually use them? Pretty rarely. S.M.A.R.T is just trash anyway.

But you do need to properly design your systems to limit the amount of "use". No un-needed writes to disk etc. Proper ambient detection, no point trying to rush to shut down servers when the AC pops it, and you just so happen to walk into a server room.

About the only thing that regularly wears out is spinning drives, and those have SMART data to give you an idea of when they'll die, but generally yif you're actually committing to high uptime you'll have redundant hardware so any one server failing doesn't take the service down, giving a bit of a grace period on hardware failures

SSDs have a fairly linear wear-out mechanism although the exact end point is unpredictable as always. HDDs fail randomly or from mechanical shock.

Shareholder primacy is a drain on so many levels. Think of all the R&D that could have been spent instead of buying back shares.

The share buybacks help offset these big companies’ RSUs for highly compensated employees. The share buybacks also allow shareholders to have a taxable event when they want to, not when the business wants them to.

It would be crazy not to do share buybacks for a growing business.

Your son should blog the experience. That could be an interesting read.

Financial risk management is a great industry for data science. I've been doing it for 15+ years. It is amazing the data rich environment which includes credit bureaus, customer transaction history, call center dynamics, and of course finances. It is a gold mine of opportunity to find new and fresh ways to observe the organization. This is also what makes risk management difficult as well. There is so much to know. LLMs are going to change risk management as it is changing every other industry. It will be interesting to see where it is headed.

That is not what Eric S. Raymond (esr) was describing.

GNUnix was developed using the Cathedral-style, Linux was developed using the bazaar-style. How Linux development was coordinated was thought to be impossible for something that had to be as solid as an operating system. The essay is a deep dive, exploring the conditions that the Linux project needed to ship an OS.

But ESR believed in right wing, libertarian adjacent politics. He's advocating for deregulated, free market ideas in the form of criticizing GNU. In doing this, he was seeking out the preferred metaphor and working backwards, rather than describing what is.

Adding the letter archive into a LLM prompt is a good read. you can use this as a source https://www.berkshirehathaway.com/letters/letters.html

perhaps I'm missing something. Why not start the learning at a later state?

If the goal is to achieve end-to-end learning that would be cheating.

If you sat down to solve a problem you’ve never seen before you wouldn’t even know what a valid “later state” looking like.

Why is it cheating? We literally teach sports this way? Often times you teach sports by learning in scaled down scenarios. I see no reason this should be different.

If the goal is to learn how to solve a Rubik's Cube when you've never seen a Rubik's Cube before, you have no idea what "halfway solved" even looks like.

This is precisely how RL worked for learning Atari games: you don't start with the game halfway solved and then claim the AI solved the end-to-end problem on its own.

The goal in these scenarios is for the machine to solve the problem with no prior information.

This isn't accurate, though? Halfway solved, for most teachings, is to have the first layer solved.

Indeed, this is a key to teaching people to know how to advance. Do not focus on a side, but learn to advance a layer.

That's effectively what you get in either case. With MLM, on the first learning iteration you might only mask exactly one token per sequence. This is equivalent to starting learning at a later state. The direction of the curriculum flows toward more and more of these being masked over time, which is equivalent to starting from earlier and earlier states. Eventually, you mask 100% of the sequence and you are starting from zero.

behind paywall. Amateur sleuth from West Virginia may have decoded the Z13 cipher and solved the mystery of the Zodiac killings identity.