> When the unit doesn’t work correctly, the anger and RMA requests are directed back at the Raspberry Pi foundation.
They don't need the resellers help for that! I've already been burned twice thinking "I'll use a raspberry pi for this small project" and having it die within a month because reliably reading/writing to an SD card is hard or something.
Between the notorious unreliability, crazy price hikes, trademark or whatever disputes, and this anti consumer ewaste generating policy, I can't think of any reason I'd ever patronize them again.. my only regret is it took me two purchases to learn this lesson!
> Find unused code in our web app. Exclude generated files and test fixtures, and check for indirect references before recommending a removal.
If I "/goal remove unused code" in Claude today, I would not even think to specify "check for indirect references" and "don't consider generated code dead code," those sorts of intuitions have been "built in" to the frontier models for a while now.
So I'm pretty confused by this, what does it do that I can't with an agent swarm?
> I would have guessed that reliably identifying LLM generated text was not possible
It depends what you mean by "reliably." If you mean, "we should be comfortable relying on this kind of tool at scale to identify and punish students, professionals, and writers who may have used AI," absolutely not.
If you take "reliably" to mean "1 in 200 false positive rate" as they disclose on their front page, absolutely that is possible (they are doing it today!). If you think there are more than 200 assignments turned in over a given year at university, you probably do not consider a tool like this fit for purpose. It's an open question whether those procuring said tool are aware of this
Unfortunately their marketing is really insisting on the former, and trying to push it into the zeitgeist that detection of AI-generated or edited text is reliable-type-1 now and long-term. They fail to make it clear that this is merely a tool that strongly suggests text follows patterns known to us at the present time of known LLMs. However, that fingerprint will drift over time, as LLMs get better, human writing style evolves, and the line between human and "smart autocorrect" becomes even blurrier (does speech-to-text push the model into "AI assisted" mode, because it tidied up your punctuation, for example?)
> If you take "reliably" to mean "1 in 200 false positive rate" as they disclose on their front page
Pangram claims a 1 in 10,000 false positive rate (rate at which human-authored texts are incorrectly classified as AI-generated). 1 in 200 sounds like the false negative rate (rate at which AI-generated texts are classified as human-authored), or perhaps a rate for a specific category of text.
Good point of precision, I read their site too quickly. I don't think my argument materially changes with that number instead however. There were over 40,000 students enrolled in my university alone. Generously assuming they only turn in one assignment per year, having 4 of them go through the "computer says you cheated, and as you can see, it's 99.9% accurate" gauntlet is not a price we should be willing to pay for... the marginal benefit of this tool over more classical ways to proctor and assess pupils
While that is not quite my bar of confidence when implementing wide-reaching technologies that have numerous unexplored knock-on effects, I guess the calculus must have been different on Infinite Loop recently.
The fundamental issue isn't technical. It's that people will see the "certified real" tag and just take the image for face value of whatever narrative someone wants to convey. They'll see the "Real Photo, Verified by Apple" and their brain will short circuit [0]
I don't think we should have this, for that reason alone (but many others too).
I’m pretty sure “certified real” aren’t the words Apple will use, nor do they use it in this document. The words to describe the technology were chosen with care: semantic verification, attestation, tamper evident, etc.
You seem to be missing the point that’s being made here. Of course apple will word this very carefully. This does not matter in how people will interpret it. They will interpret it as certified real.
As much I love a good old "Why don't you just ..." response that terrifically misses the point - in this particular case I was watching YouTube through their Apple TV app.
Now, there may be another "why don't you just ..." or "well, actually ..." response you have queued up, maybe some ramblings about a pi hole, or some anti-Apple hate, or some other solutioneering. Go ahead, tell me.
But it is the structure of incentive that forces this. The YouTube creators that make the content do need to get paid. And Google offers an option: Youtube Premium. That give both me and the creator what we want and (for the time being) removes the ads.
So either one is rich in time (doing whatever "why don't you just ..." hoop you expect me to jump through) or rich in money.
If you want creators to get paid, pay them (channel memberships, patreon, click on their sponsor links and buy something). No need to subject yourself to ads.
Ads don't just steal your time either, they invade your head. You are influenced by them and you have no choice as long as you watch them.
That is wrong on both accounts. I have little choice other than to watch ads since the structure of incentive that creates a marketplace like YouTube forces it on me.
I think about libraries and how the world would be if instead of a free public resource, you had to watch ads before you were allowed to take out a book.
It is just the case that in this modern social media world, if you want your ideas to spread you have to put them onto social media. And if I want access to those ideas I have to consume them through social media. There is nothing about that environment that is my choice, it is the environment that I find myself in.
And I have few choices to get around it. I can twist up into a pretzel trying to block the onslaught with more "why don't you just..." advice. Or I could pay to make it go away. Or I could just go off-grid and forego access.
And when you look close at the options, it becomes clear that sovereignty of my own mind has a price. Freedom of mind is not free in this modern world, if it ever really was.
You made the choice to use Apple TV, whatever that is. I just use... YouTube. In my browser. Do you not have a browser? It doesn't cost money.
You also have no obligation to watch ads for creators to get paid. They can also set up things like Patreon if they would like. The ads are very lucrative, and that's their choice. But you are free to make choices too, if you would like to.
> The YouTube creators that make the content do need to get paid
Nope.
They may get paid, they may not. That's the risk they've chosen. They may also get their Google account banned with no recourse for something minor or even non-existent.
> So either one is rich in time (doing whatever "why don't you just ..." hoop you expect me to jump through) or rich in money.
That's what they want you to think, you've been watching too many ads.
Eh, I hope Apple continues to provide this and it forces the needed discussion about how two-party consent requirements are nonsensical. Why should it be illegal for me to remember exactly a conversation that I participated in, instead of only being allowed to have a vague recollection?
Laws like this provide cover for abusers and deceivers, by preemptively spoiling objective evidence and making any accusations depend on hearsay instead.
It's not illegal for you to remember, it's illegal for you to record. There's a difference.
For me, I don't want to live in a world where if I say something embarrassing or not well thought out, someone pushes a button and the last 15 seconds is transcribed as evidence. That's a different world to the previous one where it would be someone's word fori it.
As for the fact that phone could already do this, that's not the point. Phone users have to go out of their way to make it happen so it's generally unlikely to happen. The watch feature though is always on and just waiting for you to press "save last 15 seconds"
Same with the always-on transcription feature that you dont even need to interact with. I like not having to speak with extreme precision, knowing my words are going into some record.
> Eh, I hope Apple continues to provide this and it forces the needed discussion about how two-party consent requirements are nonsensical
Really disagree on this, I think all states should be two party consent, personally.
> Why should it be illegal for me to remember exactly a conversation that I participated in, instead of only being allowed to have a vague recollection?
It's not illegal, it's just that the other person has to know that you're doing that and consent to it.
Giving them the chance to walk away or to tell a person and their Meta glasses to fuck off is important.
Our politicians and legislators lie on camera constantly. But it doesn't really seem to matter much these days. So I'm not sure why this would make a difference.
Everyone being liars means you and I are both liars, same as the politicians. But the politicians have the power and if we remove two party consent then they get to surreptitiously record you and leverage that power. It doesnt even the power playing field.
We can alreay write notes down for every conversation and then send them to the person involved saying "we talked about X, Y, and Z." That last step is the key because it lets them object in writing if you mischaracterize things. From a "catching someone in a lie" the most important step is that one, because you form a paper trail where the other party can correct or contest what was written and bring that up now, and the fact that they didn't is itself evidence in case of a dispute later. The apple watch feature doesn't do that, it just dragnets everything. Even if it recorded the audio, we are in a faked-audio world so unless you have some signal that they agreed that they said a thing ahead of time, they can always deny it later.
> They were coordinating with OpenAI regarding a publishing timeline, but could not come to an agreement,
Skimming the PDFs it seems much more dramatic than that? It sounds like at least one of them is concerned OpenAI "solved" the problem by having their internal model use the chats of the independent researchers and want to claim the credit instead? I don't know. The tone is pretty accusational though:
> the one Levent and I had quietly chosen to
attack. Almost nobody else I know of was working on it. It is not the direction
one arrives at in a few days by giving a model the problem statement. When I
heard “forced,” it was a bright red flag.
> I was shown a prompt and told the internal research model had simply been
given the problem statement. Levent had been told by Sebastien “very little
human input” had been used. This turned out not to be true. Over the course
of the call, as members of their team sent Sebastien corrections and details over
their internal chat, it emerged that an entire team had been working on the
problem, that this was one of a number of things that was tried, that work had
started on the unforced problem, that the team first set the model on easier
problems, including Euler, that even the prompt that had been shown to me
had been written by prompting Codex, and that an insane amount of compute
had been used.
> I asked when the first prompt had been sent by them. This question was
not answered directly by OpenAI for some time. Eventually it was agreed that
it had been sent in the past few days, after information about our work had
reached OpenAI.
> I asked whether the model had been trained on, or had access to, our sessions
in Codex, into which we had been putting all our drafts for the whole of this
project. I was told the model did not look up user data. I asked again, about
training, and I did not get an answer. [0]
I find the framing a little strange, a sort of David vs Goliath (with his enormous computational resources at his disposal). Since Levent is at Anthropic whose internal models are presumably as capable as anything OpenAI has. So why wasn't Anthropic behind their effort? Why did Tristan use OpenAI's models when it should have been known was a potential outcome? I understand they wanted a normal math collaboration but presumably what Levent brought was his resources (as far as I can see Navier-Stokes is not his speciality). Normally these things are hashed out formally beforehand to avoid the sort of thing now happening.
They were working on it for almost a year, and Buckmaster has evidently been interested in Navier-Stokes for a while. This seems to be more of an innocent collaboration between two researchers than a strong company PR effort. Maybe Anthropic should have stepped in and made a large team to help them finish the proof (and maybe they tried and didn't succeed, who knows).
If what he wrote is accurate, it does suggest that OAI is effectively extremely hostile to cutting edge researchers (eg, if we hear rumors about your partial success on a problem that has huge PR benefits, then we'll assemble a strike team of researchers with unlimited compute to claim the win for ourselves, possibly by training on your data). It's also not a good look for them to request author removals based on company affiliations.
I think what you have in mind is more appropriate for more normal corporate projects and the like. But academic collaborations are not usually so political/'profit' driven, if that makes sense.
Presumably because this was something Levent did in his spare time and because it was not obvious that this work would eventually lead to a breakthrough.
> Why did Tristan use OpenAI's models when it should have been known was a potential outcome?
I'm sure in the past he had less cynical feelings about OpenAI and their penchant for academic fraud.
> I understand they wanted a normal math collaboration but presumably what Levent brought was his resources (as far as I can see Navier-Stokes is not his speciality)
I think you're not giving the guy enough credit in saying that his contribution came down to having an API key for Anthropic models.
> Normally these things are hashed out formally beforehand to avoid the sort of thing now happening.
How would that have helped? That agreement (which may well still exist) would not have involved OpenAI.
So what do you think his contribution was? His preprint record shows no research on fluids - and the statement says that the first LLM-generated proof Tristan received from Levent was 'the most horrendous I have ever read.' Levent is out for mathematical scalps whether it is in his field of expertise or not, and he has the resources to do it. And I am not saying he is not a very clever person, but the idea that you can bring yourself up to the forefront of research in PDEs, in particular NS, and contribute new ideas in less than a year is implausible.
They have messed things up, because Levent has a conflict of interest between his job at Anthropic and this independent work, and Tristan should have opted out of OpenAI training on their work (he probably didn't know about this). This doesn't justify OpenAI's despicable attempt to steal their work.
Yeah these are major accusations. But the story is incomplete, the conversation is missing a lot of details. It's not clear who was working on what, and when. The entire thing feels rushed, like they wanted to get this result published and out the door quickly.
There are various forces at play here, academic honesty requires them to disclose any inputs regardless of license or ToS circumstances.
While common sense reminds us here that if you send your data to an external entity’s computer, you are no longer in control of said data. The lines have blurred here clearly over the last decade, but that should have made the theory yet more clear to everyone involved: your data will be vacuumed up unless you keep it sealed. Use your own computer if you want to be in control.
But if they didn't opt out of training, did they want OpenAI to opt out for them? Also they need to audit anyone they sent drafts to to make sure they opted out before submitting it.
I'd prefer things be opt in, and especially not start opt out, then try to trick you opt in with a popup defaulting to opt-in, like Anthropic did on consumer plans, but if they submitted anything on an opted-in plan it's not reasonable to be mad it trained on it.
Even still, I also believe for significant reasons that OpenAI would ignore the opt-out in selective cases and could be in the wrong here.
And the threats and terms they offered seem wrong either way, pending more context.
If you don't trust the other party, then it doesn't matter how the checkbox is set. The fundamental rule, IMO, is don't send precious or secret data to a third party.
They don't need the resellers help for that! I've already been burned twice thinking "I'll use a raspberry pi for this small project" and having it die within a month because reliably reading/writing to an SD card is hard or something.
Between the notorious unreliability, crazy price hikes, trademark or whatever disputes, and this anti consumer ewaste generating policy, I can't think of any reason I'd ever patronize them again.. my only regret is it took me two purchases to learn this lesson!
reply