You can't stay engaged if the menus don't bleep and bloop at you and confetti rains down on every click? That's a personal problem. A minimal UI is not automatically a bad-looking UI.
No, but I want to click to place the cursor in my 10 line prompt wherever I want. I want to double click to mark words. I want to use a scroll bar. I want to copy paste properly and not "23 lines pasted". I don’t want to read docs to find all keystrokes the TUI supports to achieve what I want.
>I want to click to place the cursor in my 10 line prompt wherever I want. I want to double click to mark words. I want to use a scroll bar. I want to copy paste properly and not "23 lines pasted"
sounds like your terminal emulator is just crap, terminal and iterm on mac just work
Unrelated to the article itself: The login-wall on some random website I've never even seen before feels mildly disrespectful of my time. Kind of similar in vibe to making you download an app to order in a restaurant.
I scan to see if there are one of those dickovers, and if there is I check the archive mirror, but in this case it also has the login wall.
Eh, click back and move on with life. I am not signing up to every fringe website for one off articles. And from the site's perspective losing me as a reader is no loss to them either, because I'm never coming back, run ad blockers, etc.
You got to have a paid subscription to read the article. The main output of remap radio is a (number of) podcast(s). The written game criticism articles is an extra.
They claim "By signing up—again, it costs nothing!—you can read the rest". If that's false, that not only a really shitty move but should be a crime - they're obtaining something of value (your e-mail address, no doubt to spam it with subscription ads) under false pretenses.
I quite liked it actually. I didn't want any more than I got, which I got for free. I would have found a paywall unsatisfactory since then I'd get nothing.
In the right space with real people, worthy of a debate.
On HN? No. Not like this. Not here. Not without filtering the participants for real human beings.
Personal note: That principle has been the bane of my autistic existence. People sometimes seem literally incapable of understanding that other people even can be different.
It is a bit strange, like being surprised some countries drive on the left side. xkcd 1053 of course, but people are still allowed to be surprised when someone went their entire life without picking up a fairly common factoid.
I'm not saying you're wrong, much like a gun really would help a victim of CPTSD feel safe.
What I was trying to point to was that "this thing helps some people" does not equal "this thing is unequivocally Good and should be entirely unchecked".
I don't even care about the AI. I just get peeved by bad lines of argumentation.
In the literal sense that a gun is not a chatbot? True.
In the sense that both your post and my analogy use the argument "this is assistive technology" to defend something which only in a very narrow slice of their thing-ness actually is assistive and in the whole rest of usages are much more, and not only Good, which both you and my "futurama caricature" were willfully ignoring? I think it's quite illustrative.
I notice the "Limitations" section talks about how content only at some point touched by Claude may return a positive, and content that returns a negative may still be Claude generated. But I really would have liked for them to state explicitly that entirely false positives where a piece is fully human-written may still be marked as generated, because too many institutions with the power to ruin someone's life over that have trouble understanding the concept.
This. FN rates are cute, but FP rates will ruin an academic career or a student's work/further study choices if their content gets marked erroneously. Surely the answer is a sequence of marks?
Keen to see if they are doing something SynthID-esque?
Do they care about false positives? As long as it’s even somewhat reliable that’s enough for them to prevent training on their own slop. I think this is a big reason to do this that’s overlooked.
Good point. But if that were their only purpose, there'd be no need to share it with anybody. In fact, they'd get the best results by not mentioning it.
That’s true. But they probably want to be able to identify other models slop as well. And with the laws popping up, it makes sense to do it the way they are.
> But I really would have liked for them to state explicitly that entirely false positives where a piece is fully human-written may still be marked as generated, because too many institutions with the power to ruin someone's life over that have trouble understanding the concept.
This is marketing material aimed, in part, at encouraging the usage you are concerned about, which is why they do not highlight that problem.
I think that false positives are inevitable due to the method of watermarking being embedded in the text itself. The output is intended to mimic human writing, therefore it's entirely conceivable that a human could by chance write text that contains the watermark. The odds may be extremely small, but it's not something you could ever guarantee.
I read the original paper they're basing this off of and I think you're right. I do wonder how much of a quality tradeoff there is with perturbing the next token probability distribution. My intuition tells me that a more "prominent" watermark will necessarily degrade output quality. If they are trying to balance quality and watermark prominence, I wonder if that affects the FPR.
You're absolutely right! Humans have been slowly thinking and writing more and more like AI. As people get more and more exposed to the stochastic patterns of large language model tools, it's normal for them to emulate the styles of communication they are exposed to. This is commonly called "brainrot" by those in Gen Z and younger cohorts.
If you find yourself getting to be afflicted by this "brainrot", be sure to go outside and take a moment to ponder what's around you. The grass is there and will be there long after we are all gone. Consider this for a moment as your organic thought processing unit starts to slowly munch away at its internal context window.
It’s worse than that, false positives are possible but someone generating text should be able to get ai to change some words and formatting to break the watermarking, then ai detectors can tell them how well they did.
I don’t know what the answer but I absolutely know it isn’t this.
I think we need a chain of custody system for content, but that would require browsers, software, websites, operating systems, phones, camera manufacturers, etc to all get on board. But each intermediary or source (optionally) cryptographicaly signs a piece of content that it either generates, edits, or passes along, and the end result at a destination, is that content is either 'trusted' if its cryptographic chain is solid, or un-trusted otherwise.
> I think we need a chain of custody system for content, but that would require browsers, software, websites, operating systems, phones, camera manufacturers, etc to all get on board.
It would also require the individual humans you are trying to control to get on board otherwise the analog hole breaks the chain, absent mindboggling levels of physical surveillance on top of the the total monitoring of all electronic data flows that this idea requires.
If I'm like a high school/uni student, where this seems to really matter at the moment, you can still just have the LLM generate it and type it word for word in whatever text editor you're using, right? Less convenient but still probably easier than doing whatever work was necessary + still typing it all up. Chain of custody would show my keyboard really typed each stroke or w/e, but the underlying work is still generated
This is a meme video but I think it hits the nail on the head.
TLDR: AI will force global online digital ID for everyone that uses the internet for the exact reason you mentioned. And that would forever change free speech forever allowing the powers that be to put the genie "back in the bottle" so to speak.
If there is any false positive rate (which, because text will naturally and by chance include tokens from the green and red sets in some pattern, there will be), tools making promises like "detect AI-generated text" are unacceptable. They are going to turn innocent people into pariahs on some unsubstantiated "this content is 37% likely to be AI" claim that the user has no way of verifying or inspecting more deeply, we just have to trust the statistical box and assign some meaning to whatever that number means. 37% of my phrases are AI? There's a 37% chance my entire text is AI written? Part of the fun is not knowing!
This is scripture homeopathy and it's irresponsible.
I'm curious about your thoughts on pangram. I only really see posts on Reddit claiming it falsely labels their content as ai generated but nobody will actually post examples of "textbook from twenty years ago" or upload screenshots of a journal (also those posts usually feel deeply ai generated without an ai detector)
Do you think this is an impossible task and we shouldn't try to solve it? Or do you think it's doable and that some ai detectors might be better than others?
This feels testable - you could go to fanfiction or similar sites with billions of words of writing from before 2016 or so and run them through it.
I tried a chapter just now and got human doing that, but I'm not invested enough to run a hundred samples today. But it sounds like it would be an alright way to audit it? I will confess I'm pretty skeptical you could ever eliminate false positives here though. I can often get an ai sense from some writing on my own but I doubt it would be better than 90% accurate, and "ai plus human editing" might screw with that anyway, stuff like that. I would have preferred we just never developed this kind of thing so I wouldn't have to guess.
That still might work better with older texts. As AI-generated text gets more prevalent, I'm guessing people will start subconsciously adopting AI writing styles.
Yeah, that's one of my questions. Everyone who talks to AI for too long seems to get worse at writing anyway, and humans mirror any form of conversation to some extent.
pangram is pretty good; i use it all of the time and pay for it. surprised that it's not mentioned that often here. they just released a new model that is supposed to lower the fpr (false positive rate) even further than it's already impossibly low score. it also detects AI in images now, though I expect the fpr to be pretty high there given its newness.
> But I really would have liked for them to state explicitly that entirely false positives where a piece is fully human-written may still be marked as generated
Are you talking about pieces that were fully human-written with zero AI editing/rewriting etc? If so, what makes you think that false positives will happen there? They aren't looking for "writing styles" or emdashes etc. They are using watermarks and metadata.
If you're talking about people using AI to copy-edit text they manually wrote, this was explicitly called out in the article:
> A detected mark provides a signal that content was processed by Claude, but is not fully conclusive. Detecting a Claude mark tells you that the content may have been processed by Claude. It does not, on its own, confirm the full provenance of the content. For example: Claude may not be the original author. People often use Claude to proofread, translate, summarize, or convert files. The output can carry a Claude mark even if the underlying ideas, text, or data originated from another source; The content may have changed after Claude processed it. Marked content may be modified, excerpted, or combined with other material after Claude processed it.
The former. I'm not sure what you mean by metadata, but my expectation was that anything that Claude could put into the plaintext to identify itself may plausibly also accidentally be produced by [a million monkeys on typewriters/one in a million human writers], since in the end, the writing is using the same language and symbols that humans use. How unique could the LLM possibly make it while still retaining its usefulness?
> How unique could the LLM possibly make it while still retaining its usefulness?
They could be doing invisible and vaguely-harmless Unicode stuff. Insertion of zero-width joiners and non-joiners, replacement of regular spaces with non-breaking spaces, building spaces from multiple hairline spaces, intentional use of non-NFC-normalized codepoint sequences for accented characters, etc.
Text with all this junk in it still reads the same; it just might wrap a little strangely, or not byte-match / collate correctly in a database (and Anthropic has never made a guarantee that their models would be capable of emitting text with these properties, so that’s fine.)
And, importantly, no regular text or document editor would insert these things (especially in the useless places you could insert them for watermarking.) You only really see them in text that’s been explicitly typeset for a specific layout (e.g. in text-containing SVGs, website mastheads, or game HUDs) or for print publication.
Of course, if this is the technique they end up using, then it’s very simple to strip it out by canonicalizing the text (i.e. Unicode-normalizing it + stripping out invisible layout characters + replacing “weird spaces” with regular ones, etc. Essentially the same thing many sites already do to user-generated content to prevent users from using Unicode features to break the page’s layout.
Sure, there are things you could do legally when falsely accused; and there are things authorities and companies should do.
But ultimately, when you are powerless and can't afford to do the fighting: I'm convinced the only way to protect yourself is to be very mindful about your writing style, and to deliberately corrupt the language through objectively wrong "stylistic elements".
Because it won't be in the training directly. It is applied after a model generates its distribution of likely tokens, biasing each token randomly based on a random key and unrelated to any meaning of the words. So half the time, the most likely token becomes more likely and half the time it becomes less likely, and the same for every other token (when temperature is above 0).
You then look at the tokens actually picked to see how closely they follow this pattern that isn't connected to the meaning of the tokens. With enough text, you can then analyze the chance of it happening by chance verses being because the generation of the tokens was done using the algorithm, and you can save a positive result until you are arbitrarily sure. There is a chance of a false positive, but the chance of a false positive approaches the chance that the murderer happened to have fingerprints that matched your and both forensics labs happened to have mixed up the dna tests and the eye witness happened to misremember the face and your phone gps happened to glitch out and put you at the murder scene at the time of the crime all happening. It is theoretically possible only in the same sense that quantum teleporting a cat is theoretically possible.
The real question is how much text do they need for a given level of certainty and what do they check for. If they flag a positive at a p value <.01, that's a problem. If they can reasonably get a p value of < 1e-12 in only a few paragraphs of text, that is effectively no false positives (but a lot of 'too short to analyze' outcomes).
How do you verify in practice then? Wouldn't you need the original prompt so you can reobtain the likely token distribution to validate again the random key(s)?
What about "watermarked long-form code"? I'm having a hard time understanding how a model could watermark not prose, but functional/semantic text like code, that actually has meaning. You can't switch our the characters, you can't use various types of whitespace, you can't add arbitrary code comments, and a lot of other restrictions. Is there any state of the art methods for watermarking code without affecting the quality/correctness?
For straight generated code it'll likely need more text, but it'll still show up.
In cases where one token is extremely likely, it'll randomly be red or green and still be picked in either case as it is simply the best (or only) option. So you'll have more tokens that don't show a pattern either way (half of these cases will match and half won't, just the same as if a human wrote it). Meaning you'll need more instances where multiple tokens were all likely to see if there is a pattern. Given the check algorithm can't identify these cases, it can only judge on the overall text, so the more strict a language, the more the length requirement scales.
Where I wonder if this keeps working is in tool calls. Often, you don't take code straight from the llm, you take the results of a tool call to edit already existing code. It might be that the result of this leads to far too few signals to pick up, meaning that this only works when one does significant generation with a single model (even swapping between different models, at least by different companies, breaks this just as much as having a human write parts of the code).
Think of it like finding a loaded dice. A dice that has a slight bias in a few dozen roles is just random chance. If that bias continues after hundreds of thousands of roles, the dice is loaded. But will a code base have enough samples, especially when edits made from tool calls? I could see this being unable to detect things at the size of a reasonable PR and only being useful for massive sets of changes and only if the person behind them didn't structure their AI usage to avoid detection.
As anybody who has put together a coding standard knows, there are a lot of options for individual expression, meaning a lot of room for things like watermarking. And of course you can add arbitrary comments; my Claude-generated code is very verbose.
> there are a lot of options for individual expression, meaning a lot of room for things like watermarking
The way I use LLMs (and I'd advice everyone to do the same) there really isn't, the agent implements things exactly how I want them, or I use the agent to massage it into the exact bit-by-bit version I imagined when I first sent the prompt afterwards. I honestly don't know what the point would be to let the agents compose worse code than what I'd do manually, although I know it's a popular approach taken by many.
> And of course you can add arbitrary comments; my Claude-generated code is very verbose.
So watermarking for all users who allow code comments from agents, no watermarking for us who force the agents to never write a single code comment? Alright, I'd be fine with that.
From what I've seen, your approach to LLMs is exceedingly rare, so I suspect it's one the people who care about watermarking aren't very concerned with.
And the reason to let Claude make worse code than a professional would by hand is basically suppressed demand. Since programmers are expensive, previously code mostly got written when a large number of dollars were on the line, or when an individual programmer did something not economically optimum (e.g., hobby project).
That left a whole lot of somewhat less valuable software unwritten. It's the economic space that no-code tools have been nibbling on for years. One way to think of things like Claude Code is as effectively no-code tools. Pre-LLM no-code tools would produce data structures that got executed by special environments without ever being seen or tuned by a human. Claude Code can be used just like that, with text as the input and python as the intermediate representation that nobody ever looks at.
That approach probably isn't sustainable for what we professional programmers would call a serious project. Claude can easily get in over its head and I expect that its code decays over time, in a fashion similar to how many human teams get in a state where they just have to rewrite everything. But faster, I'd expect.
But there are a lot of unserious projects that previously would have never been created. E.g., a quick app to manage your little league team, or a bit of in-house business stuff in the "a little hard to do with a spreadsheet" range.
> I honestly don't know what the point would be to let the agents compose worse code than what I'd do manually
You never generate throwaway code used to test an external service? or try out an interface idea? There's a lot of code that's only meant to be ran once. I often dont even care what language it's written in.
> You never generate throwaway code used to test an external service? or try out an interface idea?
And save/persist it? No, most of any experimental stuff goes into /tmp which gets cleared out on reboot, nothing I care to save in any repository. Or just "show me how this would look like" and then it's only in the session itself (and the logs/state I suppose, technically...).
I think they mean it like this: imagine you ask me a random number sequence. I give you a random number sequence. Little did you know, I used a very specific PRNG to generate it, so later I can prove with certainty that your number was generated by me, and you can't say you came up with it yourself.
There is no room for false positive here in the same way you can't randomly find a collision in a hash function if it's strong enough. Like the rate is so infinitesimal that it is effectively zero.
Now replace random number sequence with prompted string of words. And instead of using the PRNG on every word I use it every n words. If the generated text is sufficiently long I can tell by matching the expected deterministic pattern.
You can defeat it by changing the words yourself and triggering a false negative but there isn't really any room for a false positive if the text is long enough and the pattern matches perfectly. If the pattern doesn't match then I can compute a probability.
Sorry you're getting downvoted, this interpretation doesn't seem that far fetched to me.
Here's the strawman: The text-based watermarking is going to be done procedurally instead of generatively. Maybe they add some sequence of zero-width Unicode characters to all generated text at certain intervals. Then, there is effectively no false positive possible (because humans would [effectively] never type such sequences of unicode naturally). It may survive some editing (depending on how you select/edit the characters), and it's possible to be stripped (false negatives).
You can't stay engaged if the menus don't bleep and bloop at you and confetti rains down on every click? That's a personal problem. A minimal UI is not automatically a bad-looking UI.