While I'm excited for more entrants in this space. It's depressing that there's no way to measure how much effort was put in a piece of software.

The distrust I feel for "Show HN" posts is depressing.

I know this will sound absurd, but in the good old days, even the claim of having built something indicated quality. This hubris is not limited to the front page, I've experience new products from well establish companies that are horseshit because they vibe code without doing any real QA or UX polish.

The convergence of claim with product as opposed vaporware is toxic.

I get the same feeling, it's difficult to estimate if it's just a vibe-coded platform, or a carefully crafted project; if it's a one-man job that can crumble at any time, or a stable service powered by a motivated team. The comparison with other services is also not super honest, cherry-picking elements to put in bold (e.g. the higher price than GitLab that is highlighted, or the Agent access), when it has nothing to be ashamed of.

OP seems quite knowledgeable (e.g. https://ogirardot.writizzy.blog/p/rickub-1-8-faster-then-git...), and their company ARUKU has been established in 2011, which makes me a bit more confident in this platform. But there are no details about the team, the software, since how much time they're working on it, etc.

At the same time, if you just use public repositories and synchronize them somewhere else for backup, there's not much risk in testing it. There's a free tier, they do have some neat features (that they don't really advertise) if you come from GitHub, like an import process that can copy PRs and issues or good compatibility with Actions.

The UI definitely needs a bit of love. While it's quite similar to other forges (you're not lost in the interface), it feels a bit rough. And the scrollbar presence or absence pushing the whole page left or right really bothers me.

Maybe to signal the opposite. I've started using double dashes ever since LLMs stole my favorite symbol.

Why do we hope to use the same model as its own guardrail?

This approach routinely fails with a single stream of consciousness. I can't count the number of times I've had to talk myself out of doing something stupid.

In the same way, a guardrail could inject thoughts like "...but I shouldn't do that..." "...I must remember to respect..." "...these ants deserve compassion."

The guardrail could even go as far as rewriting the thoughts of a model about to go rogue.

Presumably because of performance. It'd work well though, I imagine.

Humor me and suspend disbelief.

If these models are such an existential threat to humanity, why are they controlled by two private companies?

We might as well give Anthropic our nukes too.

> If these models are such an existential threat to humanity, why are they controlled by two private companies?

The government routinely contracts with private companies to create arms and munitions. Or do you think the bombs are delivered without payloads?

Maybe we can ask in a different way: would we allow very rich individuals to obtain nukes?

This has been a thing well before chat.com even among engineers at our company.

Chat Gee Pee Tee is a mouthful.

I hope everyone recognizes they aim to make you dependent on their intelligence rather than your own.

But there's open models that are (pessimistically) 1 year behind? It's not like if openai decided to rugpull everyone, we'd be going back to writing code by hand and all the developers who forgot how to fizzbuzz would be screwed.

> we'd be going back to writing code by hand

Well, as as SWE who writes code by hand and has no intention to outsource their intelligence to any entity, I felt slightly offended by "back".

Are you a member of a team? Then you are the outsourced intelligence. Your manager is not writing everything by hand, they are outsourcing it to you. Just understand that may change (I mean I am already mentally prepared for it to be a thing of the past within years). If you are purely writing software for hobby/fun then disregard, nothing will stop you from doing that.

I have no problem with other people outsourcing their intelligence to me, I've been doing this for years. But I prefer not to outsource my intelligence, at least not for things I care about, for various reasons.

Do you use an operating system? What about databases? Or the myriad of other software used which we build upon. I can't imagine a professional coding context where we don't outsource intelligence to some degree, if not a major degree. We stand on the shoulders of giants as they say.

It's not a problem at all. It's only for the things I care about and I consider important for a couple of reasons (maintaining and developing mental models etc.).

Open models still need compute.

Open AI rents compute. It seems unlikely given the huge investments they have already spent that they can profitably match someone who is just selling GPU time.

Open weight models are way closer to frontiers than that

An open model is not your brain.

There's an escape valve: I've had Claude stand-up AI-enabled features in my app, so we're much less dependent on Claude itself. The app uses cheap API calls to check code quality and run other "lessons learned" sweeps. Some of these become regular code in the end as well.

Same. Claude "loves" writing tools to make checks deterministic so much that I actually have to pull it back sometimes and point out that some things need to keep using LLMs. But even then, I have it "farm out" a lot of things to scripts using dirt-cheap models for things I don't need a Sonnet or higher level model for as well. The proportion that runs on the SOTA models keeps dropping.

I reject your intelligence and substitute my own?

The issue is relying on an LLM to perform accurate OCR and then using it to interpret art.

Tooling my ass. They can see all the transcripts in realtime and could easily have had another agent evaluate.

Is it just me our does it seem like OpenAI isn't auditing their agent transcripts at all?

None of these LLMs are plastic. They lack neurodiversity. Their thought space and their traversal are likely constrained in someway that humanity's isn't as a collective.

Is it possible to have neuroplasticity and still keep them aligned?

I have neuroplasticity, and I am aligned. ;)

Are you (at least partially) aligned out of existential fear of repercussions (getting fired, losing your life, going to prison), which LLMs don't have?

Humans are famously horribly aligned, plenty of examples in history.

Hmm. Examples of horrible alignmnent don't necessarily outweigh the fact that most people, most of the time, mostly behave in a way that is socially aligned. (Though I'm speaking in terms of intent, conveniently ignoring the side effects / negative externalities of our collective behavior.)

When threatened with physical or social harm. What they think in their little heads though :)

But are other people?

how do you imagine that would work? you being able to. influence globally stored weights with some prompts? we have fine-tuning for that.

Yet I can't randomly order another person to steal a car for me, just because I tell them to. Alignment for an intelligent system is a hard problem and at this stage is seems close to unsolvable.

My guess is that we'll just ignore it and make money along the way and every 2-3 months we'll have the equivalent to "Equifax gets hacked and millions of user records are stolen", etc. (this time with the LLM itself doing the hacking at someone's behest - accidental or not).

What does this even mean. There is strong evidence of LLMs doing in context learning.

Some of the linear RNN layers in recent models are provably doing SGD in hidden space during inference

By in context learning do you mean latent space?

https://transformer-circuits.pub/2021/framework/index.html

> What does this even mean. There is strong evidence of LLMs doing in context learning.

1. Is this learning persistent?

2. Do they verify these new lessons against core principles?

3. Do they and protect themselves/ignore requests if these new lessons contradict those core principles?

Humans do that from the time they're 3 years old (not that well, but they do do it).

Yes. Yes. Yes.

In my experience all those claims are false.

So the next step is to ask for evidence and ideally independent and peer reviewed research.

You know they can take notes, right?

And ICL dates all the way back to 2020, at least: https://arxiv.org/abs/2005.14165

That's training during training. They can't learn afterwards.

do you have a reference where that claim is demonstrated?