Version of AI tool 'too powerful for public' released to public

Mythos AKA Project Glasswing is indeed a very powerful platform. This is why all the biggest tech companies were given access to it in order to help ensure safeguards and guardrails and to help them discover vulnerabilities.

It is not a hoax or a marketing stunt. Firefox alone has found multiple significant problems with their latest build due to it.
There have been multiple discoveries because of it. Example, a 27-year-old vulnerability in OpenBSD and a 16-year-old flaw in FFmpeg that five million automated test runs had missed. None of these had been previously found by human researchers. Mythos found them on its own, without guidance. Linux root access from a basic user etc. It's insane. Look them up.

Also a company is in the business to make money, they would not give their product away for free to all the big tech bros
unless they were scared of some of the findings. They also wouldn't be banding together like this unless there was an actual threat.

IMHO, you cannot guardrail AI. It is very hard to do so if not impossible.

As mentioned in Irobot, you must ask the right questions. The right question will open the door. (question=prompt)

Where that ends up, we shall see. It will be a bumpy ride for sure.

Cyberwarfare is coming on a far larger scale, it is only a matter of time.

Some more good reading:

The impact of artificial intelligence on organisational cyber security: An outcome of a systematic literature review

Research on Security Issues of Artificial Intelligence

Good articles but want to sell you things of course:

You can guardrail AI, the problem is the guardrails need to exist in-line but not within the LLM itself. Since LLM and generative AI is non-deterministic any guardrails that exist as a system prompt is also non-deterministic.

When we shift the topic to agent guardrails, they must also exist at a higher privilege level than the agent itself so they can't be tampered with. There are plenty of documented cases were agents couldn't perform an action based on a guardrail the agent itself had access to. The agent simply modified the guardrails themselves to allow it to perform the action.

That's some world-class journalism. Here's what they actually said, because it was soooooo difficult to find.

"At present, no company—including Anthropic—has developed safeguards strong enough to prevent such models from being misused and potentially causing severe harm. That is why we have yet to release Mythos-class models to the public."

"Next, we will work with critical partners—including US and allied governments—to expand Project Glasswing to additional partners. And in the near future, once we’ve developed the far stronger safeguards we need, we look forward to making Mythos-class models available through a general release."

pseymour said:

journalism

Click to expand...

that hasn't existed since the 90's keep up

I find BBC News to be more reputable than the MSN articles on Microsoft. Go figure.

While it's great that these large companies are getting access to and fixing issues in important systems like operating systems and browsers, overall risk isn't reduced that much because there are so many opportunities to have security gaps in the development process of products. Supply chain attacks (3rd party dependencies as an example) are a hot topic and getting popped at a crazy pace. Fixing the OS or browser does nothing to protect against such attacks.

andrew129260 said:

that hasn't existed since the 90's keep up

Click to expand...

Didn't exist back then either, mate. I was there. :)

garlin said:

I find BBC News to be more reputable than the MSN articles on Microsoft. Go figure.

Click to expand...

So, what they said is not misleading or downright wrong, just because they're generally better than total crap?

neemobeer said:

Supply chain attacks (3rd party dependencies as an example) are a hot topic and getting popped at a crazy pace. Fixing the OS or browser does nothing to protect against such attacks.

Click to expand...

so true....

I am happy about the browser though as that is the frontline for most people that dont really install anything and just use the browser and whatever os they have.

I used to work with MSN, their target demographic is basically "soccer moms". They don't post any articles which are terribly informative or offensive. It's to fill pages to make it look like MSN is real destination. Everything's outsourced without any editorial review.

I still think it's just next level marketing.
If it was that good, the CIA or MI6 or the like would have grabbed it long ago.

The one thing we 'know' for certain, is that govt. agencies grab-up things like that.

Ghot said:

I still think it's just next level marketing.
If it was that good, the CIA or MI6 or the like would have grabbed it long ago.

The one thing we 'know' for certain, is that govt. agencies grab-up things like that.

Click to expand...

And you know for certain that they haven't?

On one side, the anouncement is clearly an advertising stunt, poining out the increased capabilities to attract attention, more users and ultimately money. They're about to release on the stock market and such news make stock price go up, that's what they really care about.

The preview group aren't random big companies, they're their commercial partners and main customersm who not only beta-tested it, but also used to find their own problems, that's "on the field use experience" to fix vulnerabilities. It's very doubtful for Anthopic (or any other AI company really) to worry about such consequences, they've already shown not a single bit of ethical consideration for prior models, so it's very difficult to accept that they've got morals now. The tool was already being misused, used for nefarious purposes by malicious actors and they know it very well. That hasn't stopped previous releases and is clearly not stopping this one either.

On the other hand, if it's so powerful, it can be clearly used for good purposes. So many projects could detect their problems and act on them before the bad guys do. Security is ultimately an arms race, so having a good car on the good side is a good thing, even if the bad guys get it too. And they won't just stop using AI entirely because Mythos is not released, they'll just use something else.

As always, ethical decisions are hard.