24 Comments
User's avatar
Joseph Meehan's avatar

As a software developer who generally thinks vibecoding still isn't worth it, the real test will be can it transform *Scientist* code. Scientists need to code a lot, are generally bad at it, but also care a *lot* about precision and accuracy in a way vibecoding works against. A model that overall improves the quality of scientist code can actually do a lot of good.

Geoff Anderson's avatar

I just snorted diet Coke. I studied Physics, and in that time (firmly in the 1980's and 1990s) I wrote code to do things I needed.

It was ugly. It was functional. It had no error checking. I would spend hours to groom the data so that I got a result, and I didn't care that if I had even one flier, it would break hard.

So, I agree. "Scientist" code is a good meter stick.

Ralph Haygood's avatar

I eagerly (?) await the article retractions due to egregious blunders in "vibe-coded" software. (I've made quite a lot of software for scientific purposes.)

Kaleberg's avatar

Having dealt with the dark art of numerical analysis while trying to maximize precision, I have my doubts about how well vibecoding will deal with it. I tend to hire consultants who can tell me when A implies B and when the exact same A doesn't imply the exact same B.

Tom M's avatar

My experience is pretty optimistic on this front. I’m an economist, and I my code is trash, laboriously produced. I write just enough just often enough to be bad at it. After playing with Claude code the past couple weeks, I find that it produces code that’s less ugly in less time.

Of course, I need to vet the code and the results it produces carefully because Claude will occasionally make weird and dumb mistakes, but I’d have to do that anyway because I also make dumb mistakes. It’s especially helpful for things like data cleaning and graphics production where the correctness of the results is pretty easy to assess.

It’s not a revolutionary advance, but it does make me better and faster at one of the more tedious parts of my job.

Kaleberg's avatar

I can think of some obvious apps to vibecode. How about one that takes a list of my friends, looks up what they've posted on Facebook for the last 30 days and gives me a reverse chronological list of their posts? How about one that does a Google search and eliminates the advertisements, the web sites that are full of fatuous, probably AI generated, textual padding and shows me the remaining hits? How about one that gives me the list of Netflix offerings in a particular genre showing the teaser panel and the full descriptive text for each? How about one for Amazon that lets me search for vendors but ignore results offering products that I didn't search for and places close textual matches, like those matching the name of a book and its author, at the top?

I'm sure there's a felony contempt of business model here to prevent this kind of app from getting written, but AI might make it easier to disenshittificate the internet.

Philip Koop's avatar

So I write software for a living. My employer is very keen on "AI" and I have over a dozen LLM-models integrated in my development environment; they can be run in both "chat" and "agentic" modes. We get regular training seminars on their use.

Recently, I've seen a sea-change in the seminars. "Vibe-coding" is now discouraged, and we are to aim for "spec-based coding". This involves analyzing a requirement, specifying a solution, and breaking that solution down into tasks to be executed. Then the agents can do the tasks, although we are admonished that we always have to check the work.

The thing is that although I started my career as a systems programmer (working on the operating system of a computer that supports actually useful applications), I have long since become an "application developer". Analyzing a requirement and figuring out how to solve it is maybe 90% of what I do. Once I know what needs to be done, coding it up isn't hard; that's the air that I breathe and the water that I swim in. We are also encouraged to use various special keywords (analogous to HTML tags) to help the models along. You see the point? "Agent-based coding" is becoming more like "coding". It looks more like the latest generation of high-level programming language than like a self-licking ice cream cone.

The promise (or threat) is that AI will soon be able to do that 90% bit of my job. So far that is empirically not true, and I am not clear of the theoretical basis on which these hopes rest. The Princeton AI researcher Tom Griffiths (I recommend his book "The Laws of Thought") notes that the thing about current LLMs is that they need a *lot* of training data, compared to a human. One example is that a human infant can learn a language in about 5 years but a LLM needs the equivalent of 5,000 to 50,000 years of training data to do the same. He is working on an idea: if you think of LLM training as a Bayesian process, you ought to be able to speed it up by picking good initial weights - the equivalent of an "informed prior". This makes sense for human language: maybe you could train LLMs on a couple dozen languages, then pick out some commonality in the trained model weights to start with on the next language. But I am doubtful that you could apply that idea in my field, because there isn't an obvious common element that you could isolate and use to "pre-train" your models.

Ralph Haygood's avatar

"... the demand for skilled coders is so great, and the market for coding jobs is so promising, that we are heading for a future where more and more people make their careers out of learning to code.": Even in 2019, that was a silly notion, reflecting a basic misunderstanding of software development (which I've been doing, off and on, for 45 years in a variety of contexts, both academic and commercial). Because this misunderstanding keeps cropping up like the proverbial bad penny, I hereby announce:

Haygood's Fundamental Theorem of Software Development: The most challenging and time-consuming aspect of software development is design and specification, meaning deciding exactly what the software should do, under every condition that may arise in practice. If done well, design and specification routinely take at least twice as long as coding.

Supposing that developing software for any serious purpose consists mostly of writing code is akin to supposing that composing poetry consists mostly of writing words.

For example, I could paste into this comment the 10,544 characters (plain text) specifying how self-serve password resetting (when a user has forgotten their password) works in a web application I'm currently developing. That process needs to be designed very carefully and specified very precisely, because it's a potential point of attack on the application, and it's something a sizable proportion of the users will need to use at some point.

To be sure, there's a great deal of poorly designed software in the world. (E.g., although most web applications with user accounts have self-serve password-resetting processes, many of those processes are clumsy, confusing, and/or insecure.) Almost every day, I'm forced to (try to) use poorly designed software. I believe this situation is largely due to members of the boss class who don't understand software development (partly because they've never done any themselves) and/or don't care that their software is lousy. Often, lousy software is foisted on us by big companies or public agencies that enjoy at least de facto monopoly positions. "Yeah, our software sucks, but what're you gonna do about it, huh? Take your business somewhere else? [Howls of laughter]"

Because those bosses also hate needing employees, particularly expensive ones like software developers, they're now enthusiastically embracing klarna koding*, and so I expect software to get even lousier.

"She didn’t want to vibecode the Clueless closet-organizer app. She just wishes she could download something like that.": Indeed. Most people don't want to develop software, not even with klarna koding, much like they don't want to run servers or even have to be aware there are servers**. It's comical how oblivious "techie" types often are to these "normie" preferences.

*Like Klarna, "vibe coding" is buy now, pay later, in that if you're using it much, technical debt is probably accumulating in your codebase.

**https://moxie.org/2022/01/07/web3-first-impressions.html

Ralph Haygood's avatar

By the way, I never imagined Linux would take over the world of personal computing, for reasons that were obvious even in the 90s. To begin with, most people don't want a "free tank"; they just want a car that's comfortable and convenient. In mass-market products, ease-of-use almost always trumps versatility. Moreover, there was and still is, I think, a culture gap between most people who get excited about Linux and the "normies" who buy most personal computers. Inevitably, the development of "Linux on the desktop" has mostly been done by and hence has mostly catered to the needs and tastes of the former, not the latter. (I personally am the former; I'm writing this comment in Firefox on a Debian VM under Qubes. However, I understand the latter; I'm a freelance software developer, and they tend to be my clients.)

Rachel Jacobs's avatar

The tank metaphor for Linux feels perfect because I don't really want a tank. They have power, but what on earth do I need with all that power!

And I think you are being far too kind to AI, at least the sort we have seen thus far. Professionally, I'm a writer and editor, and I am anything but worried about AI replacing me. It does a mediocre job at best. So I can believe AI can do casual vibecoding just fine, but I'm guessing they're trying to make a market for that _because_ they can't actually make in the big leagues yet.

Alex Remington's avatar

I think Clueless has transcended its era in an interesting way - a way that many movies from the '90s did not, including movies I adore just as much such as Romy and Michelle's High School Reunion, for example.

When my friend and I did a LearnedLeague MiniLeague on 1990s Romantic Comedies, he immediately suggested we make an entire day's worth of questions about Clueless, and I immediately agreed. It's just become a timeless, universal classic, like Groundhog Day and When Harry Met Sally.

Hey, the youngs aren't wrong! It's a great movie.

Kaleberg's avatar

Kudos to Jane Austen.

NickS (WA)'s avatar

I'm in essentially the same position -- I'm curious about some of the AI tools but haven't started using them yet. One of the pieces which best captured, for me, the process of thinking about was this by Jasmine Sun (who ended up having a positive vibe coding experience): https://jasmi.news/p/claude-code

---------------------- block quote ----------------

If you tell a friend they can now instantly create any app, they’ll probably say “Cool! Now I need to think of an idea.” Then they will forget about it, and never build a thing. The problem is not that your friend is horribly uncreative. It’s that most people’s problems are not software-shaped, and most won’t notice even when they are.

...

Well, I get it. I am embarrassingly nontechnical and scared of CSS, but spent every day last week talking to Claude Code more than my friends. It is an incredible technology that has made me more AGI-pilled than ever, while also being a net decrease on my work productivity. This is my attempt to reckon with both.

...

Eventually I came up with a first task: I needed to stitch together three PDFs for a grant application. The online discourse made Claude Code sound exceptionally easy—like it requires no technical skill, can one-shot complex apps, and never ships a bug. But for the truly uninitiated, I don’t think this is true.

Here’s what using Claude Code initially felt like: cooking with ingredients from a stranger’s fridge, the blankness of a page before you start writing, solo traveling in a country where you don’t speak the language. It’s hard to know what to build, hard to know how to start, and sometimes stuff doesn’t work the way you expect. As with all these analogues, you will eventually enter a flow state. But it took my fair share of false starts to get there, and I (the human) was very much in the loop.

...

A few days later, a friend sent me a voice memo instead of a text, and a collaborator asked me for feedback on a plan shared via YouTube. Unfortunately I am a psycho who refuses to listen instead of reading. So I had Gemini convert both files to text and sent off my replies.

Oh, I noticed. I do this over and over. Copying and pasting, uploading and downloading, turning audio and video into text for me to read. Maybe *this* problem is software-shaped. ...

Albrecht Zimmermann's avatar

Fascinating example in the context of the linux discussion since: "stitch together three PDFs" since on linux there's already an app for that. Call it on the command line, give it the right parameters and you have a merged .pdf, no coding (let alone Claude Code) required. :)

NickS (WA)'s avatar

To be fair to her, "stitch together three PDFs" is given as an example of a poorly chosen vibe-coding project. But, you're right that there is the potential for vibe-coding to push people towards large amounts of duplicated effort,

Kaleberg's avatar

Apple started doing this three or four years ago for voice messages on the iPhone. They don't always provide a complete transcription. Now and then they lapse into dot dot dot, but the big words, dates and numbers tend to come through. I saw this prototyped in the mid-1970s and thought it was a great idea. It only took 40 or 50 years, but it's coming.

Cheez Whiz's avatar

If you learned to operate a computer using a Graphical User Interface you were deliberately shielded from learning how a computer works. What you learned was how to manipulate the GUI to do things on a computer. For a long, long time any UNIX GUI was crude and complex compared to the more polished Mac and then Windows. And there was the application barrier. UNIX gave you tools, not apps, with none of the compatability handholding Apple and Microsoft offered by simply being much bigger markets. If you chose to write an application, you had to roll up your sleeves and learn UNIX, a text editor, a language and its libraries, and how to compile it all. Or just buy one for Windows or MacOS.

The promise of vibecoding leans into this model. I assume AI also compiles and dumps an executable on your computer, tablet, or phone. Or do vibecoders do that? I find that really hard to believe. If they view the source Claude generates, how much do they understand? Do they pick the language to write in or does Claude? What do they do when (not if) the app breaks or does something wrong? Who you gonna call? The promise is very seductive, just as the promise that you'll never see a command line or worry about "allocating" memory for your app ever again was. But writing code is a horse of a different color. There are way too many things outside the control of Claude that can break that promise. The never-ending war of obsolesence with hardware and OS means Claude gives you an app compiled to run on that device on that version of OS, and maybe elsewhere. I guess the vision is you save your prompt and ask Claude for a new app when the old one stops working.

Albrecht Zimmermann's avatar

This! Reading the text and the replies made me understand that vibe coding is an offer for the people whose main computing device is a smartphone.

Parrish Ticer's avatar

Thanks for writing such an on point; at least for me, article. It helped me frame my thinking about the deluge of AI information that is washing over my feeds. I don’t want to code or vibe code either. I feel your pain on being stuck in Apple’s walled garden too. Keep up the good work, it’s appreciated. Cheers!

Rob Nelson's avatar

"....didn’t follow the expected path" feels like line that applies to just about every revolution, technological or otherwise. What's weird is how little that fact registers among those who enjoy prognostication.

And Clueless! My thirteen year old daughter and I are watching it tonight. At her suggestion. Baffling indeed, though I am thrilled to see it again.

Kevin Munger's avatar

https://journals.sagepub.com/doi/10.1177/2056305119849492

I think you were right the first time!

Andrew's avatar

As a professional engineer, I wanted to try the vibe coding thing a few days ago. I had this idea that the “sales” at my local Big Box Store weren’t really sales and that it’s just a trick when prices don’t actually move.

So I used an agent to search through historical prices and create a report for me. I now have a github repository that runs this weekly. I’ll let you know if it finds anything.

Simple app. Simple idea. Maybe useful. Took me about an hour. Doing it without an agent would have been 3 hours maybe (well, I would t have done it at all).

Albrecht Zimmermann's avatar

I'm a bit surprised by your equating linux enthusiasts with people who know how to code. Having studied CS, I know the latter, and linux makes my job easier in a number of ways, which is why I use it for work.

But a big appeal for me is that I can parametrize and fix it relatively easily by navigating to particular files and writing a few lines of text - notably not code - something that Windows has made increasingly difficult under the (probably not incorrect assumption) that users who do not know what they do can break their system in unrecoverable ways.

But that's exactly what I did with my early DOS and Windows machines, so that's what prepared me for linux to a certain degree, not my being able to write code.

As an example, I recently tried to update the default version of Python on my linux machine, only to find out that Python is by now such an integral part of the OS that one cannot do this without breaking everything. I knew how to roll back my changes, though, so everything ended up fine (after an hour of cursing and still with an outdated Python version). I didn't write a single line of code for it but only changed system variable settings and symbolic links.

So, I agree with "Linux demanded a relationship to software that most people didn’t want to have" but not in the sense of "people don't wanna learn how to code" but in the sense "people don't wanna be able to finetune their systems in such detail that they learn how everything works and where everything is".

Three side-notes: 1) early linux, where one needed to compile and configure everything oneself was a bridge too far for me.

2) this is why smartphones frustrate me even more than Windows machines: it's extremely difficult to go under the hood and even the kind of modifications one can do to an MS computer are almost impossible.

3) MacOS arguably splits the difference (or it did the last time I used a Mac): the underlying linux is almost as easily modifiable as other ones and the graphical interface and certain software is clearly advanced over anything linux has to offer. But Apple's business practices are awful, so no Mac for me.

Kaleberg's avatar

I use open source software now and then, so I often wind up failing to download, rebuild or compile. Libraries come and go and configurations suddenly break for no obvious reason. Apps become unavailable for months and sometimes years before someone figures out how to get them to run on a late model Mac. There are several open source utilities that help, like brew and port. Sometimes I can get each piece of something to work by using a combination of such utilities and perhaps a git download or two, but then it's almost impossible to configure them to work together.

If I were still a professional software person, this would still be a problem but perhaps one worth dealing with.