Rendered at 11:47:23 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
torben-friis 14 hours ago [-]
Here's my constant question:
Everyone's going so fast that they keep hitting walls. Review, CI, product asking for things, whatever.
Why have we not seen an improvements in products?
While every post and thread feels like a 90's wall street office, the new android and iphone ship with fewer features than usual. No indie guys come up with a linux-sized alternative OS. Switch 2 remains unhacked. Windows takes 3 seconds to show the right click menu.
Is everyone just running full speed in circles or something?
TrueDuality 14 hours ago [-]
A good chunk of what my company has been doing with AI falls into either burning down our known tech-debt and "easy wins" that no one ever had the bandwidth to approach... And improving / automating our processes. The former is having a direct and meaningful impact on the quality and availability of our services.
Our QA, formerly a fairly frequent blocker of all our releases, are doing more in-depth reviews and catching issues earlier in our release process. They have become unblocked to the point they are actively chasing down work that starts to slip.
We have cleaned up and tuned both our security alerts and operations logs and improved our tenant isolation in our service in a way that makes customer and formal audits SIGNIFICANTLY easier.
We're setting ourselves up for faster human development of the hard-things. Our development environment and infrastructure are faster, cleaner, more auditable processes, and cheaper overall to operate.
These fixes mostly don't show up in our product change logs, and definitely don't fall into "new features". It would largely be invisible to the outside world, but our costs are going down (though to be fair, not offsetting the spend on AI to date), internal productivity has improved, operational incidents are down, and customer satisfaction is up.
jurgenburgen 3 hours ago [-]
Cost is stable (or rising?), user-facing delivery is sameish, (some) engineers seem happier because they are allowed to gold-plate and prepare for future “human development of the hard-things”.
Is that a fair summary?
JimDabell 1 hours ago [-]
Them:
> a direct and meaningful impact on the quality and availability of our services.
You:
> user-facing delivery is sameish
I don’t see how you can think that is a fair summary or a good-faith argument.
theshrike79 2 hours ago [-]
Cost is stable, what users see is "the same".
Engineers are happier because they can spend more time on that extra round of polish that was just filed into low-priority "fix when time" Linear tickets.
QA is happier because the stupid bulk operations are handled with AI and they can focus on the hard to find stuff that tickles the QA mind in a special way =)
Same stuff, but with better (in-house) tooling and (most) people are happier. The Code Artesans aren't, but it's mostly their problem and they'll find companies that only do fully manual programming for specific niches.
baq 3 hours ago [-]
> Is that a fair summary?
Not op but no.
lazyasciiart 11 hours ago [-]
How is dev satisfaction?
consumer451 10 hours ago [-]
A luxury of the past, for >90% of devs?
consumer451 5 hours ago [-]
I want to add to this, because it's certainly not all doom and gloom for me, yet.
If you are product-driven, life is pretty good, and maybe has never been better? (For the moment)
netdevphoenix 3 hours ago [-]
>If you are product-driven, life is pretty good, and maybe has never been better
It has certainly been better for junior devs, mid devs, anyone looking for zirp era salary growth and supply demand ratios. For a senior code-adverse coaster looking forward to see his static salary get eaten by inflation, life has indeed never been better.
consumer451 3 hours ago [-]
Yeah, at Corp, that makes sense. I was thinking more along the lines of scrappy startup silliness. "I am one person and I deal with any and all code to get my product built."
From that POV, has it ever been better, assuming you can rise to the top of an ever more crowded market?
netdevphoenix 3 hours ago [-]
> "I am one person and I deal with code to get my product my built."
From that POV, has it ever been better, assuming you can rise to the top of an ever more crowded market?
Building a product is easier now, same way making a photograph is easier now. Making a reliable and high income living out of it though, it looks increasingly uncertain.
In a world with abundant music and photography, just saying that you made your music using Reason/ProTools or you made your photograph with a Leica isn't enough. I believe the same will happen with software. Sales, product design, customer service and operational aspects like uptime will matter more than the software itself. Ironically, this is the kind of stuff the HN type highly dislikes.
consumer451 2 hours ago [-]
> I believe the same will happen with software. Sales, product design, customer service and operational aspects like uptime will matter more than the software itself. Ironically, this is the kind of stuff the HN type highly dislikes.
Agreed. It's not even a new thing. They tried to teach me/us long ago, starting by putting it into words we might more easily understand, such as "Customer Development" lol.
When I realized that it was the sales guys that will have the power in the mid-term new world, I got genuinely upset. However, maybe they always had the power and I just didn't want to accept reality.
Now, I believe the most desired hire in the near future will be the PMM.
netdevphoenix 1 hours ago [-]
What is PMM?
zerr 3 hours ago [-]
If you are product-driven why not go into Product Ownership roles? It certainly will be more satisfying compared to prompt "engineering" clerk "career".
consumer451 3 hours ago [-]
Well, my life is currently both, as I am on the smallest possible tech and product team. To me, extrapolating on having done this since Sonnet 3.5 to now, there will be no tech aspect to the job soon enough. There are many hundreds of billions invested in making that a reality, and they are succeeding.
Until ~9 months ago, I spent most of my time being a "prompt engineer." Today I join a meeting, ten minutes later I receive the transcript. Then, I run a custom skill in my project, and I get Jira epics and stories to triage that are nearly perfect. Then, I run the second skill orchestration skill... some babysitting... and ~85% of the time that is all I need to do. Docs, code, unit and e2e, great UX... all there after every meeting, and basically two commands on my part.
I see two to three to maybe five years before anything I have to offer, in any capacity, is completely cut out of the picture.
I honestly don't understand how everyone is not on this same page. The labs are going to eat it all. The only reason I see for them to talk about "pausing," is because they finally realized that they are going to collapse the entire service economy around themselves at this rate: aka, the USA.
baq 18 minutes ago [-]
Yeah it’s going to be very rough for any knowledge worker unless the governments decide it isn’t ok for the general public to be allowed to use the tech.
It’s actually rough today with astra and fable, it’s just not been diffused enough. Tech workers like us see the writing on the wall, but a lot of others are blissfully ignorant.
zerr 2 hours ago [-]
Not sure how long the current subsidized pricing would last. Also, the vast majority of software shops can't afford those subsidized prices anyway.
baq 16 minutes ago [-]
It’s 10 devs with a small ai budget each vs 1 ex-dev now PM with a large ai budget - the choice is quite obvious for any decision maker who counts time and money
arathis 6 hours ago [-]
If we cared about the satisfaction of the worker, we’d never have fast food.
nradov 9 hours ago [-]
Which devs are you asking about, the humans or the AI agents?
skulk 11 hours ago [-]
doesn't really matter now, does it? Someone's gonna do the job, whatever it looks like.
Tempest1981 8 hours ago [-]
Start a few dozen agents, and take a long lunch, right? No?
notyourwork 7 hours ago [-]
No better:
UltraSane 7 hours ago [-]
I assume they are very satisfied with their paycheck.
verpeteren 5 hours ago [-]
This is the way!
lukevp 11 hours ago [-]
PS5 emulation has gone from barely working to running Dark Souls at 10+ FPS with virtually no graphical glitches… in 6 weeks. If you’re not familiar with normal emulator development time, this is… quite extraordinary. There have been insane progress on decompilations and many other things in the emulator space.
Everyone’s trying to figure out how to convert this speed to product features at scale, but enterprises are like container ships. Lots of might but slow to turn. The littler companies can actually take advantage of this and produce higher quality products at much faster speed. I think you’re expecting too much in the short term and too little in the long term. AI-native companies are gonna eat everyone’s lunch, once they figure out how to actually do it reliably.
decompgoldenage 9 hours ago [-]
Also, in 12 months, we went from seeing game dissassembly and decompilations projects be slow and annoying, to suddenly having enough mapower where people are taking the liberty to CHOOSE which decompilations to support. DK64's release was 80% anti-AI marketing, not because they were strictly anti-AI, but because they were telling the OTHER decompilation to fuck off and learn some goddamned standards. Completely unthinkable before.
There was a video by a layperson 2 years ago where they wanted to know the mechanics of some elusive pokemon pinball spawns where there was a whole bunch of missinformation about it online. The people behind that project had the correct addresses but they were completely unlabeled and the layman had to essentially figure out and do some of the work themselves to sort it out and figure it out. Nowadays that would never happen. An LLM will just do it for you within the day.
Eufrat 3 hours ago [-]
The fact that this is an account with a username specifically focused on this specific concern that was created 5 hours ago is real weird/suspicious.
Also, are these even actual decompiles? Sure, playing Wheel of Fortune with a LLM model that at some undetermined point spits out what seems to be a working example is sorta useful, but also…how exactly do you debug this nonsense? Don’t tell me we’ll just have the LLM thrash around a bunch again. What I’m asking is does anyone understand what is going on here other than “lol, it works, shut up”?
dotancohen 3 hours ago [-]
You must be new here. Lurkers posting under an apt username then disappearing is a feature, not a bug. He doesn't need to argue with anyone online - he said his piece and we're free to take it or leave it.
Usually, green username are a sign of "leave it", I'll agree. But this one seems very informative.
charcircuit 3 hours ago [-]
LLMs work great at decompilation, I'm not sure what you are trying to suggest. Outside of game compilation disassembly is useful for debugging various crashes in 3p code where you don't have the source. And yes LLMs do the debugging here.
Eufrat 3 hours ago [-]
I’m suggesting that having a model wander around stochastically is not necessarily a great way to gain any insight or understanding. LLMs do not think, they cannot really debug anything. Attaching intelligence or intent to them is a non sequitur, it’s not happening. A cloud that looks like a toaster does not imply anything beyond the human obsession with pattern matching, why do we keep doing this with LLMs? They can randomly wander around and sometimes produce useful output.
I don’t doubt they help by randomly permuting around in a way that people cannot do at scale, but I question the economics of this approach and I think it needs to be used judiciously. Sometimes you just have to do that especially like you said, when you do not have the source or the code is intentionally obfuscated, this can be useful, but a lot of people seem like they’re just vibe coding crap and blowing money on tokens until they get a useful result and I have no idea what the value of that is.
3 hours ago [-]
thejazzman 9 hours ago [-]
To further this point, I’ve picked up the torch and got Switch2 controllers working in Dolphin and Cemu
Wind Waker in 4K w/official Bluetooth GameCube controller
(Nintendo went out of their way to invent a new protocol so that it didn’t “just work” like OG switch controllers)
jetbalsa 5 hours ago [-]
I did the same thing with a old multiplayer game called Wulfram, we got it online and a mostly working server (most of the game logic was server side with no surviving code or binary) with the help of AI, Mostly opus and astra. https://wulfram3.com is our efforts in this.
sublinear 6 hours ago [-]
> AI-native companies are gonna eat everyone’s lunch, once they figure out how to actually do it reliably.
Couldn't you say this about any company? What isn't just a matter of "figuring it out"?
Why can't they use AI to crack this nut? :D
> If you’re not familiar with normal emulator development time...
Emulators have historically been a shitshow until someone figures out missing pieces here and there. By their nature, that tide lifts all ships. It has very little to do with using AI. I'd argue the real reason emulators aren't what they used to be is the hardware is more complex now and the scene is no longer attracting the most talented devs. Video games used to be on the cutting edge and carried a lot more cultural weight. It's pretty underwhelming to get cred for working on lame x86 hardware that plays yet another franchise reboot.
iwontberude 6 hours ago [-]
[dead]
apf6 14 hours ago [-]
Last year people were asking "if AI is so great then where are the new apps?". Then data for 2026 came out and now the IOS app store has a 84% percent year-over-year increase in new app submissions.
For that other stuff you mentioned like the right click menu. Those huge corporate projects suffer more from layers of institutional dysfunction and will be very very slow to show any improvement. Their dysfunction can't be solved with just faster coding.
Using AI to build more features is easier than using AI to improve existing projects. People will gradually figure out how to do latter too, it'll just take longer.
torben-friis 14 hours ago [-]
That's just volume though. I also know, and have no trouble believing, that github is going down partly due to the weight of all the vibecoded pushes. But that does not necessarily translate to people's needs and wants being covered, for all we know iOS just has a plague of unused POCs.
Do you have actual productive examples? As in, products with a real userbase that couldn't exist or be scaled pre-AI? Genuinely asking, I might have missed some large hits. The closest I can remember was bun rewrite kerfuffle, which seemed more a marketing action than anything.
apf6 14 hours ago [-]
> That's just volume though
If we sorted through all those new apps and ignored all the crap, I'm pretty sure we would find an overall increase in actually useful apps. I just shipped a new app myself, and I think it's useful, and I wouldn't have finished mine without AI assistance.
> products with a real userbase that couldn't exist or be scaled pre-AI?
Coding hasn't been the bottleneck for product creation for a while. So I don't think there are going to be many examples that fit that exact criteria. Any app idea was 'possible' before, it just took more developer hours to do it. So now teams are doing more things that were weren't worth the effort before, but are a lot more feasible with AI assistance.
dieselgate 13 hours ago [-]
> I'm pretty sure we would find an overall increase in actually useful apps
I think you're probably right but at what point does it become diminishing returns? I have observed the app market to be overly saturated for years and rarely download something other than a mobile-banking update.
brabel 5 hours ago [-]
So let’s stop making new apps then?? What is your point? Looks like you’re just complaining for the sake of it.
huurtehoog 10 hours ago [-]
Has economic growth accelerated? TFP, labor productivity?
Where is the net benefit people keep talking about when they claim productivity gains are obvious because "more code faster, can't you see?"
brabel 5 hours ago [-]
It’s interesting that people claimed the exact same thing when computers were being introduced to the work place int the 80s and 90s. Lots of papers showing how productivity didn’t go up at all and using a paper and pen seems to be just as efficient as using a PC.
Even in the 19th century, when electricity became widely available, there was no productivity gain for 30 years at least. This is a well understood phenomenon.
Google for “the Solow Paradox”.
huurtehoog 1 hours ago [-]
Ah you must be referring to this off the cuff comment by Robert Solow in a book review: "You can see the computer age everywhere but in the productivity statistics" (1987)
It's an interesting question and very much not resolved. It indeed led to a flurry of studies in the 1990s, and more recently to several updates and meta-analyses.
The problem of "does computer technology investment causes increased productivity" is an interesting issue in economics and statistics. It is far from clear that the (immense) investment in computers over the past several decades has caused a corresponding excess growth in productivity.
Some of the literature published after 2015 I have found on this topic:
"Information technology (IT) productivity paradox in the 21st century"
> Thus we are still unable to confirm or reject the existence of an IT
productivity paradox
"Benchmarking the IT productivity paradox: Recent evidence from the manufacturing sector"
(This one was published in 2006 but I find it relevant because it does a very well scoped analysis in manufacturing firms thus addressing the oft-mentioned argument that computer technology may leverage task productivity in a way that is hard to measure in aggregate)
> However, many scholars from both sides of the IT paradox debate agree that difficulty still exists in specifying how to assess the IT contribution, and the availability of reliable data sets
> Regardless of the final decision to differentiate or conform, our results make a compelling argument that more spending does not necessarily mean better IT productivity.
"Lessons from three decades of IT productivity research: towards a better understanding of IT‑induced productivity effects"
This is one of the most inclined to disagree with the existence of the paradox, and still very cautious in the language used for writing the conclusion, e.g.:
> But to not at least consider the ongoing technological change as an important determinant of the deceleration in productivity growth seems ill-advised."
This I'm quoting from the submitted manuscript. I haven't gotten around to reading the published version yet, but:
> Since the size of the effect helps make the right decision in business-related investments, our result of ICT elasticity being very close to zero with values, about 0.3% for productivity and no effect on profitability, supports the argument that there are better forms of investment to be made
This is just ~2 - 4 years after ChatGPT launched, and despite very shallow adoption (only ~6% of all work hours.) As a sibling comment indicates, it took almost 2 decades for the Computer Revolution to be visible in national level statistics.
Also note this study was originally published in 2024, then revised in 2025, but this preliminary evidence has been around for a while, if people wanted to find it. It's even been posted to HN a couple of times, somehow it just doesn't get the attention you'd think it should get, even if it was just to poke holes in the conclusions.
huurtehoog 1 hours ago [-]
> When we feed these estimates into a standard aggregate production model, this suggests that generative AI may have increased labor productivity by up to 1.3% since the introduction of ChatGPT. This is consistent with recent estimates of aggregate labor productivity in the U.S. nonfarm business sector. For example, productivity increased at an average rate of 1.43% per year from 2015-2019, before the COVID-19 pandemic. By contrast, from the fourth quarter of 2022 through the second quarter of 2025, aggregate labor productivity increased by 2.16% on an annualized basis. Relative to its prepandemic trend, this corresponds to excess cumulative productivity growth of 1.89 percentage points since ChatGPT was publicly released
These long term data suggest that this "excess" remains bellow historical productivity growth
The analysis bellow is a better support to your position on the side of "computer technology investment caused a delayed excess growth in productivity"
To be clear: I do not take a position. I think this is an open question, a very important one, and I am not fully convinced that the exponential growth in the investment on computer technology over the past 50 years has led to a corresponding gain in productivity, nor that it is entirely a drag and a mechanism for increasing firm size and driving asymmetric profitability concentrated in ever fewer firms as the increased concentration in the capitalization of American stock market index composition would indicate.
That said, the strongest case I have seen for the position that we are beginning to see these delayed gains is the letter I linked above, and it still takes care to conclude:
> As more data become available, it will be important to continue to monitor whether current patterns represent the early stages of a new era of booming productivity or merely a temporary uptick in an otherwise slow-growth environment.
8note 10 hours ago [-]
more code faster is competing against some of the largest tax hikes of the last ~70 years, economic uncertainty on what the taxes tomorrow will look like, combined with an energy crisis.
huurtehoog 10 hours ago [-]
Are you claiming productivity should be growing much more slowly then it is over the past few months or years? Can you show me any analysis that supports that claim?
Because I have seen plenty of analysis published in the past 30 years that puzzle over the productivity growth stagnation in the face of an exponential growth in capital expenditure in computer technology.
So by all means share with me some of the groundbreaking results showing that we can finally see more productivity growth than would be otherwise expected by the conjecture.
kylecazar 4 hours ago [-]
There's a difference between task-level productivity, which AI has been shown to increase, and effects on the macro. You're talking about the latter while most people here are engineers considering the former.
It's still unclear if task-level productivity gains bubble up, but it's also still early and I'm not sure we should expect to see immediate results there.
huurtehoog 1 hours ago [-]
So it enables doing more things but not necessarily doing more valuable things?
Yes that is in essence the crux of the problem
azan_ 4 hours ago [-]
And can you show analysis supporting your claim? Sounds like some kind of piketty bullshit.
whattheheckheck 9 hours ago [-]
[flagged]
abletonlive 9 hours ago [-]
Once the previous goalpost gets demolished on HN, there's always someone new to put a new one. We aren't quite there yet but eventually the goalpost moves to something that is just impossible to even measure. That'll be the end game surely
huurtehoog 9 hours ago [-]
Was the goal of any investment ever not 'increase productivity'?
Eufrat 3 hours ago [-]
> If we sorted through all those new apps and ignored all the crap, I'm pretty sure we would find an overall increase in actually useful apps. I just shipped a new app myself, and I think it's useful, and I wouldn't have finished mine without AI assistance.
I don’t think this is happening, at least not on any scale that makes any of this actually useful. One off success stories do not make a revolution.
It absolutely has generated a massive amount of slop, though. The problem is that you still need intent and while I think LLMs can help you free up and push through annoying boilerplate or tedious spots where known solutions exist, you still need to design these things and that’s something I’ve not seen an LLM be too helpful with.
foobarqux 9 hours ago [-]
> If we sorted through all those new apps and ignored all the crap, I'm pretty sure we would find an overall increase in actually useful apps. I just shipped a new app myself, and I think it's useful, and I wouldn't have finished mine without AI assistance.
App store revenue actually decreased! That suggests that the apps are largely not useful, unless you make the implausible assumption that nearly none of them are charging money.
I mean in a market where production cost goes down & competition increases while demand stays the same... That's exactly what I would expect.
setr 6 hours ago [-]
If they were actually useful though, demand shouldn’t stay the same — more niches should be getting filled. If they’re just clones of each other, then yes
pinkgolem 4 hours ago [-]
i mean, i had multiple gaps filled in the last few month for things i was longing for.. but all for free.
intended 5 hours ago [-]
> If we sorted through
If.
The labour of finding better goods has become harder with more content showing up.
shimman 13 hours ago [-]
What you're saying wouldn't be hard to prove, you're acting as if we aren't aware of usage statistics. We can even tell if MAU (monthly active users) are increasing or not for particular apps. So we are aware of them and what do they actually say?
ndriscoll 12 hours ago [-]
In the gaming space, retro console PC ports have absolutely exploded in the last year. People are modding old games with 4k textures, ray tracing, DLSS 5, widescreen support, uncapped framerates, dual-screen (e.g. Ayn Thor), etc. PS5 emulation has gone from almost nothing to AAA titles in-game in like 2 months, and apparently now runs on xbox (lol).
The other day someone posted a reverse engineered GPU driver for their Mac[0].
Anyway, what makes you think those pushes aren't individuals meeting their own needs? I expect it will take a while for this to really sink in, but the future is people asking the computer to build exactly the app they want.
Also, English localizations of even super-obscure foreign retro games.
Most of those might be not as good as fan translations from dedicated people with good knowledge of English and the original language (especially for very context-dependent languages like Japanese) but they are definitely good enough to follow the story and play through the game.
8note 10 hours ago [-]
whats the large hits that have happened in the past couple years?
I think ai has raised the floor quite heavily on what it takes to be a large hit.
all those vibecoded pushes are people building bits and bobs and variations on each other, instead of buying it or using a common service. If it could be done by ai, it will be written off as unimportant.
the userbase of the future is 1, maybe 10
lazyasciiart 11 hours ago [-]
Has anyone said that AI is fixing user research, actual idea generation, marketing and audience reach? Or are you assuming that better code should be making all of that irrelevant?
huurtehoog 10 hours ago [-]
It either delivers better outcomes or fails to do so for not being as general and useful as touted.
The original sales pitch from the major player in this technology hype cycle is "cure cancer, fix global warming, take over the economy". Not "more LOC".
binary132 8 hours ago [-]
AGI any second now
nharziro 10 hours ago [-]
Not OP, but I ported a legacy .NET WebForms application to Blazor. The actual code migration was completed over roughly a 48-hour period, followed by fairly extensive testing.
We were fortunate to already have a strong end-to-end test suite written in Python, so we could run the new application against the same tests and verify that the existing functionality was preserved. QA found around 20 bugs, which we fixed pretty quickly before launching.
After the migration, we also moved the application's authentication from Shibboleth to Entra OAuth and deployed it to our OpenShift infrastructure. We couldn't do that with the old WebForms application because our cluster doesn't have Windows worker nodes, so the legacy app had been stuck running on VMs. Getting it onto OpenShift gave us another operational and cost-saving benefit beyond simply modernizing the codebase.
I did this back in January, when the models finally became capable enough for this kind of work. I believe I used GPT-5.2 through Codex. Successfully completing this project is what finally got me fully on the AI bandwagon. I had used AI before, but mostly for smaller tasks like writing code, refactoring individual methods, or making isolated changes.
The application is now modernized, more stable, faster, and more functional than it was before.
I think the project worked as well as it did for two important reasons. First, I had deep domain expertise in both the application and its surrounding systems because I was the original developer. Second, our QA team's test suite was comprehensive enough to validate the functionality that actually mattered. AI dramatically accelerated the work, but there still needed to be someone who understood what the application was supposed to do and a reliable way to verify that the new implementation behaved correctly.
None of this means we couldn't have done the migration without AI. We absolutely could have. The difference is that AI made it possible to do it at practically zero cost in terms of engineering time and money compared with the alternatives.
We had wanted to move away from WebForms for years, but with the size of our team and the constant stream of new feature requests, there was never a realistic opportunity to stop development for months and focus on a rewrite. I estimate that doing the migration myself without AI would have taken at least six months to do properly. The other option would have been hiring a contractor, which I would estimate at $80,000 or more over six to nine months.
Based on my token usage, those roughly 48 hours of working with Codex to port the application cost about $150. Those were January prices, so I don't know what the equivalent cost would be today.
For me, that was the project that changed AI from something useful for assisting with individual coding tasks into something I saw as capable of fundamentally changing the economics of software engineering work.
nradov 9 hours ago [-]
People don't know what they want.
pc86 9 hours ago [-]
OK but you talked about people going fast. 84% YOY increase in volume shows people are going a lot faster.
You can move the goalposts to "yeah but it's not any good" but 90% of everything is crap anyway, and that was true long before AI.
deaux 3 hours ago [-]
I'm running several. They're not "large hits" but they're providing value to thousands of people. They couldn't exist pre-AI because core functionality depends on LLMs.
What you're missing is that slop has increased 20x but quality new releases have also increased 5x. Unless there's something in your niche though you'll mostly see the 80% slop so I get why you feel that way
onetimeusename 10 hours ago [-]
I have this theory that the economics of AI development would make it so it's much more viable to make in-house, custom, apps rather than paying subscriptions for apps. I have even seen this play out on a personal level. A friend of mine who has no dev experience vibe-coded something custom that evite does. So it's possible measuring the number of apps miscounts ones that are not public or shared. I'm not negating anything you said, just adding something about metrics on apps.
bdashdash 6 hours ago [-]
At the 60 person smb (travel industry) I work we've taken web development in house for the first time, created new user experiences that sat on our roadmap for years, created custom AI solutions on top of our Fresh desk tooling, and (predictably) created a number of internal productivity tools.
The actual bottom line result is very fuzzy. Cost are shifting, by spending less on agencies but in return opening up new positions or shuffling people internally.
We don't see a clear increase in revenue, nor can we accurately tie retaining revenue to our AI initiatives.
Workplace satisfaction scores remain similar, with a few outliers (e.g. some people's work got incredibly exciting, while a few others are terrified by all the change).
Time will tell if this all leads to anything meaningful for the business.
geraneum 8 hours ago [-]
Maybe one day but right now in order to get any meaningfully usable app with AIs you need to pay way more in AI sub than monthly sub of a bunch of apps. It doesn’t make sense right now to build especially your own if you’re not a company. Also it’s not clear how long we’ll have this heavily subsidized subscriptions.
matt_heimer 11 hours ago [-]
The OS one is funny to me, it's been hard to keep osdev.org online due to the insane amount AI bot traffic.
edgyquant 9 hours ago [-]
Thank you for your site dude! I had so much fun and learned so much building my own toy OS over a decade ago.
binary132 8 hours ago [-]
osdev is golden, truly. hope you survive the slopocalypse without too much difficulty
ChickeNES 9 hours ago [-]
> For the question where are the alternative OSes? Here is one that I've seen. There's probably more
And that's just the people operating in public instead of private :P
sarchertech 10 hours ago [-]
Last time I looked the other app stores didn’t show anywhere near that much of a bump (that was a several months ago though).
And a few months ago I went through a random sample of new apps on the App Store and the vast majority were just wrappers for AI APIs. So it was more of a new gold rush situation than a productivity bump.
erfgh 3 hours ago [-]
App submissions are up but what about revenue?
SrslyJosh 12 hours ago [-]
> Last year people were asking "if AI is so great then where are the new apps?". Then data for 2026 came out and now the IOS app store has a 84% percent year-over-year increase in new app submissions.
That's exactly the opposite of what I want to see. As an app store user, I want to see fewer, higher quality apps, not more shovelware or slop.
CuriouslyC 11 hours ago [-]
> higher quality apps, not more shovelware or slop.
Your prejudice is showing.
jayd16 13 hours ago [-]
Any notable breakthroughs in the top 100 apps on the app store that could attribute their success to AI?
globular-toast 6 hours ago [-]
We know it's easy to produce working code now, what's missing is useful software that actually solves a problem. What problem does that toy OS solve? The main point of toy OSes has been education, but you learn nothing when using an LLM. So all of this is just pointless energy consumption, it's not solving any real problems that people have.
coldtea 12 hours ago [-]
>Last year people were asking "if AI is so great then where are the new apps?". Then data for 2026 came out and now the IOS app store has a 84% percent year-over-year increase in new app submissions.
People mean where are the good new apps AI made possible. Of course it has a 84% year-over-year slop app submission.
Any real "killer apps" though?
vidarh 5 hours ago [-]
If the cost to produce goes down, unless the demand is elastic, you'd expect fewer big hits in favour of a more fragmented, more competitive space all earning less.
ehnto 10 hours ago [-]
Probably for the same reason that SV companies hiring thousands of developers struggled to improve their product much past the original product, that was built by a handful of people.
Scale in headcount was a tactic to get investment, then you had to find stuff for everyone to work on. Suddenly people have the time to engineer so hard that we get runtime JSON defined CSS rendering engines to produce the same buttons we've had since 1995, instead of just writing a stylesheet and html.
Code output velocity from AI threatens to be useful, except it's also prone to over-engineering and burning tokens on the unnecessary. It's learned from the best after all. My suspicion is there is a lot getting done, but it's just not that impactful to flagship products.
As others have noted, there is a lot of new work going into passion projects that would have never happened otherwise, and that is cool. But I wouldn't hold my breath for SV tech to become super pragmatic and effective.
dmboyd 9 hours ago [-]
With pay-per-token, there’s also an incentive for over-engineered but functionally harmless architectures. A json based css engine is probably something that can be test-cased really well in the training set.
moomoo11 9 hours ago [-]
those thousands of developers aren't working on the FB feed or whatever...
they're working on the ad tech business and other business-related systems involved including internal tools.
and all the other platform engineering shit under the hood that makes much of the web scaleable... and lots of it is open source and contributed to by various engineers from these companies.
end of the day, these are businesses. they're not charities or whatever casual shit.
nobody is stopping you from building a competing product that's lean or whatever.
why don't you do something like that? i'm sure you're a genius.
ehnto 9 hours ago [-]
No seriously, companies hired developers just to starve the competition of talent, raise more investment etc. It was a genuine thing, that left a lot of developers doing pretty meaningless work. A similar thing happens for managers in big orgs, who want bigger headcounts to inflate their power in the org. You get thumb twiddling and initiatives to justify the headcount.
It's probably changing now as VC cash floods into AI instead of web app startups, but I am not talking out of my ass, it's a well known phenomenon.
The industry does not incentivise or reward lean software, the software input for a lot of "software" companies doesn't require it, that's fine. Exactly as you said they're businesses, they do what makes money and go where incentives take them. That cuts both ways, they are not disincentived to have a lot of wasted dev hours, at least not historically.
antupis 6 hours ago [-]
I think it's more about money and scaling, bus factor is pretty big if run very lean organization eg whatsapp 2014 and that point all those developers are kinda your cofounders and probably start asking much bigger piece of pie. With small teams you kinda trade scaling and availability to velocity. It's much easier to ship but running oncall 24/7 with small team is just nightmare.
saithound 12 hours ago [-]
> Why have we not seen an improvements in products? [..] Is everyone just running full speed in circles or something?
The simplest explanation is that they don't give a flying flamingo about what you or I consider "improvements to products".
This report is an example.
There are several changes that modify CI behaviour, where the article gives no corresponding quality measurement.
They replaced type aware custom lint rules with AST-only static analysis. They don't say anything about what those new rules detect, didn't do old-vs-new rule comparison. They switched the TypeScript check from tsc to tsgo. Again, they are very proud of the performance improvement, but don't seem to care about diagnostic equivalence. The list goes on. They don't even report pass/fail agreement between the old and the new CI. They have 4x more tests, but no idea whether this big test suite works any better than the smaller old one, or even whether it works at all.
plant-ian 11 hours ago [-]
Are they really adding 2,000 tests a week to their codebase?
Aeolun 6 hours ago [-]
Anecdotally, codex is very fond of checking strings are equal between UI and test.
nottorp 4 hours ago [-]
Why not? If you let the LLM run amok, you get 4 one line functions calling each other instead of one 4 line function. Run that for 24 hours and you'll get more than 2000 tests.
maccard 5 hours ago [-]
They said all their tests are written by agents so probably, yeah.
miki123211 3 hours ago [-]
It really depends on the industry.
The blind community is benefiting enormously from coding agents. Game accessibility mods for everything under the sun (the big names in the last month or two are Civ V and Witcher, although there's plenty more), accessible 3rd party clients for annoying sites, people's favorite speech synthesizers ported to platforms they never ran on natively (or just straight down turned into portable C), plenty of small but nifty utilities and apps.
This is because that community's needs are amenable to what AI can do. Accessibility work (on somebody else's product) is a lot of demotivating and extremely difficult reverse engineering drudge work with a verifiable success criterion, and this is what AI excels at. Large-scale software dev is all about judgment and taste, and here, AI is not doing so well.
You see the same things with mathematics versus medicine. In math, the bottleneck is basically human attention, formal proofs in Lean are, again, drudge work with verifiable success. In medicine, the bottleneck is patients, paperwork and the lab environment, so even an omniscient LLM without the ability to pour fluid into a beaker wouldn't be that much of a productivity improvement.
girvo 14 hours ago [-]
The follow on question is if it's making us all so much more productive, where is the increased revenue? As far as I can tell, it's mostly the AI labs seeing that, not everyone using them (modulo small founders building new things and doing okay, I think)
pianopatrick 12 hours ago [-]
Personally I believe that this boost in productivity will not necessarily lead to greater revenue.
All the companies have the same access to AI, and AI is making them all better at doing what they were doing before (writing software). So some companies that use AI really well may be able to take market share from other companies that are slow. But I think this might just lead to a more intense competition for customers.
Kinda like what happened to music after digital recording became the thing - we have more music than before and the music is better, but being a musician became a much more intense competition to find an audience.
AppleBananaPie 11 hours ago [-]
probably a naive question but then would we expect companies not using it to lose market share to competitors that are using it?
3 hours ago [-]
pianopatrick 11 hours ago [-]
Well I don't really know what will happen in the future. My first thought is that what will happen depends on the company and industry. I can think of some non tech companies where AI might not matter for a while, like a car wash or a lumber mill. But long term likely yes for tech companies and companies where tech makes a difference.
oblio 5 hours ago [-]
People won't want to hear this, but for FAANG things are probably similar. I'm not very convinced that Meta, for example, is actually creating new revenue. It's just consolidating a lot of the existing global ad spend, it's just wrecked newspaper classifieds and the like.
p1necone 12 hours ago [-]
If every software company saw a roughly equal improvement in productivity, they'd still be splitting the same customer base amongst each other - not clear whether revenue would actually go up for anyone.
Perhaps some new markets could be entered that weren't feasible before?
zrobotics 10 hours ago [-]
But there have been trillions invested in infra for AI. Surely those investors are going to need to see a return at some point?
i_am_a_peasant 33 minutes ago [-]
apparently those trillions were necessary just to keep the boat staying afloat.
player1234 2 hours ago [-]
[dead]
ianm218 13 hours ago [-]
Isn’t cloudflare a pretty clear answer to these? They are launching more products than ever, some of them clearly vibe coded, and their revenue is exploding.
72deluxe 3 hours ago [-]
Is that because more people are signing up to use Cloudflare as a MITM to block the increasing avalanche of AI-generated traffic though??
Interesting to look at, a decent example for sure.
nsoonhui 12 hours ago [-]
It may not have to do with increased revenue, but it does have everything to do with reduced cost, especially developer's cost.
I bet every company is finding up how to level up their employees via AI, so that they can use less of them in the future.
So even without increased revenue, AI has its (mis)uses.
lacker 14 hours ago [-]
The increased revenue is in startups, like the recent couple YC batches.
csoups14 13 hours ago [-]
Do you have a source for that? Are they raising more money or receiving more money for AI-oriented products or are they actually making more money on consumer/B2B end products?
MikeNotThePope 3 hours ago [-]
Personally I’m leveling up on using AI to develop a project. My speed at knocking out PRs has increased a lot, but much of that is massaging the process of using AI, including managing CI bottlenecks and costs. I’ve never had to deal with 15+ PRs being developed in parallel before. It’s interesting work, but there’s a lot of doing stuff to get to the stuff I want to do.
Examples of things I’ve tackled this week:
- GitHub Actions costs exploding and looking into self hosting options (gonna try a box in Hetnzer)
- figuring out how to handle unattended builds
- tinkering a lot of email, from how to manage domain reputation and worrying about bots spamming from my server because they started filling out forms
- figuring out how to manage AI development from my phone
- setting up a process to prevent unintended destructive database migrations
- finding a way to know have to babysit the AI but approving a constant stream of questions and permission requests
- all the random little issues that the AI files to fix an endless amount of little things
- finding the right model(s) to use to most efficiently make use of the limits of my subscription
And so on.
72deluxe 3 hours ago [-]
Perhaps it is because the last 20 years has seen a shift in developer culture away from lower-level understanding and native applications so that all the vibe-coded apps you use on the desktop are Chromium-based web "applications".
If you apply AI to low-level applications, it is quite good and you avoid the runtime bottleneck of webness, I have found. Eg. https://github.com/Redrum624/Vitrine is very good but has a horrible UI due to the webby nature of its rendering - all vibe coded. If you used a native toolkit it'd feel completely different.
tecoholic 9 hours ago [-]
Similar to what others have said, we have had big changes in our systems, thanks to LLMs. Some examples of what we have done:
- Rewrote a data extraction and PDF bounding box algorithm - LLM provided the tooling to visualize the output of algorithm and find the right rules for our needs.
- Migrated an old SciBERT model that was bundled into a 6GB docker container that needed GPU, with to an ONNX based inference container about 900MB running on CPU.
- Setup a K3S cluster to replace our Nomad cluster.
- Optimized an algorithm that used SQLite with better indexes and optimized querying with some 60-80% performance gains.
We have reduce resource usage in specific areas of the application drastically. These were all possible before, but LLMs provided the tooling to iterate and deliver it in time and cost that seemed prohibitive before.
Now are the end users going to see the benefit and is the product magically better? Well no. The chicken answer is I am not involved in that side to know. But a more realistic answer is, user experience and product fit is not something LLMs can solve. That's still upto the humans to figure out and I think that's where this "nothing has improved" feeling comes from.
geooff_ 11 hours ago [-]
I've been bootstrapping for two years now, just me and AI tooling.
Code moves much faster, product taste doesn't. Once you have your marching orders you can make features 10x faster. Shots off target still miss though. It doesn't matter if your code gets generated 10x faster when it doesn't resonate with users.
deaux 3 hours ago [-]
"This" is lazy but yes, this. Especially in the app space, 90% of the stuff coming out has barely any "do people who aren't me actually need this, does this deliver something new and useful" thought put into it. It's a skill most people lack, they can't judge it.
Bootstrapping a solo software business has multiple core required skills. Technical skills are now largely unnecessary, but it turns out most people lack multiple of the other required skills too.
onel 3 hours ago [-]
Moose has entered the room.
I built a debian based OS focused on self-hosting: github com/onmoose/os
It's not really Linux size but I couldn't have built this if it wasn't for coding agents.
Still a ton of work needed on UX but the main functionality works so well.
Linux also has decades of work on it, we can't really expect new products to equal that in 2 years of agentic coding
vachina 3 hours ago [-]
> Is everyone just running full speed in circles or something?
Yes, just rehashes of the same thing that was done before, in another language/framework/codebase/fork.
wannabe44 6 hours ago [-]
AI doesn't fix bad ideas. As computer industry has become the cash cow for erstwhile bankers and stockmarket brokers, expect decline of general software and rise of a small niche (zig is a recent example but also suckless / plan 9 crowd, gentoo linux, some parts of FP / PLT crowd) which try to preserve their own little world despite economic incentives to do otherwise.
hexapus 9 hours ago [-]
AI is a rocket motor.
Doesn't matter where we're going, as long as we get there FAST!
It's a sickness.
Aeolun 1 hours ago [-]
Pretty convenient if you are pointed at the moon though, because there is no other way you’ll get there.
dagmx 14 hours ago [-]
In my experience everyone is just rebuilding the same wheel over and over.
A lot of people may be more empowered to create things now with less up front effort but it doesn’t lead to having better ideas or more actual system architects.
jas0n 14 hours ago [-]
Why do you assume "we not seen an improvements in products"? Who is "we"? Have you done an exhaustive (or even half-assed) analysis or is this just your "vibe"?
torben-friis 13 hours ago [-]
>Why do you assume "we not seen an improvements in products"? Who is "we"?
Have you? As in, can you name products you use daily that have been transformed night and day?
I am not talking about the existence of minor improvement, mind you. I am denying the existence of anything that is minimally consistent with the level of discourse that surrounds development itself (throwing away practices, checks and balances, hitting constant roadblocks due to the new speed, etc).
A true 100x requires no analysis. If we could suddenly build and improve houses at 100x I would know because I would be typing this from my 5th living room.
bigstrat2003 8 hours ago [-]
Because we haven't.
sensanaty 11 hours ago [-]
I've definitely noticing buggier and buggier software, that's for sure. Even our internal tooling and CI pipelines and all that kinda stuff has taken nosedive thanks to the deluge of garbage
Aeolun 13 hours ago [-]
All of the work work at my company has been internal facing. People are rightly leery of exposing things made by an LLM to the public.
xnx 7 hours ago [-]
I wanted to review 600+ YouTube streams and made a custom Tinder-like interface in less than a minute with Antigravity. I would've just slogged through using a spreadsheet before.
Using the custom interface instead save a lot of time.
0xdeadfa11 5 hours ago [-]
You can pretty much throw any PC game at Codex and get a playable VR mod with full 6dof and in most cases even motion controls
mrheosuper 5 hours ago [-]
I agree, look at the top token maxxer: Microsoft, Facebook, Twitter, are there observable improvement ? Like the performance of Windows is still...not ideal, every Windows update still has some issues. If AI is indeed 10x, shouldn't we(end user) see at least 2x improvement ?
iknowstuff 14 hours ago [-]
AI doesn’t make a responsive right-click menu a higher business priority. But we are seeing a ton of small custom projects that are as easy to dismiss as they are to abandon.
torben-friis 14 hours ago [-]
Surely it makes the backlog clear faster to the point where you reach the non-priority stuff? But I'll bite, what business priorities are being created at 100x?
xx__yy 10 hours ago [-]
Agreed, but give it time. The tipping point of good models only just arrived with Fable 5, GLM 5.3, Grok 4.6, Muse Spark 1.3, and the like. Those are the only models that do a better job than me, and I'm happy to hang up my IDE—and that was barely a month ago.
I've been fixing everything that has been sitting there—not blocking us, but slowing us down—all the things we never had the bandwidth for. We're now squeezing more out of development, CI, and production.
I'm not sure why, but I'm concerned this may be the heyday, and we might not get this again at this price or speed. So I figured we should clear the backlog while we can still afford it and still have the ability to do so.
But now things have changed, and the bigger, grander ideas are starting to brew.
EarthBlues 11 hours ago [-]
The problem, as ever, is figuring what's worth doing and what's not. LLMs are not as useful there.
lacker 14 hours ago [-]
Well you mention Android, iPhone, Nintendo, and Microsoft. Those are the biggest companies, and many of them don't even use the best coding tools. So they are not going to be the ones improving.
On the other hand, look at the YC companies. I think the last batch or two is growing revenue faster than any batches ever before. Those are the companies that are being sped up a huge amount by AI.
compiler-guy 9 hours ago [-]
It’s hard to imagine a company more all in on ai than Google. The pressure to use it is intense and they are throwing ai at basically every problem. If a proposal doesn’t have ai in the title somewhere it has no chance of funding.
I don’t really think it has made Google products better, but it is absolutely the expectation.
Paradigma11 3 hours ago [-]
I would guess most are working on improving their workflow/tools and keeping up with the rapid changes.
3 hours ago [-]
solarkraft 12 hours ago [-]
The new iOS has actually increased the quality for
the first time in years. There are open source projects now that tackle problems nobody has before.
It’s not night and day (as you’d expect from the hype), but I see some differences.
SrslyJosh 12 hours ago [-]
> The new iOS has actually increased the quality for the first time in years.
In what ways? When did previous releases stop meeting your criteria?
> There are open source projects now that tackle problems nobody has before.
Such as?
solarkraft 10 hours ago [-]
> When did previous releases stop meeting your criteria?
Ages ago. It was really noticeable when they prioritized features over quality.
> Such as?
HN is full of them. I don’t have great have specific example at the top of my head, but lots of hard reverse engineering/low level projects credit AI and that they would never have happened without it these days.
TonyStr 4 hours ago [-]
> Windows takes 3 seconds to show the right click menu
Wow. It actually does. I thought you were exaggerating.
72deluxe 3 hours ago [-]
Which OS is this on? I have a Windows 10 machine I keep alive for development and actual weekly use - is this issue a Windows 11 thing? I don't use the Windows 11 machine I have at all.
TonyStr 2 hours ago [-]
Yes, this is Windows 11 (forced by company). I believe the first right-click is blocked by loading something, though I'm not sure what, as after opening the menu it shows "Loading..." in place of "Open With Terminal" for a second. Subsequent right-clicks are ~1000ms and sometimes 2000ms to open. Probably also varies by equipment, as my company machine is loaded with a lot of bloatware, though it's not a weak machine by any means.
kevin_thibedeau 11 hours ago [-]
The real question is why are people too lazy to spin up their own phorge. It's a form of learned helplessness supported by a supposed business case to prefer buying services rather than bootstrapping your organization with proven technology.
oblio 5 hours ago [-]
Do you mean "forge"? As in Git forge, like Github?
bs7280 14 hours ago [-]
The same reason it took many years for corperate america to get a real productivity increase from computers and the internet, all the old ways of doing things had to be redone. I think the largest companies are least equipped to take advantage of AI productivity gains. Agile no longer makes sense, Org charts as no longer make sense, etc...
pianopatrick 12 hours ago [-]
I've been noticing much faster UX changes in the Youtube for Android App.
I don't know if it's AI or not, and I haven't liked all the changes, but there have been more of them than I remember before.
8note 10 hours ago [-]
incentives are not aligned to improving products.
the important thing to microsoft is still finding new ways to make money, not to make your clicks faster. they might even be using their new velocity to make clicks slower with an ad in between.
there's alao much less incentive to build things to share. people can just make their own widget app and not need yours
tonyhart7 7 hours ago [-]
"Why have we not seen an improvements in products?"
we literally do, we shipping feature faster than ever
TuxSH 11 hours ago [-]
> Switch 2 remains unhacked
What is Switch 2 security doing here? (independently of it having so few features added per update)
oblio 5 hours ago [-]
My guess they mean some kind of jailbreaking.
TuxSH 18 minutes ago [-]
Sure but it's the odd one out in a list of whishing manufacturers added features.
"Switch 2 remains unhacked" is rather disrepectful of the hacking community, and runs on the assumption that the Switch 2 has significant vulns. It's very well possible it doesn't have any high-privilege software exploit at all.
After all if one cas use AI to find vulns and/or to RE, it's even easier for the OS developer (Nintendo) to use it to find bugs in their code before release.
Fordec 12 hours ago [-]
I mean, looking at the release notes for things like say the Linux kernel it is clear that the improvements are there at a massive rate of change. But it's small stuff, it's performance improvements, it's quality of life. If you do things right people will think you've done nothing at all. What did you expect? Businesses to come up with whole new business lines, or is it just that backlogs of work is getting burned down that was going to be gotten to eventually? From what I can tell from the PM forums, the next bottleneck is the Product Org coming up with ideas worth implementing as fast as the developers can deliver them.
oblio 5 hours ago [-]
> What did you expect? Businesses to come up with whole new business lines?
For a $2tn industry, YES.
inglor_cz 3 hours ago [-]
So, my 2 cents: we are mostly using AI to analyze and fix things in our codebase that were hard to fix manually. Rare race conditions. Extra unit tests. Things that just didn't pass the effort+cost/gain ratio before.
The quality definitely improved, but our products were usable before, only not perfect. The improvement is mostly on the margin. Instead of crashing twice a week, the app crashes twice a month. A communication session won't fail in two hours, but maybe in two days. (But the average session in the real world was < 20 minutes anyway.) Some UI elements that were not necessary, but are nice to have.
the_sleaze_ 13 hours ago [-]
We 100% have. I've seen significant improvements in many of the services I use.
csoups14 13 hours ago [-]
Drop a couple names, I'd like to know who is leveraging AI well
written-beyond 13 hours ago [-]
Not doubting you, care to share what those improvements were?
bigstrat2003 8 hours ago [-]
> Is everyone just running full speed in circles or something?
Yes. The lack of actual tangible results is how you know that the claims of increased productivity are false. We haven't seen a bunch of new useful apps (or anything else for that matter), which we would have if LLMs actually worked.
azan_ 4 hours ago [-]
What do you mean by llms not working? Did you use one recently?
sandeepkd 9 hours ago [-]
well people are spending tokens like crazy for sure, Anthropic and all other labs and everything related to AI seem to be making tons of money.
On a serious note, it might take a while to realize the actual benefits or losses. Its clearly not a good signal when people whose job is to manage other people start writing their own pet AI projects. This just indicates that AI has created this big job insecurity among everyone. At the end of the day being an engineer, eventually the job is relatively safer when there is so much code being written out there.
Nursie 8 hours ago [-]
> Why have we not seen an improvements in products?
Ours is moving faster than ever in terms of feature delivery, and we've used AI to really hammer at the security aspects and clear a load of backlog stuff.
aprilthird2021 9 hours ago [-]
AI makes a lot of drudgery type coding tasks a lot easier (think code migrations, etc.)
But actually coming up with, testing, and rolling out loved new user features. AI isn't that good at that and we can't really "prompt" that out of it as easily as we can prompt a py2 to py3 conversion of old code
cute_boi 10 hours ago [-]
The improvement isn't happening because company are also actively firing people in name of layoff and aren't actively hiring.
deaux 3 hours ago [-]
If you believe the explosion of high quality 1-3 people indie games on Steam has nothing to do with AI I don't know what to tell you. The amount of slop has increased 20x but the amount of good quality ones has also increased 4x. And it's not that hard to distinguish them.
player1234 1 hours ago [-]
[dead]
aliclark 15 hours ago [-]
In my case it's not the CI that's the bottleneck. It's the human testing side. Does it work, sure. But does it actually do the thing we want (and more importantly) does it do it in a way our customers will understand and actually like?
lbrito 14 hours ago [-]
I feel no one talks about this, and yet it is glaringly obvious. Are we even solving the right problem? No one cares. Push code. Number go up.
Aeolun 6 hours ago [-]
That problem isn’t new though. It’s not the biz people that are enthusiastic over AI, it’s the devs.
mk89 6 hours ago [-]
You're so lucky to work for a company where PMs don't vibecode and tell how easy it is to do things....!
skort 14 hours ago [-]
I feel like if anything LLMs have reduced coding/software to "throw as much as possible at the wall and see what sticks". It feels quite shortsighted and wasteful, especially considering that humanity needs to get better about how it produces and consumes energy. It's sort of bleak.
eggplantemoji69 11 hours ago [-]
Which is a bit terrifying and removes the ‘engineering’ component of a critical part of the job, in exchange for slot machine / pray to ai gods…which comes with disconnect from the code, something I assume will adversely compound over time.
7 hours ago [-]
pydry 14 hours ago [-]
hasn't improved software either. it's made the overall quality worse if anything.
radarsat1 4 hours ago [-]
I was just discussing with my team that this exact thing maybe brings back to relevance the idea of "behaviour driven development" and I was reminded of this Cucumber/Gherkin lib & language that defines a kind of executable prose you can use to specify how the software should behave. It's an interpretable programming language but designed to be close to how a human might just write down their specs of what kind of actions and responses are expected from a software system.
The idea is to drive actual testing from this, but in this era, I think it's interesting as a way to use AI to generate tests, and to cross-check those tests with the natural language descriptions, in a bit of a cycle that helps refine the highest level definition of the software.
Once that's nailed down, the implementation is just details.. normal engineering concerns like maintainability etc notwithstanding of course, but you can trust more and more the AI agents to get it right. The design specs being natural enough for humans to deal with but interpretable/specific enough to actually generate tests is pretty interesting for the bottleneck you are talking about, I think.
shykes 15 hours ago [-]
Think of it as a layered problem. If the bottom layer (CI) cannot keep up with the output of agents, then solving problems at a higher layer - like user experience checks - will be exponentially slower and less reliable. Kind of like how optimizing tight inner loops makes your whole program faster.
user43928 14 hours ago [-]
It's true because it usually makes no sense to QA test a build where the test suite has not yet passed.
Sharlin 15 hours ago [-]
Uh, no. If stage X is the bottleneck, it's immaterial how much you speed up stage Y.
shykes 15 hours ago [-]
You're assuming QA reviews won't be fully automated, and triggered from a CI pipeline.
bluGill 11 hours ago [-]
QA is not automatable. QA is about finding the things that you didn't think of and so didn't write a test for it. That many people think qa is not important shows in the bad software we have.
shykes 7 hours ago [-]
> QA is about finding the things that you didn't think of and so didn't write a test for it
QA is about quality. That it has been increasingly used to mean "repetitive manual testing only" is part of the very trend you decry of qa being undervalued.
Fundamentally QA is about two groups of people collaborating in an adversarial process ("you build it we break it") to achieve a common goal: a better product. QA by definition requires a relationship of equals, or the process ceases to be adversarial and becomes useless ceremony. If you're not able to assemble two distinct groups of people, one person can wear both hats (ie you test your own software) but you'll have more blind spots.
How much of QA is done by a human or automated, is and has always been an implementation detail.
If you're in charge of QA, you're in charge not just of finding new problems, but preventing regressions as well. And you're responsible for embedding as much of that work into the devloop itself, so that builders can find and remediate problems earlier and with less pressure on your limited time. How do you achieve that? automation.
I started my career doing QA and 99% of what I was doing could be automated by an AI today. Today I still do QA on my own product- I just spend all my time on the remaining 1% and my product is better as a result.
oblio 15 hours ago [-]
We can assume lots of things, but at some point the rubber must hit the road.
What exactly are you using to back up this claim?
> QA reviews [...] fully automated
willsmith72 13 hours ago [-]
to me the last part if the most important part. product management hasn't been automated at all, and is more important than ever. you can't just keep adding features to make a great product
Sharlin 15 hours ago [-]
Clearly you have to replace obsolete human testers with agentic AI testers, duh.
At some point, with all this velocity, human users become the bottleneck, unable to keep up with and learn all the changes and new features. Luckily, there's a simple solution: just replace the human users with agentic AI users.
dwaltrip 15 hours ago [-]
Maybe we can also replace the customers with agentic consumers.
oblio 15 hours ago [-]
> The forceful executive, Henry Ford II, and the leader of the automobile workers union, Walter Reuther, both saw many examples of advanced machinery operating at the plant. The words they exchanged brilliantly encapsulated the paradox of automation:
> Henry Ford II: Walter, how are you going to get those robots to pay your union dues?
> Walter Reuther: Henry, how are you going to get them to buy your cars?
That’s what I’m doing! Results yet to be measured.
shykes 14 hours ago [-]
I think you're being sarcastic, but there is actual truth behind what you're saying.
Because every developer is now a slop cannon by default, by default users will experience churn and whiplash, and things will break all over the place. As you point out, this is bad. It's also impossible to fix without deploying agents on the QA side. Like it or hate it, agentic testing is inevitable to protect users from the churn and noise caused by the slop cannon. I don't think that replaces test engineers at all - if anything it makes the job more fun. If you've ever had to keep playwright tests in sync with the target manually, and kept the CI environment up to speed with toolchain changes, you know what I mean.
Whether the "slop cannon by default" situation could have been avoided in the first place, is another question... But we're here now and there's no going back. Might as well deal with it as best as we can.
TLDR: it's not all bad :)
vips7L 7 hours ago [-]
Amdahl’s law.
dgroshev 11 hours ago [-]
I suspect a substantial part of this is an avalanche of useless testing.
If you even review PRs still: when was the last time you didn't just skip over tests? And if you ever looked at tests in an LLM-heavy PR, how many of those tests tested something useful, and not just built-ins and trivial behaviours?
There's at least some awareness in the industry of how LLMs generate a lot of boilerplate in business logic. It feels like we're much less aware of how much of it is in tests.
sz4kerto 5 hours ago [-]
> If you even review PRs still: when was the last time you didn't just skip over tests?
The opposite. Reviewing PRs is now about just reviewing the tests, as the code is very likely to be fine if tests are relevant and they're passing.
The main enabler of agentic coding is heavy end-to-end/characterisation testing.
We can ask AI to do very large-scale refactors, for example, because if tests pass, it's 99% that everything is fine.
_zoltan_ 2 hours ago [-]
I actually look at the tests first. But for this to work, you have to have good hygiene. If the test makes sense, fails without the PR but passed with the PR, I'm almost happy.
epolanski 26 minutes ago [-]
I was (luckily) not reviewing PRs even before AI.
It's a practice, it's not the best practice. It depends on the team/org/codebase/feature/etc. It always costs time and money and effort. Lots of it from multiple people.
You can get much better output by shifting that cost into hiring much better professionals, not better developers, but overall professionals.
The kind of people you can blindly trust that the software they are writing will be good, you don't need to get involved.
Of course there are exceptions. The author may actually want a review. Or the piece of code might be touching something extremely critical to the business but also easy to get hard.
But besides that? PRs are just productivity porn, or "we do engineering right because we follow Twitter" porn.
The best performing teams I had you hired individuals that removed work and responsibilities off your shoulders without you ever having to regret it. Never added it.
I laugh off engineers that "no you have to review, because it spreads information, enhances quality" and yada yada yada, while in the real world way more critical decisions are made by a single individual without requiring somebody reviewing their work.
MikeNotThePope 3 hours ago [-]
Probably half the generated tests in my app are for markdown files.
classictraffic 15 hours ago [-]
> Moving our workloads off GitHub Actions to third-party runners with faster CPUs, higher-performance storage, and better cache infrastructure gave us faster machines to run the same pipeline on
Yeah, was not surprised to read this. Actions is convenient if you already use GitHub, but it can also be pretty slow. Given reliability is also a major issue with GitHub these days I expect to see more orgs moving to different pipelines
wereHamster 6 hours ago [-]
I wonder why larger companies don't use self hosted github runners. You can buy a pretty beefy machine (TB of RAM, 256 cores, fast NVMe disk) and tests will run faster than on any hosted platform. Plus you don't have to shard as aggressively because more fits into one machine, benefit of shared page cache, shared persistent disk, can easily preserve working directory (for example my pnpm install takes 0 seconds, because the node_modules folder is already present from the previous run).
Lucasoato 4 hours ago [-]
Microsoft is pushing so hard to get companies far away from the on-premises world. Once they're in the Cloud, there's too much vendor lock-in for big enterprises to go away.
deaux 3 hours ago [-]
It's sad that even so many greenfield businesses still default to it. There is no longer a need to. The barrier to alternatives used to be technical, now that's gone. 99.9% of web apps' CI needs are solved by a $10/month box.
oblio 5 hours ago [-]
> I wonder why larger companies don't use self hosted github runners.
Because they're running away from on-premise and the associated Ops teams.
speedgoose 15 hours ago [-]
I wonder how much faster GitHub actions could be if they weren’t running on Azure. Because Azure is either slow or very expensive.
chris_money202 14 hours ago [-]
I think its actually because GitHub isn't solely running on Azure. They are currently using a mix of on prem, AWS, and Azure. The Azure migration has been a challenge with Azure running into capacity issues due to AI load.
rtpg 7 hours ago [-]
to be fair all CI providers I've had the pleasure of working with have gnarly performance profiles for the boxes they provide.
I don't think it's out of malice, but I do feel uncomfortable with the fact that the people who sell me the CI coordination software also sell me the minutes for the boxes that run the CI software. There's _some_ alignment of interests but not as much as I would want!
stackskipton 10 hours ago [-]
It's not Azure as much as self-hosted runners are basement bin servers on clearly massively oversubscribed machines.
dneri 15 hours ago [-]
We switched our Actions workload to blacksmith.sh (not affiliated) and have been pretty happy with how fast and inexpensive they are. I wouldn't be surprised to see this trend continue.
prplfsh 1 hours ago [-]
Does anyone do approaches where developers run CI locally on their machine and then somehow push up signed or authenticated artifacts?
Tade0 56 minutes ago [-]
I second that question. Several times already I wished I had such a setup because the system was so congested that jobs would just time out.
Of course you can increase the duration, but if something is supposed to take less than 10 minutes but takes over half an hour, increasing limits won't really address the problem at hand.
Good thing Linear has been finished for a long time doesn't need more features, so AI coding can go slow. Oh damn, it's busy becoming the next Jira :(
reticulates 15 hours ago [-]
I’m conflicted about Linear’s progression. I dislike some of the features but on the whole they’ve managed to keep the software pleasant to use, it doesn’t feel to me that it is drifting towards Jira territory, rather, it feels like it is losing the carefully considered product design because now code is cheap to generate. I’m not worried about it turning into Jira but it has lost its soul. Still a great product.
mh- 14 hours ago [-]
I haven't used Linear in any serious capacity, but I feel like there's plenty of market in "Jira that doesn't feel like it hates its users" and "Jira but we care about performance".
Not saying that they're shooting for either of those segments, but someone should. Plenty of enterprise orgs that need (or are convinced they need) Jira's featureset.
Aeolun 6 hours ago [-]
To be fair, the forced cloud migration has (unfortunately) been pretty good for performance.
epolanski 15 minutes ago [-]
How could it be different?
Linear is essentially a small subset of Jira.
It's liked and works because it's simpler and covers most of the needs of teams without huge need of defining their own processes.
But be in business long enough and you find out that appealing to small teams and small orgs that don't require flexible and custom process definition doesn't make enough money and doesn't grow you forever.
So they will keep eventually growing in features and flexibility till they cover the reason Jira is so successful: it is more powerful and adaptable.
I'm no fan of Jira, but I've tried and seen enough of these tools to understand why it has the success that it has.
This is even more clear when you go beyond software development and need processes for teams like sales, HR, legal, etc. Jira can fit all of them.
Linear? It's a joke.
stavros 14 hours ago [-]
All software becomes popular because it's not the old behemoth, but it's new, fast, and simple. All software then grows and grows until it becomes the new behemoth, and the circle of life begins anew.
the_chrismo 20 minutes ago [-]
If you're not big enough to just roll your own, RWX is a wonderful option.
frangonf 13 hours ago [-]
My biggest takeaway (as a one man band 20 leagues below this spending time optimizing Gitlab free plan and self hosted runners): they did not worry about any of this until 100M ARR and more 1B valuation[0].
With the widespread increase in speed/commit cadence/content added to repos, I wonder if there's an angle here for companies with a lot of CI/CD needs to simply start self-hosting their CI/CD machines on premises.
In my experience, self hosting your CI/CD runners had the biggest impact in cost savings throughout, while also allowing for more powerful machines, which directly means quicker CI/CD runs
azkalam 16 hours ago [-]
If you can pay the setup cost, Bazel will get you build times ~10s with a warm cache for even massive projects.
criemen 15 hours ago [-]
I lead a bazel conversion for a pretty complex piece of software written in 5+ programming languages and shipping native binaries to all 3 major OSes a few years ago, and it took multiple years to get it done.
For a less complex project (1 programming language, still shipping to all 3 major OSes), with my knowledge and agents I got the bazel conversion done in 2 weeks.
The setup cost for bazel just went down by a lot, and I don't think the industry as a whole is aware of that yet.
rokob 15 hours ago [-]
Were any of those using JS/TS? I've seen Bazel perform wonders with many compiled languages, but I'm not impressed with the JS ecosystem.
criemen 15 hours ago [-]
Nothing substantial, no. I've never personally experienced builds to be so slow to begin investigating bazel as an option for JS/TS.
Tsgo, oxlint, caching dependencies etc. what linear outlined in their blog post would be more impactful for the average TS project I've worked on.
teaearlgraycold 15 hours ago [-]
The entire industry, including its outputs that LLMs are trained on, hasn’t reconsidered what’s easy vs. hard or fast vs. slow. LLMs consistently recommend against code changes because they will take “a weekend”. No, Claude. You will do the work and it will take 20 minutes.
manquer 15 hours ago [-]
Can't comment on Bazel specifically, but having worked with both nx and turbo, the bottleneck was usually network and disk IOPS rarely compute.
Even fully cached outputs needs to fetched and read from a remote server[1]. A step n-1 outout fetched from remote cache server need to written to disk and then again read by step n[3] - all disk I/O and network bound operations.
10s may be achievable/realistic goal in the Java/C++ world where Bazel normally seen. In TS eco-system most people would be over the moon to get into ballpark of 1-2m for a decently large monorepo.
We should define Build more clearly here, if you mean running just transpile/compile steps or the full series of steps that includes tests (as the linear post here is talking about). It is hard to see even a small sub-set of a large suite of test that require a virtual DOM or a real browser can run in 10s or less.
[1] Typical for say managed CI setup .
[3] Common run-of-the-mill frontend + backend stacks in different languages etc.
criemen 15 hours ago [-]
If you don't need to rebuild anything, bazel can fetch only the final artifact (not the intermediates) from the remote cache.
Also, if you have persistent CI workers with a persistent bazel instance, you save on some network roundtrips, but that's obviously harder to set up and make bulletproof.
manquer 14 hours ago [-]
Typically you need the intermediates to compute if you need the next one , so you cannot skip to final until you have the intermediaries.
The final asset/artifact is rarely small either. even best optimized artifacts can be few hundred MB docker image or more commonly multiple image layers running GBs in size .
each step is a network pull then recompute cache if stale and keep going till end .
sluongng 6 hours ago [-]
We solve it 2 ways in the Bazel ecosystem: for the intermediate artifacts, we only fetch the digest (hash + size) of the blobs to calculate the merkle tree forward. The blob itself can stay on the remote cache server.
For the bigger final artifacts, we support using Content Defined Chunking (rolling gear hashing) to only fetch the missing chunks between incremental builds. Binaries executable with stable layout benefits from this quite a lot.
We are definitely not done with all of the improvements here. But since all the major AI labs are using Bazel, we know that the tools can support “Agent Scale”. https://webazel.dev/
manquer 2 hours ago [-]
> calculate the merkle tree forward. he blob itself can stay on the remote cache server.
Not sure how that would work with building say a docker image, reproducible builds are pretty hard problem to solve, and caching intermediate layers is not always simple or even doable, we typically still need to publish to a registry which is not the cache server.
sluongng 1 hours ago [-]
The Bazel ecosystem builds container images not by using Dockerfile, which contains non-reproducible primitives such as RUN and others. We do it by actually constructing the file trees and tarballs manually, then using them to compose the JSON manifest and indices. This is done via smaller hermetic and reproducible Bazel actions and thus enables the ecosystem to scale way beyond what alternative BuildKit-based solutions can.
well for test runs you kinda need the binary to run, but you're correct, if the job is just "does this build" no download necessary.
seniorsassycat 15 hours ago [-]
Bazel seems to have a lot of tradeoffs, from setup time of the sandbox for each task, to ergonomics that lead folks to maintain parallel 'normal' tooling.
Plus, 'with a warm cache' is doing heavy lifting, what's the real cache hit rate for a week of development? Investing in improving the cold build and frequent actions is still important with bazel or any incremental builder.
I'm not sure it's useful to talk about bazel broadly, it's actual performance and behavior comes down to the rules you use. You can configure bazel like turbo/nx and cache tsc/vitest/eslint on each package.json module, and get course cached units that are evicted on every change, or you can use gazelle and target per-file actions which are only invalidated when their dependencies change. But that trades off batching unless you use workers.
sluongng 6 hours ago [-]
What i have seen on my end is that it’s pretty easy to setup a cloud coding agent with a warm Bazel cache. “Warm” here can means multiple layers: same disk to CoW/hardlink, different disk, network disk/block devices, same rack/datacenter, same AZ, etc…
And yes, there are a ton of investments going toward Bazel recently to unlock these newer use cases.
rienbdj 15 hours ago [-]
Most PRs only touch a handful of targets so cache hits are extremely high in practice.
seniorsassycat 13 hours ago [-]
Depends on which targets and how granular the caching is.
- if you touch package.json that might invalidate everything
- touch core and you'll invalidate everything
- touch one file in api and you may run all api tests (see gazelle)
I ran an experiment where I migrated a package to bazel then replayed a weeks worth of changes and it saved 20%. That's nothing to scoff at, but not the headline numbers you see after a full hot build.
hamolton 6 hours ago [-]
Is this really easy in the AI era? It seems like the repetitive, verifiable work that the agents should be good at.
gabriel-uribe 4 hours ago [-]
Self-hosted is the way. Small potatoes in comparison, but I also made the change this year as my CI usage went from <3k mins/$0/mo -> $100+/mo.
Migrated actions to a spare M2 MacBook Air at home, and when offline, uses hosted Actions as a backup. Bonus: has simulators/emulators for ease of running Maestro tests for mobile QA.
miyuru 4 hours ago [-]
>Because the third-party runners sit outside GitHub’s network, they rely on a direct IP link to reach GitHub. The provider traced the hangs to intermittent degradation on that link.
This is interesting. If I read that correctly this means the CI host had a direct peering connection with Github or Azure network.
Why were they not able solve the issue?
the blog mentions that they use a local cache to "reduce the time" but the problem is not solved right?
CharlieDigital 15 hours ago [-]
My take: it seems like systems should become smaller, more isolated, and contract-oriented.
I have been a long time proponent of monoliths, but it seems like agents would be happier with smaller, more isolated services. The more isolated, the better. Contracts between the service components only. Then it can iterate internally as long as it satisfies the contract. If it needs to, it can version the contract and keep iterating.
rsanheim 2 hours ago [-]
If we lived in a world where people or agents could define those service boundaries up front, correctly, with some reasonable foresight for change on the horizon ...then sure...I'd agree.
However, our world is not that world. Your agents may be happy in their tiny walled kingdom of toil and ineffectiveness, but you and your users will not. Poorly drawn service boundaries will drag you down more than almost any other architectural mistake.
If there is one universal amongst organizations, its that they love walls and silos. Be wary of putting them up ahead of time, cuz tearing them down once established is nearly impossible.
CharlieDigital 13 minutes ago [-]
> If there is one universal amongst organizations, its that they love walls and silos.
What's true for human organizations isn't necessarily true for agent-driven engineering. People and human teams struggle with contracts because there's always human negotiation involved. If the decisions are instead made by a team of agents, there's no more ego, ownership, miscommunications; just decisions based on whatever rules have been given to the orchestrator.
> Your agents may be happy in their tiny walled kingdom...
Yes indeed; the agents will always be happier if they can iterate faster, ship faster, build faster, with smaller context. That would be the point of using contracts as boundaries so the agent can iterate more autonomously so long as it maintains the externally facing contract or version the contract if it needs to.
CuriouslyC 11 hours ago [-]
Microservices are still bad. You want either a modular monolith or FaaS with most of the work in a domain library.
0xbadcafebee 10 hours ago [-]
None of them are bad. It's like saying a bike vs a scooter vs a car is bad. Use them correctly and for what they're intended and they work fine. Use them improperly for the wrong things and suddenly people think the tools suck, when it's the humans who misused them that suck.
CharlieDigital 10 hours ago [-]
[dead]
mraza007 16 hours ago [-]
Agentic Coding has been a huge strain on CI,
I have been using Bazel to improve our build times and ultimately building customized runners to improve our CI
Anyways great blog post from linear team a lot to learn from it
to11mtm 14 hours ago [-]
This still surprises me.
Mostly around, at least the majority of 'stuff' I've worked on (both before and after the rise of coding agents) myself or others took enough time to make sure that anything done locally, if you run the tests locally, you're at least 90% of the way there as far as what CI/CD does.
I suppose the flipside being, most of those projects had less churn (i.e. one person was working on a service at a time, and we had good contracts between services.) Also, Our local boxes were way better than our CI boxes, so there was incentive to run locally versus waiting 2-10x the time for CI to run...
saagarjha 8 hours ago [-]
> Despite our test suites almost quadrupling since the start of the year
Did the tests produce four times as much value, though?
fragmede 8 hours ago [-]
Unknowable. How much did a bug that was caught by those tests, and did not reach production save the company? What if there were many such bugs? Or none? How much extra time was lost to developers waiting around for tests to run?
zackify 11 hours ago [-]
Our CI has thousands of tests and is done in 2 minutes. Everytime it grows above 3 I add more parallelization and keep it ultra fast. Been working great for our team
MikeNotThePope 3 hours ago [-]
How are you speeding up the provisioning? Most of my tests run pretty quick, but spinning up a new runner takes probably most of the time. I haven’t tried optimizing this yet.
shykes 11 hours ago [-]
I think to solve this problem properly, we have to stop treating "build" and "test" as separate buckets of work, to be designed and scaled separately. It's all CI. As soon as you try seriously scaling out tests, you will run into build bottlenecks. To truly scale CI you need a scheduler that understands your build, test environment, and all the glue in between, well enough to schedule it in a way that actually speeds things up. That is very difficult and not something that even the best build tools can do - yes, even Bazel. Bazel can run tests but it's not nearly as good at it than at building.
sluongng 1 hours ago [-]
Yes. Bazel is half of the equation. The other half which other enterprises rely on is the server side of Bazel's Remote Build protocol. There, the scheduler can be implemented with logic that accounts for sandbox sizing as well as other requirements/constraints.
I wonder if the GitHub actions outages we keep seeing is due to themselves making self-hosted runners paid, hence bringing broke/cheap users back from hosted runners to their garbage infra.Meanwhile I'm running my Codeberg Actions on the free Oracle ARM machine 2 cores 12GB ram (previously 4 cores 24GB) and way more reliable. Keep winning bozos.
Edit: After second thought, I guess "alternative runner" providers still have to pay the self-hosted tax. So M$ actually saves and makes money by not scaling their infra and driving people to alternative providers they can tax freely. Actual geniuses.
watermelon0 14 hours ago [-]
They abandoned (or at least postponed) the idea of charging for self-hosted runners usage.
Actions management plane is the most problematic part of the platform (in regards to scale), so they are basically losing money with self-hosted runners.
Scaling of VMs shouldn't be a problem, since that is practically stateless, whereas there are lots of operations performed for every job/workflow.
cc62cf4a4f20 3 hours ago [-]
Unless this impacts net income, it's just theater.
colek42 13 hours ago [-]
We are working on something here, pushgate.dev. It forces the agent to run tests; we are also working on adding support for static analysis of bundles using Jev. Onboarding is a bit of a mess right now, but send me a message cole@testifysec.com if you are interested.
16 hours ago [-]
d_sc 16 hours ago [-]
vitest load balancing on test duration times instead of by file would be a nice performance win
moofeez 16 hours ago [-]
We actually tried this and (for us) it doesn't lead to enough of a gain that's worth the extra complexity.
pushpendraw 14 hours ago [-]
Speed up CI and the next bottleneck just moves to deploy and rollback, which do not scale the same way.
tokioyoyo 6 hours ago [-]
> deploy and rollback, which do not scale the same way
Why not? A good chunk of software engineering problems can be solved if we re-structured our definitions. Even just a year ago, people did not believe that a good chunk of prod-running code would be completely AI generated. Now we're at the point where some companies try to split "what code has to be PR-reviewed, and what doesn't have to be" to remove that bottleneck and so on.
shykes 12 hours ago [-]
You are right, but one problem at a time :)
LastTrain 7 hours ago [-]
The next bottleneck is customers. There better be a lot of them, they better have loads of money, and they better be willing to install updates for every fucking release that goes out whether they want it or not because how else are we going to pay for all this?
klntsky 6 hours ago [-]
It's strange that they did not mention reworking the tests themselves. In my experience AI-generated tests are absolute garbage - unless you specifically prompt to reason about coverage to avoid duplication and to merge some.
I did that by self-hosting Github Runners, using Podman to build images (it doesn't squirt your build context through a socket every time!), and using Docker images as content-addressable cache.
I will do a write-up about it this week...
thald 4 hours ago [-]
Another approach is to redesign CI pipelines. Most checks should be done within the agent loop for quicker feedback (using hooks, skills). Linting, unit tests etc.
CI should handle only stuff which cannot be run locally due to resource or setup reasons.
Also CI should only receive pre-validated change candidates with passed checks inside the agent loop.
inshard 15 hours ago [-]
I just feel apps like linear are increasingly getting in the way of full send agentic development where sub agent orchestration is done through agent to agent messaging, work trees, on demand git restructuring and epoch specific coordination plains, often .md files. The smaller the human component of total product development gets, the more this may be the case.
CharlieDigital 15 hours ago [-]
Question here:
> where sub agent orchestration is done through agent to agent messaging
How do you expect to see the history/record of what the agents did and why? Is it enough to see it in PRs? Do you expect tickets that have the design and history? How are you thinking of agents being able to historically resolve reasoning/why/decisions made in earlier passes?
Genuine open question here. My assumption is that a GH or Linear or Jira is still useful as a decision store. It may as well be a custom app over Postgres, but it seems like something is needed to store this and for observability. A GH/Linear/Jira is nice if only because of standard APIs and integration points (whatever you build would likely end up duplicating a subset of those).
simonedepertis 15 hours ago [-]
[flagged]
theYipster 14 hours ago [-]
Linear is a key part of my prompting technique. I write specifications in Linear, and the history I generate in it becomes a critical source of context for my agents. Prompting my agents has become “look at XYZ-124 in Linear and ask me any questions you have.”
10 hours ago [-]
yieldcrv 12 hours ago [-]
> Despite our test suites almost quadrupling since the start of the year
I think the SDLC needs to be re-evaluated, tests, specifically unit tests, are just cosmetics. "wow its so easy to get high coverage now!" as opposed to why we're doing it in the first place
I don't find agents to be using tests any differently than a junior or mid level developer. I don't find humans to be using tests any differently than a junior or mid level developer either.
Basically, the tests are never guiding the features, they aren't highlighting regressions, the test is simply modified for the updated application. Unit tests especially.
I have found end to end tests to be useful, and also unblocked. They can tell when the actual user experience has changed, and go back to the feature's implementation to stop altering the user experience in unexpected ways.
additionally, I find this across every industry that was worried about AI. basically, whatever best practice was neglected due to lack of investment into your org is now being done without additional investment into your org. that's a good thing. now we need to look at the purpose of the best practice, and if that purpose itself was already solved in the process.
alexnewman 16 hours ago [-]
My recent winnowwallet.com still has build times of over 10 minutes. I'm convinced the agent really wants build times around 5m to move at a quick pace. Also to not drive me insane. It took a ton of work to get it down from 45 minutes because my tests launch a full version of the app, and walk it through major usecases while recording video and screenshots. I then use AI to qa this. It also generates its webpage this way. All from CI/CD
aashishverma112 6 minutes ago [-]
[flagged]
nirmeetimthebes 4 hours ago [-]
[flagged]
aidiscoverywire 6 hours ago [-]
[flagged]
skyzhao1223 6 hours ago [-]
[flagged]
lhk931122 9 hours ago [-]
[dead]
hrpnk 15 hours ago [-]
[dead]
kevinbaiv 14 hours ago [-]
[flagged]
15 hours ago [-]
kittikitti 14 hours ago [-]
I cannot believe we are in 2026 and CI/CD hasn't evolved enough to even consider hot updates. Burn your CI/CD pipelines to the ground and start over again without any of the slop. Don't let anyone who calls themselves DevOps Engineers design it again, only people who are System Administrators.
dan_manges 15 hours ago [-]
(Disclosure: I'm a cofounder of RWX) For anybody wanting to solve similar problems and considering Bazel, take a look RWX. It's built around the same concepts of content-based caching and graph-based task execution, but it's far more runtime agnostic and easier to adopt. https://rwx.com
xracy 16 hours ago [-]
Maybe I'm alone in this, but as someone who is in tech, I don't know what CI is, and I don't think it's unreasonable to expect it to have an expansion within the article the first time you use it...
AI is in the cultural zeitgeist, but you gotta expand most other things at least once.
handfuloflight 16 hours ago [-]
So if you don't know what CI is, what does "I am someone in tech", mean? Marques Brownlee probably doesn't know CI is, but he's also in tech. Because if you by mean "I am in tech" that "I am a software engineer", and you don't know what CI is, then boy; I'd be worried for you.
Retric 15 hours ago [-]
CI is really only significant on the programming side of things. Networking, systems administration, hardware, etc often don’t have that workflow.
Asmod4n 15 hours ago [-]
sysadmins CI is hoping the machine still boots after an update. Unless you are lucky you got two identical machines to have a test env.
to11mtm 14 hours ago [-]
Also the case of 'learning that someone did updates directly on AWS console instead of terraform and losing 0.5 or more days cleaning up the resulting mess'
esafak 15 hours ago [-]
I see you have not discovered infrastructure-as-code (IaC). We have CI for our infra.
Retric 12 hours ago [-]
I’m familiar, the issue is IaC only works on a subset of system administration jobs.
Maintaining systems in a tunnel boring machine is a very different than an internet startup.
flohofwoe 5 hours ago [-]
I mean, "CI" is a complete misnomer. There is usually no "integration" happening while a CI pipeline runs, it's mostly just running a couple of tests.
xracy 13 hours ago [-]
I am a Site Reliability Engineer, I have been for close to a decade, and this is not as common an abbreviation as you think it is. Also, my entire job isn't in question because I didn't memorize shorthand for a term that I don't use in my day-to-day.
Really appreciate getting downvoted to hell for saying "We can't throw readers a simple bone?"
All I'm asking for is CI(Continuous Integration) in the first paragraph.
I didn't say I don't know what Continuous Integration was. I said I didn't know what "CI" stood for.
Cannot stress enough, this is like, basic communication/writing that they teach in elementary school.
But I'm so happy you're able to get off on your feelings of superiority that you always automatically know what those two letters stand for. I hope that translates to meaningful happiness and fulfillment in your life.
handfuloflight 13 hours ago [-]
If expecting more from others is what gives someone a sense of superiority, then what is that you're doing with your expectations?
Next time, Google it or ask ChatGPT. That's what elementary kids do when they don't know something.
Jcampuzano2 15 hours ago [-]
"AI is in the cultural zeitgeist"
Within the engineering tech sphere, CI/CD have also been terms that have been standard for at least a decade now.
You would probably be rejected from most interviews at the first stage if you didn't vaguely know what they mean at this point.
Linears entire product is tailored towards software engineers/engineers in general or people who work alongside engineers, so its not surprise their posts have a bit of assumed knowledge.
xracy 13 hours ago [-]
I know what a CD (Compact Disk) is.
(Continuous Integration, and Continuous Deployment, are fine if someone tells me what the letters literally stand for, but sorry for not being so tech-brained that I haven't memorized every possible TLA that's ever been used in tech).
halfcat 11 hours ago [-]
> haven’t memorized every possible TLA
TLA or TLA+?
bognition 16 hours ago [-]
CI -> Continuous Integration
Its best thought of as the testing systems that are run as part of pull request review / merge to main / build processes.
xracy 13 hours ago [-]
Thank you
ad_fontes 15 hours ago [-]
Man, what a weird thing to complain about.
Most days I feel like I must be the dumbest person on HN. I don't understand what 80%+ of submissions are about. But if it sounds interesting, I'll dig into it a bit and learn a few things along the way.
15 hours ago [-]
madrox 16 hours ago [-]
I recommend making a habit of googling or asking AI about terms you run into. As someone who is in tech, you will hear tech jargon your whole career that will not be explained if it's considered standard terminology for the audience being spoken to. It is better to learn to educate yourself than expect others to go out of their way.
xracy 13 hours ago [-]
It's wild to me that people think asking for
"Continuous Integration (CI)"
in the first line of a paragraph of a long article is "going out of your way".
It used to be the standard expectation for using shorthand.
madrox 11 hours ago [-]
It used to be a standard expectation, but that was before google or AI could have answered your question in less time and typing that your complaint on HN required. Especially when the author is writing a blog post for an audience that knows what CI is.
Do you want to be right, or do you want to learn?
ben0x539 8 hours ago [-]
I don't think it would occur to me to write out Continuous Integration because it's basically a nonsense term if you don't know what it means anyway, it's not any less opaque than just CI. The problem isn't the shorthand here, but yeah it's probably good to spell it out anyway so when someone googles it they have more confidence they've found the right thing.
Everyone's going so fast that they keep hitting walls. Review, CI, product asking for things, whatever.
Why have we not seen an improvements in products?
While every post and thread feels like a 90's wall street office, the new android and iphone ship with fewer features than usual. No indie guys come up with a linux-sized alternative OS. Switch 2 remains unhacked. Windows takes 3 seconds to show the right click menu.
Is everyone just running full speed in circles or something?
Our QA, formerly a fairly frequent blocker of all our releases, are doing more in-depth reviews and catching issues earlier in our release process. They have become unblocked to the point they are actively chasing down work that starts to slip.
We have cleaned up and tuned both our security alerts and operations logs and improved our tenant isolation in our service in a way that makes customer and formal audits SIGNIFICANTLY easier.
We're setting ourselves up for faster human development of the hard-things. Our development environment and infrastructure are faster, cleaner, more auditable processes, and cheaper overall to operate.
These fixes mostly don't show up in our product change logs, and definitely don't fall into "new features". It would largely be invisible to the outside world, but our costs are going down (though to be fair, not offsetting the spend on AI to date), internal productivity has improved, operational incidents are down, and customer satisfaction is up.
Is that a fair summary?
> a direct and meaningful impact on the quality and availability of our services.
You:
> user-facing delivery is sameish
I don’t see how you can think that is a fair summary or a good-faith argument.
Engineers are happier because they can spend more time on that extra round of polish that was just filed into low-priority "fix when time" Linear tickets.
QA is happier because the stupid bulk operations are handled with AI and they can focus on the hard to find stuff that tickles the QA mind in a special way =)
Same stuff, but with better (in-house) tooling and (most) people are happier. The Code Artesans aren't, but it's mostly their problem and they'll find companies that only do fully manual programming for specific niches.
Not op but no.
If you are product-driven, life is pretty good, and maybe has never been better? (For the moment)
It has certainly been better for junior devs, mid devs, anyone looking for zirp era salary growth and supply demand ratios. For a senior code-adverse coaster looking forward to see his static salary get eaten by inflation, life has indeed never been better.
From that POV, has it ever been better, assuming you can rise to the top of an ever more crowded market?
Building a product is easier now, same way making a photograph is easier now. Making a reliable and high income living out of it though, it looks increasingly uncertain.
In a world with abundant music and photography, just saying that you made your music using Reason/ProTools or you made your photograph with a Leica isn't enough. I believe the same will happen with software. Sales, product design, customer service and operational aspects like uptime will matter more than the software itself. Ironically, this is the kind of stuff the HN type highly dislikes.
Agreed. It's not even a new thing. They tried to teach me/us long ago, starting by putting it into words we might more easily understand, such as "Customer Development" lol.
When I realized that it was the sales guys that will have the power in the mid-term new world, I got genuinely upset. However, maybe they always had the power and I just didn't want to accept reality.
Now, I believe the most desired hire in the near future will be the PMM.
Until ~9 months ago, I spent most of my time being a "prompt engineer." Today I join a meeting, ten minutes later I receive the transcript. Then, I run a custom skill in my project, and I get Jira epics and stories to triage that are nearly perfect. Then, I run the second skill orchestration skill... some babysitting... and ~85% of the time that is all I need to do. Docs, code, unit and e2e, great UX... all there after every meeting, and basically two commands on my part.
I see two to three to maybe five years before anything I have to offer, in any capacity, is completely cut out of the picture.
I honestly don't understand how everyone is not on this same page. The labs are going to eat it all. The only reason I see for them to talk about "pausing," is because they finally realized that they are going to collapse the entire service economy around themselves at this rate: aka, the USA.
It’s actually rough today with astra and fable, it’s just not been diffused enough. Tech workers like us see the writing on the wall, but a lot of others are blissfully ignorant.
Everyone’s trying to figure out how to convert this speed to product features at scale, but enterprises are like container ships. Lots of might but slow to turn. The littler companies can actually take advantage of this and produce higher quality products at much faster speed. I think you’re expecting too much in the short term and too little in the long term. AI-native companies are gonna eat everyone’s lunch, once they figure out how to actually do it reliably.
There was a video by a layperson 2 years ago where they wanted to know the mechanics of some elusive pokemon pinball spawns where there was a whole bunch of missinformation about it online. The people behind that project had the correct addresses but they were completely unlabeled and the layman had to essentially figure out and do some of the work themselves to sort it out and figure it out. Nowadays that would never happen. An LLM will just do it for you within the day.
Also, are these even actual decompiles? Sure, playing Wheel of Fortune with a LLM model that at some undetermined point spits out what seems to be a working example is sorta useful, but also…how exactly do you debug this nonsense? Don’t tell me we’ll just have the LLM thrash around a bunch again. What I’m asking is does anyone understand what is going on here other than “lol, it works, shut up”?
Usually, green username are a sign of "leave it", I'll agree. But this one seems very informative.
I don’t doubt they help by randomly permuting around in a way that people cannot do at scale, but I question the economics of this approach and I think it needs to be used judiciously. Sometimes you just have to do that especially like you said, when you do not have the source or the code is intentionally obfuscated, this can be useful, but a lot of people seem like they’re just vibe coding crap and blowing money on tokens until they get a useful result and I have no idea what the value of that is.
Wind Waker in 4K w/official Bluetooth GameCube controller
(Nintendo went out of their way to invent a new protocol so that it didn’t “just work” like OG switch controllers)
Couldn't you say this about any company? What isn't just a matter of "figuring it out"?
Why can't they use AI to crack this nut? :D
> If you’re not familiar with normal emulator development time...
Emulators have historically been a shitshow until someone figures out missing pieces here and there. By their nature, that tide lifts all ships. It has very little to do with using AI. I'd argue the real reason emulators aren't what they used to be is the hardware is more complex now and the scene is no longer attracting the most talented devs. Video games used to be on the cutting edge and carried a lot more cultural weight. It's pretty underwhelming to get cred for working on lame x86 hardware that plays yet another franchise reboot.
For the question where are the alternative OSes? Here is one that I've seen. There's probably more - https://www.reddit.com/r/ClaudeAI/comments/1wfpydl/i_asked_c...
For that other stuff you mentioned like the right click menu. Those huge corporate projects suffer more from layers of institutional dysfunction and will be very very slow to show any improvement. Their dysfunction can't be solved with just faster coding.
Using AI to build more features is easier than using AI to improve existing projects. People will gradually figure out how to do latter too, it'll just take longer.
Do you have actual productive examples? As in, products with a real userbase that couldn't exist or be scaled pre-AI? Genuinely asking, I might have missed some large hits. The closest I can remember was bun rewrite kerfuffle, which seemed more a marketing action than anything.
If we sorted through all those new apps and ignored all the crap, I'm pretty sure we would find an overall increase in actually useful apps. I just shipped a new app myself, and I think it's useful, and I wouldn't have finished mine without AI assistance.
> products with a real userbase that couldn't exist or be scaled pre-AI?
Coding hasn't been the bottleneck for product creation for a while. So I don't think there are going to be many examples that fit that exact criteria. Any app idea was 'possible' before, it just took more developer hours to do it. So now teams are doing more things that were weren't worth the effort before, but are a lot more feasible with AI assistance.
I think you're probably right but at what point does it become diminishing returns? I have observed the app market to be overly saturated for years and rarely download something other than a mobile-banking update.
Where is the net benefit people keep talking about when they claim productivity gains are obvious because "more code faster, can't you see?"
Even in the 19th century, when electricity became widely available, there was no productivity gain for 30 years at least. This is a well understood phenomenon.
Google for “the Solow Paradox”.
https://www.standupeconomist.com/pdf/misc/solow-computer-pro...
It's an interesting question and very much not resolved. It indeed led to a flurry of studies in the 1990s, and more recently to several updates and meta-analyses.
The problem of "does computer technology investment causes increased productivity" is an interesting issue in economics and statistics. It is far from clear that the (immense) investment in computers over the past several decades has caused a corresponding excess growth in productivity.
Some of the literature published after 2015 I have found on this topic:
"Information technology (IT) productivity paradox in the 21st century"
> Thus we are still unable to confirm or reject the existence of an IT productivity paradox
https://doi.org/10.1108/IJPPM-12-2012-0129
-------
"Benchmarking the IT productivity paradox: Recent evidence from the manufacturing sector"
(This one was published in 2006 but I find it relevant because it does a very well scoped analysis in manufacturing firms thus addressing the oft-mentioned argument that computer technology may leverage task productivity in a way that is hard to measure in aggregate)
> However, many scholars from both sides of the IT paradox debate agree that difficulty still exists in specifying how to assess the IT contribution, and the availability of reliable data sets
> Regardless of the final decision to differentiate or conform, our results make a compelling argument that more spending does not necessarily mean better IT productivity.
https://doi.org/10.1016/j.mcm.2004.12.012
-------
"Lessons from three decades of IT productivity research: towards a better understanding of IT‑induced productivity effects"
This is one of the most inclined to disagree with the existence of the paradox, and still very cautious in the language used for writing the conclusion, e.g.:
> But to not at least consider the ongoing technological change as an important determinant of the deceleration in productivity growth seems ill-advised."
https://doi.org/10.1007/s11301-019-00173-6
-------
The Productivity Paradox: A Meta-Analysis
This I'm quoting from the submitted manuscript. I haven't gotten around to reading the published version yet, but:
> Since the size of the effect helps make the right decision in business-related investments, our result of ICT elasticity being very close to zero with values, about 0.3% for productivity and no effect on profitability, supports the argument that there are better forms of investment to be made
https://doi.org/10.1016/j.infoecopol.2016.11.003
https://www.stlouisfed.org/on-the-economy/2025/nov/state-gen...
This is just ~2 - 4 years after ChatGPT launched, and despite very shallow adoption (only ~6% of all work hours.) As a sibling comment indicates, it took almost 2 decades for the Computer Revolution to be visible in national level statistics.
Also note this study was originally published in 2024, then revised in 2025, but this preliminary evidence has been around for a while, if people wanted to find it. It's even been posted to HN a couple of times, somehow it just doesn't get the attention you'd think it should get, even if it was just to poke holes in the conclusions.
These long term data suggest that this "excess" remains bellow historical productivity growth
https://www.bls.gov/productivity/
https://www.bls.gov/productivity/images/pfei.png
The analysis bellow is a better support to your position on the side of "computer technology investment caused a delayed excess growth in productivity"
https://www.frbsf.org/research-and-insights/publications/eco...
To be clear: I do not take a position. I think this is an open question, a very important one, and I am not fully convinced that the exponential growth in the investment on computer technology over the past 50 years has led to a corresponding gain in productivity, nor that it is entirely a drag and a mechanism for increasing firm size and driving asymmetric profitability concentrated in ever fewer firms as the increased concentration in the capitalization of American stock market index composition would indicate.
That said, the strongest case I have seen for the position that we are beginning to see these delayed gains is the letter I linked above, and it still takes care to conclude:
> As more data become available, it will be important to continue to monitor whether current patterns represent the early stages of a new era of booming productivity or merely a temporary uptick in an otherwise slow-growth environment.
Because I have seen plenty of analysis published in the past 30 years that puzzle over the productivity growth stagnation in the face of an exponential growth in capital expenditure in computer technology.
So by all means share with me some of the groundbreaking results showing that we can finally see more productivity growth than would be otherwise expected by the conjecture.
It's still unclear if task-level productivity gains bubble up, but it's also still early and I'm not sure we should expect to see immediate results there.
Yes that is in essence the crux of the problem
I don’t think this is happening, at least not on any scale that makes any of this actually useful. One off success stories do not make a revolution.
It absolutely has generated a massive amount of slop, though. The problem is that you still need intent and while I think LLMs can help you free up and push through annoying boilerplate or tedious spots where known solutions exist, you still need to design these things and that’s something I’ve not seen an LLM be too helpful with.
App store revenue actually decreased! That suggests that the apps are largely not useful, unless you make the implausible assumption that nearly none of them are charging money.
https://www.ft.com/content/e5533f32-e4c2-4ce1-9d2e-a831e9654...
If.
The labour of finding better goods has become harder with more content showing up.
The other day someone posted a reverse engineered GPU driver for their Mac[0].
Anyway, what makes you think those pushes aren't individuals meeting their own needs? I expect it will take a while for this to really sink in, but the future is people asking the computer to build exactly the app they want.
[0] https://news.ycombinator.com/item?id=49717638
Most of those might be not as good as fan translations from dedicated people with good knowledge of English and the original language (especially for very context-dependent languages like Japanese) but they are definitely good enough to follow the story and play through the game.
I think ai has raised the floor quite heavily on what it takes to be a large hit.
all those vibecoded pushes are people building bits and bobs and variations on each other, instead of buying it or using a common service. If it could be done by ai, it will be written off as unimportant.
the userbase of the future is 1, maybe 10
The original sales pitch from the major player in this technology hype cycle is "cure cancer, fix global warming, take over the economy". Not "more LOC".
We were fortunate to already have a strong end-to-end test suite written in Python, so we could run the new application against the same tests and verify that the existing functionality was preserved. QA found around 20 bugs, which we fixed pretty quickly before launching.
After the migration, we also moved the application's authentication from Shibboleth to Entra OAuth and deployed it to our OpenShift infrastructure. We couldn't do that with the old WebForms application because our cluster doesn't have Windows worker nodes, so the legacy app had been stuck running on VMs. Getting it onto OpenShift gave us another operational and cost-saving benefit beyond simply modernizing the codebase.
I did this back in January, when the models finally became capable enough for this kind of work. I believe I used GPT-5.2 through Codex. Successfully completing this project is what finally got me fully on the AI bandwagon. I had used AI before, but mostly for smaller tasks like writing code, refactoring individual methods, or making isolated changes.
The application is now modernized, more stable, faster, and more functional than it was before.
I think the project worked as well as it did for two important reasons. First, I had deep domain expertise in both the application and its surrounding systems because I was the original developer. Second, our QA team's test suite was comprehensive enough to validate the functionality that actually mattered. AI dramatically accelerated the work, but there still needed to be someone who understood what the application was supposed to do and a reliable way to verify that the new implementation behaved correctly.
None of this means we couldn't have done the migration without AI. We absolutely could have. The difference is that AI made it possible to do it at practically zero cost in terms of engineering time and money compared with the alternatives.
We had wanted to move away from WebForms for years, but with the size of our team and the constant stream of new feature requests, there was never a realistic opportunity to stop development for months and focus on a rewrite. I estimate that doing the migration myself without AI would have taken at least six months to do properly. The other option would have been hiring a contractor, which I would estimate at $80,000 or more over six to nine months.
Based on my token usage, those roughly 48 hours of working with Codex to port the application cost about $150. Those were January prices, so I don't know what the equivalent cost would be today.
For me, that was the project that changed AI from something useful for assisting with individual coding tasks into something I saw as capable of fundamentally changing the economics of software engineering work.
You can move the goalposts to "yeah but it's not any good" but 90% of everything is crap anyway, and that was true long before AI.
What you're missing is that slop has increased 20x but quality new releases have also increased 5x. Unless there's something in your niche though you'll mostly see the 80% slop so I get why you feel that way
The actual bottom line result is very fuzzy. Cost are shifting, by spending less on agencies but in return opening up new positions or shuffling people internally.
We don't see a clear increase in revenue, nor can we accurately tie retaining revenue to our AI initiatives.
Workplace satisfaction scores remain similar, with a few outliers (e.g. some people's work got incredibly exciting, while a few others are terrified by all the change).
Time will tell if this all leads to anything meaningful for the business.
And that's just the people operating in public instead of private :P
And a few months ago I went through a random sample of new apps on the App Store and the vast majority were just wrappers for AI APIs. So it was more of a new gold rush situation than a productivity bump.
That's exactly the opposite of what I want to see. As an app store user, I want to see fewer, higher quality apps, not more shovelware or slop.
Your prejudice is showing.
People mean where are the good new apps AI made possible. Of course it has a 84% year-over-year slop app submission.
Any real "killer apps" though?
Scale in headcount was a tactic to get investment, then you had to find stuff for everyone to work on. Suddenly people have the time to engineer so hard that we get runtime JSON defined CSS rendering engines to produce the same buttons we've had since 1995, instead of just writing a stylesheet and html.
Code output velocity from AI threatens to be useful, except it's also prone to over-engineering and burning tokens on the unnecessary. It's learned from the best after all. My suspicion is there is a lot getting done, but it's just not that impactful to flagship products.
As others have noted, there is a lot of new work going into passion projects that would have never happened otherwise, and that is cool. But I wouldn't hold my breath for SV tech to become super pragmatic and effective.
they're working on the ad tech business and other business-related systems involved including internal tools.
and all the other platform engineering shit under the hood that makes much of the web scaleable... and lots of it is open source and contributed to by various engineers from these companies.
end of the day, these are businesses. they're not charities or whatever casual shit.
nobody is stopping you from building a competing product that's lean or whatever.
why don't you do something like that? i'm sure you're a genius.
It's probably changing now as VC cash floods into AI instead of web app startups, but I am not talking out of my ass, it's a well known phenomenon.
The industry does not incentivise or reward lean software, the software input for a lot of "software" companies doesn't require it, that's fine. Exactly as you said they're businesses, they do what makes money and go where incentives take them. That cuts both ways, they are not disincentived to have a lot of wasted dev hours, at least not historically.
The simplest explanation is that they don't give a flying flamingo about what you or I consider "improvements to products".
This report is an example.
There are several changes that modify CI behaviour, where the article gives no corresponding quality measurement.
They replaced type aware custom lint rules with AST-only static analysis. They don't say anything about what those new rules detect, didn't do old-vs-new rule comparison. They switched the TypeScript check from tsc to tsgo. Again, they are very proud of the performance improvement, but don't seem to care about diagnostic equivalence. The list goes on. They don't even report pass/fail agreement between the old and the new CI. They have 4x more tests, but no idea whether this big test suite works any better than the smaller old one, or even whether it works at all.
The blind community is benefiting enormously from coding agents. Game accessibility mods for everything under the sun (the big names in the last month or two are Civ V and Witcher, although there's plenty more), accessible 3rd party clients for annoying sites, people's favorite speech synthesizers ported to platforms they never ran on natively (or just straight down turned into portable C), plenty of small but nifty utilities and apps.
This is because that community's needs are amenable to what AI can do. Accessibility work (on somebody else's product) is a lot of demotivating and extremely difficult reverse engineering drudge work with a verifiable success criterion, and this is what AI excels at. Large-scale software dev is all about judgment and taste, and here, AI is not doing so well.
You see the same things with mathematics versus medicine. In math, the bottleneck is basically human attention, formal proofs in Lean are, again, drudge work with verifiable success. In medicine, the bottleneck is patients, paperwork and the lab environment, so even an omniscient LLM without the ability to pour fluid into a beaker wouldn't be that much of a productivity improvement.
All the companies have the same access to AI, and AI is making them all better at doing what they were doing before (writing software). So some companies that use AI really well may be able to take market share from other companies that are slow. But I think this might just lead to a more intense competition for customers.
Kinda like what happened to music after digital recording became the thing - we have more music than before and the music is better, but being a musician became a much more intense competition to find an audience.
Perhaps some new markets could be entered that weren't feasible before?
https://au.finance.yahoo.com/quote/NET/financials/
Interesting to look at, a decent example for sure.
I bet every company is finding up how to level up their employees via AI, so that they can use less of them in the future.
So even without increased revenue, AI has its (mis)uses.
Examples of things I’ve tackled this week:
- GitHub Actions costs exploding and looking into self hosting options (gonna try a box in Hetnzer)
- figuring out how to handle unattended builds
- tinkering a lot of email, from how to manage domain reputation and worrying about bots spamming from my server because they started filling out forms
- figuring out how to manage AI development from my phone
- setting up a process to prevent unintended destructive database migrations
- finding a way to know have to babysit the AI but approving a constant stream of questions and permission requests
- all the random little issues that the AI files to fix an endless amount of little things
- finding the right model(s) to use to most efficiently make use of the limits of my subscription
And so on.
If you apply AI to low-level applications, it is quite good and you avoid the runtime bottleneck of webness, I have found. Eg. https://github.com/Redrum624/Vitrine is very good but has a horrible UI due to the webby nature of its rendering - all vibe coded. If you used a native toolkit it'd feel completely different.
Now are the end users going to see the benefit and is the product magically better? Well no. The chicken answer is I am not involved in that side to know. But a more realistic answer is, user experience and product fit is not something LLMs can solve. That's still upto the humans to figure out and I think that's where this "nothing has improved" feeling comes from.
Code moves much faster, product taste doesn't. Once you have your marching orders you can make features 10x faster. Shots off target still miss though. It doesn't matter if your code gets generated 10x faster when it doesn't resonate with users.
Bootstrapping a solo software business has multiple core required skills. Technical skills are now largely unnecessary, but it turns out most people lack multiple of the other required skills too.
I built a debian based OS focused on self-hosting: github com/onmoose/os It's not really Linux size but I couldn't have built this if it wasn't for coding agents. Still a ton of work needed on UX but the main functionality works so well.
Linux also has decades of work on it, we can't really expect new products to equal that in 2 years of agentic coding
Yes, just rehashes of the same thing that was done before, in another language/framework/codebase/fork.
Doesn't matter where we're going, as long as we get there FAST!
It's a sickness.
A lot of people may be more empowered to create things now with less up front effort but it doesn’t lead to having better ideas or more actual system architects.
Have you? As in, can you name products you use daily that have been transformed night and day?
I am not talking about the existence of minor improvement, mind you. I am denying the existence of anything that is minimally consistent with the level of discourse that surrounds development itself (throwing away practices, checks and balances, hitting constant roadblocks due to the new speed, etc).
A true 100x requires no analysis. If we could suddenly build and improve houses at 100x I would know because I would be typing this from my 5th living room.
Using the custom interface instead save a lot of time.
I've been fixing everything that has been sitting there—not blocking us, but slowing us down—all the things we never had the bandwidth for. We're now squeezing more out of development, CI, and production.
I'm not sure why, but I'm concerned this may be the heyday, and we might not get this again at this price or speed. So I figured we should clear the backlog while we can still afford it and still have the ability to do so.
But now things have changed, and the bigger, grander ideas are starting to brew.
On the other hand, look at the YC companies. I think the last batch or two is growing revenue faster than any batches ever before. Those are the companies that are being sped up a huge amount by AI.
I don’t really think it has made Google products better, but it is absolutely the expectation.
It’s not night and day (as you’d expect from the hype), but I see some differences.
In what ways? When did previous releases stop meeting your criteria?
> There are open source projects now that tackle problems nobody has before.
Such as?
Ages ago. It was really noticeable when they prioritized features over quality.
> Such as?
HN is full of them. I don’t have great have specific example at the top of my head, but lots of hard reverse engineering/low level projects credit AI and that they would never have happened without it these days.
Wow. It actually does. I thought you were exaggerating.
I don't know if it's AI or not, and I haven't liked all the changes, but there have been more of them than I remember before.
the important thing to microsoft is still finding new ways to make money, not to make your clicks faster. they might even be using their new velocity to make clicks slower with an ad in between.
there's alao much less incentive to build things to share. people can just make their own widget app and not need yours
we literally do, we shipping feature faster than ever
What is Switch 2 security doing here? (independently of it having so few features added per update)
"Switch 2 remains unhacked" is rather disrepectful of the hacking community, and runs on the assumption that the Switch 2 has significant vulns. It's very well possible it doesn't have any high-privilege software exploit at all.
After all if one cas use AI to find vulns and/or to RE, it's even easier for the OS developer (Nintendo) to use it to find bugs in their code before release.
For a $2tn industry, YES.
The quality definitely improved, but our products were usable before, only not perfect. The improvement is mostly on the margin. Instead of crashing twice a week, the app crashes twice a month. A communication session won't fail in two hours, but maybe in two days. (But the average session in the real world was < 20 minutes anyway.) Some UI elements that were not necessary, but are nice to have.
Yes. The lack of actual tangible results is how you know that the claims of increased productivity are false. We haven't seen a bunch of new useful apps (or anything else for that matter), which we would have if LLMs actually worked.
On a serious note, it might take a while to realize the actual benefits or losses. Its clearly not a good signal when people whose job is to manage other people start writing their own pet AI projects. This just indicates that AI has created this big job insecurity among everyone. At the end of the day being an engineer, eventually the job is relatively safer when there is so much code being written out there.
Ours is moving faster than ever in terms of feature delivery, and we've used AI to really hammer at the security aspects and clear a load of backlog stuff.
But actually coming up with, testing, and rolling out loved new user features. AI isn't that good at that and we can't really "prompt" that out of it as easily as we can prompt a py2 to py3 conversion of old code
The idea is to drive actual testing from this, but in this era, I think it's interesting as a way to use AI to generate tests, and to cross-check those tests with the natural language descriptions, in a bit of a cycle that helps refine the highest level definition of the software.
Once that's nailed down, the implementation is just details.. normal engineering concerns like maintainability etc notwithstanding of course, but you can trust more and more the AI agents to get it right. The design specs being natural enough for humans to deal with but interpretable/specific enough to actually generate tests is pretty interesting for the bottleneck you are talking about, I think.
QA is about quality. That it has been increasingly used to mean "repetitive manual testing only" is part of the very trend you decry of qa being undervalued.
Fundamentally QA is about two groups of people collaborating in an adversarial process ("you build it we break it") to achieve a common goal: a better product. QA by definition requires a relationship of equals, or the process ceases to be adversarial and becomes useless ceremony. If you're not able to assemble two distinct groups of people, one person can wear both hats (ie you test your own software) but you'll have more blind spots.
How much of QA is done by a human or automated, is and has always been an implementation detail.
If you're in charge of QA, you're in charge not just of finding new problems, but preventing regressions as well. And you're responsible for embedding as much of that work into the devloop itself, so that builders can find and remediate problems earlier and with less pressure on your limited time. How do you achieve that? automation.
I started my career doing QA and 99% of what I was doing could be automated by an AI today. Today I still do QA on my own product- I just spend all my time on the remaining 1% and my product is better as a result.
What exactly are you using to back up this claim?
> QA reviews [...] fully automated
At some point, with all this velocity, human users become the bottleneck, unable to keep up with and learn all the changes and new features. Luckily, there's a simple solution: just replace the human users with agentic AI users.
> Henry Ford II: Walter, how are you going to get those robots to pay your union dues?
> Walter Reuther: Henry, how are you going to get them to buy your cars?
https://quoteinvestigator.com/2011/11/16/robots-buy-cars/
Because every developer is now a slop cannon by default, by default users will experience churn and whiplash, and things will break all over the place. As you point out, this is bad. It's also impossible to fix without deploying agents on the QA side. Like it or hate it, agentic testing is inevitable to protect users from the churn and noise caused by the slop cannon. I don't think that replaces test engineers at all - if anything it makes the job more fun. If you've ever had to keep playwright tests in sync with the target manually, and kept the CI environment up to speed with toolchain changes, you know what I mean.
Whether the "slop cannon by default" situation could have been avoided in the first place, is another question... But we're here now and there's no going back. Might as well deal with it as best as we can.
TLDR: it's not all bad :)
If you even review PRs still: when was the last time you didn't just skip over tests? And if you ever looked at tests in an LLM-heavy PR, how many of those tests tested something useful, and not just built-ins and trivial behaviours?
There's at least some awareness in the industry of how LLMs generate a lot of boilerplate in business logic. It feels like we're much less aware of how much of it is in tests.
The opposite. Reviewing PRs is now about just reviewing the tests, as the code is very likely to be fine if tests are relevant and they're passing.
The main enabler of agentic coding is heavy end-to-end/characterisation testing.
We can ask AI to do very large-scale refactors, for example, because if tests pass, it's 99% that everything is fine.
It's a practice, it's not the best practice. It depends on the team/org/codebase/feature/etc. It always costs time and money and effort. Lots of it from multiple people.
You can get much better output by shifting that cost into hiring much better professionals, not better developers, but overall professionals.
The kind of people you can blindly trust that the software they are writing will be good, you don't need to get involved.
Of course there are exceptions. The author may actually want a review. Or the piece of code might be touching something extremely critical to the business but also easy to get hard.
But besides that? PRs are just productivity porn, or "we do engineering right because we follow Twitter" porn.
The best performing teams I had you hired individuals that removed work and responsibilities off your shoulders without you ever having to regret it. Never added it.
I laugh off engineers that "no you have to review, because it spreads information, enhances quality" and yada yada yada, while in the real world way more critical decisions are made by a single individual without requiring somebody reviewing their work.
Yeah, was not surprised to read this. Actions is convenient if you already use GitHub, but it can also be pretty slow. Given reliability is also a major issue with GitHub these days I expect to see more orgs moving to different pipelines
Because they're running away from on-premise and the associated Ops teams.
I don't think it's out of malice, but I do feel uncomfortable with the fact that the people who sell me the CI coordination software also sell me the minutes for the boxes that run the CI software. There's _some_ alignment of interests but not as much as I would want!
Of course you can increase the duration, but if something is supposed to take less than 10 minutes but takes over half an hour, increasing limits won't really address the problem at hand.
Not saying that they're shooting for either of those segments, but someone should. Plenty of enterprise orgs that need (or are convinced they need) Jira's featureset.
Linear is essentially a small subset of Jira.
It's liked and works because it's simpler and covers most of the needs of teams without huge need of defining their own processes.
But be in business long enough and you find out that appealing to small teams and small orgs that don't require flexible and custom process definition doesn't make enough money and doesn't grow you forever.
So they will keep eventually growing in features and flexibility till they cover the reason Jira is so successful: it is more powerful and adaptable.
I'm no fan of Jira, but I've tried and seen enough of these tools to understand why it has the success that it has.
This is even more clear when you go beyond software development and need processes for teams like sales, HR, legal, etc. Jira can fit all of them.
Linear? It's a joke.
[0] https://linear.app/now/sharing-growth-with-the-people-buildi...
In my experience, self hosting your CI/CD runners had the biggest impact in cost savings throughout, while also allowing for more powerful machines, which directly means quicker CI/CD runs
For a less complex project (1 programming language, still shipping to all 3 major OSes), with my knowledge and agents I got the bazel conversion done in 2 weeks.
The setup cost for bazel just went down by a lot, and I don't think the industry as a whole is aware of that yet.
Tsgo, oxlint, caching dependencies etc. what linear outlined in their blog post would be more impactful for the average TS project I've worked on.
Even fully cached outputs needs to fetched and read from a remote server[1]. A step n-1 outout fetched from remote cache server need to written to disk and then again read by step n[3] - all disk I/O and network bound operations.
10s may be achievable/realistic goal in the Java/C++ world where Bazel normally seen. In TS eco-system most people would be over the moon to get into ballpark of 1-2m for a decently large monorepo.
We should define Build more clearly here, if you mean running just transpile/compile steps or the full series of steps that includes tests (as the linear post here is talking about). It is hard to see even a small sub-set of a large suite of test that require a virtual DOM or a real browser can run in 10s or less.
[1] Typical for say managed CI setup .
[3] Common run-of-the-mill frontend + backend stacks in different languages etc.
Also, if you have persistent CI workers with a persistent bazel instance, you save on some network roundtrips, but that's obviously harder to set up and make bulletproof.
The final asset/artifact is rarely small either. even best optimized artifacts can be few hundred MB docker image or more commonly multiple image layers running GBs in size .
each step is a network pull then recompute cache if stale and keep going till end .
For the bigger final artifacts, we support using Content Defined Chunking (rolling gear hashing) to only fetch the missing chunks between incremental builds. Binaries executable with stable layout benefits from this quite a lot.
We are definitely not done with all of the improvements here. But since all the major AI labs are using Bazel, we know that the tools can support “Agent Scale”. https://webazel.dev/
Not sure how that would work with building say a docker image, reproducible builds are pretty hard problem to solve, and caching intermediate layers is not always simple or even doable, we typically still need to publish to a registry which is not the cache server.
https://www.youtube.com/watch?v=biYXmAv4Ppk&t=314s should be a good talk to study up on the matter. The speaker is now working at Apple.
Plus, 'with a warm cache' is doing heavy lifting, what's the real cache hit rate for a week of development? Investing in improving the cold build and frequent actions is still important with bazel or any incremental builder.
I'm not sure it's useful to talk about bazel broadly, it's actual performance and behavior comes down to the rules you use. You can configure bazel like turbo/nx and cache tsc/vitest/eslint on each package.json module, and get course cached units that are evicted on every change, or you can use gazelle and target per-file actions which are only invalidated when their dependencies change. But that trades off batching unless you use workers.
And yes, there are a ton of investments going toward Bazel recently to unlock these newer use cases.
I ran an experiment where I migrated a package to bazel then replayed a weeks worth of changes and it saved 20%. That's nothing to scoff at, but not the headline numbers you see after a full hot build.
Migrated actions to a spare M2 MacBook Air at home, and when offline, uses hosted Actions as a backup. Bonus: has simulators/emulators for ease of running Maestro tests for mobile QA.
This is interesting. If I read that correctly this means the CI host had a direct peering connection with Github or Azure network.
Why were they not able solve the issue?
the blog mentions that they use a local cache to "reduce the time" but the problem is not solved right?
I have been a long time proponent of monoliths, but it seems like agents would be happier with smaller, more isolated services. The more isolated, the better. Contracts between the service components only. Then it can iterate internally as long as it satisfies the contract. If it needs to, it can version the contract and keep iterating.
However, our world is not that world. Your agents may be happy in their tiny walled kingdom of toil and ineffectiveness, but you and your users will not. Poorly drawn service boundaries will drag you down more than almost any other architectural mistake.
If there is one universal amongst organizations, its that they love walls and silos. Be wary of putting them up ahead of time, cuz tearing them down once established is nearly impossible.
Anyways great blog post from linear team a lot to learn from it
Mostly around, at least the majority of 'stuff' I've worked on (both before and after the rise of coding agents) myself or others took enough time to make sure that anything done locally, if you run the tests locally, you're at least 90% of the way there as far as what CI/CD does.
I suppose the flipside being, most of those projects had less churn (i.e. one person was working on a service at a time, and we had good contracts between services.) Also, Our local boxes were way better than our CI boxes, so there was incentive to run locally versus waiting 2-10x the time for CI to run...
Did the tests produce four times as much value, though?
I gave a talk about how we do it at BuildBuddy at a recent BazelCon here: https://youtu.be/iQqLtuBzkKE?t=848
Edit: After second thought, I guess "alternative runner" providers still have to pay the self-hosted tax. So M$ actually saves and makes money by not scaling their infra and driving people to alternative providers they can tax freely. Actual geniuses.
Actions management plane is the most problematic part of the platform (in regards to scale), so they are basically losing money with self-hosted runners.
Scaling of VMs shouldn't be a problem, since that is practically stateless, whereas there are lots of operations performed for every job/workflow.
Why not? A good chunk of software engineering problems can be solved if we re-structured our definitions. Even just a year ago, people did not believe that a good chunk of prod-running code would be completely AI generated. Now we're at the point where some companies try to split "what code has to be PR-reviewed, and what doesn't have to be" to remove that bottleneck and so on.
And this includes _full_ linting and unit tests.
I did that by self-hosting Github Runners, using Podman to build images (it doesn't squirt your build context through a socket every time!), and using Docker images as content-addressable cache.
I will do a write-up about it this week...
CI should handle only stuff which cannot be run locally due to resource or setup reasons.
Also CI should only receive pre-validated change candidates with passed checks inside the agent loop.
Genuine open question here. My assumption is that a GH or Linear or Jira is still useful as a decision store. It may as well be a custom app over Postgres, but it seems like something is needed to store this and for observability. A GH/Linear/Jira is nice if only because of standard APIs and integration points (whatever you build would likely end up duplicating a subset of those).
I think the SDLC needs to be re-evaluated, tests, specifically unit tests, are just cosmetics. "wow its so easy to get high coverage now!" as opposed to why we're doing it in the first place
I don't find agents to be using tests any differently than a junior or mid level developer. I don't find humans to be using tests any differently than a junior or mid level developer either.
Basically, the tests are never guiding the features, they aren't highlighting regressions, the test is simply modified for the updated application. Unit tests especially.
I have found end to end tests to be useful, and also unblocked. They can tell when the actual user experience has changed, and go back to the feature's implementation to stop altering the user experience in unexpected ways.
additionally, I find this across every industry that was worried about AI. basically, whatever best practice was neglected due to lack of investment into your org is now being done without additional investment into your org. that's a good thing. now we need to look at the purpose of the best practice, and if that purpose itself was already solved in the process.
AI is in the cultural zeitgeist, but you gotta expand most other things at least once.
Maintaining systems in a tunnel boring machine is a very different than an internet startup.
Really appreciate getting downvoted to hell for saying "We can't throw readers a simple bone?"
All I'm asking for is CI(Continuous Integration) in the first paragraph.
Cannot stress enough, this is like, basic communication/writing that they teach in elementary school.
But I'm so happy you're able to get off on your feelings of superiority that you always automatically know what those two letters stand for. I hope that translates to meaningful happiness and fulfillment in your life.
Next time, Google it or ask ChatGPT. That's what elementary kids do when they don't know something.
Within the engineering tech sphere, CI/CD have also been terms that have been standard for at least a decade now.
You would probably be rejected from most interviews at the first stage if you didn't vaguely know what they mean at this point.
Linears entire product is tailored towards software engineers/engineers in general or people who work alongside engineers, so its not surprise their posts have a bit of assumed knowledge.
(Continuous Integration, and Continuous Deployment, are fine if someone tells me what the letters literally stand for, but sorry for not being so tech-brained that I haven't memorized every possible TLA that's ever been used in tech).
TLA or TLA+?
Its best thought of as the testing systems that are run as part of pull request review / merge to main / build processes.
Most days I feel like I must be the dumbest person on HN. I don't understand what 80%+ of submissions are about. But if it sounds interesting, I'll dig into it a bit and learn a few things along the way.
"Continuous Integration (CI)"
in the first line of a paragraph of a long article is "going out of your way". It used to be the standard expectation for using shorthand.
Do you want to be right, or do you want to learn?