We’re not stagnating

Rewrite this without focussing so much on that shitty post. Write it as a positive thing.

This can probably be split into two articles - Software Isn’t Stagnating and Software Isn’t Bloated.

This post upset me a great deal. A short summary

• Look at a narrow subset of technology - the tech we use to create software
• We had great stuff before 1996.
• Nothing worthwhile or groundbreaking invented after 1996.

HN comments

This is absurd for a few reasons

1. All human progress is iterative. We like to think that progress happens when geniuses shut in for a few weeks or months and emerge with a brilliant invention or discovery. But we’re always standing on the shoulders of those who came before us. We credit James Watt for discovering the Steam Engine but really, what he invented was a more efficient condenser. Still groundbreaking, but he couldn’t have done it without building on the efforts of X, Y and Z. This doesn’t mean we trivialize every invention/discovery by saying “it’s just so-and-so but with a small change”. We recognise that every invention took a lot of effort from the inventor, but was probably an iterative improvement over the status quo.
2. The list of ground breaking technologies he mentions is also iterative. (Examples) C++ is “merely” C but with a few additional features from Y. Windows is “merely” DOS with a GUI inspired by the Mac. Just as it’s absurd to dismiss C++, it’s absurd to dismiss Wasm because it builds on an idea that previously existed. C++11, C++14, C++17 released in the previous decade all contributed in making C++ a better language. They all iterated on a popular, effective language to making it better.
3. The author is stunningly ignorant of actual improvements such as

• GPGPU programming, which allows some previously intractable problems to be solved quickly.
• Languages like Rust, which starts with a clean slate, is competitive in performance with C and C++, while eliminating classes of bugs that are responsible for 70% of bugs found in Chrome and Microsoft.
• In terms of developer productivity enabled by a library ecosystem and a package manager, npm, cargo and others would be like black magic to someone from 1996.
• LLVM, a compiler project that enables teams to create new programming languages without having to worry about supporting the quirks of different kinds of processor architectures or optimizing them. Swift and Rust are two notable new languages that rely on LLVM.
• A variety of distributed databases allowing data to be stored at scale. Each database makes a different trade off, allowing the selection of something that’s appropriate for the usecase. Need low latency and have a small dataset? Use Redis. Need Full Text Search? Use ElasticSearch. You’re a hipster? Use MongoDB.
• Version control became widespread. Before 1996 (the arbitrary year chosen by the author), it was used only in some projects and it cost money. With Git, Mercurial and others, it became free for anyone to use and widespread.
• Speaking of which, most of our developer tools are free and open source. Sometimes, they’re merely free or require a small license fee. Few people are being locked out of general programming because of cost (this is amazing!)
• CI/CD. Deploying every few hours has it’s pros and cons, but was hard 20 years ago.

But arguably the biggest miss is Linux and how it’s changed the computing industry. Since the OS is free, developers are free to spawn and destroy Linux VMs according to demand, instead of worrying about licensing. It’s hard to imagine how AWS/Azure/GCP and the rest could have existed without Linux. Yet the author doesn’t deign to mention it, probably dismissing it as just a “unix”.

So when do we rewrite?

Implicit in the criticism that software is bloated is that we should rewrite it.

If you listen to Joel Spolsky, the answer is “never”. I think the answer is “it depends”, even if that is vague and unsatisfying. I think these 4 conditions must be satisfied first.

• We understand the current system, including all the edge cases it currently handles. We must never rewrite because we don’t understand the current system. Understand first, rewrite later.
• The requirements have changed. Either we need to handle a bunch of new requirements that aren’t currently met, or the software handles requirements it no longer needs to.
• There is a technology that simplifies our architecture. For example, we might move maintaining consistency from our application to our database.
• Investing in a rewrite is something we can afford right now. In some cases, making interest payments on our tech debt might be less painful than not shipping features for a long time.

I posit that most modern software does not meet these 4 criteria and should not be rewritten.

The author also seems to want change for it’s own sake