dljfield.com

It’s hard to miss, this week, the furore over the event-stream package on NPM being compromised by a malicious actor, who was given control of the repository and subsequently snuck code into it to steal bitcoins.

NPM has a bit of a reputation at this point of being a mess (fairly or otherwise), and previous high-profile breakages in and attacks of JavaScript’s package ecosystem could be attributed, at least in part, to failures of NPM itself. Even a hack earlier this year that resulted in malicious code going out with the popular eslint package happened due to NPM credentials being stolen.

This latest issue is most similar to the case in March 2016 of a commonly used package being removed from NPM. It certainly did not go without comment that a function as simple as left-padding a string was not just a downloadable package, but a dependency so relied upon that it broke the entire JavaScript world. But, in my experience of it at the time, those comments were somewhat lost in the questions about why NPM allowed a package to be suddenly removed in the first place, and whether developers should have the right to do it with their own packages or not.

By the time October came around, and Facebook had released Yarn as an attempt to take over as much of the JavaScript infrastructure as possible improve JavaScript package management, questions about left-pad’s existence had pretty much ceased, and all focus was on technical missteps and whether NPM could survive (it did).

Yet, the existence of left-pad remains just as relevant today as it did back then.

Open Source Doesn’t Work That Way

When we think of “open source”, we generally think of projects like Linux, which are large and wide-reaching, with lots of users, lots of contributors and a set of well-trusted maintainers organising it all and keeping the code flowing. This setup is everything the ideal of open source development is supposed to be, and a model for healthy open source projects.

JavaScript has some projects matching this description of healthy open source. We might, for example, point to React as one that has lots of users, lots of contributors and a set of well-trusted maintainers keeping it all together.

But the majority of open source JavaScript doesn’t look like this at all.

The vast, vast majority of the projects up on NPM have a single maintainer and no other contributors. This isn’t unusual for an open source project, most of us will have projects that are technically open and yet are unused. The difference with JavaScript is that projects like this often are used. Heavily.

To go back to the hot button issue of the moment, event-stream has nearly two million weekly downloads, and is depended on by—at time of writing—1,539 other projects. And yet it is (or was) maintained by one person, and its contribution stats indicate that it was developed pretty much entirely by him as well.

This is bad enough, but what’s worse is that when you look at it, this one person is responsible for something like 700 other packages, and not all of those are exactly unused either.

If a healthy open source ecosystem has few projects, each maintained by multiple people, an unhealthy one has thousands of projects maintained by one. An unhealthy open source ecosystem is JavaScript.

JavaScript Did It To Itself

It is practically folklore at this point that JavaScript owes its life to ten probably manic days in the mid-nineties. Much ink has been spilled examining just how and why it has become the language powering arguably the majority of our everyday computing experience, responsible for untold billions of dollars made and lost, and just as much ink has been spilled discussing the merits of the language now that it is so popular.

However, whether JavaScript is a good or a bad language at this point is, in my opinion, a wholly moot point. Every major problem that has hit the JavaScript-using world is caused by the circumstances of its birth.

The reason we are faced with thousands upon thousands of small packages, depended upon by thousands upon thousands of other small packages, is because JavaScript lacks a major aspect of all more seriously constructed languages: a standard library.

Ignoring express and react, at the time of writing the following packages are the most depended upon according to NPM’s website:

• lodash
• request
• chalk
• async
• bluebird
• commander
• debug

Every single one of these packages, in any other major language (C++ perhaps aside), would be rendered obsolete by the standard library already including everything they do.

It doesn’t get any better as you go down the list. fs-extra stands out for describing itself as containing “methods that aren't included in the vanilla Node.js fs package. Such as mkdir -p, cp -r, and rm -rf”.

Seriously?

This lack of standard library opens up the gates for some rather perverse incentives. Because it is so easy to produce something that someone else might need to use, a time-strapped developer is incentivised to do just that. They’ll throw out many small utilities and libraries that can cover just about anything a person might need to do. This lets you collect GitHub stars that you can point to the next time you apply to a new position. Why would you get together with other people to put serious work into a major open source undertaking when you can get a new job and double your salary for letting someone left-pad their strings?

This gets combined with an audience that is increasingly comprised of very green developers, many with only months of experience and no formal training, who are both too new and incapable of producing the code themselves, and too ignorant of other languages to notice that nowhere else makes them install packages to do simple things.

And we probably can’t do anything about it.

Yeah, We’re Doomed

Obviously, the solution is to give JavaScript a standard library. Also obviously, this is almost certainly not going to happen.

Simply put, the hurdles in front of the goal are far too high, and the motivation to do anything about it is far too low. Even if the existing standards body could come to an agreement about what ought to go in a standard library, and what the API of that library should be (across both browsers and node), the vendors would almost certainly fail to produce it in full, given their track record with the existing language features.

This doesn’t even account for the fact that no existing code—of which there is a stupendous amount—makes use of the standard library, and to switch to doing so would require developers to target their code only to the most current browsers, which they won’t because doing so would break the whole web for millions of people.

And let’s be honest: a ton of JavaScript developers think the current system is a positive aspect of the language. Whether they argue that in real good faith or just for their own benefit, who knows?

Perhaps a more realistic solution here is to instead produce a single package that acts as a standard library. It loses the major advantage of being always available to every JS developer, but at the very least if it does take off it is far more likely to be a healthy project than what we have now.

I doubt such a thing will happen, though.

My own opinion is that JavaScript is far too far down its current path to ever correct course. It will rocket along as-is, occasionally patching up the issues that spring up from its fundamental flaw, until it burns out as a new generation of developers moves onto whatever it is that becomes the most popular language compiled to web assembly.

Which I suppose could be JavaScript.