JS dependencies: The Problem

This post is split into 2 parts aiming to answer the question:

How do I use library “xyz”?

Using JS dependencies is by far the most common problem for CLJS users. Many answers exist and the topic got a whole lot more complicated recently. No solution is perfect and pretty much all of them require some sort of manual tuning. In the second post I’ll go over how I have chosen to tackle this problem within shadow-cljs.

This post will focus on defining the problem and describing the current solutions that people use today.

The Problem

To run code in the browser we want to optimize the code as much as possible. This means that we want to reduce the number of requests the browser has to do to a minimum and cut all unnecessary code out of our build. For CLJS this is done by the Closure Compiler which yields very good results but requires that code is written in a certain way which pretty much no one likes writing by hand.

Luckily the CLJS compiler can do that for us and we can build on top of the extensive Closure Library. Beyond that we had to resort to all kind of trickery described below to use “other” JS dependencies that are not part of this world and are written for the more generic npm world.

CLJSJS

CLJSJS is a community effort to pre-bundle most npm packages so they can be consumed by the CLJS compiler. This is by far the easiest option but has certain “scaling” issues. Not all npm packages are available and sometimes packages contain duplicate code since they were bundle separately leading to bloated file sizes or even very hard to track bugs since you may end up with multiple versions of the same library loaded on the page.

If you just want "react" this solution works perfectly fine but it gets hairy quick with every additional dependency. One example would be the material-ui package which ships with its own version of "react", meaning that you have to manually exclude all other versions other libraries might be using.

Since CLJSJS packages typically bundle :externs the user often doesn’t have to worry about providing these. BUT the :externs are often generated and overly generic which might actually hurt your file size in bigger projects.

“Keep it separate”

The compromise is to manually collect all JS dependecies and bundle them together using tools like webpack and then just including the file separately (or via the :foreign-libs compiler option). You do need :externs for this and code-splitting gets extra complicated but this solution is very reliable when using many different npm packages that you just bundle up into one big blob.

The problem here is you’ll access the bundled JS code via certain global variables and the compiler can’t do anything to help. You also need to setup the other build setup which usually is not very easy.

Closure Compiler

Until recently the Closure Compiler did not understand CommonJS/ES6 very well. A lof of work is being done to change that and the goal is to transpile ES6/CommonJS code into a format that Closure can understand and optimize.

Given the vast differences in coding/packaging conventions out there this is a rather big task. Some packages work, some packages work with a few tweaks and some packages just don’t work at all.

Using this option is by far the best if it works. That is a big “if” though and getting it to work can be quite challenging. Ideally :externs would not be required anymore but in practice you still have to use them to work around a few kinks (for now).

I hope this will work reliably soon but we are not quite there yet.

Part #2

In the second post I will describe how shadow-cljs will handle this going forward.