Problem Solved: Source Paths

The question what exactly shadow-cljs does differently compared to other ClojureScript tools comes up every now and again.

At this point the answer is “A lot!” but that is not a very satisfying answer. The long answer however is really long and I thought I make a blog post series out of it going into a few internal details about what problems exactly I was trying to solve with certain features. These sometimes might seem like tiny little details but for me they had a big impact.

I’ll leave it up to the reader to decide whether these are actual problems. They were for me, they might not be for you. Pretty much all the features in shadow-cljs came out of personal experience when building CLJS projects over the last 5 years. YMMV.

Problem #1: Source Paths in CLJS

ClojureScript by default does not have the concept of source paths, only “inputs”. An input may either be a file or a directory. In case of a directory it is searched recursively for .clj(s|c) files which then become inputs.

The problem is that all inputs are included in all builds by default.

Suppose you want to build 2 separate things from one code base, maybe a browser client app with a complementary node.js server. For sake of simplicity I’ll trim down the code examples to an absolute minimum.

Imagine this simple file structure

.
├── deps.edn
├── build.clj
└── src
    └── demo
        ├── client.cljs
        └── server.cljs

The client

(ns demo.client)

(js/console.log "client")

The server

(ns demo.server)

(js/console.log "server")

The build file

(require 'cljs.build.api)

(cljs.build.api/build "src"
  {:output-to "out/main.js"
   :verbose true
   :target :nodejs
   :optimizations :advanced})

Compiling this and running the generated code produces

$ clj build.clj
$ node out/main.js
client
server

As expected the generated output contains ALL the code since the config does not capture which code should be included. This will not always be this obvious since not everything makes itself known like this. It is very easy to overlook files and accidentally include them in a build when you wouldn’t otherwise need them. In theory :advanced should take care of this but that does not always work.

In addition the compiler “inputs” are not known to Clojure at all. So if you want to use macros you need to include those separately via the :source-paths of lein to ensure they end up on the classpath.

Solution #1: :main

The compiler option :main solves that as it lets us select an “entry” namespace and only its dependencies will be included in the compilation.

(require 'cljs.build.api)

(cljs.build.api/build "src"
  {:output-to "out/main.js"
   :verbose true
   :target :nodejs
   :main 'demo.server
   :optimizations :advanced})

Recompile and we only get the desired server output. If you always remember to set this you will be safe.

Solution #2: Separate Source Paths

The more common solution is to split out the code into separate source paths from the beginning. So each “target” gets its own folder and each build will only pick the relevant folders.

.
├── deps.edn
├── build.clj
└── src
    └── client
        └── demo
            └── client.cljs
    └── server
        └── demo
            └── server.cljs
    └── shared
        └── demo
            └── foo.cljs

You will typically end up with an additional src/shared folder for code shared among both targets. I personally find this incredibly frustrating to work with.

I suspect that this pattern became wide-spread since :main was introduced some time after multiple source paths become a thing. I’m partially to blame for this since I was the one that added support for multiple :source-paths in lein-cljsbuild.

I’m not saying that allowing multiple :source-paths is a bad thing, there are several very valid use-cases for doing this. I only think that this pattern is overused and we already have namespaces to separate our code. I’m all for separating src/main from src/test but src/shared goes way too far IMHO.

shadow-cljs Solution

The solution in shadow-cljs is pretty straightforward.

• shadow-cljs expects “entry” namespaces to be configured for all builds. Browser builds do this via :modules, node builds via :main. This is the default and you cannot build those targets without them
• Multiple :source-paths are supported but sources are only taken into account when referenced
• Multiple :source-paths are always global. You cannot configure a separate source path per build
• :source-paths are always on the classpath

Although the implementation in shadow-cljs is entirely different it doesn’t provide anything that would not be possible with standard CLJS. I do believe that enforcing the config of “entry” namespaces however saves you from unknowingly including too much code in your builds. shadow-cljs just takes care of setting a better default so you don’t have to worry about it. You’ll see this pattern repeated in many of the shadow-cljs features.