Problem Solved: Source Paths
|The question what exactly shadow-cljs does differently compared to other ClojureScript tools comes up every now and again.
At this point the answer is “A lot!” but that is not a very satisfying answer. The long answer however is really long and I thought I make a blog post series out of it going into a few internal details about what problems exactly I was trying to solve with certain features. These sometimes might seem like tiny little details but for me they had a big impact.
I’ll leave it up to the reader to decide whether these are actual problems. They were for me, they might not be for you. Pretty much all the features in shadow-cljs
came out of personal experience when building CLJS projects over the last 5 years. YMMV.
Problem #1: Source Paths in CLJS
ClojureScript by default does not have the concept of source paths, only “inputs”. An input may either be a file or a directory. In case of a directory it is searched recursively for .clj(s|c)
files which then become inputs.
The problem is that all inputs are included in all builds by default.
Suppose you want to build 2 separate things from one code base, maybe a browser client app with a complementary node.js server. For sake of simplicity I’ll trim down the code examples to an absolute minimum.
Imagine this simple file structure
.
├── deps.edn
├── build.clj
└── src
└── demo
├── client.cljs
└── server.cljs
The client
(ns demo.client)
(js/console.log "client")
The server
(ns demo.server)
(js/console.log "server")
The build file
(require 'cljs.build.api)
(cljs.build.api/build "src"
{:output-to "out/main.js"
:verbose true
:target :nodejs
:optimizations :advanced})
Compiling this and running the generated code produces
$ clj build.clj
$ node out/main.js
client
server
As expected the generated output contains ALL the code since the config does not capture which code should be included. This will not always be this obvious since not everything makes itself known like this. It is very easy to overlook files and accidentally include them in a build when you wouldn’t otherwise need them. In theory :advanced
should take care of this but that does not always work.
In addition the compiler “inputs” are not known to Clojure at all. So if you want to use macros you need to include those separately via the :source-paths
of lein
to ensure they end up on the classpath.
Solution #1: :main
The compiler option :main
solves that as it lets us select an “entry” namespace and only its dependencies will be included in the compilation.
(require 'cljs.build.api)
(cljs.build.api/build "src"
{:output-to "out/main.js"
:verbose true
:target :nodejs
:main 'demo.server
:optimizations :advanced})
Recompile and we only get the desired server
output. If you always remember to set this you will be safe.
Solution #2: Separate Source Paths
The more common solution is to split out the code into separate source paths from the beginning. So each “target” gets its own folder and each build will only pick the relevant folders.
.
├── deps.edn
├── build.clj
└── src
└── client
└── demo
└── client.cljs
└── server
└── demo
└── server.cljs
└── shared
└── demo
└── foo.cljs
You will typically end up with an additional src/shared
folder for code shared among both targets. I personally find this incredibly frustrating to work with.
I suspect that this pattern became wide-spread since :main
was introduced some time after multiple source paths become a thing. I’m partially to blame for this since I was the one that added support for multiple :source-paths
in lein-cljsbuild
.
I’m not saying that allowing multiple :source-paths
is a bad thing, there are several very valid use-cases for doing this. I only think that this pattern is overused and we already have namespaces to separate our code. I’m all for separating src/main
from src/test
but src/shared
goes way too far IMHO.
shadow-cljs Solution
The solution in shadow-cljs
is pretty straightforward.
shadow-cljs
expects “entry” namespaces to be configured for all builds. Browser builds do this via:modules
, node builds via:main
. This is the default and you cannot build those targets without them- Multiple
:source-paths
are supported but sources are only taken into account when referenced - Multiple
:source-paths
are always global. You cannot configure a separate source path per build :source-paths
are always on the classpath
Although the implementation in shadow-cljs
is entirely different it doesn’t provide anything that would not be possible with standard CLJS. I do believe that enforcing the config of “entry” namespaces however saves you from unknowingly including too much code in your builds. shadow-cljs
just takes care of setting a better default so you don’t have to worry about it. You’ll see this pattern repeated in many of the shadow-cljs
features.