Meyvn
Next-gen build tool

Table of Contents

Intro

You Know Nothing, Jon Snow. — Ygritte

The word maven comes from the Yiddish meyvn, meaning one who understands. You can think of Meyvn as:

  • A build tool
  • A wrapper for Maven
  • A tools.deps extension.

Meyvn enables you to generate uberjars (executables) and jars (libraries), and to deploy them on remote servers, Clojars, etc.

Goals

  • Use deps.edn as single source of truth.
  • One command to do it all: compile, package, install on a remote server, etc.
  • Standalone and lightweight. It only depends on Clojure and Maven (with no external dependencies).
  • Ability to selectively shrink uberjars
  • Leverage the Maven ecosystem
  • Include a good Clojurescript story: official compiler, shared aot cache, all options available to user, etc.
  • Sustainable open source with dual licensing scheme.

Installation

  • Meyvn requires Maven. Interestingly, Meyvn does not require Clojure. Neither a Clojure installation nor the official CLI tools. This is because Meyvn relies on Maven for pulling core dependencies (it's turtles all the way down). If you do have Clojure installed, you can install Meyvn with our easy installer. Otherwise, please follow the manual installation instructions.

Easy installation

Copy and paste the following command in your terminal:

clj -Sdeps '{:deps {org.meyvn/meyvn-installer {:mvn/version "1.4.1"}}}' -M -m meyvn-installer.core

Manual installation

  • Download Meyvn to your local repository
mvn org.apache.maven.plugins:maven-dependency-plugin:3.2.0:get -DremoteRepositories=https://nexus.tuppu.net/repository/meyvn/ -Dartifact=org.meyvn:meyvn:1.6.4
  • Create a new shell script in your path.
$ touch /usr/local/bin/myvn
  • Edit your newly created shell script with the following.
$ cat /usr/local/bin/myvn
#!/bin/sh
M2_HOME=/path/to/maven/root java -jar ~/.m2/repository/org/meyvn/meyvn/1.6.4/meyvn-1.6.4.jar "$@"

M2_HOME points to the root of your Maven installation directory, replace the content with the prefix appropriate for your system.

Note: Maven can run without this environment variable on the command line, but the Maven Invoker APIs require it to be set explicitly.

  • To find the prefix for the Maven installation on your system.
$ readlink -f `which mvn` | awk '{gsub("bin/mvn", ""); print}'

Note: On Mac OS X, brew install coreutils and replace readlink with greadlink.

  • Set the executable bit.

Note: $@ references the arguments passed to the script.

$ chmod +x /usr/local/bin/myvn

Note: Some features require access credentials

Add your Meyvn credentials to your settings.xml file in /.m2 repository.

  <server>
  <id>meyvn</id>
  <username>yourname</username>
  <password>yourpassword</password>
</server>

Usage

The standard Maven lifecycle phases and goals are passed as arguments. There’s documentation, too.

For example:

myvn compile 

Or

myvn package

Or

myvn deploy

You can also chain publishing goals, just like in Maven:

myvn clean clojure:test install

Debugging the build

If you see errors with the build, run myvn -g. This will persist Meyvn’s pom file. You can now run mvn on it and debug as you normally would in Maven. You will need to specify the path to the pom file.

mvn -f meyvn-pom.xml <goal>

Configuration

Configuration is stored in meyvn.edn, which will be created in the root of your project on first run.

Here are the defaults. Aside from the :pom key which captures the project coordinates and is always used, the other keys can be enabled or disabled as needed.

{:pom {:group-id "com.changeme"
       :artifact-id "myproject"
       :version "1.0.0"
       :name "My project does a lot"}
 :packaging {:uberjar 
             {:enabled true
              :main-class "main.core"              
              :excludes {:artifacts ["org.clojure:google-closure-library"]
                         :filters ["META-INF/*.MF" "META-INF/*.SF" "META-INF/*.DSA" "META-INF/*.RSA"]}}             
             :jar
             {:enabled false
              :remote-repository {:id "clojars" ;; Username and password lives in ~/.m2/settings.xml
                                  :url "https://clojars.org/repo"}}}
 :cljs {:enabled false
        :compiler-opts {:main "main.core"
                        :optimizations :advanced
                        :output-wrapper true
                        :infer-externs true
                        :parallel-build true
                        :aot-cache true
                        :output-to "resources/js/main.js"}
        :tools-deps-alias :cljs}
 :scm {:enabled true} ; will autodetect git repository
 :testing {:enabled false
           :tools-deps-alias :test} ; only in commercial version
 :profiles {:enabled false
            :staging {:http-port "3000"}
            :production {:http-port "8000"}}
 :distribution-management {:id "ssh-repository"
                           :url "scpexe://user@domain:/home/.m2/repository"}} ; only in commercial version

How does it work?

tools.deps has the ability to translate a deps.edn file into a pom file (clj -Spom). Meyvn starts off from that pom file and augments it with features that make sense for Clojure workflows. Meyvn’s pom file is transient and does not interfere with POM files that may already be present in your project.

POM lein pom clj -Spom myvn -x pom
root
modelVersion
groupId
artifactId
version
dependencies
compilation (clj)
compilation (cljs)
uberjars
native image (graalvm)
obfuscation (proguard)
javafx (plugin)
gpg signing (plugin)
deploying (plugin)
build plan (plugin)
minification (css + images)
data reader support
jpro (plugin)
jpackage (plugin)
appimage (plugin)

Maven is invoked via an API (Apache Maven Invoker) and can be passed all lifecycle phases or goal it supports.

Clojurescript sources are compiled and included in the final artifact. Clojurescript compilation is done in its own process with the official compiler.

Uberjars

Consider the following deps.edn file:

{:paths ["src/clj"]
 :deps {org.clojure/core.async {:mvn/version "0.4.474"}
        ring {:mvn/version "1.6.3"}
        compojure {:mvn/version "1.6.1"}}
 :aliases {:cljs {:extra-deps {org.clojure/clojurescript {:mvn/version "1.10.238"}
                               reagent {:mvn/version "0.8.1"} 
                               secretary {:mvn/version "1.2.3"}}
                  :extra-paths ["src/cljs"]}}}

The Clojurescript-side of the mixed project is cleanly segregated. The :cljs alias is used when compiling the *.cljs files, but not when assembling the uberjar, helping to keep the latter small. You tell Meyvn to use this alias in the meyvn.edn configuration, under the cljs -> tools-deps-alias keys.

If there is a resources folder in the base directory, it will be included in the build.

Meyvn uses the Apache Maven Shade Plugin in order to build uberjars.

Shading dependencies is the process of including and renaming dependencies (thus relocating the classes & rewriting affected bytecode & resources) to create a private copy that you bundle alongside your own code. But the shading part is actually optional: the plugin allows to include dependencies in your jar (fat jar), and optionally rename (shade) dependencies.

Meyvn gives you access to the exclusions facility provided by the Shade plugin, equivalent to Leiningen’s uberjar-exclusions or Boot’s standard-jar-exclusions.

:excludes {:artifacts ["org.clojure:google-closure-library"]
           :filters ["META-INF/*.MF" "META-INF/*.SF" "META-INF/*.DSA" "META-INF/*.RSA"]}

Note that you don’t need to exclude INDEX/LIST as this is built-in by the Shade plugin.

Additionally, Meyvn allows you to exclude artifacts. For example, sometimes the Closure library is pulled by a transitive dependency and lands in your final uberjar. With Meyvn you can prevent that.

Data readers are merged with a custom transformer that knows how to merge EDN maps.

Regular jars

Libraries uploaded to Clojars are typically non-aot, source-only jars. Uploading to Clojars follows standard procedure. Private repositories are supported as well. For example, to upload an artifact to deps.co, adjust the remote repository setting in the jar section of meyvn.edn.

:jar
{:enabled true
 :remote-repository {:id "releases"
                     :url "https://repo.deps.co/your-org/releases"}}

In all cases, use settings.xml for storing your credentials, or refer to Maven for password encryption.

Pom files

Meyvn works with its own set of pom files. It isn’t bothered with existing pom files in your project directory. This is by design. The single source of truth is deps.edn. Together with the configuration (in meyvn.edn), it knows all that it needs to know.

The added benefit is that you can continue to maintain a pom file if you are already using a Maven workflow.

Dependency mechanism

The transitive dependency mechanism used by Maven is guided by the nearest wins conflict resolution strategy. This allows for resolution of individual conflicts: for any particular conflicting dependency, you can specify its version within your own POM, and that version becomes the nearest.

Note that if two dependency versions are at the same depth in the dependency tree, until Maven 2.0.8 it was not defined which one would win, but since Maven 2.0.9 it's the order in the declaration that counts: the first declaration wins.

With the commercial version, you can use dependency convergence, forcing the buld to fail on transitive dependencies that are not on the same version.

Testing

Consider the following deps.edn file.

{:paths ["src"]
 :deps {
   clj-time {:mvn/version "0.14.2"}
 }
 :aliases {:test {:extra-paths ["test"]
                  :extra-deps {org.clojure/test.check {:mvn/version "0.9.0"}}}}}

Again, please note the best practice of segregating paths and dependencies with aliases. To run your tests with Meyvn, make sure the relevant section in meyvn.edn looks like this:

:testing {:enabled true
          :tools-deps-alias :test}

Then run:

$ myvn clojure:test

The build will abort in case of errors.

Note: This feature is found in the commercial version only.

Interactive coding

$ myvn clojure:nrepl

This will start a nREPL server with Cider middleware that you can connect to with nREPL clients.

Note: This feature is found in the commercial version only.

Profiles

In Maven, profiles are used to parameterize builds, not the runtime environment of the executable. There are good reasons for this, but this means that after your build is done, you can't just run the executable (if it needs environment variables to be set). First you need to make sure the environment is set up properly.

Meyvn can help with that. When you enable the profiles section, Meyvn will create a Maven profile in the transient POM, and under each profile (for example, staging or production), it will write a standard edn map describing your environment into standard java properties.

(We leverage the fact that custom properties can be defined under any profile.)

On your staging/production server, those properties will be accessible in the pom alongside your jar in the local repository.

Meyvn doesn't want to force you to install clojure or Meyvn on your servers, but if you do, you can use it to list those properties and pipe into a script in typical UNIX style.

$ myvn -x list -a org.bar:foo:1.0.0 -p production

The -a switch is for artifact (in Maven coordinates) and -p is for profile.

The script could, typically, massage the properties into environment variables. How you use them depends on your final command output, really. The last mile is context-dependent.

In the absence of Meyvn on the server, you can get the properties via the Maven helper plugin.

$ mvn org.apache.maven.plugins:maven-help-plugin:3.1.0:all-profiles "-Dartifact=org.company:myproject:1.0.0
$ mvn org.apache.maven.plugins:maven-help-plugin:3.1.0:evaluate -Dexpression=project.properties -Dartifact=org.company:myproject:1.0.0

Finally, Meyvn has built-in support for runit, the UNIX init scheme with service supervision. The following command will write the environment in the format expected by runit under the path specified by the -t argument.

#+BEGINSRC sh myvn -x write -a org.bar:foo:1.0.0 -p production -t /opt/foo #+ENDSRC sh

Note: This feature is found in the commercial version only.

Auxiliary commands

Meyvn runs with the same interface as Maven. Goals and lifecycle phases are being passed to it as you would with standard Maven. The -x flag changes the mode of operation and allows you to run specialized tasks.

Simply run myvn -x to see what is available. Currently, Meyvn can generate the POM file, list newer versions of dependencies, show platform information. More functionality is to be expected.

Note: this is a commercial feature.

Will it work?

It should work for the typical Clojure workflows. Please feel free to contact me in private if you want help solving your company’s build workflow.

Please note that Windows is not supported (the Clojure command line tools are not available).

Feel free to open issues regarding the supported workflows. New workflows will be added under commercial agreements.

Roadmap

This is just the beginning. The release of the clj command line tools is still fresh, and we are just starting to see the possibilities.

The takeaway for Meyvn is that building on top of the Maven ecosystem is rewarding. It is a huge ecosystem, well documented and extremely mature. A lot of functionality just sits there, waiting to be tapped by our tooling (in areas such as continuous integration, generated documentation, testing, reporting, etc.)

The plan is to have more features as companies sponsor them. Those features will be fed back to the OSS version.

What about Boot and Leiningen?

Naturally, Boot and Leiningen can also produce artifacts, but their scope is wider, providing development-time workflows and extension mechanisms.

Meyvn delegates build tasks to Maven, and offers direct access to the Clojurescript compiler.

In other words, there is no competition, only complementary options.

Sustainable open source

We as a community know how to write open source software, but we are less knowledgeable in how to make that activity sustainable. With Meyvn, I’m attempting to lead a sustainable Open Source project. That means that Meyvn is dual licensed, with a commercial license available for sale.

The LGPLv3 licensed community version will always remain free and available to all parties. However, companies who use Meyvn in their operations are expected to acquire a commercial license.

In the coming months, I will experiment with two competing models:

  1. Commercial and community version have parity of features
  2. Commercial version has more features than community version

What enables the first model is analytics. By sending data home, I can approach companies with proposals to acquire a commercial license. The features I am adding to the commercial version are fed back to the OSS version.

Pros: The community benefits. Cons: Tracking.

The second model doesn’t need tracking, because the distinction between a basic and a feature-laden version is by itself an incentive to buy the “better version”.

Pros: No tracking. Cons: The community loses.

The first model I am putting to test is the first model (with opt-in tracking). When you opt-in, Meyvn will send the POM’s group ID and success result of each execution back to an analytics server. When you opt-out, the program quits. At this stage, I am interested in users who can relate with the mission statement, for whom finding ways to do sustainable OSS is a shared value and not mere lip service.

The NoLipService library is responsible for the reporting. To ensure transparency it is released as open source as well. It is still early days, and I welcome contributions and different implementation ideas.

License

Meyvn is released under a dual licensing scheme.

Meyvn is an Open Source project licensed under the terms of the LGPLv3 license. Please see http://www.gnu.org/licenses/lgpl-3.0.html for license text.

Meyvn Enterprise has a commercial-friendly license allowing private forks and modifications of Meyvn. Licensees get a build of Meyvn with commercial features, and devoid of NoLipService’s reporting. Additionally, licensees get access to email support.

Please contact me for more details.

Patron

Writing and maintaining Open Source Software takes time and effort. Be a mensch. Be a maven. Patronize Meyvn.

Literature

Author: Daniel Szmulewicz

Created: 2022-03-20 Sun 01:13

Validate