We had some interesting issues with Gradle this week.
We build a number of internal projects with Gradle, and some of these projects have interdependencies.
The dependency graph of our internal projects extends to several levels.
To illustrate, this diagram shows three projects, with "server" depending on "common",
and "client" depending on "server".
For internal dependencies, we use Gradle's "changing module" version selectors: "latest.integration" and "latest.release".
The problem occurred because Gradle 2.3 changed the way these version selectors are written to a published pom.xml dependency section.
Prior versions wrote the version selectors "as-is", e.g. "latest.integration". I believe that this convention originated with Ivy.
But this is not a valid Maven version.
So Gradle 2.3 changed to write a valid Maven version such as "LATEST" or "RELEASE".
A great post explaining the options available with Maven, and some of the pros and cons, is
this on one StackOverflow.
Generally speaking, it's best to use specific version numbers in dependencies for released artifacts, for repeatable builds.
But it's also desirable to have changing or dynamic dependencies for snapshot or integration builds, for continuous integration and testing.
The problem is that older versions of Gradle don't understand these "new" values ("LATEST" and "RELEASE").
Let's show this using the sample projects above.
For the purpose of this illustration, we'll use a common init.gradle shared by each project, with contents:
def homeDir = System.getProperty("user.home")
def repoUrl = "file:///$homeDir/tmp/repo"
allprojects {
apply plugin: "java"
apply plugin: "maven"
group = "example"
uploadArchives {
repositories {
mavenDeployer {
repository(url: repoUrl)
}
}
}
repositories {
maven {
url repoUrl
}
}
}
We'll use this init.gradle for every build in these examples, using this alias:
alias mygradle="./gradlew -I../init.gradle"
For the "common" project, we'll have these files:
➜ common git:(master) ✗ tree
.
├── build.gradle
├── gradle
│ └── wrapper
│ ├── gradle-wrapper.jar
│ └── gradle-wrapper.properties
├── gradlew
├── gradlew.bat
└── src
└── main
└── java
└── example
└── Common.java
In build.gradle we have:
wrapper {
gradleVersion = "2.2.1"
}
version = "01.00"
We can build "common" like this:
➜ common git:(master) ✗ mygradle uploadArchives
:compileJava
:processResources UP-TO-DATE
:classes
:jar
:uploadArchives
Uploading: example/common/01.00/common-01.00.jar to repository remote at file:////Users/jhurst/tmp/repo
Transferring 1K from remote
Uploaded 1K
BUILD SUCCESSFUL
The generated pom.xml is not very interesting:
<project
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"
xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<modelVersion>4.0.0</modelVersion>
<groupId>example</groupId>
<artifactId>common</artifactId>
<version>01.00</version>
</project>
Now we look at "server". The files are:
➜ server git:(master) ✗ tree
.
├── build.gradle
├── gradle
│ └── wrapper
│ ├── gradle-wrapper.jar
│ └── gradle-wrapper.properties
├── gradlew
├── gradlew.bat
└── src
└── main
└── java
└── example
└── Server.java
The server project uses Gradle 2.4, and declares a dependency on "common" in its build.gradle:
wrapper {
gradleVersion = "2.4"
}
dependencies {
compile "example:common:latest.integration"
}
version = "01.00"
We build "server":
➜ server git:(master) ✗ mygradle uploadArchives
:compileJava UP-TO-DATE
:processResources UP-TO-DATE
:classes UP-TO-DATE
:jar UP-TO-DATE
:uploadArchives
BUILD SUCCESSFUL
Now we have a dependency in the generated pom.xml:
<project
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"
xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<modelVersion>4.0.0</modelVersion>
<groupId>example</groupId>
<artifactId>server</artifactId>
<version>01.00</version>
<dependencies>
<dependency>
<groupId>example</groupId>
<artifactId>common</artifactId>
<version>LATEST</version>
<scope>compile</scope>
</dependency>
</dependencies>
</project>
This dependency is specified using the new, Maven-compatible version selector.
Now we go to "client":
➜ client git:(master) ✗ tree
.
├── build.gradle
├── gradle
│ └── wrapper
│ ├── gradle-wrapper.jar
│ └── gradle-wrapper.properties
├── gradlew
├── gradlew.bat
└── src
└── main
└── java
└── example
└── Client.java
The client project uses Gradle 2.2.1, and declares a dependency on "server" in its build.gradle:
wrapper {
gradleVersion = "2.2.1"
}
dependencies {
compile "example:server:latest.integration"
}
version = "01.00"
We attempt to build "client":
➜ client git:(master) ✗ mygradle uploadArchives
:compileJava
FAILURE: Build failed with an exception.
* What went wrong:
Could not resolve all dependencies for configuration ':compile'.
> Could not find example:common:LATEST.
Searched in the following locations:
file:/Users/jhurst/tmp/repo/example/common/LATEST/common-LATEST.pom
file:/Users/jhurst/tmp/repo/example/common/LATEST/common-LATEST.jar
Required by:
example:client:01.00 > example:server:01.00
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.
BUILD FAILED
The problem is that Gradle 2.2.1 does not correctly interpret "LATEST" as a changing module version selector.
It's easily fixed - we simply upgrade "client" to Gradle 2.3 or later.
You might think this is a really trivial problem. It is, once it's clear what is going on.
We found it a bit confusing at first because we didn't know where the "LATEST" string in the dependency error message was coming from.
There is a further difficulty caused by this change if you use Groovy's Grapes feature in Groovy scripts to fetch dependencies.
Grapes uses Ivy to resolve dependencies, and it does not understand this LATEST/RELEASE syntax in POM files either.
To show this, let's use a ivysettings.xml file as follows:
<ivysettings>
<resolvers>
<ibiblio
name="downloadGrapes"
m2compatible="true"
root="file:///Users/jhurst/tmp/repo"/>
</resolvers>
<settings defaultResolver="downloadGrapes"/>
</ivysettings>
We configure Groovy to use this using the grape.config system property:
export JAVA_OPTS="-Dgrape.config=$PWD/ivysettings.xml"
Let's have a Groovy script that has a dependency on the "common" module:
@Grab("example:common:01.00")
import example.Common
println Common.simpleName
When we run this, it fetches the dependency and runs successfully:
➜ groovy git:(master) ✗ groovy ./grabcommon.groovy
Common
Let's try another Groovy script that has a dependency on "server" instead:
@Grab("example:server:01.00")
import example.Server
println Server.simpleName
When we run this, we get a similar failure to that with Gradle earlier:
➜ groovy git:(master) ✗ groovy ./grabserver.groovy
org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
General error during conversion: Error grabbing Grapes --
[unresolved dependency: example#common;LATEST: not found]
java.lang.RuntimeException: Error grabbing Grapes --
[unresolved dependency: example#common;LATEST: not found]
...
This one is not so easy to solve.
Gradle originally used Ivy's dependency resolution code, but then switched to using its own code.
Groovy's Grapes feature still uses Ivy.
We need either to improve Ivy to support this syntax in POM files, or else perhaps it would be better to get Groovy Grapes to use Gradle's dependency resolution code instead of Ivy. But given that Grapes is configured using an ivysettings.xml, and Gradle does not provide any analogous way to tell a Groovy script where to look for dependencies, it is not obvious how we would switch Groovy to use Gradle's code. Besides, Groovy needs to continue to support ivysettings.xml and all Ivy features, for backwards compatibility.
A colleague of mine pointed out that one solution is to use the long form of @Grab, with transitive = false, like this:
@Grab(group = "example", module = "server", version = "01.00", transitive = false)
@Grab("example:common:01.00")
import example.Common
import example.Server
println Server.simpleName
println Common.simpleName
This works, but it excludes all of the transitive dependencies.
If you have a lot of third party dependencies and need this exclusion only for your local modules, it's not that great.
