Maven Dependency Hell

Yeah, I knew you’ve felt the same some times! Some developers are suddenly experiencing local problems, while others doesn’t, even as they’re having the same code base. The experienced developer thinks “classpath problem!”, almost as a reflex! And they’re always right, it’s yet another a damned jar that’s been automagically downloaded by maven from an other automagically loaded jar dependency. Anyway, that’s how it feels like for me. It’s often time-consuming to discover the reason, even more time-consuming if “classpath-problem” isn’t your first thought (god, no)! If not, say bye bye to productivity for that day. Been there, done that!

How to avoid it? Eliminate the automagic dependencies. Exclude these dependencies and explicitly define all used dependencies.It’s somewhat time-consuming, but it’s worth it. Maven dependency management helps you out. mvn dependency:tree and mvn dependency:analyze are useful commands, but be aware, they aren’t necessarily correct. Eliminate the dependencies one by one. Some dependencies are only used runtime and it seems like the dependency analyzer don’t do well with aspects.

Today I had yet another annoying dependency problem, which only occurred on the local environment of one of our testers (oh nooooo). He had something like this

NoClassDefFoundError net/sf/cglib/proxy/Callback.class

What’s crazy, is that I had that exact same problem a week ago. Then I tried to locate it and excluded it from a Hibernate dependency. And it worked! Well, I found more occurrences, excluded them all so I could reproduce the error (get the NoClassDefFoundError locally as well). It was the artifact joda-time-hibernate that automagically(my new favourite word) retrieved the wrong cglib. Then I explicitly defined the cglib dependency and it all plays beautifully.
To help others out, here’s a copy-paste of parts of my pom (which will surely be outdated by the time you hit this post. Slow down SpringTeam and Hibernate-team! I can’t hang on! )

<dependency>
	<groupId>org.hibernate</groupId>
	<artifactId>hibernate-entitymanager</artifactId>
	<version>3.3.1.ga</version>
	<exclusions>
	  <exclusion>
     		<groupId>cglib</groupId>
    		<artifactId>cglib</artifactId>
  	  </exclusion>
 	 </exclusions>
  </dependency>
  <dependency>
	<groupId>org.hibernate</groupId>
	<artifactId>hibernate</artifactId>
	<version>3.2.4.ga</version>
	<exclusions>
	  <exclusion>
     		 <groupId>cglib</groupId>
    		 <artifactId>cglib</artifactId>
  	  </exclusion>
 	 </exclusions>
   </dependency>
   <dependency>
	<groupId>cglib</groupId>
	<artifactId>cglib</artifactId>
	<version>2.1_3</version>
   </dependency>
   <dependency>
	<groupId>org.hibernate</groupId>
	<artifactId>hibernate-annotations</artifactId>
	<version>3.3.0.ga</version>
   </dependency>
   <dependency>
	<groupId>joda-time</groupId>
	<artifactId>joda-time</artifactId>
	<version>1.5</version>
   </dependency>
   <dependency>
	<groupId>joda-time</groupId>
	<artifactId>joda-time-hibernate</artifactId>
	<version>1.0</version>
        <exclusions>
	   <exclusion>
     	      <groupId>cglib</groupId>
      	      <artifactId>cglib-full</artifactId>
           </exclusion>
 	</exclusions>
   </dependency>

Not particulary beautiful…

We’ve had the most classpath problems with eclipse. When running mvn eclipse:clean eclipse:eclipse, all jars will be downloaded and found both in the classpath (.classpath file) and in your local m2 repository. Browsing these will help you find artifacts of different versions, which is a warning sign. Also, using Open Type ctrl+shift+T in eclipse, write the class name will identify the jar packages containing identical class signatures. These are the classes causing random problems, depending on which class is loaded first on the classpath.

Tip X: running mvn install -X yields a lot of output, including all the jars loaded. So, if you’ve found the jar name that’s causing the problems, you’ve got the key!

Edit: Run mvn install -X > mavenoutput.txt and search for all occurences of “removed – nearer found”. Eclude all these to proactively prevent classpath problems. Maven is smart, Eclipse is not (with regards to this. Or is it mvn eclipse:eclipse thats dumb?)

About Ole Morten Amundsen

Developer, programmer, entrepreneur. Java, .Net, ruby, rails, agile, lean. Opinionated enthusiast!
This entry was posted in hibernate, Java, maven and tagged , , . Bookmark the permalink.

10 Responses to Maven Dependency Hell

  1. anderssv says:

    Not sure if you are doing this, but use the dependencyManagement section in your main pom to specify versions. This way you sentralize all specification (only one place to specify version and excludes) and any automagically included dependencies (that you have already set a version for) is forced into the version you have specified. Still doesn’t eliminate it all, but helps a lot. :)

  2. anderssv says:

    And turn off the “need to login to comment” BS. ;) Use Akismet if you are afraid of spam, it works pretty good.

  3. Ole Morten says:

    “need to login..” turned off and Akismet on! Thanks.

    Yes, dependencyManagement in the main pom is a great idea. Thanks for pointing that out! It should’ve been mentioned in the post as well. Quick example given.
    main/root pom.xml:
    <dependencyManagement>
    <dependencies>
    <dependency>
    <groupId>org.aspectj</groupId>
    <artifactId>aspectjrt</artifactId>
    <version>{version.aspectj}</version>
    </dependency>
    <dependency>
    <groupId>org.aspectj</groupId>
    <artifactId>aspectjweaver</artifactId>
    <version>${version.aspectj}</version>
    </dependency>

    and in the submodule pom.xml

    <dependencies>
    <dependency>
    <groupId>org.aspectj</groupId>
    <artifactId>aspectjrt</artifactId>
    </dependency>
    <dependency>
    <groupId>org.aspectj</groupId>
    <artifactId>aspectjweaver</artifactId>
    </dependency>

    Thanks Anders!

  4. Petr Macek says:

    Greetings from Ostrava,

    you can also try using dependency plugin and save it to the file on disk (mvn dependency:tree > output) that will display also the path where the artifact came from. Not mentioning tha possibility to filter on artifact name.
    OR alternatively there is a neat jfrog util called dependency analyzer that can also draw a graph (from the developers of our beloved Artifactory:-))

    Regards Petr

  5. jon says:

    eclipse has a “Dependency Graph” view when you open a pom.xml file.

  6. I think there should be something like


    com.somestuff

    com.somestuff-lib
    1.2.3
    1.1.0

    Where baseversion is the version where the api changed last, so maven can see, if different referenced versions are only bugfixes..

    So dependency A with version 1.2.3 and B with version 1.1.9 will be changed to version 1.2.3.

    We should let maven do this stuff…

    (i am still happy with maven.. this stuff is not maven’s fault..)

  7. jke says:

    The “hell” is only beginning to emerge.
    Our case

    We built some project a single year ago using maven.

    Since we did not want the dependent libraries “wander” ahead, we fixed our versions as directed by everybody. Howeven – year has passed – and now our project does not compile anymore on a “clean” environment. Why? Simply because some repositories holding the dependencies on some our dependent libraries have disappeared. However these “broken” dependencies still exist and are really pain to fix. This is “Way too soon” breakage! We have product lifespan of 5 to 10 years.

    We will abandon maven and move back to ant and fixed “local” libraries. With those we have over 8 year old source-code, that still compile quickly right out of the cvs. The only feature we will hold from maven will be the jar naming pattern for our dependencies.

    btw. Do not let me start on dependency license management and changes on second or third level dependency licenses :)

  8. Tjad Clark says:

    Love the new word :) “Automagically”

  9. gigi says:

    Love what you do. Really saves me a lot of time and hassle. Just 1 thing

    whenever the dependency in the pom file is formatted as the following

    net.sourceforge.cobertura

    cobertura-runtime

    ${cobertura.version}

    ….
    i.e, tag value and name not in the same line, I’d see this error and formatted the pom and restart

    ./remove-extra-dependencies.rb:55: undefined method `[]’ for nil:NilClass (NoMethodError)
    from ./remove-extra-dependencies.rb:54:in `each’
    from ./remove-extra-dependencies.rb:54
    from ./remove-extra-dependencies.rb:52:in `each’
    from ./remove-extra-dependencies.rb:52

  10. Ben Caldwell says:

    I just came across this after two days of pure frustration and cursing maven!
    I had a problem where I had to use a new version of a jar to fix a bug, but one of my project dependencies had a dependency on the old version of the jar. I simply changed all my dependencies to direct, giving me control over which versions of jars I used so I could force everything to use the latest versions.
    Thanks heaps for the tip!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s