Why does Spring Boot Maven plugin mess up repackaging my reactor module sometimes?

Question

Project structure

I have a Maven reactor project in the following structure:

company-parent (other project for dependency and plugin management)

<parent>
  <groupId>org.apache</groupId>
  <artifactId>apache</artifactId>
  <version>21</version>
</parent>

<groupId>com.mycompany</groupId>
<artifactId>maven-mycompany-parent</artifactId>
<version>1</version>
<packaging>pom</packaging>

<dependencyManagement>
  <dependencies>
    <dependency>
      <groupId>org.springframework.boot</groupId>
      <artifactId>spring-boot-dependencies</artifactId>
      <version>2.1.6.RELEASE</version>
      <type>pom</type>
      <scope>import</scope>
    </dependency>
...
  </dependencies>
</dependencyManagement>
<build>
  <pluginManagement>
    <plugin>
      <groupId>org.springframework.boot</groupId>
      <artifactId>spring-boot-maven-plugin</artifactId>
      <version>2.1.6.RELEASE</version>
    </plugin>
    ...
  </pluginManagement>
  ....
</build>

project-parent

<parent>
  <groupId>com.mycompany</groupId>
  <artifactId>maven-mycompany-parent</artifactId>
  <version>1</version>
</parent>

<groupId>com.mycompany</groupId>
<artifactId>project-parent</artifactId>
<version>1.0-SNAPSHOT</version>
<packaging>pom</packaging>

<modules>
  <module>project-api</module>
  <module>project-impl</module>
</modules>

project-api module containing WSDLs and generated POJOs

<parent>
  <groupId>com.mycompany</groupId>
  <artifactId>project-parent</artifactId>
  <version>1.0-SNAPSHOT</version>
</parent>

<artifactId>project-api</artifactId>
<packaging>jar</packaging>

project-impl module containing to concrete JAX-WS implementation of the WSDLs

<parent>
  <groupId>com.mycompany</groupId>
  <artifactId>project-parent</artifactId>
  <version>1.0-SNAPSHOT</version>
</parent>

<artifactId>project-impl</artifactId>
<packaging>war</packaging>
...
<dependencies>
  <dependency>
    <groupId>com.mycompany</groupId>
    <artifactId>project-api</artifactId>
    <version>${project.version}</version>
  </dependency>
...
</dependencies>
<build>
  <plugins>
    <plugin>
      <groupId>org.apache.maven.plugins</groupId>
      <artifactId>maven-war-plugin</artifactId>
      <configuration>
        <webResources>
          <webResource>
            <directory>${project.basedir}/src/main/sql</directory>
            <targetPath>sql</targetPath>
          </webResource>
          <webResource>
            <directory>${project.basedir}</directory>
            <include>CHANGELOG.md</include>
            <filtering>true</filtering>
            <targetPath>.</targetPath>
          </webResource>
        </webResources>
        <archive>
          <manifest>
            <addClasspath>true</addClasspath>
            <addDefaultImplementationEntries>true</addDefaultImplementationEntries>       
            <addDefaultSpecificationEntries>true</addDefaultSpecificationEntries>
          </manifest>
        </archive>
      </configuration>
    </plugin>
    <plugin>
      <groupId>org.springframework.boot</groupId>
      <artifactId>spring-boot-maven-plugin</artifactId>
      <executions>
        <execution>
          <goals>
            <goal>build-info</goal>
            <goal>repackage</goal>
          </goals>
        </execution>
      </executions>
    </plugin>
    ...
  </plugins>
  ...
</build>

Problem

Sometimes, the repackaged .war cannot be executed due to file not found and classloading issues. I figured it out, that when the problem persists, the repackaged .war's WEB-INF/lib contains a lot of .jar dependencies and one, empty directory: WEB-INF/lib/project-api-1.0-SNAPSHOT.jar/.

Here's the output of unzip:

-bash-4.2$ unzip -l project-impl/target/project-impl.war | grep project-api
        0  07-17-2019 10:32   WEB-INF/lib/project-api-1.0-SNAPSHOT.jar/

And the execution fails, of course:

-bash-4.2$ java -jar project-impl/target/project-impl.war
...
[ERROR] 2019-07-17T11:24:55,711 org.spr.boo.SpringApplication Application run failed
org.springframework.beans.factory.BeanDefinitionStoreException: Failed to parse configuration class [com.mycompany.package.Application]; nested exception is java.io.FileNotFoundException: class path resource [com/mycompany/package/MyPortType.class] cannot be opened because it does not exist
        at org.springframework.context.annotation.ConfigurationClassParser.parse(ConfigurationClassParser.java:181) ~[spring-context-5.1.8.RELEASE.jar!/:5.1.8.RELEASE]
        at org.springframework.context.annotation.ConfigurationClassPostProcessor.processConfigBeanDefinitions(ConfigurationClassPostProcessor.java:315) ~[spring-context-5.1.8.RELEASE.jar!/:5.1.8.RELEASE]
        at org.springframework.context.annotation.ConfigurationClassPostProcessor.postProcessBeanDefinitionRegistry(ConfigurationClassPostProcessor.java:232) ~[spring-context-5.1.8.RELEASE.jar!/:5.1.8.RELEASE]
        at org.springframework.context.support.PostProcessorRegistrationDelegate.invokeBeanDefinitionRegistryPostProcessors(PostProcessorRegistrationDelegate.java:275) ~[spring-context-5.1.8.RELEASE.jar!/:5.1.8.RELEASE]
        at org.springframework.context.support.PostProcessorRegistrationDelegate.invokeBeanFactoryPostProcessors(PostProcessorRegistrationDelegate.java:95) ~[spring-context-5.1.8.RELEASE.jar!/:5.1.8.RELEASE]
        at org.springframework.context.support.AbstractApplicationContext.invokeBeanFactoryPostProcessors(AbstractApplicationContext.java:705) ~[spring-context-5.1.8.RELEASE.jar!/:5.1.8.RELEASE]
        at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:531) ~[spring-context-5.1.8.RELEASE.jar!/:5.1.8.RELEASE]
        at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:140) ~[spring-boot-2.1.6.RELEASE.jar!/:2.1.6.RELEASE]
        at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:742) [spring-boot-2.1.6.RELEASE.jar!/:2.1.6.RELEASE]
        at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:389) [spring-boot-2.1.6.RELEASE.jar!/:2.1.6.RELEASE]
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:311) [spring-boot-2.1.6.RELEASE.jar!/:2.1.6.RELEASE]
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:1213) [spring-boot-2.1.6.RELEASE.jar!/:2.1.6.RELEASE]
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:1202) [spring-boot-2.1.6.RELEASE.jar!/:2.1.6.RELEASE]
        at com.mycompany.package.Application.main(Application.java:18) [classes!/:3.0.0-SNAPSHOT]
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_212]
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_212]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_212]
        at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_212]
        at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:47) [project-impl.war:3.0.0-SNAPSHOT]
        at org.springframework.boot.loader.Launcher.launch(Launcher.java:86) [project-impl.war:3.0.0-SNAPSHOT]
        at org.springframework.boot.loader.Launcher.launch(Launcher.java:50) [project-impl.war:3.0.0-SNAPSHOT]
        at org.springframework.boot.loader.WarLauncher.main(WarLauncher.java:57) [project-impl.war:3.0.0-SNAPSHOT]
Caused by: java.io.FileNotFoundException: class path resource [com/mycompany/package/MyPortType.class] cannot be opened because it does not exist
        at org.springframework.core.io.ClassPathResource.getInputStream(ClassPathResource.java:180) ~[spring-core-5.1.8.RELEASE.jar!/:5.1.8.RELEASE]
        at org.springframework.core.type.classreading.SimpleMetadataReader.<init>(SimpleMetadataReader.java:51) ~[spring-core-5.1.8.RELEASE.jar!/:5.1.8.RELEASE]
        at org.springframework.core.type.classreading.SimpleMetadataReaderFactory.getMetadataReader(SimpleMetadataReaderFactory.java:103) ~[spring-core-5.1.8.RELEASE.jar!/:5.1.8.RELEASE]
        at org.springframework.boot.type.classreading.ConcurrentReferenceCachingMetadataReaderFactory.createMetadataReader(ConcurrentReferenceCachingMetadataReaderFactory.java:86) ~[spring-boot-2.1.6.RELEASE.jar!/:2.1.6.RELEASE]
        at org.springframework.boot.type.classreading.ConcurrentReferenceCachingMetadataReaderFactory.getMetadataReader(ConcurrentReferenceCachingMetadataReaderFactory.java:73) ~[spring-boot-2.1.6.RELEASE.jar!/:2.1.6.RELEASE]
        at org.springframework.core.type.classreading.SimpleMetadataReaderFactory.getMetadataReader(SimpleMetadataReaderFactory.java:81) ~[spring-core-5.1.8.RELEASE.jar!/:5.1.8.RELEASE]
        at org.springframework.context.annotation.ConfigurationClassParser.asSourceClass(ConfigurationClassParser.java:682) ~[spring-context-5.1.8.RELEASE.jar!/:5.1.8.RELEASE]
        at org.springframework.context.annotation.ConfigurationClassParser$SourceClass.getInterfaces(ConfigurationClassParser.java:1008) ~[spring-context-5.1.8.RELEASE.jar!/:5.1.8.RELEASE]
        at org.springframework.context.annotation.ConfigurationClassParser.processInterfaces(ConfigurationClassParser.java:375) ~[spring-context-5.1.8.RELEASE.jar!/:5.1.8.RELEASE]
        at org.springframework.context.annotation.ConfigurationClassParser.doProcessConfigurationClass(ConfigurationClassParser.java:323) ~[spring-context-5.1.8.RELEASE.jar!/:5.1.8.RELEASE]
        at org.springframework.context.annotation.ConfigurationClassParser.processConfigurationClass(ConfigurationClassParser.java:242) ~[spring-context-5.1.8.RELEASE.jar!/:5.1.8.RELEASE]
        at org.springframework.context.annotation.ConfigurationClassParser.parse(ConfigurationClassParser.java:191) ~[spring-context-5.1.8.RELEASE.jar!/:5.1.8.RELEASE]
        at org.springframework.context.annotation.ConfigurationClassParser.doProcessConfigurationClass(ConfigurationClassParser.java:295) ~[spring-context-5.1.8.RELEASE.jar!/:5.1.8.RELEASE]
        at org.springframework.context.annotation.ConfigurationClassParser.processConfigurationClass(ConfigurationClassParser.java:242) ~[spring-context-5.1.8.RELEASE.jar!/:5.1.8.RELEASE]
        at org.springframework.context.annotation.ConfigurationClassParser.parse(ConfigurationClassParser.java:199) ~[spring-context-5.1.8.RELEASE.jar!/:5.1.8.RELEASE]
        at org.springframework.context.annotation.ConfigurationClassParser.parse(ConfigurationClassParser.java:167) ~[spring-context-5.1.8.RELEASE.jar!/:5.1.8.RELEASE]
        ... 21 more

As I rerun mvn install, there is a ~80% chance the regenerated war will be healed as the project-api-1.0.jar file is present in the WEB-INF/lib as it should be, no empty directory.

-bash-4.2$ unzip -l project-impl/target/project-impl.war | grep project-api
  2740878  07-17-2019 11:49   WEB-INF/lib/project-api-1.0-SNAPSHOT.jar

I checked the api jar in the local repository after the wrong result mvn install run on the reactor parent, it is zip -T consistent. No error messages during the faulty .war generation. Checked a lot of forums, Spring Boot Maven Plugin's issues, found nothing relevant.

Did any of you bump into this issue? Any help would be appreciated.

PS: The full pom.xml files cannot fit into the 30000 StackOverflow character limit unfortunately.

Update 1

Maven debug log shows weird difference how the maven-war-plugin handles the project API jar and other jars:

...
[DEBUG] Processing: project-api-1.0-SNAPSHOT.jar
[DEBUG] Processing: cxf-xjc-runtime-3.3.0.jar
[DEBUG]  + WEB-INF/lib/cxf-xjc-runtime-3.3.0.jar has been copied.
[DEBUG] Processing: jakarta.xml.bind-api-2.3.2.jar
[DEBUG]  + WEB-INF/lib/jakarta.xml.bind-api-2.3.2.jar has been copied.
...

Update 2

I deducted the problem to the maven-war-plugin:3.2.2. The original war file has this problem before repackaging (see relevant post).

-bash-4.2$ unzip -l project-impl.war.original | grep project-api
        0  07-18-2019 02:58   WEB-INF/lib/project-api-1.0-SNAPSHOT.jar/

Update 3

void org.apache.maven.plugins.war.packaging.ArtifactsPackagingTask.copyFile( String sourceId, final WarPackagingContext context, final File file, String targetFilename ) throws IOException

The problem with the war packaging is that argument file does not point to the local repository's project-api-1.0-SNAPSHOT.jar, but to /absolute/path/project-parent/project-api/target/classes directory. Unsure why the argument is wrong though.

Answer 1

I figured it out, has nothing to do with spring-boot-maven-plugin. The problem was with maven-site-plugin and maven-war-plugin run after each other. Originally I invoked mvn clean site site:jar deploy from Jenkins to enable the deployment of the jar artifact with the site classifier too in one step. maven-site-plugin does something with the dependency Reactor artifacts (project-api-0.1.0-SNAPSHOT.jar here, so maven-war-plugin cannot refer to it for some reason. An empty directory will be stored in the war's WEB-INF/lib directory instead called project-api-0.1.0-SNAPSHOT.jar/. Weird.