sbt unable to import two elasticsearch libraries of different version in project

Question

I am trying to connect with two elasticsearch clusters using scala code and query elasticsearch from them. using following libraries and code in build.sbt of scala project:

libraryDependencies +="org.elasticsearch" % "elasticsearch" % "7.2.0"
libraryDependencies += "org.elasticsearch.client" % "elasticsearch-rest-high-level-client" % "7.2.0"

AND

val elastic4sVersion = "6.2.8"
libraryDependencies ++= Seq(
  "com.sksamuel.elastic4s" %% "elastic4s-core" % elastic4sVersion,
  // for the http client
  "com.sksamuel.elastic4s" %% "elastic4s-http" % elastic4sVersion,
)

These has a common client library, which gets missing when building. I can see either 6.x or 7.x but not both. I tried the shading approach

assemblyShadeRules in assembly := Seq(
  ShadeRule.rename("org.elasticsearch.client.**" -> "my_conf.@1")
    .inLibrary("org.elasticsearch.client" % "elasticsearch-rest-high-level-client" % "7.2.0")
    .inAll
)

assemblyShadeRules in assembly := Seq(
  ShadeRule.rename("org.elasticsearch.client.**" -> "my_conf_1.@1")
    .inLibrary("org.elasticsearch.client" % "elasticsearch-rest-high-level-client" % "6.2.2")
    .inAll
)
assemblyShadeRules in assembly := Seq(
  ShadeRule.rename("org.elasticsearch.elasticsearch.**" -> "my_configuration.@1")
    .inLibrary("org.elasticsearch" % "elasticsearch" % "7.2.0")
      .inAll
)

but I can not can not get the shaded versions available and get error when trying to import them in projet references.

Answer 1

Well, on JVM you can have only one version of the same .class in a classpath, so every build tool will respect that.

sbt will make sure there will be only one version of the library available in the project (unless you set it explicitly, I would assume it picks the highest version number of all conflicting versions) so if you need to have a library that is used by both dependencies I would look for versions that use the same version of the dependency. (Or explicitly override the version and use sbt-mima with sbt-missinglink to check that this change didn't broke anything).

From what I see the simplest way would be to use the table on projects page (together with maven) to pick up the right elastic4s version for the elasticsearch version you want to use.