[Bug]: Update to 1.19.4 breaks builds

[bonndan]

Module

Core

Testcontainers version

1.19.4

Using the latest Testcontainers version?

Yes

Host OS

Linux, Windows

Host Arch

x86

Docker version

Docker Desktop 4.5.1 (74721), other

What happened?

Updating testcontainers to 1.19.4 crashes multiple pipelines in different projects and on different enviroments.

Relevant log output

java.lang.NoSuchMethodError: 'void org.apache.commons.compress.archivers.tar.TarArchiveOutputStream.putArchiveEntry(org.apache.commons.compress.archivers.tar.TarArchiveEntry)'
	at org.testcontainers.images.builder.Transferable.transferTo(Transferable.java:80)
	at org.testcontainers.containers.ContainerState.copyFileToContainer(ContainerState.java:355)
	at org.testcontainers.containers.KafkaContainer.containerIsStarting(KafkaContainer.java:199)
	at org.testcontainers.containers.GenericContainer.containerIsStarting(GenericContainer.java:715)
	at org.testcontainers.containers.GenericContainer.tryStart(GenericContainer.java:492)
	at org.testcontainers.containers.GenericContainer.lambda$doStart$0(GenericContainer.java:357)

Additional Information

No response

[d3ns0n]

I do have the same problem on MacOS

[eddumelendez]

It fails because of the change in commons-compress. See diff apache/commons-compress@rel/commons-compress-1.24.0...rel/commons-compress-1.25.0#diff-19b93108b5b4d5b1f4826b7dce61b416beb5a02d928ae8868412d971601ab201R568. If the version is updated then all should be fine.

[anoopnarang]

In my project, commons-compress brought in via testcontainers (1.25) was conflicting with the one brought in via hadoop-commons (1.21). Making an exclusion on hadoop-commons fixes the issue.

But it kind of brings another issue ( tentative ), as excluding a transitive dependency in main to fix something for test.

[wilkinsona]

We've also been affected by this in Spring Boot's build. There's an awkward combination of main and test dependency versions and dealing with deprecation warnings that I haven't managed to find a good way to address.

Could Testcontainers shade commons-compress?

[d3ns0n]

@eddumelendez @anoopnarang thank you for the hint. I added commons-compress:1.25.0 as an explicit dependency to my project. Now everything is fine again 👍

[eddumelendez]

@wilkinsona sorry about the issue. So far, we can not shade the dependency because it is part of the public API 😢 . What I was thinking is downgrade the dependency so public void putArchiveEntry(final ArchiveEntry archiveEntry) still supports TarArchiveEntry. I've tested some examples with combination of Testcontainers 1.19.3 (with commons-compress 1.23.0) and commons-compress 1.25.0 and works. LMK if this would help.

[wilkinsona]

Hey, Eddu. We did manage to work around this in the end in our build with some Gradle gymnastics. That said, a downgrade would help us and allow us to simplify the build a bit. I think it may also help Maven users where version conflicts are resolved by nearest to the root of the graph rather than the highest wins that Gradle uses. This means that Maven projects that use Commons Compress and Maven are more likely to end up with an older version of Commons Compress with which Testcontainers isn't compatible. Perhaps the upgrade to the latest version could then be done in Testcontainers 1.20?

[eddumelendez]

We are going to release a new version downgrading commons-compress in order to avoid upgrading commons-compress dependency just for Testcontainers usage. If the upgrade is needed because of other reasons, new Testcontainers will still play well with it for the reasons explained in previous comment. Also, we want to reduce the impact in projects such as Spring Boot and others.

Thanks everyone!

[eddumelendez]

Testcontainers 1.19.5 is available https://repo1.maven.org/maven2/org/testcontainers/testcontainers-bom/1.19.5/