feat(azure): add Azurite, EventHubs and ServiceBus in the new Azure module, deprecating the old Azurite module #3008

mdelapenya · 2025-03-05T23:38:36Z

What does this PR do?

This PR is doing two things:

adding a new modules package for azure, in which the new Azure containers will live, each on its own subpackage. In example modules/azure/eventhubs.
deprecating the old modules/azurite module, in favor of modules/azure/azurite. The feature has been migrated to the new module, and the deprecated code is linking to the new module, including the imports.

Regarding the new module containers, we are adding:

Azurite

As mentioned, deprecates the former Azurite module, using the same features. It also adds more robust wait strategies for the different service types (Bloc, Queue and Table).

EventHubs

The EventHubs emulator needs an Azurite container, so the module starts it under the hood. For that:

a network is created for the stack
an Azurite container is created in the network, with an alias
an EventHubs container is created in the network, with an alias

As a result, container communication happens using the aliases and the well-known ports. For the end-users, it's transparent, and they just have to talk to the EventHubs container using the provided API. The testable examples include how to connect to the container.

ServiceBus

The ServiceBus emulator needs a MSSQL Server container, so the module starts it under the hood. For that:

a network is created for the stack
an MSSQL Server container is created in the network, with an alias
an ServiceBus container is created in the network, with an alias

As a result, container communication happens using the aliases and the well-known ports. For the end-users, it's transparent, and they just have to talk to the ServiceBus container using the provided API. The testable examples include how to connect to the container.

Why is it important?

More cloud modules! this time to verify interactions with Azure Cloud.

Follow-ups

I'm not porting the CosmosDB emulator module, as it's broken in the upstream. I tried to run the tc-java implementation and it fails:

java.lang.IllegalStateException: [javax.net](http://javax.net/).ssl.SSLHandshakeException: Remote host terminated the handshake
	at org.testcontainers.containers.KeyStoreBuilder.buildByDownloadingCertificate(KeyStoreBuilder.java:33)
	at org.testcontainers.containers.CosmosDBEmulatorContainer.buildNewKeyStore(CosmosDBEmulatorContainer.java:37)
	at org.testcontainers.containers.CosmosDBEmulatorContainerTest.testWithCosmosClient(CosmosDBEmulatorContainerTest.java:50)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.testcontainers.containers.FailureDetectingExternalResource$1.evaluate(FailureDetectingExternalResource.java:29)
	at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
	at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
	at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.runTestClass(JUnitTestClassExecutor.java:112)
	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:58)
	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:40)
	at org.gradle.api.internal.tasks.testing.junit.AbstractJUnitTestClassProcessor.processTestClass(AbstractJUnitTestClassProcessor.java:54)
	at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:53)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
	at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
	at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:92)
	at jdk.proxy1/jdk.proxy1.$Proxy4.processTestClass(Unknown Source)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker$2.run(TestWorker.java:181)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker.executeAndMaintainThreadName(TestWorker.java:130)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:101)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:61)
	at org.gradle.process.internal.worker.child.ActionExecutionWorker.execute(ActionExecutionWorker.java:56)
	at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:122)
	at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:69)
	at [worker.org](http://worker.org/).gradle.process.internal.worker.GradleWorkerMain.run(GradleWorkerMain.java:69)
	at [worker.org](http://worker.org/).gradle.process.internal.worker.GradleWorkerMain.main(GradleWorkerMain.java:74)
Caused by: [javax.net](http://javax.net/).ssl.SSLHandshakeException: Remote host terminated the handshake

… old one

netlify · 2025-03-05T23:40:24Z

✅ Deploy Preview for testcontainers-go ready!

Name	Link
🔨 Latest commit	`6e5f9bf`
🔍 Latest deploy log	https://app.netlify.com/sites/testcontainers-go/deploys/67d1e65db871f00008f0e158
😎 Deploy Preview	https://deploy-preview-3008--testcontainers-go.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

* main: docs: refine texts on how to set the module image (testcontainers#3012) feat(modules): add dind module (testcontainers#3004) docs: correct container variable (testcontainers#3010) chore: update Weaviate version to v1.29.0 and Weaviate Go client to v5.0.2 (testcontainers#3006)

This reverts commit fbf78a0.

mdelapenya · 2025-03-11T11:41:24Z

.github/workflows/ci-test-go.yml

@@ -65,7 +65,7 @@ jobs:

      - name: ensure compilation
        working-directory: ./${{ inputs.project-directory }}
-        run: go build
+        run: go build ./...


Needed because the new azure module has subpackages for each container type.

@stevenh I'm tempted to follow the same approach for the GCP module: to have sibpackages for each container type, so they all have their Run function, and not RunBigQuery, etc

Sounds like a good idea to me.

mdelapenya · 2025-03-11T16:24:16Z

modules/azure/azurite/azurite.go

+// AzuriteContainer represents the Azurite container type used in the module
+type AzuriteContainer struct {
+	testcontainers.Container
+	Settings options


Keeping this public to avoid the breaking change. OTOH, if we want to do the BC now, I'm fine to move this settings to private.

I think we should take the hit now.

I think it's never used, so we can fairly remove it.

* main: chore(deps): bump github.com/opencontainers/image-spec from 1.1.0 to 1.1.1, dario.cat/mergo from 1.0.0 to 1.0.1 (testcontainers#3030) chore(deps): bump github/codeql-action from 3.28.0 to 3.28.11 (testcontainers#3014) chore(deps): bump ossf/scorecard-action from 2.4.0 to 2.4.1 (testcontainers#3013) chore: readd dependabot, including a way to refresh the project files for all the modules (testcontainers#2997)

stevenh · 2025-03-12T16:36:46Z

.github/workflows/ci-test-go.yml

@@ -65,7 +65,7 @@ jobs:

      - name: ensure compilation
        working-directory: ./${{ inputs.project-directory }}
-        run: go build
+        run: go build ./...


Sounds like a good idea to me.

modules/azure/azurite/azurite.go

stevenh · 2025-03-12T16:39:04Z

modules/azure/azurite/azurite.go

+
+const (
+	// BlobPort is the default port used by Azurite
+	BlobPort = "10000/tcp"


question: do these need to be exported?

I kept the same state as in the original, deprecated modules/azurite container.

stevenh · 2025-03-12T16:39:36Z

modules/azure/azurite/azurite.go

+// AzuriteContainer represents the Azurite container type used in the module
+type AzuriteContainer struct {
+	testcontainers.Container
+	Settings options


I think we should take the hit now.

stevenh · 2025-03-12T16:40:44Z

modules/azure/azurite/azurite.go

+// MustServiceURL returns the URL of the given service, panics if an error occurs
+func (c *AzuriteContainer) MustServiceURL(ctx context.Context, srv Service) string {


question: do we need the must, I would remove if not.

It's used in the examples, or you mean we should not add this kind of Must methods.

modules/azure/eventhubs/options.go

modules/azure/servicebus/options.go

modules/azure/servicebus/servicebus.go

stevenh · 2025-03-12T17:00:22Z

modules/azure/servicebus/servicebus.go

+	return fmt.Sprintf(connectionStringFormat, hostPort, defaultSharedAccessKeyName, defaultSharedAccessKey), nil
+}
+
+// MustConnectionString returns the connection string for the eventhubs container,


question: do we need?

mdelapenya added 4 commits March 3, 2025 18:25

feat: move Azurite container to the new Azure module, deprecating the…

2e46706

… old one

feat: add eventhubs module

f81fb5d

fix: right import

3a5354c

chore: use tabs for Azurite code examples

733963d

mdelapenya added 16 commits March 6, 2025 09:29

fix: azure has no go files

69d080b

chore: rename

25da330

feat: add a way to share the MSSQL Server password in the container

056d519

feat: add servicebus emulator module

41a3225

chore: add wait strategy for the eventhubs container

de0bcdc

chore: add more robust wait strategies for the azurite container

de62a49

chore: add more tests for the azurite module

87b6843

chore: run mod tidy

5b2c75a

fix: use the new azurite imports

74ab1a5

fix: lint

0379d78

chore: remove unused import

80480d4

chore: pin mssql version

71ea08b

fix: mod tidy

05ccaad

wip: cosmosdb container

fbf78a0

Revert "wip: cosmosdb container"

fc1e31c

This reverts commit fbf78a0.

mdelapenya commented Mar 11, 2025

View reviewed changes

chore: retry in example tests

5a26e07

mdelapenya marked this pull request as ready for review March 11, 2025 13:30

mdelapenya requested a review from a team as a code owner March 11, 2025 13:30

mdelapenya requested review from eddumelendez and stevenh March 11, 2025 13:30

mdelapenya self-assigned this Mar 11, 2025

mdelapenya added the feature New functionality or new behaviors on the existing one label Mar 11, 2025

mdelapenya changed the title ~~feat: azure modules~~ feat: add Azurite, EventHubs and ServiceBus in the new Azure module, deprecating the old Azurite module Mar 11, 2025

mdelapenya changed the title ~~feat: add Azurite, EventHubs and ServiceBus in the new Azure module, deprecating the old Azurite module~~ feat(azure): add Azurite, EventHubs and ServiceBus in the new Azure module, deprecating the old Azurite module Mar 11, 2025

mdelapenya added 2 commits March 11, 2025 17:10

chore: docs and wrap errors

0b45138

docs: add method docs for MSSQL

192f58e

mdelapenya mentioned this pull request Mar 11, 2025

feat(gcloud): add option to run firestore in datastore mode #3009

Open

mdelapenya commented Mar 11, 2025

View reviewed changes

mdelapenya added 6 commits March 12, 2025 11:41

chore: refine ServiceBus wait strategies

b07a621

chore: customiser MSSQL Server

fcd6383

chore: customise Azurite container

3f4ebc7

fix: add azure to dependabot

751b652

chore: refine EventHubs wait strategies

7195326

stevenh requested changes Mar 12, 2025

View reviewed changes

mdelapenya added 8 commits March 12, 2025 18:26

chore: handle EULA variable correctly

4f1ede5

docs: proper comments

75a15d0

chore!: remove settings, as it not used

98ad9b3

fixup comments

9a4fde1

chore: options returns error

4529186

chore: do not leak network

03e5d4d

chore: simplify wait strategies

e64edb7

fix: wait for all does not handle nil waits

6e5f9bf

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(azure): add Azurite, EventHubs and ServiceBus in the new Azure module, deprecating the old Azurite module #3008

feat(azure): add Azurite, EventHubs and ServiceBus in the new Azure module, deprecating the old Azurite module #3008

mdelapenya commented Mar 5, 2025 •

edited

Loading

netlify bot commented Mar 5, 2025 •

edited

Loading

mdelapenya Mar 11, 2025

stevenh Mar 12, 2025

mdelapenya Mar 11, 2025

stevenh Mar 12, 2025

mdelapenya Mar 12, 2025

stevenh Mar 12, 2025

stevenh Mar 12, 2025

mdelapenya Mar 12, 2025

stevenh Mar 12, 2025

stevenh Mar 12, 2025

mdelapenya Mar 12, 2025

stevenh Mar 12, 2025

		// MustServiceURL returns the URL of the given service, panics if an error occurs
		func (c *AzuriteContainer) MustServiceURL(ctx context.Context, srv Service) string {

feat(azure): add Azurite, EventHubs and ServiceBus in the new Azure module, deprecating the old Azurite module #3008

Are you sure you want to change the base?

feat(azure): add Azurite, EventHubs and ServiceBus in the new Azure module, deprecating the old Azurite module #3008

Conversation

mdelapenya commented Mar 5, 2025 • edited Loading

What does this PR do?

Azurite

EventHubs

ServiceBus

Why is it important?

Follow-ups

netlify bot commented Mar 5, 2025 • edited Loading

✅ Deploy Preview for testcontainers-go ready!

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mdelapenya commented Mar 5, 2025 •

edited

Loading

netlify bot commented Mar 5, 2025 •

edited

Loading