Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Gradle crashes when FileLockCommunicator cannot open a DatagramSocket despite --offline flag #25762

Closed
marcprux opened this issue Jul 13, 2023 · 5 comments
Labels
a:feature A new functionality closed:not-fixed Indicates the issue was not fixed and is not planned to be in:daemon in:file-locking in:workers

Comments

@marcprux
Copy link

marcprux commented Jul 13, 2023

Expected Behavior

Gradle should be able to run offline and in network-restricted environments, but despite the --offline flag, it still attempts to start a datagram server. This should be disabled when the --offline flags is specified and the daemon is not being used. Or there should be some other option or environment variable to prevent the FileLockCommunicator from being used in isolated environments.

Current Behavior

Any build where the process cannot open a datagram socket will fail, even when the daemon is disabled. This can be reproduced on macOS by running the gradle process with the sandbox-exec tool configured to deny the network-inbound permission:

marc@zap gradlenet % GRADLE_OPTS="-Dorg.gradle.daemon=false -Xmx2048m" /usr/bin/sandbox-exec -p '(version 1)
(deny default)
(import "system.sb")
(allow file-read*)
(allow file-write*)
(allow process*)
(allow network-outbound)
(deny network-inbound)
' gradle build --console=plain --stacktrace --offline --no-daemon --no-build-cache --no-configuration-cache --no-watch-fs

FAILURE: Build failed with an exception.

* What went wrong:
Gradle could not start your build.
> Cannot create service of type BuildSessionActionExecutor using method LauncherServices$ToolingBuildSessionScopeServices.createActionExecutor() as there is a problem with parameter #21 of type FileSystemWatchingInformation.
   > Cannot create service of type BuildLifecycleAwareVirtualFileSystem using method VirtualFileSystemServices$GradleUserHomeServices.createVirtualFileSystem() as there is a problem with parameter #7 of type GlobalCacheLocations.
      > Cannot create service of type GlobalCacheLocations using method GradleUserHomeScopeServices.createGlobalCacheLocations() as there is a problem with parameter #1 of type List<GlobalCache>.
         > Could not create service of type FileAccessTimeJournal using GradleUserHomeScopeServices.createFileAccessTimeJournal().
            > java.net.SocketException: Operation not permitted

* Try:
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
> Get more help at https://help.gradle.org.

* Exception is:
org.gradle.initialization.exception.InitializationException: Gradle could not start your build.
	at org.gradle.initialization.exception.DefaultExceptionAnalyser.collectFailures(DefaultExceptionAnalyser.java:58)
	at org.gradle.initialization.exception.MultipleBuildFailuresExceptionAnalyser.transform(MultipleBuildFailuresExceptionAnalyser.java:55)
	at org.gradle.initialization.exception.MultipleBuildFailuresExceptionAnalyser.transform(MultipleBuildFailuresExceptionAnalyser.java:38)
	at org.gradle.initialization.exception.StackTraceSanitizingExceptionAnalyser.transform(StackTraceSanitizingExceptionAnalyser.java:33)
	at org.gradle.tooling.internal.provider.SessionFailureReportingActionExecuter.execute(SessionFailureReportingActionExecuter.java:59)
	at org.gradle.tooling.internal.provider.SessionFailureReportingActionExecuter.execute(SessionFailureReportingActionExecuter.java:38)
	at org.gradle.tooling.internal.provider.SetupLoggingActionExecuter.execute(SetupLoggingActionExecuter.java:47)
	at org.gradle.tooling.internal.provider.SetupLoggingActionExecuter.execute(SetupLoggingActionExecuter.java:31)
	at org.gradle.launcher.cli.RunBuildAction.run(RunBuildAction.java:57)
	at org.gradle.internal.Actions$RunnableActionAdapter.execute(Actions.java:167)
	at org.gradle.launcher.cli.DefaultCommandLineActionFactory$ParseAndBuildAction.execute(DefaultCommandLineActionFactory.java:259)
	at org.gradle.launcher.cli.DefaultCommandLineActionFactory$ParseAndBuildAction.execute(DefaultCommandLineActionFactory.java:230)
	at org.gradle.launcher.cli.DebugLoggerWarningAction.execute(DebugLoggerWarningAction.java:74)
	at org.gradle.launcher.cli.DebugLoggerWarningAction.execute(DebugLoggerWarningAction.java:30)
	at org.gradle.launcher.cli.WelcomeMessageAction.execute(WelcomeMessageAction.java:96)
	at org.gradle.launcher.cli.WelcomeMessageAction.execute(WelcomeMessageAction.java:40)
	at org.gradle.launcher.cli.NativeServicesInitializingAction.execute(NativeServicesInitializingAction.java:44)
	at org.gradle.launcher.cli.NativeServicesInitializingAction.execute(NativeServicesInitializingAction.java:26)
	at org.gradle.launcher.cli.ExceptionReportingAction.execute(ExceptionReportingAction.java:41)
	at org.gradle.launcher.cli.ExceptionReportingAction.execute(ExceptionReportingAction.java:26)
	at org.gradle.launcher.cli.DefaultCommandLineActionFactory$WithLogging.execute(DefaultCommandLineActionFactory.java:361)
	at org.gradle.launcher.Main.doAction(Main.java:35)
	at org.gradle.launcher.bootstrap.EntryPoint.run(EntryPoint.java:50)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
	at org.gradle.launcher.bootstrap.ProcessBootstrap.runNoExit(ProcessBootstrap.java:60)
	at org.gradle.launcher.bootstrap.ProcessBootstrap.run(ProcessBootstrap.java:37)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
	at org.gradle.launcher.GradleMain.main(GradleMain.java:34)
Caused by: org.gradle.internal.service.ServiceCreationException: Cannot create service of type BuildSessionActionExecutor using method LauncherServices$ToolingBuildSessionScopeServices.createActionExecutor() as there is a problem with parameter #21 of type FileSystemWatchingInformation.
	at org.gradle.internal.service.DefaultServiceRegistry$FactoryService.bind(DefaultServiceRegistry.java:815)
	at org.gradle.internal.service.DefaultServiceRegistry$SingletonService.prepare(DefaultServiceRegistry.java:700)
	at org.gradle.internal.service.DefaultServiceRegistry$SingletonService.getService(DefaultServiceRegistry.java:723)
	at org.gradle.internal.service.DefaultServiceRegistry$OwnServices.getService(DefaultServiceRegistry.java:450)
	at org.gradle.internal.service.DefaultServiceRegistry$CompositeServiceProvider.getService(DefaultServiceRegistry.java:1008)
	at org.gradle.internal.service.DefaultServiceRegistry.find(DefaultServiceRegistry.java:1094)
	at org.gradle.internal.service.DefaultServiceRegistry.getService(DefaultServiceRegistry.java:329)
	at org.gradle.internal.service.DefaultServiceRegistry.find(DefaultServiceRegistry.java:323)
	at org.gradle.internal.service.DefaultServiceRegistry.get(DefaultServiceRegistry.java:308)
	at org.gradle.internal.service.DefaultServiceRegistry.get(DefaultServiceRegistry.java:303)
	at org.gradle.internal.session.DefaultBuildSessionContext.execute(DefaultBuildSessionContext.java:46)
	at org.gradle.tooling.internal.provider.BuildSessionLifecycleBuildActionExecuter$ActionImpl.apply(BuildSessionLifecycleBuildActionExecuter.java:100)
	at org.gradle.tooling.internal.provider.BuildSessionLifecycleBuildActionExecuter$ActionImpl.apply(BuildSessionLifecycleBuildActionExecuter.java:88)
	at org.gradle.internal.session.BuildSessionState.run(BuildSessionState.java:69)
	at org.gradle.tooling.internal.provider.BuildSessionLifecycleBuildActionExecuter.execute(BuildSessionLifecycleBuildActionExecuter.java:62)
	at org.gradle.tooling.internal.provider.BuildSessionLifecycleBuildActionExecuter.execute(BuildSessionLifecycleBuildActionExecuter.java:41)
	at org.gradle.tooling.internal.provider.StartParamsValidatingActionExecuter.execute(StartParamsValidatingActionExecuter.java:64)
	at org.gradle.tooling.internal.provider.StartParamsValidatingActionExecuter.execute(StartParamsValidatingActionExecuter.java:32)
	at org.gradle.tooling.internal.provider.SessionFailureReportingActionExecuter.execute(SessionFailureReportingActionExecuter.java:50)
	... 23 more
Caused by: org.gradle.internal.service.ServiceCreationException: Cannot create service of type BuildLifecycleAwareVirtualFileSystem using method VirtualFileSystemServices$GradleUserHomeServices.createVirtualFileSystem() as there is a problem with parameter #7 of type GlobalCacheLocations.
	at org.gradle.internal.service.DefaultServiceRegistry$FactoryService.bind(DefaultServiceRegistry.java:815)
	at org.gradle.internal.service.DefaultServiceRegistry$SingletonService.prepare(DefaultServiceRegistry.java:700)
	at org.gradle.internal.service.DefaultServiceRegistry$SingletonService.getService(DefaultServiceRegistry.java:723)
	at org.gradle.internal.service.DefaultServiceRegistry$OwnServices.getService(DefaultServiceRegistry.java:450)
	at org.gradle.internal.service.DefaultServiceRegistry$CompositeServiceProvider.getService(DefaultServiceRegistry.java:1008)
	at org.gradle.internal.service.DefaultServiceRegistry$ParentServices.getService(DefaultServiceRegistry.java:1062)
	at org.gradle.internal.service.DefaultServiceRegistry$CompositeServiceProvider.getService(DefaultServiceRegistry.java:1008)
	at org.gradle.internal.service.DefaultServiceRegistry$CompositeServiceProvider.getService(DefaultServiceRegistry.java:1008)
	at org.gradle.internal.service.DefaultServiceRegistry.find(DefaultServiceRegistry.java:1094)
	at org.gradle.internal.service.DefaultServiceRegistry.access$1600(DefaultServiceRegistry.java:83)
	at org.gradle.internal.service.DefaultServiceRegistry$FactoryService.bind(DefaultServiceRegistry.java:813)
	... 41 more
Caused by: org.gradle.internal.service.ServiceCreationException: Cannot create service of type GlobalCacheLocations using method GradleUserHomeScopeServices.createGlobalCacheLocations() as there is a problem with parameter #1 of type List<GlobalCache>.
	at org.gradle.internal.service.DefaultServiceRegistry$FactoryService.bind(DefaultServiceRegistry.java:815)
	at org.gradle.internal.service.DefaultServiceRegistry$SingletonService.prepare(DefaultServiceRegistry.java:700)
	at org.gradle.internal.service.DefaultServiceRegistry$SingletonService.getService(DefaultServiceRegistry.java:723)
	at org.gradle.internal.service.DefaultServiceRegistry$OwnServices.getService(DefaultServiceRegistry.java:450)
	at org.gradle.internal.service.DefaultServiceRegistry$CompositeServiceProvider.getService(DefaultServiceRegistry.java:1008)
	at org.gradle.internal.service.DefaultServiceRegistry.find(DefaultServiceRegistry.java:1094)
	at org.gradle.internal.service.DefaultServiceRegistry.access$1600(DefaultServiceRegistry.java:83)
	at org.gradle.internal.service.DefaultServiceRegistry$FactoryService.bind(DefaultServiceRegistry.java:813)
	... 51 more
Caused by: org.gradle.internal.service.ServiceCreationException: Could not create service of type FileAccessTimeJournal using GradleUserHomeScopeServices.createFileAccessTimeJournal().
	at org.gradle.internal.service.DefaultServiceRegistry$FactoryMethodService.invokeMethod(DefaultServiceRegistry.java:913)
	at org.gradle.internal.service.DefaultServiceRegistry$FactoryService.createServiceInstance(DefaultServiceRegistry.java:838)
	at org.gradle.internal.service.DefaultServiceRegistry$ManagedObjectServiceProvider.getInstance(DefaultServiceRegistry.java:623)
	at org.gradle.internal.service.DefaultServiceRegistry$SingletonService.get(DefaultServiceRegistry.java:686)
	at org.gradle.internal.service.DefaultServiceRegistry$FactoryService.assembleParameters(DefaultServiceRegistry.java:851)
	at org.gradle.internal.service.DefaultServiceRegistry$FactoryService.createServiceInstance(DefaultServiceRegistry.java:837)
	at org.gradle.internal.service.DefaultServiceRegistry$ManagedObjectServiceProvider.getInstance(DefaultServiceRegistry.java:623)
	at org.gradle.internal.service.DefaultServiceRegistry$SingletonService.get(DefaultServiceRegistry.java:686)
	at org.gradle.internal.service.DefaultServiceRegistry$FactoryService.assembleParameters(DefaultServiceRegistry.java:851)
	at org.gradle.internal.service.DefaultServiceRegistry$FactoryService.createServiceInstance(DefaultServiceRegistry.java:837)
	at org.gradle.internal.service.DefaultServiceRegistry$ManagedObjectServiceProvider.getInstance(DefaultServiceRegistry.java:623)
	at org.gradle.internal.service.DefaultServiceRegistry$SingletonService.get(DefaultServiceRegistry.java:686)
	at org.gradle.internal.service.DefaultServiceRegistry.getCollectionService(DefaultServiceRegistry.java:1141)
	at org.gradle.internal.service.DefaultServiceRegistry.getCollectionService(DefaultServiceRegistry.java:1123)
	at org.gradle.internal.service.DefaultServiceRegistry.find(DefaultServiceRegistry.java:1086)
	at org.gradle.internal.service.DefaultServiceRegistry.access$1600(DefaultServiceRegistry.java:83)
	at org.gradle.internal.service.DefaultServiceRegistry$FactoryService.bind(DefaultServiceRegistry.java:813)
	... 58 more
Caused by: org.gradle.api.UncheckedIOException: java.net.SocketException: Operation not permitted
	at org.gradle.internal.UncheckedException.throwAsUncheckedException(UncheckedException.java:62)
	at org.gradle.internal.UncheckedException.throwAsUncheckedException(UncheckedException.java:41)
	at org.gradle.cache.internal.locklistener.FileLockCommunicator.<init>(FileLockCommunicator.java:51)
	at org.gradle.cache.internal.locklistener.DefaultFileLockContentionHandler.getCommunicator(DefaultFileLockContentionHandler.java:263)
	at org.gradle.cache.internal.locklistener.DefaultFileLockContentionHandler.reservePort(DefaultFileLockContentionHandler.java:255)
	at org.gradle.cache.internal.DefaultFileLockManager.lock(DefaultFileLockManager.java:109)
	at org.gradle.cache.internal.LockOnDemandCrossProcessCacheAccess.incrementLockCount(LockOnDemandCrossProcessCacheAccess.java:106)
	at org.gradle.cache.internal.LockOnDemandCrossProcessCacheAccess.acquireFileLock(LockOnDemandCrossProcessCacheAccess.java:168)
	at org.gradle.cache.internal.DefaultCacheCoordinator.onStartWork(DefaultCacheCoordinator.java:398)
	at org.gradle.cache.internal.DefaultCacheCoordinator.useCache(DefaultCacheCoordinator.java:239)
	at org.gradle.cache.internal.DefaultCacheCoordinator.useCache(DefaultCacheCoordinator.java:229)
	at org.gradle.cache.internal.DefaultCacheCoordinator.newCache(DefaultCacheCoordinator.java:320)
	at org.gradle.cache.internal.DefaultPersistentDirectoryStore.createIndexedCache(DefaultPersistentDirectoryStore.java:173)
	at org.gradle.cache.internal.DefaultCacheFactory$ReferenceTrackingCache.createIndexedCache(DefaultCacheFactory.java:199)
	at org.gradle.api.internal.changedetection.state.DefaultFileAccessTimeJournal.<init>(DefaultFileAccessTimeJournal.java:55)
	at org.gradle.internal.service.scopes.GradleUserHomeScopeServices.createFileAccessTimeJournal(GradleUserHomeScopeServices.java:227)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
	at org.gradle.internal.reflect.JavaMethod.invoke(JavaMethod.java:125)
	at org.gradle.internal.service.ReflectionBasedServiceMethod.invoke(ReflectionBasedServiceMethod.java:34)
	at org.gradle.internal.service.DefaultServiceRegistry$FactoryMethodService.invokeMethod(DefaultServiceRegistry.java:911)
	... 74 more
Caused by: java.net.SocketException: Operation not permitted
	at org.gradle.cache.internal.locklistener.FileLockCommunicator.<init>(FileLockCommunicator.java:49)
	... 91 more


BUILD FAILED in 548ms

Context (optional)

This issue prevents us from using Gradle in a secure network-constrained environment. When "--offline" is specified, it should really be offline.

Note that if you swap the permissions (i.e., (deny network-outbound) and (allow network-inbound)), then the build does complete successfully (provided that all the project's network dependencies are downloaded and available). So while it is possible to run Gradle without requesting anything from the network, it cannot be run without being able to listen on a port.

Steps to Reproduce

macOS 13.4.1, Homebrew, openjdk version "20.0.1" 2023-04-18

  1. Set the right environment flags to allow Gradle to respect the --no-daemon flag. E.g.: export GRADLE_OPTS="-Dorg.gradle.daemon=false -Xmx2048m"
  2. Create a fresh project: gradle init --type kotlin-library --dsl kotlin --console plain --no-daemon --offline --project-name=ExampleDemo --package=example.demo --test-framework=kotlintest
  3. Run the gradle build in a network-constrained sandbox:
/usr/bin/sandbox-exec -p '(version 1)
(deny default)
(import "system.sb")
(allow file-read*)
(allow file-write*)
(allow process*)
(allow network-outbound)
(deny network-inbound)
' gradle build --console=plain --stacktrace --offline --no-daemon --no-build-cache --no-configuration-cache --no-watch-fs

Gradle version

8.2.1

Build scan URL (optional)

No response

Your Environment (optional)

No response

@ov7a
Copy link
Member

ov7a commented Jul 14, 2023

Thank you for your interest in Gradle!

This issue needs a decision from the team responsible for that area. They have been informed. Response time may vary.


The system uses file sockets which are required to communicate even in a single-daemon environment. As far as I know, workers use it for communicating around locked resources.

@ov7a ov7a added in:daemon a:feature A new functionality in:workers in:file-locking 👋 team-triage Issues that need to be triaged by a specific team and removed a:bug to-triage labels Jul 14, 2023
@big-guy
Copy link
Member

big-guy commented Aug 15, 2023

Would something like is mentioned here work for you? ocaml/opam#5221 (comment)

(deny network*)
(allow network-inbound (local ip "localhost:*"))
(allow network* (remote ip "localhost:*"))

Gradle uses network-based IPC to communicate to workers (like compiler daemons for Kotlin or Java) or tests, so there needs to be some way to communicate back to the Gradle daemon.

@ov7a
Copy link
Member

ov7a commented Sep 12, 2023

Given that there is no reply, I assume that this issue can be closed. It may be reopened later if we receive more use cases and details.

@ov7a ov7a closed this as not planned Won't fix, can't repro, duplicate, stale Sep 12, 2023
@ov7a ov7a added closed:not-fixed Indicates the issue was not fixed and is not planned to be and removed 👋 team-triage Issues that need to be triaged by a specific team labels Sep 12, 2023
@marcprux
Copy link
Author

No, the process sandboxing cannot be altered, as it is part of a secure build system. When gradle is run with "--offline", then network access should not be attempted. The fact that it does is clearly a bug.

@ov7a ov7a reopened this Sep 12, 2023
@ov7a ov7a added 👋 team-triage Issues that need to be triaged by a specific team and removed closed:not-fixed Indicates the issue was not fixed and is not planned to be labels Sep 12, 2023
@lptr
Copy link
Member

lptr commented Sep 28, 2023

Sorry for the long delay in responding. This works as designed, as the goal of --offline is to prevent Gradle from downloading external resources from outside the machine, not to prevent it from using IP completely.

That said, you have a valid use case that Gradle could support. Doing so would require significant work, as we'd need to replace our current IP-based cross-process communication with other means on all the supported platforms. As the feature at hand does not fall into our current priorities, I don't expect us to work on this on the foreseeable future.

If you believe this is important for Gradle to support, a few ways you can help make it happen would be:

  • Create a separate issue about some sort of "networkless" mode.
  • Provide a PR that would implement your expectations.
  • Present some evidence that this affects a wider range of users than we currently think.

@lptr lptr closed this as not planned Won't fix, can't repro, duplicate, stale Sep 28, 2023
@ov7a ov7a added closed:not-fixed Indicates the issue was not fixed and is not planned to be and removed 👋 team-triage Issues that need to be triaged by a specific team labels Sep 28, 2023
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
a:feature A new functionality closed:not-fixed Indicates the issue was not fixed and is not planned to be in:daemon in:file-locking in:workers
Projects
None yet
Development

No branches or pull requests

4 participants