-
-
Notifications
You must be signed in to change notification settings - Fork 5.7k
New issue
Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? # to your account
The status of gitea 1.17 service is always "starting" in windows when SSH is Disabled #20609
Comments
Logs if you think there is a bug. https://docs.gitea.io/en-us/seek-help/ Or, do you mean that you think that the status must be "running" instead of "starting"? |
I have just upgraded to 1.17 windows go-git as well (windows 10), and have the same issue. The windows service is stuck on starting and never progresses to running, leaving it where we can neither stop, start or restart it. However the app is running and responsive. log snippet from 1.16.9 prior to upgrade
and log snippet after upgrade to 1.17
|
After reading the code, I think that the gitea.exe is using Windows Service API, but it might be broken recently 😂 |
Me too, I just upgraded from 1.16 to 1.17 windows go-git, and I have the same issue. Gitea service is stuck on starting and never progresses to running, although it seems does not matter, but I can not close or shutdown app, except I kill the progress manully in windows task manager. |
None of this code has changed on behalf of Gitea and it is doing precisely the same thing that it has always done - something must have changed in our dependent upstream code. |
:( so probably not resolved for 1.17.1 |
IsAnInteractiveSession() is depreciated, unsure of this is related |
yes but the problem is that IsWindowsService has a different permissions profile and won't work for most users. see #15454 Now it's possible that something has changed. Fundamentally though I don't run Windows so I can't debug this. You will need to compile and test versions to try to figure this out. |
OK, so looking at your log fragment I am not seeing:
So we should just examine this code a bit more clearly: gitea/modules/graceful/manager_windows.go Lines 89 to 131 in 3d12018
We know we get to: gitea/modules/graceful/manager_windows.go Line 100 in 3d12018
because we don't see: gitea/modules/graceful/manager_windows.go Line 92 in 3d12018
So this leads to Question 1 1. Is
|
log.Trace("Not running a service ... using the debug SVC manager") |
So if we're not blocked at 100 svc.Run(...)
should call the our Execute
function. Leading to Question 2
2. Is Execute
ever run
Assuming Execute
is run then we should see:
gitea/modules/graceful/manager_windows.go
Line 122 in 3d12018
log.Trace("Awaiting server start-up") |
so that would mean we were blocked at:
gitea/modules/graceful/manager_windows.go
Lines 116 to 120 in 3d12018
if setting.StartupTimeout > 0 { | |
status <- svc.Status{State: svc.StartPending} | |
} else { | |
status <- svc.Status{State: svc.StartPending, WaitHint: uint32(setting.StartupTimeout / time.Millisecond)} | |
} |
Now we're lacking in Trace statements here but we could easily check this with pprof.
So I think it would be very helpful to give us the cpuprofile from pprof. So please enable pprof and give us the output of cpuprofile.
In app.ini add:
[server]
ENABLE_PPROF=true
Start Gitea and then run:
curl -o cpuprofile.out http://localhost:6060/debug/pprof/profile?seconds=15
Paste the outputed cpuprofile.out up here. (You may need to zip it.)
If you want to interrogate this yourself, this command will open a webbrowser to examine it:
go tool pprof --http=:6061 http://localhost:6060/debug/pprof/profile?seconds=15
gitea/modules/graceful/manager_windows.go Lines 116 to 120 in 3d12018
I am probably wrong... But that logic looks backwards The "else" clause is executed if setting.StartupTimeout is less than or equal to zero, yet the waitHint argument uses this value. The svc.status calls look the wrong way round. Probably not related to this, unless a subtle change in golang-1.18 (used for gitea-1.17?) has a change in behaviour |
yeah that does look like a mistake but ... it's been like that for a long time - so I don't think it can be the cause necessarily. What do you mean about the status calls being the wrong way round? The PPROF profile I asked for would be the best thing to give us to work out where and if things are blocked. |
Just that the if/else calls look the wrong way round. I'll get pprof data tomorrow evening. I am predominantly a Linux user but do baby sit a windows machine that flagged this |
Yeah, it's really weird that has sneaked through and that I've made such a weird mistake. I just don't understand how it could have been there for so long. Feel free to send a PR to switch it around but it would probably be better to do it at the same time as we figure out what is causing the problem. |
I've attached my output from gitea 1.17 on windows 10 as a service |
I have attached my logs. I also upgraded to gitea-1.17.1 |
Wait, the logs provided have so far been set to INFO but to support the postulation made in #20609 (comment) they need to be set to TRACE. |
ah sorry it appears I need the goroutine profile... curl -o goroutine.out http://localhost:6060/debug/pprof/goroutine?debug=2 That should be a text readable file. If you are running 1.18/dev you could simply go to |
Yup I need trace logs for this. You should be able to set the |
new logs
With logs set to TRACE we are not seeing: log.Trace("Awaiting server start-up") so the original assumption is correct. So this is causing a problem on a WinServer-2019 but is working fine with Windows11. I am going to try to build on windows... I have the gentoo-linux ebuild to reference and I have all the build deps. As a sanity check I will swap those if/else clauses ... |
The trace logs are missing the important things I need. I don't see Now looking at the goroutine stack we see:
This is the goroutine is created by gitea/modules/graceful/manager_windows.go Lines 109 to 111 in 8cceee4
It's Goroutine is number 18: /source/modules/graceful/manager_windows.go:110
Now looking at Goroutine 16 we see that the head of the stack is:
gitea/modules/graceful/manager_windows.go Lines 228 to 251 in 8cceee4
This function will return once all of the server listeners are created so unfortunately this profile was acquired too early. But it does indicate that we're not blocked at 120 and in fact get to L124 gitea/modules/graceful/manager_windows.go Line 124 in 8cceee4
|
Would you say it’s „safe“ to run the service like that until a fix is available? Everything seems to work properly after all. Or would it be better go back to 1.16.9 for the time being? |
new goroutine log. I watched the gitea.log until the service was created. hopefully this was long enough |
The problem is we can't downgrade the db so we are where we are. it works just any backup scripts won't work (disabled mine and weekly manually do the steps). The only risk with hard terminating is if someone is writing. This is why I am also doing such things late at night and also I flush before killing. |
Still seeing awaitServer... So may be it's deadlocked somehow - but I don't see how. I think we need the full associated logs - there's something going on here. |
Is it possible that you don't have INSTALL_LOCK set? If not, well there's your problem. The server won't be ready until that lock is set. |
app.ini pasted below with ### where info would be.
As per logs. I think I have provided everything, with the level set to TRACE as well. if there is anything else please advise |
Thanks. The whole SVC thing is a red herring. The problem is: Lines 19 to 22 in 3d52edc
This block needs to call |
Thanks for the explanation. I’m going to handle it like that as well then for now. |
Workaround for the moment is to enable ssh - you don't need to enable to internal ssh server and you can even set the SSH_DOMAIN to something like |
The graceful manager waits for 4 listeners to be created or to be told that they are not needed. If it is not told about them it will indefinitely and timeout. This leads to SVC hosts not being told of being in the readyState but on Unix would lead to the termination of the process. There was an unfortunate regression in #20299 which missed this subtly and in the case whereby SSH is disabled the `builtinUnused()` is not called. This PR adds a call to `builtinUnused()` when not using the builtin ssh to allow `createServerWaitGroup.Done()` to be called. In addition it was noted that the if/else clauses for timeout informing of the SVC host were in the wrong order. These have been swapped. Fix #20609
) The graceful manager waits for 4 listeners to be created or to be told that they are not needed. If it is not told about them it will indefinitely and timeout. This leads to SVC hosts not being told of being in the readyState but on Unix would lead to the termination of the process. There was an unfortunate regression in #20299 which missed this subtly and in the case whereby SSH is disabled the `builtinUnused()` is not called. This PR adds a call to `builtinUnused()` when not using the builtin ssh to allow `createServerWaitGroup.Done()` to be called. In addition it was noted that the if/else clauses for timeout informing of the SVC host were in the wrong order. These have been swapped. Fix #20609
I just made an update from gitea v1.16.8 to gitea v1.17.1. Im runnig the gitea-1.17.1-windows-4.0-amd64.exe and have same issue as described by LJFan. The Operating System is Windows Server 2016. The Windows Event Viewer is not showing any Errors and the GiTea Logfile isn`t showing any issues either. Here the first few Lines of Gitea Log
|
|
Do I just need to set set the SSH_DOMAIN under [server] to enable SSH? If so it doesn`t work for me. |
No, you need to set DISABLE_SSH=false, START_SSH_SERVER=true (or false may also work) Or just download the latest 1.17 from https://dl.gitea.io/gitea/1.17 |
The graceful manager waits for 4 listeners to be created or to be told that they are not needed. If it is not told about them it will indefinitely and timeout. This leads to SVC hosts not being told of being in the readyState but on Unix would lead to the termination of the process. There was an unfortunate regression in go-gitea#20299 which missed this subtly and in the case whereby SSH is disabled the `builtinUnused()` is not called. This PR adds a call to `builtinUnused()` when not using the builtin ssh to allow `createServerWaitGroup.Done()` to be called. In addition it was noted that the if/else clauses for timeout informing of the SVC host were in the wrong order. These have been swapped. Fix go-gitea#20609
Description
I create a service name
gitea
following the instruction in Windows Service.However, when I start the
gitea
service, the service is always in the 'starting' statusThe status is 'running' actually because gitea behaves normally.
Gitea Version
1.17.0
Can you reproduce the bug on the Gitea demo site?
No
Log Gist
No response
Screenshots
No response
Git Version
git version 2.36.1.windows.1
Operating System
Windows Server 2022 Standard
How are you running Gitea?
I use the newest (1.17.0) prebuild version of gitea gitea-1.17.0-gogit-windows-4.0-amd64.exe.
Database
MSSQL
The text was updated successfully, but these errors were encountered: