Tests failures: System.Net.NameResolution.Tests / DnsObsolete* & Dns_GetHostEntry* #1488

ghost · 2017-09-30T02:13:22Z

Type of failures

Affected tests:

System.Net.NameResolution.Tests.GetHostByNameTest (both always fail together)
- DnsObsoleteGetHostByName_EmptyString_ReturnsHostName
- DnsObsoleteBeginEndGetHostByName_EmptyString_ReturnsHostName
System.Net.NameResolution.Tests.GetHostEntryTest (both always fail together)
- Dns_GetHostEntry_HostString_Ok
- Dns_GetHostEntryAsync_HostString_Ok

Failure text:

OSX: Device not configured
Linux: No such device or address

System.Net.Internals.SocketExceptionFactory+ExtendedSocketException : Device not configured
OR
System.Net.Internals.SocketExceptionFactory+ExtendedSocketException : No such device or address

at System.Net.Dns.InternalGetHostByName(String hostName, Boolean includeIPv6) in /Users/buildagent/agent/_work/34/s/corefx/src/System.Net.NameResolution/src/System/Net/DNS.cs:line 46
at System.Net.Dns.GetHostByName(String hostName) in /Users/buildagent/agent/_work/34/s/corefx/src/System.Net.NameResolution/src/System/Net/DNS.cs:line 41
at System.Net.NameResolution.Tests.GetHostByNameTest.DnsObsoleteGetHostByName_EmptyString_ReturnsHostName() in /Users/buildagent/agent/_work/34/s/corefx/src/System.Net.NameResolution/tests/FunctionalTests/GetHostByNameTest.cs:line 108

Note:

Dns_GetHostEntryAsync_HostString_Ok fails almost always with same error twice, wrapped in System.AggregateException.

Older failure on OSX (prior to 2017/9/15)

Assert.Contains() Failure
Not found: DCI-Mac-Build-068.local
In value:  dci-mac-build-068.local
at System.Net.NameResolution.Tests.GetHostByNameTest.DnsObsoleteGetHostByName_EmptyString_ReturnsHostName() in /Users/buildagent/agent/_work/30/s/corefx/src/System.Net.NameResolution/tests/FunctionalTests/GetHostByNameTest.cs:line 108

History of failures

Each test couple always fails or passes - maybe bad machine setup?

Day	Build	OS	Test
9/6	20170906.01	OSX10.12	DnsObsolete*
9/10	20170910.01	Ubuntu14.04	DnsObsolete* & Dns_GetHostEntry*
9/13	20170913.02	OSX10.12	DnsObsolete*
9/15	20170915.01	OSX10.12	DnsObsolete*
9/20	20170920.03	Debian90	DnsObsolete* & Dns_GetHostEntry*
9/30	20170930.01	OSX10.12	DnsObsolete* & Dns_GetHostEntry*
10/2	20171002.01	Debian87	DnsObsolete* & Dns_GetHostEntry*
10/21	20171021.01	Debian87	DnsObsolete* & Dns_GetHostEntry*
10/26	20171026.01	Suse42.2	DnsObsolete* & Dns_GetHostEntry*
11/13	20171113.03	OSX10.12	DnsObsolete* & Dns_GetHostEntry*
11/16	20171116.51	SLES12	DnsObsolete* & Dns_GetHostEntry*
11/26	20171126.02	Suse42.2	DnsObsolete* & Dns_GetHostEntry*

Similar failures to dotnet/corefx#20245, but at different times (probably runs on different worker machine).

Runfo Tracking Issue: dnsobsoletegethostbyname_emptystring_returnshostname

Build	Definition	Kind	Run Name

Build Result Summary

Day Hit Count	Week Hit Count	Month Hit Count
0	0	0

The text was updated successfully, but these errors were encountered:

Sunny-pu · 2017-10-26T02:17:09Z

[EDIT] Test failure included in history in top post.

karelz · 2017-12-28T21:14:10Z

Updated test failure history in top post.

Sunny-pu · 2018-04-19T06:45:14Z

failed again:
https://mc.dot.net/#/product/netcore/master/source/official~2Fcorefx~2Fmaster~2F/type/test~2Ffunctional~2Fcli~2F/build/20180419.01/workItem/System.Net.NameResolution.Functional.Tests

benaadams · 2019-02-24T19:22:09Z

Seen in dotnet/corefx#35267

Debian.8.Amd64.Open-x64-Release

Message :
System.Net.Internals.SocketExceptionFactory+ExtendedSocketException : Name or service not known
Stack Trace :
   at System.Net.Dns.InternalGetHostByName(String hostName) in /__w/1/s/src/System.Net.NameResolution/src/System/Net/DNS.cs:line 68
   at System.Net.Dns.GetHostByName(String hostName) in /__w/1/s/src/System.Net.NameResolution/src/System/Net/DNS.cs:line 41
   at System.Net.NameResolution.Tests.GetHostByNameTest.DnsObsoleteGetHostByName_EmptyString_ReturnsHostName() in /__w/1/s/src/System.Net.NameResolution/tests/FunctionalTests/GetHostByNameTest.cs:line 108

AriNuer · 2019-06-18T06:59:51Z

Same test failed with similar exception:

Failed test:
System.Net.NameResolution.Tests.GetHostByNameTest/DnsObsoleteBeginEndGetHostByName_EmptyString_ReturnsHostName

Message:

System.Net.Internals.SocketExceptionFactory+ExtendedSocketException : Name or service not known

Stack Trace :

   at System.Net.Dns.InternalGetHostByName(String hostName) in /_/src/System.Net.NameResolution/src/System/Net/DNS.cs:line 68
   at System.Net.Dns.ResolveCallback(Object context) in /_/src/System.Net.NameResolution/src/System/Net/DNS.cs:line 218
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw(Exception source) in /_/src/System.Private.CoreLib/shared/System/Runtime/ExceptionServices/ExceptionDispatchInfo.cs:line 69
   at System.Net.Dns.HostResolutionEndHelper(IAsyncResult asyncResult) in /_/src/System.Net.NameResolution/src/System/Net/DNS.cs:line 361
   at System.Net.Dns.EndGetHostByName(IAsyncResult asyncResult) in /_/src/System.Net.NameResolution/src/System/Net/DNS.cs:line 383
   at System.Net.NameResolution.Tests.GetHostByNameTest.DnsObsoleteBeginEndGetHostByName_EmptyString_ReturnsHostName() in /_/src/System.Net.NameResolution/tests/FunctionalTests/GetHostByNameTest.cs:line 119

Build: -20190617.81(Master)
Failing configurations:

RedHat.6.Amd64.Open-x64-Release
Debian.9.Amd64.Open-x64-Release

Details:
https://mc.dot.net/#/user/dotnet-bot/pr~2Fdotnet~2Fcorefx~2Frefs~2Fheads~2Fmaster/test~2Ffunctional~2Fcli~2Finnerloop~2F/20190617.81/workItem/System.Net.NameResolution.Functional.Tests/analysis/xunit/System.Net.NameResolution.Tests.GetHostByNameTest~2FDnsObsoleteBeginEndGetHostByName_EmptyString_ReturnsHostName

karelz · 2019-10-02T18:46:55Z

Triage: We should know why DNS resolution does not work sometimes -- we know it is not cached on Linux/Mac, but why does it happen so often in CI? Is it network flakiness? Or busy machine?

Plan: Add production diagnostics into DNS, then make runs with those changes to catch it. It will be useful also for production diagnostic by customers.

wfurt · 2020-01-28T20:18:20Z

no failure on Linux for 90 days. Some tests are still disabled on OSX and ARM64 (tracked by #27622).

karelz · 2020-01-29T09:40:47Z

@wfurt do we have any tests disabled against this specific bug? If not, let's close it.

wfurt · 2020-01-29T19:21:28Z

We have four tests disabled at this moment for OSX and the same for ARM64 (tracked in #27622).

I suspect most of the failures are not product bugs and are caused by test environment.
AFAIK we changed our macOS infrastructure as well as ARM64 support matured.
How about moving the test to Outerloop? That would allow us to collect some stability data without impacting everyone's PRs. That also seems appropriate since tests do have a dependency on external service.

cc: @dotnet/ncl in case somebody has a better suggestion.

ViktorHofer · 2020-02-18T19:22:05Z

A bunch of NameResolution tests failed again (including the ones mentioned in this issue): https://dnceng.visualstudio.com/public/_build/results?buildId=525372&view=ms.vss-test-web.build-test-results-tab&runId=16621996&resultId=137716&paneView=debug

@wfurt @davish suggestions for mitigation?

davidsh · 2020-02-18T19:26:08Z

@wfurt @davish suggestions for mitigation?

Doing name resolution in a CI environment will always be flaky most likely.

System.Net.Internals.SocketExceptionFactory+ExtendedSocketException : Name or service not known

Perhaps we should consider an approach being done here in #32501 and only run these tests on a less regular basis. I.e.:

[Trait(XunitConstants.Category, XunitConstants.IgnoreForCI)] // DNS is flaky

FWIW, Our DNS APIs have been relatively stable from a product perspective and don't see a lot of risk running these tests less often.

wfurt · 2020-02-18T19:28:08Z

We should work towards stabilizing core services (like DNS) in CI.

ViktorHofer · 2020-02-18T20:09:47Z

What about leveraging the RetryHelper for DNS tests which are inherently flaky?

davidsh · 2020-02-18T20:54:58Z

We should work towards stabilizing core services (like DNS) in CI.

What about leveraging the RetryHelper for DNS tests which are inherently flaky?

Do we understand whether the problem is intermittent (i.e. retrying tests will help) or due to a mis-configuration of some kind in CI (which retrying tests will not help).

ViktorHofer · 2020-02-18T20:59:52Z

Do we understand whether the problem is intermittent (i.e. retrying tests will help) or due to a mis-configuration of some kind in CI (which retrying tests will not help).

I assumed it's intermittent but how would we find that out?

davidsh · 2020-02-18T22:39:08Z

I assumed it's intermittent but how would we find that out?

Let's add some RetryHelper logic to some of the tests. Then run it in CI. If we see a significant improvement in test reliability, then we will know that the problem is intermittent.

Anipik · 2021-02-25T17:36:24Z

failed again here https://dev.azure.com/dnceng/public/_build/results?buildId=1011268&view=logs&j=b9b7de4d-2126-5581-6453-1576f9aaac5f&t=db048d48-40e8-5399-9b14-d587763f33ff

eerhardt · 2021-02-25T23:26:13Z

Here too: https://helixre8s23ayyeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-pull-48667-merge-af809879612c4371b8/System.Net.NameResolution.Functional.Tests/console.826d39cc.log?sv=2019-07-07&se=2021-03-17T22%3A53%3A24Z&sr=c&sp=rl&sig=qfUTGHjOCkAigQRcROLrLRTtq5MphSvppEvcO8QVJUU%3D

ViktorHofer · 2021-02-25T23:27:12Z

Tests were disabled on SLES with #48759.

This is a set of changes required to enable control flow guard enforcement within the process. Control flow guard is a security mitigation feature that validates that indirect calls only land on addresses that are valid targets of indirect calls. It has two parts: identifying valid targets of indirect calls within the process, and checking whether target of indirect call is valid before dispatching to it. This implements annotations and enforcement within the unmanaged parts of the NativeAOT runtime and the annotation-only part for the managed code. Enforcement will follow later. Three kinds of changes: * A new version of Runtime.lib that enables `/guard:cf` flag. This is in addition to the existing libraries since we don't want code to pay the perf penalty if CFG is not enabled. * Annotating methods as valid CFG targets in the AOT compiler and object file writer. * MSBuild support for new `<ControlFlowGuard>Guard</ControlFlowGuard>` property that enables all of this (passes a switch to the AOT compiler, selects the guarded version of runtime libraries to link with, and passes a switch to link.exe to enable CFG for the process).

karelz changed the title ~~Tests under: System.Net.NameResolution.Functional.Tests failed with "System.Net.Internals.SocketExceptionFactory+ExtendedSocketException : Device not configured"~~ Tests failures: System.Net.NameResolution.Tests / DnsObsolete* & Dns_GetHostEntry* Dec 28, 2017

karelz transferred this issue from dotnet/corefx Jan 9, 2020

Dotnet-GitSync-Bot added area-System.Net untriaged New issue has not been triaged by the area owner labels Jan 9, 2020

karelz added os-linux Linux OS (any supported distro) os-mac-os-x macOS aka OSX test-run-core Test failures in .NET Core test runs and removed untriaged New issue has not been triaged by the area owner labels Jan 9, 2020

karelz added this to the 5.0 milestone Jan 9, 2020

wfurt added the disabled-test The test is disabled in source code against the issue label Jan 28, 2020

Gnbrkm41 mentioned this issue Feb 1, 2020

System.Net.NameResolution.Functional.Tests have few tests that constantly fail on local machines? #30411

Closed

MihaZupan mentioned this issue Apr 3, 2020

DnsGetHostEntry_LocalHost_ReturnsFqdnAndLoopbackIPs failed in CI #34317

Closed

antonfirsov self-assigned this May 4, 2020

antonfirsov mentioned this issue May 7, 2020

Gather more diagnostics, and prepare skipping flaky DNS tests #36072

Closed

nakda mentioned this issue May 12, 2020

Inconsistent behaviour using Dns.GetHostAddresses() #36291

Closed

karelz modified the milestones: 5.0, Future Jun 5, 2020

cscharf mentioned this issue Jul 15, 2020

Can't login to admin console bitwarden/server#702

Closed

Anipik added the blocking-clean-ci Blocking PR or rolling runs of 'runtime' or 'runtime-extra-platforms' label Feb 25, 2021

Anipik mentioned this issue Feb 25, 2021

Disable GetHostName and GetHost Entry tests on Unix #48770

Closed

runfoapp bot mentioned this issue Feb 25, 2021

Infrastructure - Status/Health #702

Closed

eerhardt mentioned this issue Feb 25, 2021

Build Threading.Tasks.DataFlow and ComponentModel.Annotations for NetCoreAppCurrent #48667

Merged

antonfirsov removed their assignment Mar 9, 2021

ViktorHofer removed the blocking-clean-ci Blocking PR or rolling runs of 'runtime' or 'runtime-extra-platforms' label Mar 15, 2021

This comment has been minimized.

# to view

Anipik mentioned this issue Aug 3, 2021

Update branding for 5.0.10 #56752

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Tests failures: System.Net.NameResolution.Tests / DnsObsolete* & Dns_GetHostEntry* #1488

Tests failures: System.Net.NameResolution.Tests / DnsObsolete* & Dns_GetHostEntry* #1488

ghost commented Sep 30, 2017 •

edited by runfoapp bot

Loading

Sunny-pu commented Oct 26, 2017 •

edited by karelz

Loading

karelz commented Dec 28, 2017

Sunny-pu commented Apr 19, 2018

benaadams commented Feb 24, 2019

AriNuer commented Jun 18, 2019

karelz commented Oct 2, 2019 •

edited

Loading

wfurt commented Jan 28, 2020 •

edited by karelz

Loading

karelz commented Jan 29, 2020

wfurt commented Jan 29, 2020 •

edited by karelz

Loading

ViktorHofer commented Feb 18, 2020

davidsh commented Feb 18, 2020 •

edited

Loading

wfurt commented Feb 18, 2020

ViktorHofer commented Feb 18, 2020

davidsh commented Feb 18, 2020 •

edited

Loading

ViktorHofer commented Feb 18, 2020

davidsh commented Feb 18, 2020

Anipik commented Feb 25, 2021

eerhardt commented Feb 25, 2021

ViktorHofer commented Feb 25, 2021

This comment has been minimized.

This comment has been minimized.

Tests failures: System.Net.NameResolution.Tests / DnsObsolete* & Dns_GetHostEntry* #1488

Tests failures: System.Net.NameResolution.Tests / DnsObsolete* & Dns_GetHostEntry* #1488

Comments

ghost commented Sep 30, 2017 • edited by runfoapp bot Loading

Type of failures

Older failure on OSX (prior to 2017/9/15)

History of failures

Sunny-pu commented Oct 26, 2017 • edited by karelz Loading

karelz commented Dec 28, 2017

Sunny-pu commented Apr 19, 2018

benaadams commented Feb 24, 2019

AriNuer commented Jun 18, 2019

karelz commented Oct 2, 2019 • edited Loading

wfurt commented Jan 28, 2020 • edited by karelz Loading

karelz commented Jan 29, 2020

wfurt commented Jan 29, 2020 • edited by karelz Loading

ViktorHofer commented Feb 18, 2020

davidsh commented Feb 18, 2020 • edited Loading

wfurt commented Feb 18, 2020

ViktorHofer commented Feb 18, 2020

davidsh commented Feb 18, 2020 • edited Loading

ViktorHofer commented Feb 18, 2020

davidsh commented Feb 18, 2020

Anipik commented Feb 25, 2021

eerhardt commented Feb 25, 2021

ViktorHofer commented Feb 25, 2021

This comment has been minimized.

This comment has been minimized.

ghost commented Sep 30, 2017 •

edited by runfoapp bot

Loading

Sunny-pu commented Oct 26, 2017 •

edited by karelz

Loading

karelz commented Oct 2, 2019 •

edited

Loading

wfurt commented Jan 28, 2020 •

edited by karelz

Loading

wfurt commented Jan 29, 2020 •

edited by karelz

Loading

davidsh commented Feb 18, 2020 •

edited

Loading

davidsh commented Feb 18, 2020 •

edited

Loading