404 error on msstore endpoint resulting in Failed to update source: msstore

[mdanish-kh]

Brief description of your issue

winget list reports me Failed when searching source; results will not be included: msstore. Looking in the CLI verbose logs, found that it gets a 404 when hitting msstore endpoint. Performing a source reset didn't help either.

Relevant log snippet

2024-09-03 23:43:28.731 [REPO] Sending http GET request to: https://storeedgefd.dsx.mp.microsoft.com/v9.0/packageManifests/9NMW3C7204PX?Market=PK
2024-09-03 23:43:28.732 [REPO] Http GET request details:
GET / HTTP/1.1

Content-Type: application/json

User-Agent: winget-cli WindowsPackageManager/1.8.1911 DesktopAppInstaller/Microsoft.DesktopAppInstaller v1.23.1911.0

Version: 1.6.0




2024-09-03 23:43:28.974 [REPO] Response status: 404
2024-09-03 23:43:28.975 [REPO] Response details:
HTTP/1.1 404 Not Found

Access-Control-Allow-Credentials: true

Access-Control-Allow-Origin: 

Access-Control-Expose-Headers: *

Cache-Control: max-age=0, no-cache, no-store

Connection: keep-alive

Content-Length: 248

Content-Type: application/json; charset=utf-8

Date: Tue, 03 Sep 2024 18:43:31 GMT

Expires: Tue, 03 Sep 2024 18:43:31 GMT

MS-CV: fIMNwxgG4022fgfs.1

Pragma: no-cache

Server: Microsoft-HTTPAPI/2.0

X-OSG-Served-By: Torus-EastAsia_LEGACY000AJ7_1.0.0.0

X-Segment-On: true



{"code":"DataNotFound","data":[],"details":[],"innererror":{"code":"DataNotFound","data":[],"details":[],"message":"Product 9NMW3C7204PX is not present","source":"StoreEdgeFD"},"message":"Product 9NMW3C7204PX is not present","source":"StoreEdgeFD"}
2024-09-03 23:43:28.975 [FAIL] C:\__w\1\s\external\pkg\src\AppInstallerCommonCore\HttpClientHelper.cpp(187)\WindowsPackageManager.dll!00007FF8397F68E2: (caller: 00007FF8397F5BC9) Exception(1) tid(6e80) 8A150044 
2024-09-03 23:43:28.975 [FAIL] C:\__w\1\s\external\pkg\src\AppInstallerRepositoryCore\CompositeSource.cpp(1127)\WindowsPackageManager.dll!00007FF8399009DA: (caller: 00007FF83983CE73) LogHr(1) tid(6e80) 8A150044     Msg:[C:\__w\1\s\external\pkg\src\AppInstallerCommonCore\HttpClientHelper.cpp(187)\WindowsPackageManager.dll!00007FF8397F68E2: (caller: 00007FF8397F5BC9) Exception(1) tid(6e80) 8A150044 ] 

2024-09-03 23:43:28.975 [REPO] Failed to search source for correlation: msstore
2024-09-03 23:43:28.975 [REPO] Did not find Id [9NMW3C7204PX] in tracked source: msstore

Full log: WinGet-2024-09-03-23-43-25.211.log

Steps to reproduce

Not aware of any. Happens from time to time on my machine. Could be a CDN / my region issue

Expected behavior

Msstore source is updated successfully

Actual behavior

Failed when searching source; results will not be included: msstore

Environment

Windows Package Manager v1.8.1911
Copyright (c) Microsoft Corporation. All rights reserved.

Windows: Windows.Desktop v10.0.22631.4037
System Architecture: X64
Package: Microsoft.DesktopAppInstaller v1.23.1911.0

Winget Directories
-------------------------------------------------------------------------------------------------------------------------------
Logs                               %LOCALAPPDATA%\Packages\Microsoft.DesktopAppInstaller_8wekyb3d8bbwe\LocalState\DiagOutputDir
User Settings                      %LOCALAPPDATA%\Packages\Microsoft.DesktopAppInstaller_8wekyb3d8bbwe\LocalState\settings.json
Portable Links Directory (User)    %LOCALAPPDATA%\Microsoft\WinGet\Links
Portable Links Directory (Machine) C:\Program Files\WinGet\Links
Portable Package Root (User)       %LOCALAPPDATA%\Microsoft\WinGet\Packages
Portable Package Root              C:\Program Files\WinGet\Packages
Portable Package Root (x86)        C:\Program Files (x86)\WinGet\Packages
Installer Downloads                %USERPROFILE%\Downloads

Links
---------------------------------------------------------------------------
Privacy Statement   https://aka.ms/winget-privacy
License Agreement   https://aka.ms/winget-license
Third Party Notices https://aka.ms/winget-3rdPartyNotice
Homepage            https://aka.ms/winget
Windows Store Terms https://www.microsoft.com/en-us/storedocs/terms-of-sale

Admin Setting                             State
--------------------------------------------------
LocalManifestFiles                        Enabled
BypassCertificatePinningForMicrosoftStore Enabled
InstallerHashOverride                     Enabled
LocalArchiveMalwareScanOverride           Disabled
ProxyCommandLineOptions                   Disabled
DefaultProxy                              Disabled

[github-actions]

Hi I'm an AI powered bot that finds similar issues based off the issue title.

Please view the issues below to see if they solve your problem, and if the issue describes your problem please consider closing this one and thumbs upping the other issue to help us prioritize it. Thank you!

Open similar issues:

• Failed when searching Source (#3419), similarity score: 0.81

Closed similar issues:

• Failed when searching source; results will not be included: msstore (#4676), similarity score: 0.80
• Winget source down - 0KB cache file from CDN (#3744), similarity score: 0.80
• winget source update fails (#3721), similarity score: 0.79
• Failed when searching source; results will not be included: msstore (#3579), similarity score: 0.79

Note: You can give me feedback by thumbs upping or thumbs downing this comment.

[mdanish-kh]

Feel free to mark as Area-External Issue outside of winget-cli source if there's nothing WinGet can do here

[Trenly]

I believe it is package specific, as these are the relevant lines -

2024-09-03 23:43:28.730 [REPO] Finding available package from installed package using system reference search: Query:[none] Include:PackageFamilyName='42458pdfiumapp.freepdfreaderforadobeacrobat-freepd_41tabtrnk92zy'[Exact]
2024-09-03 23:43:28.730 [REPO]  ... searching source: msstore [StoreEdgeFD]
2024-09-03 23:43:28.730 [REPO] Performing search: Query:[none] Include:PackageFamilyName='42458pdfiumapp.freepdfreaderforadobeacrobat-freepd_41tabtrnk92zy'[Exact]
2024-09-03 23:43:28.731 [REPO]  ... setting latest tracking package to: 9NMW3C7204PX
2024-09-03 23:43:28.731 [REPO]  ... using tracking package: 9NMW3C7204PX
2024-09-03 23:43:28.731 [REPO] Search request meets optimized search criteria.
2024-09-03 23:43:28.731 [REPO] Sending http GET request to: https://storeedgefd.dsx.mp.microsoft.com/v9.0/packageManifests/9NMW3C7204PX?Market=PK
|
2024-09-03 23:43:28.974 [REPO] Response status: 404

winget search 9NMW3C7204PX -s msstore
An unexpected error occurred while executing the command:
0x8a150044 : The rest API endpoint is not found.

[mdanish-kh]

I was curious that if package metadata couldn't be found from source, then why didn't WinGet fallback to assigning it an ID of the form (MSIX|ARP)\<PackageFullName> as it does with other packages that don't seem to be available in a source

I tried to gain a deeper understanding by looking at the code, and though I can't understand all of it, I think what's happening is that the CLI tried to look into my StoreEdgeFD\installed.db (tracking index?) first to see if it could find a matching ID for the package. It did as I had the relevant package msstore ID in installed.db, and then it tried to use that ID to query the remote msstore source to get more info. The info wasn't available, so winget showed it as Failed when searching source; results will not be included: msstore, although it was only this one package that couldn't be found.

winget-cli/src/AppInstallerRepositoryCore/CompositeSource.cpp

Lines 1504 to 1536 in 19b3b5e

	// Find the tracking result with the latest timestamp.
	auto trackingCatalog = source.GetTrackingCatalog();
	SearchResult trackingResult = trackingCatalog.Search(systemReferenceSearch);
	std::shared_ptr<IPackage> trackingPackage;
	std::chrono::system_clock::time_point trackingPackageTime;
	bool trackingSet = false;
	for (const auto& trackingMatch : trackingResult.Matches)
	{
	auto candidateTime = GetLatestTrackingWriteTime(OnlyAvailable(trackingMatch.Package));
	if (!trackingPackage || candidateTime > trackingPackageTime)
	{
	trackingPackage = OnlyAvailable(trackingMatch.Package);
	trackingPackageTime = candidateTime;
	}
	}
	if (trackingPackage && trackingPackageTime > compositePackage->GetTrackingPackageWriteTime())
	{
	AICLI_LOG(Repo, Verbose, << " ... setting latest tracking package to: " << trackingPackage->GetProperty(PackageProperty::Id));
	compositePackage->SetTracking(source, trackingPackage, trackingPackageTime);
	trackingSet = true;
	}
	// Attempt to correlate local packages against this source if supported.
	SearchResult availableResult;
	if (source.GetDetails().SupportInstalledSearchCorrelation)
	{
	availableResult = result.SearchAndHandleFailures(source, systemReferenceSearch);
	}

If the msstore response for a package is Detail not found, is this an opportunity for the CLI to disregard the ID in installed.db (and also delete the entry? I see there's some logic related to timestamp but we want to delete it regardless)? That way WinGet would correctly treat the installed package as if it wasn't available from any source. And ideally, if only one package fails correlation, then it may be an opportunity to show that package in CLI output OR say that "Some packages could not be correlated from source: msstore"

---

@denelon This may not be Area-External Issue outside of winget-cli source if this is an opportunity for CLI improvement

[minerva-jupiter]

I had same issue. However when I uninstalled the package that MsStoreSource return 404, this app don't have run with any issues.
This solution is putting the cart before the horse.
And then I suggest that if any error couse in process to get source, this app list up errored package and continue running to install package with no issue.
It is too inconvenient to stop running by issue in only some packages.