Add optional cache to repository

[xq114]

你在什么场景下需要该功能？

目前运行 xrepo search <xxx> 的速度非常慢，看了一下log是卡在遍历xmake-repo的所有文件上了，现在随着xmake-repo package数量越来越多，也应该加上一个cache

描述可能的解决方案

给官方xmake-repo加一个cache文件，搜索package时不需要逐个文件夹去遍历；对没有cache的私有仓库还是走原来的方式。cache的更新可以交给ci自动更新，从dev到master sync的时候加commit

描述你认为的候选方案

不改动xmake-repo，但在本地运行xrepo search的时候生成本地cache，xmake-repo commit不变则cache也不用更新

其他信息

No response

[Issues-translate-bot]

Bot detected the issue body's language is not English, translate it automatically.

---

In what scenario do you need this function?

Currently, the speed of running xrepo search <xxx> is very slow. After looking at the log, it is stuck in traversing all the files in xmake-repo. Now as the number of xmake-repo packages increases, a cache should also be added.

Describe possible solutions

Add a cache file to the official xmake-repo. When searching for packages, you do not need to traverse each folder one by one. For private repositories without cache, use the original method. Cache updates can be handed over to ci for automatic updates, and commit is added when syncing from dev to master.

Describe your alternatives

Do not change xmake-repo, but generate a local cache when running xrepo search locally. If xmake-repo commit remains unchanged, the cache does not need to be updated.

Other information

No response

[waruqi]

其实主要的瓶颈不是遍历目录，是遍历的同时，还要挨个 load xmake.lua 去解析 description ，因为要从 desc 中搜，全量加载包配置才是最耗时的。

试下这个 patch ，应该会快不少

#5748

Before

ruki-2:tmp ruki$ time xrepo search "tbox"
The package names:
    tbox:
      -> tbox-v1.7.5: A glib-like multi-platform c library (in xmake-repo)
      -> cpp-tbox-2023.12.13: A complete Linux application software development tool library an
d runtime framework, aim at make C++ development easy. (in xmake-repo)

real    0m3.797s
user    0m1.184s

After

首次不会快，缓存后第二次会快。

ruki-2:tmp ruki$ time xrepo search "tbox"
The package names:
    tbox:
      -> tbox: A glib-like multi-platform c library (in xmake-repo)
      -> cpp-tbox: A complete Linux application software development tool library and runtime f
ramework, aim at make C++ development easy. (in xmake-repo)

real    0m1.220s
user    0m0.436s
sys     0m0.526s

[xq114]

感觉windows上加速不明显
之前

之后

第一次跑从12s降到7s，但后续用时反而增加了；可能和NTFS文件系统有关系，Unix读文件的延迟比Windows小？我记得LaTeX就因为这个原因在Windows上很慢

[xq114]

哦不对，现在用时是check visual studio导致的，search已经足够快了

[Issues-translate-bot]

Bot detected the issue body's language is not English, translate it automatically.

---

In fact, the main bottleneck is not to traverse the directory, but to load xmake.lua one by one to parse the description while traversing, because it is the most time-consuming to search from desc and fully load the package configuration.

Try this patch, it should be much faster

#5748

[Issues-translate-bot]

Bot detected the issue body's language is not English, translate it automatically.

---

It feels like the acceleration is not obvious on Windows
Before

after

The first run dropped from 12s to 7s, but the subsequent time increased instead; it may be related to the NTFS file system. The delay in reading files in Unix is ​​smaller than that in Windows? I remember LaTeX being slow on Windows for this reason

[Issues-translate-bot]

Bot detected the issue body's language is not English, translate it automatically.

---

Oh no, the current time is caused by checking visual studio. Search is already fast enough.

[waruqi]

既然没问题，那我就关了

[Issues-translate-bot]

Bot detected the issue body's language is not English, translate it automatically.

---

Since there's no problem, I'll turn it off