GSoC 2025: CubeFS

programs/summerofcode/2025.md

@@ -419,3 +419,100 @@ In the next phase, which will be implemented in this Summer Of Code (SOC) projec
   - Vincent (@CaptainVincent, vincent@secondstate.io) - Primary
   - yi (@0yi0 yi@secondstate.io)
 - Upstream Issue (URL): https://github.com/WasmEdge/WasmEdge/issues/4011
+
+#### CubeFS
+
+##### Metanode Multi-Replica Atomic Self-Healing
+
+- Description:[CubeFS](https://cubefs.io/) is a new generation cloud-native open source storage system. Metadata partition is a multi-replica system that maintains consistency using Raft. When a single replica needs to be migrated from one node to another, such as in cases of disk failure or load balancing, it requires the establishment of a new replica and the deletion of the old one. This involves multi-step operations across multiple nodes.Replica information is stored in the master system and also locally within the metanode. Due to network fluctuations in a distributed environment, inconsistencies may arise between operations in the master and metanode, as well as between metanodes. We can refer to the self-healing methods used in data partitioning to achieve consistency of replica information. Here, we do not consider automated rollback and forward mechanisms during data partition changes, similar to data partitioning. The migration speed for metadata partitioning is relatively fast and more convenient.
+
+- Expected Outcome:
+	- Similar to data partitioning, self-healing can be implemented to ensure that partition information is consistent between the master and metanode, as well as among the metanodes.
+	- Consider self-healing concurrency control. The previous implementation of data partition was integrated into the entire data partition migration system. Metapartition may need to be implemented independently.
+- Recommended Skills: Go, basic familiarity with storage system
+- Expected project size: medium (~175 hour projects)
+- Mentor(s): 
+  - Chi He (@bboyCH4, hechi1014@126.com) - primary
+  - Leon Chang (@leonrayang, chl696@sina.com)
+- Upstream Issue (URL):https://github.com/cubefs/cubefs/issues/3656
+
+
+##### Goroutine Pool Refactoring
+
+- Description:[CubeFS](https://cubefs.io/) is a new generation cloud-native open source storage system.The current system, based on a Golang framework, tracks the number of goroutines but frequently experiences goroutine leaks. There is no clear insight into which processes are utilizing the most goroutines, nor is there any timeout setting or quantitative management for certain goroutines. Overall, there is a lack of statistical output and metrics reporting. This functionality could be integrated into the util module and made available for use by various subsystems.
+- Recommended Skills:  Go, basic familiarity with storage system
+
+- Expected Outcome: 
+	- Enable comprehensive goroutine management
+	- Refactoring the current system's main processes with a goroutine pool implementation.
+- Expected project size: small (~90 hour projects)
+- Mentor(s):
+  - Xuewei Zeng (@Victor1319, 834863182@qq.com) - primary
+  - Leon Chang (@leonrayang, chl696@sina.com)
+- Upstream Issue (URL): https://github.com/cubefs/cubefs/issues/3657
+
+
+##### Inotify
+
+- Description:[CubeFS](https://cubefs.io/) is a new generation cloud-native open source storage system.Adding support for the inotify system call to CubeFS allows applications to promptly detect changes in files and directories through the file system's event notification mechanism. This is a useful feature in some real-time data systems.
+
+	- Inotify is implemented through the kernel's event notification mechanism and mainly supports the following events:
+		- File creation, modification, and deletion.
+		- Changes in directory contents.
+		- Changes in file attributes.
+	- In distributed file systems, the challenges of inotify include:**
+		- **Latency Issues:** How to promptly detect event changes on remote nodes.
+		- **Performance Issues:** Event listening may introduce additional overhead."**
+- Recommended Skills:  Go, basic familiarity with storage system
+
+- Expected Outcome: 
+	- Maintain an event queue on the client side to store file system events.
+	- Add an event logging feature in the metadata service to record all file operations from clients.
+	- Provide a subscription interface for each client to listen for remote events.
+
+- Expected project size: small (~90 hour projects)
+- Mentor(s):
+  - Mofei Zhang (@mervinkid, mofei2816@gmail.com) - primary
+  - Haifeng Liu (@bladehliu, bladehliu@qq.com)
+- Upstream Issue (URL): https://github.com/cubefs/cubefs/issues/3658
+
+
+
+##### Hierarchical flow control
+
+- Description: Hierarchical flow control is needed for the DataNode system, which has various IO types, such as repair IO, inspection IO, asynchronous deletion IO, and business read/write IO. This capability should be adjustable, prioritizing business operations to minimize the impact of asynchronous processes on the system, and should be executed during idle periods.
+- Expected Outcome: Optimize the existing flow control module by building a hierarchical queue and control logic to create a controllable, viewable, and statistically analyzable flow control system that is safe and reliable.
+- Recommended Skills: Go, basic familiarity with storage system
+- Expected project size: medium (~175 hour projects)
+- Mentor(s):
+  - Leon Chang (@leonrayang, chl696@sina.com) - primary
+  - Chi He (@bboyCH4, hechi1014@126.com)
+- Upstream Issue (URL):https://github.com/cubefs/cubefs/issues/3659
+
+
+##### Optimize Data Inspection Capability
+
+- Description: The current data cycle inspection capability primarily focuses on newly added and modified data, while existing data is not being read. This means that issues like disk bad sectors may go undetected if not read, resulting in anomalies being discovered only during subsequent migrations.
+- Expected Outcome: 
+	- Implementation Logic: Consider comparing data based on the master-slave logic between existing replicas while also verifying the CRC.
+	- Throttling: It is necessary to take into account the busy state of the HDD, limiting the I/O request volume (current disk metrics have logged the oplog) to a certain level before initiating the comparison. Future plans should also incorporate tiered throttling for additional considerations.
+	- Controllability: Support periodic definitions, avoiding excessive frequency, and also consider adjustable controls after going live.
+- Recommended Skills: Go, basic familiarity with storage system
+- Expected project size: (~90 hour projects)
+- Mentor(s):
+  - Xuewei Zeng (@Victor1319, 834863182@qq.com) - primary
+  - Leon Chang (@leonrayang, chl696@sina.com)
+- Upstream Issue (URL):https://github.com/cubefs/cubefs/issues/3660
+
+
+
+##### Layered Storage Based on S3
+
+- Description: Currently, the main branch code has integrated the foundational capabilities for layered storage, which includes three types of storage: multi-replica SSD, HDD, and EC storage. The development functionality supports multi-replica SSD and HDD. The overall goal of the project is hybrid cloud storage, which can be referenced at the link: Hybrid Cloud. Currently, support for S3 storage layering is needed, especially in public clouds, to facilitate the deployment of CubeFS. For further details, please refer to the previous issue: Issue #3088.
+- Expected Outcome: Support direct data storage to S3, providing a foundation for further cooling to S3 in the future.For now, there's no need to consider multi-layer cooling operations for the data.
+- Recommended Skills: Go, basic familiarity with storage system
+- Expected project size: medium (~175 hour projects)
+- Mentor(s):
+  - Leon Chang (@leonrayang, chl696@sina.com)
+  - Haifeng Liu (@bladehliu, bladehliu@qq.com)
+- Upstream Issue (URL):https://github.com/cubefs/cubefs/issues/3661