Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

doc(hansbug): update README #42

Merged
merged 13 commits into from
Jun 23, 2022
161 changes: 130 additions & 31 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,17 +1,22 @@
# treevalue
<div align="center">
<a href="https://opendilab.github.io/treevalue/"><img width="1000px" height="auto" src="https://github.com/opendilab/treevalue/blob/main/docs/source/_static/title-banner.png"></a>
</div>

---

[![Twitter](https://img.shields.io/twitter/url?style=social&url=https%3A%2F%2Ftwitter.com%2Fopendilab)](https://twitter.com/opendilab)
[![PyPI](https://img.shields.io/pypi/v/treevalue)](https://pypi.org/project/treevalue/)
![PyPI - Python Version](https://img.shields.io/pypi/pyversions/treevalue)
![Loc](https://img.shields.io/endpoint?url=https://gist.githubusercontent.com/HansBug/ff0bc026423888cd7c4f287eaed4b3f5/raw/loc.json)
![Comments](https://img.shields.io/endpoint?url=https://gist.githubusercontent.com/HansBug/ff0bc026423888cd7c4f287eaed4b3f5/raw/comments.json)


[![Docs Deploy](https://github.com/opendilab/treevalue/workflows/Docs%20Deploy/badge.svg)](https://github.com/opendilab/treevalue/actions?query=workflow%3A%22Docs+Deploy%22)
[![Code Test](https://github.com/opendilab/treevalue/workflows/Code%20Test/badge.svg)](https://github.com/opendilab/treevalue/actions?query=workflow%3A%22Code+Test%22)
[![Badge Creation](https://github.com/opendilab/treevalue/workflows/Badge%20Creation/badge.svg)](https://github.com/opendilab/treevalue/actions?query=workflow%3A%22Badge+Creation%22)
[![Package Release](https://github.com/opendilab/treevalue/workflows/Package%20Release/badge.svg)](https://github.com/opendilab/treevalue/actions?query=workflow%3A%22Package+Release%22)
[![codecov](https://codecov.io/gh/opendilab/treevalue/branch/main/graph/badge.svg?token=XJVDP4EFAT)](https://codecov.io/gh/opendilab/treevalue)

![GitHub Org's stars](https://img.shields.io/github/stars/opendilab)
[![GitHub stars](https://img.shields.io/github/stars/opendilab/treevalue)](https://github.com/opendilab/treevalue/stargazers)
[![GitHub forks](https://img.shields.io/github/forks/opendilab/treevalue)](https://github.com/opendilab/treevalue/network)
![GitHub commit activity](https://img.shields.io/github/commit-activity/m/opendilab/treevalue)
Expand All @@ -22,25 +27,65 @@

`TreeValue` is a generalized tree-based data structure mainly developed by [OpenDILab Contributors](https://github.com/opendilab).

Almost all the operation can be supported in form of trees in a convenient way to simplify the structure processing when the calculation is tree-based.
Almost all the operations can be supported in the form of trees in a convenient way to simplify the structure processing when the calculation is tree-based.

## Outline

* [Overview](#overview)
* [Getting Started](#getting-started)
* [Prerequisite](#prerequisite)
* [Installation](#installation)
* [Quick Usage](#quick-usage)
* [Tutorials](#tutorials)
* [External](#external)
* [Speed Performance](#speed-performance)
* [Change Log](#change-log)
* [Feedback and Contribute](#feedback-and-contribute)
* [Citation](#citation)
* [License](#license)

## Overview

When we build a complex nested structure, we need to model it as a tree structure, and the native list and dict in Python are often used to solve this problem. However, it takes a lot of codes and some complex and non-intuitive calculation logic, which is not easy to modify and extend related code and data, and parallelization is impossible.

Therefore, we need a kind of more proper data container, named `TreeValue`. It is designed for solving the following problems:

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

whether to add our overview layer graph

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it is not necessary, and the graph's is verrrrrry long which may break the view of readme.

May be hushuai can given some suggestions?

- **Ease of Use**: When the existing operations are applied to tree structures such as dict, they will become completely unrecognizable, with really low readability and maintainability.
- **Diversity of Data**: In the tree structure operation, various abnormal conditions (structure mismatch, missing key-value, type mismatch, etc.) occur from time to time, and the code will be more complicated if it needs to be handled properly.
- **Scalability and Parallelization**: When any multivariate operation is performed, the calculation logic needs to be redesigned under the native Python code implementation, and the processing will be more complicated and confusing, and the code quality is difficult to control.

## Getting Started

## Installation
### Prerequisite

`treevalue` has been fully tested in the Linux, macOS and Windows environments and with multiple Python versions, and it works properly on all these platforms.

However, **`treevalue` currently does not support PyPy**, so just pay attention to this when using it.

### Installation

You can simply install it with `pip` command line from the official PyPI site.

```shell
pip install treevalue
```

For more information about installation, you can refer to the [installation guide](https://opendilab.github.io/treevalue/main/tutorials/installation/index.html).
Or just from the source code on github

## Documentation
```shell
pip install git+https://github.com/opendilab/treevalue.git@main
```

For more information about installation, you can refer to the [installation guide](https://opendilab.github.io/treevalue/main/tutorials/installation/index.html).

The detailed documentation are hosted on [https://opendilab.github.io/treevalue](https://opendilab.github.io/treevalue/).
After this, you can check if the installation is processed properly with the following code

Only english version is provided now, the chinese documentation is still under development.
```python
from treevalue import __version__
print('TreeValue version is', __version__)
```

## Quick Start
### Quick Usage

You can easily create a tree value object based on `FastTreeValue`.

Expand Down Expand Up @@ -140,15 +185,43 @@ print(1 + (t - 0.8) ** 2 * 1.5) # math operators
# [1.0782, 1.0037, 1.5075, 1.0658]])
```

For more quick start explanation and further usage, take a look at:


### Tutorials

For more examples, explanations and further usages, take a look at:

* [Quick Start](https://opendilab.github.io/treevalue/main/tutorials/quick_start/index.html)
* [Create a Simplest Tree](https://opendilab.github.io/treevalue/main/tutorials/quick_start/index.html#create-a-simplest-tree)
* [Create a Slightly Complex Tree](https://opendilab.github.io/treevalue/main/tutorials/quick_start/index.html#create-a-slightly-complex-tree)
* [Basic Usage](https://opendilab.github.io/treevalue/main/tutorials/basic_usage/index.html)
* [Create a Tree](https://opendilab.github.io/treevalue/main/tutorials/basic_usage/index.html#create-a-tree)
* [Edit a Tree](https://opendilab.github.io/treevalue/main/tutorials/basic_usage/index.html#edit-the-tree)
* [Do Index or Slice Calculation on The Tree](https://opendilab.github.io/treevalue/main/tutorials/basic_usage/index.html#do-index-or-slice-calculation-on-the-tree)
* [Do Math Calculation on The Tree](https://opendilab.github.io/treevalue/main/tutorials/basic_usage/index.html#do-calculation-on-the-tree)
* [Make Function Tree-Supported](https://opendilab.github.io/treevalue/main/tutorials/basic_usage/index.html#make-function-tree-supported)
* [Advanced Usage](https://opendilab.github.io/treevalue/main/tutorials/advanced_usage/index.html)
* [Function Modes](https://opendilab.github.io/treevalue/main/tutorials/advanced_usage/index.html#function-modes)
* [Inheriting on Trees](https://opendilab.github.io/treevalue/main/tutorials/advanced_usage/index.html#inheriting-on-trees)
* [Process Missing Values](https://opendilab.github.io/treevalue/main/tutorials/advanced_usage/index.html#process-missing-values)
* [Functional Utilities](https://opendilab.github.io/treevalue/main/tutorials/advanced_usage/index.html#functional-utilities)
* [Structural Utilities](https://opendilab.github.io/treevalue/main/tutorials/advanced_usage/index.html#structural-utilities)
* [Tree Utilities](https://opendilab.github.io/treevalue/main/tutorials/advanced_usage/index.html#tree-utilities)
* [Flatten Utilities](https://opendilab.github.io/treevalue/main/tutorials/advanced_usage/index.html#flatten-utilities)
* [IO Utilities](https://opendilab.github.io/treevalue/main/tutorials/advanced_usage/index.html#io-utilities)
* [Object Oriented Usage](https://opendilab.github.io/treevalue/main/tutorials/advanced_usage/index.html#object-oriented-usage)
* [Costumize My TreeValue Class](https://opendilab.github.io/treevalue/main/tutorials/advanced_usage/index.html#diy-treevalue-class)
* [Visualization of TreeValue](https://opendilab.github.io/treevalue/main/tutorials/advanced_usage/index.html#draw-graph-for-treevalue)

### External

We provide an official treevalue-based-wrapper for numpy and torch called [DI-treetensor](https://github.com/opendilab/DI-treetensor) since the `treevalue` is often used with libraries like `numpy` and `torch`. It will actually be helpful while working with AI fields.



## Speed Performance

Here is the speed performance of all the operations in `FastTreeValue`, the following table is the performance comparison result with [dm-tree](https://github.com/deepmind/tree).
Here is the speed performance of all the operations in `FastTreeValue`; the following table is the performance comparison result with [dm-tree](https://github.com/deepmind/tree).

| | flatten | flatten(with path) | mapping | mapping(with path) |
| --------------------------------------------------- | :--------------: | :-------------------: | :-------------------: | :-------------------------: |
Expand Down Expand Up @@ -183,7 +256,7 @@ The following table is the performance comparison result with [tianshou Batch](h
| [treevalue](https://github.com/opendilab/treevalue) | 51.6 ns ± 0.609 ns | **64.4 ns ± 0.564 ns** | **750 ns ± 14.2 ns** | **88.9 µs ± 887 ns** | **50.2 µs ± 771 ns** | **40.3 µs ± 1.08 µs** | **62 µs ± 1.2 µs** |
| [tianshou Batch](https://github.com/thu-ml/tianshou) | **43.2 ns ± 0.698 ns** | 396 ns ± 8.99 ns | 11.1 µs ± 277 ns | 89 µs ± 1.42 µs | 119 µs ± 1.1 µs | 194 µs ± 1.81 µs | 653 µs ± 17.8 µs |

And this is the comparison between tianshou Batch and us, with `cat` , `stack` and `split` operations (**lower value means less time cost and runs faster**)
And this is the comparison between Tianshou Batch and us, with `cat` , `stack` and `split` operations (**lower value means less time cost and runs faster**)

![Time cost of cat operation](docs/source/_static/Time%20cost%20of%20cat%20operation.svg)

Expand All @@ -197,36 +270,62 @@ Test benchmark code can be found here:
* [Comparison with jax-libtree](https://github.com/opendilab/treevalue/blob/main/test/compare/jax/test_jax.py)
* [Comparison with tianshou Batch](https://github.com/opendilab/treevalue/blob/main/test/compare/tianshou/test_tianshou_batch.py)

## Extension
## Change Log

If you need to translate `treevalue` object to runnable source code, you may use the [potc-treevalue](https://github.com/potc-dev/potc-treevalue) plugin with the installation command below
<details><summary><b>Version History</b> <i>[click to expand]</i></summary>
<div>

```shell
pip install potc-treevalue
```
* 2022-05-03
1.3.1: Change definition of getitem, setitem and delitem; add pop method for TreeValue class.
* 2022-03-15
1.3.0: Add getitem, setitem and delitem for adding, editing and removing items in TreeValue class.
* 2022-02-22
1.2.2: Optimize union function; add walk utility method.
* 2022-01-26
1.2.1: Update tree printing; add keys, values, items on TreeValue; add comparision to facebook nest library.
* 2022-01-04
1.2.0: Add flatten_values and flatten_keys; fix problem in mapping function; add support for potc.
* 2021-12-03
1.1.0: Add version information; fix bug of default value; add flatten and unflatten; optimization speed performance.
* 2021-10-24
1.0.0: Greatly optimize the speed performance using cython, overhead has been reduced to a negligible level.
</div>
</details>

Or just install it with `treevalue` itself
## Feedback and Contribute

```shell
pip install treevalue[potc]
```
Welcome to **OpenDILab** community - treevalue!

In potc, you can translate the objects to runnable python source code, which can be loaded to objects afterwards by the python interpreter, like the following graph
If you meet some problem or have some brilliant ideas, you can [file an issue](https://github.com/opendilab/treevalue/issues/new/choose).

![potc system](docs/source/_static/potc-doing.svg)
<b>Scan the QR code and add us on Wechat:</b>

For more information, you can refer to
<div align="center">
<img src='https://github.com/opendilab/DI-engine/raw/main/assets/wechat.png' width="25%" />
</div>

* [potc-dev/potc](https://github.com/potc-dev/potc)
* [potc-dev/potc-treevalue](https://github.com/potc-dev/potc-treevalue)
* [Potc Plugin Installation](https://opendilab.github.io/treevalue/main/tutorials/plugins/index.html#potc-support)
Or just contact us with [slack](https://opendilab.slack.com/join/shared_invite/zt-v9tmv4fp-nUBAQEH1_Kuyu_q4plBssQ#/shared-invite/email) or email (opendilab.contact@gmail.com).

## Contribution
Please check [Contributing Guidances](https://github.com/opendilab/treevalue/blob/main/CONTRIBUTING.md).

We appreciate all contributions to improve treevalue, both logic and system designs. Please refer to CONTRIBUTING.md for more guides.
Thanks to the following contributors!

And users can join our [slack communication channel](https://join.slack.com/t/opendilab/shared_invite/zt-v9tmv4fp-nUBAQEH1_Kuyu_q4plBssQ), or contact the core developer [HansBug](https://github.com/HansBug) for more detailed discussion.
<a href="https://github.com/opendilab/treevalue/graphs/contributors">
<img src="https://contrib.rocks/image?repo=opendilab/treevalue" />
</a>

## Citation

```
@misc{treevalue,
title={{TreeValue} - Tree-Structure Computing Solution},
author={TreeValue Contributors},
publisher = {GitHub},
howpublished = {\url{https://github.com/opendilab/treevalue}},
year={2021},
}
```

## License

`treevalue` released under the Apache 2.0 license.
`treevalue` released under the Apache 2.0 license. See the [LICENSE](./LICENSE) file for details.