Latent Spectral Models (ICML 2023)

Paper Correction: There is a typo in our descriptions about evaluation metrics. The paper reported results are Relative L2.

Solving High-Dimensional PDEs with Latent Spectral Models [paper]

To tackle both the approximation and computation complexities in PDE-governed tasks. We propose the Latent Spectral Models (LSM) with the following features:

• Free from unwieldy coordinate space, LSM solves PDEs in the latent space.
• LSM holds the universal approximation capacity under theoretical guarantees.
• LSM achieves 11.5% error reduction over previous SOTA in both solid and fuild physics and performs favorable efficiency and transferability.

Figure 1. Overview of LSM.

LSM vs. Previous Methods

Different from previous methods that learn a single operator directly, inspired by classical spectral methods in numerical analysis, LSM composes complex mappings into multiple basis operators. Along with the latent space projection, LSM presents favorable approximation and convergence properties.

Figure 2. Comparison in approximating complex mappings.

Get Started

1. Install Python 3.8. For convenience, execute the following command.

pip install -r requirements.txt

2. Prepare Data. You can obtain experimental datasets from the following links.

Dataset	Task	Geometry	Link
Elasticity-P	Estimate material inner stress	Point Cloud	[Google Cloud]
Elasticity-G	Estimate material inner stress	Regular Grid	[Google Cloud]
Plasticity	Estimate material deformation over time	Structured Mesh	[Google Cloud]
Navier-Stokes	Predict future fluid velocity	Regular Grid	[Google Cloud]
Darcy	Estimate fluid pressure through medium	Regular Grid	[Google Cloud]
AirFoil	Estimate airflow velocity around airfoil	Structured Mesh	[Google Cloud]
Pipe	Estimate fluid velocity in a pipe	Structured Mesh	[Google Cloud]

3. Train and evaluate model. We provide the experiment scripts of all benchmarks under the folder ./scripts/. You can reproduce the experiment results as the following examples:

bash scripts/elas_lsm.sh # for Elasticity-P
bash scripts/elsa_interp_lsm.sh # for Elasticity-G
bash scripts/plas_lsm.sh # for Plasticity
bash scripts/ns_lsm.sh # for Navier-Stokes
bash scripts/darcy_lsm.sh # for Darcy
bash scripts/airfoil_lsm.sh # for Airfoil
bash scripts/pipe_lsm.sh # for Pipe

Note: You need to change the argument --data-path in the above script files to your dataset path.

4. Develop your own model. Here are the instructions:

   • Add the model file under folder ./models/.
   • Add the model name into ./model_dict.py.
   • Add a script file under folder ./scripts/ and change the argument --model.

Note: For clearness and easy comparison, we also include the FNO in this repository.

Results

We extensively experiment on seven benchmarks and compare LSM with 13 baselines. LSM achieves the consistent state-of-the-art in both solid and fluid physics (11.5% averaged error reduction).

Table 1. Model performance on seven benchmarks. MSE is recorded.

Showcases

Figure 3. Showcases. LSM can capture the shock wave around the airfoil precisely.

Citation

If you find this repo useful, please cite our paper.

@inproceedings{wu2023LSM,
  title={Solving High-Dimensional PDEs with Latent Spectral Models},
  author={Haixu Wu and Tengge Hu and Huakun Luo and Jianmin Wang and Mingsheng Long},
  booktitle={International Conference on Machine Learning},
  year={2023}
}

Contact

If you have any questions or want to use the code, please contact wuhx23@mails.tsinghua.edu.cn.

Acknowledgement

We appreciate the following github repos a lot for their valuable code base or datasets:

https://github.com/neuraloperator/neuraloperator

https://github.com/neuraloperator/Geo-FNO