Skip to content

Latest commit

 

History

History
125 lines (107 loc) · 7.66 KB

README.md

File metadata and controls

125 lines (107 loc) · 7.66 KB

OphNet: A Large-Scale Video Benchmark for Ophthalmic Surgical Workflow Understanding

Static Badge

News

  • [Nov, 2024] We have repaired several damaged videos. You can now download the dataset again.
  • [Oct, 2024] We realeased OphNet2024 challenge dataset ! More information can be found in Data Preparation.
  • [Jul, 2024] OphNet2024 is in preparation——larger scale, more accurate, and more experimental results!
  • [Jul, 2024] OphNet was accepted by ECCV2024.
  • [Jun, 2024] The manuscript can be found on arXiv.


Introduction


Dataset Preparation

Directory Structure

OphNet-benchmark
├── annotation
│   ├── OphNet2024_surgery.csv
│   ├── OphNet2024_loca_all.csv
│   ├── OphNet2024_loca_challenge.csv
│   ├── OphNet2024_loca_challenge_phase.csv
│   ├── OphNet2024_ori_operation_trimmed.csv
│   ├── OphNet2024_ori_phase_trimmed.csv
├── data_processing
│   ├── clipper.py

-annotation

  • OphNet2024_surgery.csv: Annotated 1,969 untrimmed videos for surgical types, with the first label as the primary surgery. Selected 743 videos for time-boundary annotation.
  • OphNet2024_loca_all.csv: The original version of the time boundary annotations.
  • OphNet2024_loca_challenge.csv: Map phase and operation labels with fewer than 15 clips to numeric IDs 51 and 106, which can be interpreted as renaming labels with fewer than 15 instances as "Others."
  • OphNet2024_loca_challenge_phase.csv: A complete phase clip in OphNet2024_challenge.csv may be split due to covering multiple operations. Therefore, in OphNet2024_challenge_phase.csv, we merge consecutive clips of the same phase.
  • OphNet2024_ori_operation_trimmed.csv & OphNet2024_ori_phase_trimmed.csv: Follow the original labels without processing the tail data, similarly divided into two granularities: phase and operation.

-data_processing

  • clipper.py: extract clips based on annotated time boundaries from untrimmed videos.

HuggingFace&Baidu Netdisk

OphNet2024
├── OphNet2024_all (≈305G, 1,969 untrimmed videos--original resolution and FPS)
│   ├── OphNet2024_all.tar.gz.00
│   ├── OphNet2024_all.tar.gz.01
│   ├── ...
├── OphNet2024_trimmed_operation (≈139G, 17,508 trimmed videos from 743 videos with time-boundary annotation--original resolution and FPS)
│   ├── OphNet2024_loca_challenge_trimmed.csv
│   ├── OphNet2024_trimmed_operation.tar.gz.00
│   ├── OphNet2024_trimmed_operation.tar.gz.01
│   ├── ...
├── OphNet2024_trimmed_phase (≈139G, 14,674 trimmed videos from 743 videos with time-boundary annotation--original resolution and FPS)
│   ├── OphNet2024_loca_challenge_phase_trimmed.csv
│   ├── OphNet2024_trimmed_phase.tar.gz.00
│   ├── OphNet2024_trimmed_phase.tar.gz.01
│   ├── ...
  • OphNet2024_loca_challenge_trimmed.csv: The OphNet2024_loca_challenge.csv file with the version containing trimmed video names will be automatically created after running data_processing/cliper.py. (/OphNet2024_trimmed_operation)
  • OphNet2024_loca_challenge_phase_trimmed.csv: The OphNet2024_loca_challenge_phase.csv file with the version containing trimmed video names will be automatically created after running data_processing/cliper.py. (/OphNet2024_trimmed_phase)

Download

  • Label Description: The table with Chinese and English versions of surgery, phase, and operation names along with their ID mappings: OphNet2024_Label

  • Untrimmed Videos Download Source: HuggingFace | Baidu Netdisk

    Use the following command to merge and extract the archive:

    cat OphNet2024_all.tar.gz.* | tar xzvf -

  • Trimmed Videos Download Source: run the script we provided for trimming:

    python data_processing/cliper.py

    or use the link to download: HuggingFace | Baidu Netdisk. Use the following command to merge and extract the archive:

    operation level

    cat OphNet2024_trimmed_operation.tar.gz.* | tar xzvf -

    phase level

    cat OphNet2024_trimmed_phase.tar.gz.* | tar xzvf -

Baselines

Coming soon...

Challenge

Coming soon...

Discussion Group

If you have any questions about OphNet, please add this WeChat ID to the WeChat group discussion:

image

TO DO

  • Release untrimmed videos
  • Release trimmed videos--operation level
  • Release trimmed videos--phase level
  • Release annotation files
  • Release baseline experimental results and checkpoints

Citation

@article{hu2024ophnet,
  title={OphNet: A Large-Scale Video Benchmark for Ophthalmic Surgical Workflow Understanding},
  author={Hu, Ming and Xia, Peng and Wang, Lin and Yan, Siyuan and Tang, Feilong and Xu, Zhongxing and Luo, Yimin and Song, Kaimin and Leitner, Jurgen and Cheng, Xuelian and others},
  journal={arXiv preprint arXiv:2406.07471},
  year={2024}
}