๐Ÿ› ๏ธ Installation Guide#

๐Ÿ˜„ Donโ€™t worry โ€” both Quick Installation and Dataset Preparation are beginner-friendly.

Quick Installation#

git clone https://github.com/InternRobotics/InternSR.git
cd InternSR
pip install -e .

Dataset Preparation#

We recommend placing all data under data/. The expected directory structure under data/ is as follows :

data/
โ”œโ”€โ”€ images/ # `images/` folder stores all image modality files from the datasets
โ”œโ”€โ”€ videos/ # `videos/` folder contains all video modality files from the datasets
โ”œโ”€โ”€ annotations/ # `annotations/` folder holds all text annotation files from the datasets

MMScan#

  1. Download the image zip files from Hugging Face (~56G), combine and unzip them under ./data/images/mmscan.

  2. Download the annotations from Hugging Face and place them under ./data/annotations.

data/
โ”œโ”€โ”€ images/
โ”‚   โ”œโ”€โ”€ mmscan/
โ”‚   โ”‚   โ”œโ”€โ”€ 3rscan
โ”‚   โ”‚   โ”œโ”€โ”€ 3rscan_depth
โ”‚   โ”‚   โ”œโ”€โ”€ matterport3d
โ”‚   โ”‚   โ”œโ”€โ”€ scannet
โ”œโ”€โ”€ annotations/
โ”‚   โ”œโ”€โ”€ embodiedscan_video_meta/
โ”‚   โ”œโ”€โ”€ โ”œโ”€โ”€ image.json
โ”‚   โ”œโ”€โ”€ โ”œโ”€โ”€ depth.json
โ”‚   โ”œโ”€โ”€ โ”œโ”€โ”€ ...
โ”‚   โ”œโ”€โ”€ mmscan_qa_val_0.1.json
โ”‚   โ”œโ”€โ”€ ...

Note: The file mmscan_qa_val_{ratio}.json contains the validation data at the specified ratio.

OST-Bench#

Download the images from Hugging Face/Kaggle(~5G) and download the .tsv file , place them as follows:

data/
โ”œโ”€โ”€ images/
โ”‚   โ”œโ”€โ”€ OST/
โ”‚   โ”‚   โ”œโ”€โ”€ <scan_id>
โ”‚   โ”‚   โ”œโ”€โ”€ ...
โ”œโ”€โ”€ annotations/
โ”‚   โ”œโ”€โ”€ OST.tsv

MMSI-Bench#

Download the .tsv file (~1G, including images) , place it as follows:

data/
โ”œโ”€โ”€ annotations/
โ”‚   โ”œโ”€โ”€ MMSI_Bench.tsv

EgoExo-Bench#

  1. Download the processed video data from the Hugging Face.

  2. Due to license restrictions, data from the Ego-Exo4D project is not included. Users should acquire it separately by following the official Ego-Exo4D guidelines.

  3. Download the .tsv file , place them as follows:

data/
โ”œโ”€โ”€ videos/
โ”‚   โ”œโ”€โ”€ EgoExo4D/tasks
โ”‚   โ”œโ”€โ”€ processed_frames
โ”‚   โ”œโ”€โ”€ processed_video
โ”œโ”€โ”€ annotations/ EgoExoBench_MCQ.tsv