This codebase was created for and tested on Python 3.8. To install the requirements, please create a virtual environment and install the packages:
python -m venv env
source env/bin/activate
pip install -r requirements.txt
The project is organized into the below folders:
evaluation_code to evaluate the model after adaptation on the Human3.6M test set.
image_to_latent_encoder, motion_autoencoder, pose_autoencoder, relation_network contain codes for pre-training the respective models required for target adaptation.
pretrained_weights contains the trained weights for all of the above models.
target_adaptation code for the adaptation procedure described in the paper.
The adaptation requires pre-trained pose autoencoder, motion autoencoder, relation networks (Flip+InPlane-50 for pose space transformation, Flip+InPlane-50-backward, and slow-backward for motion space transformation), and a source-trained image-to-latent model. The codebase includes pretrained weights for these networks. To re-train any of the models for use in adaptation, please run the corresponding train.py script from the train directory. For example, to train the pose autoencoder, run the below commands:
cd pose_autoencoder
python train.py
For more details, please refer to the README.md file in the codebase.
The adaptation procedure can be applied to a target dataset by running the train.py file in the target adaptation directory. The code includes a sample of H3.6M for running the adaptation.
cd target_adaptation
python train.py
The following command can be used to run PA-MPJPE evaluation of the trained model on the H3.6M test set. Evaluation procedure for other target settings will be published soon.
cd evaluation
python eval.py
If you find our work helpful in your research, please cite our work:
@inproceedings{kundu2021non,
title={Non-Local Latent Relation Distillation for Self-Adaptive 3D Human Pose Estimation},
author={Kundu, Jogendra Nath and Seth, Siddharth and Jamkhandi, Anirudh and YM, Pradyumna and Jampani, Varun and Chakraborty, Anirban and Babu, R Venkatesh},
booktitle={Advances in Neural Information Processing Systems},
year={2021}
}
This project is licensed under an [MIT License].
If you have any queries, please get in touch via email : jogendranathkundu@gmail.com.