README.md

---
license: mit
pipeline_tag: video-classification
---

# VideoMAEv2_TikTok

We provide pre-trained weights on the **TikTokActions** dataset for two backbones: **ViT-B** (Vision Transformer-Base) and **ViT-Giant**. Additionally, we include fine-tuned weights on **Kinetics-400** for both backbones.

## Pre-trained and Fine-tuned Weights
- **Pre-trained weights on TikTokActions**: These weights were trained using TikTok video clips categorized into multiple actions. The dataset consists of 283,582 unique videos across 386 hashtags.
- **Fine-tuned weights on Kinetics-400**: After pre-training, the models were fine-tuned on Kinetics-400, achieving state-of-the-art results.

We also provide the `log.txt` file, which includes information on the fine-tuning process.

To use the weights and fine-tuning scripts, please refer to [VideoMAEv2's GitHub repository](https://github.com/OpenGVLab/VideoMAEv2) for implementation details.

## Citation

For **VideoMAEv2**, please cite the following works:

@InProceedings{wang2023videomaev2, author = {Wang, Limin and Huang, Bingkun and Zhao, Zhiyu and Tong, Zhan and He, Yinan and Wang, Yi and Wang, Yali and Qiao, Yu}, title = {VideoMAE V2: Scaling Video Masked Autoencoders With Dual Masking}, booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)}, month = {June}, year = {2023}, pages = {14549-14560} }

@misc{videomaev2, title={VideoMAE V2: Scaling Video Masked Autoencoders with Dual Masking}, author={Limin Wang and Bingkun Huang and Zhiyu Zhao and Zhan Tong and Yinan He and Yi Wang and Yali Wang and Yu Qiao}, year={2023}, eprint={2303.16727}, archivePrefix={arXiv}, primaryClass={cs.CV} }


For **our repository**, please cite the following paper:

@article{qian2024actionrecognition, author = {Yang Qian, Yinan Sun, Ali Kargarandehkordi, Parnian Azizian, Onur Cezmi Mutlu, Saimourya Surabhi, Pingyi Chen, Zain Jabbar, Dennis Paul Wall, Peter Washington}, title = {Advancing Human Action Recognition with Foundation Models trained on Unlabeled Public Videos}, journal = {arXiv preprint arXiv:2402.08875}, year = {2024}, pages = {10}, doi = {https://doi.org/10.48550/arXiv.2402.08875} }


## Results

Our model achieves the following results on established action recognition benchmarks using the **ViT-Giant** backbone:
- **UCF101**: 99.05%
- **HMDB51**: 86.08%
- **Kinetics-400**: 85.51%
- **Something-Something V2**: 74.27%

These results highlight the power of using diverse, unlabeled, and dynamic video content for training foundation models, especially in the domain of action recognition.

## License
This project is licensed under the MIT License.