sseung0703
commited on
Commit
•
80c3093
1
Parent(s):
ea793a2
Update README.md
Browse files
README.md
CHANGED
@@ -14,6 +14,74 @@ https://discuss.huggingface.co/t/open-to-the-community-community-week-using-jax-
|
|
14 |
We will also upload our project on the Hugging Face Hub Repository.
|
15 |
[https://huggingface.co/flax-community/putting-nerf-on-a-diet/](https://huggingface.co/flax-community/putting-nerf-on-a-diet/)
|
16 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
17 |
## Demo
|
18 |
[https://huggingface.co/spaces/flax-community/DietNerf-Demo](https://huggingface.co/spaces/flax-community/DietNerf-Demo)
|
19 |
|
@@ -26,5 +94,29 @@ We will also upload our project on the Hugging Face Hub Repository.
|
|
26 |
| Cloud TPU Team | Leader : [Alex Lau](https://github.com/riven314), Members : [JaeYoung Chung](https://github.com/robot0321), [Sunghyun Kim](https://github.com/MrBananaHuman), [Aswin Pyakurel](https://github.com/masapasa) |
|
27 |
| Project Managing | [Stella Yang](https://github.com/codestella) To Watch Our Project Progress, Please Check [Our Project Notion](https://www.notion.so/Putting-NeRF-on-a-Diet-e0caecea0c2b40c3996c83205baf870d) |
|
28 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
29 |
## License
|
30 |
[Apache License 2.0](https://github.com/codestella/putting-nerf-on-a-diet/blob/main/LICENSE)
|
|
|
14 |
We will also upload our project on the Hugging Face Hub Repository.
|
15 |
[https://huggingface.co/flax-community/putting-nerf-on-a-diet/](https://huggingface.co/flax-community/putting-nerf-on-a-diet/)
|
16 |
|
17 |
+
Our JAX/Flax implementation currently supports:
|
18 |
+
|
19 |
+
<table class="tg">
|
20 |
+
<thead>
|
21 |
+
<tr>
|
22 |
+
<th class="tg-0lax"><span style="font-weight:bold">Platform</span></th>
|
23 |
+
<th class="tg-0lax" colspan="2"><span style="font-weight:bold">Single-Host GPU</span></th>
|
24 |
+
<th class="tg-0lax" colspan="2"><span style="font-weight:bold">Multi-Device TPU</span></th>
|
25 |
+
</tr>
|
26 |
+
</thead>
|
27 |
+
<tbody>
|
28 |
+
<tr>
|
29 |
+
<td class="tg-0lax"><span style="font-weight:bold">Type</span></td>
|
30 |
+
<td class="tg-0lax">Single-Device</td>
|
31 |
+
<td class="tg-0lax">Multi-Device</td>
|
32 |
+
<td class="tg-0lax">Single-Host</td>
|
33 |
+
<td class="tg-0lax">Multi-Host</td>
|
34 |
+
</tr>
|
35 |
+
<tr>
|
36 |
+
<td class="tg-0lax"><span style="font-weight:bold">Training</span></td>
|
37 |
+
<td class="tg-0lax"><img src="http://storage.googleapis.com/gresearch/jaxnerf/check.png" alt="Supported" width=18px height=18px></td>
|
38 |
+
<td class="tg-0lax"><img src="http://storage.googleapis.com/gresearch/jaxnerf/check.png" alt="Supported" width=18px height=18px></td>
|
39 |
+
<td class="tg-0lax"><img src="http://storage.googleapis.com/gresearch/jaxnerf/check.png" alt="Supported" width=18px height=18px></td>
|
40 |
+
<td class="tg-0lax"><img src="http://storage.googleapis.com/gresearch/jaxnerf/check.png" alt="Supported" width=18px height=18px></td>
|
41 |
+
</tr>
|
42 |
+
<tr>
|
43 |
+
<td class="tg-0lax"><span style="font-weight:bold">Evaluation</span></td>
|
44 |
+
<td class="tg-0lax"><img src="http://storage.googleapis.com/gresearch/jaxnerf/check.png" alt="Supported" width=18px height=18px></td>
|
45 |
+
<td class="tg-0lax"><img src="http://storage.googleapis.com/gresearch/jaxnerf/check.png" alt="Supported" width=18px height=18px></td>
|
46 |
+
<td class="tg-0lax"><img src="http://storage.googleapis.com/gresearch/jaxnerf/check.png" alt="Supported" width=18px height=18px></td>
|
47 |
+
<td class="tg-0lax"><img src="http://storage.googleapis.com/gresearch/jaxnerf/check.png" alt="Supported" width=18px height=18px></td>
|
48 |
+
</tr>
|
49 |
+
</tbody>
|
50 |
+
</table>
|
51 |
+
|
52 |
+
## Installation
|
53 |
+
|
54 |
+
```
|
55 |
+
# Clone the repo
|
56 |
+
svn export https://github.com/google-research/google-research/trunk/jaxnerf
|
57 |
+
# Create a conda environment, note you can use python 3.6-3.8 as
|
58 |
+
# one of the dependencies (TensorFlow) hasn't supported python 3.9 yet.
|
59 |
+
conda create --name jaxnerf python=3.6.12; conda activate jaxnerf
|
60 |
+
# Prepare pip
|
61 |
+
conda install pip; pip install --upgrade pip
|
62 |
+
# Install requirements
|
63 |
+
pip install -r jaxnerf/requirements.txt
|
64 |
+
# [Optional] Install GPU and TPU support for Jax
|
65 |
+
# Remember to change cuda101 to your CUDA version, e.g. cuda110 for CUDA 11.0.
|
66 |
+
pip install --upgrade jax jaxlib==0.1.57+cuda101 -f https://storage.googleapis.com/jax-releases/jax_releases.html
|
67 |
+
# install flax and flax-transformer
|
68 |
+
pip install flax transformer[flax]
|
69 |
+
```
|
70 |
+
|
71 |
+
## Dataset
|
72 |
+
Download the datasets from the [NeRF official Google Drive](https://drive.google.com/drive/folders/128yBriW1IG_3NJ5Rp7APSTZsJqdJdfc1).
|
73 |
+
Please download the `nerf_synthetic.zip` and unzip them
|
74 |
+
in the place you like. Let's assume they are placed under `/tmp/jaxnerf/data/`.
|
75 |
+
|
76 |
+
## How to use
|
77 |
+
```
|
78 |
+
python -m train \
|
79 |
+
--data_dir=/PATH/TO/YOUR/SCENE/DATA \ % e.g., nerf_synthetic/lego
|
80 |
+
--train_dir=/PATH/TO/THE/PLACE/YOU/WANT/TO/SAVE/CHECKPOINTS \
|
81 |
+
--config=configs/CONFIG_YOU_LIKE
|
82 |
+
```
|
83 |
+
you can toggle the semantic loss by “use_semantic_loss” in cofiguration files.
|
84 |
+
|
85 |
## Demo
|
86 |
[https://huggingface.co/spaces/flax-community/DietNerf-Demo](https://huggingface.co/spaces/flax-community/DietNerf-Demo)
|
87 |
|
|
|
94 |
| Cloud TPU Team | Leader : [Alex Lau](https://github.com/riven314), Members : [JaeYoung Chung](https://github.com/robot0321), [Sunghyun Kim](https://github.com/MrBananaHuman), [Aswin Pyakurel](https://github.com/masapasa) |
|
95 |
| Project Managing | [Stella Yang](https://github.com/codestella) To Watch Our Project Progress, Please Check [Our Project Notion](https://www.notion.so/Putting-NeRF-on-a-Diet-e0caecea0c2b40c3996c83205baf870d) |
|
96 |
|
97 |
+
## References
|
98 |
+
This project is based on “JAX-NeRF”.
|
99 |
+
```
|
100 |
+
@software{jaxnerf2020github,
|
101 |
+
author = {Boyang Deng and Jonathan T. Barron and Pratul P. Srinivasan},
|
102 |
+
title = {{JaxNeRF}: an efficient {JAX} implementation of {NeRF}},
|
103 |
+
url = {https://github.com/google-research/google-research/tree/master/jaxnerf},
|
104 |
+
version = {0.0},
|
105 |
+
year = {2020},
|
106 |
+
}
|
107 |
+
```
|
108 |
+
|
109 |
+
This project is based on “JAX-NeRF”.
|
110 |
+
```
|
111 |
+
@misc{jain2021putting,
|
112 |
+
title={Putting NeRF on a Diet: Semantically Consistent Few-Shot View Synthesis},
|
113 |
+
author={Ajay Jain and Matthew Tancik and Pieter Abbeel},
|
114 |
+
year={2021},
|
115 |
+
eprint={2104.00677},
|
116 |
+
archivePrefix={arXiv},
|
117 |
+
primaryClass={cs.CV}
|
118 |
+
}
|
119 |
+
```
|
120 |
+
|
121 |
## License
|
122 |
[Apache License 2.0](https://github.com/codestella/putting-nerf-on-a-diet/blob/main/LICENSE)
|