nielsr HF staff commited on
Commit
ab3c323
1 Parent(s): e2483b2

Add model card

Browse files

This PR adds a model card, with installation and usage sections, and links to https://huggingface.co/papers/2403.15378.

Cheers!

Files changed (1) hide show
  1. README.md +45 -3
README.md CHANGED
@@ -1,3 +1,45 @@
1
- ---
2
- license: apache-2.0
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ pipeline_tag: zero-shot-image-classification
4
+ library_name: openclip
5
+ ---
6
+
7
+ # LongCLIP model
8
+
9
+ This repository contains the weights of the LongCLIP model.
10
+
11
+ Paper: https://huggingface.co/papers/2403.15378
12
+
13
+ Github repository: https://github.com/beichenzbc/long-clip
14
+
15
+ ## Installation
16
+
17
+ ```bash
18
+ git clone https://github.com/beichenzbc/Long-CLIP.git
19
+ cd Long-CLIP
20
+ ```
21
+
22
+ ## Usage
23
+
24
+ ```
25
+ from model import longclip
26
+ import torch
27
+ from PIL import Image
28
+ from huggingface_hub import hf_hub_download
29
+
30
+ device = "cuda" if torch.cuda.is_available() else "cpu"
31
+ filepath = hf_hub_download(repo_id="BeichenZhang/LongCLIP-L-336px", filename="longclip-L@336px.pt")
32
+ model, preprocess = longclip.load(filepath, device=device)
33
+
34
+ text = longclip.tokenize(["A man is crossing the street with a red car parked nearby.", "A man is driving a car in an urban scene."]).to(device)
35
+ image = preprocess(Image.open("./img/demo.png")).unsqueeze(0).to(device)
36
+
37
+ with torch.no_grad():
38
+ image_features = model.encode_image(image)
39
+ text_features = model.encode_text(text)
40
+
41
+ logits_per_image = image_features @ text_features.T
42
+ probs = logits_per_image.softmax(dim=-1).cpu().numpy()
43
+
44
+ print("Label probs:", probs)
45
+ ```