+ 
+
+
+Instructions:
+ - Press keyboard arrow keys `
+
+
+
+
+
+# Download files from the Hub
+
+The `huggingface_hub` library provides functions to download files from the repositories
+stored on the Hub. You can use these functions independently or integrate them into your
+own library, making it more convenient for your users to interact with the Hub. This
+guide will show you how to:
+
+* Download and cache a single file.
+* Download and cache an entire repository.
+* Download files to a local folder.
+
+## Download a single file
+
+The `hf_hub_download()` function is the main function for downloading files from the Hub.
+It downloads the remote file, caches it on disk (in a version-aware way), and returns its local file path.
+
+
+
+
+
+
+ Repository
+
+ How to create a repository on the Hub? How to configure it? How to interact with it? +
+ + + +
+ Download files
+
+ How do I download a file from the Hub? How do I download a repository? +
+ + + +
+ Upload files
+
+ How to upload a file or a folder? How to make changes to an existing repository on the Hub? +
+ + + +
+ Search
+
+ How to efficiently search through the 200k+ public models, datasets and spaces? +
+ + + +
+ HfFileSystem
+
+ How to interact with the Hub through a convenient interface that mimics Python's file interface? +
+ + + +
+ Inference
+
+ How to make predictions using the accelerated Inference API? +
+ + + +
+ Community Tab
+
+ How to interact with the Community tab (Discussions and Pull Requests)? +
+ + + +
+ Collections
+
+ How to programmatically build collections? +
+ + + +
+ Cache
+
+ How does the cache-system work? How to benefit from it? +
+ + + +
+ Model Cards
+
+ How to create and share Model Cards? +
+ + + +
+ Manage your Space
+
+ How to manage your Space hardware and configuration? +
+ + + +
+ Integrate a library
+
+ What does it mean to integrate a library with the Hub? And how to do it? +
+ + + +
+ Webhooks server
+
+ How to create a server to receive Webhooks and deploy it as a Space? +
+ + +`push_to_hub(model, ...)` | `model = MyModel.from_pretrained(...)`
`model.push_to_hub(...)` | +| Flexibility | Very flexible.
You fully control the implementation. | Less flexible.
Your framework must have a model class. | +| Maintenance | More maintenance to add support for configuration, and new features. Might also require fixing issues reported by users. | Less maintenance as most of the interactions with the Hub are implemented in `huggingface_hub`. | +| Documentation / Type annotation | To be written manually. | Partially handled by `huggingface_hub`. | +| Download counter | To be handled manually. | Enabled by default if class has a `config` attribute. | +| Model card | To be handled manually | Generated by default with library_name, tags, etc. | + + + +# Webhooks + +Webhooks are a foundation for MLOps-related features. They allow you to listen for new changes on specific repos or to all repos belonging to particular users/organizations you're interested in following. This guide will first explain how to manage webhooks programmatically. Then we'll see how to leverage `huggingface_hub` to create a server listening to webhooks and deploy it to a Space. + +This guide assumes you are familiar with the concept of webhooks on the Huggingface Hub. To learn more about webhooks themselves, you should read this [guide](https://huggingface.co/docs/hub/webhooks) first. + +## Managing Webhooks + +`huggingface_hub` allows you to manage your webhooks programmatically. You can list your existing webhooks, create new ones, and update, enable, disable or delete them. This section guides you through the procedures using the Hugging Face Hub's API functions. + +### Creating a Webhook + +To create a new webhook, use `create_webhook()` and specify the URL where payloads should be sent, what events should be watched, and optionally set a domain and a secret for security. + +```python +from huggingface_hub import create_webhook + +# Example: Creating a webhook +webhook = create_webhook( + url="https://webhook.site/your-custom-url", + watched=[{"type": "user", "name": "your-username"}, {"type": "org", "name": "your-org-name"}], + domains=["repo", "discussion"], + secret="your-secret" +) +``` + +### Listing Webhooks + +To see all the webhooks you have configured, you can list them with `list_webhooks()`. This is useful to review their IDs, URLs, and statuses. + +```python +from huggingface_hub import list_webhooks + +# Example: Listing all webhooks +webhooks = list_webhooks() +for webhook in webhooks: + print(webhook) +``` + +### Updating a Webhook + +If you need to change the configuration of an existing webhook, such as the URL or the events it watches, you can update it using `update_webhook()`. + +```python +from huggingface_hub import update_webhook + +# Example: Updating a webhook +updated_webhook = update_webhook( + webhook_id="your-webhook-id", + url="https://new.webhook.site/url", + watched=[{"type": "user", "name": "new-username"}], + domains=["repo"] +) +``` + +### Enabling and Disabling Webhooks + +You might want to temporarily disable a webhook without deleting it. This can be done using `disable_webhook()`, and the webhook can be re-enabled later with `enable_webhook()`. + +```python +from huggingface_hub import enable_webhook, disable_webhook + +# Example: Enabling a webhook +enabled_webhook = enable_webhook("your-webhook-id") +print("Enabled:", enabled_webhook) + +# Example: Disabling a webhook +disabled_webhook = disable_webhook("your-webhook-id") +print("Disabled:", disabled_webhook) +``` + +### Deleting a Webhook + +When a webhook is no longer needed, it can be permanently deleted using `delete_webhook()`. + +```python +from huggingface_hub import delete_webhook + +# Example: Deleting a webhook +delete_webhook("your-webhook-id") +``` + +## Webhooks Server + +The base class that we will use in this guides section is `WebhooksServer()`. It is a class for easily configuring a server that +can receive webhooks from the Huggingface Hub. The server is based on a [Gradio](https://gradio.app/) app. It has a UI +to display instructions for you or your users and an API to listen to webhooks. + +
+
+
+
+And that's it! You can now trigger that webhook by updating the target repository (e.g. push a commit). Check the
+Activity tab of your Webhook to see the events that have been triggered. Now that you have a working setup, you can
+test it and quickly iterate. If you modify your code and restart the server, your public URL might change. Make sure
+to update the webhook configuration on the Hub if needed.
+
+### Deploy to a Space
+
+Now that you have a working webhook server, the goal is to deploy it to a Space. Go to https://huggingface.co/new-space
+to create a Space. Give it a name, select the Gradio SDK and click on "Create Space". Upload your code to the Space
+in a file called `app.py`. Your Space will start automatically! For more details about Spaces, please refer to this
+[guide](https://huggingface.co/docs/hub/spaces-overview).
+
+Your webhook server is now running on a public Space. If most cases, you will want to secure it with a secret. Go to
+your Space settings > Section "Repository secrets" > "Add a secret". Set the `WEBHOOK_SECRET` environment variable to
+the value of your choice. Go back to the [Webhooks settings](https://huggingface.co/settings/webhooks) and set the
+secret in the webhook configuration. Now, only requests with the correct secret will be accepted by your server.
+
+And this is it! Your Space is now ready to receive webhooks from the Hub. Please keep in mind that if you run the Space
+on a free 'cpu-basic' hardware, it will be shut down after 48 hours of inactivity. If you need a permanent Space, you
+should consider setting to an [upgraded hardware](https://huggingface.co/docs/hub/spaces-gpus#hardware-specs).
+
+### Advanced usage
+
+The guide above explained the quickest way to setup a `WebhooksServer()`. In this section, we will see how to customize
+it further.
+
+#### Multiple endpoints
+
+You can register multiple endpoints on the same server. For example, you might want to have one endpoint to trigger
+a training job and another one to trigger a model evaluation. You can do this by adding multiple `@webhook_endpoint`
+decorators:
+
+```python
+# app.py
+from huggingface_hub import webhook_endpoint, WebhookPayload
+
+@webhook_endpoint
+async def trigger_training(payload: WebhookPayload) -> None:
+ if payload.repo.type == "dataset" and payload.event.action == "update":
+ # Trigger a training job if a dataset is updated
+ ...
+
+@webhook_endpoint
+async def trigger_evaluation(payload: WebhookPayload) -> None:
+ if payload.repo.type == "model" and payload.event.action == "update":
+ # Trigger an evaluation job if a model is updated
+ ...
+```
+
+Which will create two endpoints:
+
+```text
+(...)
+Webhooks are correctly setup and ready to use:
+ - POST https://1fadb0f52d8bf825fc.gradio.live/webhooks/trigger_training
+ - POST https://1fadb0f52d8bf825fc.gradio.live/webhooks/trigger_evaluation
+```
+
+#### Custom server
+
+To get more flexibility, you can also create a `WebhooksServer()` object directly. This is useful if you want to
+customize the landing page of your server. You can do this by passing a [Gradio UI](https://gradio.app/docs/#blocks)
+that will overwrite the default one. For example, you can add instructions for your users or add a form to manually
+trigger the webhooks. When creating a `WebhooksServer()`, you can register new webhooks using the
+`add_webhook()` decorator.
+
+Here is a complete example:
+
+```python
+import gradio as gr
+from fastapi import Request
+from huggingface_hub import WebhooksServer, WebhookPayload
+
+# 1. Define UI
+with gr.Blocks() as ui:
+ ...
+
+# 2. Create WebhooksServer with custom UI and secret
+app = WebhooksServer(ui=ui, webhook_secret="my_secret_key")
+
+# 3. Register webhook with explicit name
+@app.add_webhook("/say_hello")
+async def hello(payload: WebhookPayload):
+ return {"message": "hello"}
+
+# 4. Register webhook with implicit name
+@app.add_webhook
+async def goodbye(payload: WebhookPayload):
+ return {"message": "goodbye"}
+
+# 5. Start server (optional)
+app.run()
+```
+
+1. We define a custom UI using Gradio blocks. This UI will be displayed on the landing page of the server.
+2. We create a `WebhooksServer()` object with a custom UI and a secret. The secret is optional and can be set with
+the `WEBHOOK_SECRET` environment variable.
+3. We register a webhook with an explicit name. This will create an endpoint at `/webhooks/say_hello`.
+4. We register a webhook with an implicit name. This will create an endpoint at `/webhooks/goodbye`.
+5. We start the server. This is optional as your server will automatically be started at the end of the script.
+
+
+
+# Create and share Model Cards
+
+The `huggingface_hub` library provides a Python interface to create, share, and update Model Cards.
+Visit [the dedicated documentation page](https://huggingface.co/docs/hub/models-cards)
+for a deeper view of what Model Cards on the Hub are, and how they work under the hood.
+
+
+
+
+
+
+The `repo_type_prefix` is `datasets/` for datasets, `spaces/` for spaces, and models don't need a prefix in the URL.
+
+Some interesting integrations where `HfFileSystem` simplifies interacting with the Hub are listed below:
+
+* Reading/writing a [Pandas](https://pandas.pydata.org/pandas-docs/stable/user_guide/io.html#reading-writing-remote-files) DataFrame from/to a Hub repository:
+
+ ```python
+ >>> import pandas as pd
+
+ >>> # Read a remote CSV file into a dataframe
+ >>> df = pd.read_csv("hf://datasets/my-username/my-dataset-repo/train.csv")
+
+ >>> # Write a dataframe to a remote CSV file
+ >>> df.to_csv("hf://datasets/my-username/my-dataset-repo/test.csv")
+ ```
+
+The same workflow can also be used for [Dask](https://docs.dask.org/en/stable/how-to/connect-to-remote-data.html) and [Polars](https://pola-rs.github.io/polars/py-polars/html/reference/io.html) DataFrames.
+
+* Querying (remote) Hub files with [DuckDB](https://duckdb.org/docs/guides/python/filesystems):
+
+ ```python
+ >>> from huggingface_hub import HfFileSystem
+ >>> import duckdb
+
+ >>> fs = HfFileSystem()
+ >>> duckdb.register_filesystem(fs)
+ >>> # Query a remote file and get the result back as a dataframe
+ >>> fs_query_file = "hf://datasets/my-username/my-dataset-repo/data_dir/data.parquet"
+ >>> df = duckdb.query(f"SELECT * FROM '{fs_query_file}' LIMIT 10").df()
+ ```
+
+* Using the Hub as an array store with [Zarr](https://zarr.readthedocs.io/en/stable/tutorial.html#io-with-fsspec):
+
+ ```python
+ >>> import numpy as np
+ >>> import zarr
+
+ >>> embeddings = np.random.randn(50000, 1000).astype("float32")
+
+ >>> # Write an array to a repo
+ >>> with zarr.open_group("hf://my-username/my-model-repo/array-store", mode="w") as root:
+ ... foo = root.create_group("embeddings")
+ ... foobar = foo.zeros('experiment_0', shape=(50000, 1000), chunks=(10000, 1000), dtype='f4')
+ ... foobar[:] = embeddings
+
+ >>> # Read an array from a repo
+ >>> with zarr.open_group("hf://my-username/my-model-repo/array-store", mode="r") as root:
+ ... first_row = root["embeddings/experiment_0"][0]
+ ```
+
+## Authentication
+
+In many cases, you must be logged in with a Hugging Face account to interact with the Hub. Refer to the [Authentication](../quick-start#authentication) section of the documentation to learn more about authentication methods on the Hub.
+
+It is also possible to log in programmatically by passing your `token` as an argument to `HfFileSystem`:
+
+```python
+>>> from huggingface_hub import HfFileSystem
+>>> fs = HfFileSystem(token=token)
+```
+
+If you log in this way, be careful not to accidentally leak the token when sharing your source code!
+
+
+
+# Create and manage a repository
+
+The Hugging Face Hub is a collection of git repositories. [Git](https://git-scm.com/) is a widely used tool in software
+development to easily version projects when working collaboratively. This guide will show you how to interact with the
+repositories on the Hub, especially:
+
+- Create and delete a repository.
+- Manage branches and tags.
+- Rename your repository.
+- Update your repository visibility.
+- Manage a local copy of your repository.
+
+
+
+ 
+
+
+Let's fetch the collection with, `"TheBloke/recent-models-64f9a55bb3115b4f513ec026"`:
+
+```py
+>>> from huggingface_hub import get_collection
+>>> collection = get_collection("TheBloke/recent-models-64f9a55bb3115b4f513ec026")
+>>> collection
+Collection(
+ slug='TheBloke/recent-models-64f9a55bb3115b4f513ec026',
+ title='Recent models',
+ owner='TheBloke',
+ items=[...],
+ last_updated=datetime.datetime(2023, 10, 2, 22, 56, 48, 632000, tzinfo=datetime.timezone.utc),
+ position=1,
+ private=False,
+ theme='green',
+ upvotes=90,
+ description="Models I've recently quantized. Please note that currently this list has to be updated manually, and therefore is not guaranteed to be up-to-date."
+)
+>>> collection.items[0]
+CollectionItem(
+ item_object_id='651446103cd773a050bf64c2',
+ item_id='TheBloke/U-Amethyst-20B-AWQ',
+ item_type='model',
+ position=88,
+ note=None
+)
+```
+
+The `Collection` object returned by `get_collection()` contains:
+- high-level metadata: `slug`, `owner`, `title`, `description`, etc.
+- a list of `CollectionItem` objects; each item represents a model, a dataset, a Space, or a paper.
+
+All collection items are guaranteed to have:
+- a unique `item_object_id`: this is the id of the collection item in the database
+- an `item_id`: this is the id on the Hub of the underlying item (model, dataset, Space, paper); it is not necessarily unique, and only the `item_id`/`item_type` pair is unique
+- an `item_type`: model, dataset, Space, paper
+- the `position` of the item in the collection, which can be updated to reorganize your collection (see `update_collection_item()` below)
+
+A `note` can also be attached to the item. This is useful to add additional information about the item (a comment, a link to a blog post, etc.). The attribute still has a `None` value if an item doesn't have a note.
+
+In addition to these base attributes, returned items can have additional attributes depending on their type: `author`, `private`, `lastModified`, `gated`, `title`, `likes`, `upvotes`, etc. None of these attributes are guaranteed to be returned.
+
+## List collections
+
+We can also retrieve collections using `list_collections()`. Collections can be filtered using some parameters. Let's list all the collections from the user [`teknium`](https://huggingface.co/teknium).
+```py
+>>> from huggingface_hub import list_collections
+
+>>> collections = list_collections(owner="teknium")
+```
+
+This returns an iterable of `Collection` objects. We can iterate over them to print, for example, the number of upvotes for each collection.
+
+```py
+>>> for collection in collections:
+... print("Number of upvotes:", collection.upvotes)
+Number of upvotes: 1
+Number of upvotes: 5
+```
+
+
+