Spaces:
Running
Running
You get the following: | |
text | |
======== New task ======== | |
Can you give me the name of the model that has the most downloads in the 'text-to-video' task on the Hugging Face Hub? | |
==== Agent is executing the code below: | |
most_downloaded_model = model_download_counter(task="text-to-video") | |
print(f"The most downloaded model for the 'text-to-video' task is {most_downloaded_model}.") | |
==== | |
And the output: | |
"The most downloaded model for the 'text-to-video' task is ByteDance/AnimateDiff-Lightning." | |
Manage agent toolbox | |
If you have already initialized an agent, it is inconvenient to reinitialize it from scratch with a tool you want to use. With Transformers, you can manage an agent's toolbox by adding or replacing a tool. | |
Let's add the model_download_tool to an existing agent initialized with only the default toolbox. | |
thon | |
from transformers import CodeAgent | |
agent = CodeAgent(tools=[], llm_engine=llm_engine, add_base_tools=True) | |
agent.toolbox.add_tool(model_download_tool) | |
Now we can leverage both the new tool and the previous text-to-speech tool: | |
python | |
agent.run( | |
"Can you read out loud the name of the model that has the most downloads in the 'text-to-video' task on the Hugging Face Hub and return the audio?" | |
) | |
| Audio | | |
|------------------------------------------------------------------------------------------------------------------------------------------------------| | |
| | | |
[!WARNING] | |
Beware when adding tools to an agent that already works well because it can bias selection towards your tool or select another tool other than the one already defined. | |
Use the agent.toolbox.update_tool() method to replace an existing tool in the agent's toolbox. | |
This is useful if your new tool is a one-to-one replacement of the existing tool because the agent already knows how to perform that specific task. | |
Just make sure the new tool follows the same API as the replaced tool or adapt the system prompt template to ensure all examples using the replaced tool are updated. | |
Use a collection of tools | |
You can leverage tool collections by using the ToolCollection object, with the slug of the collection you want to use. | |
Then pass them as a list to initialize you agent, and start using them! | |
from transformers import ToolCollection, ReactCodeAgent | |
image_tool_collection = ToolCollection(collection_slug="huggingface-tools/diffusion-tools-6630bb19a942c2306a2cdb6f") | |
agent = ReactCodeAgent(tools=[*image_tool_collection.tools], add_base_tools=True) | |
agent.run("Please draw me a picture of rivers and lakes.") | |
To speed up the start, tools are loaded only if called by the agent. | |
This gets you this image: | |
Use gradio-tools | |
gradio-tools is a powerful library that allows using Hugging | |
Face Spaces as tools. It supports many existing Spaces as well as custom Spaces. | |
Transformers supports gradio_tools with the [Tool.from_gradio] method. For example, let's use the StableDiffusionPromptGeneratorTool from gradio-tools toolkit for improving prompts to generate better images. | |
Import and instantiate the tool, then pass it to the Tool.from_gradio method: | |
thon | |
from gradio_tools import StableDiffusionPromptGeneratorTool | |
from transformers import Tool, load_tool, CodeAgent | |
gradio_prompt_generator_tool = StableDiffusionPromptGeneratorTool() | |
prompt_generator_tool = Tool.from_gradio(gradio_prompt_generator_tool) | |
Now you can use it just like any other tool. For example, let's improve the prompt a rabbit wearing a space suit. | |
thon | |
image_generation_tool = load_tool('huggingface-tools/text-to-image') | |
agent = CodeAgent(tools=[prompt_generator_tool, image_generation_tool], llm_engine=llm_engine) | |
agent.run( | |
"Improve this prompt, then generate an image of it.", prompt='A rabbit wearing a space suit' | |
) |