Building on HF

221 139 83

Sergio Paniego PRO

sergiopaniego

https://sergiopaniego.github.io/

AI & ML interests

None yet

Recent Activity

replied to their post 7 minutes ago

We just released TRL v0.26.0! It comes packed with updates: > Agent training with tools in GRPO > New CISPO & SAPO losses + reasoning rewards > vLLM quantization in colocate mode > Dataset shuffling in SFT > Lots of NEW examples > Tons of fixes and documentation improvements

upvoted an article 13 minutes ago

Building Deep Research: How we Achieved State of the Art

posted an update about 1 hour ago

View all activity

Organizations

replied to their post 7 minutes ago

Thanks for your interest!

The amount of memory required for training depends on many factors. If you're new, I'd recommend the following notebook that has a lot of details and runs for free on Colab:

https://colab.research.google.com/github/huggingface/trl/blob/main/examples/notebooks/sft_trl_lora_qlora.ipynb

We've even more examples that run on Colab: https://huggingface.co/docs/trl/example_overview#notebooks

Let us know how it goes!

posted an update about 1 hour ago

Post

We just released TRL v0.26.0!

It comes packed with updates:
> Agent training with tools in GRPO
> New CISPO & SAPO losses + reasoning rewards
> vLLM quantization in colocate mode
> Dataset shuffling in SFT
> Lots of NEW examples
> Tons of fixes and documentation improvements

2 replies

posted an update about 23 hours ago

Post

1583

NEW: @EssentialAI just released Rnj-1, their first 8B model.

You can easily fine-tune it with GRPO using TRL to add reasoning capabilities to a compact mode

Free Colab link: https://colab.research.google.com/github/huggingface/trl/blob/main/examples/notebooks/grpo_rnj_1_instruct.ipynb

More free TRL notebooks: https://huggingface.co/docs/trl/main/en/example_overview#notebooks

posted an update 5 days ago

Post

2747

Want to get started with fine-tuning but don’t know where to begin? 🤓☝️

We’re expanding our collection of beginner-friendly free Colab notebooks so you can learn and fine-tune models using TRL at no cost

🔬 Check out the full list of free notebooks: https://huggingface.co/docs/trl/main/en/example_overview#notebooks

🔬 If you want more advanced content, we also have a lot to cover in the community tutorials: https://huggingface.co/docs/trl/community_tutorials

And now the obvious question: what would you like us to add next?

replied to their post 6 days ago

Thanks!! I missed them since they're on another collection, my bad. Let me review and update

posted an update 7 days ago

Post

2298

NEW: @mistralai released a fantastic family of multimodal models, Ministral 3.

You can fine-tune them for free on Colab using TRL ⚡️, supporting both SFT and GRPO

Link to the notebooks:
- SFT: https://colab.research.google.com/github/huggingface/trl/blob/main/examples/notebooks/sft_ministral3_vl.ipynb
- GRPO: https://colab.research.google.com/github/huggingface/trl/blob/main/examples/notebooks/grpo_ministral3_vl.ipynb
- TRL and more examples: https://huggingface.co/docs/trl/index

2 replies

posted an update 8 days ago

Post

2133

ICYMI, transformers v5 is out!

Grab a coffee ☕ and go read the announcement blog https://huggingface.co/blog/transformers-v5

posted an update 9 days ago

Post

3071

want to use open models easily through an API?

Inference Providers might be exactly what you’re looking for sooo here’s a complete beginner-friendly walkthrough 🧐

https://www.youtube.com/watch?v=oxwsizy1Spw

2 replies

posted an update 13 days ago

Post

1716

nanochat is now in transformers!

The LLM by @karpathy is officially in the library, and we wrote a blog covering: how did we port the model, differences from the original, and how to run or train it.

go read it 🤓

nanochat-students/transformers

posted an update 15 days ago

Post

3940

you gotta go fast and go read the latest blog by @ror et al. explaining Continuous Batching in depth

https://huggingface.co/blog/continuous_batching

posted an update 16 days ago

Post

1705

Interested in RL training environments?

We just released a beginner-friendly walkthrough notebook!

Train a model to play Wordle using TRL + OpenEnv (TextArena) + GRPO + vLLM.

happy learning! 🌱

Notebook: https://github.com/huggingface/trl/blob/main/examples/notebooks/openenv_wordle_grpo.ipynb

OpenEnv guide in TRL: https://huggingface.co/docs/trl/main/en/openenv

posted an update 20 days ago

Post

324

Ya está disponible el vídeo de la charla del otro día en @nerdearla sobre IA abierta, por si queréis verla! 🤠

https://www.youtube.com/watch?v=p-JLn4xAkMw

1 reply

posted an update 21 days ago

Post

2576

we've just added several example scripts to TRL showing how to train models with GRPO using some of the new OpenEnv environments

train a model to interact with a browser (🎮 BrowserGym Env), play Wordle (🎮 Wordle Env) and moooore!

TRL (GRPO + vLLM) + OpenEnv! ⚡️

📝 go play with them: https://github.com/huggingface/trl/tree/main/examples/scripts/openenv

📝 examples list: https://huggingface.co/docs/trl/main/en/example_overview#scripts

posted an update 23 days ago

Post

1750

Who wants a TRL sticker? 🙋

https://github.com/huggingface/trl

1 reply

posted an update about 1 month ago

Post

5369

fine-tuning a 14B model with TRL + SFT on a free Colab (T4 GPU)?
thanks to the latest TRL optimizations, you actually can!
sharing a new notebook showing how to do it 😎

colab: https://colab.research.google.com/github/huggingface/trl/blob/main/examples/notebooks/sft_trl_lora_qlora.ipynb

notebooks in TRL: https://github.com/huggingface/trl/tree/main/examples/notebooks

2 replies

posted an update about 1 month ago

Post

450

Gave a smol 🤏 intro to Agents using smolagents last Monday!
Sharing the slides in case you're curious. They serve as a gentle first step into the Agents Course we developed at @huggingface 🫶🫶

Course: https://huggingface.co/learn/agents-course/unit0/introduction

Workshop material: https://github.com/sergiopaniego/talks/tree/main/intro_to_agents

posted an update about 1 month ago

Post

3136

Sharing the slides from yesterday's talk about "Fine Tuning with TRL" from the @TogetherAgent x @huggingface workshop we hosted in our Paris office 🎃!

Link: https://github.com/sergiopaniego/talks/blob/main/fine_tuning_with_trl/Fine%20tuning%20with%20TRL%20(Oct%2025).pdf

posted an update about 1 month ago

Post

428

On-Policy distillation is trendy! and super useful!

HuggingFaceH4/on-policy-distillation

posted an update about 2 months ago

Post

2911

Meet OpenEnv 👋, an open ecosystem of environments for intelligent agents. Build, share, and test agents safely and consistently.

Ideal for training with TRL (we include examples🤓), deployment, and community collaboration via the HF Hub

Blog: https://huggingface.co/blog/openenv
Hub for Environments:

openenv
OpenEnv repo: https://github.com/meta-pytorch/OpenEnv
Try it out using TRL: https://huggingface.co/docs/trl/main/en/openenv

1 reply

posted an update about 2 months ago

Post

1972

New drop! 💥 The VLM Object Understanding Comparison Space now runs with Qwen3-VL-4B and moondream3.

You can compare how models reason about images 🧠

Bonus: thanks to @ariG23498 , you now get auto-suggested prompts to explore faster.

Let’s gooo

sergiopaniego/vlm_object_understanding

Sergio Paniego PRO

AI & ML interests

Recent Activity

Organizations

sergiopaniego's activity