|
I received the M.S. degree (with distinction in research) from the School of Engineering at EPFL in February 2024, specializing in Automatic and Systems. Previously, I completed my master thesis at TML lab, EPFL, supervised by Nicolas Flammarion, and will continue my research at TML lab as a research assistant. I am interested in developing effective methods that enable AI systems to adapt efficiently to new environments and tasks. During my master studies, I was also fortunate to spend time as a semester project student at VITA lab, EPFL, supervised by Alexandre Alahi. I got my bachelor's degree from Zhejiang University. I am actively looking for PhD positions starting from 2025 Fall and research assistant positions starting from 2024 Fall. If you think my background is a good fit, please don't hesitate to get in touch! |
|
|
[May 02, 2024] One work on data selection for Instruction Fine-Tuning LLMs was accepted to ICML 2024. See (some of) you in Vienna again! [Mar 04, 2024] Our work Long Is More for Alignment: A Simple but Tough-to-Beat Baseline for Instruction Fine-Tuning was accepted to Workshop DMLR@ICLR'24. Many thanks to anonymous reviewers for their constructive and helpful reviews! [Feb 29, 2024] I successfully defended my master thesis with a perfect grade of 6.0/6.0 and received nominations for EPFL Master Thesis Awards. Great thanks to my committee: Nicolas Flammarion, Giancarlo Ferrari Trecate, Tao Lin, and my supervisors: Maksym Andriushchenko, Francesco Croce. [Feb 07, 2024] Our new paper Long Is More for Alignment: A Simple but Tough-to-Beat Baseline for Instruction Fine-Tuning is available online (see a Twitter/X thread for a summary). Great thanks to Maksym for spreading the word. [Jan 16, 2024] One work on Parameter-efficient Fine-tuning was accepted to ICLR 2024. See you in Vienna! [Apr 25, 2023] One work on Test-time Adaptation was accepted to ICML 2023. |
|
|
|
Hao Zhao, Maksym Andriushchenko, Francesco Croce, Nicolas Flammarion ICML 2024, abridged in Workshop DMLR@ICLR'24 [arXiv] [HuggingFace] [Code] We show that filtering via length heuristics is an effective and efficient approach to select instruction tuning data for alignment. In addition, we propose a lightweight refinement of long instructions which can further improve the abilities of the fine-tuned LLMs. In particular, we obtain the 2nd highest-ranked Llama-2-7B-based model on AlpacaEval 2.0 while training on only 1,000 examples and no extra preference data. |
|
Haobo Song*, Hao Zhao*, Soumajit Majumder, Tao Lin ICLR 2024 [arXiv] [Code] We propose CapaBoost, a simple yet effective strategy that enhances model capacity by leveraging low-rank updates through parallel weight modules in target layers. By applying static random masks to the shared weight matrix, CapaBoost constructs a diverse set of weight matrices, effectively increasing the rank of incremental weights without adding parameters. Notably, our approach can be seamlessly integrated into various existing parameter-efficient fine-tuning methods. |
|
Hao Zhao* Yuejiang Liu*, Alexandre Alahi, Tao Lin ICML 2023 [arXiv] [Code] We present TTAB, a test-time adaptation benchmark that encompasses ten state-of-the-art algorithms, a diverse array of distribution shifts, and two evaluation protocols. Through extensive experiments, our benchmark reveals three common pitfalls in prior efforts. Our findings underscore the need for future research in the field to conduct rigorous evaluations on a broader set of models and shifts, and to re-examine the assumptions behind the empirical success of TTA. |
|
[2021.9 - 2024.2] M.S. in Automatic and Systems, EPFL, Switzerland [2017.9 - 2021.6] B.Eng. in Mechanical Engineering, Zhejiang University, China |
|
I borrowed this website layout from here! |