- 4
- 58 876
Kaiwen Wang
เข้าร่วมเมื่อ 3 ต.ค. 2011
Computer Science PhD at Cornell
ex-Facebook Engineer
ex-Facebook Engineer
Conditional Language Policy for Steerable Alignment
📝 "Conditional Language Policy: A General Framework for Steerable Multi-Objective Finetuning", to appear at EMNLP 2024.
🔗 Paper link: arxiv.org/abs/2407.15762
Abstract: Reward-based finetuning is crucial for aligning language policies with intended behaviors (e.g., creativity and safety). A key challenge is to develop steerable language models that trade-off multiple (conflicting) objectives in a flexible and efficient manner. This paper presents Conditional Language Policy (CLP), a general framework for finetuning language models on multiple objectives. Building on techniques from multi-task training and parameter-efficient finetuning, CLP learn steerable models that effectively trade-off conflicting objectives at inference time. Notably, this does not require training or maintaining multiple models to achieve different trade-offs between the objectives. Through extensive experiments and ablations on two summarization datasets, we show that CLP learns steerable language models that outperform and Pareto-dominate the existing approaches for multi-objective finetuning.
Keywords: Reinforcement Learning, Multi-Objective Finetuning, Multi-task Learning, Parameter Efficient Training
🔗 Paper link: arxiv.org/abs/2407.15762
Abstract: Reward-based finetuning is crucial for aligning language policies with intended behaviors (e.g., creativity and safety). A key challenge is to develop steerable language models that trade-off multiple (conflicting) objectives in a flexible and efficient manner. This paper presents Conditional Language Policy (CLP), a general framework for finetuning language models on multiple objectives. Building on techniques from multi-task training and parameter-efficient finetuning, CLP learn steerable models that effectively trade-off conflicting objectives at inference time. Notably, this does not require training or maintaining multiple models to achieve different trade-offs between the objectives. Through extensive experiments and ablations on two summarization datasets, we show that CLP learns steerable language models that outperform and Pareto-dominate the existing approaches for multi-objective finetuning.
Keywords: Reinforcement Learning, Multi-Objective Finetuning, Multi-task Learning, Parameter Efficient Training
มุมมอง: 40
วีดีโอ
Day of a Computer Science PhD at Cornell
มุมมอง 46K2 ปีที่แล้ว
A day in the life of a Computer Science PhD at Cornell University (Ithaca Campus). Music Adventures - A Himitsu Destination - MBB Sparks - Chaël Rainbow - JayJen
Construction of suffix arrays
มุมมอง 12K5 ปีที่แล้ว
A tutorial on the efficient construction of suffix arrays in O(n log(n))
Mazify! CMU 15-112 Term Project
มุมมอง 1.1K7 ปีที่แล้ว
Mazify! 15-112 Term Project for Fall 2016 at Carnegie Mellon University. Written in Python3, with Tkinter and Pillow, using techniques such as recursive backtracking, k-means segmentation and graph theory. Github: github.com/kaiwenw/Mazify-
trollolololololololololololol
Can a Cornell phd econometrics student take classes in ml, 3 to 4 courses in CS department?
Hello can you help to write thesis in machine language?
thank you for using upbeat music instead of wrist-slitting depressing lo fi !
unsupervised😂😂 ur a ML STUDENT BRO
how old are u please?
Just subscribed. Looking at Cornell’s program. Thanks for sharing
Great content and subscribed! Would love hear what you think of our grad school channel. Thank you.
Great explanation of prefix doubling. Thanks for this very helpful video 🙏
Fellow first-year Cornell PhD student here, not sure why i'm watching this bc my life is pretty much exactly the same as what's in this vlog hahaha. great vid tho!
Thank you for making this video! So cool!
Keep it up!
How old are you?
you need to tamp your espresso
Only south asian profs
Where did you get your undergraduate degree
first year cs student at the Hebrew university of jerusalem and i gotta say , doing a PHD in computer science sounds pretty hellish to me right now
CS is not easy but very in demand , rewarding and pays very well. Keep up the good work 👌 .
Lol he starts at 10h30 and goes to sleep before midnight, what’s hard about that
@@Alexandre-xz6jm Smh, let's see you go ahead and do a Ph.D. CS Program at an Ivy League university since it's not hard............ Assuming you could even be accepted in the first place.
@@Alexandre-xz6jm "Computer science isn't hard compared to majors like engineering" , uhh no it is actually. You should stop
Nice, I am doing software engineering although I kinda hate academia in general
Bro that book is extremely cringe. Kissinger is one of the most evil people in modern history
Gonna be starting a PhD in stats at Binghamton university next year. Cool to see what the lifestyle is like for other students
A great video! I'm also considering a PhD (not sure if at Cornell) after I finish my studies at Cornell!
I’m a phd student at Cornell!!! I also did undergrad there
Hi Kaiwen, Currently physics phd student at Brown looking at theoretical deep learning/stat mech. Would love to hear more about your story leaving facebook/employment/money to pursue a PhD!
Does Brown have good engineering programs? How would you compare it to a state school like Michigan, UNC or UF?
Same here. In a similar position doing ML at MANGA considering a PhD. Would love to hear more details on this!
@@GarrettLawrence Although I've only been here for grad school, from what I can see it's very undergraduate focused. Admittedly the resources at Brown's disposal in the Engineering department don't compare to Michigan, UNC, etc., but the education that you'll receive at either school should be fine. However, Brown is really really known for their CS department and a lot of students go into software engineering at FANG or go into finance (I've already been recruited for quant stuff). I assume this isn't the case at schools like Michigan, Georgia Tech, or similar institutes, but again I can't attest to their programs since I don't attend these schools. Word of advice: don't take out debt if you can, most of the time a rank 50 school on a full-ride will be more worth it than a rank 30 school you pay for, but again, it is much more nuanced than this.
Awesome. More video like this.
Cool
Great video! Had to sub! :)
Very inspiring vlog :)
Keep it up 👍
very cool!
loved it, keep going
Great VLOG! Please keep updating!
How to survive PhD class! hehehe good luck :)
Your vibe is so comforting I love watching and you should record the yoga too!
I really like it 👍
Hey! I loved your vlog on what it would be like to be researching in order to become a PhD at Cornell. I actually applied to Cornell so fingers crossed I get in. Currently I'm a senior in highschool and looking forward to doing Computer Science in college as my major; I have some experience with coding but would really like to become more advanced and eventually work with algorithms and datasets, etc. Do you have any suggestions for research that I should be doing, texts I should be reading, or ways that I can be getting ahead of the game and making myself the most advanced that I possibly can be at this stage? I really look forward to hearing your wisdom. And if it is possible I would love to maintain some sort of contact (even though I understand this is a bit odd of a request) so that I may ask anymore questions--if you're willing to answer them of course. Amazing vlog, thank you so much for sharing. I would love to see more!
best of luck!
kaggle
wow! awesome video!
Awesome vlog! Love this! :D
Amazingly clear and explanative video. This was exactly what I've been looking for. Thank you, it helped me a LOT!
Thanks dude.
Bro... Thank you 😭😭😍😍
Hey can u please explain it using strings instead of numbers and making tables for each state, i would be very grateful for that
Excellent video
is that the manber and myers's algorithm ?
thanks for making this video.
My pleasure!
@@kaiwenw can you create a video for suffix array as there is little information about them on youtube
anyone noticed his adam's apple lol, great video though
Could you please provide the code in c++?
I want to hit like multiple😍😍 times but youtube does not allow this. Thanks for such a great video. 😃😃
Thank you so much 😀
This was a great explanation of the "prefix doubling" approach of constructing suffix arrays! You really helped me get this, and (fingers crossed) getting one step closer to understanding the linear time suffix array construction algorithms.
Thanks for uploading this video. This is very clear and helpful. :D
Been looking all over the internet for an explanation like this. Thank you for a great and informative video. If you make more similar videos I'd definitely watch them.
Thanks!! What other content do you want to see?
More useful data structures would be nice, perhaps lcp array as a follow up to this video for example.