Meta Data Scientist Mock Interview - Call Suggestion on Portal | Product Metrics + AB Testing
ฝัง
- เผยแพร่เมื่อ 20 ก.ค. 2024
- 🚀Land your dream data job using datainterview.com/.
====== ✅ Details ======
🤔 Try these Meta's product-sense interview questions:
"Q1 - How would you measure engagement on Meta's Portal device?"
"Q2 - How would you prioritize the metrics you listed in Q1?"
"Q3 - How would you design an AB experiment to test a 'Call Suggestion' feature?"
Here's a mock interview based on Meta's technical phone screen with a product sense. The interviewer was a data scientist at Google and PayPal, and the candidate is preparing for product data science roles at FAANG companies.
Watch the mock interview to prepare for your own product data science interview. Make sure to check out datainterview.com/ for more content!
👍 Make sure to subscribe, like and share!
====== ⏱️ Timestamps ======
0:00 Intro
01:55 Measure engagement on Portal
10:55 Prioritize metrics
18:19 AB testing on the 'Call Suggestion' feature
29:40 Solution + Feedback
====== 📚 Other Useful Contents ======
1. Principles and Frameworks of Product Metrics | TH-cam Case Study
Link: / principles-and-framewo...
2. How to Crack the Data Scientist Case Interview
Link: / crack-the-data-scienti...
3. How to Crack the Amazon Data Scientist Interview
Link: / crack-the-amazon-data-...
====== Connect ======
📗 LinkedIn - / danleedata
📘 Medium - / datainterview - วิทยาศาสตร์และเทคโนโลยี
Great mock interview! In the future, I think it would be helpful to mention what level of experience the candidate has (ie: New grad, 3 years, etc) so that people watching with varying levels of experience can have a better sense of where they should be or what's expected.
Thanks for the suggestion! -- Dan
Hey Dan, this is a really great mock interview. Enjoyed so much watching it!
Little thought of mine, one additional thing to consider for the A/B testing framework would be the network effects. If the call feature is designed to increase the number of calls per user, then because you always need two people for a call to happen, you can have the call receiver being in the control group, leading to spill over effects between treatment and control.
Thanks for the input! -- Dan
You mentioned the randomization portion makes sense, but I don't think it does? This seems like a really good example of potential for network effects impacting the results - if 2 friends are binned in control and variant, then variant friend may call control friend more frequently leading to a rise in both variant and control metrics.
So either the metric would need to be adjusted to be "# of calls STARTED by user" or the randomization should be done using network information, randomizing sets of user networks.
On picking what feature is best. I think here one can do SVD or PCA and look at the weights of the largest eigenvector (in PCA and matrix V in SVD) on each initial feature. Here you should see the best combo and the largest contributor.
Tell this to Meta :)
Great point on the UI+Algorithm test. However I think the test should be a simple A/B/C test and not a paired T test which is typically performed on the same population at two different points of time.
Do u have any resources on how to learn which test to select?
This is goal! thank you!
Since it's a multivariate test why not use ANOVA and Tuckey HSD pairwise Comparisons?
How pair wise T-test is gonna Capture the interaction?
This is a good example to show why following the frameworks blindly is a terrible idea. Most of the metrics have nothing to do with engagement. It almost seemed that the interviewee had a checklist and just filling up the details. The interviewer did a poor in not emphasizing that the metrics are for measuring engagement.
Although I agree with your overall point, I think calling it a "terrible idea" is a bit harsh. The framework does help the interviewee to structure the order of what things to consider. he more so missed out on fleshing out the metrics in a way that connects them to the business case. this is particularly difficult to do when you are given a product that you are not familiar with. most people do not use Portal. If this were a case study on instagram, by contrast, it would probably be easier for an interviewee to think of more meaningful and useful engagement metrics. Just my two cents.
Very helpful mock!
step 6: pair wise t-test. How did you come to decide this any resoruces?
For the t-test, since this is a multivariate test, wouldn't we apply Bonferroni correction?
Hey Dan, I would like to subscribe your premium option. I was wondering how many courses fo AB testing? and Is there mock videos for AB testing too?
Hey Pitt, there are 12 lessons that cover core concepts in AB testing, and some of the mock interview videos in the premium subscription course contains AB testing questions as well. Please check out the course page for more info: datainterview.com/pricing/
"Q2 - How would you prioritize the metrics you listed in Q1?" Shouldn't it be the executive who picks out the top 3 metrics? I would present the result of Q1 and ask the executive to rank the priority... Also, can you elaborate on why choosing t-test instead of z-test?
Can you do Data Engineering mock interviews? Like Data Modeling and ETL design?
Yes!
@@DataInterview Sweet! I have a Meta Data Engineer interview next week!
I’m wondering if there is a way to sign up for this kind of mock interview?
Hi Ruby, please check out the coaching page to enroll for such a mock interview. Here's the link: datainterview.com/coaching/
So hard to understand Mark's accent
What is CLT? can anyone plz explain.
When Mark wrote it down he was talking about Customer Lifetime Value, probably just a typo.
i don't understand a word he's saying
You can either improve your listening skills or turn on the captions. The truth is that there will be many non-native speakers working in Meta. Will you quit your job just because of the ascent your colleagues have?