What gets discussed:
The dreaded take-home exercises! Almost every role in tech has a take-home assignment after the first interview. The prompt varies by company, and there is limited information on:
Amit has 15 years of product and project management experience. In this interview, Amit shares insights on:
(a) How take-home-exercises gets used in large-tech companies?
(b) What is typically the criteria for evaluating responses?
(c) What makes for great responses?
(d) What you should NOT do in take-home exercises
(e) Critical tips to improve your odds at success!
(1) Typically for product managers, at what stage of the process do you encounter a take-home exercise?
When I think of hiring product managers, I usually go through a three step process. The first is usually understanding the functional capabilities. Do they understand the skill set? Do they understand how to go through different stages of product development? So that's the first functional skill set
The second is usually a fit. And the fit question depends on which organization you work for. You know, most companies have a certain value set or a certain type that they look for in an individual. So that can vary based on the company that you interview with. So you look for fit and there are certain questions around that fit that you ask.
And then the third is usually where the take home assignment comes in. And that usually in my experience, has been in the form of case study. In general, you could have the first two rounds with about three to four people. And if you go through these two stages, and those three, four people say you are a good candidate, then you move on with a take-home assignment.
(2) How have you done take-home exercises before? What skills do you test in a take-home exercise?
The way I have done take-home assignments in the past is through a case study. Typically, the case study is really comprehensive, where you create a hypothetical scenario and say, this particular company has this type of problem, and then you go on to describe that problem in detail, provide some actual data around the problem.
Because it's not just a test of your business skills, It's also your analytical skills. So you're testing whether you can take a complex problem and decompose it into specific, you know, pieces that you can actually solve for. And in order to solve for those pieces you need, you know, your data, intuition and your business understanding.
(3) What does a case study look like? Can you share an example of questions a candidate can typically expect in such a case study?
So we'll describe the problem. We'll give you a few specific questions. You know, for example, let's just say, an ecommerce company is struggling with growth. A question you could say, you know, this particular company is has observed that the growth rate has slowed down, and then you go on to give additional information supporting that when the information could be around the product mix, their sales in the last three years, which you're expected to analyze and understand why the company's done, right.
So that analytical information helps you understand that, and then you go on to ask specific questions and say, we know which product category would you focus on to address the sales? What clearer action would you take? What are some of the potential strategies that you can evaluate? Once you evaluate those options? How will you prioritize what to do next?
So these are some questions type of questions that you add in general, you will have about one to five questions. So it's a very comprehensive case study. And there's a spectrum some people could just give you one or two questions. In some cases, we go deep and we want to test the person's ability to really think through different aspects of the problem. So you get a lot of questions, but I think no more than five. I think the sweet spot is about three questions.
So those specific questions will take you down a certain path and we want to understand how not only can you think wide but how can you think deep in a particular area of the business.
(4) What is the expectation from such a take-home exercise? What instructions are given to the candidate?
We typically expect a 10 page Keynote or PowerPoint slide deck, which articulates the problem statement and your solution to that problem statement, and present your homework or present your analysis of the work that you did. So if you've done a spreadsheet model that shows how you thought about improving the growth of the business, include that analysis in your 10 slide deck so that we see what kind of work you've done to arrive at your recommendation and very clearly articulate what the expectations are, identify the problem, give us your analysis and give you a recommendation at the end.
So those are the three components of the presentation that you need to create and that's the deliverable. Typically, you have a week to work on this at your own pace. You know, feel free to bring in external research that you can. Obviously we don't expect you ask your friend to do that for you. So making sure that you are keeping honest and have the right integrity of the process is important as well.
(5) How much time should someone actually spend time on such case-studies? What do you see on average from candidates?
The reason we give a week is because a lot of times people are working full time and doing this on the side. So we want to make sure they have a weekend in between to be able to spend some time. From an expectation perspective, I would say anywhere between six to 10 hours is a really good time. To pull together a very comprehensive study, comprehensive answer.
But in some cases, you'll see people who extremely detailed they'll have the 10 slides or they will have 20 slides as backup in the appendix. And you know, if you ask a question as a follow up, they say let me show you how to slide up. So I think there are some people who will go above and beyond and may spend 20 hours but that depends on an individual situation and that's not an expectation but the hours to build.
(6) How do you evaluate these responses? What is the rubric for evaluation? What does the evaluation process look like?
Did your understanding of the problem come through? Did you demonstrate enough analytical insight into that problem? Were you able to break it down into specific pieces and did you go deep enough? And did you explore all the possible avenues or not? These are the kinds of things that I looked for.
And usually I have a spreadsheet while I'm listening to the candidate, I'm actually assigning a score across those four or five aspects that I'm looking for the candidate to demonstrate their strengths and and then it's usually between zero and five. So zero means you have failed at this completely and five is that you have delivered an exceptional performance. Usually most people will have between three and four. In terms of their performance. If you've done a good job of selecting the right candidate, nobody should be going below three. And I think then you just basically assign weights.
So if for this role, analytical skills are the most important and communication is not as important than you would say analytical skills have to have a higher score than the communication. But if communication is equally important than analytics, equal weight, so I think that's where it's a it's a role dependent weightage. So yeah, you basically score and by the way, in these interviews, in the case study rounds, we have all the people who interviewed this individual, such that you're not just relying on one data point.
Now they have two data views and now they were the user in the case studies so that way they have a better understanding of the candidates ability and you'll be surprised. There are times that people who excel in these one on one interviews or fate or function interviews completely fail in the case study interview. That's where the case study interview, and the take home assignment is a very good test to separate the wheat from the chaff, if you will. So that's how we started.
(7) What separates good and great responses?
I think for me, the ability to tie intuition and data are like very important. You could say when you're solving a problem, you know, the data is telling me this, but I believe the data is only telling me half the story because I think this may be at play and here's why. So that's a much well rounded answer to say while I looked at the data and data told me x.
So I think I try and see or appreciate candidates who don't rely on one or the other input in its entirety, but are able to stitch a story together across different points of input, right? Whether data is an input or some research that they did is an input or their own experiences in input. So where they bring in and say, You know what? I think based on the facts you share, this looks like a good answer, but my experience tells me something different and therefore I believe we should take it the following way.
So people who are able to bring in this originality and combine it well with data and information they collected from the secondary research usually stand out as more effective than the ones who don't.
(8) How much does presentation matter in these deliverables?
I would be lying if I say if somebody puts in more effort and creates a more beautiful product doesn't have an age. I don't think that's true. Usually, when you see something that's in a more mature state, genuinely appeals to you more because one person putting the effort into it looks better.
But I would say it's not true in its entirety, right, or that's not the only variable at play. The other variables at play are how well did you think through the user experience, as you say, user experience or or product working? Well, it's not just how it looks. It's also how well it works. So if we think about that lens, then the user experience the person created, which is what's the interaction between one screen and the other, or one feature than the other? Or did they come up with the most efficient route to solve a user's problem? Worse is they came up with like 15 steps but just made them look more beautiful. I think all those factors are in place.
So not just the amount of effort that they put in and how beautiful they may look, but was it efficient and and the best way to solve the problem, I think both factors combined is what I generally look for when I'm trying to evaluate.
(9) What are ‘turn-offs’ for hiring managers? What is the #1 reason for failed submissions?
Poorly organized information. If you put too much information on one page, or it's not grouped in a logical manner. That's like the biggest turnoff right? And so if you don't have a hierarchy of your thought process, you will start from, you know, like, if you're telling a story, right, there is a beginning, middle and end and if you are following that storytelling, simple structure, then yes, that kind of turns off or is an easy disqualifier.
(10) What is your #1 recommendation for candidates to improve their odds at acing their take-home challenges?
My biggest recommendation is always spend a lot of time on understanding your audience. Know Who are you going to be interviewing with? What do they care about as much as you can? Right? Sometimes, you get the name of the person you're interviewing with ahead of time. Go look up their LinkedIn profile, go find them on medium, write some stuff on medium understand how they think about, like, what's the word you have product management or whatever that may be. It's, it's probably one of the oldest things like if, you know, you'll be able to tell your content accordingly. So it's it's very seldom that there is a right or wrong, right? There's usually a spectrum. So and I think that's why if you can tailor answers to your audience and understand what they're looking for. I think that would be really helpful.
And the second thing you can do in that same vein is understanding job description. Very good read, like every word of the description and see, what are they asking for? And when you speak to the recruiter, or when you have your earlier laughs before this case that go deep and ask what is what makes a successful candidate in this role right in your scene as a product manager, and what was special about it, because then you start to decode what makes a good product manager for that role for that company. Because generic answers don't work. A good product manager at Google may or may not be able to manage your Facebook because they operate differently. So I think that's where understanding your audience and understanding the role that you're applying for is the most important foundational step.
And then the third step is prepare well give it your best shot. Because if you can walk away from that interview thinking that there was nothing else I could do like I did my 110% I think that's whether you get that job or not secondary, but you should have the regret that I didn't give it another shot. So that's what I would say like, go deep in your own mind and find the best you know, out, you know, whatever you have excited then make sure that translates that into like case study or work assignment or whatever you do, because then you can be just happy that you give it a give it the best shot.