People and Parrots
Bottom line(s) on top
- The challenges in any organization are not just the direct and obvious problems that need to be solved, but also the organizational context they are being solved in.
- The hallmark of great thinkers and problem solvers is their ability to employ a combination of pattern matching (taking organizational context into account) and “first principles” thinking to attack a problem. In the absence of this, you get “When I worked at <name drop FAANG category companies>, we did X”.
- The most successful problem solvers have exposure to a wide range of patterns (“training data”), carefully consider organizational and other context when framing the problem (”the prompt”) - sometimes a large amount of it (”number of tokens”) and can fill in any gaps, sometimes through a first-principles approach (”intelligence” / “the magic”).
If you’re interested in the long version, read on.
The perfect hire
Why is work experience valued?
I suppose the reasoning is that if someone has worked in the same or adjacent area, with demonstrated success, then there’s a better likelihood that they will succeed in the new job they are interviewing for.
That’s the stuff recruiters and hiring managers usually look for when reviewing applications. The “relevant experience”.
Then the interview happens - and it often ends up being a conversation where experiences that are listed on paper are dug into - to assess if someone has truly demonstrated success. The more senior the role, the greater the scrutiny (one would expect, anyway). It goes something like this:
Interviewer: “Tell me about a time when…”
Interviewee: “<Situation> - <Task> - <Action> - <Result>” (Breaks it down perfectly, is very articulate and convincing)
Interviewer: “<Star Struck> You’re perfect! When can you start!?”
This is an obvious parody of the interview process, but also not too far from the truth.
(Kudos BTW to the interviewee who considered the question, and was able to pattern match a situation to produce a winning response. That’s a skill in itself.)
But wait… the interviewee took a course of action to overcome a particular challenge - why did they choose that particular course of action?
Patterns
As humans, we’re pretty good at pattern recognition. That’s one of the reasons we are (arguably) the apex predators on this planet. Human ingenuity has been about recognizing patterns and sub-patterns to think about and connect them in interesting new ways. Exposure to many different problems and patterns is therefore an advantage.
LLMs (you knew this was coming!) have famously been branded stochastic parrots, and to some, this has even become a pejorative term. If you think of their training data as exposing an LLM to (and / or helping an LLM build) a massive trove of patterns, you can see why there is excitement around this technology. A magic genie that has the answer to any question you ask it. Except here’s the rub - it requires you, the human, to craft a good “prompt” i.e. present the “situation” or problem to be solved in a way that the LLM recognizes, allowing it to “match a pattern” (very loosely speaking) and produce the right response.
That’s what humans do almost all the time when faced with a challenge - a challenge appears, an internal prompt is crafted, a response is generated, the challenge diminishes or morphs, another prompt, another response, and on it goes. All in a loop.
But this is where we start to differentiate between pattern parrots and good thinkers - the ideal “problem solver” that all organizations look to hire, to build and maintain the capacity to overcome challenges.
What happens when the challenge and the organizational context in which it occurs is novel?
Back to the Interview
Interviews tend to test your ability to recognize patterns. Question after question is presented with “prompts” or situations. These are situations that the organization or interviewee has deemed valuable for you to have succeeded at, for you to then be successful at the job you are targeting. Or in some cases (insidiously), generic tough situations - “If they have overcome that, they must be good”.
On receiving a response that outlines a certain approach, very few interviewees will press further and ask:
- Why did you choose that approach? / How did you know to choose that approach?
- What else did you consider?
- How often have you been in this situation?
What is the interviewee able to tell their interviewer about facing novel situations? And if they’ve never faced that situation, there’s a valuable conversation to be had about how they would go about solving it.
That’s where the real signal is - where you can see the ability of people to think through and navigate a pattern that’s not quite like anything they’ve seen before.
Bottom line(s)
- The challenges in any organization are not just the direct and obvious problems that need to be solved, but also the organizational context they are being solved in.
- The hallmark of great thinkers and problem solvers is their ability to employ a combination of pattern matching (taking organizational context into account) and “first principles” thinking to attack a problem. In the absence of this, you get “When I worked at <name drop FAANG category companies>, we did X”.
- The most successful problem solvers have exposure to a wide range of patterns (“training data”), carefully consider organizational and other context when framing the problem (”the prompt”) - sometimes a large amount of it (”number of tokens”) and can fill in any gaps, sometimes through a first-principles approach (”intelligence” / “the magic”).
I’m Siddhartha (Sid) Shankar, and I am currently a Senior Engineering Manager at GitHub serving the CodeQL dynamic languages team, supporting JavaScript/TypeScript, Python and Ruby. I have had the privilege of leading and managing engineering teams since 2015. I’m at my best when engaging in opportunities that require bringing people together - often across teams - to deliver value to customers in a sustainable and pragmatic way. More about the author →
This page is open source. Please help improve it.
Edit