Skip to main content

Live Coding Exercises: How NOT to hire potentially Brilliant Engineers.

I've intimated this view before but, I abhor "live coding" exercises for engineering interviews and will never have them as part of any interview process I conduct. They are simply unrealistic to real world engineering in every possible way, they only test familiarity (or luck) with a tiny subset of solution methods to a specif subset of problems...that you either "nail" or get spectacularly wrong depending on who is observing you.

They are mostly entirely unfair to the candidate on top of the pressure of having a gun under them while coding, only in the most extreme cases is coding under the gun and that's just competitions where the code is far from real world engineering related...so why test for general coding ability with such tests?? Stupid.

I posit, it is significantly more effective to see examples of a candidates finished working code in the form of a project or projects they've created. How long it took some one to get some uber algorithm working only matters should their estimates for co
mpletion bump against the business time line for getting it done...those time lines in code are almost always on the order of days or weeks NOT minutes. Also, such exercises tell you *nothing* about their ability to work around distributed problems, design algorithms or solutions that scale efficiently say across computers and that is the true talent one needs to have as a versatile problem solving coder...the minutia of solution for atomic algorithms all can be found on stack overflow in a 30 second search...real engineers are going to find and use those rather than beating their head over the syntactic niggles of trying to live code a solution that's already been done *out there*.

Ultimately there is this wide view that engineering is not a creative act which is ridiculous. As some one who is as driven by design and art as creative expression as he is by writing software or building hardware the two pillars have many strong similarities. Creativity doesn't come on a clock and forcing an engineer to do so on a clock can put many brilliant engineers in a schism that masks their ability with anxiety. Some of the greatest works ever created came in moments of utter and serene *mental peace*. This environment is the one an interviewer should be making for their candidates...one that is shaped to *their* creative process.

I interview for the laziest possible programmer, where in programming terms the lazy guy is good at building things once that scale and are efficient consistently even if they are not built in 5 minutes because that programmer is thinking *like an engineer* while building the solution. Forest- Trees thinking I like to call it.  I'd rather a coder take a bit longer to get something rock solid *at scale* than that they build something that has a very limited efficiency scope and is not scalable and none of that talent is identifiable by silly live coding exercises.

Comments

Scott said…
This is a harder problem than you give it credit for.

>How long it took some one to get some uber algorithm working only matters should their estimates for completion bump against the business time line for getting it done

This is generally untrue. If you have two devs who do the same quality work and one is twice as fast as the other, the faster one is worth twice as much to your company. They can implement four features while the other dev implements two. Faster devs either change your business timeline or they affect what features you can have within that timeline.

Other than that, when someone brings in code to show you, you have to consider: did they write it at all? Did they write it themselves? Did they get help from a smarter friend -- maybe that's the person you should be hiring?
David Saintloth said…
Your mentioned issue has been thought of and the reality is that "fast" is not a consistent thing for Engineers, you should read my engineers versus programmers post or my rube goldberg rules software development post to understand why.

Engineering complex applications is not the same as writing algorithms. You can't have some one build an efficient scalable web application on an unknown platform as an interview exercise yet that is precisely what you'd be hiring them to be on a team charged with doing.

It makes no sense to ask for the bark view when what you want to see is how the trees are distributed across the landscape. "Bark view" is important but almost never on a temporally important scale (only when shit breaks in fact!) other than that...building so that shit doesn't break ensures robust , scalable systems.

As for the issue of having code brought to you, obviously you aren't going to just look at it you are going to ask them to explain how it works to suss out authorship this is trivial. Further you will be better off asking for *live* working code...say being executed from a web application url they provide to you...that is the BEST by far example of engineering demonstrated as you get proof a) the code works b) they know where and how it is integrated into their system proving it is their system and c) they can walk you through the code itself and show you how it does that. Check and Mate, time to hire that person!!
David Saintloth said…
Engineers versus Programmers:

http://sent2null.blogspot.com/2012/03/engineers-versus-programmers.html

Rube Goldberg Rules:

http://sent2null.blogspot.com/2013/01/rube-goldberg-rules-software-development.html

Popular posts from this blog

On the idea of "world wide mush" resulting from "open" development models

A recent article posted in the Wall Street Journal posits that the collectivization of various types of goods or services created by the internet is long term a damaging trend for human societies.

http://online.wsj.com/article/SB10001424052748703481004574646402192953052.html

I think that the author misses truths that have been in place that show that collectivization is not a process that started with the internet but has been with us since we started inventing things.

It seems that Mr. Lanier is not properly defining the contexts under which different problems can benefit or suffer from collectivization. He speaks in general terms of the loss of the potential for creators to extract profit from their work but misses that this is and was true of human civilization since we first picked up a rock to use as a crude hammer. New things make old things obsolete and people MUST adapt to what is displaced (be it a former human performance of that task or use of an older product) so as to main…

Engineers versus Programmers

I have found as more non formally trained people enter the coding space, the quality of code that results varies in an interesting way.

The formalities of learning to code in a structured course at University involve often strong focus on "correctness" and efficiency in the form of big O representations for the algorithms created.

Much less focus tends to be placed on what I'll call practical programming, which is the type of code that engineers (note I didn't use "programmers" on purpose) must learn to write.

Programmers are what Universities create, students that can take a defined development environment and within in write an algorithm for computing some sequence or traversing a tree or encoding and decoding a string. Efficiency and invariant rules are guiding development missions. Execution time for creating the solution is often a week or more depending on the professor and their style of teaching code and giving out problems. This type of coding is devo…