Testing Software Engineers? Engineer these Tests

February 9, 2014 at 11:15 PM by Jordan Taylor

quiet_please_software_engineer_at_work_print-r92ab291d59f7420e81e66d7349ab816b_z0x_8byvr_512Being persistent with your exclusive sourcing strategy will attract promising software engineers to your company, but ascertaining if they’re a good hire will mean implementing a code test or offering them a short-term contract.

Tom Pinckney, manager of eBay’s NYC R&D Center, highlights the two-way value of using a code test as a recruiting tool:

“At SiteAdvisor, Hunch and now eBay we've long been fans of making practical onsite coding projects an integral part of the interview process…What I have slowly started to learn is that this is also about the candidate learning about us: what we value in a developer, what we think is important, what kind of problems we work on, and how we solve those problems.”

Questions to Ask Before the Code Test

Before the code test you can start with simple questions that increase in complexity if needed. Some interviewers prefer to see how candidates handle easier questions (moving toward topics like distributed systems, algorithms, and computational theory are usually reserved for more senior level programmers). Candidate language of choice differs, but they should be able to handle technical questions on at least one language (and, of course, you know what you’re hiring for). Here’s a very small sample of knowledge and experienced-based questions you might ask:



  • What are some differences between Python 2.x and 3.x?
  • Tell me why GIL is important?
  • How would you setup multiple projects where each one uses different versions of Python and third party libraries?


  • What are three ways to invoke a method in ruby?
  • Can you tell me the difference between classes and modules?
  • What is a Proc?


  • What are some things you don’t like about Python or Ruby?
  • Do you use tabs or spaces?
  • How would you build something like YouTube and mange scaling issues?

Coding Test

Every company is going to approach this differently: some ask candidates to write code during a phone screen; some still find value in Fizz Buzz; some use programming test web apps; some give 8-hour tests (Facebook); and others simply create a short-term contract (usually for one 8 hour day) so their team can work side-by-side with candidates.

Rather than diving into all the specifics, let’s highlight some great resources for several of these approaches.

Phone screen test: Coder Pad is “real-time, collaborative, and fully in-browser REPL” that helps you weed out candidates that mostly likely won’t pass an in-person interview.

coder pad

Programming test web app: TalentGuide is the perfect tool for saving time in your screening process.

talent guide


Classic “Fizz-Buzz test”: Fizz Buzz has been around for quite some time but it still has value—“an interview question designed to help filter out the 99.5% of programming job candidates who can't seem to program their way out of a wet paper bag.” Here’s the test:

Write a program that prints the numbers from 1 to 100. But for multiples of 3 print “Fizz” instead of the number and for the multiples of five print “Buzz”. For numbers which are multiple of both 3 and 5, print “FizzBuzz”.


Using a code tests as a recruiting/assessment tool is the best way to accurately, and objectively, measure a candidate’s skills and experience. At Entelo, we prefer using the short-term contract method, but it’s good to know there are other strategies out there. Whether you utilized the methods above, or give out a take-home project, remember to give candidates real problems to solve and save the theory for another day.

Also, here are some things you should reflect on after the testing process: how quick can they think on their feet; were their answers to the knowledge and experience questions coherent and correspondent; did I learn something new from them; what can we now say about the candidates learning and adaptability skills?

Tom Pickney sums it all up:

“I increasingly believe that an interesting, challenging, and educational code test helps recruit great programmers as much as it helps us identify them. Even in the cases where we don't end up hiring the candidate or they turn us down, they've many times told us that the coding projects were fun and taught them something. So whether a hire comes out of the process or not it was still a win-win for the candidate.”

Additional resources

dante9999 – Reddit: Python interview questions

Zack Bloom – HubSpot: What We’ve Learned About Interviewing Engineers

Jimmy Bogard – LosTechies: FizzBuzz is Dead! Long Live FizzBuzz!

Alex MacCaw – Sourcing.io: A Startup’s Guide to Hiring

Ryan Sobol – GitHub: 15 Questions to Ask During a Ruby Interview

Joel Spolsky – Joel on Software: The Joel Test: 12 Steps to Better Code

Rob Stevenson – Entelo: 3 Ways to Distinguish an Average Engineer from a Great One

How to Email an Engineer