I have been applying to a lot of jobs lately and I've done quite a few "prove you know what's up" tests and projects. I think it might be a good exercise to go over them in my head and talk about how everyone could win from these tests.
Note: this write up is meant for remote development work!
I think tests for entry level/junior positions are necessary to prove:
If the position you are trying to fill is for an expert or senior level position that applicant should already have ample code samples and open source contributions to go through. I don't think it's necessary to check for fundamentals in this situation. It'd be more important to me that the person is a good fit personality wise for the team.
For a senior position it might be more valuable to give them a problem and ask how they would organize a team to solve it? Not sure, I'm still a Junior!
A good employment test should give the interviewee an opportunity to showcase their skillset.
For example, if testing someone for a Django/Python environment:
In one test I made up a set of models and as a bonus the employers asked me to write up a strategy to test the models.
Not only does this give me an opportunity to strut-my-stuff, but it makes the problem just a little more interesting. I started to open up my brain and think of ways to test the code which made me think more clearly about the overall architecture—and it was fun!
If your interviewee is completely baffled by the test because it's so specific and they could solve the problem using their own tools: that is a bad experience for everyone. Valuable time is soaked up during the test Googling how to use library X and framework Y.
Instead, the applicant should be encouraged to use whatever strategy they see fit to solve the problem and then they should be tasked with explaining why they chose that route. Often times when you meet someonew new, they have a few tricks up their sleeve you may have never seen before!
The test was super vague, as if the person who wrote the test didn't care much about the end result. It bothered me that there was a one paragraph explanation and so much openness. No real time limit, no concrete goals. Also, when I asked for more details it took almost a week to get a response. The lack of response made me lose interest in that position.