About a month ago I decided to host Eclipse distribution downloads on the nWire web site. Alongside with the standard distributions, you will find distributions that already include nWire pre-installed. Packaging and repackaging was a very simple task and, on the face of it, everything was in order. Nevertheless, I decided to do the right thing and test that everything works.
The challenge is to test on as many platforms as possible: Windows XP, Vista, 7 and Linux, on 32-bit and 64-bit. It was narrowed down to 24 tests of unzipping and running Eclipse. Our resident QA engineer was preoccupied with other tasks and didn't have all the platforms installed, so I decided to outsource this task.
Choosing a Service and Signing-up
There are many sites that offer freelancers for hire. I heard of oDesk and Elance and they seem to be the biggest sites to date. Other sites worth mentioning are Guru.com and vWorker (aka rent-a-coder). Elance requires a $10 registration fee, so I decided to try oDesk. Plain and simple. On the other hand, unlike Elance oDesk does not work with PayPal. Since I use PayPal to sell my software, paying with PayPal is very cost effective to me: it saves the very expensive round trip of converting USDs to my local currency and vice versa. In the long run, I would prefer paying with PayPal.
The sign-up process is quick in oDesk, but, after you sign up, you need to verify your credit card. oDesk will charge you with a random fee (still hope to get it back as promised) and you need to enter this fee as a verification. You cannot hire before you verify, at least for payment up-front contracts. This turned out to be an annoyance, since I wanted to get this job out the door as quickly as possible. I had to wait for the charges to appear in my account and than phone the credit card company and ask for them. At hindsight, I'd rather pay $10 on PayPal...
oDesk has a desktop client. You don't need it if you're hiring. On the Mac, it looks really out of place, so I recommend not installing it if you don't have to. They also have an iPhone app which I haven't tried.
Posting the Job
Posting the job was easy. The interface is simple and intuitive, although somewhat outdated. I was very careful to specify the exact requirements and I think I did OK (you can read it yourself). I estimated how much time it will take, checked for common rates and decided to make it a fixed-price contract at $30. I like knowing in advance how much it's going to cost, especially when I don't have any experience (read: trust) with the person I'm working with.
I got about 20 job applications. The job was posted on July 13th. Here's the distribution of applicants per day: July 13 - 5, the next day 5 as well, then 3, 1, 3, 1 and on July 19 - 2. The last two were spam. Completely spam. They both offered to do it for about $250 which was absurd. Other than those two, all offers were bone-fide.
Sorting the applicants was frustrating. The list kept growing and it was hard to keep track of the conversation with each applicant. I was really missing a way to write notes for each applicant. The applicants list UI is poorly executed and very annoying, mostly because the sort order keeps changing and opening new windows is inconsistent.
Candidates came from India, Western-Europe and Russia, Philippines and South America. I received many applicants from India, over 50%. None of them read the requirements properly. It was required to have access to all the testing platforms. I had to ask each applicant individually whether they have it and all said no. It was really a waste of time.
Finally, I managed to find some applicants that had the platforms. I chose Max. He had relevant experience, but not on oDesk. He was new, like me, and didn't have any feedback. We scheduled to talk over Skype. He's from Ukraine, so we are at the same timezone. I sent him a full specification of the tests before the conversation. We talked, I was under the impression that he understands the job and we moved on.
His offer was $33. The majority of applicants added 10% to the price. Some did the opposite. You might want to subtract 10% from what you think is fair, knowing offers will add 10% anyway.
Since it was a fixed-budget job, I didn't care about getting a work log. I just wanted the job done as quickly as possible. Max was quick to execute and provided organized screenshots of all relevant scenarios as a proof that it works. We used Dropbox (here an affiliated link) to share the files.
One of the tests failed because the distribution was corrupted. Max also had some issues with the 64 bit platforms. I had to step in and help in solving it, but he was proficient: I just provided the links to the solutions and he did the rest. Overall, it took about 2 days because of the 64 bit issue. Once the tests were complete and the corrupted file replaced, I now can safely say that everything works.
Eventually, I was happy with the results. I added 10% to the agreed price, which totaled at $37. I think it was a fair deal.
Conclusion and Tips
I got what I came for, and I got it at a good price IMHO. I'm happy. I think the key was good definition of the task which led to choosing the right person.Choosing a person with no oDesk feedback was not a problem in this case.
I will use such a service again in the future. Will I use oDesk again? Maybe. Not having PayPal as an option could be a deal breaker when the stakes become higher. I didn't like the interface and it is infested by spam to some extent.
While looking into other sites, I found that some sites are better for specific needs. For example, some sites have more cheap QA people while other have more talented marketing people. If you have a specific need, I suggest running a search for your target employees on each site and comparing results.
If you are going to post a job, make sure to highlight the requirements. Prepare a canned list of questions verifying the applicant fits the requirements (even though you specified them in the description) and send them to each applicant without even reading the application or the applicant details. You see, many freelancers just send canned applications without even reading your requirements. When you send questions, they will need to answer them. If they don't answer, forget about them. If they answer that they don't fit the profile: thanks, but no thanks. Finally, you will get a short-list of people who actually read your questions and understand what you need.