How to Screw Up a Product Testing Cycle
Posted by rbpasker on January 2, 2008
Dear Product Manager,
Thanks for sending me your product! One of the most rewarding things I get to do in my professional life is get an early look at new products, and because I consider these early looks a privilege, I promise to reciprocate to you by providing quality feedback as quickly as possibly.
I know that your product may be raw, and might lack the stability, richness, or completeness that the ultimate product will have. I won’t ding you or your product for features or stability you didn’t promise me!
And thanks for setting up an email responder and forums and bugtrackers and mailing lists to collect feedback. And I’m glad your management identified you as the point man for this testing cycle. Both of these are very important.
Nevertheless, because our interaction during this testing cycle has been so poor, I have filed your product in the Bit Bucket, and I won’t be sending you any more feedback.
Here are some of the most common reasons:
You didn’t read my email / forum post / bug report. I told you that I read all the FAQs and docs, and I even gave you a list of things I tried. Why, when you responded, did you send me a link to another FAQ or doc page, or ask me if i tried this or that? Don’t send me a canned or ill-thought out response. I did my half of the job, now do yours: read my email.
You didn’t get me to the point where I could really try your product. When I sent you that note that said, “I’m still stuck on step 2 of the install,” I put your product aside, waiting for a response. Its took you 6 days to get back to me! Now I have to swap the whole mess back into my brain’s working set, and start over. You should have sent me the missing file, answered that one question, or fixed that little bug, so I could at least try your product out.
You didn’t test your product yourself before sending it to me. Yes, I know it works for you, but I have a “customer” environment. The URLs are hardcoded to point to a server behind your firewall. You’re referencing a file in your source code repository. You should have taken home a pristine laptop, and tried it yourself before sending it to me.
Your product has platform or version dependencies, but you didn’t try my configuration before you sent it to me, or you didn’t ask me if my configuration matches what your product requires.
You said you would send me a fix, but you didn’t. I’ve waited a week. I can see the bug in Bugtraq. How much longer do I need to wait?
You sent me a new release, but you didn’t mention the showstopper bug I reported. Is it fixed, but you didn’t add it to the release notes? Has it been forgotten? Is it in the queue?
Hopefully, you get the point by now: when you’re in the role of testing program manager, you have to keep us testers productive and informed! If you do, we’ll work hard for you, and help you make a better product.