For those of us working to influence Google Autocomplete, Mechanical Turk is a great tool to use. You can easily create an account, upload your .csv file and have random people from all over the United States (or the world) perform your Google searches for you.
If you’ve done any research on how to change Google Autocomplete, you know that diversity is key. The fastest path to success is to get your searches done from as many different computers as possible.
One would hope that Mechanical Turk would be a great help in this effort, but based on a couple of tests I’ve run recently, it’s not ideal. Let’s look at the data.
Test Batch #1
I uploaded a list of 18 unique keywords I wanted searched. Each unique keyword search is called a HIT and by default, MTurk only allows 1 person to work on any one HIT. So, it’s possible that 1 person could perform all 18 searches, since each one is a unique HIT.
In my test batch, I required that each keyword be searched 10 times. So, worst case, I’d get 10 different people performing all 18 searches. Best case, I’d get 180 different people searching for a keyword 1 time.
My result was that I got 21 different people to perform the searches. I was hoping for a little more diversity.
Test Batch #2
On my 2nd test batch, I uploaded the exact same .csv a day later – 18 unique keywords X 10 searches per keyword. Again, the worst case scenario would be I’d get 10 people performing the searches; best case would be 180.
In this batch, I only had 14 different people performing the searches. Not very good.
When I merged the two result files together, I counted 33 different people performed the searches. Thus, only two people worked on both batches.
Test Batch #3
On a 3rd batch, I uploaded 11 keywords as unique HITs and required 15 searches for each keyword. Best case, I’d get 165 unique searchers – worst case, I’d get 15.
I fared much better in this instance, getting 36 different people to perform the searches.
Additional Thoughts On Mechanical Turk & Google Autocomplete
There are some other limitations to Mechanical Turk that make it not an ideal solution for trying to change Google’s suggestions.
- Batches are completed all at once. I’m not a developer, so I upload batches under the standard interface. In this interface, I have not found a way to “drip” the HITs over time. So, it’s likely that all of your searches are going to occur within a day or two, as opposed to being spread out more naturally. Not ideal.
- Not location specific. More and more, Google is localizing autofill results based on where you search from. For instance, if you Google something in Portland, Oregon, you might see entirely different suggestions if you Google it from New York. Mechanical Turk only allows you to specify workers by country, not by city.
- Can’t collect geographic information. I track every single search our team performs, so I know when it was done, where it was done and what was searched. Unfortunately, Turk’s TOS do not allow you to collect ANY identifying information on workers. I tried adding a question, “What is your city and state?” and my HIT was rejected. Without city/state information, my reporting data is incomplete.
Conclusion
The upside to Mechanical Turk is that it works. Once you’ve designed your HIT template, it’s easy to upload a task. Of course, you’ll have to spend time doing setup for each client, in order to make sure the search is performed in a way that Google recognizes it.
The big downside is you can’t identify where your searches are coming from. This is a big problem, since I believe that Google is going to continue to integrate localized suggestions in the future, even more so than it already has.
We’ve seen how local search results have become much more local in the last 12 months, I suspect that highly location-specific autocomplete is not far behind.
We use Mechanical Turk on a limited basis to help get search volume and a little diversity, but to rely on it as the only method for changing Google’s autocomplete would be risky.
I’d be interested in hearing feedback from others who’ve tested it to change Google suggestions. What kind of diversity did you get? Can anyone say whether the developer API allows you to geo-target or drip searches over time?
If you have negative suggestions in autocomplete and need help, please call 503-890-6663 for a quote to get it fixed.