Spock Loves Amazon Web Services

Though we don’t usually blog about technical details here, we’re so crushingly enamored with Amazon S3 and EC2 that we felt compelled to write a short post about it.

Since the first months of Spock when we were busy prototyping early versions of the site, we’ve made heavy use of Amazon Simple Storage Service (S3). When you’ve got millions of profile pictures to display and each photo can have many thumbnail sizes, that’s a lot of data to store, manage, and serve. We started by serving the photos off our web servers, but that quickly got to be a management nightmare. We did some quick math and realized we’ve save quite a bit of money using S3 to serve our photos instead. Though there was some work in the initial integration, after we finished, we’ve probably spent less than 15 minutes/month thinking about photo storage. Since then we’ve used S3 for various other tasks like serving our Spock Challenge data set, which is gigabytes big and was heavily downloaded.

Aside from S3, we’ve also made good use of EC2. Amazon’s Elastic Compute Cloud (EC2) is a great way of getting computing power when you need it, and more when you need more. And really, when you’re crawling, classifying, indexing, and extracting people info from the entire World Wide Web, you definitely need more. EC2 is great because you can create one virtual machine with all the software you want, and then instantly clone it across hundreds of virtual boxes. The more we use it, the more we begin to wonder why we bother to operate any physical machines ourselves.

So yeah, we’re pretty psyched with Amazon Web Services, and we’re looking forward to leveraging other neat services they roll out in the future.

InformationWeek: Spock’s Social Search Engine

Social computing and search have been destined to merge at least since Google’s PageRank algorithm started counting Web links as personal endorsements of relevance.

And really, search has always been personal: We search for our own names, for the names of friends, enemies, and everyone in between. Lately, search companies have responded, adding personalization options to make our searches return results based on our own sense of relevance.

The rapid rise of social networking has forced the issue: It’s now clear that social networks can help organize the world’s information and make it personally accessible and useful in a way that computer algorithms haven’t been able to match.

Spock.com, scheduled to open to the public next month, is the latest child of the union of social computing and search. It is a search engine for people, like Wink.com and, to a lesser extent, ZoomInfo.com. It qualifies as a social technology because unlike people-oriented, privacy-challenged search engines like Zabasearch.com, Spock invites the people in its index to participate in how they get listed.

In the absence of privacy, control is the next best thing and Spock stands out for giving its users a least a little say over how they and others get represented online.

Spock, its creators insist, is thus named because it’s a memorable consumer name and because it stands for Single Point of Contact and Knowledge. Perhaps the domain name was just available at the right price.

Search for a name on Spock, say “Steve Jobs,” and you’ll see a familiar search results page, with a picture of Apple Computer CEO and co-founder Steven Paul Jobs alongside related links, tags, and text as the top result.

Click on the link that is his name and you’ll see a profile page that presents biographical information from Wikipedia, a list of related people (Steve Wozniak, John Lasseter, Larry Ellison, to name a few), links to other Web sites with relevant information, and a selection of tags.

Tags are hyperlinked words that users or the Spock Web crawler have associated with this person. Steve Jobs is tagged with “Apple,” “CEO,” “iPod,” and “adopted,” to name a few. Clicking on these tags executes a Spock search using the term in question. “CEO,” for example, returns Jobs first, followed by Microsoft CEO Steve Ballmer, Google CEO Eric Schmidt, Microsoft Chairman Bill Gates, and Oracle CEO Larry Ellison. Jobs’s position at the top of that list is a reflection of how Spock and its users rate him as a CEO.

Spock users can vote publicly on whether or not a tag is accurate. The Spock entry for George Bush, for example, lists the tag “miserable failure,” in keeping with the efforts to have Google associate Bush with that term. The Spock vote for the validity of “miserable failure” is currently 42 for and 18 against.

Spock users can create private annotations on other people’s contact pages. Only the creator of the notes can see them, not the person profiled or anyone else. It also offers users the ability to make contact information public, private, or viewable only to select “favorites.”

Spock allows users to claim their own search profiles. Really, Spock almost demands it — few who care about how they’re represented online will turn down the opportunity to have some say in their portrayal. Users are invited to import their address books as friend lists, to add links to their Web sites, to upload pictures of themselves, and to enter additional information to flesh out their profiles. And if someone tags you with a tag you believe mischaracterized you, you get a vote, though only one, against it. That’s where it pays to have friends who’ll vote with you.

Spock aims to deter abuses of its system by making people accountable. Only registered users can vote, so proposing an offensive tag for someone is likely to have consequences if that person is also a Spock user. Spock’s success may hinge on how well it manages to keep its community cordial. After all, not many people will want to use a search engine that finds them lacking.

Click Here to read more

Fox News: Spock Search Engine Focuses on People

SAN FRANCISCO — A search engine startup promises to deliver more targeted results on queries about people, whether it’s your ex-girlfriend, the guy from the bar last night, or Paris Hilton.

The idea is to help you avoid sorting through the thousands of results — the vast majority likely to be irrelevant Web pages — delivered by the major Internet search companies.

Click Here to read more

Jay Bhatti in BusinessWeek Magazine

Business_week Searching For John Q. Public
by Catherine Holahan | July 30, 2007 | PDF of Article

Jay Bhatti was in Businessweek magazine.  They covered his work at Spock and how search engines like Spock could take a leadership position in search.

Searching For John Q. Public
New people-search engines hope to do better than Google in finding the less-than-famous

Jay Bhatti thinks he has spotted Google’s (GOOG ) weakness. The search Goliath isn’t good with people. A Google search for an individual may return tens of thousands of links in milliseconds, but it’s hard to tell unless you click on one if it’s the person you’re looking for. The results list won’t distinguish between, say, James Brown the soul singer and James Brown the sportscaster.

That’s where Bhatti’s company, Spock, comes in. His people-specialized search engine, scheduled for public launch the first week of August, scans social networks such as LinkedIn, MySpace, Facebook, and other sites where people regularly post information about themselves and others. It then pulls that information into a concise summary about a person, such as his occupation, interests, age, marital status, and hometown. A click on the summary reveals related Web sites and known associates.

Spock is one of dozens of niche search companies trying to capture some of the more than $60 billion that is projected to be spent on search marketing over the next four years. Bhatti wants Spock to become the site you visit to find details on a particular individual. “Searching for people in a general search engine is like trying to look for a needle in a haystack,” says Bhatti, Spock’s co-founder.

Specialization may be the only viable strategy for search startups. Google’s lead in general search seems insurmountable. Nearly 50% of all searches are done on Google, according to June figures from market researcher comScore (SCOR ). Its share has remained high despite competition from Yahoo! (YHOO ) (25.1% of searches) and Microsoft (MSFT ) (13.2%).

In some niches, the search game is still a wide-open field. The key is identifying the right niche. Travel, health, and finance are already crowded with competitors. In personal search, however, there is no clear victor. Startups include Wink, which is similar to Spock, and ZoomInfo, a search engine specializing in executives. They’re gunning for the roughly 30% of the 7 billion-plus Web searches performed in the U.S. each month that relate to individuals. About half of those queries concern celebrities. The other half target names that don’t have a million Web mentions: business contacts, former friends, ex-lovers, and the like.

These people-search engines sound like a stalker’s dream come true. But Spock and other such sites contend they return information a Web surfer could eventually find anyway; they just highlight it more effectively. And they don’t list phone numbers and addresses. But privacy advocates say that many people who fill social network pages with personal information falsely believe that only their “friends” will see it–or at least only users of that social network. Typically, social networks require people to have an account to read posts and visit users’ profile pages. However, unless a user specifies that his or her profile is private, it can be indexed by search engines. Even private profiles have some publicly searchable information, such as the user’s name, photo, gender, age, and hometown. “What people haven’t understood…is that information they thought was being limited to the people in their networks is accessible to search engines that can crawl through these sites,” says Marc Rotenberg, executive director of the Electronic Privacy Information Center in Washington.

The big search engines are ill-designed to highlight information on little-known people. Google, for example, returns results based on the prominence of search terms in a given Web page and the number of sites linking to that page, among other things. It’s a great model for finding authoritative sources about oft-discussed people such as Michael Jordan the basketball player. It’s not so great when you’re searching for information about a guy by the same name who’s toiling away in an office building in Utah.

Of course, the guy in Utah may not want to be found. So Spock and Wink say they give users better control of their online identities by showing them personal info online they may not know about. Whether that’s a sufficient reason for people to switch from Googling themselves and start ego-surfing via Spock or Wink remains to be seen.

JacobGrier.com: The Search by Spock

This week I was invited to join a cool new people search engine called Spock. Tim O’Reilly sums it up better than I can:

You can search for a specific person — but you can do that on Google. More importantly, you can search for a class of person, say politicians, or people associated with a topic — say Ruby on Rails. The spock robot automatically creates tags for any person it finds (and it gathers information on people from Wikipedia, social networking sites like LinkedIn…) but it also lets users add tags of their own, and vote existing tags up or down to strengthen the associations between people and topics.

Click Here to read more

PodTech Interview: Using Spock to find information about people on the Web

14:28 | Robert Scoble | Jul 10th, 2007 3:46pm

Here, co-founder of Spock, Jay Bhatti, gives us a demo of how Spock makes searching for people more powerful, easier, and more fun

http://www.podtech.net/home/3552/using-spock-to-find-information-about-people-on-the-web

Spock Testimonials

Spock users have been writing in about how much they like Spock. Here are some recent quotes:

I gotta say someone invited me to spock from twitter… I didn’t expect much, but it’s funny I have been looking for a couple of people for years, and never turned up much on my search. But today I put both those names in and your site responded with what looks like real leads. This is powerful stuff I just wish you were in atlanta, so I could work for you guys/gals… This is awsome.
– Jayson

These are the July 4th holidays and you are working so diligently that everyone should be amazed. You have to be incredibly committed to the concept to be so very giving and thorough. Thank you for being so conscientious; if everyone in your organization is similarly committed to this concept, you will truly have an outstanding website.
– Choel

Thank you.
Your support is amazingly fast and I’m growing onto Spock 🙂
Cheers,
Rick

Thank you for the this great people search engine and invite. Love it!
Regards,
Jurjen de Vries

Thank you very much!! I can appreciate the amount of work that goes into a new web product and I am excited about yours.
– Ben

So far I’m really digging Spock – you guys have done a fantastic job and should be very proud of where the product is at.
Cheers,
Chad

Spock rocks. clean interface. no ads. useful info in results. prototypical usage of web2.0 features. proved entirely useful in the first five searches! Can we get zaba data in the results? Thanks for a great beta!
Jason Robinson

Do you have something to say about Spock? Leave us some feedback, or write a comment below!