Jay Bhatti: Search, Consumer Web, and Marketing » Spock Research

5 Things Microsoft Should Do To Improve Search

Jay Bhatti — Tue, 15 Jun 2010 15:21:18 +0000

I really want Microsoft to do well in search, because the overall search experience for consumers would be improved if Google had a viable competitor.

(Also, I have friends there, and I want to see them do well.)

Having built Spock.com, a vertical search engine company, and having worked with both Google and Microsoft on various partnerships during that time, I think I can share a few simple things that Microsoft can do to improve search.

None of this is rocket science. It’s computer science. Which is why it’s so surprising to see Bing not pushing harder on these things that would decrease the gap between them and Google.

Improve Bing Crawling for the Long Tail

Image: Wikimedia

Google’s crawlers are impressive to say the least. If you launch a site tomorrow that has over 1,000 pages, within 2 days, Google will discover your site and crawl all those pages. Now, it might not put them in the index if the pages do not meet Google’s quality standards, but at least Google found and crawled your site.

With Bing, I don’t see that same level of execution. Heck, some popular sites that have thousands of pages indexed on Google are not to even be found on Bing (except for the home page). Bing likes to say that they are on par with Google when it comes to search experience. That may be true for the head terms like “credit cards”. But, when you type in long-tail terms like “freehold township credit union” the results from Google and Bing are drastically different. With Google’s being more relevant to the user.

If Microsoft really wants to compete with Google, then it needs to improve long-tail search. The only way to do that is to crawl faster, deeper, and with a lot more scale. Google has over 200,000 servers. I am sure Microsoft can match Google dollar to dollar on spending for servers. But if Steve Ballmer wants to really win, then get the right engineers in the crawling group working on the right set of challenges, like building crawlers that can crawl at least 50% of the web. That would be a good start.

Get AdCenter on Par With Adwords

This one is not hard. Microsoft should just copy what AdWords does and stop confusing advertisers.

Several months ago, I was invited to give the AdCenter team advice on how to improve its experience for advertisers. While I was there, a group of researchers walked us through some advanced thinking on AdCenter. It made no sense to any of the advertisers in the room.

We told Microsoft that they should just work to get AdCenter on par with Adwords from a user experience point of view. We showed them the features that Adwords had for years that AdCenter still lacked. Some of us were surprised when it seemed that we knew more about the feature to feature comparison of AdCenter to Adwords than the AdCenter team.

Stop using AdWords!

The only team that should be using Adwords at Microsoft is the AdCenter team, and that only to learn from it. Why give free revenue to Google?

Google won’t let employees use Windows based computers in the future, which means 20,000 less Windows license fees for Microsoft. If Google wants to do that, maybe Microsoft should stop giving Google so much revenue with all the AdWords accounts that are being used across Microsoft.

Yes, Microsoft marketing teams across the company use AdWords like crazy to market their products. Several people at Microsoft have told me that there are over 50 Adwords accounts at Microsoft. Let’s make that zero.

No more gimmicks

Gimmicks like slideshows and xrank need to go. They are artificial and do not result in real growth, When Google came out in 1999, they focused on core search experience, while other search engines offered prizes to people for using their engines. Guess who won! Microsoft needs to end all gimmicks and only focus on core search and nothing else.

No more MSN.

Microsoft should focus on building utility web products like Bing search, Office Online, and SkyDrive. See what happened to Yahoo when it tried to be both a search company and a content company?

Microsoft should learn from them and cut all content related products. It’ll be a tough pill to swallow. Especially given all the people who work in MSN. But if Microsoft wants to win in search, online documents, and the cloud, then it cannot be spending all those resources on content related activities. Focus on utility products. That is where the money and future reside.

Does the iPad deliver?

patrick — Wed, 10 Feb 2010 19:51:50 +0000

As one of the more anticipated events in the tech community, Apple’s latest invention did not disappoint. In officially unveiling the new iPad, Steve Jobs took on the Amazon Kindle and once again raised the bar for its competitors. While pundits have criticized the iPad for lacking a true identity (and they are certainly right to an extent), similar to the iPhone, the iPad should encourage further innovation and will impact multiple industries. Taking a closer look at the iPad strictly as an e-reader, it is an interesting product that has advantages and disadvantages over the Kindle.

iPad Advantages:

Is a multi media device that can play movies, tv, and games, and allows for color screen web browsing
9.7” touch screen is larger than the Kindle’s 7.5”
Standard 16GB flash storage is larger than the Kindle
iPad is compatible with all iPhone apps
iBooks store and deals with top publishers will ensure content will be available
Excellent for newspapers and text books, the iPad can show high resolution images, video and take notes

Kindle Advantages:

At $259 the Kindle is cheaper than the $499 iPad – which does not include a wireless plan
Free Wifi on the Kindle ensures that you can purchase and download books anywhere
eLink screen is better for reading
Books on the Kindle cost under $10, whereas the iBooks store will have variable pricing
iPad will be using ATT for its wireless service which is notoriously poor in cities such as New York and San Francisco
The Kindle has a significantly better battery life of 1-3 weeks, while the iPad is around 10 hours

For those looking strictly for an e-reader, the Kindle is probably the better choice. With a cheaper price, longer battery life, eLink screen, free wifi, and less expensive books, the Kindle is by far the better option for an avid reader. However for those (and I can imagine this represents the majority) looking for something that services newspapers, magazines, and text books, the iPad represents a significant upgrade. Not only is the color touch screen a substantial improvement, but the option to take notes, and store data, along with the multimedia capability, make the Kindle appear wildly outdated.

While my gut tells me that similar to the iPod, the iPad will win out; its success will be based on the outcome of three different things:

Will the iPad distinguish itself enough from the iPhone/iPod touch – As many critics have pointed out, the iPad is too big to replace your iPhone and not powerful enough to replace your laptop or even Mac Mini. Should an improved version come out with the ability to take pictures, run excel and word, and support Adobe Flash, it could start to take off very quickly. The lack of a USB Port is likely to drive consumers’ nuts as well.
What the Kindle does with its pricing model– Should the Kindle lower its current price to the $100-$200 range, consumers will be more inclined to look past the Kindle’s shortcomings. While the iPad is likely to come down in price as well, the need to purchase a data package and the prevalence of smart phones will drive customers away.
Can the iPad replace text books – While the iPad’s appeal is based on its ability to be more than just an e-reader, if it does a good job at marketing itself as an alternative to text books or work documents, Apple will have created something special. Given how iTunes has put a significant dent in illegally downloaded music, the iBook store could be something publishers aggressively embrace which would in turn help make the iPad a more common alternative.

While Apple has missed on products in the past, the public’s love affair with its products will likely ensure the iPad’s success. Even if pundits who criticized the iPad are correct, at the very least it should encourage the Kindle and the highly anticipated HP Slate to step up their game.

How Bad is Microsoft About Product Quality

Jay Bhatti — Sun, 13 Dec 2009 06:49:41 +0000

Microsoft recently added Bing Local. A direct competitor to Google Local Business Listings. However, check out the business claiming page on Bing today:

https://ssl.bing.com/listings/ListingCenter.aspx

As a former Microsoft Product Manager, it pains me to no end to see how poor Microsoft performs on delivering quality products.

Spreadsheets won’t change the world but leaders will

Jay Bhatti — Tue, 29 Sep 2009 03:26:23 +0000

Issue date: 12/3/01 – PDF | Link

This article was written by Jay Bhatti on 12/03/2001 for the Wharton School

It was still a few years before World War II. General Douglas MacArthur was a pretty popular man. Given his status, the executives at JP Morgan offered him a very high level position at the bank. One that would have made him very rich and amongst the most high-class people in New York City. His wife at the time (he later divorced her!) was constantly egging him on to take the job. She was, after all, very keen on being associated with the elite and having the best that life had to offer – in short, she was tired of living on a military salary. Yet, in a stunning move, General MacArthur turned down this golden position. When asked why, he simply said, “Bankers don’t make history.” So what happened to General MacArthur after that bad career move? He went on to lead the Pacific Fleet to victory in WWII, and, in short, made history. In October 1944 the world watched as he dramatically liberated the Philippines from Japanese control. On September 2, 1945, he presided over the Japanese surrender on board the U.S.S. Missouri, bringing an end to World War II. In the next five and a half years, as Supreme Commander of the Allied Powers in Japan, MacArthur and his staff helped a devastated Japan rebuild itself, institute a democratic government, and chart a course that has made it one of the world’s leading industrial powers. While his decision to ding JPMorgan may have been surprising at the time, one thing is clear: Douglas MacArthur fulfilled his self-imposed destiny of becoming one of history’s greatest men.

Now the point of the story is not to encourage Wharton MBAs to join the Navy, but rather to debate about what our underlying mission is as an institution. Is it to create spreadsheet monkeys and consulting goons? Or is it to help develop the future leaders of society?

I cannot count how many times I have heard the following quote “I don’t want to major in Finance, but I am at Wharton, and it would look bad if I did not have some solid NPV skills!”

If you look at the past 100 years of human history, what have been the attributes that have had the greatest impact on society? I would say that leadership, entrepreneurship, and innovation have by far been the factors that most shaped the world we live in today. Should it not make sense that we as a leading institution of higher education focus our energies on these very things? I am not suggesting that we eliminate all the finance and quant courses at Wharton. What I am suggesting is that we place a greater emphasis on those classes and programs that would best equip students to lead, innovate, and create.

For example, instead of having math camp for two weeks at pre-term, would it not be more valuable to have a week long class that discusses what is meant by Wharton leadership? Maybe we can have discussions/sessions with some of the remarkable leaders that have come from Wharton and how they impacted society?

Hey, I know this sounds fluffy, but if students can more closely align their passions with what they do with most of their time, don’t you think that they will be better leaders? I mean, if you are really passionate about public service, and instead you go and work for an I-bank, would you wake up in the morning excited about going to work? From day one at this school, the focus of the school should be to find out what each student is most passionate about and then work with the student to prepare him/her to become the best they can be in their respective field! Place a stronger emphasis on classes that discuss leadership (with the right professors!), put into place stronger programs that would make more students consider entrepreneurship, and foster a spirit of innovation and risk taking in every class. Have more joint-programs/classes with the engineering school to allow MBA students to find out what is really happening in the world of technology and science. For example, the first computer in the world was made here, but did any Wharton MBA take advantage of this? FYI, the engineering school is doing some really cool research on nanotechnology. Maybe you should pay a visit.

In my “Seminar on Leadership” classes this semester, we had the opportunity to have an hour-long chat session with Dean Harker. In this session, Dean Harker stressed the point of Wharton moving to the next level. He mentioned that since the early 1980’s, Wharton’s mission was to be #1. Having accomplished that, he said the next challenge was to change the culture of the school: to move away from the rankings, to more closely focus on becoming the best learning institution in the world, to make the alumni and students more connected with each other, and to develop leaders who represent the Wharton style of leadership. It is time for the school to go back to what it was created to do: “To help develop leaders in professional, community, and personal character.” Part of this effort requires us to place less of an emphaisis on the calculator and more on the human element of the equation. After all, you manage a calculator, but you lead people.

I am willing to bet my bottom dollar that the people who will have the most impact on this world in the next 100 years will be those that can lead with passion, take huge risks, create new enterprises, and understand that innovation is the engine that moves society. These are the attributes that the school needs to develop in its students. Let’s focus our energies on this and not as much on the quant classes (save them for the undergrads). In short, Wharton should work harder to develop 800-pound gorillas and not spreadsheet monkeys.

Age Distribution of People on the Web

Jay Bhatti — Mon, 21 Sep 2009 20:07:27 +0000

Despite Privacy Concerns, 74% of people openly show their age on the web!

As a people search engine, Spock crawls and indexes millions of web documents and social network profiles everyday.

As a result, we end up gathering interesting demographic data about people. For example, a vast majority of people who have a social networking profile or web document about themselves on the web are 25 or younger. In addition, 74% openly show their age on social networks, blogs, and other social mediums. Even with added privacy controls, a vast majority of Internet users openly show their age.

Age Breakdown of People Who Have a Public Identity on the Web

25 or Younger: 37%

26 to 45: 23%

46 to 65: 8%

66 or older: 6%

No Age Listed: 26%

The above data is based on Spock crawling and analyzing over 600 million social networking profiles and 2 billion web documents that reference people (wikipedia, IMBD, corporate bio pages, etc).

Why are there so many documents and profiles on the web about people under the age of 25? We call this the social network effect. Social networks have a combined 600 million plus profiles, many of which are owned by people in college or high-school.

A common question asked is what happens to the age breakdown if you exclude the impact of social networks. We compiled the table below, which breaks people out by segment. Social Networking profiles are compiled in the “normal people” segment.

Age Breakdown (by Segment) of People who have a public identity or document about them on the web.

Age	Normal People	Famous People	Semi-Famous People	Overall
25 or Younger	38%	3%	1%	37%
26 to 45	23%	16%	2%	23%
46 to 65	7%	13%	3%	8%
66 or Older	5%	38%	6%	6%
No Age Listed	27%	30%	88%	26%

Examples of each segment: Normal People, Famous People, Semi-Famous People.

The data shows that people on social networks are more likely to have an age associated with their webpage then web documents about famous or semi-famous people. In conclusion, it appears that when given the option, people are very likely to display their age on the web.

Men three times more likely to brag about professional accomplishments then women

Jay Bhatti — Mon, 21 Sep 2009 11:08:03 +0000

In a Spock Research study of 3 million corporate bio pages on the internet, Spock discovered that men were three times more likely to overly boast about their professional accomplishments then their female counterparts.

For example, men were 3.15 times more likely to have the words “accomplished”, “responsible for”, “served as” and “led” in their corporate bio page then women. In our analysis of corporate bio pages of people from similar industries and job titles, it did not appear that women were any less accomplished, just that they were less willing to place subjective terms like “led” or “responsible for” in their bio page then men.

However, when we looked at bio pages for educational information, women did equally if not better then men for terms like “graduated with honors”, “doctorate”, “certified”, or “graduated magna cum laude”.

What does this tell us? One hypothesis is that men seem to be more aggressive in marketing and promoting themselves online as opposed to women.

Spock mentioned along with Facebook and Google in ZDNet

Jay Bhatti — Mon, 21 Sep 2009 11:03:40 +0000

Spock was mentioned as one of the leading compaines alongside Facebook and Google in ZDNet. Click here to see the article.

PDF of ZDNET Article

Spock mentioned in redOrbit Magazine

Jay Bhatti — Tue, 21 Jul 2009 11:02:46 +0000

Click here to see that article about Spock and how it is changing the web.

The Changing Rules of Google SEO

Jay Bhatti — Thu, 09 Jul 2009 00:19:04 +0000

Penalty Type	When	Detail	Actions You Can Take
Google Vince Update	March 09	A Googler named Vince created this change and hence the name. This is not a penalty, rather an update in Google’s algorithm. Vince update seems to favor bigger brands and has pushed some of these big name sites further up the rankings. Google’s explanation is that, It is more about factoring trust more into the algorithm for more generic queries. From what Matt has said this update is probably looking at the overall weight and trust of a site (and the big brands have spent enough marketing pounds to win here) and the theme of the site.	Do site awareness, and brand promotion, in addition to traditional SEO work.
Google -6 Penalty	Late 2006	Google didn’t admit doing this to sites, per Matt Cutt. One possible trigger is that many of these sites have highly optimized pages tightly focused around a single core phrase or keyword. Google now argues that the effect was caused by a glitch in the system and that an attempt to filter out bad sites had caught good sites in the process. Most sites should get their original rankings back soon.	Do not overly stress on a single keyword.
Google -30 Penalty	Introduced in late 2006, but Google starts to aggressively enforce it in mid 2009.	A penalty widely-speculated given to thin affiliate, refer or doorway sites which do not add much value for the site visitors. However, many non-affiliate sites also have reported this penalty. Sites with excessive low quality inbound or outbound links and lots of non-unique content may have a minus 30 ranking penalty applied. Syndrome: your well-ranked keywords (1st page) suddenly drop 30 positions. Some of the practice below may help trigger -30 filter: Guestbook spamming: If you try to get inbound links by spamming guest books and blogs then Google might apply the filter to your web site. JavaScript redirects: JavaScript redirections might be misinterpreted as a spamming attempt. Better use 301 htaccess redirect if you must redirect URLs on your pages. Doorway pages: Google doesn’t like doorway pages. If you must use special landing pages for PPC ads and other ads, make sure that these pages cannot be spidered by Google and other search engines. You can use robots.txt to do that	The only solution to avoid this penalty is to have unique content on your site, get links from well trusted sites and link to high quality sites. For detailed information, please refer to Google’s Webmaster Guidelines. Clean up the site first and submit a reconsideration request to Google.
Google -50 Penalty	Sept 2009	Over-optimized key anchor text on link building. This is the most recent one that generates some discussions amongst webmaster and SEO sites. Syndrome: your well-ranked keywords (1st page) suddenly drop 50+ positions. Good article on this topic: http://www.cemper.com/seo-knowhow/google-minus-50-penalty-cure	De-optimize anchor text of back linking. Use natural languages, not keywords screaming SEO. If you’re link building, make sure your anchor text varies on each site that links to you. You do not need to have keywords stuffed on every single link. Use “nofollow” at times. Clean up the bad practice first and submit a reconsideration request to Google.
Google -60 Penalty	Mid 2008	Bad back linking practice: spam back linking or potential link farming. It looks that Google applies this penalty to websites that buy links. Many of the websites that seem to have been penalized had many inbound links from websites that linked to them from every single page of their website (so-called site-wide links). Site-wide links are an indicator of paid links, which Google sees as an unwanted way to artificially inflate search engine rankings. The head of Google’s anti-spam team Matt Cutts has often said that websites that buy paid links will be penalized and it looks as if Google tries to do the job properly. If this penalty for paid links really exists then even websites that follow Google’s rules can get in trouble. Your competitors could harm your website simply by buying links or by creating mini-net websites with sitewide links to your website. Syndrome: keyword rank drops 60 positions.	Avoid site-wide linking. Sever links from bad neighborhood. Sever links from low quality directory sites Sever links from link farms Avoid paid links Build quality links from relevant and well-trusted sites Use varied and descriptive anchor text on links that link back to your site.
Google -950 Penalty	Jan 07	Spam Penalty, or Over Optimization Penalty. A much dreaded site or keyword drops 950 positions in ranking. Spam liking, spam documentation, content duplication, sloppy HTML that generates many validation errors. Overall, Google 950 penalty is Google’s means to discourage webmasters from engaging in any kind of spam activity and subtly directing them to follow the ideal SEO. Speculation: it’s possibly related to the Spam Detection Patent invented by Googler Anna Lynn Patterson.	Stop link farming Stop SEO spamming (over linking, keyword stuffing, bad back linking, hidden text and links, cloaking, excessive redirect JavaScript, doorway…etc) Provide unique and value adding content Clean up first, and submit a reconsideration request to Google
Delisted by Google		A hacked or a pure spam site will be delisted by Google, meaning your site will be excluded from search results. Some of the proven reasons why a site gets delisted are: 1) Repeated spelling and syntactical errors. If your website repeatedly contains a particular misspelled word, or it’s primarily made up of junk content (such as those computer generated content), you are at a high risk of being delisted from Google search. 2) Adding a large number of external links in a short time. One possible scenario is where your server is hacked and spammers add lots of links to your website without you knowing. Most of these links are hidden. You won’t see them unless you study the source code. Another possible scenario is when you are too active in link exchange. Let’s take link directories for example, most link directories will have an option for you to link back to them. If you spend one whole day exchanging links with 200 link directories, your website is at risk. 3) Sitemap error. 4) Hidden links and hidden text. Excessive use of both can get your website delisted from Google. It’s cloaking. 5) Doorway pages that redirect visitors without their knowledge use some form of cloaking. This is against Google’s principle, which is “Don’t deceive your users or present different content to search engines than you display to users.”	Well, you just violated all possible Google webmaster guidelines. Start from scratch and rebuild your site. Clean up first, and submit a re-inclusion request to Google

Jay Bhatti Discusses Spock Acquisition on ABC News

Jay Bhatti — Wed, 24 Jun 2009 10:52:20 +0000

Jay Bhatti was recently in ABC News talking about the recent sale of Spock.com to Intelius – Check out the video here