Topic-sensitive PageRank

Haveliwala's 2002 method for computing PageRank per topic, a research idea Google never confirmed using.

Answer first

Topic-sensitive PageRank is a 2002 research method by Taher Haveliwala at Stanford that computes a separate PageRank score for each of a set of topics, then blends them by how well a query matches each topic. It is general information retrieval theory. Google acquired Haveliwala's startup Kaltix in 2003 and holds later personalisation patents by the same team, but has never confirmed using topic-sensitive PageRank. The useful takeaway for sites is to earn links from pages about your topic.

What is topic-sensitive PageRank?

Topic-sensitive PageRank is a variant of PageRank proposed by Taher Haveliwala of Stanford University in a paper presented at the WWW2002 conference in May 2002, Topic-Sensitive PageRank. Instead of one global importance score per page, it computes a set of scores, one per topic.

It builds on an idea already present in Brin and Page's original PageRank paper, which noted that the random jump could be biased towards chosen pages to allow personalisation.

To yield more accurate search results, we propose computing a set of PageRank vectors, biased using a set of representative topics, to capture more accurately the notion of importance with respect to a particular topic.

Taher H. Haveliwala, Stanford University, Topic-Sensitive PageRank, WWW2002 proceedings

How does topic-sensitive PageRank work?

Offline, the paper computes 16 PageRank vectors, each biased towards the URLs listed in one top-level category of the ODP. The bias works through the random jump: instead of teleporting to any page, the surfer teleports only to pages in that topic, so pages close to that topic in the link graph score higher.

At query time, the query (and any context, such as the page the user is reading) is classified against the 16 topics, and the page's final score is a weighted blend of its topic scores. A page heavily linked from cooking sites would score well for a cooking query even if its global PageRank were modest.

By using these (precomputed) biased PageRank vectors to generate query-specific importance scores for pages at query time, we show that we can generate more accurate rankings than with a single, generic PageRank vector.

Taher H. Haveliwala, Stanford University, Topic-Sensitive PageRank, WWW2002 proceedings

What evidence is there that Google uses topic-sensitive PageRank?

This is General theory. We found no Google patent that claims topic-sensitive PageRank as described in the 2002 paper, and Google has never said it uses it.

There are connections worth stating accurately. According to Wikipedia's Kaltix article, Haveliwala co-founded Kaltix, a personalised search start-up that Google acquired on 30 September 2003. Google later obtained patents with Haveliwala, Glen Jeh and Sepandar Kamvar as inventors, including US7716223, which describes boosting results from sites associated with a user's topics of interest, and US7260573, which describes personalised page importance scores. These are personalisation patents, not topic-sensitive PageRank, and a patent is not proof of use.

A user interest profile identifies topics of interest to a user.

Taher H. Haveliwala, Glen M. Jeh and Sepandar D. Kamvar (Google), US7716223B2, Variable personalization of search results in a search engine, Google Patents

What does topic-sensitive PageRank mean for sites?

Even without confirmation, the underlying claim is testable and consistent with Google's confirmed statement that its link analysis systems use links to determine what pages are about. A link from a page on your topic plausibly tells a search engine more than a link from an unrelated page.

The falsifiable audit is to classify your referring domains by topic. If most links come from sites with no topical connection to yours (general directories, unrelated blogs, sponsorship pages), your link profile says little about what you are an authority on, whatever the theory in use.

  • Prioritise links and mentions from publications, associations and resources in your sector.
  • Link internally within topic clusters so topical pages reinforce each other.
  • Treat off-topic link volume as noise, not authority.

What are the key dates for topic-sensitive PageRank?

  • 2002-05: Topic-Sensitive PageRank paper presented at WWW2002 (source)
  • 2003-09-30: Google acquires Kaltix, co-founded by Haveliwala (source)
  • 2004-03-29: Priority date of Google personalisation patent US7716223 (Haveliwala, Jeh, Kamvar) (source)

Frequently asked questions about topic-sensitive PageRank

Does Google use topic-sensitive PageRank?

Google has not said so and we found no Google patent claiming it. It is a 2002 Stanford research method, so we label it general theory.

Who invented topic-sensitive PageRank?

Taher Haveliwala, then at Stanford University, in a paper presented at the WWW2002 conference. He later co-founded Kaltix, which Google acquired in 2003.

Are relevant links better than unrelated links?

Google confirms links help it determine what pages are about, which makes topically related links more informative. The size of any effect is not published.