August 4th, 2011
By ICSI researchers Christian Kreibich, Nicholas Weaver and Vern Paxson, with Peter Eckersley.
Earlier this year, two research papers
reported the observation of strange phenomena in the Domain Name System (DNS) at several US ISPs. On these ISPs'
networks, some or all traffic to major search engines, including Bing, Yahoo! and (sometimes) Google, is being directed
to mysterious third party proxies.
A report in New Scientist today
documents that the traffic is being rerouted through a company called Paxfire. This blog post, coauthored with one of the teams
that discovered the phenomenon, will explain the situation in more detail.
Who is rerouting this search traffic?
The published research papers did not identify the controller of the proxy servers
that were receiving the traffic, but parallel investigations by the ICSI Networking Group and EFF have since revealed a
company called Paxfire
says that it may retain copies of users' "queries", a vague term that could be construed to mean either the domain
names that they look up or the searches they conduct, or both. The redirections mostly occur transparently to the user
and few if any of the affected ISP customers are likely to have ever heard of Paxfire, let alone consented to this
collection of their communications with search engines.
The proxies in question are operated either directly by Paxfire, or by the ISPs using web proxies provided by Paxfire.
Major users of the Paxfire system include Cavalier, Cogent, Frontier, Fuse, DirecPC, RCN, and Wide Open West. Charter
also used Paxfire in the past, but appears to have discontinued this practice.
Why do they do this?
In short, the purpose appears to be monetization of users' searches. ICSI Networking's investigation has revealed that
Paxfire's HTTP proxies selectively siphon search requests out of the proxied traffic flows and redirect them through one
or more affiliate marketing programs, presumably resulting in commission payments to Paxfire and the ISPs involved. The
affiliate programs involved include Commission Junction, the Google Affiliate Network, LinkShare, and Ask.com. When
looking up brand names such as "apple", "dell", "groupon", and "wsj", the affiliate programs direct the queries to the
corresponding brands' websites or to search assistance pages instead of providing the intended search engine results
What can I do about it?
If you want to know if the network you're currently on is subject to this type of traffic redirection, you can run a Netalyzr
test. And the best protection against the privacy and security risks created by this type of hijacking is to visit
sites using HTTPS rather than HTTP, which can easily be achieved using EFF's HTTPS Everywhere
More technical details below...
A detailed explanation
For most users of the World Wide Web, visiting a website equals clicking on a link to the site or entering the site's
name into their browser, and receiving the corresponding page from the site. Users generally assume that the site's name
is identical to the site itself, and essentially trust the site's authenticity if it looks as usual and the browser does
not pop up phishing warnings or other signs of trouble. Paxfire's misdirection of search traffic undermines this trust.
The ICSI Networking group develops and operates the ICSI Netalyzr
, a tool that tests the characteristics of users' Internet connections. Netalyzr's measurements show that approximately
a dozen US Internet Service Providers (ISPs), including DirecPC, Frontier, Hughes, and Wide Open West, deliberately and
with no visible indication route thousands of users' entire web search traffic via Paxfire's web proxies.
To explain these redirections further, we first need to delve into the workings of the Internet a bit. Since the
Internet does not route traffic to names but to network addresses, contacting a website involves translating the site's
name (say "www.google.com") to the IP address (say 18.104.22.168) of a computer that runs Google's web server. It is to
this address that the browser actually sends its request. The Domain Name System (DNS) is in charge of facilitating this
mapping of names to addresses. It is the Internet's equivalent of telephone books.
Usually, ISPs provide DNS servers (directory assistance, essentially) for their users. When a user's computer asks to
map a name to an IP address, the user's system contacts the ISP's DNS server, which looks up the correct IP address for
the name and returns it to the user. As currently implemented, this process does not provide any guaranteed correctness.
In essence, users must trust their ISP's DNS servers to correctly return IP addresses that indeed belong to the site the
user intends to visit. In some instances, however, this trust may not be warranted.
For a while now, a number of ISPs have worked in cooperation with Paxfire and similar businesses like Barefruit
to profit from mistakes that users make when typing names into their browsers. Paxfire provides a product for ISPs that
rewrites DNS errors (effectively conveying "the name you asked for doesn't exist") to responses sending users to search
pages that host advertisements, for which Paxfire then shares the corresponding ad-related revenue with the ISPs. This
practice has already been controversial.
Rerouting of requests to and responses from search engines
Paxfire's product also includes an optional, unadvertised, and more alarming feature that drastically expands Paxfire's
window into users' traffic. Instead of activating only upon error, this product redirects the customers' entire web search traffic destined for Yahoo!, Bing, and sometimes Google, to a small number of separate web traffic proxies.
These proxies collect the users' web searches and the corresponding search results, mostly forwarding them to and from
the intended search engines. This allows Paxfire and/or the ISPs to directly monitor all searches made by the ISPs'
customers and build up corresponding profiles, a process on which Paxfire holds a patent
. It also puts Paxfire in a position to modify the underlying traffic if it decides to.
Under specific conditions
, the Paxfire proxies do not merely relay traffic to and from the search engines. When the user initiates searches for
specific keywords from the browser's URL bar or search bar, the proxy no longer relays the query to the intended search
engine, but instead redirects the browser's request through affiliate networks, as the equivalent of a click on
advertisements. Using the names of popular websites, we have so far identified 170 brand-related keywords that trigger
redirections via affiliate programs and result either on the brands' sites or on search assistance pages unrelated to
the intended search engine results page.
The subset of customers affected varies from temporally localized deployments to apparently entire customer bases. The
DNS-based redirection operates in a surgical fashion, affecting only search engines but not other services such as
Google Maps or Yahoo! Mail, and remains completely invisible to the user. The treatment of Google queries varies.
Charter and Cogent appear to redirect only Bing and Yahoo, while DirecPC, Frontier and Wide Open West also used to
redirect Google to Paxfire proxies located within their own networks. Google has recently put significant pressure (see the answer to the question)
on the ISPs to get them to stop redirecting Google searches. As of August 2011, all major ISPs involved have stopped
proxying Google, but they still proxy Yahoo and Bing.