Does DuckDuckGo Want To Search the Web?

Search engines have been losing the battle against content farms for a decade or so. Around 1 March (2023 – ed.) I noticed that DuckDuckGo was including random things that it thought were physically close to my IP address in search results, things like maps, business directories, or local news stories that did not have most or any of the keywords but were things people might often click on. That made me look more into what DuckDuckGo actually is these days.

DDG explain their search as follows:

When people search, we believe they’re really looking for answers, as opposed to just links. For many categories of searches (restaurants, lyrics, weather, etc.), there is usually a specialized search engine (e.g., Tripadvisor), content site (e.g., Musixmatch), or dedicated source (e.g., Dark Sky) that does a better job of actually answering searches than a general search engine can with just links. Our long-term goal is to get you Instant Answers from these best sources.

Most of our search result pages feature one or more Instant Answers. … Of course, we have more traditional links and images in our search results too, which we largely source from Bing. Our focus is synthesizing all these sources to create a superior search experience.

https://help.duckduckgo.com/duckduckgo-help-pages/results/sources/

So the results which are local but not relevant probably come from Bing whose managers are in a frenzy about new chatbots like chatGPT. And like some other search engines, DDG envisions itself as more like an encyclopedia than a directory. The critical decision may have been made in March 2022 when DDG stopped partnering with Yandex after the Russian invasion of Ukraine. Since DDG does not create its own index of the web, and does not work with Google, that left only one other big commercially-available index.

In July 2016, DDG said that it got most search results from three sources: “We also of course have more traditional links in the search results, which we primarily source from Yahoo!, and in some regions and scenarios, Yandex and Bing.” They already had that phrase about answers not just links.

In March 2023, DDG quietly removed quotes and Boolean operators from their search syntax. They no longer promise that if you search "a b c" they will show you sites with all three of those terms before sites with just one or two, or that if you search for "tesla -automobile" you will get more pages about the Serbian-American inventor than pages about the car brand.

In Google and the Culture of Searching, I talked about how after about 2010 Google increasingly discouraged true keyword search (sites which actually have a word or phrase vs. sites which have something vaguely similar), boolean search ("a AND b BUT NOT z") or metadata search (eg. searching for a title or author in your library catalogue). If you try to search for an obscure word these days, most search engines will give you results for a vaguely similar word which people talk a lot about on the Internet (Datini becomes dating, tatbeet becomes tablet).

DuckDuckGo is the Internet privacy company for everyone who’s had enough of online tracking and wants to take back their privacy now.

https://help.duckduckgo.com/duckduckgo-help-pages/

The Internet privacy company not the Internet search company? That is a big change. Around 2022 DDG released a DuckDuckGo Private Browser as part of this new mission. Web browsers have been regulatory captured: the standards are so complicated that only a giant company can afford to implement them. While I wish DuckDuckGo all the best, trying to compete with Google Chrome and AdBlock Plus and uBlock origins will be tough!

When I mostly gave up on it in 2013, Google search was struggling with two problems: its weighting of results by who links to them was failing because more and more links were on black boxes like Facebook, and Google made its money selling ads on the same content farms and crank sites which invested vast amounts of money and effort to rise on Google search rankings. Sites that exist to make money or spread insinuations about (((globalists))) have more Google ads than sites created by knowledgeable people to share their knowledge. It seems that DuckDuckGo has given up on building something better, and I think that is unfortunate. DuckDuckGo was the last American web service company whose project I could believe was making the world a better place.

PS. I’m told that DDG has also joined the chatbot fad with something called DuckDuckAssist. They seem to have resources for everything except searching the web.

PPS. I am starting to follow DDG’s instructions and do more searches on Wikipedia or the Internet Archive or Etsy rather than search the whole web via DDG. Of course that means DDG don’t get paid to show me ads so I am not sure if that is good business strategy for them.

Since 2012 I have stopped using Windows (Linux all the way), Microsoft Office (LibreOffice + various text editors), Dropbox, Skype, and iTunes (free music players + streaming) and greatly reduced my use of Google search and a smartphone-friendly notetaking tool.

Edit 2023-03-24: comments on DDG by Alexiares at https://www.moonspeaker.ca/about.html who points to Rohan Kumar, “A look at search engines with their own indexes,” https://seirdy.one/posts/2021/03/10/search-engines-with-own-indexes/ (also on the Gemini protocol) On 12 Feb 2024, see also https://openwebsearch.eu/

Edit 2023-04-24: after months of blog and reddit posts, tech news is taking notice eg. https://www.ghacks.net/2023/04/24/duckduckgo-disables-most-search-filters-from-search/

Edit 2023-05-16: see SEP “Search Engines and Ethics” and Brin, Sergei, and Page, Lawrence, 1998. “The Anatomy of a Large-Scale Hypertextual Web Search Engine,” in Seventh International World-Wide Web Conference (WWW 7), Amsterdam: Elsevier. http://infolab.stanford.edu/pub/papers/google.pdf This paper warns that funding search with ads next to the results created a conflict of interest among the search engine, the advertisers, and the searchers.

Edit 2023-08-07: DDG has money for product placement in S2 E3 of Star Trek: Strange New Worlds. They have money for ads, a browser, and a chatbot tool but not searching?

Edit 2023-10-19: I was not cynical enough. Giving results for “dating” instead of “Datini” and “tablet” instead of “tatbeet” lets a search engine show more and more valuable ads next to them (Wired has retracted its claim that in court documents, Google confessed to actively replacing search strings with string which advertisers would pay more for, but Google have clearly and steadily widened their control over what results they give and reduced searchers’ control).

Edit 2023-11-08: A.J. Kohn, “Its Goog Enough!” 8 November 2023 argues that Google up-ranks things which more people click on, creating a positive feedback loop (people who know Wikipedia are likely to click on other Wikipedia pages) and encouraging people with a highly-ranked and reputable domain to stick random money-making things on it. https://www.blindfiveyearold.com/its-goog-enough

Edit 2024-05-15: another statement of the problem by Casey Newton at https://www.platformer.news/google-io-ai-search-sundar-pichai/ … if search engines start serving up synthetic answers rather than linking to sites, they break the social contract where they provided money and attention in exchange for people creating things for them to link to (cp. Baldur Bjarnason on how Google ads and up-ranking blogs encouraged the web 2.0 explosion in the oughties)

(scheduled 3 March 2023)