Every year, Google processes five trillion searches. This huge number is a clear indicator of the monopoly it has on the search engine market, not to mention the environmental and social issues associated with Big Tech search engines. So, what would it take for us to have free, unbiased access to search? We spoke to Professor Michael Granitzer, Chair of Data Science at the University of Passau, for his insight on all things search engines. Together with the Open Search Foundation, Grantizer is leading the EU-funded OpenWebSearch project, which is developing a new, European search index and has already indexed 2.7 billion URLs. Here’s how Granitzer hopes to make the future of search brighter with an open-source index developed for the public good.
RESET: Why is European digital sovereignty important in terms of search? And what are the dangers of a monopoly in the search engine space?
Michael Granitzer: I think nobody in Europe would accept that there is only one or two newspapers where you get your daily information from. But this is the situation we have when it comes to search. We consume a lot of information from the web, and one of the major entry points is web search. Of course, social media is another entry point that keeps becoming stronger, but web search is still there. And it is controlled by a few gatekeepers.
I don’t think it’s a risk that these aren’t European gatekeepers, or that they’re big companies. But the essential part is the monopoly: that only a few companies control the search landscape. We have to ask, what would happen in an extreme case, if they just shut down their services? Finding out information would be impossible because we wouldn’t have a search engine anymore.
How are you building the European Open Web Index, and what are you hoping to achieve?
Our mission is to break up the silo of a single search engine. We’re doing this by crawling the web, collecting web pages and preparing them to be consumed by search engines. Preparing them involves cleaning advertisements and navigation links, then extracting the main content. This index can be used by individuals or organisations to build their own search engines.
For example, one scenario would be helping companies that run searches on their own data sets. They might want to augment that search with web data, which our index could provide. Our partner, the German Aerospace Center, is doing that for scientific research. The team has internal scientific resources on geospatial data, and they augment this data with our web index, creating a special-purpose search engine for their use case.
How can we ensure a green digital future?
Growing e-waste, carbon emissions from AI, data centre water usage—is rampant digitalisation compatible with a healthy planet? Our latest project explores how digital tools and services can be developed with sustainability in mind.
Is the index completely accessible for everyone? Would another use case be, for example, that I’m an individual who wants to build a search engine? If so, can I access it?
Yes, you can find our datasets at openwebindex.eu. We also hope to launch a search engine for researchers in the next few months that allows you to search all of our data.
And how long does this indexing take?
We’re indexing 24/7. We currently have around 25 to 30 machines that are crawling roughly 100 million URLs per day.
In terms of scale, could your search index rival Brave or Google?
To be on a bar with what’s out there, we’d need to increase our efforts by a factor of 20 or 30. We could do it, but it’s a matter of resources and cost. We’d have to hire people to maintain the service; we’d have to buy more storage.
We want to make this data collaboratively available, similar to Wikipedia. If we can drive this project as a community, [we could use that support to scale together].
What data centres do you work with?
We’re based in Passau and work with European HPC data centres, like Leibniz Supercomputing Centre in Munich (just 200 kilometres from here), IT4Innovations Czech EuroHPC JU Center, CSC EuroHPC JU Center in Finland and CERN in Switzerland. So we are an academic project with a network of supercomputing centres focused on high-performance computing and AI. But we are open to working with commercial data centres and anyone who wants to join our mission.
If you had to recommend a search engine that’s currently out there, which one would you recommend? Which search engine do you use?
I think it would be bad advice to recommend just one. Instead, I’d advise critically reviewing your current habits to see if something else could fit better. Personally, I use Qwant for some tasks, DuckDuckGo for others, and sometimes Ecosia.
If people try out new search engines, they might discover parts of the web that could be especially interesting to them. That’s why I don’t recommend just one. Go out and find other search engines and invest skills in finding new, fresh information.
A guide to privacy-focused, climate-friendly search
Want a breakdown of your search engine options? Check out our guide: How to chose a search engine in the age of AI.
And what’s your view of AI overviews being added to search engines?
Well, I think it’s helpful. AI is a tool. You just have to be aware that it can be very wrong, so you always have to cross-check and double-check to see if something is truly correct. But it’s not so different from reading a blog article written by a non-expert. I think people often forget that you can encounter so much bad content on the web that people automatically believe. So, AI can help as part of your research, just don’t take its answers for granted, as you shouldn’t with anything you read on the web. And watch out for losing your own research competencies if you rely too much on AI!
What do you wish more people knew about search engines? Are there any common misconceptions you want to set straight?
That’s tricky. Maybe the biggest misconception is that you can’t find information if you don’t use Google. And the second would be that these services are free. Search engines cost a lot of money to run and maintain. As the searcher, you are the product. That’s the most important part that people need to understand. We pay so much in advertising taxes, and Google is running an advertisement monopoly. That’s the big misconception that people need to understand: that they are paying a huge price for not using alternative search engines.
How do you address the environmental impact of search with the European Open Web Index?
If you look at what is currently happening in big tech, they have completely abandoned any agenda to try to become CO2 neutral. We have to question what’s needed. Is AI-based search really necessary? Why are we building data centres in the thousands? By having search as a public good, we could steer the digital ecosystem in a more sustainable direction. Sustainable in the sense of energy and green technology, but also socially. We saw what happened with Grok AI generating fake sexist images and posting them to X. We need alternatives to Big Tech that follow an ethical, moral and humanistic approach; that’s why we’re building the European Open Web Index.
What’s your vision for the future of search?
We have to think of the future society as having man and models: AI models with humans operating them. In an ideal scenario, the AI model is running on my machine and controlling my data. It’s a tool that helps me, conducts searches on my behalf and understands what I want to do. I’m talking about small language models rather than large language models. A model that helps me search, aggregate and synthesise information based on search endpoints that I choose.
Again, back to the newspaper example: I can choose a set of newspapers to get my information from. But the newspaper is still written by a human. Decentralised, with very different players, each bringing their speciality in how they create content. And I have my personal agent who can help me collect and analyse all of this. That, for me, would be ideal.
