The rise of startups in AI

Digitization of the world has produced a vast amount of data to feed artificial intelligence algorithms. In the natural language processing (NLP) field domain, OpenAI’s GPT-3 is a significant milestone built with billions of parameters. GPT-3 is a language prediction model and uses deep learning to generate human-like text, identify themes and emotions, and other advanced AI features.

We've all experienced rudimentary NLP systems, such as those that handle customer service phone calls - often frustratingly. The state of the art has made remarkable advances lately. For example, Avoma, a TSVC portfolio company, has an NLP-based bot can attend your Zoom, Google Meet, and other videoconference meetings, transcribe meeting minutes for you, and, increasingly, understand context and meaning to identify action items and other important aspects of what was said.  NLP-based systems will handle more and more human tasks like this.

The evolution of the search engine

As exciting as it is to see NLP taking over human tasks, what's even more exciting is the confluence of NLP technologies with data technologies to enable new capabilities that are far beyond what humans can do. Search is just such a capability. We believe that AI will alter the landscape of search engines - from keyword matching to context matching - giving search engines a better understanding of what users are looking for.

The way current search engines process information is to focus on keywords, matching them with an internal index to come up with the most relevant results. However, keyword matching is subject to gaming by Search Engine Optimization (SEO) which negatively influences the value of results returned.

About a decade ago, researchers introduced the semantic web, an ambitious attempt to drive machine-understandable web content and  context. It could potentially have improved web search. Unfortunately it failed; the semantic web must be constructed by humans and requires huge effort. Ontology construction with millions or even billions of semantic nodes is impractical. Today, with technological advancements in AI, language models can interpret currently available web resources and train to understand the users’ input in a different way.

Metaphor - Changing the search game

Search engines are hugely valuable; we use them dozens of time a day. Yet the industry has a lot of room for improvement, and artificial intelligence is how much of this improvement will be delivered. The process of guiding machines with keywords is too rigid. The goal should be to have AI truly understand what content we need. Just like when we seek help from an expert, we don’t ask him/her to search for answers; rather, we want the expert’s input that fits our own scenarios.

We believe that once AI is mature enough, the search engine will fade away and be replaced by an intelligent content assistant. Under the influence of machine learning and advancement in AI, there is a potential to replace the search engine with intelligent content assistants and shift the focus from keywords matching to context-based instead.

Wanderer2, developed by Metaphor, is the first publicly available language model that can perform a web search on a large scale. Unlike other search engines, Wanderer2 was trained to search and predict web pages through context rather than keywords alone. For example, if a user types in “The most promising startups in health tech are” in the search section, Wanderer2 can read the context of the question and predict the most likely outcomes the user is looking for. In this case, Wanderer2 will show health tech startups with relevance measures according to context.  Wanderer2 was trained on millions of links from Hacker News which give it an edge over standard search engines on many technical topics.

Besides web search, Wanderer2 can also perform similarity search. Wanderer2 can search millions of links and find the most similar web pages on the internet. With an ability to find similar companies, posts, and blogs, it has the potential to create more values and optimal results for users.

Alex Gajewski, Metaphor's founder, intends to redefine search, creating a next-generation experience with organic and context-based results. In the company's own words, they are “building the world’s most advanced search engine with language models.” With the fast past of machine learning and technical advancement in artificial intelligence, we believe Alex’s approach to search technology can initially revolutionize search within important vertical industries. From there, the sky's the limit.

https://www.deepcrawl.com/knowledge/technical-seo-library/how-do-search-engines-work/

https://omarzahran.medium.com/the-evolution-of-the-search-engine-c9b0bb08bfb0

https://www.wordstream.com/articles/internet-search-engines-history

https://www.searchenginejournal.com/seo-101/meet-search-engines/

https://www.hotbot.com/blog/the-pros-and-cons-of-using-a-private-search-engine/

‍