Note: AI2 is also a provider of Wonderful Semantic Scholar Database. New research tools (Paper Finder0 is freely accessible) discussed below. By the way, today’s announcement follows some related news items (other research tools) from AI2. Posts from January 2025 and November 2024 are also linked at the bottom of this post.
Today we are releasing AI2 Paper Finder, a LLM-driven literature search system.
I believe that AI-based literature search should follow the research and thought processes that human researchers use when searching for relevant papers in their field. The AI2 Paper Finder is built on this philosophy and is excellent at finding papers that are difficult to find using existing search tools.
Consider your own research process and how to search for papers. For example, imagine you are looking for a paper in English with annotations of one of the speakers’ property (emotion, age, gender, etc.) and introducing a dataset of unscripted dialogue between two speakers (written or transcription).
How do you approach it today?
You’ll start by choosing either Semantic Scholar, Google Scholar, or regular Google searches, or by asking LLMs like GPT. Next, create a search term (based on your knowledge of both your domain and your selected tool) and see what you see. The results may not be exactly what you need, but they are a good start. From search results, your intuition will guide you to more follow-up: learn new vocabulary, remind you of related concepts that lead to new search queries, find promising leads and follow the work of a particular author. The point is that literature search is a multi-step process that involves learning and iteration as you go.
This thought process was built into an AI2 paper finder. Once you enter a query, you can see the system break down the query into related components, search for papers, follow citations, evaluate relevance, and run follow-up queries based on results.
AI2 Paper Finder does not need to simplify queries as keywords to perform effective searches. This does not require you to perform an effective search like in the first example.
(clip)
We look forward to sharing our first iteration of AI2 Paper Finder with the community. This is a work in progress and I look forward to interacting with everyone, using it for my day to day work, and providing feedback on the outcomes. Try it, look at its boundaries and let us know if it doesn’t work as expected! The AI2 Paper Finder is far from perfect, actively monitoring feedback and looking for opportunities to improve.
Paperfinder vs. Other tools
How does AI2 Paper Finder differ from other literature search solutions? First and foremost, we aim to be as open as possible. We are very interested in the research questions that Openness makes possible, and we are committed to fully and openly explaining every aspect of the system. To that end, a detailed technical report will soon be released. We also aim to open up the query streams we receive. This will mine interesting queries and release them as community-wide benchmarks until users opt in. Issues related to academic copyrights complicate the process of open-sourcing today’s code, but we hope that our commitment to openness empowers other parts of the community that participate in us to tackle big research questions.
Other initiatives (including AI2 ScholarQA) have compiled research summary. The summary is based on search, but unlike paper discoveries. The difference is not only in the format in which the results are presented (lists or summaries), but also in the way information is intended to be consumed. The summary is meant as a summary and is not intended to be exhaustive. If you get one prominent paper from the area, it’s okay to ignore the others. Paper discoveries hope for a much more comprehensive result. While the summary is primarily intended to learn about new topics, paper discoveries can help you dig deeper into areas you already know.
