5 min
Most of the modern day NLP applications are deployed for chatbots, which is a conversational task involving questions which require mostly one word answers. This makes open-domain question answering (QA) a crucial benchmarking task in natural language understanding (NLU).
AI researchers and linguists have been collaborating to figure out a way to supplement the pursuit of General AI with a structure, universal at its core and flexible in its deployment.
Most existing QA tasks are constrained — both to specific knowledge domains and to answers of a single word or phrase from the input passage. They require identifying a simple fact in a single web document, which is then presented as the answer, but existing QA systems can’t offer rich explanations the way people do.
To help advance question answering (QA) and create smarter assistants, Facebook AI has shared the first large-scale data set, code, and baseline models for long-form QA, which requires machines to provide long, complex answers.
Overview Of ELI5 Dataset
[caption id="attachment_43543" align="alignnone" width="832"]
via Facebook AI[/caption]
This new long-form QA dataset challenges existing algorithms because it requires processing many web documents comprising hundreds of thousands of words, identifying the relevant information in those documents, and writing a longform response to an often open-ended question.
[caption id="attachment_43541" align="alignnone" width="686"]
via Facebook AI[/caption]
To build the data set, the researchers have leveraged a public subreddit titled “Explain Like I’m Five” (ELI5), in which an online community answers questions with responses that 5-year-olds can comprehend. The data set comprises 270K threads of diverse, open-ended questions that require multi-sentence answers.
QA models for ELI5 mimic what many people do when they're asked a question: Just like how humans launch their browsers to find/learn about a topic, these QA models too, make an attempt at exploiting the same strategy.
ELI5 combines the challenges of synthesizing information from multiple sources, answering questions, and generating text into a real-world task, making it a more realistic and difficult task than prior QA data sets
Generating Abstract Models
[caption id="attachment_43542" align="alignnone" width="720"]
via Facebook AI[/caption]
This open source data set has also been used to introduce two models:
- extractive models, which produce answers that are copied word for word from the supporting documents, and
- abstractive models, which can rewrite the information in the supporting documents as needed.
Filling The Fluency Gap
The black cat has crossed the road — this sentence might sound simple but when one tries to translate it to native languages, all the historical inferences and semantic sophistications come into play. For some feline lover, that sentence might remind them of fluffy cat pictures and crossing the road is unimportant to them. For some, it could be an ominous warning, and for others a sign of prosperity. A harmless statement like this can throw amateur linguists into disarray. At a more generalised level, it gets tricky for the machines to respond to queries which are interlaced with references and trends. The ELI5 data set and the accompanying baseline models help us make progress toward this goal of attaining a human like understanding of language. The researchers firmly believe that these models can collectively affect the way people access information, particularly in the development of more intelligent assistants. Read the full paper here.- Jev Isn’t An Overnight Fix For Your Lack of AI Fundamentals
- Internet on Alert as AI Uncovers Critical Flaws in PHP Used by 70% of Websites
- From Claude Code Mods to Super Mario: What People Are Building on Jev
- 8 Reasons Why Anthropic’s New Report is an AI Doomer’s Worst Nightmare
- How VECV Built a Real-Time Data Backbone for 2,50,000 Connected Vehicles
- Anthropic Stands Alone Against Open Models