EdgeLM: Edge Demonstrations for Language Models' Table Understanding
Preprint on arXiv.
The problem
Tables in the real world arrive with problems. Values are missing, two records describing the same customer do not match, columns that mean the same thing carry different names. Somebody has to find the wrong cells, merge the duplicates, line up the columns and fill the blanks before anything built on the data can be trusted. Traditionally each of those jobs got its own system, with its own rules, its own features and its own training data.
Language models can take on all of them at once, which is why so much recent work points them at tables. The way you steer one is by putting a handful of solved examples in the prompt and letting it infer the pattern from them. Those examples are called demonstrations, and which ones you choose changes the answer a great deal.
The standard way to choose them is to retrieve whatever most resembles the case at hand. It is the obvious choice and a reasonable one, but it has a blind spot.
Why it matters
Examples that look just like the query tend to agree with each other. If you are asking whether two records refer to the same person, and every example you show is a pair that matched, you have told the model several times over what a match looks like and nothing about what a near miss looks like. The examples confirm the answer the model was already leaning towards.
The cases that need help are the ones where two records look almost identical and a small difference flips the answer. Those are exactly the cases where similar examples say the least. The easy cases, which similarity handles well, did not need the help in the first place.
Main idea
Retrieve examples that are close to the query and also show where the line falls.
EdgeLM looks for two kinds. The first is nearby examples whose correct answers differ from each other. Two records almost alike, one a match and one not, put the boundary in front of the model instead of one side of it. The second is nearby examples the model itself got wrong when asked with no help at all. These are collected once in advance for each model, and they mark the places where it is unreliable, so showing one together with the right answer corrects the particular mistake it is prone to.
Nothing else changes. There is no retraining and no task specific engineering. The prompt has the same number of slots as before, filled differently.
The result
EdgeLM was tested on five table tasks, which are error detection, entity matching, schema matching, filling missing values and anomaly detection, across fifteen datasets and five language models, three open weight and two proprietary. Every strategy compared shares the same prompt, the same pool of examples and the same test set, so the only thing that varies is which examples fill the slots.
It comes out best or tied for best in almost every combination, and it never loses to plain similarity. The gains are largest where the baselines leave the most room, on anomaly detection, where the thing being looked for is rare and easily missed, and on error detection. Where the task is nearly solved already, as schema matching is, the gain shrinks to very little, but it stays positive.