Opinionated retrieval pipeline for the agentic era

You're an analyst and someone hands you a stack of ten thousand SEC filings. "Find me companies in unrelated industries that went through the same strategic pattern as this one, and tell me how it played out for each of them." they ask. You'd need to read the filings, decide which events matter, figure out which events across different companies are actually the same kind of move, and then trace the outcomes over time. That’s hours, if not days, of work with no anchoring starting point. You wouldn’t know where to look, because there’s no index for strategic precedents.

The industry has spent the last three years building systems that chunk up documents, embed them, and retrieve the most similar fragments when you ask a question. That machinery — the full RAG stack — was a genuine innovation when language models were only good for one-shotting queries with direct answers. Fast forward to today and it’s evident that agentic systems can read full documents and reason over their findings in multi-step sequences and access to tools.

The Scaffold Era

For a couple of years, the dominant pattern for building AI research tools went like this. You take a large corpus — say, SEC filings for every public company — and you break each document into small fragments. Five hundred tokens each, maybe a thousand. You convert each fragment into a high-dimensional vector using an embedding model and store those vectors in a database. When a user asks a question, the question is vectorized to find the closest fragments by cosine similarity, then a reranker is used to surface the best matches, and feed the result to a language model for synthesis. This worked for most cases, but an agent comes close to that level of execution without the burden of the infrastructure.

For the class of questions that RAG was designed to answer — "what does this document say about X?" — the agent approach is already better. The scaffold era is ending. The task of helping someone understand what's in a document hasn't gone away, but the fragile machinery that is built to approximate it has.

Two Kinds of Systems

But there's something getting lost in the transition. While retrieval systems aren’t dying, there’s a conflation between two fundamentally different kinds of systems.

The first kind has no opinion about its domain. It ingests arbitrary documents and answers arbitrary questions at query time by finding similar text. The system doesn't know whether it's reading a 10-K or a recipe book. It doesn't care. It just retrieves fragments. This is what agents absorb completely.

The second kind has a specific, durable thesis about what matters. It does analytical work at ingestion time, not query time. The output is structured analytical artifacts, purpose-built objects that encode the pipeline's opinion about what's significant in the raw material. Those artifacts get classified, connected to each other, and indexed into a searchable substrate that looks nothing like a vector database full of text fragments.

The first kind is a librarian. It shelves books and remembers where they are, while the second kind is closer to a factory floor. Raw material comes in one end. It goes through a series of opinionated transformation steps and what comes out the other end is manufactured analytical objects. Those objects are what make certain categories of research questions answerable at all. That’s the system Slinky Adopts.

The Factory Floor

Step 1: Transforming text to situations

Slinky is predicated on the idea that the mountainous fragments of raw text published by — and about — companies in their regulatory filings, on websites, or even in the news, can be weaved into precedents. This means reducing all disclosed information into a neutral microcosm that can carry events with strategic significance and support analogical reasoning. The first opinion the pipeline holds is about the unit of analysis. A situation: a discrete, named, dated strategic event with a specific structure.

When I say "specific structure," I mean the pipeline produces an object with defined fields. A situation has an actor: which company or executive is doing something. It has an action: what they're doing. It has an outcome: what happened or is expected to happen. It has a tier score that represents how strategically significant the event is. And it has a confidence score that represents how much the pipeline trusts its own extraction.

A situation extracted from a 10-K is a searchable, comparable, and stackable artifact, applicable across thousands of companies. You can sort by date. You can find every situation in your corpus where the action was "divested a business unit" and the tier score was above a threshold. Try doing that with chunks.

The extraction step is where the pipeline earns its opinion. Most text in a filing is noise. A risk factor disclosure that says "we may face competition from larger companies with greater resources" appears in virtually every 10-K ever written. That is not a situation. It's boilerplate. But a disclosure buried in Item 7 that says the company is shutting down its consumer hardware division and redeploying the engineering team to an enterprise platform is an actual situation. The pipeline has to know the difference, and knowing the difference is an opinion about what constitutes a strategically meaningful event.

Step 2: Threading situations into arcs

A single situation is a data point. By itself, it tells you what happened, but it doesn't tell you where a company is headed.

Once situations are extracted, they get connected over time into arcs. Arcs are longitudinal sequences of related events for the same company. A company initiates a cost-cutting program. Six months later, it announces a leadership restructuring in its sales organization. A year after that, it divests a business unit that was never profitable. Individually, each of those is a data point an analyst might note and move on from, but together, they form a recognizable strategic arc.

The arc is the pipeline's second opinion: that isolated events are less meaningful than the sequence they belong to. If you're looking at a company and all you have is the latest quarter's filing, you see a snapshot. But if you have the arc, you see a trajectory. You see whether a cost-cutting announcement is the first move in a long restructuring or the tail end of one that's almost finished.

"They just announced layoffs" is a fact. "They just announced layoffs, and this is the third contraction event in eighteen months following a failed international expansion" is context. The arc provides that context automatically, because the pipeline threads events as they arrive.

Step 3: Matching arcs to patterns

Each extracted situation gets matched against a library of canonical archetypes (i.e patterns), which are recurring strategic playbooks that appear across industries, company sizes, and time periods. A pattern might be "pricing power erosion following market saturation." Another might be "platform consolidation during leadership transition." These are patterns that show up again and again in different costumes across different sectors.

When a situation matches a canonical with sufficient confidence, it inherits that canonical's precedent set. That is the mechanism that connects a SaaS company's current pricing squeeze to a consumer electronics manufacturer that experienced the same pattern five years ago, to a retail chain that navigated it in a different market a decade before that. The companies have nothing in common by industry. They have everything in common by strategic pattern.

The pattern is the pipeline's third opinion: that strategy is fundamentally repetitive, and the most useful thing you can do with a new event is ask where you've seen it before. That opinion is what makes lateral thinking — the kind that connects disparate companies across disparate industries — mechanically possible at query time. This is akin to sifting through thousands of filings to pattern-match, but instead you're querying a pre-computed index of patterns that already exist.

The Structured Memory

With all that threading and clustering, what comes off the factory floor is not blobs of text, but a structured analytical memory: a durable, queryable substrate of situations, arcs, and patterns that encode a specific thesis about what matters in corporate strategy.

The difference from text retrieval matters enormously for the kind of question you can answer. A retrieval system — whether it's RAG or an agent navigating raw documents — can help you understand what a single company disclosed in a single filing. That's truth-seeking and it works great.

But ask a different kind of question. Ask: which companies in unrelated industries have navigated the same canonical arc in the past five years, what stage did each reach, and what outcomes did they land on? That question requires having already done the work of extracting situations, threading them into arcs, matching arcs to patterns, and indexing the result. The pre-computation is the product. Without it, the question is unanswerable; not because the information doesn't exist in the raw filings, but because no one can affordably and consistently perform that analytical labor across thousands of documents at query time.