Systems and tools Live across 50 states

Market signals

This system gathers fifteen kinds of early evidence on every state and occupation, grades each by how much it can be trusted, and combines them into two flags whose logic anyone can follow.

Economics · data science. Tens of thousands of live signal observations feed every downstream surface.
01

The problem

Official statistics come out quarters late, and decisions often have to be made before then, so earlier evidence has to fill the gap. That includes job postings, layoff notices, unusual wage moves, federal awards and even the tone of regional economic reports. All of it is noisy and uneven and easy to over-read. The work is in grading how much each piece can be trusted and combining the pieces so that whoever has to act on them can follow the logic.

02

How it works

01Fifteen evidence types, one table

Every signal, quantitative or qualitative, is recorded by state and occupation, so a wage anomaly, a wave of retirements and a change in sentiment can be read side by side.

Data science · engineering

02Tiered confidence

A strong source can raise a flag by itself. Weaker sources only count when two of them point the same way. Unconfirmed signals are kept and labeled, because a rumor that is later confirmed is useful to have on record.

Economics · data science

03Two flags, with readable logic

All the signals roll up to two flags, market stress and rising market, and each is an explicit OR over named signals. A reader can always answer "why is this flag up" by naming the signal that raised it.

Economics

04Language as a data source

Qualitative sources such as regional economic commentary and the trade press are read by an LLM using a fixed extraction schema. That turns the text into signals by state and occupation, like the statistics, at very little cost per release.

LLM systems · data science
Evidence ten of fifteen shown

Each type carries its source and what it is allowed to claim.

01Wage anomalyoccupational wage statistics, location quotients
02Employment shiftpayroll series, shift-share decomposition
03Retirement pressurecensus microdata age structure
04Skill driftoccupational skill profiles, across vintages
05AI exposuretask-level exposure measures
06Layoff noticesstate advance-notice filings
07Federal investmentfederal spending records
08Posting intensityjob postings, deduplicated
09Licensure volumecredential and licensure rosters
10Regional sentimentregional economic reports, LLM-extracted
Each signal type is recorded by state and occupation. Strong sources can raise a flag by themselves, and weaker ones only count when they agree with each other.
From prose to a signal switch the source

The hardest sources to use are written text, such as a regional economic report, a press release or a layoff filing. An LLM reads each one using a fixed extraction schema and returns structured signals by state and occupation, like the statistics, each with the sentence it came from.

Illustrative extractions. The schema fixes the fields, the occupation resolves to a federal code, and the basis keeps every signal traceable to its source sentence.
Two flags composition, spelled out

The rollup is an OR over named signals, so the answer to "why is this flag up" is always a signal you can point to.

market stress = retirement wave ∨ skill decline ∨ sector contraction
rising market = growth cluster ∨ skill emergence ∨ AI tailwind ∨ sector tailwind
Strong evidencefires alone
Weaker evidenceneeds two, agreeing in direction
Unconfirmedkept, and labeled unconfirmed
Each flag records which signals raised it and at what confidence tier.
03

Tools used

Python, pandasThe state-by-occupation signal table and the rollup.
Ollama, Qwen, AnthropicLLM extraction of written sources under a constrained schema.
statsmodelsShift-share decomposition and wage-anomaly detection.
requests, BeautifulSoupIngestion of postings, filings, and federal spending records.
Supabase, PostgresTens of thousands of live signal observations.
The occupation resolverPosting standardization to the federal crosswalk.
04

Final state

The system runs live across all fifty states, with tens of thousands of signal observations drawn from a larger base of source rows, each refreshed on its source's own schedule. It feeds the opportunity index, the state dashboards and the labor market twin. The signal types and confidence rules shown here are the production rules, and the example notes are illustrative.

← Back to systems and tools