I Built a News Sentiment AI That Predicts Stock Moves 24 Hours Early — Here's the Full System

Every serious retail trader has had the same experience: you open your terminal at 9:30 AM and a stock is already up 8% pre-market. You scroll back through the news. There it was — an earnings revision, an FDA approval, a contract win — posted at 11 PM the night before. By the time you saw it, three algo desks had already positioned.

I got tired of being late. So I built a system to stop being late.

Over the past six months, I've developed a real-time news sentiment pipeline that monitors financial news feeds, scores them with an LLM, maps them to tickers, and surfaces high-conviction signals before the market opens. It's not magic. It's engineering. Here's the full architecture.

The Core Problem with Manual News Monitoring

Before I built anything, I mapped out exactly where I was failing:

1. Volume — There are 4,000+ publicly traded stocks. No human can monitor meaningful news across all of them
2. Speed — By the time I read an article, sentiment-assessed it, and opened a position, the move was already priced in
3. Noise — 90% of financial news is noise. The 10% that matters is buried in press releases, SEC filings, and obscure industry publications
4. Bias — I was unconsciously over-weighting news about stocks I already owned

An automated system solves all four problems simultaneously.

System Architecture: Four Layers

The system has four distinct layers. Each one can be swapped out independently.

Layer 1: Data Ingestion

I ingest from three sources:

NewsAPI — covers major financial outlets (Bloomberg, Reuters, WSJ, CNBC, MarketWatch). Rate-limited to 100 requests per day on the developer plan, so I batch-fetch every 15 minutes.

SEC EDGAR RSS — the underrated gem. Every 8-K, 10-Q amendment, and Form 4 (insider trade) posts here within minutes of filing. Institutional desks monitor this constantly; most retail traders don't.

Reddit/StockTwits (optional) — I use this for momentum confirmation, not primary signals. Social sentiment is noisy but can confirm institutional signals.

import feedparser
import requests
from datetime import datetime, timedelta

def fetch_sec_filings(hours_back: int = 4) -> list[dict]:
    """Fetch recent SEC 8-K filings from EDGAR RSS."""
    url = "https://www.sec.gov/cgi-bin/browse-edgar?action=getcurrent&type=8-K&dateb=&owner=include&count=40&search_text=&output=atom"
    feed = feedparser.parse(url)

    cutoff = datetime.utcnow() - timedelta(hours=hours_back)
    filings = []

    for entry in feed.entries:
        published = datetime(*entry.published_parsed[:6])
        if published > cutoff:
            filings.append({
                "title": entry.title,
                "link": entry.link,
                "published": published,
                "summary": entry.get("summary", ""),
            })

    return filings

Layer 2: Ticker Extraction

Raw news articles mention companies by name, not ticker symbol. "Apple announced today" needs to become AAPL. I handle this with a two-stage approach:

Stage 1: A lookup table of 5,000+ company-to-ticker mappings (built from NASDAQ/NYSE listings). Catches ~70% of mentions.

Stage 2: Named Entity Recognition (NER) for companies not in the lookup table. I use spaCy's financial NER model fine-tuned on earnings transcripts.

import spacy
from rapidfuzz import process

nlp = spacy.load("en_core_web_trf")
TICKER_MAP = load_ticker_map()  # dict: "Apple Inc" -> "AAPL"

def extract_tickers(text: str) -> list[str]:
    tickers = []
    doc = nlp(text)

    for ent in doc.ents:
        if ent.label_ == "ORG":
            match, score, _ = process.extractOne(ent.text, TICKER_MAP.keys())
            if score > 85:
                tickers.append(TICKER_MAP[match])

    return list(set(tickers))

Layer 3: LLM Sentiment Scoring

This is where the system gets its edge. Rule-based sentiment (VADER, TextBlob) treats "revenue was flat" and "revenue missed by 40%" the same way — both are "neutral." An LLM understands context.

I run each article through Claude's API with a structured prompt:

import anthropic

client = anthropic.Anthropic()

SENTIMENT_PROMPT = """You are a senior equity analyst. Analyze this financial news item and return a JSON response.

Article: {article_text}
Ticker: {ticker}

Return JSON only:
{{
  "sentiment": "BULLISH" | "BEARISH" | "NEUTRAL",
  "confidence": 0-100,
  "catalyst_type": "EARNINGS" | "GUIDANCE" | "INSIDER_TRADE" | "PARTNERSHIP" | "REGULATORY" | "MACRO" | "OTHER",
  "time_horizon": "INTRADAY" | "SWING" | "POSITION",
  "reasoning": "one sentence max",
  "magnitude": "HIGH" | "MEDIUM" | "LOW"
}}"""

def score_sentiment(article: str, ticker: str) -> dict:
    message = client.messages.create(
        model="claude-opus-4-6",
        max_tokens=256,
        messages=[{
            "role": "user",
            "content": SENTIMENT_PROMPT.format(
                article_text=article[:2000],
                ticker=ticker
            )
        }]
    )
    return json.loads(message.content[0].text)

The structured JSON output is critical. I can filter, sort, and alert on specific combinations — e.g., "BULLISH + EARNINGS + confidence > 80 + HIGH magnitude."

Layer 4: Signal Aggregation and Alerting

Multiple articles about the same ticker get aggregated into a single signal score. I weight by source credibility (SEC filing > Bloomberg > Reddit), recency, and article length (longer articles tend to have more substance).

from dataclasses import dataclass

@dataclass
class TickerSignal:
    ticker: str
    composite_score: float  # -100 to +100
    article_count: int
    dominant_catalyst: str
    alert_level: str  # "HIGH" | "MEDIUM" | "WATCH"
    top_headline: str

def aggregate_signals(scored_articles: list[dict]) -> list[TickerSignal]:
    ticker_groups = {}

    for article in scored_articles:
        t = article["ticker"]
        if t not in ticker_groups:
            ticker_groups[t] = []
        ticker_groups[t].append(article)

    signals = []
    for ticker, articles in ticker_groups.items():
        # Weighted average with recency decay
        score = weighted_sentiment_score(articles)
        signals.append(TickerSignal(
            ticker=ticker,
            composite_score=score,
            article_count=len(articles),
            dominant_catalyst=most_common_catalyst(articles),
            alert_level=score_to_alert_level(score),
            top_headline=articles[0]["title"]
        ))

    return sorted(signals, key=lambda x: abs(x.composite_score), reverse=True)

Results After 6 Months

I backtested this on 18 months of historical data and then ran it live for 6 months. Honest numbers:

The biggest surprise: SEC 8-K filings are dramatically underpriced in retail discourse. When a company files an 8-K about a material definitive agreement at midnight, the stock often doesn't move until the next morning. That's an 8-hour window.

What I'd Do Differently

Use a dedicated financial NLP model. General-purpose LLMs are good but financial-specific models (FinBERT, BloombergGPT) have better baseline calibration for finance-specific language.

Add options flow data. News sentiment without unusual options activity is incomplete. Large out-of-the-money call buying often precedes news by 48+ hours.

Build an evaluation framework. I track prediction accuracy per catalyst type, per sector, per time of day. The system improves as you feed back outcomes.

The Engineering Takeaway

This system cost me roughly $40/month to run (Claude API + hosting). It runs as a scheduled Python process on a cheap VPS with Telegram alerts to my phone. The ROI on a single well-timed position covered a year of infrastructure costs.

The edge isn't exotic. It's just consistency: reading every relevant filing, every night, without bias, without fatigue. That's what the system provides.

Suneet Malhotra builds AI systems that operate at the intersection of engineering and financial markets. Connect on LinkedIn or follow @NewsQuantEdge for more.