Wednesday, November 18, 2026
Data from SEC filings, checked daily
My DeskFree
Public filings, plainly told
Introduction to big data and machine learn
Summarize with AIChatGPTClaudePerplexityGoogle AIGrokCopilot
CFA® Level I · Quantitative Methods · Lesson 11 of 11

Introduction to big data and machine learning

What the methods do and when an analyst would use them.

Written by Sepehr Parsa7 min read

What is big data?

Big data means data sets too large or too complex for traditional tools. It is usually described by:

  • Volume: the quantity of data.
  • Velocity: how fast it arrives, sometimes in real time.
  • Variety: structured (tables), semi-structured and unstructured (text, images, audio).
  • Veracity: how reliable and accurate it is. Big data is often messy.

Sources of alternative data

Source Examples
Individuals Social media posts, product reviews, search trends
Business processes Credit card transactions, corporate data exhaust, supply chain records
Sensors and devices Satellite images of car parks, shipping and location data, internet-of-things devices

Analysts use these to spot trends before they appear in official figures, for example estimating a retailer’s sales from foot traffic. Using such data raises legal and ethical questions, such as privacy and whether data counts as material nonpublic information.

Machine learning

Machine learning (ML) finds patterns in data and uses them to make predictions, often with little human direction on what to look for.

Supervised learning

The training data has labelled outcomes, and the model learns to predict them.

  • Regression problems predict a number (next month’s return).
  • Classification problems predict a category (default or no default).
  • Examples of methods: penalized regression (which limits model complexity), support vector machines, k-nearest neighbours, classification and regression trees, random forests.

Unsupervised learning

The data has no labels. The model finds structure on its own.

  • Clustering: group similar observations, such as stocks that behave alike (k-means, hierarchical clustering).
  • Dimension reduction: summarize many correlated variables with a few factors (principal components analysis).

Deep learning and reinforcement learning

Neural networks with many layers (deep learning) handle complex, non-linear patterns such as images, speech and text. Reinforcement learning learns by trial and error to maximize a reward, as in game-playing programs.

Overfitting and how to avoid it

A model can learn the noise in its training data as well as the real pattern. It then looks excellent on that data and fails on new data: overfitting. The opposite, underfitting, is a model too simple to capture the pattern.

To manage this, split the data:

  1. Training set: fit the model.
  2. Validation set: tune and choose between models.
  3. Test set: measure final performance on data the model has never seen.

A big gap between training and test performance signals overfitting. Simpler models, fewer features and penalties on complexity help.

Limits for analysts

  • ML finds correlations, not causes. A pattern can disappear when conditions change.
  • Some models are black boxes: hard to explain to clients or regulators.
  • Results depend on data quality. Biased or incomplete data produces biased models.

Exam focus

Expect to classify a described method as supervised, unsupervised or deep learning, to recognize overfitting, to know the purpose of training, validation and test sets, and to name the characteristics of big data.

Check yourself

Tap an answer. You get the explanation straight away.

  1. A model that predicts whether a bond will default, trained on past bonds labelled 'defaulted' or 'did not default', is an example of:

  2. A model fits its training data almost perfectly but performs poorly on new data. It most likely suffers from:

  3. Grouping stocks into clusters by the similarity of their return patterns, with no predefined categories, is an example of:

Questions people ask

Is machine learning on the CFA exam mathematical?

At Level I the questions are conceptual: which method fits a problem, what overfitting is and how to reduce it. You will not be asked to train a model.

What is the difference between bias error and variance error?

Bias error comes from a model that is too simple and misses the pattern (underfitting). Variance error comes from a model so flexible it fits noise (overfitting).

What is alternative data?

Data outside traditional financial reports, such as satellite images, card transactions, web traffic or sensor data.

Unlimited practiceGenerate more Quantitative Methods questionsFresh questions with worked solutions every time, in the Level I style.
Progress is saved in this browser.
Topic finishedTake the Level I practice examTimed, in the real exam format, with an explanation for every answer.

CFA Institute does not endorse, promote or warrant the accuracy or quality of WallStPal. CFA® and Chartered Financial Analyst® are registered trademarks owned by CFA Institute.

Exam format, timing and topic weights are from CFA Institute's Level I, II and III exam pages, checked November 15, 2026. Check CFA Institute for the year you sit. Lessons and practice questions are WallStPal's own and are not CFA Institute material.

Report an error

Tell us what looks wrong on this page. We check every report against the original filing and fix mistakes on our corrections page.

Share this trade
XTelegram
My Desk · free

Sign in to follow

Follow stocks, members of Congress, famous investors and funds, and see what's new each time you come back.