From descriptive to predictive: Your first machine learning model

An overview of common low-hanging fruits to help you get started with machine learning
July 7, 2023

Machine learning promises a future of smarter, faster decisions and the ability to apply human creativity to higher-value activities. But in order to make the best use of machine learning, you must have two things:

  1. A specific, concrete business problem that can be solved using prediction, pattern recognition or automated decision-making
  2. A solid foundation of data infrastructure and mature data operations

It’s easy to get caught up in the hype and jump the gun, especially before mature data operations are in place. Before pursuing machine learning, you should have a regular cadence of reports, governed and secured data, and widespread data-driven decision making. In short, before pursuing machine learning projects that can be highly open-ended, complicated and risky, an organization should exhaust the plentiful potential of simply having data available where it’s needed.

Once that solid foundation is in place, though, what’s next? Here, we’ll introduce a mental model for identifying machine learning opportunities and discuss a number of practical examples.

How to think about machine learning opportunities

Machine learning fundamentally comes in the following flavors:

  • Unsupervised, in which a model uncovers and identifies patterns in order to either group entities together (cluster analysis) or to identify fundamental factors that affect phenomena (principal components analysis). 
  • Supervised, in which a model makes predictions about future outputs based on future inputs after training on a set of known inputs and outputs. Semi-supervised learning often attempts to solve the same problems as supervised learning but with less data, bootstrapping from a small initial training set.
  • Reinforcement learning, in which artificial agents are programmed to maximize rewards and learn through trial-and-error

Concretely, this means that any business problem you want to solve using machine learning should meet at least one of the following criteria:

  • It is a pattern recognition exercise, solved by programmatically uncovering patterns that have previously escaped notice or gaining deeper insight into known (or suspected) patterns.
  • It is a predictive exercise, using known factors about what has already happened to make a robust prediction about what will (or can) happen, or to classify things into known categories.
  • It involves using an artificial agent to bootstrap its learning and automatically make decisions. Applications include everything from self-driving vehicles and drones, chatbots and generative AI.

One additional consideration is the possibility to achieve pattern recognition, prediction and automation using heuristics, i.e. hand-coded rules based on experience. So before jumping into a potentially complicated modeling exercise, it is worth carefully examining whether heuristics can achieve those goals with sufficient precision and accuracy. In this sense, machine learning is more a matter of improving the precision and accuracy of certain capabilities than creating entirely new ones.

Practical machine learning use cases

Based on studies and the experiences of our customers, the following are some practical examples of predictive models to pursue and their subsequent ML flavors. Note that some of these are industry-specific and will depend on your organization’s goals, maturity and other characteristics (such as whether it is B2C or B2B).

Financial forecastingHeuristic methods for financial modeling like percent of sales, straight line and moving average are commonly used to predict financial trends, but linear regression has the potential for greater accuracy. This is a classic predictive use case and applicable to organizations of all kinds in all industries.

Personalization and recommendations for customers – In businesses with lots of small, repeated interactions, especially B2C, you can quickly build a recommendation model using approaches such as collaborative filtering. This is mainly a pattern recognition exercise, although as categories become better-known and understood, it can evolve into a predictive exercise too.

Customer and product segmentation – Constructing categories and archetypes for customers, products and other entities that your organization routinely interacts with can allow you to more effectively target your efforts. As with personalization and recommendation, you can approach this as raw pattern recognition or prediction.

Marketing and sales forecasting – How do specific customer events affect consumption? How do interactions with email marketing affect event attendance? There is even a natural language processing example – how do keywords impact click rates? In general, can you assign a probability of success to each transaction based on pricing and other known attributes? These are yet more instances of supervised learning that can help a team optimize its actions.

Supply chain, logistics and other operations optimization – With the proliferation of just-in-time manufacturing and inventorying, being able to forecast and predict needs and potential shortages are critical for maintaining leanness and responsiveness in industries such as manufacturing. This is especially important in the face of economic headwinds and shocks.

Anomaly detection – Detecting anomalies ranging from fraud, cyberattacks and other malicious activities to bugs, accidents and other product or user experience hiccups is important. Anomaly detection ranges from supervised, unsupervised or semi-supervised depending on the exact nature of the problem.

Business process automation – Beyond recommendations and predictions, you can also empower a machine learning model to execute decisions automatically. Need to quickly and automatically allocate ad spending? What about customer service chat bots to direct website visitors to the right place? What about QA on the assembly line? There are endless possibilities. Artificial agents are most commonly associated with reinforcement learning, although the use cases described above can also involve supervised learning.

Looking to the future

Between pattern recognition, prediction and automated decision making, there are countless opportunities to use machine learning to solve business problems. Machine learning and artificial intelligence continue to advance. Keep an eye out for technologies such as generative AI that offer the potential for new categories of general-purpose productivity multipliers.

None of this is especially relevant without a strong foundation of data science, though! As per our blog on how to prepare to use ML, every machine learning use case requires the ability to centralize large amounts of data and effectively use it. That’s something we know a little something about — so to learn more, schedule a demo or consider a trial.

Start for free

Join the thousands of companies using Fivetran to centralize and transform their data.

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
Data insights
Data insights

From descriptive to predictive: Your first machine learning model

From descriptive to predictive: Your first machine learning model

July 7, 2023
July 7, 2023
From descriptive to predictive: Your first machine learning model
An overview of common low-hanging fruits to help you get started with machine learning

Machine learning promises a future of smarter, faster decisions and the ability to apply human creativity to higher-value activities. But in order to make the best use of machine learning, you must have two things:

  1. A specific, concrete business problem that can be solved using prediction, pattern recognition or automated decision-making
  2. A solid foundation of data infrastructure and mature data operations

It’s easy to get caught up in the hype and jump the gun, especially before mature data operations are in place. Before pursuing machine learning, you should have a regular cadence of reports, governed and secured data, and widespread data-driven decision making. In short, before pursuing machine learning projects that can be highly open-ended, complicated and risky, an organization should exhaust the plentiful potential of simply having data available where it’s needed.

Once that solid foundation is in place, though, what’s next? Here, we’ll introduce a mental model for identifying machine learning opportunities and discuss a number of practical examples.

How to think about machine learning opportunities

Machine learning fundamentally comes in the following flavors:

  • Unsupervised, in which a model uncovers and identifies patterns in order to either group entities together (cluster analysis) or to identify fundamental factors that affect phenomena (principal components analysis). 
  • Supervised, in which a model makes predictions about future outputs based on future inputs after training on a set of known inputs and outputs. Semi-supervised learning often attempts to solve the same problems as supervised learning but with less data, bootstrapping from a small initial training set.
  • Reinforcement learning, in which artificial agents are programmed to maximize rewards and learn through trial-and-error

Concretely, this means that any business problem you want to solve using machine learning should meet at least one of the following criteria:

  • It is a pattern recognition exercise, solved by programmatically uncovering patterns that have previously escaped notice or gaining deeper insight into known (or suspected) patterns.
  • It is a predictive exercise, using known factors about what has already happened to make a robust prediction about what will (or can) happen, or to classify things into known categories.
  • It involves using an artificial agent to bootstrap its learning and automatically make decisions. Applications include everything from self-driving vehicles and drones, chatbots and generative AI.

One additional consideration is the possibility to achieve pattern recognition, prediction and automation using heuristics, i.e. hand-coded rules based on experience. So before jumping into a potentially complicated modeling exercise, it is worth carefully examining whether heuristics can achieve those goals with sufficient precision and accuracy. In this sense, machine learning is more a matter of improving the precision and accuracy of certain capabilities than creating entirely new ones.

Practical machine learning use cases

Based on studies and the experiences of our customers, the following are some practical examples of predictive models to pursue and their subsequent ML flavors. Note that some of these are industry-specific and will depend on your organization’s goals, maturity and other characteristics (such as whether it is B2C or B2B).

Financial forecastingHeuristic methods for financial modeling like percent of sales, straight line and moving average are commonly used to predict financial trends, but linear regression has the potential for greater accuracy. This is a classic predictive use case and applicable to organizations of all kinds in all industries.

Personalization and recommendations for customers – In businesses with lots of small, repeated interactions, especially B2C, you can quickly build a recommendation model using approaches such as collaborative filtering. This is mainly a pattern recognition exercise, although as categories become better-known and understood, it can evolve into a predictive exercise too.

Customer and product segmentation – Constructing categories and archetypes for customers, products and other entities that your organization routinely interacts with can allow you to more effectively target your efforts. As with personalization and recommendation, you can approach this as raw pattern recognition or prediction.

Marketing and sales forecasting – How do specific customer events affect consumption? How do interactions with email marketing affect event attendance? There is even a natural language processing example – how do keywords impact click rates? In general, can you assign a probability of success to each transaction based on pricing and other known attributes? These are yet more instances of supervised learning that can help a team optimize its actions.

Supply chain, logistics and other operations optimization – With the proliferation of just-in-time manufacturing and inventorying, being able to forecast and predict needs and potential shortages are critical for maintaining leanness and responsiveness in industries such as manufacturing. This is especially important in the face of economic headwinds and shocks.

Anomaly detection – Detecting anomalies ranging from fraud, cyberattacks and other malicious activities to bugs, accidents and other product or user experience hiccups is important. Anomaly detection ranges from supervised, unsupervised or semi-supervised depending on the exact nature of the problem.

Business process automation – Beyond recommendations and predictions, you can also empower a machine learning model to execute decisions automatically. Need to quickly and automatically allocate ad spending? What about customer service chat bots to direct website visitors to the right place? What about QA on the assembly line? There are endless possibilities. Artificial agents are most commonly associated with reinforcement learning, although the use cases described above can also involve supervised learning.

Looking to the future

Between pattern recognition, prediction and automated decision making, there are countless opportunities to use machine learning to solve business problems. Machine learning and artificial intelligence continue to advance. Keep an eye out for technologies such as generative AI that offer the potential for new categories of general-purpose productivity multipliers.

None of this is especially relevant without a strong foundation of data science, though! As per our blog on how to prepare to use ML, every machine learning use case requires the ability to centralize large amounts of data and effectively use it. That’s something we know a little something about — so to learn more, schedule a demo or consider a trial.

Related blog posts

When are you ready for artificial intelligence and machine learning?
Data insights

When are you ready for artificial intelligence and machine learning?

Read post
In-warehouse machine learning and the modern data stack
Data insights

In-warehouse machine learning and the modern data stack

Read post
Machine learning in one lesson
Data insights

Machine learning in one lesson

Read post
No items found.
Automating credit card fraud detection with Google BigQuery ML and Fivetran
Blog

Automating credit card fraud detection with Google BigQuery ML and Fivetran

Read post
Accelerate GenAI apps with Fivetran Google Cloud BQ and Vertex AI
Blog

Accelerate GenAI apps with Fivetran Google Cloud BQ and Vertex AI

Read post
Prompt engineering and the responsible use of generative AI
Blog

Prompt engineering and the responsible use of generative AI

Read post

Start for free

Join the thousands of companies using Fivetran to centralize and transform their data.

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.