With scikit-learn, you can generate test datasets in Python

 

Machine learning algorithms or test harnesses can be tested using test datasets, which are small, created datasets.

It is possible to explore specific algorithm behaviour using test datasets because the data have well-defined properties, such as linearity or non-linearity. In addition to generating samples from configurable regression and classification problems, scikit-learn provides a suite of functions for generating samples from the Python language.

We will explore test problems and how to use them in Python with scikit-learn in this tutorial.

You will learn the following after completing this tutorial:

• Predictive tests for multi-class classifications: how to generate them.
• What is the best way to generate binary classification prediction test problems?
• The process of generating linear regression prediction problems.

Test Datasets

How do you know if a machine learning algorithm is correctly implemented when developing and implementing it? It appears that they work even when there are bugs.

In testing and debugging algorithms and test harnesses, test datasets are small, contrived problems. Also, they can be used to better understand algorithm behavior when hyperparameters are changed.

Test datasets should have the following desirable properties:

• The process of generating them is simple and quick.
• The outcomes are known or understood so that they can be compared with predictions.
• As stochastic processes, they allow randomly varying solutions to the same problem every time.
• Their size and two-dimensionality make them easy to visualize.
• Scaling them up is trivial.

For beginners and for those developing new test harnesses, I recommend using test datasets.

A suite of test problems can be generated using scikit-learn, a Python library for machine learning.

Here we will discuss how to generate test problems for classification and regression algorithms.

Classification Test Problems

Assigning labels to observations is the problem of classification.

Three classification problems will be discussed here: blobs, moons, and circles.

Blobs Classification Problem

You can generate Gaussian blobs of points using the make_blobs() function.

A number of properties can be controlled, including how many blobs will be generated and how many samples will be generated.

Given the linearly separable nature of the blobs, the problem lends itself to linear classification problems.

As a multi-class classification prediction problem, the following example generates a 2D dataset of samples with three blobs. There are two inputs and one, two, or three class values for each observation.

Please see the following example for a complete explanation.

A handy 2D plot shows points for the different classes using different colors after running the example generates the inputs and outputs.

Because the problem generator is stochastic, your specific dataset and plot will vary. Not a bug, but a feature.

Following are examples using this same example structure.

Moons Classification Problem

The make_moons() function generates a swirl pattern, or two moons, for binary classification.

You can control how many samples to generate and how noisy the moon shapes should be.

Algorithms capable of learning nonlinear class boundaries will be able to solve this test problem.

Below is an example of generating a moon dataset with moderate noise.

The complete example is listed below.

Upon running the example, the dataset is generated and plotted for review, with samples again colored according to their class.

Circles Classification Problem

The make_circles() function generates a problem in which datasets are grouped into concentric circles.

The shapes can also be controlled, as with the moon test problem.

A complex nonlinear manifold can be learned by algorithms that are capable of solving this test problem.

Below is an example of generating a noise-filled circle dataset.

Please see the following example for a complete explanation.

For review, the dataset is generated and plotted by running the example.

Regression Test Problems

An observation is used to predict a quantity using regression.

The make_regression() function will generate a dataset with inputs and outputs that follow a linear relationship.

Many configuration options are available, including the number of samples, the number of input features, and the level of noise.

A linear regression function can be learned from this dataset.

A modest amount of noise will be generated in the output feature and one input feature in the example below.

Please see the following example for a complete explanation.

By running the example, the X and Y relationship will be generated and plotted, which is quite boring, since it is linear.

Summary

The purpose of this tutorial was to introduce you to test problems in Python and how to use them in scikit-learn.

Your specific learnings were:

• Predictive tests for multi-class classifications: how to generate them.
• A method for generating binary classification prediction test problems.
• The process of generating linear regression prediction problems.

Are there any questions you would like to ask?

If you have any questions, please leave them in the comments below, and I will do my best to answer them.

Take advantage of my referral link today and become a medium member. For just $5 a month, you will have access to everything Medium has to offer. By becoming a member, I will receive $2 from $5, which will assist me in maintaining this blog.