Multi-Layer Perceptron (MLP): How It Works, With Code (2026)

An MLP is a type of ANN that consists of an input layer, one or more hidden layers, and an output layer. All the connections in MLP are fully connected. Every connection has a weight associated with it. Every neuron performs an activation function to make the model non-linear. Thus, MLP is able to learn non-linear mapping which is not possible for a perceptron with a single layer.

MLPs are a type of feedforward neural network. In the MLP, information flows only in one direction – input layer → hidden layer(s) → output layer. There are no feedback connections. Thus, MLP differs from other types of neural networks, like recurrent neural networks.

What Is a Multi-Layer Perceptron?

A Multi-Layer Perceptron is a feedforward artificial neural network composed of at least three layers of neurons; an input layer, one or more hidden layers, and an output layer.

Each layer serves a distinct function.

• Input Layer: The input layer consists of neurons representing one input feature. For example, a dataset with 10 numeric features requires 10 input neurons.
• Hidden layer(s): Each hidden layer processes the inputs passed to it from the previous layer through weighted sum, bias, and activation functions. MLPs can have one or multiple hidden layers. Practitioners call neural networks with multiple hidden layers deep neural networks. 
• Output Layer: This layer produces the final prediction. For binary classification, the output layer needs one neuron; for 10-class classification, like digit recognition, it needs 10 output neurons. 

Every neuron in one layer connects to every neuron in the next layer. This is why an MLP is also called a fully connected network or a dense network.

A Brief History of the Multi-Layer Perceptron

The Multi-Layer Perceptron developed through five documented milestones between 1943 and 1989.

Year	Contributor(s)	Contribution
1943	Warren McCulloch, Walter Pitts.	Published the first mathematical model of an artificial neuron, capable of producing a binary output from weighted binary inputs.
1958	Frank Rosenblatt	Introduced the Perceptron, a single-layer neural network that could learn its own weights from data, published in Psychological Review.
1969	Marvin Minsky, Seymour Papert	Published Perceptrons, proving that a single-layer perceptron cannot represent the XOR function, since XOR is not linearly separable.
1986	David Rumelhart, Geoffrey Hinton, Ronald Williams	Published “Learning representations by back-propagating errors” in Nature, demonstrating an effective method for training multi-layer networks.
1989	George Cybenko; Kurt Hornik, Maxwell Stinchcombe, Halbert White	Independently proved the Universal Approximation Theorem: a feedforward network with one hidden layer and a sigmoid activation function can approximate any continuous function to any desired precision.

The discovery made by Minsky and Papert in 1969 is the fundamental reason for the existence of MLP. The single perceptron creates only one linear decision boundary. The XOR function requires two decision boundaries, something that the single-layer neural net can’t do.

How a Multi-Layer Perceptron Works

An MLP processes data through four sequential mechanisms: forward propagation, loss calculation, backpropagation, and optimization.

Forward Propagation

Forward propagation transfers the input dataset through the network’s layers to produce the output. For each neuron, the network executes the following operations.

1. Weighted sum. Each input gets multiplied by its weight, and the result gets summed up along with the bias term of the neuron:

z = Σ(wᵢxᵢ) + b

Here, xᵢ is an input feature, wᵢ is its weight, and b is the bias.

2. Activation function. The neuron’s weighted sum passes through the activation function which makes the neural network nonlinear. Some activation functions include:

• Sigmoid — maps any real number to a value between 0 and 1.
• ReLU (Rectified Linear Unit) — gives the input value when it is positive; gives 0 otherwise: f(z) = max(0, z).
• Tanh (Hyperbolic Tangent) — maps any real number to a value between −1 and 1.
• Softmax — converts a vector of raw scores into a probability distribution across multiple output classes.

Without an activation function, an MLP with any number of layers reduces to a single linear transformation, mathematically equivalent to a one-layer network.

Loss Functions

The loss function measures how different the model’s output is from the target value.  The problem itself determines which loss function you use. 

• For binary classification problems, we use the Binary Cross-Entropy Loss Function.
• For multi-class classification problems, we use Categorical Cross-Entropy Loss Function.
• For regression problems, we use Mean Squared Error (MSE).

Backpropagation

Backpropagation is a process which determines how much each weight in the network is responsible for the overall loss, using the chain rule in calculus. There are three stages in the process after each forward pass:

1. The network computes the gradient of the loss function with respect to the weights of the output layer.
2. Backpropagation pushes the error through each hidden layer, computing gradients for all weights and biases in the network. 
3. The optimizer adjusts each weight to minimize the loss. 

Rumelhart, Hinton, and Williams formalized this method in 1986, and it remains the standard training algorithm for feedforward neural networks as of 2026.

Optimization

The optimizer makes decisions about how to change the weights in proportion to their gradients. The two most widely used optimizers in MLP training are:

• Stochastic Gradient Descent (SGD) modifies the weights on the basis of one or a small number of training samples (a batch): w = w − η · ∂L/∂w, where η is the learning rate.
• Adam Optimizer builds upon the Stochastic Gradient Descent approach by using momentum (moving averages of gradients) and an individual learning rate per weight. Adam Optimizer is currently set by default in TensorFlow and PyTorch.

MLP vs. Perceptron vs. CNN vs. RNN

Four architectures are commonly compared to the MLP. Each is suited to a different type of input data.

Architecture	Best Suited For	Key Structural Difference	Still in Common Use (2026)
Perceptron	Linearly separable binary classification	Single layer, no hidden layer	Rarely, used mainly for teaching
Multi-Layer Perceptron (MLP)	Structured, tabular data	Fully connected hidden layers	Yes, for tabular and baseline tasks
Convolutional Neural Network (CNN)	Images, spatial data	Convolutional filters that share weights across spatial regions	Yes, standard for computer vision
Recurrent Neural Network (RNN)	Sequential data, time series	Connections that loop, passing information across time steps	Limited, largely replaced by Transformer architectures for long sequences

An MLP treats every input feature independently, with no assumption about spatial or sequential relationships between features. This makes an MLP inefficient for image data, where CNNs exploit spatial locality, and for sequential data, where Transformer-based architectures now dominate.

When to Use a Multi-Layer Perceptron

An MLP is the appropriate architecture choice under three conditions.

1. The input data is structured or tabular, such as a spreadsheet of customer, sensor, or transaction information. 
2. A non-linear relationship exists between the inputs and outputs. This means the task needs a more complex structure, like an MLP — simple linear or logistic regression won’t do. 
3. The data set has no clear spatial structure (like an image) or sequential structure (like text). In these cases, CNN or Transformer architectures perform much better. 

An MLP is not the recommended architecture for raw image classification, natural language processing, or long sequential data. CNN, RNN, and Transformer architectures outperform it on accuracy and computational efficiency in these cases. 

Advantages of Multi-Layer Perceptrons

• Non-linearity. Activation functions enable the MLP to learn non-linear dependencies between the input data and output data.
• Flexibility. You can use MLPs for classification and regression tasks without changing their architecture. 
• Efficiency of parallel computing. GPUs execute the matrix operations MLPs perform efficiently, which speeds up training. 
• Universal approximation. The Universal Approximation Theorem, stated in 1989, shows that an MLP with one hidden layer and enough neurons can approximate any continuous function. 

Limitations of Multi-Layer Perceptrons

• Expensive computationally. More layers, more neurons, and more examples in training cause increased training time.
• Prone to overfitting. Without regularization, an MLP would simply learn how to perfectly memorize the training set rather than finding patterns in it.
• Sensitivity to feature scaling. You have to normalize or standardize all input features. 
• Problem of vanishing gradient. Gradients computed during backpropagation in deep networks exponentially decay when moving from top to bottom layers. It was first described by Sepp Hochreiter in his 1991 doctoral thesis. It causes learning slowdown in earlier layers of the deep network.

How to Build a Multi-Layer Perceptron in Python

The following steps build and train an MLP using TensorFlow and the MNIST handwritten digit dataset, which contains 70,000 grayscale images across 10 digit classes.

Step 1: Import Libraries and Load the Dataset

import tensorflow as tf

import numpy as np

import matplotlib.pyplot as plt

(x_train, y_train), (x_test, y_test) = tf.keras.datasets.mnist.load_data()

Step 2: Normalize the Data

Pixel values range from 0 to 255. Dividing by 255 scales every value to a range between 0 and 1, which speeds up convergence during training.

x_train = x_train / 255.0

x_test = x_test / 255.0

Step 3: Build the Model

model = tf.keras.models.Sequential([

    tf.keras.layers.Flatten(input_shape=(28, 28)),

    tf.keras.layers.Dense(256, activation='relu'),

    tf.keras.layers.Dense(128, activation='relu'),

    tf.keras.layers.Dense(10, activation='softmax')

])

The Flatten layer converts each 28×28 pixel image into a one-dimensional array of 784 values. The two Dense layers are the hidden layers, with 256 and 128 neurons respectively. The final Dense layer is the output layer, with 10 neurons representing the 10 digit classes.

Step 4: Compile the Model

model.compile(

    optimizer='adam',

    loss='sparse_categorical_crossentropy',

    metrics=['accuracy']

)

Step 5: Train the Model

history = model.fit(

    x_train, y_train,

    epochs=10,

    batch_size=2000,

    validation_split=0.2

)

Step 6: Evaluate the Model

test_loss, test_accuracy = model.evaluate(x_test, y_test)

print(f"Test accuracy: {test_accuracy}")

A correctly trained MLP on the MNIST dataset reaches a test accuracy above 92% within 10 training epochs, using the architecture above.

Real-World Example: Multi-Layer Perceptron for Sentiment Analysis

The firm in the hospitality industry obtains guest reviews in written form and has to classify them into either positive or negative ones. This is a case of a binary text classification problem.

The process requires three stages.

1. Text vectorization. Pure text cannot be inputted to neural networks. A technique such as Term Frequency-Inverse Document Frequency (TF-IDF) transforms the text into vector form. Each word’s weight depends on its frequency in that document relative to the whole data set. 
2. Training of the model. The vectors obtained using the TF-IDF approach and the positive/negative labels of the reviews train the MLP. It learns what combinations of words correspond to positive and negative sentiments.
3. Prediction. The trained MLP assigns new, unlabeled reviews into classes “positive” or “negative”.

Independent testing on this type of task shows a single-layer Perceptron model reaching approximately 67% mean accuracy. Increasing hidden-layer neuron count in an MLP from 2 to 5 neurons per layer, while keeping 3 hidden layers, produces a measurable accuracy improvement over the low-capacity configuration. 

Frequently Asked Questions