Introduction to Optimization

Get insight into the concepts of optimization and how they work.

We'll cover the following

What is optimization?

Optimization is a powerful problem-solving approach used in various fields, such as engineering, economics, mathematics, computer science, and many others, to find the best solution for a given problem. It is the process of finding the best solution, also known as the optimal solution, that maximizes or minimizes a given objective function subject to constraints or limitations.

The objective function is the measure of the effectiveness or efficiency of a solution, while the constraints are the conditions or limitations that the solution must satisfy. The optimal solution is the one that satisfies all the constraints and provides the highest or lowest value for the objective function.

Optimization problems can be classified into two categories:

  • Continuous optimization deals with problems where the variables can take any value within a given range.

  • Discrete optimization deals with problems where the variables can only take discrete values within the range

Process of optimization

The steps in the process of optimization generally include the following:

  1. Defining the objective function: The first step in optimization is to define the objective function that needs to be optimized. This function represents the problem that needs to be solved.

  2. Formulating the problem: The next step is to formulate the problem by defining the constraints that need to be satisfied. These constraints can be in the form of equations, inequalities, or other conditions.

  3. Choosing an optimization algorithm: After defining the problem, the next step is to choose an optimization algorithm that can efficiently solve the problem. There are many optimization algorithms available, and the choice of algorithm depends on the nature of the problem.

  4. Solving the problem: After choosing the optimization algorithm, the problem is solved by running the algorithm to find the solution that optimizes the objective function.

  5. Evaluating the solution: After obtaining the solution, it is important to evaluate its quality and verify whether it satisfies the constraints of the problem.

  6. Fine-tuning the solution: If the solution is not satisfactory, the problem formulation or the optimization algorithm might need to be revised and the process of optimization repeated until a satisfactory solution is obtained.

  7. Implementing the solution: Once a satisfactory solution is obtained, it is implemented in practice to achieve the desired outcome.

 This is an iterative process, as shown below:

Press + to interact
A schematic of the optimization process
A schematic of the optimization process

Methods of optimization

There are several optimization algorithms, such as:

  • Linear programming: It is a popular optimization algorithm used to solve problems where the objective function and constraints are linear.

  • Nonlinear programming: It generally deals with problems where the objective function and constraints are nonlinear.

  • Dynamic programming: It is used to solve problems where the solution depends on a sequence of decisions.

  • Stochastic programming: It is used to solve problems where the variables are subject to uncertainty or randomness.

These algorithms use different mathematical techniques and computational methods to find the optimal solution.

Example: Production schedule optimization

One example of optimization in real life using Python is optimizing the production schedule of a manufacturing plant. Suppose that a manufacturing plant produces several types of products using a set of machines. The objective is to maximize the production output while maximizing the profits, subject to various constraints such as machine capacities and demand.

Press + to interact

Let’s consider the following example:
Suppose you work in a manufacturing plant that has three products: A, B, and C. The manufacturing plant has two machines where machine M1 can produce products A and B, and machine M2 can produce products B and C.

You are tasked with devises a production plan for the manufacturing plant that maximizes the profit. More details about the problem are given below.

Decision variables:

  • x1x_1: Quantity of product A to be produced.

  • x2x_2: Quantity of product B to be produced.

  • x3x_3: Quantity of product C to be produced.

Objective: Maximize the total profit where:

  • Profit on product A is 8.

  • Profit on product B is 10.

  • Profit on product C is 12.

So the objective function is given as:

Constraints:

  • Machine 1 can produce up to 100 units per day.

  • Machine 2 can produce up to 150 units per day.

  • Demand for product C is at least 20 units per day.

  • Production for product B cannot be more than 50 per day.

  • For all products, the quantities cannot be negative.

Now, using these equations, this problem can be solved using Python. To solve this in Python, a couple of libraries are available. One such library is the scipy.optimize module, which provides a collection of optimization algorithms for solving various types of optimization problems.

Press + to interact
import numpy as np
from scipy.optimize import linprog
# Define the objective function coefficients (profits)
c = np.array([-8, -10, -12])  # Negative sign to maximize the objective
# Define the constraint coefficients
A = np.array([[1, 1, 0], [0, 1, 1]])  # Coefficients for the machine capacity constraints
b = np.array([100, 150])  # Right-hand side values for the machine capacity constraints
Aeq = np.array([[0, 0, 1],[0,1,0]])  # Coefficients for the demand constraint
beq = np.array([20,50])  # Right-hand side value for the demand constraint
# Define the variable bounds
bounds = [(0, None), (0, None), (0, None)]  # Non-negativity constraints
# Solve the linear programming problem
result = linprog(c, A_ub=A, b_ub=b, A_eq=Aeq, b_eq=beq, bounds=bounds, method='highs')
# Print the solution
if result.success:
    print(f"The number of units produced for product 1 are {int(result.x[0])}")
    print(f"The number of units produced for product 2 are {int(result.x[1])}")
    print(f"The number of units produced for product 3 are {int(result.x[2])}")
    print(f"The total profits are {int(-result.fun)}")
else:
    print('Optimization failed. Constraints may not be feasible.')

In this code:

  • Lines 1–2: We import numpy for mathematical operations and linprog from scipy.optimize for linear programming optimization.

  • Lines 5–11: We define coefficients of the objective function and constraints 1–4 in the form of numpy arrays.

  • Line 14: We define the variable bounds to enforce nonnegativity constraints.

  • Line 17: We call the solver method from the scipy.optimize library.

  • Lines 19–23: We print the results if optimization is solved, and we write line 26 to print the results if optimization fails.

Applications of optimization

Optimization has numerous practical applications, such as production planning, resource allocation, logistics, finance, scheduling, and many others. For example, in production planning, optimization can be used to determine the optimal amount of resources to allocate to different production activities to maximize efficiency and minimize costs. In finance, optimization can be used to design investment portfolios that provide the highest return for a given level of risk.

Optimization is a powerful tool for decision-making and problem-solving. It provides a systematic approach to finding the best solution to a given problem based on mathematical modeling and rigorous analysis. The use of optimization has led to significant improvements in efficiency, productivity, and profitability in various fields.

However, optimization also has its limitations. The models used in optimization are simplifications of real-world problems, and the assumptions made in these models might not always hold true in practice. Moreover, optimization algorithms can be computationally expensive and can require substantial computational resources.