Documentation

`gatf.algorithms`

Implementations of NSGA3, muPlusLambda, and Particle Swarm optimizers.

class gatf.algorithms.NSGA3(obj_func, n_objectives, n_variables, population_size=92, tournament_size=2, upperbound=1.0, lowerbound=-1.0, generations=1000, mutation_prob=0.1, crossover_prob=0.7, mutation_eta=1.0, crossover_eta=1.0, record_gens=10, verbose=True, seed=None)

Bases: object

The NSGA3 algorithm developmed by Deb and Jain 2014 1. Implemented using Tensorflow for use with objective functions created with Tensorflow (ie. Gaussian Processes). Uses tournament selection to choose individuals to apply simulated binary crossover and polynomial mutation.

This algorithm is a minimizer. Objectives to be maximized should be returned from the objective function as the negative (ie. \(-f(X)\)).

To promote diversity and avoid the non-dominated front being filled by duplicate solutions, duplicates are removed at each generation and only unique solutions are retained for the next generation.

A couple metrics are calculated by this algorithm, including mutual domination rate (MDR) and the % of filled reference directions. These can be used to assess the success of the optimization. However, no early stopping for convergence is implemented.

Parameters

obj_func (function Y = f(X)) – Objective function that takes only a tensor X of shape (None,n_variables) and returns a tensor of fitness values of shape (None,n_objectives) and with dtype=tf.float64.
n_objectives (int) – The number of objectives to optimize. Must be >1.
n_variables (int) – The number of variables in X.
population_size (int, optional) – The size of the population to generate. May not be exact. The default is 92.
tournament_size (int, optional) – Tournament selection is used to choose individuals to perform crossover with. Must be between [2, population_size]. The default is 2.
upperbound (float, list/tuple of floats, optional) – Upper bounds for X. The default is 1.0.
lowerbound (float, list/tuple of floats, optional) – Lower bounds for X. The default is -1.0.
generations (int, optional) – Number of generations to evolve. The default is 1000.
mutation_prob (float, optional) – The probability of mutating an element in an individual solution. Must be between [0,1]. The default is 0.1.
crossover_prob (float, optional) – The probability of performing crossover on an element in an individual solution. Must be between [0,1]. The default is 0.7.
mutation_eta (float, optional) – Controls the degree of mutation. Higher values generate children that are more similar to their parents. The default is 1.0.
crossover_eta (float, optional) – Controls the degree of crossover. Higher values generate children that are more similar to their parents. The default is 1.0.
record_gens (int, optional) – How often to record statistics and calculate convergence metrics. The default is 10 generations.
verbose (bool) – Whether to print progress to screen. The default is True.
seed (int, optional) – Random seed. The default is None.

Returns

Optimization results as dict-like key/value pairs including ‘X’, ‘fitness’, and ‘statistics’. The statistics key contains a population.Statistics object with the keys ‘gen’, ‘mdr’, and ‘refs’.

Return type

gatf.population.Population

Notes

The input signatures are defined for Tensorflow functions to avoid graph retracing and, therefore, this algorithm expects a tf.float64 datatype. Objective functions should be designed to output the proper datatype or an error with be thrown. Custom objectives should also make use of the @tf.function decorator to allow compilation into the Tensorflow graph.

>>> @tf.function(input_signature=(tf.TensorSpec(shape=[None,n_objectives], dtype=tf.float64),))
>>> def my_custom_objective(X):
>>>    # calculate fitness here
>>>    return fitness

Providing an input signature is not absolutely necessary but recommended if a warning is raised that the graph is being retraced.

References

1: Deb, Kalyanmoy, and Himanshu Jain. “An Evolutionary Many-Objective Optimization Algorithm Using Reference-Point-Based Nondominated Sorting Approach, Part I: Solving Problems With Box Constraints.” IEEE Transactions on Evolutionary Computation 18, no. 4 (August 2014): 577–601. https://doi.org/10.1109/TEVC.2013.2281535.

Examples

The NSGA3 algorithm is first instantiated with the desired objective and hyperparameters, then it is called to begin the optimization.

>>> # Initialize NSGA3
>>> optimizer = NSGA3(my_custom_objective, n_objectives, n_variables, ...)
>>> # Run the optimizer
>>> results = optimizer()

The returned results object contains key-accessible values (like a dict).

>>> # Get the final population
>>> results['X']
<tf.Tensor: shape=(population_size,n_variables), dtype=float64, numpy=...>
>>> # Get the associated fitnesses
>>> results['fitness']
<tf.Tensor: shape=(population_size,n_objectives), dtype=float64, numpy=...>

Additionally, statistics are recorded throughout the optimization. The statistics object contains lists of values that are either numbers or tensors.

>>> # Get the generations that were recorded
>>> results['statistics']['gen']
[0,10,20,...]
>>> # Get the statistic recorded associated with each generation
>>> results['statistics']['mdr']
[<tf.Tensor: shape=(), dtype=float64, numpy=...>, <tf.Tensor: shape=(), dtype=float64, numpy=...>, ...]

These can be made into a single numpy array for plotting

>>> np.array(results['statistics']['mdr'])
array([...])

class gatf.algorithms.PSO(obj_func, n_variables, population_size=15, upperbound=1.0, lowerbound=-1.0, generations=1000, inertia=0.8, cognitive_coeff=0.1, social_coeff=0.1, func_tolerance=0.0, position_tolerance=1e-08, record_gens=10, verbose=True, seed=None)

Bases: object

Particle swarm optimization. Moves particles with a velocity calculated by that particle’s inertia, best ever position, and the global best particle’s position.

This algorithm is a minimizer. Objectives to be maximized should be returned from the objective function as the negative (\(-f(X)\)).

This algoritm may break if the positions (X) converge or the objective function values converge. This can be set using the position_tolerance and func_tolerance arguments. Convergence will only be checked every record_gens generations.

Parameters

obj_func (function Y = f(X)) – Objective function that takes only a tensor X of shape (None,n_variables) and returns a tensor of fitness values of shape (None,1) with dtype tf.float64.
n_variables (int) – The number of variables in X.
population_size (int, optional) – The size of the population to generate. The default is 15.
upperbound (float, list, tuple, optional) – Upper bounds for X. The default is 1.0.
lowerbound (float, list, tuple, optional) – Lower bounds for X. The default is -1.0.
generations (int, optional) – Number of generations to evolve. The default is 1000.
inertia (float, optional) – Inertia of particles. Higher inertia makes it harder for particles to change directions. If this value is too high the optimizer may never converge. The default is 0.8.
cognitive_coeff (float) – The influence of a particle’s best ever position on its velocity. The default is 0.1.
social_coeff (float) – The influence of the global best particles position on a particle’s velocity. The default is 0.1.
func_tolerance (float, optional) – If the absolute difference between the minimum and maximum fitness is < func_tolerance, the algorithm will break. The default is 0.0.
position_tolerance (float, optional) – If the absolute difference between the minimum and maximum fitness is < func_tolerance, the algorithm will break. The default is 1e-8.
record_gens (int) – How often to record statistics and check for convergence. The default is 10 generations.
verbose (bool) – Whether to print progress to screen. The default is True.
seed (int, None) – Random seed. The default is None.

Returns

Optimization results as dict-like key/value pairs including ‘X’, ‘fitness’, and ‘statistics’. Also includes the best-ever individual ‘X_best’ and its associated fitness ‘fitness_best’. The statistics key contains a population.Statistics object with the keys ‘gen’, ‘X_min’, ‘X_mean’, and ‘X_max’.

Return type

gatf.population.Population

Notes

The input signatures are defined for Tensorflow functions to avoid graph retracing and, therefore, this algorithm expects a tf.float64 datatype. Objective functions should be designed to output the proper datatype or an error with be thrown. Custom objectives should also make use of the @tf.function decorator to allow compilation into the Tensorflow graph.

>>> @tf.function(input_signature=(tf.TensorSpec(shape=[None,n_objectives], dtype=tf.float64),))
>>> def my_custom_objective(X):
>>>    # calculate fitness here
>>>    return fitness

Providing an input signature is not absolutely necessary but recommended if a warning is raised that the graph is being retraced.

Examples

The PSO algorithm is first instantiated with the desired objective and hyperparameters, then it is called to begin the optimization.

>>> # Initialize PSO
>>> optimizer = PSO(my_custom_objective, n_variables, ...)
>>> # Run the optimizer
>>> results = optimizer()

The returned results object contains key-accessible values (like a dict).

>>> # Get the final best solution
>>> results['X_best']
<tf.Tensor: shape=(1,n_variables), dtype=float64, numpy=...>
>>> # Get the associated fitnesses
>>> results['fitness_best']
<tf.Tensor: shape=(1,1), dtype=float64, numpy=...>

Additionally, statistics are recorded throughout the optimization. The statistics object contains lists of values that are either numbers or tensors.

>>> # Get the generations that were recorded
>>> results['statistics']['gen']
[0,10,20,...]
>>> # Get the statistic recorded associated with each generation
>>> results['statistics']['X_mean']
[<tf.Tensor: shape=(), dtype=float64, numpy=...>, <tf.Tensor: shape=(), dtype=float64, numpy=...>, ...]

These can be made into a single numpy array for plotting

>>> np.array(results['statistics']['X_mean'])
array([...])

class gatf.algorithms.muPlusLambda(obj_func, n_variables, population_size=15, tournament_size=2, upperbound=1.0, lowerbound=-1.0, generations=1000, mutation_prob=0.3, crossover_prob=0.7, mutation_eta=1.0, crossover_eta=1.0, func_tolerance=0.0, position_tolerance=1e-08, record_gens=10, verbose=True, seed=None)

Bases: object

The muPlusLambda algorithm uses tournament selection to choose individuals for mating. Simulated binary crossover is applied followed by polynomial mutation. The population carried on to the next generation is chosen from the resultant children and the parents.

This algorithm is a minimizer. Objectives to be maximized should be returned from the objective function as the negative (\(-f(X)\)).

This algoritm may break if the positions (X) converge or the objective function values converge. This can be set using the position_tolerance and func_tolerance arguments. Convergence will only be checked every record_gens generations.

Parameters

obj_func (function Y = f(X)) – Objective function that takes only a tensor X of shape (None,n_variables) and returns a tensor of fitness values of shape (None,1).
n_variables (int) – The number of variables in X.
population_size (int, optional) – The size of the population to generate. May not be exact. The default is 15.
tournament_size (int, optional) – Tournament selection is used to choose individuals to perform crossover with. Must be >2 and <population_size. The default is 2.
upperbound (float, list, tuple, optional) – Upper bounds for X. The default is 1.0.
lowerbound (float, list, tuple, optional) – Lower bounds for X. The default is -1.0.
generations (int, optional) – Number of generations to evolve. The default is 1000.
mutation_prob (float, optional) – The probability of mutating an element in an individual solution. The default is 0.3.
crossover_prob (float, optional) – The probability of performing crossover on an element in an individual solution. The default is 0.7.
mutation_eta (float, optional) – Controls the degree of mutation. Higher values generate children that are more similar to their parents. The default is 1.0.
crossover_eta (float, optional) – Controls the degree of crossover. Higher values generate children that are more similar to their parents. The default is 1.0.
func_tolerance (float, optional) – If the absolute difference between the minimum and maximum fitness is < func_tolerance, the algorithm will break. The default is 0.0.
position_tolerance (float, optional) – If the absolute difference between the minimum and maximum fitness is < func_tolerance, the algorithm will break. The default is 1e-8.
record_gens (int, optional) – How often to record statistics and check for convergence. The default is 10 generations.
verbose (bool) – Whether to print progress to screen. The default is True.
seed (int, optional) – Random seed. The default is None.

Returns

Optimization results as dict-like key/value pairs including ‘X’, ‘fitness’, and ‘statistics’. Also includes the best-ever individual ‘X_best’ and its associated fitness ‘fitness_best’. The statistics key contains a population.Statistics object with the keys ‘gen’, ‘X_min’, ‘X_mean’, and ‘X_max’.

Return type

gatf.population.Population

Notes

The input signatures are defined for Tensorflow functions to avoid graph retracing and, therefore, this algorithm expects a tf.float64 datatype. Objective functions should be designed to output the proper datatype or an error with be thrown. Custom objectives should also make use of the @tf.function decorator to allow compilation into the Tensorflow graph.

>>> @tf.function(input_signature=(tf.TensorSpec(shape=[None,n_objectives], dtype=tf.float64),))
>>> def my_custom_objective(X):
>>>    # calculate fitness here
>>>    return fitness

Providing an input signature is not absolutely necessary but recommended if a warning is raised that the graph is being retraced.

Examples

The muPlusLambda algorithm is first instantiated with the desired objective and hyperparameters, then it is called to begin the optimization.

>>> # Initialize muPlusLambda
>>> optimizer = muPlusLambda(my_custom_objective, n_variables, ...)
>>> # Run the optimizer
>>> results = optimizer()

The returned results object contains key-accessible values (like a dict).

>>> # Get the final best solution
>>> results['X_best']
<tf.Tensor: shape=(1,n_variables), dtype=float64, numpy=...>
>>> # Get the associated fitnesses
>>> results['fitness_best']
<tf.Tensor: shape=(1,1), dtype=float64, numpy=...>

Additionally, statistics are recorded throughout the optimization. The statistics object contains lists of values that are either numbers or tensors.

>>> # Get the generations that were recorded
>>> results['statistics']['gen']
[0,10,20,...]
>>> # Get the statistic recorded associated with each generation
>>> results['statistics']['X_mean']
[<tf.Tensor: shape=(), dtype=float64, numpy=...>, <tf.Tensor: shape=(), dtype=float64, numpy=...>, ...]

These can be made into a single numpy array for plotting

>>> np.array(results['statistics']['X_mean'])
array([...])

`gatf.operators`

Crossover and mutation operations applied to a population to generate diversity.

class gatf.operators.PolynomialMutation(lowerbound, upperbound, prob_per_var=None, eta=1.0)

Bases: object

Uses polynomial mutation to randomly mutate individuals. Adapted from the Pymoo library 3.

Parameters

lowerbound (1D tensor) – Lower bounds for X.
upperbound (1D tensor) – Upper bounds for X.
prob_per_var (float, optional) – The probability that a variable is mutated. If None, the probability is 1 / n_var. The default is None.
eta (float, optional) – Controls the degree of mutation. Higher values generate children that are more similar to their parents. The default is 1.0.

Return type

Tensor with the same shape and type as X.

References

3: https://github.com/anyoptimization/pymoo/blob/master/pymoo/operators/mutation/pm.py

class gatf.operators.SimulatedBinaryCx(lowerbound, upperbound, prob_per_var=0.7, eta=1.0)

Bases: object

Uses simulated binary crossover to create child solutions from parents. Generates 2 * int(n_individuals / 2) children which is equal to n_individuals only when it is even. Adapted from the Pymoo library 4

Parameters

lowerbound (1D tensor) – Lower bounds for X.
upperbound (1D tensor) – Upper bounds for X.
prob_per_var (float, optional) – The probability of performing crossover on an element in an individual solution. The default is 0.7.
eta (float, optional) – Controls the degree of crossover. Higher values generate children that are more similar to their parents. The default is 1.0.

Return type

Tensor with the same shape and type as X.

References

4: https://github.com/anyoptimization/pymoo/blob/master/pymoo/operators/crossover/sbx.py

`gatf.population`

Population and statistics objects and methods to randomly initialize populations.

class gatf.population.Population(initial_population)

Bases: MutableMapping

Dict-like object containing keys ‘X’, ‘fitness’, and ‘statistics’ and, optionally, ‘X_best’, and ‘fitness_best’. Setting Population[‘X’] = value also sets Population[‘fitness’] = None. The ‘statistics’ key is a population.Statistics object that also has a dict-like structure. No other keys can be added and ‘statistics’ can be accessed but the object cannot be replaced by a new value.

Notes

Setting Population.X = value does not set ‘fitness’ to None. Therefore, only set X using Population[‘X’] = value

class gatf.population.Statistics(*args)

Bases: MutableMapping

For storing population statistics per generation. Dict-like object where Statistics[key] = value appends value to a list or creates a new key and places value in a list.

return_last_n(key, n)

Fetches the last n values from entry key.

keystr: The key of the values to return.
nint: Number of values to return

Returns: A list contain n values from key. If key is not valid, returns None.
Return type: list

gatf.population.initialize_normal_population(population_size, lowerbound, upperbound, mean=None, stddev=None)

Generates a normally distributed population within the bounds provided. Note this function is not intended to be called directly.

Parameters

population_size (int) – The number of random individuals to generate.
lowerbound (tensor) – Rank 1 tensor of shape (n_dim,) where n_dim is the dimensions of X.
upperbound (tensor) – Rank 1 tensor of shape (n_dim,) where n_dim is the dimensions of X.
mean (tensor, optional) – Rank 1 tensor of shape (n_dim,) where n_dim is the dimensions of X. If None, mean is set to (upperbound - lowerbound) / 2 + lowerbound. The default is None.
stddev (tensor, optional) – Rank 1 tensor of shape (n_dim,) where n_dim is the dimensions of X. If None, it is set to (upperbound - lowerbound) / 3 such that 99.7% of samples will fall between between the bounds provided the mean is centred in the space. Any sampled outside of the bounds will be clipped. The default is None.

Return type

Rank 2 tensor of shape (population_size, n_dims)

gatf.population.initialize_uniform_population(population_size, lowerbound, upperbound)

Generates a uniformly distributed population within the bounds provided. Note this function is not intended to be called directly.

Parameters

population_size (int) – The number of random individuals to generate.
lowerbound (tensor) – Rank 1 tensor of shape (n_dim,) where n_dim is the dimensions of X.
upperbound (tensor) – Rank 1 tensor of shape (n_dim,) where n_dim is the dimensions of X.

Return type

Rank 2 tensor of shape (population_size, n_dims)

gatf.population.uniform_reference_points(n_objectives, partitions=4)

Generates uniformly spaced reference points on a unit simplex. Note this function is not intended to be called directly but is used by gatf.algorithms.NSGA3. Adapted from the Pymoo library 2.

Parameters

n_objectives (int) – The number of objectives to be optimized.
partitions (int) – The number of partitions used to create the reference points.

Return type

Rank 2 tensor of shape (n_points, n_objectives)

References

2: https://github.com/DEAP/deap/blob/master/deap/tools/emo.py

Documentation

gatf.algorithms

gatf.operators

gatf.population

`gatf.algorithms`

`gatf.operators`

`gatf.population`