How does the chapter's GA approach handle the optimization of hyperparameters and architecture for the Bi-LSTM model?

The GA encodes the Bi-LSTM's hyperparameters and architecture as floating-point chromosomes and uses mean squared error (MSE) on the training data as the fitness function. Each generation applies selection, crossover, mutation, and elitism to evolve batch size, sampling interval, learning rate, per-layer neuron counts, hidden layer count, optimizer, and activation function. Optimizers and activations are float-coded and mapped to discrete options, and later hidden layers can be dropped when their encoded value is negative. Population-size tests showed that about 50 individuals is best for avoiding premature convergence.

The chapter describes a GA-based neural architecture search combined with hyperparameter optimization for Bi-LSTM models. Each candidate solution is a floating-point chromosome containing fields for batch size (20 to 40), data sampling interval (0.25 to 2 hours), learning rate (0.0005 to 0.05), number of neurons in several hidden layers, negative lower bounds for later layers, an optimizer choice, and an activation function choice. The GA initializes a population of 100 chromosomes with a random seed for reproducibility, and fitness is evaluated as the MSE obtained while training the Bi-LSTM and checking its predictions; a lower MSE identifies a better individual. At each generation, the GA selects, reproduces, applies crossover and mutation, and retains the top individuals through elitism. Categorical settings such as optimizer and activation are handled by generating a float in the range 0 to 2.99, rounding it to 0, 1, or 2, and mapping it to Adam, SGD, and Adagrad for optimizers, or sigmoid, tanh, and ReLU for activations. To search architecture depth, later hidden-layer fields have ranges that begin with negative values, so a negative encoded value removes the corresponding hidden layer; the optimal final chromosome therefore determines the number of hidden layers. The researchers tested initial population sizes from 10 to 200 and found that small starting populations caused premature convergence and resulted in more hidden layers, while a population size around 50 provided the best overall performance for all four targets.