Learning Automaton Fundamentals and History

Learning automata have been around for decades, with the first concept introduced in the 1950s.

The field of learning automata has its roots in the work of mathematicians and computer scientists.

The first learning automaton, called the "finite automaton", was developed by mathematician Claude Shannon in the 1950s.

This early model laid the foundation for the development of more complex learning automata.

Learning automata are designed to learn from their environment and adapt to changing situations, much like humans do.

Their ability to learn and adapt has made them a crucial component in many fields, including artificial intelligence and robotics.

A learning automaton is an adaptive decision-making unit that learns the optimal action through repeated interactions with its environment. This process involves choosing actions according to a specific probability distribution, which is updated based on the environment's response.

The learning automaton is situated in a random environment, and its actions are chosen from a set of possible outputs, or actions. The set of possible outputs is denoted as α = { α1, ..., αr }, where r is the number of possible actions.

The learning automaton's behavior can be formalized using a set of possible inputs X, a set of possible internal states Φ, and a set of possible outputs α. The initial state probability vector is p(0) = ≪ p1(0), ..., ps(0) ≫.

The learning automaton updates its state probability vector p(t) to p(t+1) using a computable function A, which takes into account the current input, state, and time step t. The output at each time step is generated by a function G: Φ → α.

A simple example of a learning automaton's environment is one that responds with 0 or 1, where 0 represents a non-penalty response and 1 represents a penalty response. In this case, the learning automaton should learn to minimize the number of penalty responses.

Learning automata come in various forms, each with its own unique characteristics.

Finite action-set learning automata are a class of learning automata where the number of possible actions is finite.

In other words, the size of the action-set is finite, making it easier to analyze and understand their behavior.

This type of learning automaton is particularly useful in situations where the number of possible actions is limited.

For example, in a game where you can only choose from a few moves, a finite action-set learning automaton would be a good fit.

It's worth noting that this type of learning automaton is often used in mathematical terms, where the size of the action-set is a key consideration.

Expand your knowledge: Action Model Learning

Learning automaton has been studied and applied to various engineering systems for decades, making it a powerful reinforcement learning method.

The state-of-the-art LA-based methods can only select the optimal action or optimal subset, but not an arbitrary target subset like selecting the best and worst actions or the ones in a given rank range.

This limitation is a problem that needs to be solved, which is where the novel pursuit learning scheme, DEP RI-AS, comes in.

The learning automaton (LA) has been a powerful reinforcement learning method studied and applied to various engineering systems for decades.

It's been analyzed and applied to various fields, but its limitations have been identified.

The state-of-the-art LA-based methods can only select the optimal action or optimal subset, which is not suitable for selecting an arbitrary target subset.

This limitation has been a problem in many applications.

The proposed discretized equal pursuit reward-inaction algorithm for arbitrary subset selection (DEP RI-AS) aims to solve this problem.

It pursues the currently estimated arbitrary action subset and makes the probabilities of selecting each action in the subset equal.

This approach increases the convergence speed of the algorithm.

The proof of its -optimality property has been presented, demonstrating its effectiveness.

Simulation results show that DEP RI-AS outperforms other methods in selecting a given subset of user-desired actions.

Readers also liked: Learning with Errors

In the early 1960s, Michael Lvovitch Tsetlin in the Soviet Union published a collection of papers on using matrices to describe automata functions.

Tsetlin's work was a significant milestone in the field of learning automata, and it laid the foundation for further research.

Research on learning automata also took place in the United States in the 1960s, but the term "learning automaton" wasn't introduced until Narendra and Thathachar used it in a survey paper in 1974.

A visual demo of a single Learning Automaton was developed by the μSystems Research Group at Newcastle University.