ELeaRNT: Evolutionary Learning of Rich Neural Network Topologies

1
ELeaRNT Evolutionary Learning of Rich Neural
Network Topologies

• Authors
• Slobodan Miletic 3078/2010 sloba10_at_gmail.com
• Nikola Jovanovic 3077/2010 nikolaj_ub_at_yahoo.co.u
  k

2
Introduction

• Genetic algorithm
• mimics natural evolution
• candidate solution
• mutation
• Neural network
• based on biological neurons
• network consists of neurons grouped in layers

3
Problem definition

• Current design methods are manual and inefficient
  
• Hard to define number of neurons and connections
• No automated design method
• for specific optimal topology

4
Problem importance

• Neural networks have a large use area
• Creating new neural networks
• takes money and time

5
Problem trend

• Computers are getting more powerful
• New neural network uses are found
• If not solved, this problem would slow down the
  evolution of neural networks

6
Existing solutions

• Trial Error
• Manual algorithm
• Few parameters for optimization
• Long, costly, and not very efficient

7
Existing solutions

• Destructive Algorithm
• Starts with very big networks
• Gets results by pruning the initial network
• A lot of time is spent on unnecessary training
• of big networks

8
Existing solutions

• Constructive Algorithm
• Starts with a small neural network
• Adds nodes and connections
• Uses input/error rate to form new nodes
• Can miss optimal solution

9
Proposed solution

• Whats better?
• General algorithm fitness function change
  enables generation
• of different neural networks types
• Created neural networks outperform
• neural network models designed by hand
• Besides the best network, it creates several
  suboptimal networks
• that can also be used as a solutions

10
Proposed solution

• Whats new?
• No similar general algorithm on the market
• Original set of genetic algorithm mutations

11
Proposed solution

• Whats its future?
• Computer power and parallelism are increasing
• which allows more complex neural networks
• Automated neural network generation algorithm
• like this will allow creation
• of complex neural networks

12
Solution details
3
1 point Crossover
2 point Crossover
5
g
3
f
13
Solution details
1 point Crossover
2 point Crossover
1 point Crossover
14
Solution details
5
g
b
2 point Crossover
1
a
15
Conclusion

• New way to create neural networks
• Results fully comparable
• with hand designed networks
• Space for further improvement
• More then one created network can be used

16

• QUESTIONS?