Genetic Algorithms (GAs) and Evolutionary Programming (EP) are investigated here in both optimization and machine learning. Adaptive and standard versions of the two algorithms are used to solve novel applications in search and rule extraction. Simulations and analysis show that while both algorithms may look similar in many ways their performance may differ for some applications. Mathematical modeling helps in gaining better understanding for GA and EP applications. Proper tuning and loading is a key for acceptable results. The ability to instantly adapt within an unpredictable and unstable search or learning environment is the most important feature of evolution-based techniques such as GAs and EP.