Rule-embedded neocognitron (REN) is proposed where the knowledge base of a neocognitron is constructed through incorporating production rules into its interlayer connections. Prototype patterns training is not required. The semantic of interlayer connections is established. The resulting network can now be analyzed according to the rule structure and problematic portions can be corrected. We demonstrate the ease with which performance can be improved by applying REN on handwritten numeral recognition. The same set of handwritten numerals initiated by Fukushima is used to test this methodology. It is found that the performance is comparable with that of Fukushima's neocognitron with supervised training.