| Title:
|
Extraction of fuzzy logic rules from data by means of artificial neural networks (English) |
| Author:
|
Holeňa, Martin |
| Language:
|
English |
| Journal:
|
Kybernetika |
| ISSN:
|
0023-5954 |
| Volume:
|
41 |
| Issue:
|
3 |
| Year:
|
2005 |
| Pages:
|
[297]-314 |
| Summary lang:
|
English |
| . |
| Category:
|
math |
| . |
| Summary:
|
The extraction of logical rules from data has been, for nearly fifteen years, a key application of artificial neural networks in data mining. Although Boolean rules have been extracted in the majority of cases, also methods for the extraction of fuzzy logic rules have been studied increasingly often. In the paper, those methods are discussed within a five-dimensional classification scheme for neural-networks based rule extraction, and it is pointed out that all of them share the feature of being based on some specialized neural network, constructed directly for the rule extraction task. As an important representative, a method for the extraction of rules in a general fuzzy disjunctive normal form is described in detail and illustrated on real-world applications. Finally, the paper proposes an algorithm demonstrating a principal possibility to extract fuzzy logic rules from multilayer perceptrons with continuous activation functions, i. e., from the kind of neural networks most universally used in applications. However, complexity analysis of the individual steps of that algorithm reveals that it involves computations with doubly-exponential complexity, due to which it can not without simplifications serve as a practically applicable alternative to methods based on specialized neural networks. (English) |
| Keyword:
|
knowledge extraction from data |
| Keyword:
|
artificial neural networks |
| Keyword:
|
fuzzy logic |
| Keyword:
|
Lukasiewicz logic |
| Keyword:
|
disjunctive normal form |
| MSC:
|
03B52 |
| MSC:
|
03E72 |
| MSC:
|
62-07 |
| MSC:
|
62M45 |
| MSC:
|
68T30 |
| MSC:
|
68T37 |
| idZBL:
|
Zbl 1249.68158 |
| idMR:
|
MR2181420 |
| . |
| Date available:
|
2009-09-24T20:09:00Z |
| Last updated:
|
2015-03-23 |
| Stable URL:
|
http://hdl.handle.net/10338.dmlcz/135657 |
| . |
| Reference:
|
[1] Adamo J. M.: Data Mining for Association Rules and Sequential Patterns: Sequential and Parallel Algorithms.Springer–Verlag, Berlin 2001 Zbl 0998.68228 |
| Reference:
|
[2] Aguzzoli S., Mundici D.: Weierstrass approximations by Lukasiewicz formulas with one quantified variable.In: 31st IEEE Internat. Symposium on Multiple-Valued Logic, 2001 |
| Reference:
|
[3] Alexander J. A., Mozer M. C.: Template-based procedures for neural network interpretation.Neural Networks 12 (1999), 479–498 10.1016/S0893-6080(99)00009-X |
| Reference:
|
[4] Amato P., Nola, A. Di, Gerla B.: Neural networks and rational Lukasiewicz logic.J. Multiple-Valued Logic and Soft Computing (accepted for publication) |
| Reference:
|
[5] Amato P., Porto M.: An algorithm for the automatic generation of logical formula representing a control law.Neural Network World 10 (2000), 777–786 |
| Reference:
|
[6] Andrews R., Diederich, J., Tickle A. B.: Survey and critique of techniques for extracting rules from trained artificical neural networks.Knowledge-based Systems 8 (1995), 378–389 10.1016/0950-7051(96)81920-4 |
| Reference:
|
[7] Bern M., Chew L. P., Eppstein, D., Ruppert J.: Dihedral bounds for mesh generation in high dimensions.In: Proc. Sixth ACM-SIAM Symposium on Discrete Algorithms, ACM, San Francisco 1995, pp. 189–196 Zbl 0849.68116, MR 1321850 |
| Reference:
|
[8] Berthold M., Hand D., editors: Intelligent Data Analysis.An Introduction. Springer–Verlag, Berlin 1999 MR 1723394, 10.1007/978-3-662-03969-4 |
| Reference:
|
[9] Chen J., Liu J.: Using mixture principal component analysis networks to extract fuzzy rules from data.Indust. Engrg. Chemistry Research 39 (2000), 2355–2367 10.1021/ie9905613 |
| Reference:
|
[10] Cignoli L. O., D’Ottaviano I. M. L., Mundici D.: Algebraic Foundations of Many-valued Reasoning.Kluwer Academic Publishers, Dordrecht 2000 Zbl 0937.06009, MR 1786097 |
| Reference:
|
[11] Garcez A. S. d’Avila, Broda, K., Gabbay D. M.: Symbolic knowledge extraction from artificial neural networks: A sound approach.Artificial Intelligence 125 (2001), 155–207 MR 1805645, 10.1016/S0004-3702(00)00077-1 |
| Reference:
|
[12] Daňková M., Perfilieva I.: Logical approximation II.Soft Computing 7 (2003), 228–233 Zbl 1029.03503, 10.1007/s00500-002-0209-3 |
| Reference:
|
[13] Raedt L. De: Interactive Theory Revision: An Inductive Logic Programming Approach.Academic Press, London 1992 |
| Reference:
|
[14] Duch W., Adamczak, R., Grabczewski K.: Extraction of logical rules from neural networks.Neural Processing Lett. 7 (1998), 211–219 10.1023/A:1009670302979 |
| Reference:
|
[15] Duch W., Adamczak, R., Grabczewski K.: A new methodology of extraction, optimization and application of crisp and fuzzy logical rules.IEEE Trans. Neural Networks 11 (2000), 277–306 |
| Reference:
|
[16] Dzeroski S., Lavrac N.: Relational Data Mining.Springer–Verlag, Berlin 2001 Zbl 1003.68039 |
| Reference:
|
[17] Edelsbrunner H.: Algorithms in Combinatorial Geometry.Springer–Verlag, Heidelberg 1987 Zbl 0634.52001, MR 0904271 |
| Reference:
|
[18] Esteva F., Godo, L., Montagna F.: The L$\Pi $ and L$\Pi \frac{1}{2}$ logics: Two complete fuzzy systems joining Lukasiewicz and product logic.Arch. Math. Logic 40 (2001), 39–67 MR 1816606, 10.1007/s001530050173 |
| Reference:
|
[19] Faber J., Novák M., Svoboda, P., Tatarinov V.: Electrical brain wave analysis during hypnagogium.Neural Network World 13 (2003), 41–54 |
| Reference:
|
[20] Finn G. D.: Learning fuzzy rules from data.Neural Computing Appl. 8 (1999), 9–24 10.1007/s005210050003 |
| Reference:
|
[21] Freitas A. A.: Data Mining and Knowledge Discovery with Evolutionary Algorithms.Springer–Verlag, Berlin 2002 Zbl 1013.68075 |
| Reference:
|
[22] Gehrke M., Walker C. L., Walker E. A.: Normal forms and truth tables for fuzzy logics.Fuzzy Sets and Systems 138 (2003), 25–51 Zbl 1027.03023, MR 2012239 |
| Reference:
|
[23] Hájek P.: Metamathematics of Fuzzy Logic.Kluwer Academic Publishers, Dordrecht 1998 Zbl 1007.03022, MR 1900263 |
| Reference:
|
[24] Hájek P., Havránek T.: Mechanizing Hypothesis Formation.Springer–Verlag, Berlin 1978 Zbl 0371.02002, MR 0501342 |
| Reference:
|
[25] Healy M. J., Caudell T. P.: Acquiring rule sets as a product of learning in a logical neural architecture.IEEE Trans. Neural Networks 8 (1997), 461–474 10.1109/72.572088 |
| Reference:
|
[26] Holeňa M.: Extraction of logical rules from data by means of piecewise-linear neural networks.In: Proc. 5th Internat. Conference on Discovery Science, Springer–Verlag, Berlin 2002, pp. 192–205 Zbl 1024.68560 |
| Reference:
|
[27] Holeňa M., Baerns M.: Artificial neural networks in catalyst development.In: Experimental Design for Combinatorial and High Throughput Materials Development (J. N. Cawse, ed.), Wiley, Hoboken 2003, pp. 163–202 |
| Reference:
|
[28] Holeňa M., Baerns M.: Feedforward neural networks in catalysis.A tool for the approximation of the dependency of yield on catalyst composition, and for knowledge extraction. Catalysis Today 81 (2003), 485–494 |
| Reference:
|
[29] Horčík R., Cintula P.: Product Lukasiewicz logic.Arch. Math. Logic 43 (2004), 477–503 Zbl 1059.03011, MR 2060396, 10.1007/s00153-004-0214-6 |
| Reference:
|
[30] Ishikawa M.: Rule extraction by successive regularization.Neural Networks 13 (2000), 1171–1183 10.1016/S0893-6080(00)00072-1 |
| Reference:
|
[31] Lu H., Setiono, R., Liu H.: Effective data mining using neural networks.IEEE Trans. Knowledge and Data Engrg. 8 (1996), 957–961 10.1109/69.553163 |
| Reference:
|
[32] Maire F.: Rule-extraction by backpropagation of polyhedra.Neural Networks 12 (1999), 717–725 10.1016/S0893-6080(99)00013-1 |
| Reference:
|
[33] McNaughton R.: A theorem about infinite-valued sentential logic.J. Symbolic Logic 16 (1951), 1–13 Zbl 0043.00901, MR 0041799, 10.2307/2268660 |
| Reference:
|
[34] Mitra S., De R. K., Pal S. K.: Knowledge-based fuzzy MLP for classification and rule generation.IEEE Trans. Neural Networks 8 (1997), 1338–1350 10.1109/72.641457 |
| Reference:
|
[35] Mitra S., Hayashi Y.: Neuro-fuzzy rule generation: Survey in soft computing framework.IEEE Trans. Neural Networks 11 (2000), 748–768 10.1109/72.846746 |
| Reference:
|
[36] Muggleton S.: Inductive Logic Programming.Academic Press, London 1992 Zbl 1132.68007 |
| Reference:
|
[37] Mundici D.: A constructive proof of McNaughton’s theorem in infinite-valued logic.J. Symbolic Logic 59 (1994), 596–602 Zbl 0807.03012, MR 1276635, 10.2307/2275410 |
| Reference:
|
[38] Narazaki H., Watanabe, T., Yamamoto M.: Reorganizing knowledge in neural networks: An exploratory mechanism for neural networks in data classification problems.IEEE Trans. Systems Man Cybernet. 26 (1996), 107–117 10.1109/3477.484442 |
| Reference:
|
[39] Nauck D., Nauck, U., Kruse R.: Generating classification rules with the neuro-fuzzy system NEFCLASS.In: Proc. Biennial Conference of the North American Fuzzy Information Processing Society NAFIPS’96, 1996, pp. 466–470 |
| Reference:
|
[40] Novák V., Perfilieva I.: Some consequences of herbrand and McNaughton theorems in fuzzy logic.In: Discovering World with Fuzzy Logic: Perspectives and Approaches to Formalization of Human-Consistent Logical Systems (V. Novák and I. Perfilieva, eds.), Springer–Verlag, Heidelberg 1999, pp. 271–295 MR 1858104 |
| Reference:
|
[41] Novák V., Perfilieva, I., Močkoř J.: Mathematical Principles of Fuzzy Logic.Kluwer Academic Publishers, Dordrecht 1999 Zbl 0940.03028, MR 1733839 |
| Reference:
|
[42] Perfilieva I.: Neural nets and normal forms from fuzzy logic point of view.Technical Report, Institute for Research and Applications of Fuzzy Modelling, 2001 |
| Reference:
|
[43] Perfilieva I.: Normal forms for fuzzy logic functions and their approximation ability.Fuzzy Sets and Systems 124 (2001), 371–384 Zbl 0994.03019, MR 1860857 |
| Reference:
|
[44] Perfilieva I.: Logical approximaion.Soft Computing 7 (2003), 73–78 10.1007/s00500-002-0173-y |
| Reference:
|
[45] Perfilieva I.: Normal forms in BL-algerbra and their contribution fo universal approximation of functions.Fuzzy Sets and Systems 143 (2004), 111–127 MR 2060276, 10.1016/j.fss.2003.06.009 |
| Reference:
|
[46] Perfilieva I.: Normal forms in BL and L$\Pi $ algebras of functions.Soft Computing 8 (2004), 291–298 Zbl 1077.03047, 10.1007/s00500-003-0274-2 |
| Reference:
|
[47] Perfilieva I., Kreinovich V.: A new universal approximation result for fuzzy systems, which reflects CNF-DNF duality.Internat. J. Intelligent Systems 17 (2002), 1121–1130 Zbl 1028.68169, 10.1002/int.10063 |
| Reference:
|
[48] Polkowski L.: Rough Sets.Mathematical Foundations. Physica–Verlag, Heidelberg 2002 Zbl 1166.68307, MR 1967129 |
| Reference:
|
[49] Quinlan J.: C4.5: Programs for Machine Learning. Morgan Kaufmann Publishers, San Francisco 1992 Zbl 0900.68112 |
| Reference:
|
[50] Setiono R.: Extracting rules from neural networks by pruning and hidden unit splitting.Neural Computation 9 (1997), 05–225 Zbl 0872.68155, 10.1162/neco.1997.9.1.205 |
| Reference:
|
[51] Tickle A. B., Andrews R., Golea, M., Diederich J.: The truth will come to light: Directions and challenges in extracting rules from trained artificial neural networks.IEEE Trans. Neural Networks 9 (1998), 1057–1068 10.1109/72.728352 |
| Reference:
|
[52] Towell G. G., Shavlik J. W.: Extracting refined rules from knowledge-based neural networks.Mach. Learning 13 (1993), 71–101 10.1007/BF00993103 |
| Reference:
|
[53] Triantaphyllou E., (eds.) G. Felici: Data Mining and Knowledge Discovery Approaches Based on Rule Induction Techniques.Kluwer Academic Publishers, Dordrecht 2003 Zbl 1117.68028 |
| Reference:
|
[54] Tsukimoto H.: Extracting rules from trained neural networks.IEEE Trans. Neural Networks 11 (2000), 333–389 10.1109/72.839008 |
| Reference:
|
[55] Wong M. L., Leung K. S.: Data Mining Using Grammar Based Genetic Programming and Applications.Kluwer Academic Publishers, Dordrecht 2000 Zbl 0944.68172 |
| Reference:
|
[56] Zhang C., Zhang, S., Heymer B. E.: Association Rule Mining: Models and Algoritms.Springer–Verlag, Berlin 2002 |
| . |
