دوره 17، شماره 2 - ( مجله کنترل، جلد 17، شماره 2، تابستان 1402 )
جلد 17 شماره 2,1402 صفحات 23-1 |
برگشت به فهرست نسخه ها
Download citation:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:
Mohammadzadeh Ayooghi V, Aliyari-Shoorehdeli M. Deep Learning based Models for Nonlinear System Identification. JoC 2023; 17 (2) :1-23
URL: http://joc.kntu.ac.ir/article-1-1008-fa.html
URL: http://joc.kntu.ac.ir/article-1-1008-fa.html
محمدزاده ایوقی وحید، علیاری شوره دلی مهدی. مدل های مبتنی بر یادگیری عمیق در شناسایی سیستم های غیرخطی. مجله کنترل. 1402; 17 (2) :1-23
1- دانشگاه صنعتی خواجه نصیرالدین طوسی، تهران، ایران
2- گروه مکاترونیک، دانشکده مهندسی برق، دانشگاه صنعتی خواجه نصیرالدین طوسی، تهران، ایران
2- گروه مکاترونیک، دانشکده مهندسی برق، دانشگاه صنعتی خواجه نصیرالدین طوسی، تهران، ایران
چکیده: (2761 مشاهده)
مدل های مبتنی بر یادگیری عمیق عملکرد بسیار مناسبی در مدل سازی مسائل پیچیده در بینایی ماشین و پردازش زبان طبیعی از خود نشان داده‌اند که این موضوع به ماهیت غیرخطی و فراپارامتری این مدل ها نسبت داده می شود. روش های شناسایی سیستم های غیرخطی، می تواند از ابزارهای توسعه‌یافته در حوزه یادگیری عمیق بهره مند شوند که این امر موجب گسترده شدن ابزارهای موجود برای انتخاب یک مدل مناسب خواهد شد. ازاین‌رو، در این مقاله قصد داریم تا روش ها و ساختارهای موجود در یادگیری عمیق را از دیدگاه شناسایی سیستم های غیرخطی مرور کنیم. اگرچه مرور نسبتاً جامعی از ابزارهای قابل‌استفاده در حوزه شناسایی سیستم های غیرخطی ارائه خواهد شد، اما تمرکز اصلی این پژوهش بر کاربرد مدل های متغیر پنهان در شناسایی فضای حالت غیرخطی است. مدل های متغیر پنهان دسته ای از مدل های یادگیری عمیق هستند که در گروه مدل های مولد قرار می گیرند. نسخه اصلی این مدل ها، تنها قابلیت تولید داده های ایستا را داراست. اما با ترکیب شبکه های عصبی بازگشتی و خود رمزنگار تغییراتی، قابلیت تولید داده های ترتیبی برای این مدل ها نیز فراهم شده است. همچنین نسخه ساختاریافته ای از این مدل ها نیز که منطبق بر سیستم های کنترل است، ارائه خواهد شد. مطالعه انجام شده نشان می دهد که عملکرد این مدل ها با مدل های پیشین و کلاسیک موجود، قابل‌قیاس است.
واژه‌های کلیدی: مدل های یادگیری عمیق، شناسایی سیستم های غیرخطی، مدل های متغیر پنهان، خود رمزنگار تغییراتی
نوع مطالعه: پژوهشي |
موضوع مقاله:
شماره ویژه (رویکرد های نو در مهندسی کنترل)
دریافت: 1402/5/5 | پذیرش: 1402/6/20 | انتشار الکترونیک پیش از انتشار نهایی: 1402/6/28 | انتشار: 1402/6/30
دریافت: 1402/5/5 | پذیرش: 1402/6/20 | انتشار الکترونیک پیش از انتشار نهایی: 1402/6/28 | انتشار: 1402/6/30
فهرست منابع
1. [1] L. Ljung, "System identification, Theory for the user." System science series, Prentice Hall, Upper Saddle River, NJ, USA, Second edition, 1999.
2. [2] Zadeh, L. "On the identification problem." IRE Transactions on Circuit Theory 3.4 (1956): 277-281. [DOI:10.1109/TCT.1956.1086328]
3. [3] Nelles, Oliver. "Nonlinear system identification, from classical approach to neural networks, fuzzy systems, and Gaussian process". Springer, Berlin, Heidelberg, 2020. [DOI:10.1007/978-3-030-47439-3]
4. [4] T. Soderstrom, P. Stocia, "System identification", Prentice-Hall, Inc, 1988.
5. [5] Schoukens, Johan, and Lennart Ljung. "Nonlinear system identification: A user-oriented road map." IEEE Control Systems Magazine 39.6 (2019): 28-99. [DOI:10.1109/MCS.2019.2938121]
6. [6] Billings, Stephen A. "Identification of nonlinear systems-a survey." IEE Proceedings D (Control Theory and Applications). Vol. 127. No. 6. IET Digital Library, 1980. [DOI:10.1049/ip-d.1980.0047]
7. [7] Zheng, Qingsheng, and Evanghelos Zafiriou. "Nonlinear system identification for control using Volterra-Laguerre expansion." Proceedings of 1995 American Control Conference-ACC'95. Vol. 3. IEEE, 1995.
8. [8] Korenberg, Michael J., and Ian W. Hunter. "The identification of nonlinear biological systems: Wiener kernel approaches." Annals of Biomedical Engineering 18 (1990): 629-654. [DOI:10.1007/BF02368452]
9. [9] Schoukens, Maarten, and Koen Tiels. "Identification of block-oriented nonlinear systems starting from linear approximations: A survey." Automatica 85 (2017): 272-292. [DOI:10.1016/j.automatica.2017.06.044]
10. [10] Chiuso, Alessandro, and Gianluigi Pillonetto. "System identification: A machine learning perspective." Annual Review of Control, Robotics, and Autonomous Systems 2 (2019): 281-304. [DOI:10.1146/annurev-control-053018-023744]
11. [11] Pillonetto, Gianluigi, et al. "Kernel methods in system identification, machine learning and function estimation: A survey." Automatica 50.3 (2014): 657-682. [DOI:10.1016/j.automatica.2014.01.001]
12. [12] Bishop, Christopher M., and Nasser M. Nasrabadi. Pattern recognition and machine learning. Vol. 4. No. 4. New York: springer, 2006.
13. [13] Heaton, Jeff. Ian goodfellow, yoshua bengio, and aaron courville: "Deep learning." (2018): 305-307. [DOI:10.1007/s10710-017-9314-z]
14. [14] Bengio, Yoshua, Aaron Courville, and Pascal Vincent. "Representation learning: A review and new perspectives." IEEE transactions on pattern analysis and machine intelligence 35.8 (2013): 1798-1828. [DOI:10.1109/TPAMI.2013.50]
15. [15] Noroozi, Mehdi, and Paolo Favaro. "Unsupervised learning of visual representations by solving jigsaw puzzles." European conference on computer vision. Cham: Springer International Publishing, 2016. [DOI:10.1007/978-3-319-46466-4_5]
16. [16] Kingma, Diederik P., and Max Welling. "Auto-encoding variational bayes." arXiv preprint arXiv: 1312.6114 (2013).
17. [17] Fraccaro, Marco. "Deep latent variable models for sequential data." English. PhD thesis DTU University (2018).
18. [18] Barber, David. Bayesian reasoning and machine learning. Cambridge University Press, 2012. [DOI:10.1017/CBO9780511804779]
19. [19] Koller, Daphne, and Nir Friedman. Probabilistic graphical models: principles and techniques. MIT press, 2009.
20. [20] Hinton, Geoffrey E. "A practical guide to training restricted Boltzmann machines." Neural Networks: Tricks of the Trade: Second Edition. Berlin, Heidelberg: Springer Berlin Heidelberg, 2012. 599-619. [DOI:10.1007/978-3-642-35289-8_32]
21. [21] Fischer, Asja, and Christian Igel. "An introduction to restricted Boltzmann machines." Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications: 17th Iberoamerican Congress, CIARP 2012, Buenos Aires, Argentina, September 3-6, 2012. Proceedings 17. Springer Berlin Heidelberg, 2012.
22. [22] Raia Hadsell, Sumit Chopra, and Yann LeCun. Dimensionality reduction by learning an invariant mapping. In CVPR, 2006.
23. [23] Gianluigi Pillonetto, Aleksandr Aravkin, Daniel Gedon, Lennart Ljung, Antônio H. Ribeiro, Thomas B. Schön, Deep network for system identification, arXiv, 2023.
24. [24] Ljung, Lennart, et al. "Deep learning and system identification." IFAC-PapersOnLine 53.2 (2020): 1175-1181. [DOI:10.1016/j.ifacol.2020.12.1329]
25. [25] M. Forgione and D. Piga. Dynonet: A neural network architecture for learning dynamical systems. Int. J. Adapt. Control Signal Process., 35(4):612-626, 2021. [DOI:10.1002/acs.3216]
26. [26] J. Hendriks, F.K. Gustafsson, A.H. Ribeiro, A. Wills, and T.B. Schon. Deep energy-based NARX models. In Proceedings of the 19th IFAC Symposium on System Identification (SYSID), 2021. [DOI:10.1016/j.ifacol.2021.08.410]
27. [27] LeCun, Yann, et al. "Energy-based models in document recognition and computer vision." Ninth International Conference on Document Analysis and Recognition (ICDAR 2007). Vol. 1. IEEE, 2007. [DOI:10.1109/ICDAR.2007.4378728]
28. [28] Andersson, Carl, et al. "Deep convolutional networks in system identification." 2019 IEEE 58th conference on decision and control (CDC). IEEE, 2019. [DOI:10.1109/CDC40024.2019.9030219]
29. [29] Lea, Colin, et al. "Temporal convolutional networks: A unified approach to action segmentation." Computer Vision-ECCV 2016 Workshops: Amsterdam, The Netherlands, October 8-10 and 15-16, 2016, Proceedings, Part III 14. Springer International Publishing, 2016.
30. [30] R. Calandra, J. Peters, C. Rasmussen, and M.P. Deisenroth. Manifold gaussian processes for regression. In 2016 International Joint Conference on Neural Networks (IJCNN), pages 3338-3345,2016. [DOI:10.1109/IJCNN.2016.7727626]
31. [31] Y. Cho and L. Saul. Kernel methods for deep learning. In Advances in Neural Information Processing Systems, volume 22, 2009.
32. [32] Nagel, Tobias, and Marco F. Huber. "Autoencoder-inspired Identification of LTI systems." 2021 European Control Conference (ECC). IEEE, 2021. [DOI:10.23919/ECC54610.2021.9655185]
33. [33] D. Gedon, N. Wahlstr¨om, T.B. Sch¨on, and L. Ljung. Deep state space models for nonlinear system identification. In Proceedings of the 19th IFAC Symposium on System Identification (SYSID), 2021. [DOI:10.1016/j.ifacol.2021.08.406]
34. [34] M. Karl, M. Soelch, J. Bayer, and P. van der Smagt. Deep variational Bayes filters: Unsupervised learning of state space models from raw data, 2017.
35. [35] M. Watter, J. Springenberg, J. Tobias, J. Boedecker, and M. Riedmiller. Embed to control: A locally linear latent dynamics model for control from raw images. In Proceedings of the 28th International Conference on Neural Information Processing Systems, Volume 2, pages 2746-2754, Cambridge, MA, USA, 2015. MIT Press.
36. [36] Rangapuram, Syama Sundar, et al. "Deep state space models for time series forecasting." Advances in neural information processing systems 31 (2018).
37. [37] Courts, Jarrad, et al. "Variational state and parameter estimation." IFAC-PapersOnLine 54.7 (2021): 732-737. [DOI:10.1016/j.ifacol.2021.08.448]
38. [38] Courts, Jarrad, et al. "Variational System Identification for Nonlinear State-Space Models." arXiv preprint arXiv: 2012.05072 (2020).
39. [39] Menghani, Gaurav. "Efficient deep learning: A survey on making deep learning models smaller, faster, and better." ACM Computing Surveys 55.12 (2023): 1-37. [DOI:10.1145/3578938]
40. [40] Xu, Canwen, and Julian McAuley. "A survey on model compression and acceleration for pretrained language models." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 37. No. 9. 2023. [DOI:10.1609/aaai.v37i9.26255]
41. [41] Choudhary, Tejalal, et al. "A comprehensive survey on model compression and acceleration." Artificial Intelligence Review 53 (2020): 5113-5155. [DOI:10.1007/s10462-020-09816-7]
42. [42] Cheng, Yu, et al. "A survey of model compression and acceleration for deep neural networks." arXiv preprint arXiv:1710.09282 (2017).
43. [43] Cheng, Yu, et al. "Model compression and acceleration for deep neural networks: The principles, progress, and challenges." IEEE Signal Processing Magazine 35.1 (2018): 126-136. [DOI:10.1109/MSP.2017.2765695]
44. [44] Liang, Tailin, et al. "Pruning and quantization for deep neural network acceleration: A survey." Neurocomputing 461 (2021): 370-403. [DOI:10.1016/j.neucom.2021.07.045]
45. [45] Rokh, Babak, Ali Azarpeyvand, and Alireza Khanteymoori. "A comprehensive survey on model quantization for deep neural networks." arXiv preprint arXiv: 2205.07877 (2022).
46. [46] Gholami, Amir, et al. "A survey of quantization methods for efficient neural network inference." Low-Power Computer Vision. Chapman and Hall/CRC, 2022. 291-326. [DOI:10.1201/9781003162810-13]
47. [47] Liang, Tailin, et al. "Pruning and quantization for deep neural network acceleration: A survey." Neurocomputing 461 (2021): 370-403. [DOI:10.1016/j.neucom.2021.07.045]
48. [48] Frankle, Jonathan, and Michael Carbin. "The lottery ticket hypothesis: Finding sparse, trainable neural networks." arXiv preprint arXiv: 1803.03635 (2018).
49. [49] Hinton, Geoffrey, Oriol Vinyals, and Jeff Dean. "Distilling the knowledge in a neural network." arXiv preprint arXiv: 1503.02531 (2015).
50. [50] Gou, Jianping, et al. "Knowledge distillation: A survey." International Journal of Computer Vision 129 (2021): 1789-1819. [DOI:10.1007/s11263-021-01453-z]
51. [51] Hastie, Trevor, et al. The elements of statistical learning: data mining, inference, and prediction. Vol. 2. New York: springer, 2009. [DOI:10.1007/978-0-387-84858-7]
52. [52] Doersch, Carl. "Tutorial on variational autoencoders." arXiv preprint arXiv: 1606.05908 (2016).
53. [53] Burda, Yuri, Roger Grosse, and Ruslan Salakhutdinov. "Importance weighted autoencoders." arXiv preprint arXiv: 1509.00519 (2015) [40] Barber, David, A. Taylan Cemgil, and Silvia Chiappa, eds. Bayesian time series models. Cambridge University Press, 2011.
54. [54] Makhzani, Alireza, et al. "Adversarial autoencoders." arXiv preprint arXiv: 1511.05644 (2015). [39] Burda, Yuri, Roger Grosse, and Ruslan Salakhutdinov. "Importance weighted autoencoders." arXiv preprint arXiv: 1509.00519 (2015)
55. [55] Girin, Laurent, et al. "Dynamical variational autoencoders: A comprehensive review." arXiv preprint arXiv:2008.12595 (2020). [DOI:10.1561/9781680839135]
56. [56] Beal, Matthew James. Variational algorithms for approximate Bayesian inference. University of London, University College London (United Kingdom), 2003.
57. [57] Hochreiter, Sepp, and Jürgen Schmidhuber. "Long short-term memory." Neural computation 9.8 (1997): 1735-1780. [DOI:10.1162/neco.1997.9.8.1735]
58. [58] Chung, Junyoung, et al. "Empirical evaluation of gated recurrent neural networks on sequence modeling." arXiv preprint arXiv:1412.3555 (2014).
59. [59] Su, Yuanhang, and C-C. Jay Kuo. "Recurrent neural networks and their memory behavior: a survey." APSIPA Transactions on Signal and Information Processing 11.1 (2022). [DOI:10.1561/116.00000123]
60. [60] Graves, Alex. "Generating sequences with recurrent neural networks." arXiv preprint arXiv:1308.0850 (2013).
61. [61] Krishnan, Rahul G., Uri Shalit, and David Sontag. "Deep kalman filters." arXiv preprint arXiv: 1511.05121 (2015).
62. [62] Bahdanau, Dzmitry, Kyunghyun Cho, and Yoshua Bengio. "Neural machine translation by jointly learning to align and translate." arXiv preprint arXiv: 1409.0473 (2014).
63. [63] Vaswani, Ashish, et al. "Attention is all you need." Advances in neural information processing systems 30 (2017).
64. [64] Han, Kai, et al. "A survey on vision transformer." IEEE transactions on pattern analysis and machine intelligence 45.1 (2022): 87-110. [DOI:10.1109/TPAMI.2022.3152247]
65. [65] Khan, Salman, et al. "Transformers in vision: A survey." ACM computing surveys (CSUR) 54.10s (2022): 1-41. [DOI:10.1145/3505244]
66. [66] Liu, Yang, et al. "A survey of visual transformers." IEEE Transactions on Neural Networks and Learning Systems (2023). [DOI:10.1109/TNNLS.2022.3227717]
67. [67] Lin, Tianyang, et al. "A survey of transformers." AI Open (2022). [DOI:10.1016/j.aiopen.2022.10.001]
68. [68] Rosendahl, Jan, et al. "Analysis of positional encodings for neural machine translation." Proceedings of the 16th International Conference on Spoken Language Translation. 2019.
69. [69] Beintema, Gerben I., Maarten Schoukens, and Roland Tóth. "Deep subspace encoders for nonlinear system identification." Automatica 156 (2023): 111210. [DOI:10.1016/j.automatica.2023.111210]
70. [70] Masti, Daniele, and Alberto Bemporad. "Learning nonlinear state-space models using autoencoders." Automatica 129 (2021): 109666. [DOI:10.1016/j.automatica.2021.109666]
71. [71] Lopez, Ryan, and Paul J. Atzberger. "Variational autoencoders for learning nonlinear dynamics of physical systems." arXiv preprint arXiv:2012.03448 (2020).
72. [72] Chung, Junyoung, et al. "A recurrent latent variable model for sequential data." Advances in neural information processing systems 28 (2015).
73. [73] Bayer, Justin, and Christian Osendorfer. "Learning stochastic recurrent networks." arXiv preprint arXiv: 1411.7610 (2014).
74. [74] Fraccaro, Marco, et al. "Sequential neural models with stochastic layers." Advances in neural information processing systems 29 (2016).
75. [75] Fraccaro, Marco, et al. "A disentangled recognition and nonlinear dynamics model for unsupervised learning." Advances in neural information processing systems 30 (2017).
ارسال پیام به نویسنده مسئول
| بازنشر اطلاعات | |
 |
این مقاله تحت شرایط Creative Commons Attribution-NonCommercial 4.0 International License قابل بازنشر است. |
