Existing methods of machine learning (ML), a foundational technology of artificial intelligence, often rely on extensive human intervention and manual preconfiguration, including manual collection, selection, and annotation of data, manual construction of the basic architecture of deep neural networks, determining the algorithm type of optimization algorithms and their hyperparameters, etc. These limitations prevent ML from effectively dealing with complex data and changing multi-task environments in the real world.
To address these existing challenges in current ML, a research team from Xi'an Jiaotong University developed a new approach called Simulation Learning Methodology (SLeM). The core concept of SLeM is to simulate and extract the learning methodology of ML, which has traditionally been set by humans, and transform it into an automatic learning process. Essentially, the SLeM framework represents an ML-for-ML paradigm, where ML tools are used to design and optimize the basic components of ML.
The team developed a set of algorithms for ML automation based on the SLeM framework and demonstrated its effectiveness in enhancing the adaptive learning capabilities of existing ML methods.
“Recently, many AutoML methods have been proposed to realize ML automation. However, most of the existing AutoML methods are heuristic in nature, making it difficult to establish a solid theoretical foundation. In contrast, the SLeM framework provides a unified mathematical formulation for ML automation and provides theoretical insights into the task transfer generalization capability of SLeM,” said Professor Zongben Xu, lead author of the paper and Academician of the Chinese Academy of Sciences.
The development of advanced large-scale language models (LLMs) has become a foundation of artificial intelligence, greatly expanding their ability to solve a wide range of applications and tasks. However, the theoretical evidence underlying the superior task generalization ability of LLMs has not been fully addressed by the ML community. The novel SLeM approach provides a promising perspective and tool to advance the study and understanding of the task generalization ability of large-scale language models (LLMs).
More information:
Zongben Xu et al., “Simulation Learning Methodology (SLeM): An Approach to Automating Machine Learning,” National Science Review (2024). DOI: 10.1093/nsr/nwae277
Provided by Science China Press
Citation: Simulating Learning Methodology: An Approach to Machine Learning Automation (August 29, 2024) Retrieved August 29, 2024 from https://techxplore.com/news/2024-08-simulating-methodology-approach-machine-automation.html
This document is subject to copyright. It may not be reproduced without written permission, except for fair dealing for the purposes of personal study or research. The content is provided for informational purposes only.