中文标题#
HHNAS-AM：使用自适应突变策略的分层混合神经架构搜索
英文标题#
HHNAS-AM: Hierarchical Hybrid Neural Architecture Search using Adaptive Mutation Policies
中文摘要#
神经架构搜索（NAS）由于其能够发现优于手动设计的架构而引起了广泛的研究兴趣。 学习文本表示对于文本分类和其他语言相关任务至关重要。 在文本分类中使用的 NAS 模型没有混合分层结构，并且对架构结构没有限制，因此搜索空间变得非常大且大部分是冗余的，所以现有的强化学习模型无法有效地导航搜索空间。 此外，进行扁平架构搜索会导致一个无序的搜索空间，难以遍历。 为此，我们提出了 HHNAS-AM（具有自适应突变策略的分层混合神经架构搜索），这是一种高效探索多样化架构配置的新方法。 我们引入了一些架构模板来进行搜索，这些模板组织了搜索空间，其中搜索空间是基于领域特定提示设计的。 我们的方法使用基于 Q 学习的突变策略，根据前几次迭代的性能反馈动态适应，从而更有效地加速搜索空间的遍历。 所提出的模型是完全概率的，能够有效地探索搜索空间。 我们在数据库 id（db_id）预测任务上评估了我们的方法，在多次实验中它始终能够发现高性能的架构。 在 Spider 数据集上，我们的方法在测试准确率上比现有基线提高了 8%。
英文摘要#
Neural Architecture Search (NAS) has garnered significant research interest due to its capability to discover architectures superior to manually designed ones. Learning text representation is crucial for text classification and other language-related tasks. The NAS model used in text classification does not have a Hybrid hierarchical structure, and there is no restriction on the architecture structure, due to which the search space becomes very large and mostly redundant, so the existing RL models are not able to navigate the search space effectively. Also, doing a flat architecture search leads to an unorganised search space, which is difficult to traverse. For this purpose, we propose HHNAS-AM (Hierarchical Hybrid Neural Architecture Search with Adaptive Mutation Policies), a novel approach that efficiently explores diverse architectural configurations. We introduce a few architectural templates to search on which organise the search spaces, where search spaces are designed on the basis of domain-specific cues. Our method employs mutation strategies that dynamically adapt based on performance feedback from previous iterations using Q-learning, enabling a more effective and accelerated traversal of the search space. The proposed model is fully probabilistic, enabling effective exploration of the search space. We evaluate our approach on the database id (db_id) prediction task, where it consistently discovers high-performing architectures across multiple experiments. On the Spider dataset, our method achieves an 8% improvement in test accuracy over existing baselines.
文章页面#
PDF 获取#
抖音扫码查看更多精彩内容