脑科学与人工智能Arxiv每日论文推送 2023.2.5
脑科学与人工智能Arxiv每日论文推送 2023.2.5
【1】通过反向对比损失鼓励类内多样性，以实现更好的单源领域泛化
Encouraging Intra-Class Diversity Through a Reverse Contrastive Loss for Better Single-Source Domain Generalization
作者： Thomas Duboudin et.al
链接： https:// arxiv.org/abs/2106.0791 6
摘要： 传统的深度学习算法在训练数据领域之外进行测试时，往往无法进行泛化。这个问题可以通过在训练时使用来自目标领域的未标记数据来缓解，但由于一旦部署了学习的模型，数据分布在现实生活中的应用会发生动态变化，因此创建对未知和不可预见的领域转变具有鲁棒性的网络至关重要。在本文中，我们重点讨论了神经网络无法做到这一点的原因之一：深度网络只关注最明显的、可能是虚假的线索来进行预测，对有用但效率稍低或更复杂的模式视而不见。这种行为已经被发现，有几种方法部分地解决了这个问题。为了研究它们的有效性和局限性，我们首先设计了一个公开可用的基于MNIST的基准，以精确测量算法寻找 "隐藏 "模式的能力。然后，我们通过我们的基准来评估最先进的算法，并表明这个问题基本上没有得到解决。最后，我们提出了一个部分反转的对比损失，以鼓励类内多样性，并找到不那么强烈相关的模式，其效率由我们的实验来证明。
Traditional deep learning algorithms often fail to generalize when they are tested outside of the domain of the training data. The issue can be mitigated by using unlabeled data from the target domain at training time, but because data distributions can change dynamically in real-life applications once a learned model is deployed, it is critical to create networks robust to unknown and unforeseen domain shifts. In this paper we focus on one of the reasons behind the inability of neural networks to be so: deep networks focus only on the most obvious, potentially spurious, clues to make their predictions and are blind to useful but slightly less efficient or more complex patterns. This behaviour has been identified and several methods partially addressed the issue. To investigate their effectiveness and limits, we first design a publicly available MNIST-based benchmark to precisely measure the ability of an algorithm to find the ''hidden'' patterns. Then, we evaluate state-of-the-art algorithms through our benchmark and show that the issue is largely unsolved. Finally, we propose a partially reversed contrastive loss to encourage intra-class diversity and find less strongly correlated patterns, whose efficiency is demonstrated by our experiments.
【2】基于动态记忆的原型网络的元学习，用于少数事件的检测
Meta-Learning with Dynamic-Memory-Based Prototypical Network for Few-Shot Event Detection
作者： Shumin Deng et.al
链接： https:// arxiv.org/abs/1910.1162 1
摘要： 事件检测（ED）是事件提取的一个子任务，涉及到识别触发器和对事件提及的分类。现有的方法主要依赖于监督学习，并需要大规模的标记事件数据集，但不幸的是，这些数据集在许多现实生活中并不容易获得。在本文中，我们考虑并重新表述了具有有限标记数据的ED任务，将其作为一个少数人学习问题。我们提出了一个基于动态记忆的原型网络（DMB-PN），它利用动态记忆网络（DMN）不仅为事件类型学习更好的原型，而且还为事件的提及产生更稳健的句子编码。与简单地通过平均计算事件原型的虚构原型网络不同，我们的模型更加稳健，并且由于DMN的多跳机制，能够多次从事件提及中提炼出背景信息。实验表明，DMB-PN不仅比一系列基线模型更好地处理了样本稀缺问题，而且在事件类型种类相对较多、实例数量极少的情况下表现得更加稳健。
Event detection (ED), a sub-task of event extraction, involves identifying triggers and categorizing event mentions. Existing methods primarily rely upon supervised learning and require large-scale labeled event datasets which are unfortunately not readily available in many real-life applications. In this paper, we consider and reformulate the ED task with limited labeled data as a Few-Shot Learning problem. We propose a Dynamic-Memory-Based Prototypical Network (DMB-PN), which exploits Dynamic Memory Network (DMN) to not only learn better prototypes for event types, but also produce more robust sentence encodings for event mentions. Differing from vanilla prototypical networks simply computing event prototypes by averaging, which only consume event mentions once, our model is more robust and is capable of distilling contextual information from event mentions for multiple times due to the multi-hop mechanism of DMNs. The experiments show that DMB-PN not only deals with sample scarcity better than a series of baseline models but also performs more robustly when the variety of event types is relatively large and the instance quantity is extremely small.
【3】时间性知识图谱的无监督实体排列
Unsupervised Entity Alignment for Temporal Knowledge Graphs
作者： Xiaoze Liu et.al
链接： https:// arxiv.org/abs/2302.0079 6
摘要： 实体对齐（EA）是一项基本的数据集成任务，它可以识别不同知识图谱（KGs）之间的等价实体。时间知识图（TKGs）通过引入时间戳扩展了传统的知识图，它已经受到越来越多的关注。最先进的时间感知的EA研究表明，TKGs的时间信息有助于EA的性能。然而，现有的研究并没有彻底利用TKGs的时间信息的优势。而且，他们通过预先对齐实体对来执行EA，这可能是劳动密集型的，因此效率很低。
在本文中，我们提出了DualMatch，它有效地融合了关系信息和时间信息的EA。DualMatch将TKG上的EA转移到一个加权图的匹配问题。更具体地说，DualMatch配备了一个无监督的方法，在不需要种子对齐的情况下实现了EA。DualMatch有两个步骤。(i) 使用新型的无标签编码器Dual-Encoder将时间和关系信息分别编码为嵌入；(ii) 使用新型的基于图匹配的解码器GM-Decoder将这两种信息融合并转化为对齐。DualMatch能够在有或没有监督的情况下对TKGs进行EA，这是因为它能够有效地捕捉时间信息。在三个真实世界的TKG数据集上进行的广泛实验表明，DualMatch在H@1和MRR方面分别比最先进的方法高出2.4%-10.7%和1.7%-7.6%。
Entity alignment (EA) is a fundamental data integration task that identifies equivalent entities between different knowledge graphs (KGs). Temporal Knowledge graphs (TKGs) extend traditional knowledge graphs by introducing timestamps, which have received increasing attention. State-of-the-art time-aware EA studies have suggested that the temporal information of TKGs facilitates the performance of EA. However, existing studies have not thoroughly exploited the advantages of temporal information in TKGs. Also, they perform EA by pre-aligning entity pairs, which can be labor-intensive and thus inefficient.
In this paper, we present DualMatch which effectively fuses the relational and temporal information for EA. DualMatch transfers EA on TKGs into a weighted graph matching problem. More specifically, DualMatch is equipped with an unsupervised method, which achieves EA without necessitating seed alignment. DualMatch has two steps: (i) encoding temporal and relational information into embeddings separately using a novel label-free encoder, Dual-Encoder; and (ii) fusing both information and transforming it into alignment using a novel graph-matching-based decoder, GM-Decoder. DualMatch is able to perform EA on TKGs with or without supervision, due to its capability of effectively capturing temporal information. Extensive experiments on three real-world TKG datasets offer the insight that DualMatch outperforms the state-of-the-art methods in terms of H@1 by 2.4% - 10.7% and MRR by 1.7% - 7.6%, respectively.
【4】SHINE: 基于深度学习的无障碍停车管理系统
SHINE: Deep Learning-Based Accessible Parking Management System
作者： Dhiraj Neupane et.al
链接： https:// arxiv.org/abs/2302.0083 7
摘要： 科学和技术的提高帮助城市的扩张，这是前所未有的。由于拥有私家车有不可否认的好处，在世界许多地方，包括韩国，汽车的数量已经飙升。然而，这些车辆数量的逐渐增加导致了与停车有关的问题，包括滥用残疾人停车位（以下简称无障碍停车位）。由于监控摄像机的高帧率，传统的车牌识别（LPR）系统在实时性方面是无效的。另一方面，自然和人工噪音以及照明和天气条件的差异使这些系统难以检测和识别。随着停车4.0概念的不断发展，许多传感器、物联网和基于深度学习的方法已被应用于自动LPR和停车管理系统。然而，研究表明，在韩国需要一个强大而有效的模型来管理无障碍停车位。我们提出了一个名为 "SHINE "的新颖系统，它使用基于深度学习的物体检测算法来检测车辆、车牌和残疾人徽章（以下简称卡、徽章或通行证），然后通过与中央服务器协调来验证使用无障碍车位的权利。这个模型，达到92.16%的平均精度，被认为可以解决无障碍车位滥用的问题。
The enhancement of science and technology has helped expand urban cities like never before. Due to the undeniable benefits of owning a private vehicle, the number of cars has rocketed in many parts of the world, including South Korea. However, these gradual increments in the number of vehicles lead to parking-related problems, including the abuse of disabled parking spaces (referred to as accessible parking spaces hereafter). Due to the high frame rate of surveillance cameras, traditional license plate recognition (LPR) systems are ineffective in real-time. On the other hand, natural and artificial noise and differences in lighting and weather conditions make detection and recognition difficult for these systems. With the growing concept of parking 4.0, many sensors, IoT and deep learning-based approaches have been applied to automatic LPR and parking management systems. However, the studies show a need for a robust and efficient model for managing accessible parking spaces in South Korea. We have proposed a novel system called 'SHINE', which uses the deep learning-based object detection algorithm for detecting the vehicle, license plate, and disability badges (referred to as cards, badges, or access badges hereafter) and then authenticates the rights to use the accessible parking spaces by coordinating with the central server. This model, achieving 92.16% mean average precision, is believed to solve the problem of accessible parking space abuse.
【5】带完成度的神经共邻用于链接预测
Neural Common Neighbor with Completion for Link Prediction
作者： Xiyuan Wang et.al
链接： https:// arxiv.org/abs/2302.0089 0
摘要： 尽管香草消息传递神经网络（MPNN）在各种图任务中表现突出，但在链接预测任务中通常是失败的，因为它只使用两个单独目标节点的表示，而忽略了它们之间的成对关系。为了捕捉成对关系，一些模型将人工特征添加到输入图中，并使用MPNN的输出来产生成对的表示。相比之下，其他模型直接使用人工特征作为成对表示。尽管这种简化避免了对每个链接单独应用GNN，从而提高了可扩展性，但由于手工制作的和不可学习的成对特征，这些模型仍然有很大的性能提升空间。为了在保持可扩展性的同时提升性能，我们提出了神经共邻（NCN），它使用可学习的成对表示。为了进一步提升NCN，我们研究了未观察到的链接问题。图的不完全性无处不在，导致训练集和测试集之间的分布偏移、共邻信息的丢失以及模型的性能下降。因此，我们提出了两种干预方法：共邻完成和目标链接删除。将这两种方法与NCN相结合，我们提出了带完成度的神经共邻（NCNC）。NCN和NCNC以很大的幅度超过了最近的强大基线。NCNC在链接预测任务中实现了最先进的性能。
Despite its outstanding performance in various graph tasks, vanilla Message Passing Neural Network (MPNN) usually fails in link prediction tasks, as it only uses representations of two individual target nodes and ignores the pairwise relation between them. To capture the pairwise relations, some models add manual features to the input graph and use the output of MPNN to produce pairwise representations. In contrast, others directly use manual features as pairwise representations. Though this simplification avoids applying a GNN to each link individually and thus improves scalability, these models still have much room for performance improvement due to the hand-crafted and unlearnable pairwise features. To upgrade performance while maintaining scalability, we propose Neural Common Neighbor (NCN), which uses learnable pairwise representations. To further boost NCN, we study the unobserved link problem. The incompleteness of the graph is ubiquitous and leads to distribution shifts between the training and test set, loss of common neighbor information, and performance degradation of models. Therefore, we propose two intervention methods: common neighbor completion and target link removal. Combining the two methods with NCN, we propose Neural Common Neighbor with Completion (NCNC). NCN and NCNC outperform recent strong baselines by large margins. NCNC achieves state-of-the-art performance in link prediction tasks.
【6】QR-CLIP：为位置和时间推理引入明确的开放世界知识
QR-CLIP: Introducing Explicit Open-World Knowledge for Location and Time Reasoning
作者： Weimin Shi et.al
链接： https:// arxiv.org/abs/2302.0095 2
摘要： 日常图像可能传达抽象的含义，需要我们记忆并从中推断出深刻的信息。为了鼓励这种类似人类的推理，在这项工作中，我们教机器预测它的拍摄地点和时间，而不是执行传统的分割或分类等基本任务。受霍恩的QR理论的启发，我们设计了一个新颖的QR-CLIP模型，包括两个部分。1）数量模块首先回顾更多的开放世界知识作为候选语言输入；2）相关性模块仔细估计视觉和语言线索并推断出位置和时间。实验显示了我们的QR-CLIP的有效性，在位置和时间推理方面，它比以前的SOTA在每个任务上的表现平均高出约10%和130%的相对提升。这项研究为位置和时间推理奠定了技术基础，并表明有效引入开放世界的知识是这些任务的灵丹妙药之一。
Daily images may convey abstract meanings that require us to memorize and infer profound information from them. To encourage such human-like reasoning, in this work, we teach machines to predict where and when it was taken rather than performing basic tasks like traditional segmentation or classification. Inspired by Horn's QR theory, we designed a novel QR-CLIP model consisting of two components: 1) the Quantity module first retrospects more open-world knowledge as the candidate language inputs; 2) the Relevance module carefully estimates vision and language cues and infers the location and time. Experiments show our QR-CLIP's effectiveness, and it outperforms the previous SOTA on each task by an average of about 10% and 130% relative lift in terms of location and time reasoning. This study lays a technical foundation for location and time reasoning and suggests that effectively introducing open-world knowledge is one of the panaceas for the tasks.
【7】训练神经网络架构的能源效率：实证研究
Energy Efficiency of Training Neural Network Architectures: An Empirical Study
作者： Yinlena Xu et.al
链接： https:// arxiv.org/abs/2302.0096 7
摘要： 深度学习模型的评估传统上集中在准确性、F1得分和相关措施等标准上。越来越多的高计算能力的环境允许创建更深层次和更复杂的模型。然而，训练这种模型所需的计算带来了巨大的碳足迹。在这项工作中，我们通过使用深度卷积神经网络的实证研究，研究DL模型架构与其在训练过程中产生的能源消耗和二氧化碳排放方面的环境影响之间的关系。具体来说，我们研究了：（1）架构和计算所在的位置对能源消耗和排放的影响；（2）准确性和能源效率之间的权衡；以及（3）使用基于软件和基于硬件的工具测量能源消耗的方法上的差异。
The evaluation of Deep Learning models has traditionally focused on criteria such as accuracy, F1 score, and related measures. The increasing availability of high computational power environments allows the creation of deeper and more complex models. However, the computations needed to train such models entail a large carbon footprint. In this work, we study the relations between DL model architectures and their environmental impact in terms of energy consumed and CO2 emissions produced during training by means of an empirical study using Deep Convolutional Neural Networks. Concretely, we study: (i) the impact of the architecture and the location where the computations are hosted on the energy consumption and emissions produced; (ii) the trade-off between accuracy and energy efficiency; and (iii) the difference on the method of measurement of the energy consumed using software-based and hardware-based tools.
【8】用于早期诊断阿尔茨海默病的语义连贯性标志物
Semantic Coherence Markers for the Early Diagnosis of the Alzheimer Disease
作者： Davide Colla et.al
链接： https:// arxiv.org/abs/2302.0102 5
摘要： 在这项工作中，我们探讨了如何利用语言模型来分析语言，并通过plexity度量来区分精神障碍者和健康人。复杂度最初被认为是一种信息论的衡量标准，用来评估一个给定的语言模型在多大程度上适合预测一个文本序列，或者说，一个词序列在多大程度上适合一个特定的语言模型。我们对公开的数据进行了广泛的实验，并采用了不同的语言模型，如N-grams，从2-grams到5-grams，以及GPT-2，一种基于转化器的语言模型。我们研究了复杂度分数是否可用于区分健康受试者和阿尔茨海默病（AD）受试者的转录本。我们表现最好的模型在对阿尔茨海默病患者和对照组受试者进行分类时达到了完全的准确性和F分数（精度/特异性和召回/敏感性均为1.00）。这些结果表明，迷惑性可以是一个有价值的分析指标，有可能应用于支持精神障碍症状的早期诊断。
In this work we explore how language models can be employed to analyze language and discriminate between mentally impaired and healthy subjects through the perplexity metric. Perplexity was originally conceived as an information-theoretic measure to assess how much a given language model is suited to predict a text sequence or, equivalently, how much a word sequence fits into a specific language model. We carried out an extensive experimentation with the publicly available data, and employed language models as diverse as N-grams, from 2-grams to 5-grams, and GPT-2, a transformer-based language model. We investigated whether perplexity scores may be used to discriminate between the transcripts of healthy subjects and subjects suffering from Alzheimer Disease (AD). Our best performing models achieved full accuracy and F-score (1.00 in both precision/specificity and recall/sensitivity) in categorizing subjects from both the AD class and control subjects. These results suggest that perplexity can be a valuable analytical metrics with potential application to supporting early diagnosis of symptoms of mental disorders.
【9】因果解除和链接预测
Causal Lifting and Link Prediction
作者： Leonardo Cotta et.al
链接： https:// arxiv.org/abs/2302.0119 8
摘要： 目前最先进的链接预测的因果模型假定有一套内在的节点因素--在节点出生时定义的先天特性--支配着图中链接的因果演变。然而，在一些因果任务中，链接的形成是路径依赖的，即链接干预的结果取决于现有的链接。例如，在一个在线零售商的顾客-产品图中，85英寸电视广告（处理）的效果可能取决于消费者是否已经有了85英寸电视。不幸的是，现有的因果关系方法在这些情况下是不切实际的。链接之间的级联功能依赖（由于路径依赖）要么无法识别，要么需要不切实际的控制变量数量。为了弥补这一缺陷，这项工作开发了第一个能够处理链接预测中路径依赖的因果模型。它引入了因果提升的概念，这是因果模型中的一个不变性，一旦满足，就可以利用有限的干预数据识别因果联系预测查询。在估计方面，我们展示了结构性成对嵌入--一种基于对称性的图中节点对的联合表示--如何表现出较低的偏差，并正确表示任务的因果结构，而不是现有的节点嵌入方法，如GNNs和矩阵分解。最后，我们在因果联系预测任务的三个不同场景下的四个数据集上验证了我们的理论发现：知识库完成、协方差矩阵估计和消费者-产品推荐。
Current state-of-the-art causal models for link prediction assume an underlying set of inherent node factors -- an innate characteristic defined at the node's birth -- that governs the causal evolution of links in the graph. In some causal tasks, however, link formation is path-dependent, i.e., the outcome of link interventions depends on existing links. For instance, in the customer-product graph of an online retailer, the effect of an 85-inch TV ad (treatment) likely depends on whether the costumer already has an 85-inch TV. Unfortunately, existing causal methods are impractical in these scenarios. The cascading functional dependencies between links (due to path dependence) are either unidentifiable or require an impractical number of control variables. In order to remedy this shortcoming, this work develops the first causal model capable of dealing with path dependencies in link prediction. It introduces the concept of causal lifting, an invariance in causal models that, when satisfied, allows the identification of causal link prediction queries using limited interventional data. On the estimation side, we show how structural pairwise embeddings -- a type of symmetry-based joint representation of node pairs in a graph -- exhibit lower bias and correctly represent the causal structure of the task, as opposed to existing node embedding methods, e.g., GNNs and matrix factorization. Finally, we validate our theoretical findings on four datasets under three different scenarios for causal link prediction tasks: knowledge base completion, covariance matrix estimation and consumer-product recommendations.
【10】天线调谐的图谱Q网络的多代理强化学习
Multi-agent Reinforcement Learning with Graph Q-Networks for Antenna Tuning
作者： Maxime Bouton et.al
链接： https:// arxiv.org/abs/2302.0119 9
摘要： 未来几代的移动网络预计将包含越来越多的天线，其复杂性和参数也越来越多。优化这些参数对于确保网络的良好性能是必要的。移动网络的规模使得使用人工干预或手工设计的策略来优化天线参数成为一种挑战。强化学习是解决这一挑战的有希望的技术，但现有的方法往往使用局部优化来扩展到大型网络部署。我们提出了一种新的多代理强化学习算法，在全球范围内优化移动网络的配置。通过使用价值分解方法，我们的算法可以从全局奖励函数中进行训练，而不是依赖不同小区的网络性能的临时分解。该算法使用一个图形神经网络架构，该架构可以通用于不同的网络拓扑结构并学习协调行为。我们在模拟环境中的天线倾斜调谐问题和联合倾斜和功率控制问题上实证证明了该算法的性能。
Future generations of mobile networks are expected to contain more and more antennas with growing complexity and more parameters. Optimizing these parameters is necessary for ensuring the good performance of the network. The scale of mobile networks makes it challenging to optimize antenna parameters using manual intervention or hand-engineered strategies. Reinforcement learning is a promising technique to address this challenge but existing methods often use local optimizations to scale to large network deployments. We propose a new multi-agent reinforcement learning algorithm to optimize mobile network configurations globally. By using a value decomposition approach, our algorithm can be trained from a global reward function instead of relying on an ad-hoc decomposition of the network performance across the different cells. The algorithm uses a graph neural network architecture which generalizes to different network topologies and learns coordination behaviors. We empirically demonstrate the performance of the algorithm on an antenna tilt tuning problem and a joint tilt and power control problem in a simulated environment.
【11】使用变异模式分解的时态融合变压器用于风力发电的预测
Temporal fusion transformer using variational mode decomposition for wind power forecasting
作者： Meiyu Jiang et.al
链接： https:// arxiv.org/abs/2302.0122 2
摘要： 风力涡轮机的功率输出取决于多种因素，包括不同高度的风速、风向、温度和涡轮机特性。尤其是风速和风向，具有复杂的周期，波动剧烈，导致风功率输出的巨大不确定性。本研究使用变模分解（VMD）来分解风功率序列，使用时态融合变压器（TFT）来预测未来1h、3h和6h的风功率。实验结果表明，VMD优于其他分解算法，TFT模型优于其他分解模型。
The power output of a wind turbine depends on a variety of factors, including wind speed at different heights, wind direction, temperature and turbine properties. Wind speed and direction, in particular, have complex cycles and fluctuate dramatically, leading to large uncertainties in wind power output. This study uses variational mode decomposition (VMD) to decompose the wind power series and Temporal fusion transformer (TFT) to forecast wind power for the next 1h, 3h and 6h. The experimental results show that VMD outperforms other decomposition algorithms and the TFT model outperforms other decomposition models.
【12】扩散模型容易受到成员推理攻击吗？
Are Diffusion Models Vulnerable to Membership Inference Attacks?
作者： Jinhao Duan et.al
链接： https:// arxiv.org/abs/2302.0131 6
摘要： 基于扩散的生成模型在图像合成方面显示出巨大的潜力，但对它们可能带来的安全和隐私风险却缺乏研究。在本文中，我们研究了扩散模型对成员推断攻击（MIAs）的脆弱性，这是一个常见的隐私问题。我们的结果表明，现有的为GANs或VAE设计的MIA对扩散模型基本上是无效的，这是因为不适用的场景（例如，要求GANs的判别器）或不适当的假设（例如，合成图像和成员图像之间的距离更近）。为了解决这个问题，我们提出了阶梯式误差比较成员推断（SecMI），这是一个黑盒式的MIA，通过评估每个时间步的前向过程后验估计的匹配度来推断成员资格。SecMI遵循MIA中常见的过拟合假设，即成员样本通常具有较小的估计误差，与保留样本相比。我们既考虑了标准的扩散模型，如DDPM，也考虑了文本到图像的扩散模型，如稳定的扩散。实验结果表明，我们的方法在六种不同的数据集上以高置信度精确推断出这两种情况下的成员身份
Diffusion-based generative models have shown great potential for image synthesis, but there is a lack of research on the security and privacy risks they may pose. In this paper, we investigate the vulnerability of diffusion models to Membership Inference Attacks (MIAs), a common privacy concern. Our results indicate that existing MIAs designed for GANs or VAE are largely ineffective on diffusion models, either due to inapplicable scenarios (e.g., requiring the discriminator of GANs) or inappropriate assumptions (e.g., closer distances between synthetic images and member images). To address this gap, we propose Step-wise Error Comparing Membership Inference (SecMI), a black-box MIA that infers memberships by assessing the matching of forward process posterior estimation at each timestep. SecMI follows the common overfitting assumption in MIA where member samples normally have smaller estimation errors, compared with hold-out samples. We consider both the standard diffusion models, e.g., DDPM, and the text-to-image diffusion models, e.g., Stable Diffusion. Experimental results demonstrate that our methods precisely infer the membership with high confidence on both of the two scenarios across six different datasets
【13】FV-MgNet: 用于可解释时间序列预测的全连接V型周期MgNet
FV-MgNet: Fully Connected V-cycle MgNet for Interpretable Time Series Forecasting
作者： Jianqing Zhu et.al
链接： https:// arxiv.org/abs/2302.0041 9
摘要： 通过研究受限线性模型的迭代方法，我们提出了一类新的全连接V型周期MgNet，用于长期时间序列预测，这是预测中最困难的任务之一。MgNet是一个CNN模型，它是基于解决离散偏微分方程（PDEs）的多网格（MG）方法而提出的用于图像分类。我们在现有的MgNet中用全连接操作取代卷积操作，然后将其应用于预测问题。受MG中V型循环结构的启发，我们进一步提出了FV-MgNet，即全连接MgNet的V型循环版本，以分层提取特征。通过评估FV-MgNet在流行数据集上的性能并与最先进的模型进行比较，我们发现FV-MgNet以更少的内存占用和更快的推理速度取得了更好的结果。此外，我们开发了消融实验来证明FV-MgNet的结构是众多变体中的最佳选择。
By investigating iterative methods for a constrained linear model, we propose a new class of fully connected V-cycle MgNet for long-term time series forecasting, which is one of the most difficult tasks in forecasting. MgNet is a CNN model that was proposed for image classification based on the multigrid (MG) methods for solving discretized partial differential equations (PDEs). We replace the convolutional operations with fully connected operations in the existing MgNet and then apply them to forecasting problems. Motivated by the V-cycle structure in MG, we further propose the FV-MgNet, a V-cycle version of the fully connected MgNet, to extract features hierarchically. By evaluating the performance of FV-MgNet on popular data sets and comparing it with state-of-the-art models, we show that the FV-MgNet achieves better results with less memory usage and faster inference speed. In addition, we develop ablation experiments to demonstrate that the structure of FV-MgNet is the best choice among the many variants.
【14】具有强化性能控制和可观察性的MLOps
MLOps with enhanced performance control and observability
作者： Indradumna Banerjee et.al
链接： https:// arxiv.org/abs/2302.0106 1
摘要： 在过去的几年里，数据的爆炸性增长和不断增加的复杂性，使得MLOps系统更容易失败，需要在这些系统中嵌入新的工具来避免这种失败。在这个演示中，我们将在MLOps系统的可观察性模块中介绍一些关键的工具，这些工具针对的是一些困难的问题，如数据drfit和模型版本控制以实现最佳的模型选择。我们相信，在我们的MLOps管道中整合这些功能，将大大有助于建立一个对早期ML系统故障免疫的强大系统。
The explosion of data and its ever increasing complexity in the last few years, has made MLOps systems more prone to failure, and new tools need to be embedded in such systems to avoid such failure. In this demo, we will introduce crucial tools in the observability module of a MLOps system that target difficult issues like data drfit and model version control for optimum model selection. We believe integrating these features in our MLOps pipeline would go a long way in building a robust system immune to early stage ML system failures.
【15】统计学和机器学习模型对近实时日排放预测的比较研究
A comparative study of statistical and machine learning models on near-real-time daily emissions prediction
作者： Xiangqian Li
链接： https:// arxiv.org/abs/2302.0056 1
摘要： 二氧化碳排放量的快速上升是导致全球变暖和气候变化的主要原因，对人类的生存构成巨大威胁，并对全球生态系统产生深远影响。因此，及时准确地预测和分析其变化趋势，有效地控制二氧化碳的排放是非常必要的，从而为二氧化碳排放减缓措施提供参考。本文旨在根据2020年1月1日至2022年9月30日中国所有行业（电力、工业、地面运输、住宅、国内航空、国际航空）的单变量日时间序列数据，选择一个合适的模型来预测近实时的日排放量。我们提出了六个预测模型，其中包括三个统计模型。灰色预测（GM(1,1)）、自回归综合移动平均（ARIMA）和带外生因素的季节性自回归综合移动平均（SARIMAX）；三个机器学习模型：人工神经网络（ANN）、随机森林（RF）和长短期记忆（LSTM）。为了评估这些模型的性能，有五个标准。平均平方误差（MSE）、平均平方根误差（RMSE）、平均绝对误差（MAE）、平均绝对百分比误差（MAPE）和决定系数（）被导入并详细讨论。在结果中，三个机器学习模型的表现优于三个统计模型，其中LSTM模型在日排放预测的五个标准值上表现最好，其MSE值为3.5179e-04，RMSE值为0.0187，MAE值为0.0140，MAPE值为14.8291%，判定系数为0.9844。
The rapid ascent in carbon dioxide emissions is a major cause of global warming and climate change, which pose a huge threat to human survival and impose far-reaching influence on the global ecosystem. Therefore, it is very necessary to effectively control carbon dioxide emissions by accurately predicting and analyzing the change trend timely, so as to provide a reference for carbon dioxide emissions mitigation measures. This paper is aiming to select a suitable model to predict the near-real-time daily emissions based on univariate daily time-series data from January 1st, 2020 to September 30st, 2022 of all sectors (Power, Industry, Ground Transport, Residential, Domestic Aviation, International Aviation) in China. We proposed six prediction models, which including three statistical models: Grey prediction (GM(1,1)), autoregressive integrated moving average (ARIMA) and seasonal autoregressive integrated moving average with exogenous factors (SARIMAX); three machine learning models: artificial neural network (ANN), random forest (RF) and long short term memory (LSTM). To evaluate the performance of these models, five criteria: Mean Squared Error (MSE), Root Mean Squared Error (RMSE), Mean Absolute Error (MAE), Mean Absolute Percentage Error (MAPE) and Coefficient of Determination () are imported and discussed in detail. In the results, three machine learning models perform better than that three statistical models, in which LSTM model performs the best on five criteria values for daily emissions prediction with the 3.5179e-04 MSE value, 0.0187 RMSE value, 0.0140 MAE value, 14.8291% MAPE value and 0.9844 value.
【16】图解化:用图解式人工智能解释对假设进行归纳推理的合理性
Diagrammatization: Rationalizing with diagrammatic AI explanations for abductive reasoning on hypotheses
作者： Brian Y. Lim et.al
链接： https:// arxiv.org/abs/2302.0124 1
摘要： 为可解释人工智能（XAI）开发了许多可视化的东西，但它们往往需要用户进一步推理才能解释。我们认为，XAI应该支持归纳推理--对最佳解释的推理--用图解推理来表达假设的产生和评估。受Peircean图解推理和5步归纳过程的启发，我们提出了Diagrammatization，一种基于领域假设提供图解、归纳解释的方法。我们为一个临床应用实现了DiagramNet，从心脏听诊中预测诊断，并以基于形状的杂音图进行解释。在建模研究中，我们发现DiagramNet不仅能提供忠实的杂音形状解释，而且比基线模型有更好的预测性能。我们在对医科学生的定性用户研究中进一步证明了图解的有用性，表明与临床相关的图解比技术性的显著性图解更受欢迎。这项工作有助于为以用户为中心的XAI提供符合领域惯例的归纳性解释。
Many visualizations have been developed for explainable AI (XAI), but they often require further reasoning by users to interpret. We argue that XAI should support abductive reasoning - inference to the best explanation - with diagrammatic reasoning to convey hypothesis generation and evaluation. Inspired by Peircean diagrammatic reasoning and the 5-step abduction process, we propose Diagrammatization, an approach to provide diagrammatic, abductive explanations based on domain hypotheses. We implemented DiagramNet for a clinical application to predict diagnoses from heart auscultation, and explain with shape-based murmur diagrams. In modeling studies, we found that DiagramNet not only provides faithful murmur shape explanations, but also has better prediction performance than baseline models. We further demonstrate the usefulness of diagrammatic explanations in a qualitative user study with medical students, showing that clinically-relevant, diagrammatic explanations are preferred over technical saliency map explanations. This work contributes insights into providing domain-conventional abductive explanations for user-centric XAI.
