Author: Max White

Authors: Aviv Bick、Kevin Y. Li、Eric P. Xing、J. Zico Kolter、Albert Gu Paper: https://arxiv.org/abs/2408.10189 Distilling Quadratic Knowledge to Subquadratic Models: A Deep Dive into MOHAWK Introduction Transformer architectures have revolutionized natural language processing (NLP) by providing state-of-the-art performance across various tasks. However, their quadratic-time self-attention mechanism poses significant computational challenges, especially for long sequences. This has led to the exploration of subquadratic alternatives, such as state space models (SSMs). Despite their potential, these models have not benefited from the extensive training resources that Transformers have. This paper introduces MOHAWK, a novel method to distill knowledge from pretrained Transformer models into subquadratic models…

Authors: Jialu Wang、Kaichen Zhou、Andrew Markham、Niki Trigoni Paper: https://arxiv.org/abs/2408.09680 Introduction In the realm of automation and intelligent systems, accurate location information is paramount. This is especially true for terminal devices and edge-cloud IoT systems, such as autonomous vehicles and augmented reality applications. However, achieving reliable positioning across diverse IoT applications remains a challenge due to significant training costs and the necessity of densely collected data. To address these issues, the study introduces MambaLoc, an innovative model that leverages the selective state space (SSM) model for visual localization. MambaLoc demonstrates exceptional training efficiency and robustness in sparse data environments, making it a…

Authors: Nastaran Bassamzadeh、Chhaya Methani Paper: https://arxiv.org/abs/2408.08335 Introduction The paper “Plan with Code: Comparing approaches for robust NL to DSL generation” by Nastaran Bassamzadeh and Chhaya Methani from Microsoft Corporation delves into the intricacies of generating Domain Specific Languages (DSLs) from Natural Language (NL) inputs. The authors highlight the challenges faced by Large Language Models (LLMs) in generating accurate DSLs, particularly due to the high rate of hallucinations and syntax errors when dealing with custom function names. The focus of the study is on workflow automation in the Robotic Process Automation (RPA) domain, presenting optimizations for using Retrieval Augmented Generation (RAG)…

Authors: Elisa Böhl、Stefan Ellmauthaler、Sarah Alice Gaggl Paper: https://arxiv.org/abs/2408.08150 Introduction Answer Set Programming (ASP) is a declarative programming paradigm that has gained traction in both theoretical and practical applications. ASP is particularly useful for solving complex problems such as planning and configuration. This paper explores different techniques to reuse logic program parts (multi-shot) by solving the arcade game Snake. The game is interesting because winning can be assured by solving the NP-hard problem of Hamiltonian Cycles. The paper demonstrates five implementations in clingo and compares their performance. Preliminaries Answer Set Programming ASP is based on stable model semantics and involves defining…

Authors: Stefano Puliti、Emily R. Lines、Jana Müllerová、Julian Frey、Zoe Schindler、Adrian Straker、Matthew J. Allen、Lukas Winiwarter、Nataliia Rehush、Hristina Hristova、Brent Murray、Kim Calders、Louise Terryn、Nicholas Coops、Bernhard Höfle、Samuli Junttila、Martin Krůček、Grzegorz Krok、Kamil Král、Shaun R. Levick、Linda Luck、Azim Missarov、Martin Mokroš、Harry J. F. Owen、Krzysztof Stereńczak、Timo P. Pitkänen、Nicola Puletti、Ninni Saarinen、Chris Hopkinson、Chiara Torresan、Enrico Tomelleri、Hannah Weiser、Rasmus Astrup Paper: https://arxiv.org/abs/2408.06507 Benchmarking Tree Species Classification from Proximally-Sensed Laser Scanning Data: Introducing the FOR-species20K Dataset Introduction In recent years, there has been a significant push towards automating the retrieval of key forest variables from various remotely sensed data, with laser scanning technology providing the most detailed and accurate 3D information. Laser scanning and other 3D technologies have demonstrated exceptional…

Authors: Jinseong Park、Seungyun Lee、Woojin Jeong、Yujin Choi、Jaewook Lee Paper: https://arxiv.org/abs/2408.06672 Introduction Time series generation is a crucial task in various real-world applications such as simulation, data augmentation, and hypothesis testing. Traditional methods like variational autoencoders (VAE) and generative adversarial networks (GANs) have been widely used, but diffusion models have recently emerged as a powerful alternative. These models are known for their ability to generate high-quality and diverse time series data. However, standard Gaussian priors used in these models may not be suitable for time series data due to their unique characteristics like fixed time order and data scaling. In this paper,…

Authors: Nursena Koprucu、Meher Shashwat Nigam、Shicheng Xu、Biruk Abere、Gabriele Dominici、Andrew Rodriguez、Sharvaree Vadgam、Berfin Inal、Alberto Tono Paper: https://arxiv.org/abs/2408.06693 Introduction Recent advancements in deep generative models have significantly improved performance in both classification and out-of-distribution (OOD) classification for images. Inspired by Geoffrey Hinton’s emphasis on generative modeling, this paper explores the use of 3D diffusion models for object classification. The proposed approach, “Diffusion Classifier for 3D Objects” (DC3DO), leverages the density estimates from these models to enable zero-shot classification of 3D shapes without additional training. This method demonstrates a 12.5% improvement on average compared to its multi-view counterparts, showcasing superior multimodal reasoning. Methodology Multi-View Diffusion…