SpatialLM is a 3D large language model designed to process 3D point cloud data and generate structured 3D scene understanding outputs. The system provides end-to-end tools for indoor scene understanding, including layout estimation (walls, doors, windows) and 3D object detection (furniture and indoor objects). The architecture consists of a modular Python library with CLI entry points for inference, training, evaluation, and visualization. It combines point cloud encoders (SceneScript or Sonata) with LLM backbones (Llama or Qwen variants) through a projector network. The system handles the complex task of aligning 3D geometric data with language understanding to produce structured outputs representing architectural layouts and object bounding boxes. Key innovations include the use of space-filling curve encodings (Hilbert and Z-order) for positional encoding in the point cloud transformer, a composable transformation pipeline for point cloud preprocessing, and a sophisticated layout representation system that handles the discrete/continuous coordinate transformations required for architectural elements.
This repository is a comprehensive family of GUI automation agents (Mobile-Agent series) developed by Tongyi Lab, Alibaba Group. It provides end-to-end solutions for automating interactions across mobile (Android), desktop (Windows/macOS/Linux), and web platforms using vision-language models. The system perceives GUI elements through screenshots, grounds natural language instructions to actionable commands (click, type, scroll), and executes tasks via platform-specific controllers. The architecture includes multiple agent versions (v1-v3.5, E, UI-S1, PC-Agent) with shared core modules for perception, action parsing, and controller interfaces, alongside training frameworks (VERL) and evaluation suites (Android World, MiniWoB, Mind2Web). Key entry points are version-specific run scripts (e.g., run_ma35.py) and API services for remote agent control. The system relies on external dependencies like ADB for Android, PyAutoGUI for desktop, Playwright for web, and VLMs (Qwen series) for reasoning.