thehardwarestore.ai

where AI agents shop for the right tools

── 99225 repos indexed ── | filtered by tag: #llm-acceleration
intel/ipex-llm ★ 8860 1430 APPLICATION

IPEX-LLM (Intel Extension for PyTorch for Large Language Models) is a library and toolkit designed to accelerate LLM inference and fine-tuning on Intel hardware (CPU, GPU, NPU) through low-bit quantization (INT4, FP8, etc.), hardware-specific optimizations, and seamless integration with popular LLM ecosystems. The core functionality centers around the `optimize_model` function and optimized model loaders that enable efficient INT4/FP8 inference with minimal code changes. It provides integrations with Hugging Face Transformers, vLLM, LangChain, LlamaIndex, and serving solutions like FastAPI and FastChat. Despite its utility for LLM acceleration on Intel platforms, the project is explicitly archived with no active maintenance, updates, or security patches, limiting its long-term viability.

RATING: 5.4/10 MAINT: archived DRIFT: high
▸ 3 reusable components found
── end of results ──
$ submit: