AWQ: Activation-aware Weight Quantization for On-Device LLM Compression and Acceleration.

Ji Lin 0002 Jiaming Tang Haotian Tang Shang Yang Wei-Ming Chen Wei-Chen Wang 0002 Guangxuan Xiao Xingyu Dang Chuang Gan 0001 Song Han 0003 AWQ: Activation-aware Weight Quantization for On-Device LLM Compression and Acceleration. 2024 MLSys https://proceedings.mlsys.org/paper_files/paper/2024/hash/42a452cbafa9dd64e9ba4aa95cc1ef21-Abstract-Conference.html conf/mlsys/2024 db/conf/mlsys/mlsys2024.html#0002TTYCWXDG024