← On-Device LLMs Hands-On: Prepare and Compress

🔒

Sign in to check your access, or get the course to unlock all lectures.

Section 6 · Lecture 2 · 9:12

int4 woq inference execution providers

← why onnx optimum export ep benchmark graph speculative app tips →

Curriculum

0/19 done

0. Welcome

welcome and course overview1m

1. On-Device AI and Mobile Hardware

Materials for this section

1 notebook

01_ondevice_ai_intro.ipynb

2. Model Compression Theory

Materials for this section

1 notebook

02_model_compression_theory.ipynb

3. Small Language Models

Materials for this section

1 notebook

03_slm_overview.ipynb

4. llama.cpp and GGUF

Materials for this section

1 notebook

04_llama_cpp_gguf_quantization.ipynb

5. MLC LLM

Materials for this section

1 notebook

05_mlc_llm_compilation.ipynb

6. ONNX Runtime Conversion

Materials for this section

1 notebook

06_onnx_llm_conversion.ipynb

Course-wide files

Applies to every section

1 doc

Notebook errata — licence warnings

Progress is remembered in this browser only; a lecture ticks at 90% watched. Materials download after purchase.

int4 woq inference execution providers — On-Device LLMs Hands-On: Prepare and Compress | SOTAAZ Blog