AI On-Device Inference: Core ML, ONNX Runtime, MLC LLM

End-of-lesson check

15 questions · take it digitally for instant feedback at tendril.neural-forge.io/learn/quiz/end-tools-AI-and-on-device-inference-platforms-creators

1. What factor most constrains the choice of LLM model size for mobile deployment?

   1. The model's training dataset size
   2. Platform hardware capabilities such as memory and processing power
   3. Developer personal preference
   4. The year the model was originally released

2. Which task is AI specifically capable of assisting with in on-device LLM deployment planning?

   1. Installing and configuring models on user devices
   2. Replacing mobile-platform engineering teams
   3. Physically testing thermal performance on devices
   4. Generating device-tier benchmarking plans

3. What is a key limitation when using AI to plan on-device LLM deployment?

   1. AI cannot calculate how many parameters a model has
   2. AI is unable to suggest any optimization techniques
   3. AI cannot predict thermal and battery behavior without actual device testing
   4. AI cannot work with text-based models

4. Which framework is specifically designed for native integration with iOS devices?

   1. TensorFlow Lite
   2. Core ML
   3. MLC LLM
   4. ONNX Runtime Mobile

5. What distinguishes on-device inference from cloud-based AI processing?

   1. Models must be larger than 10 billion parameters
   2. Models run directly on the user's device without needing a network connection
   3. Models require constant internet connectivity to function
   4. Models cannot be updated after deployment

6. When comparing deployment frameworks for a 4B parameter model, which factor is essential to evaluate?

   1. Supported quantization methods that reduce model size
   2. The color scheme of each framework's logo
   3. The number of social media users of each framework
   4. The founding year of each company

7. What does model quantization accomplish in on-device deployment?

   1. It increases the number of model parameters
   2. It reduces the memory footprint by using lower precision numbers
   3. It makes the model require more computing power
   4. It converts text models into image models

8. What challenge do developers face when deploying on-device LLM updates?

   1. Updates can be pushed instantaneously at any time
   2. Updates are unnecessary for on-device models
   3. Update cadence is constrained by app store review and platform release policies
   4. Updates require rewriting the entire model

9. Why might an on-device LLM that performs well in benchmarks still fail in real-world use?

   1. Real users do not actually use LLMs
   2. Benchmarks are always inaccurate
   3. Models perform better in laboratories than in real life
   4. Benchmarks do not capture thermal throttling and battery consumption under sustained load

10. What role does AI play in the platform comparison process for on-device deployment?

    1. Physically testing each device
    2. Drafting comparison matrices on supported models and quantization options
    3. Writing the actual mobile application code
    4. Making final engineering decisions about which platform to use

11. Which platform is primarily designed for deploying large language models across diverse hardware including mobile devices?

    1. Scikit-learn
    2. ONNX Runtime Mobile
    3. Core ML
    4. MLC LLM

12. What is a primary advantage of ONNX Runtime Mobile for cross-platform deployment?

    1. It requires rewriting models in a specific programming language
    2. It only works on Apple devices
    3. It supports deployment to both iOS and Android from a single model format
    4. It cannot run transformer-based models

13. What does the 'model conversion path' refer to in framework selection?

    1. The sequence of user interface screens
    2. The path users take to download the app
    3. The route data travels through a neural network
    4. The process of transforming a trained model into a format the deployment platform can run

14. Why is thermal behavior difficult to predict before device testing?

    1. Thermal performance depends on complex interactions between model, device hardware, and ambient conditions
    2. All devices have identical thermal characteristics
    3. Thermal behavior is always predictable from model size
    4. Thermal issues only occur in server environments

15. Which statement about on-device inference feasibility is most accurate?

    1. On-device inference is only possible on desktop computers
    2. Inference always requires cloud servers
    3. On-device LLM inference is now feasible on phones and laptops due to hardware advances
    4. Mobile devices cannot run any AI models