Stable Diffusion on CoreML Apple Silicon

https://machinelearning.apple.com/research/stable-diffusion-coreml-apple-silicon https://towardsdatascience.com/turn-vs-code-into-a-one-stop-shop-for-ml-experiments-49c97c47db27 https://github.com/huggingface/diffusers

Abstract

For simplicity I created a conda env which includes the apple dependencies for stable diffusion. Following the Setup Process is simpler in this way.

Setup Process

conda env create -f environment.yml
pip install -r requirements.txt
huggingface-cli login

DVC Experiment Tracking

dvc init

Conversion Process

python -m python_coreml_stable_diffusion.torch2coreml --convert-unet --convert-text-encoder --convert-vae-decoder --convert-safety-checker -o models

For convenience you could add this to dvc.yaml

  convert_model:
    cmd: python -m python_coreml_stable_diffusion.torch2coreml --convert-unet --convert-text-encoder --convert-vae-decoder --convert-safety-checker -o models 
    outs:
      - models

Usage

python -m python_coreml_stable_diffusion.pipeline --prompt "a photo of an astronaut riding a horse on mars" -i models -o data/processed --compute-unit ALL --seed 193

DVC Pipelines

Setting up a dvc remote + dvc.yaml with the above code is rather simple and can be done straight forward.

dvc repro

Misc

conda env export > environment.yml