Register
Login
Resources
Docs Blog Datasets Glossary Case Studies Tutorials & Webinars
Product
Data Engine LLMs Platform Enterprise
Pricing Explore
Connect to our Discord channel

accelconf.yaml 663 B

You have to be logged in to leave a comment. Sign In
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
  1. compute_environment: LOCAL_MACHINE
  2. debug: false
  3. distributed_type: FSDP
  4. downcast_bf16: 'no'
  5. fsdp_config:
  6. fsdp_auto_wrap_policy: SIZE_BASED_WRAP
  7. fsdp_backward_prefetch: BACKWARD_PRE
  8. fsdp_cpu_ram_efficient_loading: false
  9. fsdp_forward_prefetch: false
  10. fsdp_min_num_params: 20000
  11. fsdp_offload_params: true
  12. fsdp_sharding_strategy: FULL_SHARD
  13. fsdp_state_dict_type: SHARDED_STATE_DICT
  14. fsdp_sync_module_states: true
  15. fsdp_use_orig_params: true
  16. machine_rank: 0
  17. main_training_function: main
  18. mixed_precision: fp16
  19. num_machines: 1
  20. num_processes: 2
  21. rdzv_backend: static
  22. same_network: true
  23. tpu_env: []
  24. tpu_use_cluster: false
  25. tpu_use_sudo: false
  26. use_cpu: false
Tip!

Press p or to see the previous file or, n or to see the next file

Comments

Loading...