chunked/content_aware_chunking/_deepspeed/chunk_105.txt

When offload_optimizer is enabled, you could use a non-DeepSpeed optimizer (except for LAMB) as long as it has both a CPU and GPU implementation.