Synchronous training across multiple replicas on one machine.