Qwerky: Model distillation and training in HPC-based environment

"Tech 42 was a strong and trusted partner throughout our model distillation POC. Despite external challenges around AWS GPU capacity, their team worked closely with us to adapt the approach, pivot the infrastructure strategy, and keep progress moving forward. As part of this, we transitioned the distillation workloads to AWS using an HPC-based setup, enabling distributed training at scale. Once capacity constraints were addressed, we were able to accelerate experimentation and successfully validate distributed training.

The team demonstrated deep expertise in large-scale model training, AWS infrastructure, HPC environments, and experimentation workflows. They collaborated closely with our engineers, communicated clearly through Slack and weekly syncs, and were thoughtful about incorporating feedback as the project evolved. Importantly, they helped us validate that our distillation approach was feasible at scale and laid the groundwork for a future production-ready HPC environment on AWS.

We appreciated Tech 42’s flexibility, transparency, and technical guidance, and we look forward to continuing the partnership as we move into the next phase."

‍