Transform From Scratch

This project is created during my internship at Prof. Kenjiro Taura Laboratory at University of Tokyo. The main focus of this projet is to compare the performance achieved of Chapel to C++, Python in training a transformer model. The Chapel and C++code is implemented from scratch and the python is obtained from GitHub

Branch

• Master : contain all versions including the cuda implementation from scratch as well, which is a side project.
• SingleThread : contain the single thread implementation of C++, Chapel, 2 versions of Pytorch.
• MultiThread : contain the multithread thread implementation of C++, Chapel, 2 versions of Pytorch.

Build

C++

cd ./C++
make -j

Chapel

cd ./Chapel
chpl ./main.chpl --fast

cuda

cd ./cuda
make -j

python For Pytorch A (orignal implementation from GitHub)

cd ./python/Transformer-from-Scratch
python ./train.py

For Pytorch B (orignal implementation from GitHub with transformerBlock replaced by torch.nn.transformer)

cd ./python/Pytorch
python ./train.py