Scaling Issue in new Pytorch Build method

[vishwascm]

Hi, I am reporting my observations on two methods involved. They are with the following two commits 1) ac72dca, which uses Pytorch 2.5.0 and 2) 2dd009f , which uses Pytorch 2.6.0.

Steps followed in first method:

cd Tool-Solutions/docker/pytorch-aarch64
./build.sh --build-type pytorch --onednn acl

The above build will create new image with pytorch installed. I will create a container from it and run detr hugging face model (hug_detr.py script given at end). OMP_NUM_THREADS=64 python hug_detr.py is the command used to run the script, the script is run for 10 times and average time is taken. For 32 threads OMP_NUM_THREADS=32 is used.

Steps followed in second method:

cd Tool-Solutions/docker/pytorch-aarch64
./build.sh
sudo ./dockerize.sh results/torch-2.6.0.dev20241104-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl

The above build will create new image with pytorch installed. I will run detr hugging face model (hug_detr.py script given at end). OMP_NUM_THREADS=64 python hug_detr.py is the command used to run the script, the script is run for 10 times and average time is taken. For 32 threads OMP_NUM_THREADS=32 is used.

Detr Model Script (hug_detr.py):

from transformers import AutoImageProcessor, DetrForObjectDetection
import torch
from time import time
import numpy as np
from PIL import Image
import requests
import os
from torch.profiler import profile, record_function, ProfilerActivity

url = "http://images.cocodataset.org/val2017/000000039769.jpg"
image = Image.open(requests.get(url, stream=True).raw)
image_processor = AutoImageProcessor.from_pretrained("facebook/detr-resnet-50")
model = DetrForObjectDetection.from_pretrained("facebook/detr-resnet-50")
inputs = image_processor(images=image, return_tensors="pt")

with torch.no_grad():
    for i in range(4):
        print(f"Warm up of cycle {i}",end='\r', flush=True)
        outputs=model(**inputs)

times=[]
itr=6
with torch.no_grad():
    for i in range(itr):
        print(f"Actual run of cycle {i}",end='\r', flush=True)
        start_time = time()
        outputs = model(**inputs)
        times.append(time() - start_time)

print("average time (milliseconds) for DETR Inference: %.2f " %(np.average(times[0:itr])*1e3))