build_chatbot_on_spr.ipynb

{"cells":[{"cell_type":"markdown","metadata":{},"source":["NeuralChat is a customizable chat framework designed to create user own chatbot within few minutes on multiple architectures. This notebook is used to demonstrate how to build a talking chatbot on 4th Generation of Intel® Xeon® Scalable Processors Sapphire Rapids.\n","\n","The 4th Generation of Intel® Xeon® Scalable processor provides two instruction sets viz. AMX_BF16 and AMX_INT8 which provides acceleration for bfloat16 and int8 operations respectively."]},{"cell_type":"markdown","metadata":{},"source":["# Prepare Environment"]},{"cell_type":"markdown","metadata":{},"source":["Install intel extension for transformers:"]},{"cell_type":"code","execution_count":1,"metadata":{"execution":{"iopub.execute_input":"2024-07-13T14:45:10.670999Z","iopub.status.busy":"2024-07-13T14:45:10.670623Z","iopub.status.idle":"2024-07-13T14:45:28.426795Z","shell.execute_reply":"2024-07-13T14:45:28.425915Z","shell.execute_reply.started":"2024-07-13T14:45:10.670967Z"},"trusted":true},"outputs":[{"name":"stdout","output_type":"stream","text":["Collecting intel-extension-for-transformers\n","  Downloading intel_extension_for_transformers-1.4.2-cp310-cp310-manylinux_2_28_x86_64.whl.metadata (26 kB)\n","Requirement already satisfied: packaging in /opt/conda/lib/python3.10/site-packages (from intel-extension-for-transformers) (21.3)\n","Requirement already satisfied: numpy in /opt/conda/lib/python3.10/site-packages (from intel-extension-for-transformers) (1.26.4)\n","Collecting schema (from intel-extension-for-transformers)\n","  Downloading schema-0.7.7-py2.py3-none-any.whl.metadata (34 kB)\n","Requirement already satisfied: pyyaml in /opt/conda/lib/python3.10/site-packages (from intel-extension-for-transformers) (6.0.1)\n","Collecting neural-compressor (from intel-extension-for-transformers)\n","  Downloading neural_compressor-2.6-py3-none-any.whl.metadata (15 kB)\n","Requirement already satisfied: transformers in /opt/conda/lib/python3.10/site-packages (from intel-extension-for-transformers) (4.42.3)\n","Requirement already satisfied: deprecated>=1.2.13 in /opt/conda/lib/python3.10/site-packages (from neural-compressor->intel-extension-for-transformers) (1.2.14)\n","Requirement already satisfied: opencv-python-headless in /opt/conda/lib/python3.10/site-packages (from neural-compressor->intel-extension-for-transformers) (4.10.0.84)\n","Requirement already satisfied: pandas in /opt/conda/lib/python3.10/site-packages (from neural-compressor->intel-extension-for-transformers) (2.2.2)\n","Requirement already satisfied: Pillow in /opt/conda/lib/python3.10/site-packages (from neural-compressor->intel-extension-for-transformers) (9.5.0)\n","Requirement already satisfied: prettytable in /opt/conda/lib/python3.10/site-packages (from neural-compressor->intel-extension-for-transformers) (3.9.0)\n","Requirement already satisfied: psutil in /opt/conda/lib/python3.10/site-packages (from neural-compressor->intel-extension-for-transformers) (5.9.3)\n","Requirement already satisfied: py-cpuinfo in /opt/conda/lib/python3.10/site-packages (from neural-compressor->intel-extension-for-transformers) (9.0.0)\n","Requirement already satisfied: requests in /opt/conda/lib/python3.10/site-packages (from neural-compressor->intel-extension-for-transformers) (2.32.3)\n","Requirement already satisfied: scikit-learn in /opt/conda/lib/python3.10/site-packages (from neural-compressor->intel-extension-for-transformers) (1.2.2)\n","Collecting pycocotools (from neural-compressor->intel-extension-for-transformers)\n","  Downloading pycocotools-2.0.8-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (1.1 kB)\n","Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /opt/conda/lib/python3.10/site-packages (from packaging->intel-extension-for-transformers) (3.1.1)\n","Requirement already satisfied: filelock in /opt/conda/lib/python3.10/site-packages (from transformers->intel-extension-for-transformers) (3.13.1)\n","Requirement already satisfied: huggingface-hub<1.0,>=0.23.2 in /opt/conda/lib/python3.10/site-packages (from transformers->intel-extension-for-transformers) (0.23.4)\n","Requirement already satisfied: regex!=2019.12.17 in /opt/conda/lib/python3.10/site-packages (from transformers->intel-extension-for-transformers) (2023.12.25)\n","Requirement already satisfied: safetensors>=0.4.1 in /opt/conda/lib/python3.10/site-packages (from transformers->intel-extension-for-transformers) (0.4.3)\n","Requirement already satisfied: tokenizers<0.20,>=0.19 in /opt/conda/lib/python3.10/site-packages (from transformers->intel-extension-for-transformers) (0.19.1)\n","Requirement already satisfied: tqdm>=4.27 in /opt/conda/lib/python3.10/site-packages (from transformers->intel-extension-for-transformers) (4.66.4)\n","Requirement already satisfied: wrapt<2,>=1.10 in /opt/conda/lib/python3.10/site-packages (from deprecated>=1.2.13->neural-compressor->intel-extension-for-transformers) (1.14.1)\n","Requirement already satisfied: fsspec>=2023.5.0 in /opt/conda/lib/python3.10/site-packages (from huggingface-hub<1.0,>=0.23.2->transformers->intel-extension-for-transformers) (2024.5.0)\n","Requirement already satisfied: typing-extensions>=3.7.4.3 in /opt/conda/lib/python3.10/site-packages (from huggingface-hub<1.0,>=0.23.2->transformers->intel-extension-for-transformers) (4.9.0)\n","Requirement already satisfied: python-dateutil>=2.8.2 in /opt/conda/lib/python3.10/site-packages (from pandas->neural-compressor->intel-extension-for-transformers) (2.9.0.post0)\n","Requirement already satisfied: pytz>=2020.1 in /opt/conda/lib/python3.10/site-packages (from pandas->neural-compressor->intel-extension-for-transformers) (2023.3.post1)\n","Requirement already satisfied: tzdata>=2022.7 in /opt/conda/lib/python3.10/site-packages (from pandas->neural-compressor->intel-extension-for-transformers) (2023.4)\n","Requirement already satisfied: wcwidth in /opt/conda/lib/python3.10/site-packages (from prettytable->neural-compressor->intel-extension-for-transformers) (0.2.13)\n","Requirement already satisfied: matplotlib>=2.1.0 in /opt/conda/lib/python3.10/site-packages (from pycocotools->neural-compressor->intel-extension-for-transformers) (3.7.5)\n","Requirement already satisfied: charset-normalizer<4,>=2 in /opt/conda/lib/python3.10/site-packages (from requests->neural-compressor->intel-extension-for-transformers) (3.3.2)\n","Requirement already satisfied: idna<4,>=2.5 in /opt/conda/lib/python3.10/site-packages (from requests->neural-compressor->intel-extension-for-transformers) (3.6)\n","Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/conda/lib/python3.10/site-packages (from requests->neural-compressor->intel-extension-for-transformers) (1.26.18)\n","Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.10/site-packages (from requests->neural-compressor->intel-extension-for-transformers) (2024.7.4)\n","Requirement already satisfied: scipy>=1.3.2 in /opt/conda/lib/python3.10/site-packages (from scikit-learn->neural-compressor->intel-extension-for-transformers) (1.11.4)\n","Requirement already satisfied: joblib>=1.1.1 in /opt/conda/lib/python3.10/site-packages (from scikit-learn->neural-compressor->intel-extension-for-transformers) (1.4.2)\n","Requirement already satisfied: threadpoolctl>=2.0.0 in /opt/conda/lib/python3.10/site-packages (from scikit-learn->neural-compressor->intel-extension-for-transformers) (3.2.0)\n","Requirement already satisfied: contourpy>=1.0.1 in /opt/conda/lib/python3.10/site-packages (from matplotlib>=2.1.0->pycocotools->neural-compressor->intel-extension-for-transformers) (1.2.0)\n","Requirement already satisfied: cycler>=0.10 in /opt/conda/lib/python3.10/site-packages (from matplotlib>=2.1.0->pycocotools->neural-compressor->intel-extension-for-transformers) (0.12.1)\n","Requirement already satisfied: fonttools>=4.22.0 in /opt/conda/lib/python3.10/site-packages (from matplotlib>=2.1.0->pycocotools->neural-compressor->intel-extension-for-transformers) (4.47.0)\n","Requirement already satisfied: kiwisolver>=1.0.1 in /opt/conda/lib/python3.10/site-packages (from matplotlib>=2.1.0->pycocotools->neural-compressor->intel-extension-for-transformers) (1.4.5)\n","Requirement already satisfied: six>=1.5 in /opt/conda/lib/python3.10/site-packages (from python-dateutil>=2.8.2->pandas->neural-compressor->intel-extension-for-transformers) (1.16.0)\n","Downloading intel_extension_for_transformers-1.4.2-cp310-cp310-manylinux_2_28_x86_64.whl (45.3 MB)\n","\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m45.3/45.3 MB\u001b[0m \u001b[31m38.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m:00:01\u001b[0m00:01\u001b[0m\n","\u001b[?25hDownloading neural_compressor-2.6-py3-none-any.whl (1.5 MB)\n","\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.5/1.5 MB\u001b[0m \u001b[31m59.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[?25hDownloading schema-0.7.7-py2.py3-none-any.whl (18 kB)\n","Downloading pycocotools-2.0.8-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (427 kB)\n","\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m427.8/427.8 kB\u001b[0m \u001b[31m27.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[?25hInstalling collected packages: schema, pycocotools, neural-compressor, intel-extension-for-transformers\n","Successfully installed intel-extension-for-transformers-1.4.2 neural-compressor-2.6 pycocotools-2.0.8 schema-0.7.7\n"]}],"source":["!pip install intel-extension-for-transformers"]},{"cell_type":"markdown","metadata":{},"source":["Install Requirements:"]},{"cell_type":"code","execution_count":2,"metadata":{"execution":{"iopub.execute_input":"2024-07-13T14:45:37.342357Z","iopub.status.busy":"2024-07-13T14:45:37.342025Z","iopub.status.idle":"2024-07-13T14:46:30.603452Z","shell.execute_reply":"2024-07-13T14:46:30.602572Z","shell.execute_reply.started":"2024-07-13T14:45:37.342331Z"},"trusted":true},"outputs":[{"name":"stdout","output_type":"stream","text":["Cloning into 'intel-extension-for-transformers'...\n","remote: Enumerating objects: 1681988, done.\u001b[K\n","remote: Counting objects: 100% (116638/116638), done.\u001b[K\n","remote: Compressing objects: 100% (12340/12340), done.\u001b[K\n","remote: Total 1681988 (delta 63056), reused 114702 (delta 61456), pack-reused 1565350\u001b[K\n","Receiving objects: 100% (1681988/1681988), 594.70 MiB | 32.85 MiB/s, done.\n","Resolving deltas: 100% (898963/898963), done.\n","Updating files: 100% (3217/3217), done.\n"]}],"source":["!git clone https://github.com/intel/intel-extension-for-transformers.git"]},{"cell_type":"code","execution_count":3,"metadata":{"execution":{"iopub.execute_input":"2024-07-13T14:47:11.300773Z","iopub.status.busy":"2024-07-13T14:47:11.299832Z","iopub.status.idle":"2024-07-13T14:50:22.109763Z","shell.execute_reply":"2024-07-13T14:50:22.108840Z","shell.execute_reply.started":"2024-07-13T14:47:11.300725Z"},"trusted":true},"outputs":[{"name":"stdout","output_type":"stream","text":["/content/intel-extension-for-transformers/intel_extension_for_transformers/neural_chat\n","Looking in indexes: https://pypi.org/simple, https://pytorch-extension.intel.com/release-whl/stable/cpu/us/\n","Collecting accelerate==0.28.0 (from -r requirements_cpu.txt (line 2))\n","  Downloading accelerate-0.28.0-py3-none-any.whl.metadata (18 kB)\n","Collecting cchardet (from -r requirements_cpu.txt (line 3))\n","  Downloading cchardet-2.1.7.tar.gz (653 kB)\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m653.6/653.6 kB\u001b[0m \u001b[31m16.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m00:01\u001b[0m\n","\u001b[?25h  Preparing metadata (setup.py) ... \u001b[?25ldone\n","\u001b[?25hCollecting einops (from -r requirements_cpu.txt (line 4))\n","  Downloading einops-0.8.0-py3-none-any.whl.metadata (12 kB)\n","Collecting evaluate (from -r requirements_cpu.txt (line 5))\n","  Downloading evaluate-0.4.2-py3-none-any.whl.metadata (9.3 kB)\n","Requirement already satisfied: fastapi in /opt/conda/lib/python3.10/site-packages (from -r requirements_cpu.txt (line 6)) (0.108.0)\n","Collecting fschat==0.2.32 (from -r requirements_cpu.txt (line 7))\n","  Downloading fschat-0.2.32-py3-none-any.whl.metadata (20 kB)\n","Requirement already satisfied: huggingface_hub in /opt/conda/lib/python3.10/site-packages (from -r requirements_cpu.txt (line 8)) (0.23.4)\n","Collecting intel_extension_for_pytorch==2.3.0 (from -r requirements_cpu.txt (line 9))\n","  Downloading https://intel-extension-for-pytorch.s3.amazonaws.com/ipex_stable/cpu/./intel_extension_for_pytorch-2.3.0%2Bcpu-cp310-cp310-linux_x86_64.whl (98.3 MB)\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m98.3/98.3 MB\u001b[0m \u001b[31m12.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m00:01\u001b[0m00:01\u001b[0m\n","\u001b[?25hCollecting lm-eval==0.4.3 (from -r requirements_cpu.txt (line 10))\n","  Downloading lm_eval-0.4.3-py3-none-any.whl.metadata (38 kB)\n","Requirement already satisfied: neural-compressor in /opt/conda/lib/python3.10/site-packages (from -r requirements_cpu.txt (line 11)) (2.6)\n","Collecting neural_speed==1.0a0 (from -r requirements_cpu.txt (line 12))\n","  Downloading neural_speed-1.0a0-cp310-cp310-manylinux_2_28_x86_64.whl.metadata (6.3 kB)\n","Collecting numpy==1.23.5 (from -r requirements_cpu.txt (line 13))\n","  Downloading numpy-1.23.5-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (2.3 kB)\n","Collecting oneccl_bind_pt (from -r requirements_cpu.txt (line 14))\n","  Downloading https://intel-extension-for-pytorch.s3.amazonaws.com/ipex_stable/cpu/./oneccl_bind_pt-2.3.0%2Bcpu-cp310-cp310-linux_x86_64.whl (41.3 MB)\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m41.3/41.3 MB\u001b[0m \u001b[31m26.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m00:01\u001b[0m00:01\u001b[0m\n","\u001b[?25hCollecting optimum (from -r requirements_cpu.txt (line 15))\n","  Downloading optimum-1.21.2-py3-none-any.whl.metadata (19 kB)\n","Collecting optimum-intel (from -r requirements_cpu.txt (line 16))\n","  Downloading optimum_intel-1.18.1-py3-none-any.whl.metadata (16 kB)\n","Collecting peft==0.6.2 (from -r requirements_cpu.txt (line 17))\n","  Downloading peft-0.6.2-py3-none-any.whl.metadata (23 kB)\n","Collecting pydantic==1.10.13 (from -r requirements_cpu.txt (line 18))\n","  Downloading pydantic-1.10.13-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (149 kB)\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m149.6/149.6 kB\u001b[0m \u001b[31m9.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[?25hRequirement already satisfied: python-dotenv in /opt/conda/lib/python3.10/site-packages (from -r requirements_cpu.txt (line 19)) (1.0.0)\n","Collecting python-multipart (from -r requirements_cpu.txt (line 20))\n","  Downloading python_multipart-0.0.9-py3-none-any.whl.metadata (2.5 kB)\n","Collecting rouge_score (from -r requirements_cpu.txt (line 21))\n","  Downloading rouge_score-0.1.2.tar.gz (17 kB)\n","  Preparing metadata (setup.py) ... \u001b[?25ldone\n","\u001b[?25hCollecting sacremoses (from -r requirements_cpu.txt (line 22))\n","  Downloading sacremoses-0.1.1-py3-none-any.whl.metadata (8.3 kB)\n","Collecting shortuuid (from -r requirements_cpu.txt (line 23))\n","  Downloading shortuuid-1.0.13-py3-none-any.whl.metadata (5.8 kB)\n","Requirement already satisfied: starlette in /opt/conda/lib/python3.10/site-packages (from -r requirements_cpu.txt (line 24)) (0.32.0.post1)\n","Collecting tiktoken==0.4.0 (from -r requirements_cpu.txt (line 25))\n","  Downloading tiktoken-0.4.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (5.2 kB)\n","Collecting torch==2.3.0 (from -r requirements_cpu.txt (line 26))\n","  Downloading torch-2.3.0-cp310-cp310-manylinux1_x86_64.whl.metadata (26 kB)\n","Collecting torchaudio==2.3.0 (from -r requirements_cpu.txt (line 27))\n","  Downloading torchaudio-2.3.0-cp310-cp310-manylinux1_x86_64.whl.metadata (6.4 kB)\n","Requirement already satisfied: transformers>=4.35.2 in /opt/conda/lib/python3.10/site-packages (from -r requirements_cpu.txt (line 28)) (4.42.3)\n","Collecting transformers_stream_generator (from -r requirements_cpu.txt (line 29))\n","  Downloading transformers-stream-generator-0.0.5.tar.gz (13 kB)\n","  Preparing metadata (setup.py) ... \u001b[?25ldone\n","\u001b[?25hRequirement already satisfied: uvicorn in /opt/conda/lib/python3.10/site-packages (from -r requirements_cpu.txt (line 30)) (0.25.0)\n","Collecting yacs (from -r requirements_cpu.txt (line 31))\n","  Downloading yacs-0.1.8-py3-none-any.whl.metadata (639 bytes)\n","Requirement already satisfied: packaging>=20.0 in /opt/conda/lib/python3.10/site-packages (from accelerate==0.28.0->-r requirements_cpu.txt (line 2)) (21.3)\n","Requirement already satisfied: psutil in /opt/conda/lib/python3.10/site-packages (from accelerate==0.28.0->-r requirements_cpu.txt (line 2)) (5.9.3)\n","Requirement already satisfied: pyyaml in /opt/conda/lib/python3.10/site-packages (from accelerate==0.28.0->-r requirements_cpu.txt (line 2)) (6.0.1)\n","Requirement already satisfied: safetensors>=0.3.1 in /opt/conda/lib/python3.10/site-packages (from accelerate==0.28.0->-r requirements_cpu.txt (line 2)) (0.4.3)\n","Requirement already satisfied: aiohttp in /opt/conda/lib/python3.10/site-packages (from fschat==0.2.32->-r requirements_cpu.txt (line 7)) (3.9.1)\n","Requirement already satisfied: httpx in /opt/conda/lib/python3.10/site-packages (from fschat==0.2.32->-r requirements_cpu.txt (line 7)) (0.27.0)\n","Collecting markdown2[all] (from fschat==0.2.32->-r requirements_cpu.txt (line 7))\n","  Downloading markdown2-2.5.0-py2.py3-none-any.whl.metadata (2.2 kB)\n","Collecting nh3 (from fschat==0.2.32->-r requirements_cpu.txt (line 7))\n","  Downloading nh3-0.2.18-cp37-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (1.7 kB)\n","Requirement already satisfied: prompt-toolkit>=3.0.0 in /opt/conda/lib/python3.10/site-packages (from fschat==0.2.32->-r requirements_cpu.txt (line 7)) (3.0.42)\n","Requirement already satisfied: requests in /opt/conda/lib/python3.10/site-packages (from fschat==0.2.32->-r requirements_cpu.txt (line 7)) (2.32.3)\n","Requirement already satisfied: rich>=10.0.0 in /opt/conda/lib/python3.10/site-packages (from fschat==0.2.32->-r requirements_cpu.txt (line 7)) (13.7.0)\n","Requirement already satisfied: datasets>=2.16.0 in /opt/conda/lib/python3.10/site-packages (from lm-eval==0.4.3->-r requirements_cpu.txt (line 10)) (2.20.0)\n","Collecting jsonlines (from lm-eval==0.4.3->-r requirements_cpu.txt (line 10))\n","  Downloading jsonlines-4.0.0-py3-none-any.whl.metadata (1.6 kB)\n","Requirement already satisfied: numexpr in /opt/conda/lib/python3.10/site-packages (from lm-eval==0.4.3->-r requirements_cpu.txt (line 10)) (2.10.1)\n","Requirement already satisfied: pybind11>=2.6.2 in /opt/conda/lib/python3.10/site-packages (from lm-eval==0.4.3->-r requirements_cpu.txt (line 10)) (2.13.1)\n","Collecting pytablewriter (from lm-eval==0.4.3->-r requirements_cpu.txt (line 10))\n","  Downloading pytablewriter-1.2.0-py3-none-any.whl.metadata (37 kB)\n","Collecting sacrebleu>=1.5.0 (from lm-eval==0.4.3->-r requirements_cpu.txt (line 10))\n","  Downloading sacrebleu-2.4.2-py3-none-any.whl.metadata (58 kB)\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m58.0/58.0 kB\u001b[0m \u001b[31m3.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[?25hRequirement already satisfied: scikit-learn>=0.24.1 in /opt/conda/lib/python3.10/site-packages (from lm-eval==0.4.3->-r requirements_cpu.txt (line 10)) (1.2.2)\n","Collecting sqlitedict (from lm-eval==0.4.3->-r requirements_cpu.txt (line 10))\n","  Downloading sqlitedict-2.1.0.tar.gz (21 kB)\n","  Preparing metadata (setup.py) ... \u001b[?25ldone\n","\u001b[?25hCollecting tqdm-multiprocess (from lm-eval==0.4.3->-r requirements_cpu.txt (line 10))\n","  Downloading tqdm_multiprocess-0.0.11-py3-none-any.whl.metadata (5.7 kB)\n","Requirement already satisfied: zstandard in /opt/conda/lib/python3.10/site-packages (from lm-eval==0.4.3->-r requirements_cpu.txt (line 10)) (0.22.0)\n","Requirement already satisfied: dill in /opt/conda/lib/python3.10/site-packages (from lm-eval==0.4.3->-r requirements_cpu.txt (line 10)) (0.3.8)\n","Collecting word2number (from lm-eval==0.4.3->-r requirements_cpu.txt (line 10))\n","  Downloading word2number-1.1.zip (9.7 kB)\n","  Preparing metadata (setup.py) ... \u001b[?25ldone\n","\u001b[?25hRequirement already satisfied: more-itertools in /opt/conda/lib/python3.10/site-packages (from lm-eval==0.4.3->-r requirements_cpu.txt (line 10)) (10.2.0)\n","Requirement already satisfied: tqdm in /opt/conda/lib/python3.10/site-packages (from peft==0.6.2->-r requirements_cpu.txt (line 17)) (4.66.4)\n","Requirement already satisfied: typing-extensions>=4.2.0 in /opt/conda/lib/python3.10/site-packages (from pydantic==1.10.13->-r requirements_cpu.txt (line 18)) (4.9.0)\n","Requirement already satisfied: regex>=2022.1.18 in /opt/conda/lib/python3.10/site-packages (from tiktoken==0.4.0->-r requirements_cpu.txt (line 25)) (2023.12.25)\n","Requirement already satisfied: filelock in /opt/conda/lib/python3.10/site-packages (from torch==2.3.0->-r requirements_cpu.txt (line 26)) (3.13.1)\n","Requirement already satisfied: sympy in /opt/conda/lib/python3.10/site-packages (from torch==2.3.0->-r requirements_cpu.txt (line 26)) (1.13.0)\n","Requirement already satisfied: networkx in /opt/conda/lib/python3.10/site-packages (from torch==2.3.0->-r requirements_cpu.txt (line 26)) (3.2.1)\n","Requirement already satisfied: jinja2 in /opt/conda/lib/python3.10/site-packages (from torch==2.3.0->-r requirements_cpu.txt (line 26)) (3.1.2)\n","Requirement already satisfied: fsspec in /opt/conda/lib/python3.10/site-packages (from torch==2.3.0->-r requirements_cpu.txt (line 26)) (2024.5.0)\n","Collecting nvidia-cuda-nvrtc-cu12==12.1.105 (from torch==2.3.0->-r requirements_cpu.txt (line 26))\n","  Downloading nvidia_cuda_nvrtc_cu12-12.1.105-py3-none-manylinux1_x86_64.whl.metadata (1.5 kB)\n","Collecting nvidia-cuda-runtime-cu12==12.1.105 (from torch==2.3.0->-r requirements_cpu.txt (line 26))\n","  Downloading nvidia_cuda_runtime_cu12-12.1.105-py3-none-manylinux1_x86_64.whl.metadata (1.5 kB)\n","Collecting nvidia-cuda-cupti-cu12==12.1.105 (from torch==2.3.0->-r requirements_cpu.txt (line 26))\n","  Downloading nvidia_cuda_cupti_cu12-12.1.105-py3-none-manylinux1_x86_64.whl.metadata (1.6 kB)\n","Collecting nvidia-cudnn-cu12==8.9.2.26 (from torch==2.3.0->-r requirements_cpu.txt (line 26))\n","  Downloading nvidia_cudnn_cu12-8.9.2.26-py3-none-manylinux1_x86_64.whl.metadata (1.6 kB)\n","Collecting nvidia-cublas-cu12==12.1.3.1 (from torch==2.3.0->-r requirements_cpu.txt (line 26))\n","  Downloading nvidia_cublas_cu12-12.1.3.1-py3-none-manylinux1_x86_64.whl.metadata (1.5 kB)\n","Collecting nvidia-cufft-cu12==11.0.2.54 (from torch==2.3.0->-r requirements_cpu.txt (line 26))\n","  Downloading nvidia_cufft_cu12-11.0.2.54-py3-none-manylinux1_x86_64.whl.metadata (1.5 kB)\n","Collecting nvidia-curand-cu12==10.3.2.106 (from torch==2.3.0->-r requirements_cpu.txt (line 26))\n","  Downloading nvidia_curand_cu12-10.3.2.106-py3-none-manylinux1_x86_64.whl.metadata (1.5 kB)\n","Collecting nvidia-cusolver-cu12==11.4.5.107 (from torch==2.3.0->-r requirements_cpu.txt (line 26))\n","  Downloading nvidia_cusolver_cu12-11.4.5.107-py3-none-manylinux1_x86_64.whl.metadata (1.6 kB)\n","Collecting nvidia-cusparse-cu12==12.1.0.106 (from torch==2.3.0->-r requirements_cpu.txt (line 26))\n","  Downloading nvidia_cusparse_cu12-12.1.0.106-py3-none-manylinux1_x86_64.whl.metadata (1.6 kB)\n","Collecting nvidia-nccl-cu12==2.20.5 (from torch==2.3.0->-r requirements_cpu.txt (line 26))\n","  Downloading nvidia_nccl_cu12-2.20.5-py3-none-manylinux2014_x86_64.whl.metadata (1.8 kB)\n","Collecting nvidia-nvtx-cu12==12.1.105 (from torch==2.3.0->-r requirements_cpu.txt (line 26))\n","  Downloading nvidia_nvtx_cu12-12.1.105-py3-none-manylinux1_x86_64.whl.metadata (1.7 kB)\n","Collecting triton==2.3.0 (from torch==2.3.0->-r requirements_cpu.txt (line 26))\n","  Downloading triton-2.3.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (1.4 kB)\n","Collecting nvidia-nvjitlink-cu12 (from nvidia-cusolver-cu12==11.4.5.107->torch==2.3.0->-r requirements_cpu.txt (line 26))\n","  Downloading nvidia_nvjitlink_cu12-12.5.82-py3-none-manylinux2014_x86_64.whl.metadata (1.5 kB)\n","Requirement already satisfied: pandas in /opt/conda/lib/python3.10/site-packages (from evaluate->-r requirements_cpu.txt (line 5)) (2.2.2)\n","Requirement already satisfied: xxhash in /opt/conda/lib/python3.10/site-packages (from evaluate->-r requirements_cpu.txt (line 5)) (3.4.1)\n","Requirement already satisfied: multiprocess in /opt/conda/lib/python3.10/site-packages (from evaluate->-r requirements_cpu.txt (line 5)) (0.70.16)\n","Requirement already satisfied: deprecated>=1.2.13 in /opt/conda/lib/python3.10/site-packages (from neural-compressor->-r requirements_cpu.txt (line 11)) (1.2.14)\n","Requirement already satisfied: opencv-python-headless in /opt/conda/lib/python3.10/site-packages (from neural-compressor->-r requirements_cpu.txt (line 11)) (4.10.0.84)\n","Requirement already satisfied: Pillow in /opt/conda/lib/python3.10/site-packages (from neural-compressor->-r requirements_cpu.txt (line 11)) (9.5.0)\n","Requirement already satisfied: prettytable in /opt/conda/lib/python3.10/site-packages (from neural-compressor->-r requirements_cpu.txt (line 11)) (3.9.0)\n","Requirement already satisfied: py-cpuinfo in /opt/conda/lib/python3.10/site-packages (from neural-compressor->-r requirements_cpu.txt (line 11)) (9.0.0)\n","Requirement already satisfied: schema in /opt/conda/lib/python3.10/site-packages (from neural-compressor->-r requirements_cpu.txt (line 11)) (0.7.7)\n","Requirement already satisfied: pycocotools in /opt/conda/lib/python3.10/site-packages (from neural-compressor->-r requirements_cpu.txt (line 11)) (2.0.8)\n","Collecting coloredlogs (from optimum->-r requirements_cpu.txt (line 15))\n","  Downloading coloredlogs-15.0.1-py2.py3-none-any.whl.metadata (12 kB)\n","Requirement already satisfied: sentencepiece in /opt/conda/lib/python3.10/site-packages (from optimum-intel->-r requirements_cpu.txt (line 16)) (0.2.0)\n","Requirement already satisfied: setuptools in /opt/conda/lib/python3.10/site-packages (from optimum-intel->-r requirements_cpu.txt (line 16)) (69.0.3)\n","Requirement already satisfied: scipy in /opt/conda/lib/python3.10/site-packages (from optimum-intel->-r requirements_cpu.txt (line 16)) (1.11.4)\n","Requirement already satisfied: onnx in /opt/conda/lib/python3.10/site-packages (from optimum-intel->-r requirements_cpu.txt (line 16)) (1.16.1)\n","Requirement already satisfied: absl-py in /opt/conda/lib/python3.10/site-packages (from rouge_score->-r requirements_cpu.txt (line 21)) (1.4.0)\n","Requirement already satisfied: nltk in /opt/conda/lib/python3.10/site-packages (from rouge_score->-r requirements_cpu.txt (line 21)) (3.2.4)\n","Requirement already satisfied: six>=1.14.0 in /opt/conda/lib/python3.10/site-packages (from rouge_score->-r requirements_cpu.txt (line 21)) (1.16.0)\n","Requirement already satisfied: click in /opt/conda/lib/python3.10/site-packages (from sacremoses->-r requirements_cpu.txt (line 22)) (8.1.7)\n","Requirement already satisfied: joblib in /opt/conda/lib/python3.10/site-packages (from sacremoses->-r requirements_cpu.txt (line 22)) (1.4.2)\n","Requirement already satisfied: anyio<5,>=3.4.0 in /opt/conda/lib/python3.10/site-packages (from starlette->-r requirements_cpu.txt (line 24)) (4.2.0)\n","Requirement already satisfied: tokenizers<0.20,>=0.19 in /opt/conda/lib/python3.10/site-packages (from transformers>=4.35.2->-r requirements_cpu.txt (line 28)) (0.19.1)\n","Requirement already satisfied: h11>=0.8 in /opt/conda/lib/python3.10/site-packages (from uvicorn->-r requirements_cpu.txt (line 30)) (0.14.0)\n","Requirement already satisfied: idna>=2.8 in /opt/conda/lib/python3.10/site-packages (from anyio<5,>=3.4.0->starlette->-r requirements_cpu.txt (line 24)) (3.6)\n","Requirement already satisfied: sniffio>=1.1 in /opt/conda/lib/python3.10/site-packages (from anyio<5,>=3.4.0->starlette->-r requirements_cpu.txt (line 24)) (1.3.0)\n","Requirement already satisfied: exceptiongroup>=1.0.2 in /opt/conda/lib/python3.10/site-packages (from anyio<5,>=3.4.0->starlette->-r requirements_cpu.txt (line 24)) (1.2.0)\n","Requirement already satisfied: pyarrow>=15.0.0 in /opt/conda/lib/python3.10/site-packages (from datasets>=2.16.0->lm-eval==0.4.3->-r requirements_cpu.txt (line 10)) (16.1.0)\n","Requirement already satisfied: pyarrow-hotfix in /opt/conda/lib/python3.10/site-packages (from datasets>=2.16.0->lm-eval==0.4.3->-r requirements_cpu.txt (line 10)) (0.6)\n","Requirement already satisfied: wrapt<2,>=1.10 in /opt/conda/lib/python3.10/site-packages (from deprecated>=1.2.13->neural-compressor->-r requirements_cpu.txt (line 11)) (1.14.1)\n","Requirement already satisfied: attrs>=17.3.0 in /opt/conda/lib/python3.10/site-packages (from aiohttp->fschat==0.2.32->-r requirements_cpu.txt (line 7)) (23.2.0)\n","Requirement already satisfied: multidict<7.0,>=4.5 in /opt/conda/lib/python3.10/site-packages (from aiohttp->fschat==0.2.32->-r requirements_cpu.txt (line 7)) (6.0.4)\n","Requirement already satisfied: yarl<2.0,>=1.0 in /opt/conda/lib/python3.10/site-packages (from aiohttp->fschat==0.2.32->-r requirements_cpu.txt (line 7)) (1.9.3)\n","Requirement already satisfied: frozenlist>=1.1.1 in /opt/conda/lib/python3.10/site-packages (from aiohttp->fschat==0.2.32->-r requirements_cpu.txt (line 7)) (1.4.1)\n","Requirement already satisfied: aiosignal>=1.1.2 in /opt/conda/lib/python3.10/site-packages (from aiohttp->fschat==0.2.32->-r requirements_cpu.txt (line 7)) (1.3.1)\n","Requirement already satisfied: async-timeout<5.0,>=4.0 in /opt/conda/lib/python3.10/site-packages (from aiohttp->fschat==0.2.32->-r requirements_cpu.txt (line 7)) (4.0.3)\n","Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /opt/conda/lib/python3.10/site-packages (from packaging>=20.0->accelerate==0.28.0->-r requirements_cpu.txt (line 2)) (3.1.1)\n","Requirement already satisfied: wcwidth in /opt/conda/lib/python3.10/site-packages (from prompt-toolkit>=3.0.0->fschat==0.2.32->-r requirements_cpu.txt (line 7)) (0.2.13)\n","Requirement already satisfied: charset-normalizer<4,>=2 in /opt/conda/lib/python3.10/site-packages (from requests->fschat==0.2.32->-r requirements_cpu.txt (line 7)) (3.3.2)\n","Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/conda/lib/python3.10/site-packages (from requests->fschat==0.2.32->-r requirements_cpu.txt (line 7)) (1.26.18)\n","Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.10/site-packages (from requests->fschat==0.2.32->-r requirements_cpu.txt (line 7)) (2024.7.4)\n","Requirement already satisfied: markdown-it-py>=2.2.0 in /opt/conda/lib/python3.10/site-packages (from rich>=10.0.0->fschat==0.2.32->-r requirements_cpu.txt (line 7)) (3.0.0)\n","Requirement already satisfied: pygments<3.0.0,>=2.13.0 in /opt/conda/lib/python3.10/site-packages (from rich>=10.0.0->fschat==0.2.32->-r requirements_cpu.txt (line 7)) (2.17.2)\n","Collecting portalocker (from sacrebleu>=1.5.0->lm-eval==0.4.3->-r requirements_cpu.txt (line 10))\n","  Downloading portalocker-2.10.0-py3-none-any.whl.metadata (8.5 kB)\n","Requirement already satisfied: tabulate>=0.8.9 in /opt/conda/lib/python3.10/site-packages (from sacrebleu>=1.5.0->lm-eval==0.4.3->-r requirements_cpu.txt (line 10)) (0.9.0)\n","Requirement already satisfied: colorama in /opt/conda/lib/python3.10/site-packages (from sacrebleu>=1.5.0->lm-eval==0.4.3->-r requirements_cpu.txt (line 10)) (0.4.6)\n","Requirement already satisfied: lxml in /opt/conda/lib/python3.10/site-packages (from sacrebleu>=1.5.0->lm-eval==0.4.3->-r requirements_cpu.txt (line 10)) (5.2.2)\n","Requirement already satisfied: threadpoolctl>=2.0.0 in /opt/conda/lib/python3.10/site-packages (from scikit-learn>=0.24.1->lm-eval==0.4.3->-r requirements_cpu.txt (line 10)) (3.2.0)\n","Requirement already satisfied: protobuf in /opt/conda/lib/python3.10/site-packages (from transformers[sentencepiece]<4.43.0,>=4.26.0->optimum->-r requirements_cpu.txt (line 15)) (3.20.3)\n","Collecting humanfriendly>=9.1 (from coloredlogs->optimum->-r requirements_cpu.txt (line 15))\n","  Downloading humanfriendly-10.0-py2.py3-none-any.whl.metadata (9.2 kB)\n","Requirement already satisfied: httpcore==1.* in /opt/conda/lib/python3.10/site-packages (from httpx->fschat==0.2.32->-r requirements_cpu.txt (line 7)) (1.0.5)\n","Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/lib/python3.10/site-packages (from jinja2->torch==2.3.0->-r requirements_cpu.txt (line 26)) (2.1.3)\n","Collecting wavedrom (from markdown2[all]->fschat==0.2.32->-r requirements_cpu.txt (line 7))\n","  Downloading wavedrom-2.0.3.post3.tar.gz (137 kB)\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m137.7/137.7 kB\u001b[0m \u001b[31m9.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[?25h  Preparing metadata (setup.py) ... \u001b[?25ldone\n","\u001b[?25hCollecting latex2mathml (from markdown2[all]->fschat==0.2.32->-r requirements_cpu.txt (line 7))\n","  Downloading latex2mathml-3.77.0-py3-none-any.whl.metadata (14 kB)\n","Requirement already satisfied: python-dateutil>=2.8.2 in /opt/conda/lib/python3.10/site-packages (from pandas->evaluate->-r requirements_cpu.txt (line 5)) (2.9.0.post0)\n","Requirement already satisfied: pytz>=2020.1 in /opt/conda/lib/python3.10/site-packages (from pandas->evaluate->-r requirements_cpu.txt (line 5)) (2023.3.post1)\n","Requirement already satisfied: tzdata>=2022.7 in /opt/conda/lib/python3.10/site-packages (from pandas->evaluate->-r requirements_cpu.txt (line 5)) (2023.4)\n","Requirement already satisfied: matplotlib>=2.1.0 in /opt/conda/lib/python3.10/site-packages (from pycocotools->neural-compressor->-r requirements_cpu.txt (line 11)) (3.7.5)\n","Collecting DataProperty<2,>=1.0.1 (from pytablewriter->lm-eval==0.4.3->-r requirements_cpu.txt (line 10))\n","  Downloading DataProperty-1.0.1-py3-none-any.whl.metadata (11 kB)\n","Collecting mbstrdecoder<2,>=1.0.0 (from pytablewriter->lm-eval==0.4.3->-r requirements_cpu.txt (line 10))\n","  Downloading mbstrdecoder-1.1.3-py3-none-any.whl.metadata (4.0 kB)\n","Collecting pathvalidate<4,>=2.3.0 (from pytablewriter->lm-eval==0.4.3->-r requirements_cpu.txt (line 10))\n","  Downloading pathvalidate-3.2.0-py3-none-any.whl.metadata (11 kB)\n","Collecting tabledata<2,>=1.3.1 (from pytablewriter->lm-eval==0.4.3->-r requirements_cpu.txt (line 10))\n","  Downloading tabledata-1.3.3-py3-none-any.whl.metadata (3.7 kB)\n","Collecting tcolorpy<1,>=0.0.5 (from pytablewriter->lm-eval==0.4.3->-r requirements_cpu.txt (line 10))\n","  Downloading tcolorpy-0.1.6-py3-none-any.whl.metadata (6.4 kB)\n","Collecting typepy<2,>=1.3.2 (from typepy[datetime]<2,>=1.3.2->pytablewriter->lm-eval==0.4.3->-r requirements_cpu.txt (line 10))\n","  Downloading typepy-1.3.2-py3-none-any.whl.metadata (9.3 kB)\n","Requirement already satisfied: mpmath<1.4,>=1.1.0 in /opt/conda/lib/python3.10/site-packages (from sympy->torch==2.3.0->-r requirements_cpu.txt (line 26)) (1.3.0)\n","Requirement already satisfied: mdurl~=0.1 in /opt/conda/lib/python3.10/site-packages (from markdown-it-py>=2.2.0->rich>=10.0.0->fschat==0.2.32->-r requirements_cpu.txt (line 7)) (0.1.2)\n","Requirement already satisfied: contourpy>=1.0.1 in /opt/conda/lib/python3.10/site-packages (from matplotlib>=2.1.0->pycocotools->neural-compressor->-r requirements_cpu.txt (line 11)) (1.2.0)\n","Requirement already satisfied: cycler>=0.10 in /opt/conda/lib/python3.10/site-packages (from matplotlib>=2.1.0->pycocotools->neural-compressor->-r requirements_cpu.txt (line 11)) (0.12.1)\n","Requirement already satisfied: fonttools>=4.22.0 in /opt/conda/lib/python3.10/site-packages (from matplotlib>=2.1.0->pycocotools->neural-compressor->-r requirements_cpu.txt (line 11)) (4.47.0)\n","Requirement already satisfied: kiwisolver>=1.0.1 in /opt/conda/lib/python3.10/site-packages (from matplotlib>=2.1.0->pycocotools->neural-compressor->-r requirements_cpu.txt (line 11)) (1.4.5)\n","Collecting chardet<6,>=3.0.4 (from mbstrdecoder<2,>=1.0.0->pytablewriter->lm-eval==0.4.3->-r requirements_cpu.txt (line 10))\n","  Downloading chardet-5.2.0-py3-none-any.whl.metadata (3.4 kB)\n","Collecting svgwrite (from wavedrom->markdown2[all]->fschat==0.2.32->-r requirements_cpu.txt (line 7))\n","  Downloading svgwrite-1.4.3-py3-none-any.whl.metadata (8.8 kB)\n","Downloading accelerate-0.28.0-py3-none-any.whl (290 kB)\n","\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m290.1/290.1 kB\u001b[0m \u001b[31m18.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[?25hDownloading fschat-0.2.32-py3-none-any.whl (211 kB)\n","\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m211.4/211.4 kB\u001b[0m \u001b[31m14.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[?25hDownloading lm_eval-0.4.3-py3-none-any.whl (1.7 MB)\n","\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.7/1.7 MB\u001b[0m \u001b[31m53.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[?25hDownloading neural_speed-1.0a0-cp310-cp310-manylinux_2_28_x86_64.whl (23.1 MB)\n","\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m23.1/23.1 MB\u001b[0m \u001b[31m70.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m:00:01\u001b[0m00:01\u001b[0m\n","\u001b[?25hDownloading numpy-1.23.5-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.1 MB)\n","\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m17.1/17.1 MB\u001b[0m \u001b[31m80.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m:00:01\u001b[0m00:01\u001b[0m\n","\u001b[?25hDownloading peft-0.6.2-py3-none-any.whl (174 kB)\n","\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m174.7/174.7 kB\u001b[0m \u001b[31m11.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[?25hDownloading pydantic-1.10.13-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.1 MB)\n","\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m3.1/3.1 MB\u001b[0m \u001b[31m77.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m:00:01\u001b[0m\n","\u001b[?25hDownloading tiktoken-0.4.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.7 MB)\n","\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.7/1.7 MB\u001b[0m \u001b[31m62.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[?25hDownloading torch-2.3.0-cp310-cp310-manylinux1_x86_64.whl (779.1 MB)\n","\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m779.1/779.1 MB\u001b[0m \u001b[31m2.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m:00:01\u001b[0m00:01\u001b[0m\n","\u001b[?25hDownloading torchaudio-2.3.0-cp310-cp310-manylinux1_x86_64.whl (3.3 MB)\n","\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m3.3/3.3 MB\u001b[0m \u001b[31m9.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m0:00:01\u001b[0m\n","\u001b[?25hDownloading nvidia_cublas_cu12-12.1.3.1-py3-none-manylinux1_x86_64.whl (410.6 MB)\n","\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m410.6/410.6 MB\u001b[0m \u001b[31m3.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m:00:01\u001b[0m00:01\u001b[0m\n","\u001b[?25hDownloading nvidia_cuda_cupti_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (14.1 MB)\n","\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m14.1/14.1 MB\u001b[0m \u001b[31m21.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m:00:01\u001b[0m00:01\u001b[0m\n","\u001b[?25hDownloading nvidia_cuda_nvrtc_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (23.7 MB)\n","\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m23.7/23.7 MB\u001b[0m \u001b[31m69.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m:00:01\u001b[0m00:01\u001b[0m\n","\u001b[?25hDownloading nvidia_cuda_runtime_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (823 kB)\n","\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m823.6/823.6 kB\u001b[0m \u001b[31m38.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[?25hDownloading nvidia_cudnn_cu12-8.9.2.26-py3-none-manylinux1_x86_64.whl (731.7 MB)\n","\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m731.7/731.7 MB\u001b[0m \u001b[31m1.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m:00:01\u001b[0m00:01\u001b[0m\n","\u001b[?25hDownloading nvidia_cufft_cu12-11.0.2.54-py3-none-manylinux1_x86_64.whl (121.6 MB)\n","\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m121.6/121.6 MB\u001b[0m \u001b[31m6.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m:00:01\u001b[0m00:01\u001b[0m\n","\u001b[?25hDownloading nvidia_curand_cu12-10.3.2.106-py3-none-manylinux1_x86_64.whl (56.5 MB)\n","\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m56.5/56.5 MB\u001b[0m \u001b[31m29.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m:00:01\u001b[0m00:01\u001b[0m\n","\u001b[?25hDownloading nvidia_cusolver_cu12-11.4.5.107-py3-none-manylinux1_x86_64.whl (124.2 MB)\n","\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m124.2/124.2 MB\u001b[0m \u001b[31m13.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m00:01\u001b[0m00:01\u001b[0m\n","\u001b[?25hDownloading nvidia_cusparse_cu12-12.1.0.106-py3-none-manylinux1_x86_64.whl (196.0 MB)\n","\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m196.0/196.0 MB\u001b[0m \u001b[31m4.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m:00:01\u001b[0m00:01\u001b[0m\n","\u001b[?25hDownloading nvidia_nccl_cu12-2.20.5-py3-none-manylinux2014_x86_64.whl (176.2 MB)\n","\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m176.2/176.2 MB\u001b[0m \u001b[31m9.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m:00:01\u001b[0m00:01\u001b[0m\n","\u001b[?25hDownloading nvidia_nvtx_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (99 kB)\n","\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m99.1/99.1 kB\u001b[0m \u001b[31m5.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[?25hDownloading triton-2.3.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (168.1 MB)\n","\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m168.1/168.1 MB\u001b[0m \u001b[31m9.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m:00:01\u001b[0m00:01\u001b[0m\n","\u001b[?25hDownloading einops-0.8.0-py3-none-any.whl (43 kB)\n","\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m43.2/43.2 kB\u001b[0m \u001b[31m231.5 kB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0ma \u001b[36m0:00:01\u001b[0m\n","\u001b[?25hDownloading evaluate-0.4.2-py3-none-any.whl (84 kB)\n","\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m84.1/84.1 kB\u001b[0m \u001b[31m903.4 kB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0ma \u001b[36m0:00:01\u001b[0m\n","\u001b[?25hDownloading optimum-1.21.2-py3-none-any.whl (424 kB)\n","\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m424.7/424.7 kB\u001b[0m \u001b[31m4.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m:00:01\u001b[0m\n","\u001b[?25hDownloading optimum_intel-1.18.1-py3-none-any.whl (224 kB)\n","\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m224.4/224.4 kB\u001b[0m \u001b[31m2.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0mta \u001b[36m0:00:01\u001b[0m\n","\u001b[?25hDownloading python_multipart-0.0.9-py3-none-any.whl (22 kB)\n","Downloading sacremoses-0.1.1-py3-none-any.whl (897 kB)\n","\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m897.5/897.5 kB\u001b[0m \u001b[31m8.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m:00:01\u001b[0m\n","\u001b[?25hDownloading shortuuid-1.0.13-py3-none-any.whl (10 kB)\n","Downloading yacs-0.1.8-py3-none-any.whl (14 kB)\n","Downloading sacrebleu-2.4.2-py3-none-any.whl (106 kB)\n","\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m106.7/106.7 kB\u001b[0m \u001b[31m891.8 kB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0ma \u001b[36m0:00:01\u001b[0m\n","\u001b[?25hDownloading coloredlogs-15.0.1-py2.py3-none-any.whl (46 kB)\n","\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m46.0/46.0 kB\u001b[0m \u001b[31m304.1 kB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0ma \u001b[36m0:00:01\u001b[0m\n","\u001b[?25hDownloading jsonlines-4.0.0-py3-none-any.whl (8.7 kB)\n","Downloading nh3-0.2.18-cp37-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (769 kB)\n","\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m769.2/769.2 kB\u001b[0m \u001b[31m7.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m:00:01\u001b[0m\n","\u001b[?25hDownloading pytablewriter-1.2.0-py3-none-any.whl (111 kB)\n","\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m111.1/111.1 kB\u001b[0m \u001b[31m913.9 kB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0ma \u001b[36m0:00:01\u001b[0m\n","\u001b[?25hDownloading tqdm_multiprocess-0.0.11-py3-none-any.whl (9.8 kB)\n","Downloading DataProperty-1.0.1-py3-none-any.whl (27 kB)\n","Downloading humanfriendly-10.0-py2.py3-none-any.whl (86 kB)\n","\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m86.8/86.8 kB\u001b[0m \u001b[31m742.8 kB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m \u001b[36m0:00:01\u001b[0m\n","\u001b[?25hDownloading mbstrdecoder-1.1.3-py3-none-any.whl (7.8 kB)\n","Downloading pathvalidate-3.2.0-py3-none-any.whl (23 kB)\n","Downloading tabledata-1.3.3-py3-none-any.whl (11 kB)\n","Downloading tcolorpy-0.1.6-py3-none-any.whl (8.1 kB)\n","Downloading typepy-1.3.2-py3-none-any.whl (31 kB)\n","Downloading latex2mathml-3.77.0-py3-none-any.whl (73 kB)\n","\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m73.7/73.7 kB\u001b[0m \u001b[31m750.2 kB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0ma \u001b[36m0:00:01\u001b[0m\n","\u001b[?25hDownloading markdown2-2.5.0-py2.py3-none-any.whl (47 kB)\n","\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m47.2/47.2 kB\u001b[0m \u001b[31m435.7 kB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0ma \u001b[36m0:00:01\u001b[0m\n","\u001b[?25hDownloading nvidia_nvjitlink_cu12-12.5.82-py3-none-manylinux2014_x86_64.whl (21.3 MB)\n","\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m21.3/21.3 MB\u001b[0m \u001b[31m12.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m:00:01\u001b[0m00:01\u001b[0m\n","\u001b[?25hDownloading portalocker-2.10.0-py3-none-any.whl (18 kB)\n","Downloading chardet-5.2.0-py3-none-any.whl (199 kB)\n","\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m199.4/199.4 kB\u001b[0m \u001b[31m2.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m:00:01\u001b[0m\n","\u001b[?25hDownloading svgwrite-1.4.3-py3-none-any.whl (67 kB)\n","\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m67.1/67.1 kB\u001b[0m \u001b[31m665.6 kB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0ma \u001b[36m0:00:01\u001b[0m\n","\u001b[?25hBuilding wheels for collected packages: cchardet, rouge_score, transformers_stream_generator, sqlitedict, word2number, wavedrom\n","  Building wheel for cchardet (setup.py) ... \u001b[?25ldone\n","\u001b[?25h  Created wheel for cchardet: filename=cchardet-2.1.7-cp310-cp310-linux_x86_64.whl size=143397 sha256=c4284d238b2e93c8418cadc788cf1503fa8488fa9106c092f3c0ff75f391446e\n","  Stored in directory: /root/.cache/pip/wheels/ee/e0/ab/e01326f15c59438d080b1496dbab8091e952ec72f35e3c437e\n","  Building wheel for rouge_score (setup.py) ... \u001b[?25ldone\n","\u001b[?25h  Created wheel for rouge_score: filename=rouge_score-0.1.2-py3-none-any.whl size=24934 sha256=aa8bbdd7a94191013d6b569e1cf4add7ac130b114325dc8826665355cb5a66ff\n","  Stored in directory: /root/.cache/pip/wheels/5f/dd/89/461065a73be61a532ff8599a28e9beef17985c9e9c31e541b4\n","  Building wheel for transformers_stream_generator (setup.py) ... \u001b[?25ldone\n","\u001b[?25h  Created wheel for transformers_stream_generator: filename=transformers_stream_generator-0.0.5-py3-none-any.whl size=12424 sha256=d52acabbca72113b566510d70e85b6bae0d2f2ffe0d00583f00439075dc78e17\n","  Stored in directory: /root/.cache/pip/wheels/95/4a/90/140f7b67d125906f6a165f38aad212ecb4a695ad0d87582437\n","  Building wheel for sqlitedict (setup.py) ... \u001b[?25ldone\n","\u001b[?25h  Created wheel for sqlitedict: filename=sqlitedict-2.1.0-py3-none-any.whl size=16863 sha256=dfcbb5959a10c4667ff928e074155a3813c8a27e068a1e8f286f4c913f6a44a6\n","  Stored in directory: /root/.cache/pip/wheels/79/d6/e7/304e0e6cb2221022c26d8161f7c23cd4f259a9e41e8bbcfabd\n","  Building wheel for word2number (setup.py) ... \u001b[?25ldone\n","\u001b[?25h  Created wheel for word2number: filename=word2number-1.1-py3-none-any.whl size=5568 sha256=0568d19c087323e978ab7a6b934915ce2329794ca20d221267b649af0adbf420\n","  Stored in directory: /root/.cache/pip/wheels/84/ff/26/d3cfbd971e96c5aa3737ecfced81628830d7359b55fbb8ca3b\n","  Building wheel for wavedrom (setup.py) ... \u001b[?25ldone\n","\u001b[?25h  Created wheel for wavedrom: filename=wavedrom-2.0.3.post3-py2.py3-none-any.whl size=30052 sha256=15a858cd6497219f59b3c760be5eef64c26e0856481b64b340ea4a65d46b4038\n","  Stored in directory: /root/.cache/pip/wheels/9c/52/8c/38b454b42f712f325e26f633287484c7dc1ad469e1580c5954\n","Successfully built cchardet rouge_score transformers_stream_generator sqlitedict word2number wavedrom\n","Installing collected packages: word2number, sqlitedict, oneccl_bind_pt, nh3, cchardet, yacs, triton, tqdm-multiprocess, tcolorpy, svgwrite, shortuuid, sacremoses, python-multipart, pydantic, portalocker, pathvalidate, nvidia-nvtx-cu12, nvidia-nvjitlink-cu12, nvidia-nccl-cu12, nvidia-curand-cu12, nvidia-cufft-cu12, nvidia-cuda-runtime-cu12, nvidia-cuda-nvrtc-cu12, nvidia-cuda-cupti-cu12, nvidia-cublas-cu12, numpy, neural_speed, markdown2, latex2mathml, jsonlines, humanfriendly, einops, chardet, wavedrom, tiktoken, sacrebleu, rouge_score, nvidia-cusparse-cu12, nvidia-cudnn-cu12, mbstrdecoder, intel_extension_for_pytorch, coloredlogs, typepy, nvidia-cusolver-cu12, torch, fschat, transformers_stream_generator, torchaudio, evaluate, DataProperty, accelerate, tabledata, peft, optimum, pytablewriter, optimum-intel, lm-eval\n","  Attempting uninstall: pydantic\n","    Found existing installation: pydantic 2.5.3\n","    Uninstalling pydantic-2.5.3:\n","      Successfully uninstalled pydantic-2.5.3\n","  Attempting uninstall: numpy\n","    Found existing installation: numpy 1.26.4\n","    Uninstalling numpy-1.26.4:\n","      Successfully uninstalled numpy-1.26.4\n","  Attempting uninstall: torch\n","    Found existing installation: torch 2.1.2\n","    Uninstalling torch-2.1.2:\n","      Successfully uninstalled torch-2.1.2\n","  Attempting uninstall: accelerate\n","    Found existing installation: accelerate 0.32.1\n","    Uninstalling accelerate-0.32.1:\n","      Successfully uninstalled accelerate-0.32.1\n","\u001b[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\n","cudf 24.6.1 requires cubinlinker, which is not installed.\n","cudf 24.6.1 requires cupy-cuda11x>=12.0.0, which is not installed.\n","cudf 24.6.1 requires ptxcompiler, which is not installed.\n","cuml 24.6.1 requires cupy-cuda11x>=12.0.0, which is not installed.\n","dask-cudf 24.6.1 requires cupy-cuda11x>=12.0.0, which is not installed.\n","tensorflow-decision-forests 1.8.1 requires wurlitzer, which is not installed.\n","ucx-py 0.38.0 requires libucx<1.16,>=1.15.0, which is not installed.\n","ucxx 0.38.0 requires libucx>=1.15.0, which is not installed.\n","albumentations 1.4.0 requires numpy>=1.24.4, but you have numpy 1.23.5 which is incompatible.\n","apache-beam 2.46.0 requires dill<0.3.2,>=0.3.1.1, but you have dill 0.3.8 which is incompatible.\n","apache-beam 2.46.0 requires pyarrow<10.0.0,>=3.0.0, but you have pyarrow 16.1.0 which is incompatible.\n","chex 0.1.86 requires numpy>=1.24.1, but you have numpy 1.23.5 which is incompatible.\n","cudf 24.6.1 requires cuda-python<12.0a0,>=11.7.1, but you have cuda-python 12.5.0 which is incompatible.\n","featuretools 1.31.0 requires numpy>=1.25.0, but you have numpy 1.23.5 which is incompatible.\n","libpysal 4.9.2 requires packaging>=22, but you have packaging 21.3 which is incompatible.\n","libpysal 4.9.2 requires shapely>=2.0.1, but you have shapely 1.8.5.post1 which is incompatible.\n","momepy 0.7.2 requires shapely>=2, but you have shapely 1.8.5.post1 which is incompatible.\n","osmnx 1.9.3 requires shapely>=2.0, but you have shapely 1.8.5.post1 which is incompatible.\n","pointpats 2.5.0 requires numpy>=1.24, but you have numpy 1.23.5 which is incompatible.\n","pointpats 2.5.0 requires shapely>=2, but you have shapely 1.8.5.post1 which is incompatible.\n","pyldavis 3.4.1 requires numpy>=1.24.2, but you have numpy 1.23.5 which is incompatible.\n","pylibraft 24.6.0 requires cuda-python<12.0a0,>=11.7.1, but you have cuda-python 12.5.0 which is incompatible.\n","rmm 24.6.0 requires cuda-python<12.0a0,>=11.7.1, but you have cuda-python 12.5.0 which is incompatible.\n","spaghetti 1.7.6 requires shapely>=2.0.1, but you have shapely 1.8.5.post1 which is incompatible.\n","spopt 0.6.1 requires shapely>=2.0.1, but you have shapely 1.8.5.post1 which is incompatible.\n","tensorflow 2.15.0 requires keras<2.16,>=2.15.0, but you have keras 3.4.1 which is incompatible.\n","tensorstore 0.1.63 requires ml-dtypes>=0.3.1, but you have ml-dtypes 0.2.0 which is incompatible.\n","woodwork 0.31.0 requires numpy>=1.25.0, but you have numpy 1.23.5 which is incompatible.\n","xarray 2024.6.0 requires packaging>=23.1, but you have packaging 21.3 which is incompatible.\n","ydata-profiling 4.6.4 requires pydantic>=2, but you have pydantic 1.10.13 which is incompatible.\u001b[0m\u001b[31m\n","\u001b[0mSuccessfully installed DataProperty-1.0.1 accelerate-0.28.0 cchardet-2.1.7 chardet-5.2.0 coloredlogs-15.0.1 einops-0.8.0 evaluate-0.4.2 fschat-0.2.32 humanfriendly-10.0 intel_extension_for_pytorch-2.3.0+cpu jsonlines-4.0.0 latex2mathml-3.77.0 lm-eval-0.4.3 markdown2-2.5.0 mbstrdecoder-1.1.3 neural_speed-1.0a0 nh3-0.2.18 numpy-1.23.5 nvidia-cublas-cu12-12.1.3.1 nvidia-cuda-cupti-cu12-12.1.105 nvidia-cuda-nvrtc-cu12-12.1.105 nvidia-cuda-runtime-cu12-12.1.105 nvidia-cudnn-cu12-8.9.2.26 nvidia-cufft-cu12-11.0.2.54 nvidia-curand-cu12-10.3.2.106 nvidia-cusolver-cu12-11.4.5.107 nvidia-cusparse-cu12-12.1.0.106 nvidia-nccl-cu12-2.20.5 nvidia-nvjitlink-cu12-12.5.82 nvidia-nvtx-cu12-12.1.105 oneccl_bind_pt-2.3.0+cpu optimum-1.21.2 optimum-intel-1.18.1 pathvalidate-3.2.0 peft-0.6.2 portalocker-2.10.0 pydantic-1.10.13 pytablewriter-1.2.0 python-multipart-0.0.9 rouge_score-0.1.2 sacrebleu-2.4.2 sacremoses-0.1.1 shortuuid-1.0.13 sqlitedict-2.1.0 svgwrite-1.4.3 tabledata-1.3.3 tcolorpy-0.1.6 tiktoken-0.4.0 torch-2.3.0 torchaudio-2.3.0 tqdm-multiprocess-0.0.11 transformers_stream_generator-0.0.5 triton-2.3.0 typepy-1.3.2 wavedrom-2.0.3.post3 word2number-1.1 yacs-0.1.8\n","/content\n"]}],"source":["%cd ./intel-extension-for-transformers/intel_extension_for_transformers/neural_chat/\n","!pip install -r requirements_cpu.txt\n","%cd ../../../"]},{"cell_type":"markdown","metadata":{},"source":["# Build your chatbot 💻"]},{"cell_type":"markdown","metadata":{},"source":["## Text Chat"]},{"cell_type":"markdown","metadata":{},"source":["Giving NeuralChat the textual instruction, it will respond with the textual response."]},{"cell_type":"code","execution_count":5,"metadata":{"execution":{"iopub.execute_input":"2024-07-13T14:52:30.935926Z","iopub.status.busy":"2024-07-13T14:52:30.935185Z","iopub.status.idle":"2024-07-13T14:54:26.053404Z","shell.execute_reply":"2024-07-13T14:54:26.052321Z","shell.execute_reply.started":"2024-07-13T14:52:30.935889Z"},"trusted":true},"outputs":[{"name":"stderr","output_type":"stream","text":["/opt/conda/lib/python3.10/site-packages/torchvision/io/image.py:13: UserWarning: Failed to load image Python extension: '/opt/conda/lib/python3.10/site-packages/torchvision/image.so: undefined symbol: _ZN3c1017RegisterOperatorsD1Ev'If you don't plan on using image functionality from `torchvision.io`, you can ignore this warning. Otherwise, there might be something wrong with your environment. Did you have `libjpeg` or `libpng` installed before building `torchvision` from source?\n","  warn(\n","/opt/conda/lib/python3.10/site-packages/transformers/deepspeed.py:24: FutureWarning: transformers.deepspeed module is deprecated and will be removed in a future version. Please import deepspeed modules directly from transformers.integrations\n","  warnings.warn(\n"]},{"name":"stdout","output_type":"stream","text":["Loading model Intel/neural-chat-7b-v3-1\n"]},{"data":{"application/vnd.jupyter.widget-view+json":{"model_id":"953b1251fddb4f469f8da219a4b02f89","version_major":2,"version_minor":0},"text/plain":["config.json:   0%|          | 0.00/625 [00:00<?, ?B/s]"]},"metadata":{},"output_type":"display_data"},{"data":{"application/vnd.jupyter.widget-view+json":{"model_id":"91fe1fe841c14b85a5e7e379c449ff5a","version_major":2,"version_minor":0},"text/plain":["tokenizer_config.json:   0%|          | 0.00/953 [00:00<?, ?B/s]"]},"metadata":{},"output_type":"display_data"},{"data":{"application/vnd.jupyter.widget-view+json":{"model_id":"4a71f32c2c9b4ab6b592381f236c229e","version_major":2,"version_minor":0},"text/plain":["tokenizer.model:   0%|          | 0.00/493k [00:00<?, ?B/s]"]},"metadata":{},"output_type":"display_data"},{"data":{"application/vnd.jupyter.widget-view+json":{"model_id":"c043f245f94044b99f7d21c875338240","version_major":2,"version_minor":0},"text/plain":["tokenizer.json:   0%|          | 0.00/1.80M [00:00<?, ?B/s]"]},"metadata":{},"output_type":"display_data"},{"data":{"application/vnd.jupyter.widget-view+json":{"model_id":"2f9bad0cb08c4e059f6db7785616cd54","version_major":2,"version_minor":0},"text/plain":["special_tokens_map.json:   0%|          | 0.00/145 [00:00<?, ?B/s]"]},"metadata":{},"output_type":"display_data"},{"data":{"application/vnd.jupyter.widget-view+json":{"model_id":"37dd696690574979b2c96008ae0fb934","version_major":2,"version_minor":0},"text/plain":["model.safetensors.index.json:   0%|          | 0.00/25.1k [00:00<?, ?B/s]"]},"metadata":{},"output_type":"display_data"},{"data":{"application/vnd.jupyter.widget-view+json":{"model_id":"de1b1d67be4e4d20b6cd4ef80f86c252","version_major":2,"version_minor":0},"text/plain":["Downloading shards:   0%|          | 0/2 [00:00<?, ?it/s]"]},"metadata":{},"output_type":"display_data"},{"data":{"application/vnd.jupyter.widget-view+json":{"model_id":"1768108d6199431790ba78b7ba751570","version_major":2,"version_minor":0},"text/plain":["model-00001-of-00002.safetensors:   0%|          | 0.00/9.94G [00:00<?, ?B/s]"]},"metadata":{},"output_type":"display_data"},{"data":{"application/vnd.jupyter.widget-view+json":{"model_id":"bc682ea99bda4e9a939656b1b25cb7e7","version_major":2,"version_minor":0},"text/plain":["model-00002-of-00002.safetensors:   0%|          | 0.00/4.54G [00:00<?, ?B/s]"]},"metadata":{},"output_type":"display_data"},{"data":{"application/vnd.jupyter.widget-view+json":{"model_id":"bf296e3819114d8c91d7a68894ba4d08","version_major":2,"version_minor":0},"text/plain":["Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]"]},"metadata":{},"output_type":"display_data"},{"data":{"application/vnd.jupyter.widget-view+json":{"model_id":"058c9df5993440e39dbfd91e9232b42d","version_major":2,"version_minor":0},"text/plain":["generation_config.json:   0%|          | 0.00/111 [00:00<?, ?B/s]"]},"metadata":{},"output_type":"display_data"},{"name":"stdout","output_type":"stream","text":["The Intel Xeon Scalable Processors represent a family of high-performance central processing units (CPUs) designed for data centers, cloud computing, and other demanding workloads. These processors offer significant improvements in performance, efficiency, and scalability compared to their predecessors.\n","\n","Key features include:\n","1. Advanced architecture: The Intel Xeon Scalable Processors utilize the latest microarchitecture, which enables better performance and power efficiency.\n","2. Enhanced security: With built-in hardware-level security features like Intel Software Guard Extensions (SGX), these processors provide robust protection against potential threats.\n","3. Scalability: The modular design allows for flexible configurations, enabling users to choose the right combination of cores, memory capacity, and I/O capabilities to meet specific needs.\n","4. Optimized for virtualization: The Intel Xeon Scalable Processors are optimized for virtual environments, supporting multiple operating systems and applications on a single server.\n","5. High-speed connectivity: Integrated support for high-speed interconnects such as Intel Omni-Path Architecture (OPA) and Intel QuickAssist Technology (QAT) ensures efficient communication between servers and devices\n"]}],"source":["# BF16 Optimization\n","from intel_extension_for_transformers.neural_chat import build_chatbot, PipelineConfig\n","from intel_extension_for_transformers.transformers import MixedPrecisionConfig\n","config = PipelineConfig(optimization_config=MixedPrecisionConfig())\n","chatbot = build_chatbot(config)\n","response = chatbot.predict(query=\"Tell me about Intel Xeon Scalable Processors.\")\n","print(response)\n"]},{"cell_type":"markdown","metadata":{},"source":["### Example 1:"]},{"cell_type":"code","execution_count":6,"metadata":{"execution":{"iopub.execute_input":"2024-07-13T14:55:40.318348Z","iopub.status.busy":"2024-07-13T14:55:40.317284Z","iopub.status.idle":"2024-07-13T14:55:51.625121Z","shell.execute_reply":"2024-07-13T14:55:51.624256Z","shell.execute_reply.started":"2024-07-13T14:55:40.318312Z"},"trusted":true},"outputs":[{"name":"stdout","output_type":"stream","text":["The main difference between a Graphics Processing Unit (GPU) and a Central Processing Unit (CPU) lies in their primary functions and architectures.\n","\n","A CPU is the brain of a computer, responsible for executing instructions and managing various tasks. It's designed to handle general-purpose computing tasks like processing data, running programs, and controlling peripheral devices. CPUs have a single instruction stream and can perform multiple tasks simultaneously through multithreading or multiprocessing.\n","\n","On the other hand, a GPU is specialized hardware optimized for handling complex mathematical calculations and rendering visuals. GPUs excel at parallel processing, which means they can execute many tasks simultaneously using thousands of small processing units called CUDA cores. This makes them ideal for graphics-intensive applications such as gaming, video editing, and scientific simulations.\n","\n","In summary, while both CPUs and GPUs are essential components of modern computers, they serve different purposes. CPUs focus on general-purpose computing tasks, whereas GPUs specialize in high-performance graphics and parallel processing.\n"]}],"source":["response1 = chatbot.predict(query=\"What is difference between GPU and CPU.\")\n","print(response1)"]},{"cell_type":"markdown","metadata":{},"source":["### Example 2:"]},{"cell_type":"code","execution_count":7,"metadata":{"execution":{"iopub.execute_input":"2024-07-13T14:57:35.238602Z","iopub.status.busy":"2024-07-13T14:57:35.238225Z","iopub.status.idle":"2024-07-13T14:57:47.823387Z","shell.execute_reply":"2024-07-13T14:57:47.822403Z","shell.execute_reply.started":"2024-07-13T14:57:35.238572Z"},"trusted":true},"outputs":[{"name":"stdout","output_type":"stream","text":["Finetuning of an AI model is a process where you take an existing pre-trained model and further adapt it for a specific task or domain. This technique helps improve the performance of the model on new data sets or tasks without starting from scratch.\n","\n","Imagine you have a smart robot that can perform various tasks like cooking, cleaning, and playing music. At first, this robot was designed to only cook. However, as time went by, you wanted it to learn how to clean your house too. Instead of building a completely new robot for cleaning, you decide to teach the existing cooking robot some additional skills. That's what we call finetuning - adapting the original model (robot) to perform better in a different area (cleaning).\n","\n","To finetune an AI model, you typically follow these steps:\n","\n","1. Choose a pre-trained model: Start with a model that has already been trained on a large dataset for a particular task. For example, if you want to create a model that can recognize images, you might use a pre-trained image recognition model.\n","\n","2. Prepare the new dataset: Collect a new set of data related to the desired task or domain.\n"]}],"source":["response2 = chatbot.predict(query=\"Explain finetuning of a ai model.\")\n","print(response2)"]},{"cell_type":"markdown","metadata":{},"source":["### Example 3:"]},{"cell_type":"code","execution_count":8,"metadata":{"execution":{"iopub.execute_input":"2024-07-13T14:59:42.692411Z","iopub.status.busy":"2024-07-13T14:59:42.692030Z","iopub.status.idle":"2024-07-13T14:59:51.227377Z","shell.execute_reply":"2024-07-13T14:59:51.226403Z","shell.execute_reply.started":"2024-07-13T14:59:42.692379Z"},"trusted":true},"outputs":[{"name":"stdout","output_type":"stream","text":["TPU stands for Tensor Processing Unit, which is a specialized hardware designed for machine learning tasks. It's optimized for running deep neural networks efficiently. GPUs (Graphics Processing Units) were initially developed for rendering graphics in video games but have since been repurposed for various computing tasks, including machine learning.\n","\n","Comparing TPUs and GPUs can be complex as it depends on the specific application and workload. Generally, TPUs offer higher performance for certain machine learning tasks like training large models or performing inference at scale. However, GPUs still excel in other areas such as general-purpose computing, gaming, and handling diverse workloads. So, while TPUs may be better suited for some AI applications, GPUs remain versatile and valuable across multiple domains.\n"]}],"source":["response3 = chatbot.predict(query=\"What are TPU, are they better than GPU\")\n","print(response3)"]},{"cell_type":"markdown","metadata":{},"source":["### Example 4:"]},{"cell_type":"code","execution_count":9,"metadata":{"execution":{"iopub.execute_input":"2024-07-13T15:01:14.757431Z","iopub.status.busy":"2024-07-13T15:01:14.757055Z","iopub.status.idle":"2024-07-13T15:01:20.267672Z","shell.execute_reply":"2024-07-13T15:01:20.266772Z","shell.execute_reply.started":"2024-07-13T15:01:14.757401Z"},"trusted":true},"outputs":[{"name":"stdout","output_type":"stream","text":["A Transformer is a type of neural network architecture designed for natural language processing tasks. It was introduced in 2017 by researchers at Google AI. The model has shown impressive results in various NLP applications such as machine translation, text summarization, and question answering. Transformers learn to represent words and sentences through self-attention mechanisms, which allow them to understand the context and relationships between different parts of the input data. This makes them highly effective in understanding complex linguistic structures and generating meaningful outputs.\n"]}],"source":["response4 = chatbot.predict(query=\"What is a Transformer\")\n","print(response4)"]},{"cell_type":"markdown","metadata":{},"source":["### Example 5:"]},{"cell_type":"code","execution_count":10,"metadata":{"execution":{"iopub.execute_input":"2024-07-13T15:02:52.866635Z","iopub.status.busy":"2024-07-13T15:02:52.865987Z","iopub.status.idle":"2024-07-13T15:02:57.222109Z","shell.execute_reply":"2024-07-13T15:02:57.221198Z","shell.execute_reply.started":"2024-07-13T15:02:52.866601Z"},"trusted":true},"outputs":[{"name":"stdout","output_type":"stream","text":["A General Artificial Intelligence (AGI) refers to a hypothetical future AI system that can perform any intellectual task as well as or better than a human being. It would possess a high level of intelligence, creativity, problem-solving abilities, and adaptability across various domains. AGIs are still in the realm of science fiction, with current AI systems focusing on specific tasks and limited capabilities.\n"]}],"source":["response5 = chatbot.predict(query=\"What is a AGI\")\n","print(response5)"]}],"metadata":{"kaggle":{"accelerator":"gpu","dataSources":[],"isGpuEnabled":true,"isInternetEnabled":true,"language":"python","sourceType":"notebook"},"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.10.13"}},"nbformat":4,"nbformat_minor":4}