2.3 _HW_pandas_functions.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "8fb1e820",
   "metadata": {},
   "source": [
    "## 3. Функции и работа с данными\n",
    "\n",
    "### Задание 1\n",
    "Напишите функцию, которая классифицирует фильмы из материалов занятия по следующим правилам:\n",
    "- оценка 2 и меньше - низкий рейтинг\n",
    "- оценка 4 и меньше - средний рейтинг\n",
    "- оценка 4.5 и 5 - высокий рейтинг\n",
    "\n",
    "Результат классификации запишите в столбец class.\n",
    "\n",
    "## 3. Functions and working with data\n",
    "\n",
    "### Task 1\n",
    "Write a function that classifies films from the lesson materials according to the following rules:\n",
    "\n",
    "- score 2 and less - low rating\n",
    "- score 4 and less - average rating\n",
    "- score 4.5 and 5 - high rating\n",
    "\n",
    "Write the classification result in the 'class' column."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4f55e495",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "1e9ab763",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>title</th>\n",
       "      <th>rating</th>\n",
       "      <th>class</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>\"Great Performances\" Cats (1998)</td>\n",
       "      <td>1.750000</td>\n",
       "      <td>Low</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>$9.99 (2008)</td>\n",
       "      <td>3.833333</td>\n",
       "      <td>Average</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>'Hellboy': The Seeds of Creation (2004)</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>Low</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>'Neath the Arizona Skies (1934)</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>Low</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>'Round Midnight (1986)</td>\n",
       "      <td>2.250000</td>\n",
       "      <td>Average</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>'Salem's Lot (2004)</td>\n",
       "      <td>3.500000</td>\n",
       "      <td>Average</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>'Til There Was You (1997)</td>\n",
       "      <td>2.625000</td>\n",
       "      <td>Average</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>'burbs, The (1989)</td>\n",
       "      <td>3.052632</td>\n",
       "      <td>Average</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>'night Mother (1986)</td>\n",
       "      <td>5.000000</td>\n",
       "      <td>High</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>(500) Days of Summer (2009)</td>\n",
       "      <td>3.755556</td>\n",
       "      <td>Average</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                     title    rating    class\n",
       "0         \"Great Performances\" Cats (1998)  1.750000      Low\n",
       "1                             $9.99 (2008)  3.833333  Average\n",
       "2  'Hellboy': The Seeds of Creation (2004)  2.000000      Low\n",
       "3          'Neath the Arizona Skies (1934)  0.500000      Low\n",
       "4                   'Round Midnight (1986)  2.250000  Average\n",
       "5                      'Salem's Lot (2004)  3.500000  Average\n",
       "6                'Til There Was You (1997)  2.625000  Average\n",
       "7                       'burbs, The (1989)  3.052632  Average\n",
       "8                     'night Mother (1986)  5.000000     High\n",
       "9              (500) Days of Summer (2009)  3.755556  Average"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ratings = pd.read_csv('ratings.csv')\n",
    "movies = pd.read_csv('movies.csv')\n",
    "\n",
    "df = movies.merge(ratings, on = 'movieId', how = 'left')\n",
    "df_groupby_object = df.groupby(['title'])\n",
    "films_classification = df_groupby_object.rating.mean().reset_index()\n",
    "\n",
    "def rating_classification(row):\n",
    "    \n",
    "    if row['rating'] <= 2:\n",
    "        return 'Low'\n",
    "    elif row['rating'] <= 4:\n",
    "        return 'Average'\n",
    "    else:\n",
    "        return 'High'\n",
    "\n",
    "films_classification['class'] = films_classification.apply(rating_classification, axis = 1)\n",
    "\n",
    "films_classification.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a747ab4f",
   "metadata": {},
   "source": [
    "### Задание 2\n",
    "\n",
    "Используем файл keywords.csv.\n",
    "\n",
    "Необходимо написать гео-классификатор, который каждой строке сможет выставить географическую принадлежность определенному региону. Т. е. если поисковый запрос содержит название города региона, то в столбце ‘region’ пишется название этого региона. Если поисковый запрос не содержит названия города, то ставим ‘undefined’.\n",
    "\n",
    "Правила распределения по регионам Центр, Северо-Запад и Дальний Восток:\n",
    "\n",
    "Результат классификации запишите в отдельный столбец region.\n",
    "\n",
    "### Task 2\n",
    "\n",
    "We use the keywords.csv file.\n",
    "\n",
    "It is necessary to write a geo-classifier that will be able to set the geographical affiliation of a certain region to each row. I.e., if the search query contains the name of the city of the region, then the name of this region is written in the ‘region’ column. If the search query does not contain the name of the city, then put ‘undefined'.\n",
    "\n",
    "Rules of distribution by region Center, North-West and Far East:\n",
    "\n",
    "Write the classification result in a separate column region."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "459f1042",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "                        keyword   shows          region\n",
      "127                авито москва  979292           Центр\n",
      "370    авито ру санкт петербург  425134    Северо-Запад\n",
      "564   погода в санкт петербурге  292489    Северо-Запад\n",
      "849             авито ярославль  209581           Центр\n",
      "1063        фарпост владивосток  176951  Дальний Восток\n",
      "1236       банк санкт петербург  174375    Северо-Запад\n",
      "1259                     москва  153050           Центр\n",
      "1346            санкт петербург  144232    Северо-Запад\n",
      "1396            гисметео москва  140279           Центр\n",
      "1482           госуслуги москва  133061           Центр\n"
     ]
    }
   ],
   "source": [
    "geo_data = {\n",
    "'Центр': ['москва', 'тула', 'ярославль'],\n",
    "'Северо-Запад': ['петербург', 'псков', 'мурманск'],\n",
    "'Дальний Восток': ['владивосток', 'сахалин', 'хабаровск']\n",
    "}\n",
    "\n",
    "df = pd.read_csv('keywords.csv')\n",
    "\n",
    "def regions(row):\n",
    "    for region, cities in geo_data.items():\n",
    "        for city in cities:\n",
    "            if city in row['keyword']:\n",
    "                return region\n",
    "    return 'undefined'\n",
    "\n",
    "df['region'] = df.apply(regions, axis=1)\n",
    "\n",
    "show = df[df['region'] != 'undefined']\n",
    "print(show[0:10])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "91066054",
   "metadata": {},
   "source": [
    "### Задание 3 (бонусное)\n",
    "\n",
    "Есть мнение, что “раньше снимали настоящее кино, не то что сейчас”. Ваша задача проверить это утверждение, используя файлы с рейтингами фильмов из прошлого домашнего занятия (файл movies.csv и ratings.csv из базы grouplens.org.../movielens). Т. е. проверить верно ли, что с ростом года выпуска фильма его средний рейтинг становится ниже.\n",
    "\n",
    "При этом мы не будем затрагивать субьективные факторы выставления этих рейтингов, а пройдемся по следующему алгоритму:\n",
    "\n",
    "В переменную years запишите список из всех годов с 1950 по 2010.\n",
    "\n",
    "Напишите функцию production_year, которая каждой строке из названия фильма выставляет год выпуска. Не все названия фильмов содержат год выпуска в одинаковом формате, поэтому используйте следующий алгоритм:\n",
    "\n",
    "для каждой строки пройдите по всем годам списка years\n",
    "если номер года присутствует в названии фильма, то функция возвращает этот год как год выпуска\n",
    "если ни один из номеров года списка years не встретился в названии фильма, то возвращается 1900 год\n",
    "Запишите год выпуска фильма по алгоритму пункта 2 в новый столбец ‘year’\n",
    "\n",
    "Посчитайте средний рейтинг всех фильмов для каждого значения столбца ‘year’ и отсортируйте результат по убыванию рейтинга\n",
    "\n",
    "### Task 3 (bonus)\n",
    "\n",
    "There is an opinion that “they used to make a real movie, not like now.” Your task is to verify this statement using files with movie ratings from the last home lesson (the movies.csv and ratings.csv file from the grouplens.org.../movielens database). That is, to check whether it is true that with the growth of the year of release of the film, its average rating becomes lower.\n",
    "\n",
    "At the same time, we will not touch on the subjective factors of setting these ratings, but will go through the following algorithm:\n",
    "\n",
    "In the years variable, write a list of all the years from 1950 to 2010.\n",
    "\n",
    "Write the production_year function, which sets the year of release for each line of the movie title. Not all movie titles contain the year of release in the same format, so use the following algorithm:\n",
    "\n",
    "for each row, go through all the years in the years list\n",
    ". if the year number is present in the movie title, the function returns this year as the year of release.\n",
    "if none of the year numbers of the years list is found in the title of the film, then the year 1900 is returned.\n",
    "Write down the year of release of the film according to the algorithm of paragraph 2 in the new column ‘year’\n",
    "\n",
    "Calculate the average rating of all films for each value of the ‘year’ column and sort the result in descending order of rating"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "c8c16443",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>rating</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>year</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1957</th>\n",
       "      <td>4.014241</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1972</th>\n",
       "      <td>4.011136</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1952</th>\n",
       "      <td>4.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1954</th>\n",
       "      <td>3.994220</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1951</th>\n",
       "      <td>3.983539</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1974</th>\n",
       "      <td>3.978704</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1962</th>\n",
       "      <td>3.952446</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1950</th>\n",
       "      <td>3.915254</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1977</th>\n",
       "      <td>3.905786</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1964</th>\n",
       "      <td>3.841492</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        rating\n",
       "year          \n",
       "1957  4.014241\n",
       "1972  4.011136\n",
       "1952  4.000000\n",
       "1954  3.994220\n",
       "1951  3.983539\n",
       "1974  3.978704\n",
       "1962  3.952446\n",
       "1950  3.915254\n",
       "1977  3.905786\n",
       "1964  3.841492"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ratings = pd.read_csv('ratings.csv')\n",
    "movies = pd.read_csv('movies.csv')\n",
    "\n",
    "df = ratings.merge(movies, on = 'movieId')\n",
    "years = list(range(1950, 2011))\n",
    "\n",
    "def production_year(title):\n",
    "    for year in years:\n",
    "        if str(year) in title:\n",
    "            return year\n",
    "    return 1900\n",
    "\n",
    "df['year'] = df['title'].apply(production_year)\n",
    "\n",
    "ratings_years = df[['rating', 'year']].groupby(by = 'year').mean().sort_values('rating', ascending=False)\n",
    "ratings_years.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "73aa3c96",
   "metadata": {},
   "source": [
    "![](Pandas_logo.png)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}