Initial commit

fnesveda · Oct 15, 2019 · c13cd0a · c13cd0a
commit c13cd0a
Show file tree

Hide file tree

Showing 35 changed files with 3,028 additions and 0 deletions.
diff --git a/README.md b/README.md
@@ -0,0 +1,40 @@
+<h1 align="center">Demand Management in Smart Grids</h1>
+<br>
+<p align="center">A diploma thesis investigating the options of controlling power demand of households to reduce peaks in total power consumption in smart grids.</p>
+<p align="center"><img src="docs/images/screenshots/main.svg" width="800" height="480" alt="Simulation results for February 2018"></p>
+
+The problem
+-----------
+While the widespread adoption of electric vehicles in the future could serve to reduce our reliance on fossil fuels, provided that the energy used to charge those vehicles comes from cleaner sources, the increased electricity usage that would come as a side effect could constitute a challenge to the electrical grid. Most trips by household-owned vehicles are trips to work and back, and most people plug in their car to charge it as soon as they arrive home. As the bulk of the people arrive home around the same time, this can cause peaks in electricity consumption in the afternoon hours.
+
+Because traditional coal or nuclear power plants have a long ramp-up time, they are ill-suited to be used to cover these sudden peaks, and specialized peaking power plants such as gas turbines or pumped hydroelectric energy storage must be used, which is costly and not always feasible. The situation is further complicated as power plants utilizing renewable energy sources are being introduced to the grid. These plants, such as solar panel farms or wind turbines, often have an uncontrollable and intermittent power output, requiring even more energy storage to compensate.
+
+To combat these negative effects, there arises a need to be able to better control the electricity demand of the grid by matching the power requirements of electricity subscribers to the generation profile of the power plants.
+
+There are several proposed strategies to achieve that goal, but they have several key issues, one of them being their privacy implications, as most of them require the EV owners to inform the distributor or another third party about their travel schedule and charging needs, which some might not be willing to do. Another weak point is that the strategies control only electric cars, and do not take into account other appliances that could be controlled. Furthemore, most of the strategies require that subscribers are willing to adhere to them, even though it might not be directly beneficial to them.
+
+Our proposed solution
+---------------------
+To tackle the issues with the current strategies, we decided to employ the strategy of motivating the subscribers to use energy at specific times by means of time-of-use electricity price tariffs, with different timing of low and high prices for each connected household.
+
+The electricity distributor decides on a total target electricity demand profile which it would like to achieve on its grid according to its needs (e.g. a flat demand profile for a grid consisting of power plants with slow demand-response characteristics, or a profile with higher demand during specific intervals for a grid containing solar plants or wind turbines when expecting a lot of sunlight or wind).
+
+A part of this demand does not come from subscribers whose demand can be influenced easily, but instead from other sectors like businesses and industrial customers. The part of the demand the distributor can influence, like the demand of households, can be used for balancing the demand of the rest of the grid to achieve the desired total demand. The distributor must therefore separate the total target demand profile into the uninfluenceable base demand and the influenceable target household demand.
+
+The electricity prices for each household are then assigned so that more households get a high price when their demand should be low, and a low price when their demand should be high. The electricity prices are finally distributed to the households, and the decision on when to turn their appliances on is left to them, expecting them to act in their own interest and optimize their appliance usage to achieve the lowest total cost of electricity.
+
+This achieves the goals described earlier — privacy of the subscribers is protected, since they do not need to divulge the information about the usage of their appliances; multiple types of appliances can be controlled, since they only need to be able to adjust their operation to the electricity price; and the subscribers are incentivized to use energy in coordination with the needs of the distributor, as it will also be cheaper for them.
+
+A more detailed description of our solution can be found in the [thesis text](thesis/thesis.pdf).
+
+[Simulator](simulator/)
+-----------------------
+For evaluating the effectiveness of the demand control algorithm, we created a simulator of the smart grid. The simulator currently simulates the Texas power grid, based on data from the [Electricity Reliability Council of Texas](http://www.ercot.com), the [Pecan Street organization](https://www.pecanstreet.org/) and the [National Household Travel Survey](https://nhts.ornl.gov/), which it can directly download and parse.
+
+Its structure was designed to resemble a real electrical distribution grid, with each part of the grid being a separate entity, operating independently, keeping its own clock and performing calculations exactly when needed, according to an environment-provided clock signal. This was done to allow for easier possible future extension of the simulator or its separation into multiple programs communicating over some network interface to enable simulating communication delays and other physical limitations of the distribution grid.
+
+The simulator implements our algorithm described in the thesis, and two simpler demand control algorithms used for comparison with the first one — an algorithm where appliances stretch their power demand across their whole usage interval, and an algorithm where they perform their operation as early as possible.
+
+The simulator is written in Python, and it's purpose-built for this thesis, so extending it for different uses would be quite comples, but the legwork is there. There are certainly some improvements that could be made, most prominently the use of multithreading, but that shouldn't be too complicated now with the multiprocessing improvements in Python 3.8.
+
+To try the simulator, download the [latest release](https://www.github.com/fnesveda/DemandManagement/releases/latest/) and unpack it. Usage instructions can be found in the included [README file](simulator/README.md).
diff --git a/docs/images/.gitattributes b/docs/images/.gitattributes
@@ -0,0 +1 @@
+*.svg binary
diff --git a/docs/images/screenshots/main.svg b/docs/images/screenshots/main.svg
diff --git a/simulator/README.md b/simulator/README.md
@@ -0,0 +1,67 @@
+Smart grid simulator
+====================
+
+This is a simulator of a smart grid made as a part of the masters thesis "Demand control in smart grids" at the Faculty of Mathematics and Physics of the Charles University in Prague.
+It serves to simulate an electrical grid with households and appliances based on data from the National Household Travel Survey
+and the Pecan Street organization research program,
+while the appliances act according to several demand control algorithms described in the thesis,
+which is included in the file _thesis.pdf_.
+
+Requirements
+------------
+
+The simulator can be run in an UNIX environment containing the Bash shell and a Python interpreter version 3.6 or newer.
+Several third-party Python libraries are required, these are listed in the file _requirements.txt_
+and available for installation with the shell command `pip3 install -r requirements.txt`.
+
+Data download
+-------------
+Before the actual usage of the simulator, the data necessary for the simulation must be downloaded first.
+This is performed in three steps:
+
+1. In the file _simulator/data/config.txt_, put the desired dates in the `fromdate` and `todate` fields, in the format `YYYY-MM-DD`. For a succesful simulation,
+the data must be downloaded for an interval starting one week before the starting date of the simulation
+and ending one week after the ending date of the simulation.
+
+2. In the file _simulator/data/dataport/KEY_, put the username and password
+to access the Dataport database in the `username` and `password` fields, respectively.
+These access credentials can be obtained at https://dataport.pecanstreet.org/.
+
+3. Execute the data download by using the command script `./downloadData.sh` in a terminal while being in the directory with the project.
+Depending on the length of the downloaded interval, the download can take several hours and use several gigabytes of data.
+Progress of the download is printed to the terminal during the run of the script.
+
+Running the simulator
+---------------------
+
+After the data is downloaded, the simulator can be executed by using the command
+`./run.py startingDate simulationLength householdCount outputFolder` in a terminal while being in the directory with the project,
+while replacing `startingDate` with the date from which to run the simulation in the format `YYYY-MM-DD`,
+`simulationLength` with the number of days to simulate,
+`householdCount` with the number of households to simulate
+and `outputFolder` with the destination folder in which the simulation results should be saved,
+for example: `./run.py 2018-01-01 365 10000 out/`.
+
+While the simulation is running, the simulator prints information about its progress to the terminal.
+When the simulation finishes, the results are saved in the specified folder in two files, _desc.txt_ and _data.csv_.
+The file _desc.txt_ contains information about the simulation parameters,
+and the file _data.csv_ is a standard comma-separated values file containing the simulation results organized to eight columns:
+
+Column              | Description
+--------------------|------------
+Datetime            | The date and time of the result
+PredictedBaseDemand | The predicted grid base demand at that date and time (in kilowatts)
+ActualBaseDemand    | The actual grid base demand at that date and time (in kilowatts)
+TargetDemand        | The target power demand at that date and time (in kilowatts)
+SmartDemand         | The power demand of the simulated households at that date and time if the smart algorithm was used (in kilowatts)
+UncontrolledDemand  | The power demand of the simulated households at that date and time if the uncontrolled algorithm was used (in kilowatts)
+SpreadOutDemand     | The power demand of the simulated households at that date and time if the spread out algorithm was used (in kilowatts)
+PriceRatio          | The ratio of how many households should have a lower electricity price at that date and time
+
+Displaying the results
+----------------------
+
+To display the simulation results, one can use the included Jupyter notebook
+_Results.ipynb_. After opening the notebook, first the folder with the simulation
+results must be specified in the variable _resultsFolder_ in the third code cell,
+and then the cells in the notebook can be executed to show the results.
diff --git a/simulator/Results.ipynb b/simulator/Results.ipynb
@@ -0,0 +1,247 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "**Matplotlib preparation**"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import matplotlib\n",
+    "%matplotlib notebook\n",
+    "\n",
+    "import matplotlib.pyplot as plt\n",
+    "import matplotlib.dates as mdates\n",
+    "\n",
+    "matplotlib.rcParams['figure.figsize'] = (9.8, 5)\n",
+    "matplotlib.rcParams['figure.constrained_layout.use'] = True\n",
+    "matplotlib.rcParams['font.size'] = 13\n",
+    "\n",
+    "# without this matplotlib emits a warning \n",
+    "# even when not plotting pandas data at all, because pandas messes with it\n",
+    "from pandas.plotting import register_matplotlib_converters\n",
+    "register_matplotlib_converters()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "**Imports necessary for loading the data**"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import datetime\n",
+    "import pandas"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "**Folder with the simulation results**"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "resultsFolder = \"out/\""
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "**Loading the data**"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "desc = {}\n",
+    "with open(f\"{resultsFolder}/desc.txt\", \"r\") as descfile:\n",
+    "\tfor line in descfile:\n",
+    "\t\tkey, val = line.strip(\"\\n\").split(\"=\")\n",
+    "\t\tdesc[key] = val\n",
+    "\n",
+    "startingDT = datetime.datetime.strptime(desc[\"startingDatetime\"], \"%Y-%m-%d %H:%M:%S\")\n",
+    "simulationLength = int(desc[\"simulationLength\"])\n",
+    "houseCount = int(desc[\"houseCount\"])\n",
+    "\n",
+    "\n",
+    "data = pandas.read_csv(f\"{resultsFolder}/data.csv\", parse_dates=[0])\n",
+    "\n",
+    "datetimes = data[\"Datetime\"].values\n",
+    "predictedDemand = data[\"PredictedBaseDemand\"].values\n",
+    "actualDemand = data[\"ActualBaseDemand\"].values\n",
+    "targetDemand = data[\"TargetDemand\"].values\n",
+    "smartDemand = data[\"SmartDemand\"].values\n",
+    "uncontrolledDemand = data[\"UncontrolledDemand\"].values\n",
+    "spreadOutDemand = data[\"SpreadOutDemand\"].values\n",
+    "priceRatio = data[\"PriceRatio\"].values"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "**Plots of the simulation results**"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "plt.figure()\n",
+    "plt.gca().xaxis.set_major_formatter(mdates.DateFormatter(\"%d. %m.\\n%H:%M\"))\n",
+    "plt.gca().xaxis.set_minor_formatter(mdates.DateFormatter(\"%H:%M\"))\n",
+    "plt.gca().xaxis.set_major_locator(mdates.HourLocator(byhour=[0]))\n",
+    "plt.title(\"Base demand prediction vs. actual base demand\")\n",
+    "plt.plot(datetimes, predictedDemand, label=\"Predicted base demand\")\n",
+    "plt.plot(datetimes, actualDemand, label=\"Actual base demand\")\n",
+    "plt.plot(datetimes[0], [0])\n",
+    "plt.grid()\n",
+    "plt.legend(loc=\"upper center\", ncol=2)\n",
+    "plt.xlabel(\"Date and time\")\n",
+    "plt.ylabel(\"Demand [kW]\")\n",
+    "plt.show()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "plt.figure()\n",
+    "plt.gca().xaxis.set_major_formatter(mdates.DateFormatter(\"%d. %m.\\n%H:%M\"))\n",
+    "plt.gca().xaxis.set_minor_formatter(mdates.DateFormatter(\"%H:%M\"))\n",
+    "plt.gca().xaxis.set_major_locator(mdates.HourLocator(byhour=[0]))\n",
+    "plt.title(\"Ideal target demand the smart homes should try to reach\")\n",
+    "plt.plot(datetimes, predictedDemand, label=\"Predicted base demand\")\n",
+    "plt.plot(datetimes, predictedDemand + targetDemand, label=\"Predicted base demand + target demand\")\n",
+    "plt.plot(datetimes[0], [0])\n",
+    "plt.grid()\n",
+    "plt.legend(loc=\"upper center\", ncol=2)\n",
+    "plt.xlabel(\"Date and time\")\n",
+    "plt.ylabel(\"Demand [kW]\")\n",
+    "plt.show()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "scrolled": false
+   },
+   "outputs": [],
+   "source": [
+    "plt.figure()\n",
+    "plt.gca().xaxis.set_major_formatter(mdates.DateFormatter(\"%d. %m.\\n%H:%M\"))\n",
+    "plt.gca().xaxis.set_minor_formatter(mdates.DateFormatter(\"%H:%M\"))\n",
+    "plt.gca().xaxis.set_major_locator(mdates.HourLocator(byhour=[0]))\n",
+    "plt.title(\"Actual target demand the smart homes will try to reach\")\n",
+    "plt.plot(datetimes, actualDemand, label=\"Actual base demand\")\n",
+    "plt.plot(datetimes, actualDemand + targetDemand, label=\"Actual base demand + target demand\")\n",
+    "plt.plot(datetimes[0], [0])\n",
+    "plt.grid()\n",
+    "plt.legend(loc=\"upper center\", ncol=2)\n",
+    "plt.xlabel(\"Date and time\")\n",
+    "plt.ylabel(\"Demand [kW]\")\n",
+    "plt.show()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "plt.figure()\n",
+    "plt.gca().xaxis.set_major_formatter(mdates.DateFormatter(\"%d. %m.\\n%H:%M\"))\n",
+    "plt.gca().xaxis.set_minor_formatter(mdates.DateFormatter(\"%H:%M\"))\n",
+    "plt.gca().xaxis.set_major_locator(mdates.HourLocator(byhour=[0]))\n",
+    "plt.title(\"Target demand of the smart homes and the cheaper prices ratio generated from it\")\n",
+    "plt.plot(datetimes, targetDemand, label=\"Target demand\")\n",
+    "plt.plot(datetimes, priceRatio * houseCount, label=\"Price ratio\")\n",
+    "plt.plot(datetimes[0], [0])\n",
+    "plt.grid()\n",
+    "plt.legend(loc=\"upper center\", ncol=2)\n",
+    "plt.xlabel(\"Date and time\")\n",
+    "plt.ylabel(\"Demand [kW]\")\n",
+    "plt.show()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "plt.figure()\n",
+    "plt.gca().xaxis.set_major_formatter(mdates.DateFormatter(\"%d. %m.\\n%H:%M\"))\n",
+    "plt.gca().xaxis.set_minor_formatter(mdates.DateFormatter(\"%H:%M\"))\n",
+    "plt.gca().xaxis.set_major_locator(mdates.HourLocator(byhour=[0]))\n",
+    "plt.title(\"The simulated demand of the households according to different optimization algorithms\")\n",
+    "plt.plot(datetimes, actualDemand, label=\"Base demand\", zorder=0)\n",
+    "plt.plot(datetimes, actualDemand + targetDemand, label=\"Target demand\", zorder=4)\n",
+    "plt.plot(datetimes, actualDemand + uncontrolledDemand, label=\"Uncontrolled demand\", zorder=1)\n",
+    "plt.plot(datetimes, actualDemand + spreadOutDemand, label=\"Spread out demand\", zorder=2)\n",
+    "plt.plot(datetimes, actualDemand + smartDemand, label=\"Smart demand\", zorder=3)\n",
+    "plt.plot(datetimes[0], [0])\n",
+    "plt.grid()\n",
+    "plt.legend(loc=\"lower center\", ncol=3)\n",
+    "plt.xlabel(\"Date and time\")\n",
+    "plt.ylabel(\"Demand [kW]\")\n",
+    "plt.show()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.7.3"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}
diff --git a/simulator/downloadData.sh b/simulator/downloadData.sh
@@ -0,0 +1,6 @@
+#!/usr/bin/env bash
+
+DIR=$(dirname "${BASH_SOURCE[0]}")
+pushd "${DIR}" > /dev/null
+simulator/data/download.sh
+popd > /dev/null
diff --git a/simulator/requirements.txt b/simulator/requirements.txt
@@ -0,0 +1,7 @@
+matplotlib>=3.1.1
+numpy>=1.16.4
+pandas>=0.24.2
+psycopg2-binary>=2.8.3
+requests>=2.22.0
+scipy>=1.3.0
+SQLAlchemy>=1.3.5