Saving and loading snapshots multiple times

[cuihantao]

@MaKla89

Here's a fix for your use case. See my notes enclosed as well.

import andes
import os
import numpy as np
import pandas as pd
from andes.interop.pandapower import to_pandapower, \
    make_link_table, runopp_map, add_gencost
import matplotlib as plt

# plt.use('module://backend_interagg')   # Just needed for using SciView in PyCharm
# plt.use('QTagg')

%matplotlib inline
pd.options.display.width = 0

# -----------------------------------------------
#          Initialise Andes System
# -----------------------------------------------

gv_ps = andes.load(andes.get_case('ieee14/ieee14_ieeet1.xlsx'),
                   setup=False,
                   no_output=True,
                   default_config=True)

gv_ps.Toggle.u.v = [0, 0]
gv_ps.setup()
gv_ps.PFlow.run()

# Change PQ-load to constant power
gv_ps.PQ.config.p2p = 1.0
gv_ps.PQ.config.p2i = 0
gv_ps.PQ.config.p2z = 0
gv_ps.PQ.config.q2q = 1.0
gv_ps.PQ.config.q2i = 0
gv_ps.PQ.config.q2z = 0

gv_ps.TDS.config.tf = 10
gv_ps.TDS.run()




# -----------------------------------------------
#             Inject Disturbance(s)
# -----------------------------------------------

gv_ps.PQ.alter("Ppf", "PQ_8", 0.5)

# -----------------------------------------------
#     Continue Dynamic Sim until Snapshot-Time
# -----------------------------------------------

gv_ps.TDS.config.tf = 25
gv_ps.TDS.run()

# -----------------------------------------------
#   Initialize PP-OPF with Snapshot-Measurments
#   (SWITCH TO OPERATOR-VIEW / PERCEIVED VIEW !)
# -----------------------------------------------

gv_ps.Slack.u.v[:] = 1
gv_ps.PV.u.v[:] = 1


andes.utils.snapshot.save_ss("snapshot.pkl", gv_ps)
gv_ps = andes.utils.snapshot.load_ss("snapshot.pkl")  # <------ left-hand side changed to `gv_ps`
# RecursionError happens upon the second run, if a new variable name, say `pv_ps`, is assigned for the loaded ANDES system. 
# This is probably a bug in `dill`'s scoping algorithm, and it is really hard to track down.
# If the modified variable name provides the intended use case, you can stick with the same variable name.

pv_ps = to_pandapower(gv_ps, verify=False)  # < ----- MUST NOT verify power flow after initializing TDS in ANDES.
# ANDES does not allow running PFlow for systems with initialized TDS as it will break variable addressing

gv_ps.Slack.u.v[:] = 0
gv_ps.PV.u.v[:] = 0

pv_ps.load.loc[:, "p_mw"] = gv_ps.PQ.Ppf.v*100
pv_ps.load.loc[:, "q_mvar"] = gv_ps.PQ.Qpf.v*100

gv_ps.TDS.config.tf = 40
gv_ps.TDS.run()

# -----------------------------------------------
#    Run PP-OPF as Operator's Decision-Making
# -----------------------------------------------

gen_cost = np.array([
        [2, 0, 0, 3, 0.0430293, 20, 0],
        [2, 0, 0, 3, 0.25,      20, 0],
        [2, 0, 0, 3, 0.01,      40, 0],
        [2, 0, 0, 3, 0.01,      40, 0],
        [2, 0, 0, 3, 0.01,      40, 0]
])

add_gencost(pv_ps, gen_cost)
link_table = make_link_table(gv_ps)
gv_ps_opt = runopp_map(pv_ps, link_table)






# -----------------------------------------------
#   Continue Dynamic Sim while Decision-Making
# -----------------------------------------------

gv_ps.TDS.config.tf = 40
gv_ps.TDS.run()

# -----------------------------------------------
#   Map Control-Setpoints to ANDES-Changes
#    (SWITCH TO GLOBAL VIEW / GOD-VIEW !)
# -----------------------------------------------

gv_ps_gov_idx = list(gv_ps_opt['gov_idx'][~gv_ps_opt['gov_idx'].isna()])
gv_ps.TurbineGov.set(src='pref0', idx=gv_ps_gov_idx, attr='v', value=gv_ps_opt['p'][~gv_ps_opt['gov_idx'].isna()])





# -----------------------------------------------
#  Continue ANDES after application of setpoints
# -----------------------------------------------

gv_ps.TDS.config.tf = 60
gv_ps.TDS.run()






# -----------------------------------------------
#           Trying Second OPF-Loop
# -----------------------------------------------

os.remove("snapshot.pkl")
andes.utils.snapshot.save_ss("snapshot.pkl", gv_ps)  # Trying to create another deep-copy at a later point in TDS
pv_ps = andes.utils.snapshot.load_ss("snapshot.pkl")





# -----------------------------------------------
#              Prints and Plots
# -----------------------------------------------

gv_ps.TDS.load_plotter()
gv_ps.TDS.plotter.plot(gv_ps.Bus.v)
gv_ps.TDS.plotter.plot(gv_ps.Bus.a)
gv_ps.TDS.plotter.plot(gv_ps.TDS.plotter.find("omega")[0])
gv_ps.TDS.plotter.plot(gv_ps.TDS.plotter.find("pref")[0])
gv_ps.TDS.plotter.plot(gv_ps.TDS.plotter.find("pout")[0])

[cuihantao]

I have no idea yet about the RecursionError when a new variable is used. Probably it has to do with the ensuing line that assigns the return of to_pandapower to the same variable.

The code below works fine with loaded snapshots in different names. I would appreciate it if you poke around and let me know.

import andes
from matplotlib import pyplot

%matplotlib inline

andes.config_logger()

ss = andes.system.example()

ss.PFlow.run()
ss.TDS.init();

# --- First save and load ---

from andes.utils.snapshot import save_ss, load_ss

save_ss("snapshot.pkl", ss)

ss_new = load_ss("snapshot.pkl")

ss_new.TDS.run()

ss_new.TDS.plt.plot(ss.GENROU.omega)

# --- Second save and load ---
save_ss("snapshot.pkl", ss_new)

ss_new2 = load_ss("snapshot.pkl")

ss_new2.TDS.config.tf = 30
ss_new2.TDS.run()

ss_new2.TDS.plt.plot(ss_new2.GENROU.omega)

[MaKla89]

Thanks for the massive amount of energy and time you put into this :)

1. You are right, if I cycle / change the names for the temporary grid, it is working
2. I am somewhat happy that this bug is NOT a total embarassment, but seems to be a nasty little bugger ;)
3. I'm still ashamed as your SECOND comment (about NOT running PF-calculations after ANDES-TDS....) made that whole "temporary system"-concept irrelevant. This is nice to know, but I made quite some fuzz about nothing, it seems :D

[cuihantao]

Thanks!

To your third point, I didn't read all the code to understand the need for that temporary system. Time-domain simulation, at each step, involves the solution of algebraic equations, which includes all the power flow equations. It is just that the to_pandapower converter must not try to solve power flow again in ANDES. The converter will give you an output pandapower system anyway. I hope at least you could find workarounds if this is truly an issue.

Not being able to run the power flow after TDS initialization is a coding issue. Variable addresses get extended upon TDS initialization as is the case in PSS/E.

[jinningwang]

Hello Hantao @cuihantao,

pv_ps = to_pandapower(gv_ps, verify=False)  # < ----- MUST NOT verify power flow after initializing TDS in ANDES.
# ANDES does not allow running PFlow for systems with initialized TDS as it will break variable addressing

I might have more comments on this. It is correct that gv_ps will run PFlow if the verify=True.

I think maybe I can improve this by creating a copied ANDES system using the file source and all the config in gv_ps so the source ANDES system remained untouched.
What do you think about this?

Also, I am thinking that we can include a chapter or subchapter in the documentation to further explain the interop module. I can help to give a draft version of the pandapower interface. Do you think this is helpful?

Regards,
Jinning

[cuihantao]

It is doable, but that sounds like manually copying data into a new andes.System. Creating a new system from the source file won't solve the problem because, e.g., load and generation data may have been changed.

Can the proposed document go to https://docs.andes.app/en/latest/examples/pandapower.html?

[jinningwang]

It is doable, but that sounds like manually copying data into a new andes.System. Creating a new system from the source file won't solve the problem because, e.g., load and generation data may have been changed.

I suppose this change to be helpful in that the initial conversion to pandapower would not change the original andes.System at least.
As for the possible data change, I am thinking that a new function track_system can be developed to copy data of andes.System to its converted pandapower system. However, this is not that compellable to me right now.

Can the proposed document go to https://docs.andes.app/en/latest/examples/pandapower.html?

Sure, I can include more explanations in this example file.

My final conclusion is that I can include more explanations in the example of pandapower interface and keep all the thoughts for the development of AMS.

Regards,
Jinning

[MaKla89]

It seems, the discussion-feature / tab on the repository has been deactivated currently. Hence, I'll reply via mail: I really like the idea of the track_system function (if I understood it correctly) and was starting work on that myself. Currently it's basically trying to "update" the PP-system by always creating a new one via: def create_pps(ssa: andes.system, verbose=False):

if verbose: inspect_andes_system(ssa) ssa.Slack.u.v[:] = 1 ssa.PV.u.v[:] = 1 ssp = to_pandapower(ssa, verify=False) # Verfication useless as Post-TDS andes systems cant do Power Flow anymore ssa.Slack.u.v[:] = 0 ssa.PV.u.v[:] = 0 ssp.load.loc[:, "p_mw"] = ssa.PQ.Ppf.v * 100 ssp.load.loc[:, "q_mvar"] = ssa.PQ.Qpf.v * 100 if verbose: inspect_pandapower_system(ssp) return ssp

Second step is the OPF-execution, again based on Jinning Wangs ideas:

def run_pp_opf(ssa: andes.system, ssp: pandapower.auxiliary.pandapowerNet, gen_cost=gen_cost_default_IEEE14, verbose=False): add_gencost(ssp, gen_cost) link_table = make_link_table(ssa) # if verbose: # print("Link-Table:") # print(link_table) # print() ssa_opt = runopp_map(ssp, link_table) if verbose: print("Optimal Results:") print(ssa_opt) print() return ssa_opt

And (at an optionally later point in time) I bring back in the OPF-setpoints: def inject_opf_setpoints(ssa: andes.system, ssa_opt, verbose=False):

ssa_gov_idx = list(ssa_opt['gov_idx'][~ssa_opt['gov_idx'].isna()]) if verbose: print("Generator P setpoints BEFORE OPF") print(ssa.TurbineGov.get(src='pref0', idx=ssa_gov_idx, attr='v')) print() ssa.TurbineGov.set(src='pref0', idx=ssa_gov_idx, attr='v', value=ssa_opt['p'][~ssa_opt['gov_idx'].isna()]) if verbose: print("Generator P setpoints AFTER OPF") print(ssa.TurbineGov.get(src='pref0', idx=ssa_gov_idx, attr='v')) print() ssa_exciter_idx = list(ssa_opt['exc_idx'][~ssa_opt['exc_idx'].isna()]) if verbose: print("Generator V setpoints BEFORE OPF") print(ssa.Exciter.get(src='vref0', idx=ssa_exciter_idx, attr='v')) print() ssa.Exciter.set(src='vref0', idx=ssa_exciter_idx, attr='v', value=ssa_opt['vm_pu'][~ssa_opt['exc_idx'].isna()]) if verbose: print("Generator V setpoints AFTER OPF") print(ssa.Exciter.get(src='vref0', idx=ssa_exciter_idx, attr='v')) print() Ultimately, this merely tries to create a PP-system based on a

POST-TDS-andes system by replacing the PQ-bus-values for active and reactive power with the TDS-values "Ppf" and "Qpf" (assuming constant load services. Anything else would defy the point of bringing-in OPF-results). After OPF both active generator output (TurbineGovs --> pref0) and voltage-setpoints for generators (Exciters --> vref0) are mapped back from OPF to ANDES. I must admit that especially the voltage-mapping does not seem to be correctly done, but it is doing SOMETHING ;) At the beginning, both Slack-bus and PV-buses need to be re-activated, otherwise the to_pandapower-function will not work. I DO re-disable both them right after the creation of the PP-system again, just to recreate the state of the ANDES system from where it stopped after the last TDS. I do not yet fully understand why the TDS disables slack-nodes (or where it does that, actually), but the TDS will work / continue with any combination of enabled or disabled Slack and PV-busses. This is my current state, thanks a lot for the inspirations and code so far, both of you! :)

…

Message ID: ***@***.*** com>

[cuihantao]

Dear @MaKla89 ,

Thank you for your thoughts!

I did not turn off Discussions. The repository has been transferred to our organizational account curent. All discussion records are retained at https://github.com/CURENT/andes/discussions. You are welcome to post future questions or suggestions at the new location. The previous URL https://github.com/cuihantao/andes is now a fork so it doesn't have Discussions enabled.

I must admit that especially the voltage-mapping does not seem to be correctly done, but it is doing SOMETHING ;)

This is due to the variety of exciter models. vref0 does not always mean terminal voltage setpoint. vref0 is an internal constant computed upon initialization to satisfy steady-state conditions. For some models, vref0 is the terminal voltage but not for others. You will need to inspect the control block to tell.

I do not yet fully understand why the TDS disables slack-nodes (or where it does that, actually), but the TDS will work / continue with any combination of enabled or disabled Slack and PV-busses.

I am glad to explain that. Slack enforces voltage magnitude and angle at any instant. Such strong control capability is too ideal to be useful in transient stability studies. In general, static generators, including PV and Slack, has to be replaced by generators connected to the same bus. The dynamic generators will inject the same amount of power as the static ones. The static ones, once substituted, need to be removed for power flow consistency.

To save some coding effort, instead of eliminating the replaced static generators, I retained them and turn them off.

[jinningwang]

Hello @MaKla89,

For the voltage issue, I may have more to comment. When developing this interface, I also went into this issue.
For a relatively short time window, e.g. 1 hour, the voltage can be kind of constant. Then the voltage of dynamic generators is also very close to the voltage that is initially mapped to StaticGen.
For a longer time window where the assumption is not true anymore, it is still not solved how to map the voltage to the dynamic generators. One possible solution can be adding PI controllers to regulate the output voltage of generators. However, I am not sure if that is realistic for power system operation.
Welcome any other discussion if you have more thoughts on this.

Regards,
Jinning

[MaKla89]

Thanks, both of you :)

@cuihantao : Thanks for the clarification on both, the moved repository and the reasons for disabled slack nodes and static generators, that already helps a lot :) Also thanks for the hint regarding the vref0 setpoint. Of course, this is a bit inconvenient, but I'll try to check it. vref0was the first (and so far only) voltage-related service I was able to change AND see an impact on the simulation results.

@jinningwang : Ah! Very interesting observation, thank you! May I ask HOW you were trying to map the voltage setpoints? Which exciter-variables / services did you alter corresponding to the OPF-results? :) I do mostly just simulate timeframes of up to a few minutes, but it would still be interesting to understand why (and when exactly) that behaviour tilts.

My observations with the current state as I proposed: The OPF seems to provide a nice set of setpoints for Pref0, leading to an excellent active power balance and a full frequency restoration. Regarding the voltage, though, things look a bit off. Take a look at my attached example-results:

As an explanation: The first 20 seconds serve as an initialisation period. It means TDS is run until t=1s , then the first round of OPF is done to get initial sepoints from the OPF, then the TDS is continued (with OPF-setpoints) until t=20s. At t=20s a disturbance (active load step) is introduced, which can be observed nicely. Both inertia-response and FCR-response of the GENROU-generators can be seen in a nice way, containing the frequency-drop successfully. At t=30s I do create a new, updated pandapower-network with TDS-results and I run the OPF. I do resume the TDS (WITHOUT any changes yet) until t=40s, which is where I assume the setpoints are successfully communciated back from central OPF-instance to all controllers. And you can see that the OPF nicely acts as an FRR-service, bringing frequency back to 1 p.u. .

The voltages on the other look a bit off regarding the OPF-results. The voltag angle is odd, but I think thats okay as all voltage angles stay small when referenced to Bus_1 voltage angle. They just "drifted away" if compared to the original slack angle (which does not exist any longer anyways). In other scenarios WITHOUT any disturbance I found the OPF-solution leading to voltage-band violations, but I'll need further assessment on that first. BUT you can already see that the angle is drifting even before the disturbance even happens (from 1s < t < 20s). Might not be relevant as this angle-plot is wonky anyways, but it is an observation for now ;)

[jinningwang]

Hi @MaKla89,

For the voltage, I can map the results of generator voltage from pandapower into StaticGen.v0 in ANDES before the system runs PFlow and TDS. Then the dynamic generator will be assigned the voltage according to the corresponding StaticGen defined by the IdxParam gen.
In this way, you can only map the voltage one time because once the TDS has started the StaticGen will be removed. I haven't figured out a way to track the voltage setting points from OPF results if the TDS has begun.

Regarding the voltage angles, it is okay as all the angles run away together as long as they do not run away from each other.

Regards,
Jinning

[MaKla89]

Ah, thanks for your reply, @jinningwang ! I agree with you on the angles, eventhough it looks odd as a plot.

About the voltage-mapping: I forgot that you can directly map them onto v0, that is good point. Well, it seems like my approach with vref0 (equivalent to pref0) seems to make okay-ish sense, but I'm not 100% convinced it is correct. We can try around together, in case you need this too (which I don't expect). Otherwise I'll keep you posted if I find a solution or a validation for vref0 actually being correct :)