Implement exogenous processes.

[tobiasraabe]

• respy version used, if any: any
• Python version, if any: any
• Operating System: any

Exogenous processes are an important part of every modern structural model. Processes in
general govern the transition between states and the normal model has an endogenous
human capital accumulation process. Endogenous means that the individual herself
controls the transition which actually happens with probability one.

In contrast, exogenous processes are not controlled by individuals and transition
between states happens with some probability dependent on the individuals
characteristics (the state).

We continue the explanation with example model kw_94_one. Furthermore, these changes
build on #310.

Processes affecting state characteristics

The probably easiest example of an exogenous processes is one which models severe
illnesses. Assume that with probability 1% every individual may transition to a state
where all choices except home are affected by a huge negative non-pecuniary penalty.

Interface

The interface for an exogenous process resembles the interface for observables. First,
there are the parameters which define the transition probabilities of the process.

exogenous_process_illness_healthy,probability,0.99
exogenous_process_illness_sick,probability,0.01

exogenous_process is the keyword and illness is the name of the process. healthy
and sick are the two possible states of the individual. probability signals that the
following value is a probability and all values under exogenous_process_illness_* have
to sum up to one.

This is the same interface as for previous choices and observables. The user can create
more complex processes by replacing probability with the name of covariates. Then, the
parameter values are assumed to be the coefficient for a logit function. More
information here.

Secondly, the outcomes of the process have to affect the non-pecuniary rewards of all
choices except home.

nonpec_a,sick,-15000,Penalty for being sick.
nonpec_b,sick,-15000,Penalty for being sick.
nonpec_edu,sick,-15000,Penalty for being sick.

Implementation

Here are the necessary changes.

• Similar to observables, the realizations of the exogenous process duplicate the state
  space which increases the dense state space grid in _create_dense_state_space_grid.

• Because the process is exogenous, we can pre-compute the transition probabilities. One
  would loop over every _SingleDimStateSpace (jargon for the core state combined with
  one realization for each of the dense dimensions) and create vectors for each
  exogenous process for every realization of the exogenous process in the next period.
  The vectors would have as many elements as there are core states.

  For probabilities, one could argue to keep only scalars and not repeat for all states.
  Broadcasting will take of that later. For logit coefficients, we will have vectors.

• A major happens to the generation of continuation values. Without exogenous processes,
  the continuation value is given by one expected value function in a future period for
  one choice. With exogenous processes, there exist two continuation values for each
  realization of the exogenous process. The two continuation values have to be
  aggregated via the probabilities.

  How does it work if we call get_continuation_values() on _SingleDimStateSpace
  (like for kw_94_one)? We have a matrix with shape (n_states, n_choices) which
  contains the index of the child state in the next period for each choice. Then, we use
  the indices to reorder the vector of expected value functions which has one value per
  state to get a matrix of continuation values.

  If we have exogenous processes, there exists a _MultiDimStateSpace which collects
  all _SingleDimStateSpaces under the attribute sub_state_spaces. The multi state
  space calls the get_continuation_value() functions of the single state spaces and
  collects all the matrices.

  Assuming we have only one exogenous process, we would take the vector of transition
  probabilities to state of sickness and compute the dot product with the matrix of
  corresponding continuation values. Then, do the same for the healthy states. At last,
  compute the sum of the two matrices for the aggregated continuation values.

  What happens for multiple exogenous processes? At first, we need to calculate the
  combined transition probability to a state. Thus, for each realization of an exogenous
  process get the vector of transition probabilities. Then, perform element-wise
  multiplication with the vectors. Then, continue as before with one probability vector
  for each matrix of continuation values.

• The last change happens in _apply_law_of_motion while simulating single periods.
  Using the transition probabilities, we sample the realizations of the exogenous
  process for the next period.

Processes affecting the choice set (more experimental)

Some structural models have exogenous processes which affect the available actions in
one period. They are used to model the availability of job offers, training programs,
etc..

Retirement

The easiest example of such a process is retirement which will happen with some
probability starting with some age. If an individual decides to retire, she cannot
return to the labor market. We assume retirement is possible with age 45 (assuming 40
periods starting with age 16 this leaves 10 final periods) and the transition
probability rises to one at age 55 such that all individuals are retired in the last
period.

Interface

Retirement is added as a new choice similar to home production without experience to the
reward parameters.

nonpec_retirement,constant,10000

The probability of retirement is defined below.

exogenous_process_retirement_in_labor_force,constant,0
exogenous_process_retirement_retired,constant,-0.7
exogenous_process_retirement_retired,before_age_45,-700
exogenous_process_retirement_retired,period_since_age_45,-0.4
exogenous_process_retirement_retired,period_since_age_45_sq,0.1

If this process is estimated alongside the other model parameters, fix the first
parameter to have a base category.

The corresponding covariates

covariates:
  constant: "1"
  before_age_45: period < 28
  period_since_age_45: period - 29
  period_since_age_45_sq: before_age_45 ** 2

The parameters form an exogenous retirement process whose transition probabilities from
age 45 to 55 are

[0.73, 0.27],
[0.75, 0.25],
[0.73, 0.27],
[0.67, 0.33],
[0.55, 0.45],
[0.38, 0.62],
[0.2 , 0.8 ],
[0.08, 0.92],
[0.02, 0.98],
[0.  , 1.  ],

So, everyone will finally transition to retirement from age 55 to 56.

Implementation

In addition to the former example and implementation details, we need the following.

• At first, we add a similar choice to home which is called retirement; no experience!

• core_state_space_filters can be used to filter states with retirement as the
  previous choice before age 45 to reduce the computational burden.

• To indicate whether a choice is available, we add new dense dimension which signals
  with 0 in_labor_force and 1 retired. Depending on value of this dense dimension,
  is_inadmissible is set to True for choice retirement.

• [Optional] In the part of the state space where retirement is on, all periods before
  29 will not ever be reached and no choice is available. It should be possible to skip
  these dimensions during the backward induction. Starting with period 29, only
  retirement is possible. Thus, we should be able to simplify the MC simulation to this
  vector as well.

@peisenha @janosg @mo2561057 Call for feedback :)