Skip to content

Commit

Permalink
Improving the write-up.
Browse files Browse the repository at this point in the history
  • Loading branch information
@mmhs013
mmhs013 committed Jul 23, 2019
1 parent 95bdc72 commit 9f83b95
Show file tree
Hide file tree
Showing 6 changed files with 77 additions and 76 deletions.
8 changes: 4 additions & 4 deletions Examples/Example_pyMannKendall.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
"Three datasets are used here for demonstrating the use of this package. the datasets are\n",
"- **Daily Female Births Dataset:** This dataset describes the number of daily female births in California in 1959. This dataset is available in [here](https://raw.githubusercontent.com/jbrownlee/Datasets/master/daily-total-female-births.csv).\n",
"\n",
"- **Shampoo Sales Dataset:** This dataset describes the monthly number of sales of shampoo over a 3 year period. This dataset is available in [here](https://raw.githubusercontent.com/jbrownlee/Datasets/master/shampoo.csv)\n",
"- **Shampoo Sales Dataset:** This dataset describes the monthly number of sales of shampoo over 3 years. This dataset is available in [here](https://raw.githubusercontent.com/jbrownlee/Datasets/master/shampoo.csv)\n",
"\n",
"- **Air Passengers Dataset:** This famous dataset describes monthly international airline passengers (in thousands) from January 1949 to December 1960. It is widely used as a nonstationary seasonal time series. This dataset is available in [here](https://www.kaggle.com/freespirit08/time-series-for-beginners-with-arima/data)\n"
]
Expand Down Expand Up @@ -171,7 +171,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"From this acf plot, here show slightly autocorrelation in the first lag. We can ignore it. So, in our demonstration, we *assume* that there is no autocorrelation in **Daily Female Births Dataset**. So, to check the trend in this dataset, we can use *Original Mann Kendall test*. "
"From this ACF plot, here show slightly autocorrelation in the first lag. We can ignore it. So, in our demonstration, we *assume* that there is no autocorrelation in **Daily Female Births Dataset**. So, to check the trend in this dataset, we can use the *Original Mann Kendall test*. "
]
},
{
Expand Down Expand Up @@ -334,7 +334,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"From this acf plot, here show autocorrelation in the first lag. So, modified Mann Kendall test should be applied in here. We can use **Hamed and Rao Modified MK Test**, **Yue and Wang Modified MK Test**, **Modified MK test using Pre-Whitening method** or **Modified MK test using Trend free Pre-Whitening method** for this Shampoo Sales Dataset."
"From this ACF plot, here show autocorrelation in the first lag. So, modified Mann Kendall test should be applied in here. We can use **Hamed and Rao Modified MK Test**, **Yue and Wang Modified MK Test**, **Modified MK test using Pre-Whitening method** or **Modified MK test using Trend free Pre-Whitening method** for this Shampoo Sales Dataset."
]
},
{
Expand Down Expand Up @@ -541,7 +541,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"From this figure, we clearly observed a noticeable seasonality. So we should use **Seasonal MK Test** for this monthly dataset. And Because of this dataset is monthly data, so here we use **period=12**."
"From this figure, we observed a noticeable seasonality. So we should use **Seasonal MK Test** for this monthly dataset. And Because of this dataset is monthly data, so here we use **period=12**."
]
},
{
Expand Down
33 changes: 18 additions & 15 deletions Paper/paper.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -22,38 +22,41 @@ bibliography: paper.bib

# Summary

Trend analysis is one of the important measurements in studying time series data. Both parametric and non-parametric tests are commonly used in trend analysis. Parametric test require data to be independent and normally distributed. On the other hand, non-parametric trend tests require only that the data be independent and can tolerate outliers in the data [@hamed1998modified]. However, Parametric tests are more powerful than nonparametric ones.
Trend analysis is one of the important measurements in studying time series data. Both parametric and non-parametric tests are commonly used in trend analysis. Parametric test requires data to be independent and normally distributed. On the other hand, non-parametric trend tests require only that the data be independent and can tolerate outliers in the data [@hamed1998modified]. However, Parametric tests are more powerful than nonparametric ones.

The Mann–Kendall trend test [@mann1945nonparametric; @kendall1975rank] is one of the widely used non-parametric tests to detect significant trends in time series. However, the original Mann-Kendall test didn't consider serial correlation or seasonality effects [@bari2016analysis; @hirsch1982techniques]. But, in many real situations, the observed data are autocorrelated and this autocorrelation will result in misinterpretation of trend tests results [@hamed1998modified, @cox1955some]. Contrariwise, water quality, hydrologic, as well as climatic and other natural time series also have seasonality. To overcome those limitations of original Mann-Kendall test, various modified Mann-Kendall test have developed.
The Mann–Kendall trend test [@mann1945nonparametric; @kendall1975rank] is one of the widely used non-parametric tests to detect significant trends in time series. However, the original Mann-Kendall test didn't consider serial correlation or seasonality effects [@bari2016analysis; @hirsch1982techniques]. But, in many real situations, the observed data are autocorrelated and this autocorrelation will result in misinterpretation of trend tests results [@hamed1998modified, @cox1955some]. Contrariwise, water quality, hydrologic, as well as climatic and other natural time series also have seasonality. To overcome those limitations of original Mann-Kendall test, various modified Mann-Kendall test has developed.

Again, Python is one of the widely used tools for data analysis. A large number of data analysis and research tools are also developed using python. But, till now, there is no Mann-Kendall trend relation python package available. ``pyMannKendall`` package is going to fill up this gap.

``pyMannKendall`` is written in pure python and use vectorization approach to increase its performance. Currently, this package has 11 Mann-Kendall Tests and 2 sen’s slope estimator function. Brief description of functions are below:

1. **Original Mann Kendall test (*original_test*):** Original Mann Kendall test [@mann1945nonparametric; @kendall1975rank] is a nonparametric test, which does not consider serial correlation or seasonal effects.
1. **Original Mann-Kendall test (*original_test*):** Original Mann-Kendall test is a nonparametric test, which does not consider serial correlation or seasonal effects.

2. **Hamed and Rao Modified MK Test (*hamed_rao_modification_test*):** This modified MK test proposed by Hamed and Rao [@hamed1998modified] to address serial autocorrelation issues. They suggested variance correction approach to improve trend analysis. User can consider first n significant lag by insert lag number in this function. By default, it considered all significant lags.
2. **Hamed and Rao Modified MK Test (*hamed_rao_modification_test*):** This modified MK test proposed by Hamed and Rao (1998) to address serial autocorrelation issues. They suggested a variance correction approach to improve trend analysis. User can consider first n significant lag by insert lag number in this function. By default, it considered all significant lags.

3. **Yue and Wang Modified MK Test (*yue_wang_modification_test*):** This is also a variance correction method for considered serial autocorrelation proposed by Yue, S., & Wang, C. Y. (2004) [@yue2004mann]. User can also set their desired significant n lags for the calculation.
3. **Yue and Wang Modified MK Test (*yue_wang_modification_test*):** This is also a variance correction method for considered serial autocorrelation proposed by Yue, S., & Wang, C. Y. (2004). User can also set their desired significant n lags for the calculation.

4. **Modified MK test using Pre-Whitening method (*pre_whitening_modification_test*):** This test suggested by Yue and Wang (2002) [@yue2002applicability] to using Pre-Whitening the time series prior to the application of trend test.
4. **Modified MK test using Pre-Whitening method (*pre_whitening_modification_test*):** This test suggested by Yue and Wang (2002) to using Pre-Whitening the time series before the application of trend test.

5. **Modified MK test using Trend free Pre-Whitening method (*trend_free_pre_whitening_modification_test*):** This test also proposed by Yue and Wang (2002) [@yue2002influence] to remove trend component and then Pre-Whitening the time series prior to application of trend test.
5. **Modified MK test using Trend free Pre-Whitening method (*trend_free_pre_whitening_modification_test*):** This test also proposed by Yue and Wang (2002) to remove trend component and then Pre-Whitening the time series before application of trend test.

6. **Multivariate MK Test (*multivariate_test*):** This is an MK test for multiple parameters proposed by Hirsch (1982) [@hirsch1982techniques]. Actually, he used this method for seasonal mk test, where he considered every month is a parameter.
6. **Multivariate MK Test (*multivariate_test*):** This is an MK test for multiple parameters proposed by Hirsch (1982). He used this method for seasonal mk test, where he considered every month as a parameter.

7. **Seasonal MK Test (*seasonal_test*):** For seasonal time series data, Hirsch, R.M., Slack, J.R. and Smith, R.A. (1982) [@hirsch1982techniques] proposed this test to calculate the seasonal trend.
7. **Seasonal MK Test (*seasonal_test*):** For seasonal time series data, Hirsch, R.M., Slack, J.R. and Smith, R.A. (1982) proposed this test to calculate the seasonal trend.

8. **Regional MK Test (*regional_test*):** Based on Hirsch (1982) [@hirsch1982techniques] proposed seasonal mk test, Helsel, D.R. and Frans, L.M., (2006) [@helsel2006regional] suggest regional mk test to calculate overall trend a regional scale.
8. **Regional MK Test (*regional_test*):** Based on Hirsch (1982) proposed seasonal mk test, Helsel, D.R. and Frans, L.M., (2006) suggest regional mk test to calculate the overall trend in a regional scale.

9. **Correlated Multivariate MK Test (*correlated_multivariate_test*):** This multivariate mk test proposed by Hipel (1994) [@hipel1994time] for when parameters are correlated.
9. **Correlated Multivariate MK Test (*correlated_multivariate_test*):** This multivariate mk test proposed by Hipel (1994) where the parameters are correlated.

10. **Correlated Seasonal MK Test (*correlated_seasonal_test*):** This method proposed by Hipel (1994) [@hipel1994time] used, when time series significantly correlated with the preceding one or more months/seasons.
10. **Correlated Seasonal MK Test (*correlated_seasonal_test*):** This method proposed by Hipel (1994) used, when time series significantly correlated with the preceding one or more months/seasons.

11. **Partial MK Test (*partial_test*):** In a real event, many factors are affecting the main studied response parameter, which can bias the trend results. To overcome this problem, Libiseller (2002) [@libiseller2002performance] proposed this partial mk test. It required two parameters as input, where, one is response parameter and other is an independent parameter.
11. **Partial MK Test (*partial_test*):** In a real event, many factors are affecting the main studied response parameter, which can bias the trend results. To overcome this problem, Libiseller (2002) proposed this partial mk test. It required two parameters as input, where, one is response parameter and other is an independent parameter.

12. **Theil–Sen’s Slope Estimator (*sens_slope*):** This method proposed by Theil (1950) [@theil1950rank] and Sen (1968) [@sen1968estimates] to estimate the magnitude of the monotonic trend.
12. **Theil-sen's Slope Estimator (*sens_slope*):** This method proposed by Theil (1950) and Sen (1968) to estimate the magnitude of the monotonic trend.

13. **Seasonal Sen’s Slope Estimator (*seasonal_sens_slope*):** This method proposed by Hipel (1994) [@hipel1994time] to estimate the magnitude of the monotonic trend, when data has seasonal effects.
13. **Seasonal sen's Slope Estimator (*seasonal_sens_slope*):** This method proposed by Hipel (1994) to estimate the magnitude of the monotonic trend, when data has seasonal effects.


`pyMannKendall` is a pure Python implemented non-parametric Mann-Kendall trend analysis package, which bring together almost all types of Mann-Kendall Test, which might help researchers to check the Mann-Kendall trend in python.

# References
24 changes: 12 additions & 12 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,28 +7,28 @@
[![PyPI - Python Version](https://img.shields.io/pypi/pyversions/pymannkendall.svg)](https://pypi.org/project/pymannkendall/)
[![status](http://joss.theoj.org/papers/14903dbd55343be89105112e585d262a/status.svg)](http://joss.theoj.org/papers/14903dbd55343be89105112e585d262a)

## What is the Mann Kendall Test ?
The Mann Kendall Trend Test (sometimes called the MK test) is used to analyze time series data for consistently increasing or decreasing trends (monotonic trends). It is a non-parametric test, which means it works for all distributions (i.e. data doesn't have to meet the assumption of normality), but data should have no serial correlation. If data has a serial correlation, it could be affected in significant level (p-value). That's why results might be misinterpreted. To overcome this problem, researchers proposed several modified Mann Kendall tests (Hamed and Rao Modified MK Test, Yue and Wang Modified MK Test, Modified MK test using Pre-Whitening method etc.). Seasonal Mann Kendall test also developed to remove the effect of seasonality.
## What is the Mann-Kendall Test ?
The Mann-Kendall Trend Test (sometimes called the MK test) is used to analyze time series data for consistently increasing or decreasing trends (monotonic trends). It is a non-parametric test, which means it works for all distributions (i.e. data doesn't have to meet the assumption of normality), but data should have no serial correlation. If the data has a serial correlation, it could affect in significant level (p-value). It could lead to misinterpretation. To overcome this problem, researchers proposed several modified Mann-Kendall tests (Hamed and Rao Modified MK Test, Yue and Wang Modified MK Test, Modified MK test using Pre-Whitening method, etc.). Seasonal Mann-Kendall test also developed to remove the effect of seasonality.

Mann Kendall Test is a powerful trend test, so several others modified Mann Kendall tests like Multivariate MK Test, Regional MK Test, Correlated MK test, Partial MK Test etc. were developed for the spacial condition. `pyMannkendal` is a pure Python implementation of non-parametric Mann Kendall trend analysis, which bring together almost all types of Mann Kendall Test. Currently, this package has 11 Mann-Kendall Tests and 2 sen's slope estimator function. Brief description of functions are below:
Mann-Kendall Test is a powerful trend test, so several others modified Mann-Kendall tests like Multivariate MK Test, Regional MK Test, Correlated MK test, Partial MK Test, etc. were developed for the spacial condition. `pyMannkendal` is a pure Python implementation of non-parametric Mann-Kendall trend analysis, which bring together almost all types of Mann-Kendall Test. Currently, this package has 11 Mann-Kendall Tests and 2 sen's slope estimator function. Brief description of functions are below:

1. **Original Mann Kendall test (*original_test*):** Original Mann Kendall test is a nonparametric test, which does not consider serial correlation or seasonal effects.
1. **Original Mann-Kendall test (*original_test*):** Original Mann-Kendall test is a nonparametric test, which does not consider serial correlation or seasonal effects.

2. **Hamed and Rao Modified MK Test (*hamed_rao_modification_test*):** This modified MK test proposed by Hamed and Rao (1998) to address serial autocorrelation issues. They suggested variance correction approach to improve trend analysis. User can consider first n significant lag by insert lag number in this function. By default, it considered all significant lags.
2. **Hamed and Rao Modified MK Test (*hamed_rao_modification_test*):** This modified MK test proposed by Hamed and Rao (1998) to address serial autocorrelation issues. They suggested a variance correction approach to improve trend analysis. User can consider first n significant lag by insert lag number in this function. By default, it considered all significant lags.

3. **Yue and Wang Modified MK Test (*yue_wang_modification_test*):** This is also a variance correction method for considered serial autocorrelation proposed by Yue, S., & Wang, C. Y. (2004). User can also set their desired significant n lags for the calculation.

4. **Modified MK test using Pre-Whitening method (*pre_whitening_modification_test*):** This test suggested by Yue and Wang (2002) to using Pre-Whitening the time series prior to the application of trend test.
4. **Modified MK test using Pre-Whitening method (*pre_whitening_modification_test*):** This test suggested by Yue and Wang (2002) to using Pre-Whitening the time series before the application of trend test.

5. **Modified MK test using Trend free Pre-Whitening method (*trend_free_pre_whitening_modification_test*):** This test also proposed by Yue and Wang (2002) to remove trend component and then Pre-Whitening the time series prior to application of trend test.
5. **Modified MK test using Trend free Pre-Whitening method (*trend_free_pre_whitening_modification_test*):** This test also proposed by Yue and Wang (2002) to remove trend component and then Pre-Whitening the time series before application of trend test.

6. **Multivariate MK Test (*multivariate_test*):** This is an MK test for multiple parameters proposed by Hirsch (1982). Actually, he used this method for seasonal mk test, where he considered every month is a parameter.
6. **Multivariate MK Test (*multivariate_test*):** This is an MK test for multiple parameters proposed by Hirsch (1982). He used this method for seasonal mk test, where he considered every month as a parameter.

7. **Seasonal MK Test (*seasonal_test*):** For seasonal time series data, Hirsch, R.M., Slack, J.R. and Smith, R.A. (1982) proposed this test to calculate the seasonal trend.

8. **Regional MK Test (*regional_test*):** Based on Hirsch (1982) proposed seasonal mk test, Helsel, D.R. and Frans, L.M., (2006) suggest regional mk test to calculate overall trend a regional scale.
8. **Regional MK Test (*regional_test*):** Based on Hirsch (1982) proposed seasonal mk test, Helsel, D.R. and Frans, L.M., (2006) suggest regional mk test to calculate the overall trend in a regional scale.

9. **Correlated Multivariate MK Test (*correlated_multivariate_test*):** This multivariate mk test proposed by Hipel (1994) for when parameters are correlated.
9. **Correlated Multivariate MK Test (*correlated_multivariate_test*):** This multivariate mk test proposed by Hipel (1994) where the parameters are correlated.

10. **Correlated Seasonal MK Test (*correlated_seasonal_test*):** This method proposed by Hipel (1994) used, when time series significantly correlated with the preceding one or more months/seasons.

Expand All @@ -40,14 +40,14 @@ Mann Kendall Test is a powerful trend test, so several others modified Mann Kend

## Function details:

All Mann-kendall test functions has almost similer input parameters. Those are:
All Mann-Kendall test functions have almost similar input parameters. Those are:

- **x**: a vector of data
- **alpha**: significance level (0.05 default)
- **lag**: No. of First Significant Lags (Only available in hamed_rao_modification_test and yue_wang_modification_test)
- **period**: seasonal cycle. For monthly data it is 12, weekly data it is 52 (Only available in seasonal tests)

And all Mann-kendall tests return a named tuple which contained:
And all Mann-Kendall tests return a named tuple which contained:

- **trend**: tells the trend (increasing, decreasing or no trend)
- **h**: True (if trend is present) or False (if trend is absence)
Expand Down
Loading

0 comments on commit 9f83b95

Please sign in to comment.