Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ticket migrate #13

Merged
merged 39 commits into from
Nov 11, 2021
Merged
Show file tree
Hide file tree
Changes from 24 commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
6e1c827
Parse Trac component, priority, and keywords into GitHub labels.
danuker Oct 5, 2021
9d60fa7
Remove obsolete comment.
danuker Oct 5, 2021
1102559
Add assignee mapping.
danuker Oct 6, 2021
b6f72ad
Read Trac DB, convert to RST.
danuker Oct 8, 2021
a9779e6
Add requests dependency.
danuker Oct 11, 2021
e345b6e
Order tickets to preserve Trac ID.
danuker Oct 15, 2021
a2bdfd8
Parse TracWiki content.
danuker Oct 15, 2021
472ee6f
Fix multi-repo case and tests.
danuker Oct 15, 2021
198d18d
Convert links to Markdown.
danuker Oct 18, 2021
a6adc83
Add trac2down.
danuker Oct 18, 2021
f2b42f6
Initialize TSV record of created tickets.
danuker Oct 27, 2021
fa024ca
Formatting and order works on staging.
danuker Oct 27, 2021
5e92053
Write dates in ticket body. Print ID preservation stats.
danuker Oct 28, 2021
9369649
Monkeypatch the SUT to use the test config.
danuker Oct 28, 2021
f3685e1
Sort code from abstract to low-level.
danuker Oct 28, 2021
91e1c68
Add links to GH PR.
danuker Oct 28, 2021
67b2316
Ticket closing, status label, pr link, stats.
danuker Oct 29, 2021
57ff302
Ignore tickets created.
danuker Oct 29, 2021
6e3d5d3
Assign project; more UX features.
danuker Nov 2, 2021
8cea5e4
Add file to stop Git complaining.
danuker Nov 2, 2021
755c84a
Merge branch 'main' into ticket_migrate
danuker Nov 2, 2021
d01f21f
Fix CI tests.
danuker Nov 2, 2021
342d2da
Remove needless six import.
danuker Nov 2, 2021
be84c88
More test monkeypatching.
danuker Nov 2, 2021
9ce2943
Delete generated TSV.
danuker Nov 3, 2021
199868a
Address feedback.
danuker Nov 3, 2021
faa73fc
Address feedback.
danuker Nov 3, 2021
64fa971
Close created projects. Refactor DRY_RUN behavior.
danuker Nov 3, 2021
9fc059e
Update README.
danuker Nov 3, 2021
cc916d9
Process ticket links.
danuker Nov 4, 2021
5f0ef18
Fix some bugs.
danuker Nov 4, 2021
afeef86
Fix empty project case.
danuker Nov 5, 2021
2101e5a
Add bulk issue creation (Golden Comet).
danuker Nov 5, 2021
63f3cee
Read last submitted GitHub ID.
danuker Nov 8, 2021
7f5d8a7
Updates based on feedback.
danuker Nov 10, 2021
59160d2
Acknowledge less debugging possibility.
danuker Nov 11, 2021
8543716
Mention creator with at-sign.
danuker Nov 11, 2021
7247ad7
Issue linking.
danuker Nov 11, 2021
5adc2bd
Detail script purpose.
danuker Nov 11, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 1 addition & 5 deletions .github/workflows/python-app.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,14 +23,10 @@ jobs:
- name: Install dependencies
run: |
python -m pip install pip==20.3.1
pip install flake8==3.8.4 pytest==6.1.2
pip install flake8==3.8.4 pytest==6.1.2 requests==2.26.0
- name: Lint with flake8
run: |
# stop the build if there are Python syntax errors or undefined names
flake8
- name: Prepare config
run: |
cp config.py.sample config.py
- name: Test with pytest
run: |
pytest
5 changes: 5 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,11 @@ config-chevah.py
*.psql
*.db3

# Trac migration output
tickets_expected.tsv
tickets_created.tsv
comments_created.tsv

# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
Expand Down
11 changes: 11 additions & 0 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ Hacks used to migrate from Trac sqlite or Postgresql dump to GitHub.

Works with Python 3.8.

# Wiki migration

For wiki migration, you will need git available in your dev environment.

This is a 2 stage process:
Expand Down Expand Up @@ -58,3 +60,12 @@ Things that are not yet auto-converted:
* Sub-pages listing macro `[[TitleIndex(Development/)]]`
* Local table of content `[[PageOutline]]`
* Manually create _Sidebar.rst and _Footer.rst GitHub wiki meta-pages.

# Ticket migration

1. Copy `config.py.sample` over `config.py`, and edit all the settings.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that this step is also needed for Wiki migration

2. If you are sure you want to create tickets, change `DRY_RUN` to `False`
in `ticket_migrate.py`.
3. Run `./ticket_migrate.py ../trac.db`, where `../trac.db` is the path
to the Trac SQLite DB dump.

22 changes: 21 additions & 1 deletion config.py.sample
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
# Trac User to GitHub user mapping
# Trac User to GitHub user mapping.
USER_MAPPING = {
'adi': ('adiroiban', 'Adi Roiban <adi.roiban@chevah.com>'),
'danuker': ('danuker', 'Dan Haiduc <danuthaiduc@gmail.com>'),
}

TRAC_TICKET_PREFIX = 'https://trac.chevah.com/ticket/'
Expand All @@ -10,3 +11,22 @@ TRAC_TICKET_PREFIX = 'https://trac.chevah.com/ticket/'
DEFAULT_GITHUB_USER = None

FILE_EXTENSION = '.mediawiki'

# Trac ticket Component to GitHub repository mapping.
REPOSITORY_MAPPING = {
'client': 'client',
'commons': 'commons',
'trac-migration-staging': 'trac-migration-staging',
}

# GitHub repository for Trac tickets with Component not in the mapping.
FALLBACK_REPOSITORY = 'server'
Copy link
Contributor Author

@danuker danuker Nov 3, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FALLBACK_REPOSITORY = None
or
FALLBACK_REPOSITORY = SkipMigration()

We can more easily exclude these repositories from the select_tickets function. I will add a feature to skip already-submitted tickets seen in tickets_created.tsv.


# Owner of GitHub repositories where to create issues.
OWNER = 'chevah'

# The user to create the issues through the API.
# Create a token with `repo` permissions here:
# https://github.com/settings/tokens
OAUTH_USER = 'danuker'
OAUTH_TOKEN = 'ghp_qwertyuiop'
28 changes: 28 additions & 0 deletions config_test.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# Trac User to GitHub user mapping.
# This is the configuration used by the test suite.

USER_MAPPING = {
'adi': ('adiroiban', 'Adi Roiban <adi.roiban@chevah.com>'),
'danuker': ('danuker', 'Dan Haiduc <danuthaiduc@gmail.com>'),
}

TRAC_TICKET_PREFIX = 'https://trac.chevah.com/ticket/'

# Trac ticket Component to GitHub repository mapping.
REPOSITORY_MAPPING = {
'client': 'client',
'commons': 'commons',
'trac-migration-staging': 'trac-migration-staging',
}

# GitHub repository for Trac tickets with Component not in the mapping.
FALLBACK_REPOSITORY = 'server'

# Owner of GitHub repositories where to create issues.
OWNER = 'chevah'

# The user to create the issues through the API.
# Create a token with `repo` permissions here:
# https://github.com/settings/tokens
OAUTH_USER = 'danuker'
OAUTH_TOKEN = 'ghp_qwertyuiop'
Loading