Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Testnet -> dev #90

Merged
merged 55 commits into from
Oct 21, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
55 commits
Select commit Hold shift + click to select a range
8ed9070
add deploy-testnet.yml
lachlanglen May 29, 2024
b1349bc
add testnet deployment workflow & appspec
lachlanglen May 30, 2024
c385194
testnet workflow debugging
lachlanglen May 30, 2024
6b163b0
upload appspec as artifact
lachlanglen May 30, 2024
1577af5
upload zipped codebase to s3 in workflow
lachlanglen May 30, 2024
643402d
continue working on after_install_testnet
lachlanglen May 31, 2024
aec742d
separate watchtower log groups by django env
lachlanglen May 31, 2024
98d9036
add testnet potlock TLA to indexer handler
lachlanglen May 31, 2024
d505368
add POTLOCK_TLA to settings
lachlanglen May 31, 2024
9b505ce
add dev.potlock.io and test-dev.potlock.io to allowed hosts
lachlanglen May 31, 2024
a2f6a8c
remove admin dashboard edit access
lachlanglen May 31, 2024
4a9c082
remove admin edit permission for all models
lachlanglen May 31, 2024
9f32cfd
add timestamp to deployment zip filename
lachlanglen May 31, 2024
b14512b
add env vars to deploy-testnet
lachlanglen May 31, 2024
c4f545c
add after_install_dev script
lachlanglen May 31, 2024
9b8e6e7
add after_install debug logs
lachlanglen May 31, 2024
9b12842
handle testnet potfactory & pot contract patterns
lachlanglen Jun 3, 2024
202e37f
test throttle rate
lachlanglen Jun 4, 2024
65bb172
update throttling to 100/minute
lachlanglen Jun 4, 2024
bcf01ab
Merge branch 'dev' into testnet
lachlanglen Jun 4, 2024
1a3e8d7
update readme with API urls
lachlanglen Jun 4, 2024
1ce0fc0
Merge branch 'dev' into testnet
lachlanglen Jun 4, 2024
7155a7b
add poetry lock --no-update to after_install_testnet
lachlanglen Jun 4, 2024
3f53f6e
add poetry lock --no-update to after_install_dev
lachlanglen Jun 4, 2024
89cd09b
Merge branch 'dev' into testnet
lachlanglen Jun 5, 2024
ba61d8e
Merge branch 'dev' into testnet
lachlanglen Jul 31, 2024
0f84a02
Merge pull request #76 from PotLock/dev
Prometheo Aug 8, 2024
1b7644e
Merge pull request #77 from PotLock/dev
Prometheo Aug 8, 2024
8de7254
Merge pull request #78 from PotLock/dev
Prometheo Aug 8, 2024
568e8ef
models for grant picks indexing on stellar
Prometheo Aug 27, 2024
322db1a
Merge pull request #82 from PotLock/dev
Prometheo Aug 28, 2024
472dcc8
Merge pull request #84 from PotLock/dev
Prometheo Sep 10, 2024
f98d52f
Merge pull request #85 from PotLock/dev
Prometheo Sep 18, 2024
de52767
feat: stellar indexing for grantpicks...
Prometheo Sep 18, 2024
a61d175
fix: comment hardcoded start block
Prometheo Sep 18, 2024
ead9805
Merge pull request #86 from PotLock/grantpick
Prometheo Sep 19, 2024
d1401ac
feat: add api endpoint for rounds, projects and votes
Prometheo Sep 19, 2024
8b14697
fix: stellar event ingesting
Prometheo Sep 19, 2024
a09cda3
add filtering and sorting to rounds list
Prometheo Sep 20, 2024
6418d57
update ledger always
Prometheo Sep 21, 2024
44ff64e
use ledger timestamp
Prometheo Sep 21, 2024
0dc2907
list improvement and stellar token price
Prometheo Sep 26, 2024
f3f4346
fix log error
Prometheo Sep 26, 2024
7447f70
add contact serializer to rounds
Prometheo Sep 26, 2024
7518252
add project stats, whitelist grantpicks
Prometheo Oct 3, 2024
34a5a5e
endpoint for user upvoted lists, add pairs to vote
Prometheo Oct 3, 2024
40b60b8
remove upvote
Prometheo Oct 3, 2024
3a4a31d
remove caching from lists
Prometheo Oct 7, 2024
5e3f19a
add reg count to list details
Prometheo Oct 7, 2024
f01709e
use onchain id for lists route
Prometheo Oct 8, 2024
9abb172
filter appllication by status
Prometheo Oct 9, 2024
8856e60
update admins from event and update cors
Prometheo Oct 10, 2024
16ae0a3
fix update list
Prometheo Oct 10, 2024
87c8282
fix async iteration
Prometheo Oct 10, 2024
cbabcb0
prepare prod scripts
Prometheo Oct 21, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 51 additions & 0 deletions .github/workflows/deploy-prod.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
name: Prod deploy to EC2 on Push

on:
push:
branches: [prod]

env:
AWS_REGION: "us-east-1"

# Permission can be added at job level or workflow level
permissions:
id-token: write # This is required for requesting the JWT
contents: read # This is required for actions/checkout
jobs:
DeployToCodeDeploy:
runs-on: ubuntu-latest
steps:
- name: Git clone the repository
uses: actions/checkout@v3

- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v1.7.0
with:
role-to-assume: arn:aws:iam::471112976510:role/GitHubAction-AssumeRoleWithAction
role-session-name: GitHub_to_AWS_via_FederatedOIDC
aws-region: ${{ env.AWS_REGION }}

- name: Generate appspec.yml for prod
run: cp appspec.yml appspec.yml

- name: Set environment variables
id: vars
run: |
echo "DATETIME=$(date +'%Y-%m-%d_%H-%M-%S')" >> $GITHUB_ENV
echo "FILENAME=django-indexer-${DATETIME}.zip" >> $GITHUB_ENV
echo "S3_BUCKET=django-indexer" >> $GITHUB_ENV

- name: Create zip of repository
run: zip -r "${{ env.FILENAME }}" .

- name: Upload repository to S3
run: aws s3 cp "${{ env.FILENAME }}" "s3://${{ env.S3_BUCKET }}/"

- name: Create CodeDeploy Deployment
id: deploy
run: |
aws deploy create-deployment \
--application-name django-indexer \
--deployment-group-name django-indexer-prod \
--deployment-config-name CodeDeployDefault.AllAtOnce \
--s3-location bucket=${{ env.S3_BUCKET }},bundleType=zip,key=${{ env.FILENAME }}
46 changes: 44 additions & 2 deletions accounts/api.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,8 @@
DonationSerializer,
PaginatedDonationsResponseSerializer,
)
from lists.models import ListRegistration, ListRegistrationStatus
from lists.serializers import PAGINATED_LIST_REGISTRATION_EXAMPLE, ListRegistrationSerializer, PaginatedListRegistrationsResponseSerializer
from lists.models import List, ListRegistration, ListRegistrationStatus
from lists.serializers import PAGINATED_LIST_EXAMPLE, PAGINATED_LIST_REGISTRATION_EXAMPLE, ListRegistrationSerializer, ListSerializer, PaginatedListRegistrationsResponseSerializer, PaginatedListsResponseSerializer
from pots.models import Pot, PotApplication, PotApplicationStatus, PotPayout
from pots.serializers import (
PAGINATED_PAYOUT_EXAMPLE,
Expand Down Expand Up @@ -441,3 +441,45 @@ def get(self, request: Request, *args, **kwargs):
results = self.paginate_queryset(registrations, request, view=self)
serializer = ListRegistrationSerializer(results, many=True)
return self.get_paginated_response(serializer.data)



class AccountUpvotedListsAPI(APIView, CustomSizePageNumberPagination):

@extend_schema(
parameters=[
OpenApiParameter("account_id", str, OpenApiParameter.PATH),
*pagination_parameters,
],
responses={
200: OpenApiResponse(
response=PaginatedListsResponseSerializer,
description="Returns a paginated list of user upvoted lists",
examples=[
OpenApiExample(
"example-1",
summary="Simple example",
description="Example response for lists",
value=PAGINATED_LIST_EXAMPLE,
response_only=True,
),
],
),
404: OpenApiResponse(description="Account not found"),
500: OpenApiResponse(description="Internal server error"),
},
)
def get(self, request: Request, *args, **kwargs):
account_id = kwargs.get("account_id")
try:
account = Account.objects.get(id=account_id)
except Account.DoesNotExist:
return Response(
{"message": f"Account with ID {account_id} not found."}, status=404
)
upvoted_lists = List.objects.filter(upvotes__account=account)
results = self.paginate_queryset(upvoted_lists, request, view=self)
serializer = ListSerializer(results, many=True)
return self.get_paginated_response(serializer.data)


21 changes: 21 additions & 0 deletions api/urls.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,11 +13,13 @@
AccountListRegistrationsAPI,
AccountPayoutsReceivedAPI,
AccountPotApplicationsAPI,
AccountUpvotedListsAPI,
AccountsListAPI,
DonorsAPI,
)
from base.api import StatsAPI
from donations.api import DonationContractConfigAPI
from grantpicks.api import AccountProjectListAPI, ProjectListAPI, ProjectRoundVotesAPI, ProjectStatsAPI, RoundApplicationsAPI, RoundDetailAPI, RoundsListAPI
from lists.api import (
ListDetailAPI,
ListRandomRegistrationAPI,
Expand Down Expand Up @@ -82,6 +84,11 @@
AccountListRegistrationsAPI.as_view(),
name="accounts_api_by_id_registrations",
),
path(
"v1/accounts/<str:account_id>/upvoted-lists",
AccountUpvotedListsAPI.as_view(),
name="accounts_api_upvoted_lists",
),
# donate contract config
path(
"v1/donate_contract_config",
Expand Down Expand Up @@ -129,4 +136,18 @@
),
# stats
path("v1/stats", StatsAPI.as_view(), name="stats_api"),

# grantpicks
path("v1/rounds", RoundsListAPI.as_view(), name="rounds_api"),
path("v1/round/<int:round_id>/", RoundDetailAPI.as_view(), name="rounds_api_by_id"),
path("v1/round/<int:round_id>/<int:project_id>/votes", ProjectRoundVotesAPI.as_view(), name="project_round_votes_api_by_id"),
path("v1/projects", ProjectListAPI.as_view(), name="projects_api"),
path(
"v1/rounds/<str:round_id>/applications",
RoundApplicationsAPI.as_view(),
name="rounds_applications_api",
),
path("v1/<str:account_id>/projects", AccountProjectListAPI.as_view(), name="user_projects_api"),
path("v1/<str:account_id>/<int:project_id>/project-stats", ProjectStatsAPI.as_view(), name="projects_api"),

]
12 changes: 12 additions & 0 deletions base/celery.py
Original file line number Diff line number Diff line change
Expand Up @@ -40,12 +40,24 @@
"schedule": crontab(minute="*/5"), # Executes every 5 minutes
"options": {"queue": "beat_tasks"},
},
"fetch_stellar_events_every_minute": {
"task": "indexer_app.tasks.stellar_event_indexer",
"schedule": crontab(minute="*/1"), # Executes every 1 minutes
"options": {"queue": "beat_tasks"},
},
"process_stellar_event_every_minute": {
"task": "indexer_app.tasks.process_stellar_events",
"schedule": crontab(minute="*/1"), # Executes every 1 minutes
"options": {"queue": "beat_tasks"},
},
}

app.conf.task_routes = {
"indexer_app.tasks.update_account_statistics": {"queue": "beat_tasks"},
"indexer_app.tasks.fetch_usd_prices": {"queue": "beat_tasks"},
"indexer_app.tasks.update_pot_statistics": {"queue": "beat_tasks"},
"indexer_app.tasks.stellar_event_indexer": {"queue": "beat_tasks"},
"indexer_app.tasks.process_stellar_events": {"queue": "beat_tasks"},
}

SPOT_INDEXER_QUEUE_NAME = "spot_indexing"
29 changes: 24 additions & 5 deletions base/settings.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,13 +28,14 @@

# SECURITY WARNING: keep the secret key used in production secret!
# TODO: update before prod release
SECRET_KEY = "django-insecure-=r_v_es6w6rxv42^#kc2hca6p%=fe_*cog_5!t%19zea!enlju"
SECRET_KEY = os.environ.get("PL_DJANGO_SECRET_KEY", "django-insecure-=r_v_es6w6rxv42^#kc2hca6p%=fe_*cog_5!t%19zea!enlju")

ALLOWED_HOSTS = [
"ec2-100-27-57-47.compute-1.amazonaws.com",
"127.0.0.1",
"dev.potlock.io",
"test-dev.potlock.io",
"api.potlock.io"
# "alpha.potlock.io",
]

Expand All @@ -58,13 +59,27 @@
REDIS_PORT = os.environ.get("PL_REDIS_PORT", 6379)
SENTRY_DSN = os.environ.get("PL_SENTRY_DSN")

POTLOCK_TLA = "potlock.testnet" if ENVIRONMENT == "testnet" else "potlock.near"
NADABOT_TLA = "nadabot.testnet" if ENVIRONMENT == "testnet" else "nadabot.near"

# POTLOCK_TLA = "potlock.testnet" if ENVIRONMENT == "testnet" else "potlock.near"
POTLOCK_TLA = "potlock.testnet" if ENVIRONMENT == "testnet" else ("staging.potlock.near" if ENVIRONMENT == "dev" else "potlock.near")
# NADABOT_TLA = "nadabot.testnet" if ENVIRONMENT == "testnet" else "nadabot.near"
NADABOT_TLA = "nadabot.testnet" if ENVIRONMENT == "testnet" else ("staging.nadabot.near" if ENVIRONMENT == "dev" else "nadabot.near")
STELLAR_CONTRACT_ID = "CCVVNTUD6CFPKZ2C4JAZIQGCAK2S6D6KPP5IELGHTHHJPYV2B62GPJTK" if ENVIRONMENT == "testnet" else ("" if ENVIRONMENT == "dev" else "")
STELLAR_PROJECTS_REGISTRY_CONTRACT = "CAUINLSA42RCTY35UFGOM2NLKMSRM6FW6NP7AR4GTUZHUQZZWB6CRBSJ" if ENVIRONMENT == "testnet" else ("" if ENVIRONMENT == "dev" else "")
NEAR_SOCIAL_CONTRACT_ADDRESS = (
"v1.social08.testnet" if ENVIRONMENT == "testnet" else "social.near"
)

# TODO: split settigns file by enviroment
if ENVIRONMENT == "testnet":
POTLOCK_PATTERN = r'\.potlock\.testnet$'
NADABOT_PATTERN = r'\.nadabot\.testnet$'
elif ENVIRONMENT == "dev":
POTLOCK_PATTERN = r'\.staging\.potlock\.near$'
NADABOT_PATTERN = r'\.staging\.nadabot\.near$'
else: # mainnet/prod
POTLOCK_PATTERN = r'(?<!\.staging)\.potlock\.near$'
NADABOT_PATTERN = r'(?<!\.staging)\.nadabot\.near$'

FASTNEAR_RPC_URL = (
"https://rpc.web4.testnet.page"
if ENVIRONMENT == "testnet"
Expand Down Expand Up @@ -106,6 +121,7 @@
"tokens",
"nadabot",
"chains",
"grantpicks"
]

DEFAULT_PAGE_SIZE = 30
Expand Down Expand Up @@ -168,6 +184,8 @@
CORS_ALLOWED_ORIGINS = [
"http://localhost:3000",
"http://127.0.0.1:8080",
"http://dev.local",
"https://dev.local",
"https://test.potlock.org",
"https://test.potlock.xyz",
"https://test.potlock.io",
Expand Down Expand Up @@ -206,7 +224,8 @@
]

CORS_ALLOWED_ORIGIN_REGEXES = [
"^https:\/\/potlock-next-[\w-]+-potlock\.vercel\.app\/?$"
"^https:\/\/potlock-next-[\w-]+-potlock\.vercel\.app\/?$",
"^https?:\/\/.*\.?grantpicks\.com$"
]

# REDIS / CACHE CONFIGS
Expand Down
Empty file added grantpicks/__init__.py
Empty file.
3 changes: 3 additions & 0 deletions grantpicks/admin.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
from django.contrib import admin

# Register your models here.
Loading
Loading