Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds cron job handler functions #26

Merged
merged 17 commits into from
Dec 15, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 35 additions & 4 deletions src/handlers/scheduledEventHandler.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,10 @@ import { KVNamespace } from '@cloudflare/workers-types';

import config from '../config/config';
import { NAMESPACE_NAME } from '../constants';
import { env, NicknameUpdateResponseType } from '../types/global.types';
import { updateUserRoles } from '../services/discordBotServices';
import { getMissedUpdatesUsers } from '../services/rdsBackendService';
import { DiscordUserRole, env, NicknameUpdateResponseType } from '../types/global.types';
import { chunks } from '../utils/arrayUtils';
import { generateJwt } from '../utils/generateJwt';

export async function ping(env: env) {
Expand Down Expand Up @@ -57,8 +60,36 @@ export async function callDiscordNicknameBatchUpdate(env: env) {
console.error('Error while trying to update the last nickname change timestamp');
}
}

console.log(data);

return data;
}

export const addMissedUpdatesRole = async (env: env) => {
const MAX_ROLE_UPDATE = 25;
try {
let cursor: string | undefined = undefined;
for (let index = MAX_ROLE_UPDATE; index > 0; index--) {
if (index < MAX_ROLE_UPDATE && !cursor) break;

const missedUpdatesUsers = await getMissedUpdatesUsers(env, cursor);

if (!!missedUpdatesUsers && missedUpdatesUsers.usersToAddRole?.length > 1) {
const discordUserIdRoleIdList: DiscordUserRole[] = missedUpdatesUsers.usersToAddRole.map((userId) => ({
userid: userId,
roleid: config(env).MISSED_UPDATES_ROLE_ID,
}));

const discordUserRoleChunks = chunks(discordUserIdRoleIdList, MAX_ROLE_UPDATE);
for (const discordUserRoleList of discordUserRoleChunks) {
try {
await updateUserRoles(env, discordUserRoleList);
} catch (error) {
console.error('Error occurred while updating discord users', error);
}
}
}
cursor = missedUpdatesUsers?.cursor;
}
} catch (err) {
console.error('Error while adding missed updates roles');
}
};
27 changes: 27 additions & 0 deletions src/services/discordBotServices.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
import config from '../config/config';
import { DiscordRoleUpdatedList, DiscordUserRole, env } from '../types/global.types';
import { generateDiscordBotJwt } from '../utils/generateJwt';

export const updateUserRoles = async (env: env, payload: DiscordUserRole[]): Promise<DiscordRoleUpdatedList> => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this being handled as a part of cron job repo? Don't you think this can be done in RDS backend code itself?
Essentially the cron jobs repo should only contain code that contains the scheduler and the task that is invoked.

Maybe I am missing something here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If a http request takes a long time, server can choose to timeout that request with an error.

  • Currently discord allows us to update roles at an average rate of 1/sec. This means our request can be timed out while the RDS backend is updating a large number of roles at this rate.
  • Since Cloudflare docs mention that it does not have any hard limits on the duration for which cron job can run, I've moved this here.

https://developers.cloudflare.com/workers/platform/limits/

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, but this repo then no longer remains just "cron-jobs". We might have to find a solution for this.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe create an issue/ RFC that would tackle this?

try {
const url = config(env).DISCORD_BOT_API_URL;
const token = await generateDiscordBotJwt(env);

const response = await env.DISCORD_BOT.fetch(`${url}/roles?action=add-role`, {
method: 'POST',
headers: {
Authorization: `Bearer ${token}`,
'Content-Type': 'application/json',
},
body: JSON.stringify(payload),
});
if (!response.ok) {
throw Error(`Role Update failed with status: ${response.status}`);
}
const data: DiscordRoleUpdatedList = await response.json();
return data;
} catch (error) {
console.error('Error while updating discord user roles');
throw error;
}
};
32 changes: 32 additions & 0 deletions src/services/rdsBackendService.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
import config from '../config/config';
import { DiscordUsersResponse, env } from '../types/global.types';
import { generateJwt } from '../utils/generateJwt';

export const getMissedUpdatesUsers = async (env: env, cursor: string | undefined) => {
try {
const baseUrl = config(env).RDS_BASE_API_URL;

const url = new URL(`${baseUrl}/tasks/users/discord`);
url.searchParams.append('q', 'status:missed-updates');
if (cursor) {
url.searchParams.append('cursor', cursor);
}
const token = await generateJwt(env);
const response = await fetch(url, {
method: 'GET',
headers: {
Authorization: `Bearer ${token}`,
'Content-Type': 'application/json',
},
});
if (!response.ok) {
throw new Error(`Fetch call to get user discord details failed with status: ${response.status}`);
}

const responseData: DiscordUsersResponse = await response.json();
return responseData?.data;
} catch (error) {
console.error('Error occurrent while fetching discord user details');
throw error;
}
};
38 changes: 38 additions & 0 deletions src/tests/fixtures/missedRoleHandler.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
export const missedUpdatesUsersResponse = {
message: 'Discord details of users with status missed updates fetched successfully',
data: {},
};
export const missedUpdatesUsersMock = {
usersToAddRole: ['user1', 'user2'],
tasks: 10,
missedUpdatesTasks: 5,
cursor: 'some-cursor',
};
export const missedUpdatesUsersMockWithNoUsers = {
usersToAddRole: [],
tasks: 10,
missedUpdatesTasks: 5,
};
export const missedUpdatesUsersMockWithoutCursor = {
usersToAddRole: ['user1', 'user2'],
tasks: 10,
missedUpdatesTasks: 5,
};

export const updateRolesResponseMock = {
userid: 'user1',
roleid: '1',
success: true,
};
export const discordUserRoleMock = [
{ userid: 'user1', roleid: '1' },
{ userid: 'user2', roleid: '2' },
];

export const discordRoleUpdateResult = [
{
userid: '1',
roleid: '2',
success: true,
},
];
67 changes: 67 additions & 0 deletions src/tests/handlers/missedRoleHandler.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
import { addMissedUpdatesRole } from '../../handlers/scheduledEventHandler';
import { updateUserRoles } from '../../services/discordBotServices';
import { getMissedUpdatesUsers } from '../../services/rdsBackendService';
import {
missedUpdatesUsersMock,
missedUpdatesUsersMockWithNoUsers,
missedUpdatesUsersMockWithoutCursor,
} from '../fixtures/missedRoleHandler';

jest.mock('.../../../../services/rdsBackendService', () => ({
getMissedUpdatesUsers: jest.fn(),
}));
jest.mock('.../../../../services/discordBotServices', () => ({
updateUserRoles: jest.fn(),
}));
describe('addMissedUpdatesRole', () => {
beforeEach(() => {
jest.resetAllMocks();
});
afterEach(() => {
jest.clearAllMocks();
});

it('should call getMissedUpdatesUsers and updateUserRoles when there are users to add role', async () => {
(getMissedUpdatesUsers as jest.Mock)
.mockResolvedValueOnce(missedUpdatesUsersMock)
.mockResolvedValueOnce(missedUpdatesUsersMockWithoutCursor);
await addMissedUpdatesRole({});
expect(getMissedUpdatesUsers).toHaveBeenCalledTimes(2);
expect(updateUserRoles).toHaveBeenCalledTimes(2);
});

it('should not call updateUserRoles when there are no users to add role', async () => {
(getMissedUpdatesUsers as jest.Mock).mockResolvedValueOnce(missedUpdatesUsersMockWithNoUsers);

await addMissedUpdatesRole({});
expect(getMissedUpdatesUsers).toHaveBeenCalledTimes(1);
expect(updateUserRoles).toHaveBeenCalledTimes(0);
});

it('should create chunks of userId and update roles multiple times when count is greater than 25', async () => {
const mockValue: any = { ...missedUpdatesUsersMockWithoutCursor, usersToAddRole: new Array(75).fill('id') };
(getMissedUpdatesUsers as jest.Mock).mockResolvedValueOnce(mockValue);

await addMissedUpdatesRole({});
expect(getMissedUpdatesUsers).toHaveBeenCalledTimes(1);
expect(updateUserRoles).toHaveBeenCalledTimes(3);
});

it('should handle errors', async () => {
(getMissedUpdatesUsers as jest.Mock).mockRejectedValueOnce(new Error('Error fetching missed updates users'));
const consoleSpy = jest.spyOn(console, 'error');
await addMissedUpdatesRole({});
expect(consoleSpy).toHaveBeenCalledWith('Error while adding missed updates roles');
});

it('should continue updating user roles even when a call fails', async () => {
(updateUserRoles as jest.Mock).mockRejectedValueOnce(new Error('Error occurred'));
const consoleSpy = jest.spyOn(console, 'error');
const mockValue: any = { ...missedUpdatesUsersMockWithoutCursor, usersToAddRole: new Array(75).fill('id') };
(getMissedUpdatesUsers as jest.Mock).mockResolvedValueOnce(mockValue);
await addMissedUpdatesRole({});
expect(getMissedUpdatesUsers).toHaveBeenCalledTimes(1);
expect(consoleSpy).toHaveBeenCalledTimes(1);
expect(updateUserRoles).toHaveBeenCalledTimes(3);
});
});
49 changes: 49 additions & 0 deletions src/tests/services/discordBotService.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
import config from '../../config/config';
import { updateUserRoles } from '../../services/discordBotServices';
import { discordRoleUpdateResult, discordUserRoleMock } from '../fixtures/missedRoleHandler';

jest.mock('../../utils/generateJwt', () => ({
generateDiscordBotJwt: jest.fn().mockResolvedValueOnce('mocked-jwt-token'),
}));
describe('discordBotService', () => {
describe('updateUserRoles', () => {
let fetchSpy: jest.Mock<any, any, any>;
beforeEach(() => {
fetchSpy = jest.fn();
jest.clearAllMocks();
});

it('should make a successful API call and return the expected data', async () => {
fetchSpy.mockResolvedValueOnce({
ok: true,
status: 200,
json: jest.fn().mockResolvedValueOnce(discordRoleUpdateResult),
} as unknown as Response);
const result = await updateUserRoles({ DISCORD_BOT: { fetch: fetchSpy } }, discordUserRoleMock);

expect(fetchSpy).toHaveBeenCalledWith(`${config({}).DISCORD_BOT_API_URL}/roles?action=add-role`, {
method: 'POST',
headers: {
Authorization: 'Bearer mocked-jwt-token',
'Content-Type': 'application/json',
},
body: JSON.stringify(discordUserRoleMock),
});
expect(result).toEqual([...discordRoleUpdateResult]);
});

it('should throw error when api call fails', async () => {
fetchSpy.mockResolvedValueOnce({
ok: false,
status: 400,
} as unknown as Response);
await expect(updateUserRoles({ DISCORD_BOT: { fetch: fetchSpy } }, [])).rejects.toThrow('Role Update failed with status: 400');
});
it('should handle unknown errors', async () => {
const consoleSpy = jest.spyOn(console, 'error');
fetchSpy.mockRejectedValueOnce(new Error('Error occurred'));
await expect(updateUserRoles({ DISCORD_BOT: { fetch: fetchSpy } }, [])).rejects.toThrow('Error occurred');
expect(consoleSpy).toHaveBeenCalledWith('Error while updating discord user roles');
});
});
});
68 changes: 68 additions & 0 deletions src/tests/services/rdsBackendService.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
import config from '../../config/config';
import { getMissedUpdatesUsers } from '../../services/rdsBackendService';
import { missedUpdatesUsersMock, missedUpdatesUsersResponse } from '../fixtures/missedRoleHandler';

jest.mock('../../utils/generateJwt', () => ({
generateJwt: jest.fn().mockResolvedValue('mocked-jwt-token'),
}));

describe('rdsBackendService', () => {
describe('updateUserRoles', () => {
let cursor: undefined | string;
beforeEach(() => {
jest.clearAllMocks();
});

it('should make a successful API call and return the expected data', async () => {
jest.spyOn(global, 'fetch').mockResolvedValueOnce({
ok: true,
status: 200,
json: jest.fn().mockResolvedValueOnce({ ...missedUpdatesUsersResponse, data: missedUpdatesUsersMock }),
} as unknown as Response);
const result = await getMissedUpdatesUsers({}, cursor);
const url = new URL(`${config({}).RDS_BASE_API_URL}/tasks/users/discord`);
url.searchParams.append('q', 'status:missed-updates');
expect(fetch).toHaveBeenCalledWith(url, {
method: 'GET',
headers: {
Authorization: 'Bearer mocked-jwt-token',
'Content-Type': 'application/json',
},
});
expect(result).toEqual({ ...missedUpdatesUsersMock });
});
it('should make a successful API call with cursor', async () => {
jest.spyOn(global, 'fetch').mockResolvedValueOnce({
ok: true,
status: 200,
json: jest.fn().mockResolvedValueOnce({ ...missedUpdatesUsersResponse, data: missedUpdatesUsersMock }),
} as unknown as Response);
const result = await getMissedUpdatesUsers({}, 'cursorValue');
const url = new URL(`${config({}).RDS_BASE_API_URL}/tasks/users/discord`);
url.searchParams.append('q', 'status:missed-updates');
url.searchParams.append('cursor', 'cursorValue');
expect(fetch).toHaveBeenCalledWith(url, {
method: 'GET',
headers: {
Authorization: 'Bearer mocked-jwt-token',
'Content-Type': 'application/json',
},
});
expect(result).toEqual({ ...missedUpdatesUsersMock });
});
it('should throw error when api call fails', async () => {
jest.spyOn(global, 'fetch').mockResolvedValueOnce({
ok: false,
status: 400,
} as unknown as Response);
await expect(getMissedUpdatesUsers({}, cursor)).rejects.toThrow('Fetch call to get user discord details failed with status: 400');
});

it('should handle unknown errors', async () => {
const consoleSpy = jest.spyOn(console, 'error');
jest.spyOn(global, 'fetch').mockRejectedValueOnce(new Error('Error occurred'));
await expect(getMissedUpdatesUsers({}, cursor)).rejects.toThrow('Error occurred');
expect(consoleSpy).toHaveBeenCalledWith('Error occurrent while fetching discord user details');
});
});
});
5 changes: 4 additions & 1 deletion src/types/global.types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,10 @@ export type NicknameUpdateResponseType = {
unsuccessfulNicknameUpdates: number;
};
};

export type DiscordUsersResponse = {
message: string;
data: DiscordUserIdList;
};
export type DiscordUserIdList = {
usersToAddRole: string[];
tasks: number;
Expand Down
6 changes: 5 additions & 1 deletion src/worker.ts
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
import { callDiscordNicknameBatchUpdate, ping } from './handlers/scheduledEventHandler';
import { addMissedUpdatesRole, callDiscordNicknameBatchUpdate, ping } from './handlers/scheduledEventHandler';
import { env } from './types/global.types';

const EVERY_4_HOURS = '0 */4 * * *';
const EVERY_6_HOURS = '0 */6 * * *';
const EVERY_12_HOURS = '0 */12 * * *';

export default {
async scheduled(req: ScheduledController, env: env, ctx: ExecutionContext) {
Expand All @@ -12,6 +13,9 @@ export default {
break;
case EVERY_6_HOURS:
return await callDiscordNicknameBatchUpdate(env);
case EVERY_12_HOURS: {
return await addMissedUpdatesRole(env);
}
default:
console.error('Unknown Trigger Value!');
}
Expand Down
Loading