This repository has been archived by the owner on Aug 15, 2023. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 15
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
build(guardoni): fix cli testing scripts
- Loading branch information
1 parent
5ffc2d1
commit d40f7b1
Showing
24 changed files
with
230 additions
and
112 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,50 @@ | ||
| import { AppError, toAppError } from '../errors/AppError'; | ||
| import csvParse from 'csv-parse'; | ||
| import * as csvStringify from 'csv-stringify'; | ||
| import * as TE from 'fp-ts/lib/TaskEither'; | ||
| import { GetLogger } from '../logger'; | ||
| const csvLogger = GetLogger('csv') | ||
|
|
||
| export const csvParseTE = ( | ||
| content: Buffer, | ||
| options: csvParse.Options | ||
| ): TE.TaskEither< | ||
| AppError, | ||
| { | ||
| records: any; | ||
| info: csvParse.Info; | ||
| } | ||
| > => | ||
| TE.tryCatch( | ||
| () => | ||
| new Promise((resolve, reject) => { | ||
| csvParse(content, options, (error, records, info) => { | ||
| if (error) { | ||
| csvLogger.error('CSV Parse error: %O', error); | ||
| return reject(error); | ||
| } | ||
| csvLogger.debug('CSV Parse results: %O', records); | ||
| return resolve({ records, info }); | ||
| }); | ||
| }), | ||
| toAppError | ||
| ); | ||
|
|
||
| export const csvStringifyTE = ( | ||
| records: any[], | ||
| options: csvStringify.Options | ||
| ): TE.TaskEither<AppError, string> => | ||
| TE.tryCatch( | ||
| () => | ||
| new Promise((resolve, reject) => { | ||
| csvStringify.stringify(records, options, (error, info) => { | ||
| if (error) { | ||
| csvLogger.error('CSV Stringify error: %O', error); | ||
| return reject(error); | ||
| } | ||
| csvLogger.debug('CSV Stringify results: %O', records); | ||
| return resolve(info); | ||
| }); | ||
| }), | ||
| toAppError | ||
| ); |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,130 @@ | ||
| #!/usr/bin/env node | ||
|
|
||
| /* eslint-disable camelcase */ | ||
|
|
||
| import { $, os, fetch, fs, path } from 'zx'; | ||
| import { normalizePlatform, getGuardoniCliPkgName } from './utils.mjs'; | ||
| import dotenv from 'dotenv'; | ||
| import assert from 'assert'; | ||
| import { | ||
| readFixtureJSONPaths, | ||
| readFixtureJSON, | ||
| } from '../../../packages/shared/build/test/utils/parser.utils.js'; | ||
| import { csvStringifyTE } from '../../../packages/shared/build/utils/csv.utils.js'; | ||
| import { foldTEOrThrow } from '../../../packages/shared/build/utils/fp.utils.js'; | ||
|
|
||
| dotenv.config({ path: '.env.development' }); | ||
|
|
||
| // eslint-disable-next-line no-void | ||
| void (async function () { | ||
| const [p, nature, count] = process.argv.slice(2); | ||
| console.log({ p, nature, count }); | ||
|
|
||
| if (!p) { | ||
| console.error('Platform parameter not present'); | ||
| process.exit(1); | ||
| } | ||
|
|
||
| if (!nature) { | ||
| console.error('Nature parameter not present'); | ||
| process.exit(1); | ||
| } | ||
|
|
||
| const version = await $`node -p -e "require('./package.json').version"`; | ||
| const platform = normalizePlatform(os.type()); | ||
| const profile = 'profile-test-99'; | ||
|
|
||
| fs.removeSync(path.resolve(process.cwd(), 'profiles', profile)); | ||
|
|
||
| const cli = `./dist/${getGuardoniCliPkgName( | ||
| version.stdout.replace('\n', ''), | ||
| platform | ||
| )}`; | ||
|
|
||
| const flags = [ | ||
| '--basePath=./', | ||
| `--executablePath=${process.env.PUPPETEER_EXEC_PATH}`, | ||
| '-c=guardoni.config.json', | ||
| '--headless=false', | ||
| '--verbose', | ||
| ]; | ||
|
|
||
| const researchTag = `guardoni-${p}-test-${nature}`; | ||
|
|
||
| const experimentFlags = [ | ||
| ...flags, | ||
| `--publicKey=${process.env.PUBLIC_KEY}`, | ||
| `--secretKey=${process.env.SECRET_KEY}`, | ||
| `--researchTag=${researchTag}`, | ||
| ]; | ||
|
|
||
| const fixturePaths = readFixtureJSONPaths( | ||
| path.resolve( | ||
| process.cwd(), | ||
| `../${p}trex/backend/__tests__/fixtures/htmls/${nature}` | ||
| ) | ||
| ); | ||
|
|
||
| const take = parseInt(count, 10) ?? fixturePaths.length; | ||
| await $`echo "Taking ${take} fixtures"`; | ||
| const fixtures = fixturePaths | ||
| .slice(0, take) | ||
| .map((f) => readFixtureJSON(f, process.env.PUBLIC_KEY)); | ||
|
|
||
| const sources = fixtures.reduce( | ||
| (acc, f, i) => | ||
| acc.concat( | ||
| f.sources.map((s, ii) => ({ | ||
| title: f.metadata.title, | ||
| url: s.href, | ||
| urltag: `${p}-test-${nature}-${i}-${ii}`, | ||
| watchFor: '5s', | ||
| incrementScrollByPX: 400, | ||
| totalScroll: 1000, | ||
| })) | ||
| ), | ||
| [] | ||
| ); | ||
|
|
||
| // reject cookie modal | ||
| await $`${cli} ${flags} ${p}-navigate`; | ||
|
|
||
| const experiment = await foldTEOrThrow( | ||
| csvStringifyTE(sources, { header: 'true' }) | ||
| ); | ||
|
|
||
| const experimentFile = path.resolve( | ||
| process.cwd(), | ||
| `./experiments/${p}-${nature}-test-from-fixtures-temp.csv` | ||
| ); | ||
|
|
||
| fs.writeFileSync(experimentFile, experiment, 'utf-8'); | ||
|
|
||
| const yt_home_experiment_register_out = | ||
| await $`${cli} ${flags} ${p}-register ${experimentFile}`; | ||
|
|
||
| const yt_home_experiment_id = yt_home_experiment_register_out.stdout | ||
| .split('\n') | ||
| .find((s) => s.startsWith('experimentId:')) | ||
| .replace('experimentId: \t', '') | ||
| .trim(); | ||
|
|
||
| const yt_home_experiment_run_out = | ||
| await $`${cli} ${experimentFlags} ${p}-experiment ${yt_home_experiment_id} | grep 'publicKey: ' `; | ||
|
|
||
| const yt_home_experiment_public_key = yt_home_experiment_run_out.stdout | ||
| .replace('publicKey: \t ', '') | ||
| .replace('\n', ''); | ||
|
|
||
| assert.strictEqual(yt_home_experiment_public_key, process.env.PUBLIC_KEY); | ||
|
|
||
| const backend = p === 'tk' ? process.env.TK_BACKEND : process.env.YT_BACKEND; | ||
| const personalURL = `${backend}/v2/personal/${yt_home_experiment_public_key}/${nature}/json`; | ||
| await $`echo ${personalURL}`; | ||
|
|
||
| // const metadata = await fetch( | ||
| // `http://localhost:9000/api/v2/metadata?publicKey=${yt_home_experiment_public_key}&experimentId=${yt_home_experiment_id}&nature=home` | ||
| // ).then((r) => r.json()); | ||
|
|
||
| fs.removeSync(experimentFile); | ||
| })(); |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
File renamed without changes.
File renamed without changes.
File renamed without changes.
Oops, something went wrong.