refactor: Deno #48

YDX-2147483647 · 2023-11-06T07:46:55Z

原因：开发、部署更简单，速度更快。

(To be squashed and merged)

https://docs.deno.com/runtime/manual/advanced/jsx_dom/jsdom https://docs.deno.com/runtime/manual/advanced/jsx_dom/deno_dom

The proxy plugin is ignored.

YDX-2147483647 · 2023-11-07T06:03:35Z

现在功能基本正常，只是超时的错误还未捕获。

deno run -A ./src/examples/proxy-cli.ts

{"level":"info","message":"Signed in successfully.","plugin":"proxy","timestamp":"2023-11-07T05:57:44.397Z"}
{"level":"info","message":"发现48个通知来源。","plugin":"cli","timestamp":"2023-11-07T05:57:47.580Z"}
{"level":"http","message":"Request https://century.bit.edu.cn/tz/index.htm with proxy.","plugin":"proxy","timestamp":"2023-11-07T05:57:47.584Z"}
{"level":"http","message":"Request https://student.bit.edu.cn/tzgg/index.htm with proxy.","plugin":"proxy","timestamp":"2023-11-07T05:57:47.585Z"}
{"level":"http","message":"Request https://student.bit.edu.cn/xngs/index.blk.htm with proxy.","plugin":"proxy","timestamp":"2023-11-07T05:57:47.585Z"}
{"level":"http","message":"Request http://cxcy.bit.edu.cn/pt/pt/Notice with proxy.","plugin":"proxy","timestamp":"2023-11-07T05:57:47.586Z"}
{"level":"http","message":"Request http://dekt.bit.edu.cn/portal/CourseList.jsp with proxy.","plugin":"proxy","timestamp":"2023-11-07T05:57:47.586Z"}
{"level":"http","message":"Request https://dzb.bit.edu.cn/cms/web/notify/search?page=1&status=7&rows=20&order=1&sortFiled=publishDate with proxy.","plugin":"proxy","timestamp":"2023-11-07T05:57:47.588Z"}
{"level":"http","message":"Request http://mec.bit.edu.cn/pcmd/ajax.php?vpn-12-o1-mec.bit.edu.cn&act=getmanage_nologin&w=新闻公告 with proxy.","plugin":"proxy","timestamp":"2023-11-07T05:57:47.589Z"}
{"level":"warn","message":"未从“第二课堂”获取到任何通知。将忽略。","plugin":"cli","timestamp":"2023-11-07T05:57:47.667Z"}
{"hook":"fetch_each.error","level":"error","message":"TypeError: error sending request for url (https://ls.bit.edu.cn/xytz/index.htm): error trying to connect: tcp connect error: Connection timed out (os error 110)","plugin":"cli","timestamp":"2023-11-07T05:59:58.200Z"}
{"hook":"fetch_each.error","level":"error","message":"TypeError: error sending request for url (https://sae.bit.edu.cn/tzgggb/index.htm): error trying to connect: tcp connect error: Connection timed out (os error 110)","plugin":"cli","timestamp":"2023-11-07T05:59:58.201Z"}
{"hook":"fetch_each.error","level":"error","message":"TypeError: error sending request for url (https://rw.bit.edu.cn/rwszjyb/tzxw/index.htm): error trying to connect: tcp connect error: Connection timed out (os error 110)","plugin":"cli","timestamp":"2023-11-07T05:59:58.201Z"}
{"hook":"fetch_each.error","level":"error","message":"TypeError: error sending request for url (https://rw.bit.edu.cn/tzgg/index.htm): error trying to connect: tcp connect error: Connection timed out (os error 110)","plugin":"cli","timestamp":"2023-11-07T05:59:58.201Z"}
{"hook":"fetch_each.error","level":"error","message":"TypeError: error sending request for url (https://smt.bit.edu.cn/tzgg/index.htm): error trying to connect: tcp connect error: Connection timed out (os error 110)","plugin":"cli","timestamp":"2023-11-07T05:59:58.201Z"}
{"hook":"fetch_each.error","level":"error","message":"TypeError: error sending request for url (https://jxzx.bit.edu.cn/jxyx/index.htm): error trying to connect: tcp connect error: Connection timed out (os error 110)","plugin":"cli","timestamp":"2023-11-07T05:59:58.202Z"}
{"hook":"fetch_each.error","level":"error","message":"TypeError: error sending request for url (https://marx.bit.edu.cn/tzgg/index.htm): error trying to connect: tcp connect error: Connection timed out (os error 110)","plugin":"cli","timestamp":"2023-11-07T05:59:58.204Z"}
{"hook":"fetch_each.error","level":"error","message":"TypeError: error sending request for url (https://xxgk.bit.edu.cn/tzgg/index.htm): error trying to connect: tcp connect error: Connection timed out (os error 110)","plugin":"cli","timestamp":"2023-11-07T05:59:58.204Z"}
{"hook":"fetch_each.error","level":"error","message":"TypeError: error sending request for url (https://cs.bit.edu.cn/tzgg/index.htm): error trying to connect: tcp connect error: Connection timed out (os error 110)","plugin":"cli","timestamp":"2023-11-07T05:59:58.204Z"}
{"hook":"fetch_each.error","level":"error","message":"TypeError: error sending request for url (https://abroad.bit.edu.cn/xwtz/tzgg/index.htm): error trying to connect: tcp connect error: Connection timed out (os error 110)","plugin":"cli","timestamp":"2023-11-07T05:59:58.204Z"}
{"hook":"fetch_each.error","level":"error","message":"TypeError: error sending request for url (https://sme.bit.edu.cn/gbxwzx/tzgg/index.htm): error trying to connect: tcp connect error: Connection timed out (os error 110)","plugin":"cli","timestamp":"2023-11-07T05:59:58.204Z"}
{"hook":"fetch_each.error","level":"error","message":"TypeError: error sending request for url (https://cst.bit.edu.cn/xxfw/tzgg/index.htm): error trying to connect: tcp connect error: Connection timed out (os error 110)","plugin":"cli","timestamp":"2023-11-07T05:59:58.205Z"}
{"hook":"fetch_each.error","level":"error","message":"TypeError: error sending request for url (https://smen.bit.edu.cn/tzgg1/index.htm): error trying to connect: tcp connect error: Connection timed out (os error 110)","plugin":"cli","timestamp":"2023-11-07T05:59:58.205Z"}
{"hook":"fetch_each.error","level":"error","message":"TypeError: error sending request for url (https://renshichu.bit.edu.cn/gbtzgg/index.htm): error trying to connect: tcp connect error: Connection timed out (os error 110)","plugin":"cli","timestamp":"2023-11-07T05:59:58.205Z"}
{"hook":"fetch_each.error","level":"error","message":"TypeError: error sending request for url (https://hi.bit.edu.cn/rxbd/dttzyx/index.htm): error trying to connect: tcp connect error: Connection timed out (os error 110)","plugin":"cli","timestamp":"2023-11-07T05:59:58.205Z"}
{"hook":"fetch_each.error","level":"error","message":"TypeError: error sending request for url (https://physics.bit.edu.cn/tzgg4/index.htm): error trying to connect: tcp connect error: Connection timed out (os error 110)","plugin":"cli","timestamp":"2023-11-07T05:59:58.205Z"}
{"hook":"fetch_each.error","level":"error","message":"TypeError: error sending request for url (https://design.bit.edu.cn/zs/tz2/index.htm): error trying to connect: tcp connect error: Connection timed out (os error 110)","plugin":"cli","timestamp":"2023-11-07T05:59:58.205Z"}
{"hook":"fetch_each.error","level":"error","message":"TypeError: error sending request for url (https://me.bit.edu.cn/tzgg/index.htm): error trying to connect: tcp connect error: Connection timed out (os error 110)","plugin":"cli","timestamp":"2023-11-07T05:59:58.205Z"}
{"hook":"fetch_each.error","level":"error","message":"TypeError: error sending request for url (https://xyy.bit.edu.cn/tzgg/index.htm): error trying to connect: tcp connect error: Connection timed out (os error 110)","plugin":"cli","timestamp":"2023-11-07T05:59:58.205Z"}
{"hook":"fetch_each.error","level":"error","message":"TypeError: error sending request for url (https://xxgk.bit.edu.cn/zxgkxx/index.htm): error trying to connect: tcp connect error: Connection timed out (os error 110)","plugin":"cli","timestamp":"2023-11-07T05:59:58.205Z"}
{"hook":"fetch_each.error","level":"error","message":"TypeError: error sending request for url (https://xuteli.bit.edu.cn/tzgg/index.htm): error trying to connect: tcp connect error: Connection timed out (os error 110)","plugin":"cli","timestamp":"2023-11-07T05:59:58.206Z"}
{"hook":"fetch_each.error","level":"error","message":"TypeError: error sending request for url (https://sie.bit.edu.cn/index.htm): error trying to connect: tcp connect error: Connection timed out (os error 110)","plugin":"cli","timestamp":"2023-11-07T05:59:58.206Z"}
{"hook":"fetch_each.error","level":"error","message":"TypeError: error sending request for url (https://jxzx.bit.edu.cn/tzgg/index.htm): error trying to connect: tcp connect error: Connection timed out (os error 110)","plugin":"cli","timestamp":"2023-11-07T05:59:58.206Z"}
{"hook":"fetch_each.error","level":"error","message":"TypeError: error sending request for url (https://grd.bit.edu.cn/tzgg1/index.htm): error trying to connect: tcp connect error: Connection timed out (os error 110)","plugin":"cli","timestamp":"2023-11-07T05:59:58.206Z"}
{"hook":"fetch_each.error","level":"error","message":"TypeError: error sending request for url (https://itc.bit.edu.cn/tzgg/index.htm): error trying to connect: tcp connect error: Connection timed out (os error 110)","plugin":"cli","timestamp":"2023-11-07T05:59:58.206Z"}
{"hook":"fetch_each.error","level":"error","message":"TypeError: error sending request for url (https://opt.bit.edu.cn/tzgg2/tzgg/index.htm): error trying to connect: tcp connect error: Connection timed out (os error 110)","plugin":"cli","timestamp":"2023-11-07T05:59:58.206Z"}
{"hook":"fetch_each.error","level":"error","message":"TypeError: error sending request for url (https://jwb.bit.edu.cn/tzgg/index.htm): error trying to connect: tcp connect error: Connection timed out (os error 110)","plugin":"cli","timestamp":"2023-11-07T05:59:58.206Z"}
{"hook":"fetch_each.error","level":"error","message":"TypeError: error sending request for url (https://isc.bit.edu.cn/eventsnotices/index.htm): error trying to connect: tcp connect error: Connection timed out (os error 110)","plugin":"cli","timestamp":"2023-11-07T05:59:58.206Z"}
{"hook":"fetch_each.error","level":"error","message":"TypeError: error sending request for url (https://math.bit.edu.cn/tzgg/index.htm): error trying to connect: tcp connect error: Connection timed out (os error 110)","plugin":"cli","timestamp":"2023-11-07T05:59:58.207Z"}
{"hook":"fetch_each.error","level":"error","message":"TypeError: error sending request for url (https://mingde.bit.edu.cn/tzgg/index.htm): error trying to connect: tcp connect error: Connection timed out (os error 110)","plugin":"cli","timestamp":"2023-11-07T05:59:58.207Z"}
{"level":"warn","message":"未找到以往通知，您可能是第一次运行。将忽略。","timestamp":"2023-11-07T05:59:58.216Z"}
{"level":"info","message":"已按需保存到“output/notices.json”。","plugin":"cli","timestamp":"2023-11-07T05:59:58.219Z"}

```shell rg "from '[^.]*'" -t ts --no-filename | sort | uniq | rg -v 'node:' ```

YDX-2147483647 · 2023-11-10T15:30:21Z

deno 的 Fetch API 似乎和浏览器一样，没有超时一说，可能卡住。

更新：从2023年11月8–23日在校内测试服务器上要求每日运行一次，只有10日、15日异常，都是超时。

2023年12月12日更新：上次至今，11月24–12月1日、12月3日、12月5–11日都卡住了，其余都成功更新。从2023年12月12日起，改用 #53，让我们拭目以待。

2023年12月21日更新：12月12–17、20–21日正常，18、19日卡住。此后修改了 crontab 脚本，超过 10 min 强制结束。

2023年12月29日更新：12月23、26–28日正常，22、24、25日超时。

2023年12月28日在另一校外 Windows 服务器上每日运行一次，不另设时间上限。

2023年12月28–29日正常。

The log records the following errors. ```shell jq 'select(.level == "error" and (.message | contains("Timeout") | not))' output/combined.log | bat -l json ``` ``` HTTPError: Request failed with status code 400 Bad Request SyntaxError: Unexpected token '<', "<!DOCTYPE "... is not valid JSON ```

YDX-2147483647 added 10 commits November 5, 2023 22:43

chore: Update imports for deno

f341003

refactor: jsdom → deno-dom

2105d8c

https://docs.deno.com/runtime/manual/advanced/jsx_dom/jsdom https://docs.deno.com/runtime/manual/advanced/jsx_dom/deno_dom

chore: Update imports for deno

902f3bc

refactor: Drop node-fetch

48aeb4f

chore: Update launch.json for deno

374e136

test: Use deno

ee4bfcf

build: Map std imports

d846277

style: deno fmt src. *.md */*.json

ae714df

lint: deno lint src

01d731b

The proxy plugin is ignored.

chore: Drop unused dependencies

db2c100

YDX-2147483647 force-pushed the deno branch from 5fec9fc to db2c100 Compare November 6, 2023 11:00

YDX-2147483647 added 7 commits November 6, 2023 20:09

chore: Install virtual-BIT-network from npm instead of GitHub

805ed0f

chore: Vendor virtual-BIT-network

d2d2a5e

chore: Update imports for deno

f51fa78

test: Use deno

51a69e7

style: deno fmt

5df28df

lint

4332010

lint: deno check

a48d1d2

YDX-2147483647 added 9 commits November 8, 2023 00:35

chore: Prepend imports with npm:

2d069da

refactor: Drop cheerio

e3c3c5b

chore: Update imports for deno

8b17e8a

```shell rg "from '[^.]*'" -t ts --no-filename | sort | uniq | rg -v 'node:' ```

refactor: check-sources-urls

613dc59

Merge branch 'main' into deno

5b49a52

chore: Drop useless configurations

7c881bb

refactor: Remove `node:process'

5dfc656

refactor: Drop node:fs APIs

667dceb

chore: npm run → deno task

5f51990

YDX-2147483647 mentioned this pull request Nov 8, 2023

[config] Add sources #40

Merged

YDX-2147483647 added 7 commits November 10, 2023 23:30

Merge branch 'main' into deno

c7725e4

Merge branch 'main' into deno

2668607

Merge branch 'main' into deno

2d639a6

style: deno task fmt

eb04086

refactor: fetch → ky

90286e6

refactor: Deno.errors.TimedOut → ky.TimeoutError

89fd771

feat(plugin): Retry after timeout

9e6458c

YDX-2147483647 mentioned this pull request Dec 17, 2023

feat: Retry and timeout #53

Merged

YDX-2147483647 added 2 commits December 17, 2023 20:03

Merge pull request #53 from YDX-2147483647/ky

366ab51

Merge branch 'main' into deno

64617d9

YDX-2147483647 mentioned this pull request Dec 22, 2023

Bulletin Issues Transferred denoland/nodetodenochallenge#17

Closed

YDX-2147483647 added 3 commits December 25, 2023 21:24

style: deno task fmt

e2d30d0

Merge branch 'main' into deno

d5a2b3c

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

refactor: Deno #48

refactor: Deno #48

YDX-2147483647 commented Nov 6, 2023 •

edited

Loading

YDX-2147483647 commented Nov 7, 2023

YDX-2147483647 commented Nov 10, 2023 •

edited

Loading

refactor: Deno #48

Are you sure you want to change the base?

refactor: Deno #48

Conversation

YDX-2147483647 commented Nov 6, 2023 • edited Loading

YDX-2147483647 commented Nov 7, 2023

YDX-2147483647 commented Nov 10, 2023 • edited Loading

YDX-2147483647 commented Nov 6, 2023 •

edited

Loading

YDX-2147483647 commented Nov 10, 2023 •

edited

Loading