Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The script works strangely #33

Open
Taimerlan opened this issue Mar 6, 2024 · 5 comments
Open

The script works strangely #33

Taimerlan opened this issue Mar 6, 2024 · 5 comments

Comments

@Taimerlan
Copy link

Hi all,
I'm trying to index a site with 1 million pages, after a long time it gives out:
2024-03-06_23-15-44
Web Search Indexing API quota is not consumed, no errors.
2024-03-06_23-18-49
The cache is also not created in the folder. There is only one file.
Can anyone encounter it?

@AntoineKM
Copy link
Contributor

Hey @Taimerlan, what command did you run to start the script?

@Taimerlan
Copy link
Author

Taimerlan commented Mar 7, 2024

Hey @Taimerlan, what command did you run to start the script?

npm run index ....

Tried through a plugin on the site to start the index pages, all was successful.
But the script used to work when he could not take all pages from the site map, the maximum took 80-90 thousand, and when I made the site map quickly available, he captured all pages, maybe because of the number of pages such a problem?

Update: Trying to index small site, it was successful.

@AntoineKM
Copy link
Contributor

Hey @Taimerlan, what command did you run to start the script?

npm run index ....

Tried through a plugin on the site to start the index pages, all was successful. But the script used to work when he could not take all pages from the site map, the maximum took 80-90 thousand, and when I made the site map quickly available, he captured all pages, maybe because of the number of pages such a problem?

Update: Trying to index small site, it was successful.

Are you using the latest version of this repository ?

@Taimerlan
Copy link
Author

Hello, sorry for late answer. First time I use the new version with ts extention. Where as I understand there was an update to stop after an error. #29
Now I tried the most recent version, but the same thing happens. I can send you the service_account.json file and the site address to your e-mail.

@AntoineKM
Copy link
Contributor

I realize it's due to what I said here: #38 (comment)
We'll try to improve it, but it's really a big black spot at the moment.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants