-
Notifications
You must be signed in to change notification settings - Fork 58
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Moved URL's aren't crawled #24
Comments
This is also happening with 302 responses. |
Hi @n3wjack. Sorry about the delay in getting back to you - didn't spot this issue. If you've got code that's "nearly there", perhaps we could start a feature branch and work on it there? Let me know and I'll create a branch for you. Tom |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
A url that returns a 301 (moved permanently) doesn't get crawled afterwards.
I seem to be having a bunch of those where url's not ending in a slash give a 301 to the page containing the trailing slash.
I'm guessing this is not intended.
If not, let me know and I have some code ready that fixes this which needs some cleanup and perhaps some unit tests.
The text was updated successfully, but these errors were encountered: