Replies: 1 comment 1 reply
-
Hello, One way to do this is to use the This is very simple example, the pseudo URL can be made more precise.
For more lively discussion, I recommend using our Discord community server. |
Beta Was this translation helpful? Give feedback.
1 reply
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
I am using
Apify.utils.enqueueLinks
and I'd like to prevent URLs that are not in the root domain so I don't crawl anything other than the website itself.For example if I
rushi.com
with several pages, and one of those pages links toblog.github.com
I don't want to recurse into that URL because github.com != rushi.comWhat I'm looking for is Apify's equivalent of wget's
--no-parent
option. I also want to ensure relative URLs are picked up sobaseUrl: request.loadedUrl
needs to be presentThis is the code I have so far direct from the examples
Beta Was this translation helpful? Give feedback.
All reactions