-
Notifications
You must be signed in to change notification settings - Fork 3
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Implementation of Job Website Crawling #7
base: main
Are you sure you want to change the base?
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
크롤러를 스프링서버에서 Scheduler를 써서 돌리려면 파이썬 파일을 jar안에 넣어서 불러줘야 하는데, 이 부분은 어떻게 하실 예정인가요?
|
||
private final PyhonRunner pyhonRunner; | ||
|
||
@Scheduled(cron = "0 0 12 * * ?") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
좀 더 서비스가 한가한 시간에 하면 좋지 않을까요? :-)
} | ||
|
||
@Test | ||
void runMyPythonRunner() throws IOException { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
좀 더 제대로 테스트를 하고 싶으면, 어떻게 할 수 있을까요?
|
||
private Process runPythonCrawling() throws IOException, InterruptedException { | ||
ProcessBuilder pythonBuilder = new ProcessBuilder( | ||
"python3","../crawling/src/main/python/crawl_job_website.py" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
jar로 묶으면, "../crawling/src/main/python/crawl_job_website.py" 경로가 없기 때문에 실패할 텐데요. 이걸 어떻게 해결할 수 있을지 생각해보시면 좋을 것 같습니다.
힌트는 gradle에서 빌드를 할 때, python의 빌드결과를 먼저 app의 resource에 넣어주면, jar에 들어가게 되고, https://www.baeldung.com/spring-classpath-file-access#resource-utils 등을 사용해서 python 파일을 읽으면 됩니다.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
작업 내용