Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Распарсіць назвы арганізацый і дату адказу #13

Closed
gsk990 opened this issue Aug 2, 2018 · 58 comments
Closed

Распарсіць назвы арганізацый і дату адказу #13

gsk990 opened this issue Aug 2, 2018 · 58 comments
Assignees
@gsk990

Comments

@gsk990
Copy link
Member

gsk990 commented Aug 2, 2018

напрыклад, 115.бел/problem/16628

portia spider?
згенерыраваць спіс ID заявак і парсіць?
@gsk990 gsk990 self-assigned this Aug 2, 2018
@gsk990
Copy link
Member Author

gsk990 commented Aug 29, 2018

portia не працуе з гэтымі старонкамі, выдае 0 вынікаў
спрабую з google spreadsheet importxml, першая спроба: не атрымалася, зашмат запытаў, скрэйпер завіс
другая спроба:
згенерыраваць спіс ID заявак БЕЗ арганізацый, зараз дадаю ў шыт, пачынаю парсіць па частках, па 1000-5000 заявак

усяго 63355 заявак без арганізацый
list-of-problems-without-organizations.txt

@gsk990
Copy link
Member Author

gsk990 commented Aug 29, 2018 •
edited
Loading

вось тут будзе ЦСВ з вынікамі парсінга:
https://docs.google.com/spreadsheets/d/e/2PACX-1vQRML8wrCcm5Gw2nmINA8ovnFdogsqzzRDCk_b1F372qVFszJwwIVrgrCoQXMk5iN7yDS161vHZiQTD/pub?gid=0&single=true&output=csv

вось скрэйпінг шыт
https://docs.google.com/spreadsheets/d/1wRPS4y_zPyUrMUkzr3F9-xbnhR60EW6hAyytD1B8mXw/edit#gid=0

@gsk990
Copy link
Member Author

gsk990 commented Aug 30, 2018 •
edited
Loading

new scraping sheet -> CSV export https://docs.google.com/spreadsheets/d/e/2PACX-1vQRML8wrCcm5Gw2nmINA8ovnFdogsqzzRDCk_b1F372qVFszJwwIVrgrCoQXMk5iN7yDS161vHZiQTD/pub?gid=0&single=true&output=csv

@nickolayka
Copy link

что-то не работает (

@gsk990
Copy link
Member Author

gsk990 commented Aug 31, 2018 via email •
edited
Loading

@gsk990
Copy link
Member Author

gsk990 commented Sep 1, 2018

працэс ідзе, але з паўзамі. вось частковы датасет, 3187 заявак з арганізацыямі - з тых, у каторых не было прыпісаных арганізацый. Фармат - ХТМЛ спіс.
harvested_results-3187.txt

@gsk990
Copy link
Member Author

gsk990 commented Sep 2, 2018

на сёння - 4250 заявак распарсілі

@gsk990
Copy link
Member Author

gsk990 commented Sep 2, 2018

4792!

@gsk990
Copy link
Member Author

gsk990 commented Sep 2, 2018

5585

@gsk990
Copy link
Member Author

gsk990 commented Sep 3, 2018

6000

@gsk990
Copy link
Member Author

gsk990 commented Sep 3, 2018

6350

@gsk990
Copy link
Member Author

gsk990 commented Sep 3, 2018

6950

@gsk990
Copy link
Member Author

gsk990 commented Sep 3, 2018

7204

@gsk990
Copy link
Member Author

gsk990 commented Sep 3, 2018

7400

@gsk990
Copy link
Member Author

gsk990 commented Sep 3, 2018

7725

@gsk990
Copy link
Member Author

gsk990 commented Sep 3, 2018

7979

@gsk990
Copy link
Member Author

gsk990 commented Sep 3, 2018

8589

@gsk990
Copy link
Member Author

gsk990 commented Sep 3, 2018

9441

@gsk990
Copy link
Member Author

gsk990 commented Sep 4, 2018

10074

@gsk990
Copy link
Member Author

gsk990 commented Sep 4, 2018

11659

@gsk990
Copy link
Member Author

gsk990 commented Sep 4, 2018

12870

@gsk990
Copy link
Member Author

gsk990 commented Sep 4, 2018

13428

@gsk990
Copy link
Member Author

gsk990 commented Sep 5, 2018

14312

@gsk990
Copy link
Member Author

gsk990 commented Sep 5, 2018

14715

@gsk990
Copy link
Member Author

gsk990 commented Sep 6, 2018

16418

@gsk990
Copy link
Member Author

gsk990 commented Sep 6, 2018

16685

@gsk990
Copy link
Member Author

gsk990 commented Sep 6, 2018

17390

@gsk990
Copy link
Member Author

gsk990 commented Sep 6, 2018

17657

@gsk990
Copy link
Member Author

gsk990 commented Sep 8, 2018

23000

@gsk990
Copy link
Member Author

gsk990 commented Sep 9, 2018

25626

@gsk990
Copy link
Member Author

gsk990 commented Sep 9, 2018

25767

@gsk990
Copy link
Member Author

gsk990 commented Sep 9, 2018

26263

@gsk990
Copy link
Member Author

gsk990 commented Sep 9, 2018

26659

@gsk990
Copy link
Member Author

gsk990 commented Sep 10, 2018

31953

@gsk990
Copy link
Member Author

gsk990 commented Sep 10, 2018

32446

@gsk990
Copy link
Member Author

gsk990 commented Sep 11, 2018

34856

@gsk990
Copy link
Member Author

gsk990 commented Sep 12, 2018

41257

@gsk990
Copy link
Member Author

gsk990 commented Sep 12, 2018

41364

@gsk990
Copy link
Member Author

gsk990 commented Sep 12, 2018

43214

@gsk990
Copy link
Member Author

gsk990 commented Sep 12, 2018

43991

@gsk990
Copy link
Member Author

gsk990 commented Sep 12, 2018

45739

@gsk990
Copy link
Member Author

gsk990 commented Sep 12, 2018

46740

@gsk990
Copy link
Member Author

gsk990 commented Sep 12, 2018

49042

@gsk990
Copy link
Member Author

gsk990 commented Sep 13, 2018

53650

@gsk990
Copy link
Member Author

gsk990 commented Sep 13, 2018

54658

@gsk990
Copy link
Member Author

gsk990 commented Sep 13, 2018

54808

@gsk990
Copy link
Member Author

gsk990 commented Sep 13, 2018

55390

@gsk990
Copy link
Member Author

gsk990 commented Sep 14, 2018

56726

@gsk990
Copy link
Member Author

gsk990 commented Sep 14, 2018

57629

@gsk990
Copy link
Member Author

gsk990 commented Sep 15, 2018

59390

@gsk990
Copy link
Member Author

gsk990 commented Sep 15, 2018

60642!

@gsk990
Copy link
Member Author

gsk990 commented Sep 15, 2018

60964

@gsk990
Copy link
Member Author

gsk990 commented Sep 16, 2018

65896

@gsk990
Copy link
Member Author

gsk990 commented Sep 16, 2018

68654

@gsk990 gsk990 mentioned this issue Jan 19, 2019
Closed
@gsk990
Copy link
Member Author

gsk990 commented Feb 8, 2019

усё распаршана і запісана ў #8

@gsk990 gsk990 closed this as completed Feb 8, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@gsk990 gsk990

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@gsk990 @radachynskaja @nickolayka