Group post title

[azimjiwani]

Hey! The scraper works great! Is it possible to get a post's title in a FB group from the scraper? I haven't seen the posts' title, just the body text.

Thanks in advance!

[neon-ninja]

Can you post a screenshot or URL of what you mean? Posts don't have titles, AFAIK

[azimjiwani]

Heres the link: https://www.facebook.com/groups/110354088989367
The scraper returns the body text for all the posts, but not the title (in bold)

[neon-ninja]

I see - you're referring to the listing titles in a buy sell group. This commit should suffice to extract them - 4be60e4. It surfaces the listing information with the keys listing_title, listing_price, and listing_location.

Test code:

pprint.pprint(next(get_posts(
    post_urls=["https://m.facebook.com/groups/110354088989367/permalink/4312189348805799/"],
    cookies="cookies.txt"
)))

output

{'available': True,
 'comments': 0,
 'comments_full': None,
 'factcheck': None,
 'image': None,
 'image_lowquality': None,
 'images': [],
 'images_description': [],
 'images_lowquality': [],
 'images_lowquality_description': [],
 'is_live': False,
 'likes': 0,
 'link': 'http://www.civil.uwaterloo.ca/xie/277LesterStreet.htm',
 'listing_location': 'Downtown Waterloo',
 'listing_price': '$1',
 'listing_title': '5 Beds 4 Baths Apartment',
 'original_request_url': 'https://m.facebook.com/groups/110354088989367/permalink/4312189348805799/',
 'post_id': '4312189348805799',
 'post_text': '5 BEDROOM UNITS AVAILABLE AT 277 LESTER STREET FOR FALL 2021!!\n'
              '\n'
              'http://www.civil.uwaterloo.ca/xie/277LesterStreet.htm',
 'post_url': 'https://m.facebook.com/groups/110354088989367/permalink/4312189348805799/',
 'reaction_count': None,
 'reactions': None,
 'reactors': None,
 'shared_post_id': None,
 'shared_post_url': None,
 'shared_text': '',
 'shared_time': None,
 'shared_user_id': None,
 'shared_username': None,
 'shares': 0,
 'text': '5 BEDROOM UNITS AVAILABLE AT 277 LESTER STREET FOR FALL 2021!!\n'
         '\n'
         'http://www.civil.uwaterloo.ca/xie/277LesterStreet.htm',
 'time': datetime.datetime(2021, 5, 22, 2, 12, 57),
 'user_id': 1497972649,
 'user_url': None,
 'username': 'Tiffany Xie',
 'video': None,
 'video_duration_seconds': None,
 'video_height': None,
 'video_id': None,
 'video_quality': None,
 'video_size_MB': None,
 'video_thumbnail': None,
 'video_watches': None,
 'video_width': None,
 'w3_fb_url': None}

[azimjiwani]

Hey, its still not working for me. I cloned the repo as well as installed with pip. Still getting the same output as before

…

On May 27, 2021, at 5:04 PM, Nick Young ***@***.***> wrote: I see - you're referring to the listing titles in a buy sell group. This commit should suffice to extract them - 4be60e4 <4be60e4>. It surfaces the listing information with the keys listing_title, listing_price, and listing_location. Test code: pprint.pprint(next(get_posts( post_urls=["https://m.facebook.com/groups/110354088989367/permalink/4312189348805799/"], cookies="cookies.txt" ))) output {'available': True, 'comments': 0, 'comments_full': None, 'factcheck': None, 'image': None, 'image_lowquality': None, 'images': [], 'images_description': [], 'images_lowquality': [], 'images_lowquality_description': [], 'is_live': False, 'likes': 0, 'link': 'http://www.civil.uwaterloo.ca/xie/277LesterStreet.htm', 'listing_location': 'Downtown Waterloo', 'listing_price': '$1', 'listing_title': '5 Beds 4 Baths Apartment', 'original_request_url': 'https://m.facebook.com/groups/110354088989367/permalink/4312189348805799/', 'post_id': '4312189348805799', 'post_text': '5 BEDROOM UNITS AVAILABLE AT 277 LESTER STREET FOR FALL 2021!!\n' '\n' 'http://www.civil.uwaterloo.ca/xie/277LesterStreet.htm', 'post_url': 'https://m.facebook.com/groups/110354088989367/permalink/4312189348805799/', 'reaction_count': None, 'reactions': None, 'reactors': None, 'shared_post_id': None, 'shared_post_url': None, 'shared_text': '', 'shared_time': None, 'shared_user_id': None, 'shared_username': None, 'shares': 0, 'text': '5 BEDROOM UNITS AVAILABLE AT 277 LESTER STREET FOR FALL 2021!!\n' '\n' 'http://www.civil.uwaterloo.ca/xie/277LesterStreet.htm', 'time': datetime.datetime(2021, 5, 22, 2, 12, 57), 'user_id': 1497972649, 'user_url': None, 'username': 'Tiffany Xie', 'video': None, 'video_duration_seconds': None, 'video_height': None, 'video_id': None, 'video_quality': None, 'video_size_MB': None, 'video_thumbnail': None, 'video_watches': None, 'video_width': None, 'w3_fb_url': None} — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#296 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AHT6QLN6YPANHNMUWVR6SBTTP2XU3ANCNFSM45TOHXPA>.

[azimjiwani]

With the cookies parameter, an additional error arises: ValueError: not enough values to unpack (expected 7, got 1)

[neon-ninja]

I haven't released a new version containing this feature. You can use the command pip install git+https://github.com/kevinzg/facebook-scraper.git to install the latest master branch. It works fine for me with or without cookies. There must be something wrong with your cookies file if you get that error.

[azimjiwani]

Hey, Looking at your implementation code, listing title only appears in the response when a permalink is provided in the URL. Is there a way for listing location, price, and title are returned in the response without providing a permalink?

…

On May 28, 2021, at 11:59 PM, Nick Young ***@***.***> wrote: I haven't released a new version containing this feature. You can use the command pip install git+https://github.com/kevinzg/facebook-scraper.git to install the latest master branch. It works fine for me with or without cookies. There must be something wrong with your cookies file if you get that error. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#296 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AHT6QLKA72Q3DFOKSDP4VY3TQBRDBANCNFSM45TOHXPA>.

[neon-ninja]

Should still work without permalinks, here's some test code:

for post in get_posts(group=110354088989367, pages=1, timeout=60, options={"allow_extra_requests": False}):
    if post.get("listing_title"):
        print(post["post_id"], post["time"], post["listing_title"], post["listing_price"], post["listing_location"])

outputs

4366147620076638 2021-06-08 11:46:48 2 Beds 2 Baths Apartment $1,650 Downtown Waterloo
4365752733449460 2021-06-08 08:34:05 2 beds 1 bath Flat $850 Downtown Waterloo
4347187368639330 2021-06-02 05:13:44 5 Beds 5 Baths Apartment $585 Clair Hills
4354242324600501 2021-06-04 14:08:40 4 Beds 4 Baths Apartment $900 N2L
4366090706748996 2021-06-08 11:21:02 Looking for sublet - 2/4 bedrooms $1 N2L
4361432910548109 2021-06-06 23:29:24 Looking for a room from July FREE 
4362589303765803 2021-06-07 07:15:17 2/2 Bond Suite Available for Fall 2021 $1,700 N2L 3W9
4365883890103011 2021-06-08 09:37:34 1 Bed 1 Bath Apartment $1,600 Downtown Waterloo
4366051253419608 2021-06-08 11:02:29 Cute retro blue bike for sale $400 L7E 1Y5
4366012306756836 2021-06-08 10:43:16 Looking for 4 bedroom sublet Sept-Dec 2021 $1 Waterloo
4366006993424034 2021-06-08 10:40:25 Looking for a 4/5 bedroom- fall 2021 FREE Uptown West
4365440586814008 2021-06-08 06:20:07 2 Beds 1 Bath Apartment $750 Downtown Waterloo
4365830870108313 2021-06-08 09:12:06 1 Bed 1 Bath Apartment $1 Downtown Waterloo

This PR (https://github.com/kevinzg/facebook-scraper/pull/321/files) inadvertently introduced a bug with group post extraction, which 223f136 should fix - perhaps this was your problem

[azimjiwani]

works great now, thanks!