Today I feel the insatiable urge of building a tool to scrape the shit out of and export my timeline into an feed, or at least something that can be easily integrated into the .

I haven't posted anything on my Facebook profile for months nor accessed their website unless some friend shared a direct link to a photo or a video with me. Granted, I feel like the Fediverse is a much healthier place that doesn't make me feel as guilty as if I were chain-smoking and consuming junk food while driving a huge CO2-spewing SUV.

But, even if I met a lot of amazing people here, and I have even managed to bridge Twitter profiles and content from a vast trove of RSS feeds, and I have built bots that take a lot of interesting content directly to my door, and I have even managed to keep messaging my friends on Messenger/WhatsApp through Matrix and Bitlbee bridges, there's still an uncomfortable truth that doesn't make me sleep at night: Zuckerberg is still holding most of my family and friends as hostages, they will probably never move to the Fediverse, and I'm missing out on the lives of my loved ones (as well as on a lot of interesting events happening around me) because that content is behind a huge impenetrable wall.

I'm sick of hearing "Facebook should be compelled to federate, or at least open up their APIs for personal usage, but we don't know where to start". Or "the is great on paper, but it's hard to enforce". If regulators don't take the matter into their hands, then I will. And, if Facebook dares to sue me or lock my account, I'm ready to sue them back for violating the DMA. I'm ready to take this matter in front of courts and spend my money on lawyers, because I want Facebook and their highly immoral "high switching costs" strategy to die amid the worst conceivable pains in this universe - or at least I want them to be forced to open up the data of my loved ones.

I was looking around for some up-to-date Facebook scrapers, but all I could find was this project github.com/kevinzg/facebook-sc (which only scrapes public pages) and some commercial solutions that provide Facebook scraping for profiling and ads purposes (which makes me wanna spit and puke on the people behind those businesses for proudly showing off the worst that a human being can be capable of and making a profit out of it).

I made a Facebook scraper around 10 years ago, but back then their pages were relatively simple, and a bit of beautifulsoup scripting was enough to scrape the shit out of them. I've now taken a look at the developers console while browsing the website, and I've been horrified by how much effort they've put to prevent exactly what I was trying to do - the whole Facebook feed is basically a bunch of <script type="application/json"> tags that download some custom minified JavaScript for each post, that in turn is used to decrypt some other JSON requests.

So I'm appealing to all the hackers and tinkerers out there: are there FLOSS projects that already do what I'm trying to do (basically allow you to sign in to Facebook with your account, get an access token, and scrape posts and comments from your own timeline)? If not, are there any volunteers out there who would like to join forces with me in a new dog-and-cat war with Facebook - starting with reverse engineering whatever mechanism they've put in place to obfuscate the HTML on their timelines?

Follow

p.s. there's something like this apparently: github.com/harismuneer/Ultimat.

The UFS apparently used to be open-source, and now the whole Github project consists of a README that invites people to pay $119 for a license and a Zoom call where the author shares the code and the instructions to install it.

I want to build something really FLOSS also to make sure that these borderline scammers who profit from a real-world need and exploit the open-source community have no bread left to eat.

· · Web · 0 · 1 · 1
Sign in to participate in the conversation
Mastodon

A platform about automation, open-source, software development, data science, science and tech.