How to Scrape Facebook Page Posts Data to Excel (Spy Competitor)

In this tutorial, I will teach you step-by-step how to scrape or extract ANY public Facebook Page Posts Data to Excel using Facebook Graph API without programming. It’s very easy just follow below steps.

You can use this trick to spy on your competitor’s Facebook fan page and download all its data to Excel for analysis.

Following the tutorial you will be able to scrape the following data from your competitor Facebook page posts to Excel:

• The number of likes,shares and comments for each post
• The number of Facebook reactions (Like,Love,Haha,Wow,Sad,Angry) for each post
• The message of each post (Facebook status messages)
• Facebook post comments for each post
• Facebook IDs for the users who have liked, commented on or shared a post

Below sample data was extracted from Donald Trump Facebook page.

Okay,Let’s get started!

Step 1:Get Access Token from the Facebook Graph API Explorer

First,We need to get an access token from the Facebook Graph API explorer.We will use the token to get authentication to pull data from Facebook databases.This is a must do step!You will need to login using your private Facebook account.

After successfully logging in you will be redirected to the Facebook Graph API explorer interface as shown in the window below. You will see the access token (see the red rectangular box below) already generated for you. Copy and paste it to store safely somewhere because we will need it in the later steps.

An access token which never expires

The above-generated token normally expires after 2 hours and you can always get a new one by clicking Get Token. 

But What if you want an access token which never expires?

You must create a Facebook for Developers account to be able to create an access token which never expires. Don’t worry it so easy than it sounds. Click this link and follow the wizard. After setting up your account then go to your  dashboard>settings>basic then click show App secret.

The never expiring access token will be your app ID|app secret .Don’t forget the pipe “|” between the app ID and app secret. For example, this is my never expiring access token 1179322318819014|a5axxxxxxxxxxxx.

Step 2:Get your competitor’s Facebook Fan Page ID from the Graph API

Facebook assigns unique IDs to every Facebook fan page. In this step, we will get the Facebook unique ID of our competitor Facebook fan page. Our competitor page for this tutorial will be  https://www.facebook.com/DiamondPlatnumz255/.Diamond Platnumz is a famous musician from Tanzania.

In order to get the unique ID, you will need to enter the following URL in your browser.
“https://graph.facebook.com/{fan-page-name}?access_token={access-token}”

Replace the {fan-page-name} with your competitor fan-page-name (in our case, DiamondPlatnumz255) and replace the {access-token} with the access token from the earlier step. Access tokens have an expiration time, so if you find it has expired you will need to log in again and get a new one.

So the URL I entered on the browser is like this one below. You must use your own access-token, not mine, or it won’t work.

https://graph.facebook.com/DiamondPlatnumz255?access_token=EAACEdE (Truncated)

After that, you will get an XML format document like the one below. The ID we are looking for is at the top (see the red rectangular box) along with all of the other details of the page. Copy and save this ID somewhere because we will need it for the later steps.

Step 3:Pulling data from your competitor’s Facebook page

Now let us pull raw data from our competitor Facebook page. Make sure you completed Step 1 and 2 above.

We will need to use the following URL to get all the data. Be sure to replace all the information in brackets with the correct information for your competitor.The query will return data in chronological order

https://graph.facebook.com/v2.6/{fan-page-ID}/posts/?fields=message,link,permalink_url,created_time,type,name,id,comments.limit(0).summary(true),shares,likes.limit(0).summary(true),reactions.limit(0).summary(true)&limit=100&access_token={your-access-token}

This query will return 100 rows of posts if they do exist. If the page has more than 100 posts then there will be pagination at the bottom of the results page allowing you to retrieve next set of data.

The brackets will be replaced as follows by the information we collected from previous steps such as Fan page ID and access token.

{fan-page-ID} = 204153042939851

{your-access-token}  = EAACEdE (Truncated)

The URL I entered on the browser is like this one below with all the fields filled in but since the access-token is so long it is not shown here in full. You must use your own access-token, not mine, or it won’t work

https://graph.facebook.com/v2.6/204153042939851/posts/?fields=message,link,permalink_url,created_time,type,name,id,comments.limit(0).summary(true),shares,likes.limit(0).summary(true),reactions.limit(0).summary(true)&limit=100&access_token=EAACEdE (Truncated)

If everything is correct the browser will return a JSON response which looks like the window below

Please observe the pagination at the bottom of the page as we will refer to it in step 4 for retrieving more data.

Scraping data between specific dates

The previous query will return data in chronological order.What if you want to scrape data between specific dates?

Run below query make sure to change the since and until dates. Do include your page name/ID and access token as well.

https://graph.facebook.com/v2.6/YourPagaNameOrID/posts/?fields=message,link,permalink_url,created_time,type,name,id,comments.limit(0).summary(true),shares,likes.limit(0).summary(true),reactions.limit(0).summary(true)&until=2015-01-30&since=2015-01-01&limit=100&access_token=

Step 4:Converting JSON file to CSV/Excel

In this step, we will copy the returned JSON response and convert it into CSV or Excel using free to use online JSON to CSV/Excel converter.

While you are still on the JSON response page copy the JSON response and paste to the online converter just like in the image below.In order to make sure you are copying everything correctly on Windows click CTRL+A then CTRL+C or if you are using MAC Command-A then Command-C.

Click “Download the entire CSV” to get the converted data on your computer.

As expected, we retrieved 100 rows of data and permalinks. We also received information about the messages, like count, comments count, share count,reactions count and the created time for each post.

After removing columns with NONE,TRUE,FALSE and ranked I ended up with a nice looking CSV like below ready for analysis or further processing.

If you want to extract more data you need to take advantage of pagination returned by the previous result.While you are still on the previous page of JSON result.Scroll to the bottom of the page till you see “paging” then copy the URL  after “next”.Copy everything starting from the https till before the closing “.

After copying it will be like below URL

https://graph.facebook.com/v2.6/204153042939851/posts?fields=message,link,permalink_url,created_time,type,name,id,comments.limitu0025280u002529.summaryu002528trueu002529,shares,likes.limitu0025280u002529.summaryu002528trueu002529,reactions.limitu0025280u002529.summaryu002528trueu002529&limit=100&access_token=EAACEdE (Truncated)

Unfortunately,if you enter this URL into your browser as it is,it will bring an error.Reason being it has these values(u002528,u002529) which can’t be interpreted by the browser.Actually,these values represent the ( and ) respectively.

So we need to clean the URL before entering into the browser.Copy and paste the URL into Notepad text editor for Windows or TextEdit for MAC.Then use the find and replace all feature.Find u002528 and replace all with ( then find u002529 and replace all with ).

Finally, I got this clean URL below which I can input on the browser to get the next set of 100 results.Remember to clean the URL everytime you want to retrieve more data.

https://graph.facebook.com/v2.6/204153042939851/posts?fields=message,link,permalink_url,created_time,type,name,id,comments.limit(0).summary(true),shares,likes.limit(0).summary(true),reactions.limit(0).summary(true)&limit=100&access_token=EAACEd(Truncated)D&until=1465296827&__paging_token=enc_AdDWxw7(Truncated)

Now that we have all the data we need we are ready to go on to the final step of the competitor analysis.

Step 5:Competitor analysis

Now it is time to do the analysis for the competitor’s page. In the previous step, we scraped data from 100 posts between 2016-08-18 to 2016-06-07 – that’s almost 2 months of data! However, you can go even further and collect more data depending on what you need. The competitor page was created in 2011 so there is far more data to pull and analyse.

Referring back to the goals we set at the beginning of the tutorial, below is what we obtained from the data:

Posts which received the most likes in descending order

You can repeat the same process for the posts which received the most shares and comments. After that, it will be easy to tell which post type works best if you are aiming to get likes, shares and comments on your own content.

From the created time for each post you can also tell what time of the day and week works bests to achieve likes, shares and comments. You can also see the frequency with which your competitor updates his or her page.

Step 6:Exporting Facebook posts comments to Excel

Sometimes you may need to retrieve a competitor’s Facebook post comments to analyse how users or fans interact with certain posts. Through these comments, you can get a lot of insights into your competitor such as the weaknesses or strengths of your competitor’s products.

In order to do that, you will need to have the post ID of the post you want to retrieve comments from and the access token.

First, we need to get the post ID. Referring to the Excel spreadsheet we obtained in Step 4 above, you will find a column with posts ID named as ID.

Refer back to Step 1 to get the access token if you still don’t have one yet.

The format of the URL will be like below

https://graph.facebook.com/{post-ID}/comments?access_token=XXX

Using the details of the example competitor page we used earlier, the URL will look like below. You can play with the limit parameter depending on the number of comments there are on the post, but there is a limit to it so you can’t input a very large number.

https://graph.facebook.com/204153042939851_761338343887982/comments?limit=900&access_token=EACEdE (Truncated)

You will get the results in JSON format similar to the window below. You will then need to use a JSON to CSV converter to get the results into Excel format so they can be easily analysed.

If the post has a lot of comments then you need to play with the limit parameter as the Facebook Graph API has a limit on the amount of data it can retrieve at once. The exact limit varies so you may need a few tries to find the maximum.

If you have more comments than Facebook can retrieve at once the pagination option will appear by default at the bottom of the page as shown in the image below.You can then input this URL into the browser,this kind of URL doesn’t need cleaning as we did in Step 4.

Step 7:Extracting Facebook IDs of users who like,comment or share a post

Step 7a:Extracting Facebook IDs of users who comment on a post

Referring to the result we obtained in Step 6 above, you will see that the result included IDs of all users who commented on the post.

Step 7b:Extracting Facebook IDs of users who like a post

Now let us get the Facebook IDs of the users who like the post. We need to put the following URL in the browser again – again, make sure that your access token has not expired! If it has expired then you will need to get a new one by following the earlier steps. The only thing which has changed on the URL is swapping of “comments” and “likes” – all the other parameters do not change.

https://graph.facebook.com/204153042939851_761338343887982/likes?limit=2000&access_token=EAACEdE (Truncated)

You will get a window similar to the one below with Facebook IDs of users who like the post.

Step 7c:Extracting Facebook IDs of users who share a post

Finally, we will get the Facebook IDs of the users who shared the post. The procedure is similar – we just need to swap “likes” and “sharedposts” and input the following URL in your browser.

https://graph.facebook.com/204153042939851_761338343887982/sharedposts?limit=2000&access_token=EAACEdE (Truncated)

You will get the following window which shows the sharer’s Facebook user ID and even who he/she shared with.

Extracting Facebook Reactions for each post into Excel

Now I will show you how to extract Facebook reactions for each post from your Facebook page or that of your competitor into Excel. The data will give you insights into how people react to different content in your industry and help you better tailor your own content.

Step 1:Sign up for a free account with Quintly

A free trial account will enable you to try all of the packages for 14 days without putting in a credit card. You will be able to export data into Excel/PowerPoint format, or alternatively you can sign up for a totally free account which allows you to analyse a maximum of 3 Facebook profiles and 1 month of data. However, the free account does not enable you to export data into Excel or PPT format – you can only export data from your free account into PDF/PNG/JPG format.

Step 2:Add a profile

After creating an account with Quintly, go to “Manage Profiles” and then click “Add Profile”.

On the pop up window search for the Facebook page that you want to analyse and then click on “Add Selected Profiles” as shown below.

Step 3:Analysing Facebook reactions data

Click  “Facebook” on the dashboard menu, and then click  “Own Posts w/ Reactions Table”. You can also specify the date range (don’t forget that you are limited to within 30 days for a free account). Once you have done this, export the results into either Excel, CSV, JPG, PNG, PDF or PPTX for further analysis.

Below is an example of exported data with each post’s reactions data in Excel.

Conclusion

Knowing the strengths and weaknesses of your competitor’s Facebook page is vital for optimising your own Facebook fan page.

I highly recommend you to try Quintly as your Professional Social Media Analytics tool. It provides in-depth analytics, competitive benchmarking, graphs and reports, not only for Facebook but across all Social Media platforms such as Twitter, LinkedIn, Google+, Pinterest and Instagram.

They have a free 14-day trial which doesn’t require you to put a credit card. Give it a try NOW before the offers expire.

You might also be interested in:

• How to download Facebook videos and photos
• How to download your competitor’s tweets and followers to Excel

If you have any questions feel free to comment below.