Facebook gives its users the possibility to restrict the visibility of their friend list. It is known that by exploiting the “mutual friends” function, is possible to bypass the visibility restrictions under certain conditions. We will present a way and a tool to automate the procedure in order to maximize the number of hidden friends that can be found.
Furthermore, using the same tool we will see how to build a network graph of the relationships among the friends found, in order to represent the friendships between them.
We will also see how it is possible to plot the graph using the tool Gephi and propose a way to identify sub-communities within the graph.
Finally, we will apply the procedure to a practical example scenario: we’ll retrieve some of Mark Zuckerberg’s hidden friends, plot them on a graph, identify sub-communities and export the final graph in a format that can be explored with a browser: here is the final result.
Please beware that Facebook Terms of Services prohibit scraping without permission: therefore you should not use any scraping tool without proper written authorization. All the information provided in this and related articles are for educational purposes only. The authors or @Mediaservice.net are in no way responsible for any misuse of the information or the code provided.
Reconstructing a hidden friend list
Facebook lets its users decide whether to display their own friends on their profile page, as well as to whom. In the settings page it also displays the following warning:
Remember, your friends control who can see their friendships on their own Timelines. If people can see your friendship on another timeline, they’ll be able to see it in News Feed, search and other places on Facebook. If you set this to Only me, only you will be able to see your full friends list on your timeline. Other people will see only mutual friends.
This is because Facebook also offers the “Mutual Friends” function, which shows friends that two accounts have in common, provided that at least one of them has their friend list visible to the current user.
This issue has been known for years and, despite the recently increased attention on users’ privacy declared by Facebook, it is not considered a concern by the company.
The functionality is available at the URL https://www.facebook.com/browse/mutual_friends/?uid=ID1&node=ID2, where ID1 and ID2 are the numerical ids of the two Facebook accounts. Later we will learn how to find any user’s Facebook ID.
Please beware that results returned by Facebook are neither complete nor consistent. Calling the same function from different locations or accounts might yield different results.
Thanks to the mutual friends functionality it is possible to reconstruct a good portion of a target user’s friend list: given an account that is known to have their friend list visible to the current user and at least one mutual friend with the target user (called pivot), starting from their common friends, it is possible to iteratively apply the mutual friends functionality, using all new friends found at each step as an input to another iteration, until the function’s fixed point is reached (i.e.: all friends have been pivoted on and no new friends are found).
In order to show the potential of this method, I have written a simple Python tool that executes the above described procedure automatically. Please beware that Facebook Terms of Services prohibit scraping without permission: therefore you should not use the tool without proper authorization.
The tool is called ffff and can be downloaded from Github. It is written in Python 3 and uses the selenium library and Mozilla geckodriver in order to open up a browser and recursively visit the mutual friends pages. While it does so, it keeps track of the relationships found in order to build a network graph (more on this later). In order to not be detected as scraping by Facebook, it pauses for a random number of seconds between each request; this interval can be reduced, at the risk of being blocked by the platform. At the end of the procedure, it yields a csv file with the list of friends found and a gexf file with the related graph.
Build a network graph of friends
While iteratively applying the mutual friends functions to all target’s friends, we can also keep track of the relationships existing among the target friends themselves: if we find that target user T and pivot user P have a common friend C, we know not only that C is friends with our target, but also that P and C are friends.
Iterating this procedure, we are able to build a network graph of the Facebook friendships found along the way, where nodes are users and edges are existing friendship relationships.
One could think of assigning a weight to an edge proportionally to how many times the relationship has been observed. The tool ffff allows you to do so (option -w), but in our scenario it is not meaningful information, also considering that it is strongly influenced by the privacy settings of each user (the more users in a community have a public friend list, the more often relationships in that community will be observed).
Once we have a graph, we can run different algorithms on it in order to try to identify existing communities within the network. In a practical scenario, for example, this might help to identify the community of work colleagues, family, closest friends, etc.
Pratical Example: Zuck’s friends communities
As an example, we will try to fetch some of Mark Zuckerberg’s Facebook friends, who is certainly aware of this possibility.
Please note that you can obviously build a relationship graph of a user with a public friend list: in that case, you wouldn’t need a pivot user; just use his/her Facebook ID both as the target and the pivot.
Find the target’s Facebook ID
We need to know Mark Zuckerberg’s Facebook ID. In order to find the numerical ID of a Facebook account there are several ways: a simple Google search returns many online services that offer to do it for you. Our favorite way, though, is to do it autonomously: open the target’s Facebook profile page (in our example, Mark Zuckerberg’s) in a browser and inspect the HTML source code (Ctrl+U). In the code, search (Ctrl+F) for the pattern fb://profile/; there should be two matches, both followed by the same integer value: this number is the Facebook account id. In our example, since Mark Zuckerberg is one of the first users of the platform, the number is very small (4); expect to have much bigger numbers for regular users.
A quicker alternative to the above method is shown in the next paragraph.
Finding pivot users
We also need to find at least one account that has a public friend list (or a friend list visible to our account) and at least one mutual friend with our target.
Generally, a good way to do this would be to look at users who have interacted with the target, for example by commenting or reacting to his content. Also notice that in order to maximize the number of friends found, it is better to have many pivot users belonging to different communities (e.g.: work colleagues, family members, sport friends, etc.). The more, the better.
I have written an auxiliary script that automatically finds users who have interacted with a given target profile; it’s called fint.py and is included in the ffff’s project’s repository. It works in the same way ffff does, therefore I will just give a quick example usage (use the switch -h to show all available options). Refer to the section “Find hidden friends with ffff.py” or the project’s page on Github for more general information.
python fint.py -fu email@example.com -fp fbpassword -d geckodriver.exe -t 4 -ls 10 -lp 10 -lc 5 -lr 5
The above command will scrape authors of at most 100 comments (-lc) and of at most 1000 reactions (-lr), on a maximum of 10 stories (-ls) and 10 photos (-lp) of target user 4 (-t). The output is a file called 4-pivots.txt that contains a list of Facebook ids that can be fed as an input to ffff (option -P).
Since our target is a public figure and somewhat special with regards to Facebook, and we are not really concerned with building a large portion of his friend list, we are content with a single pivot friend. To obtain it we tried a different way: a simple search returns an article that lists some of M.Z. known friends. The fourth on the list, Brian Chesky, currently has a public friend list, which makes it a perfect candidate to be our pivot user.
We now need to find his Facebook ID. Above, we have already shown some easy methods to do so. If you are happy with using one of them, you can skip to the following section. Otherwise, I will present a different method to retrieve a user ID that might look complicated at first, but it’s probably the quickest when you are manually browsing the Facebook website.
If we visit Brian Chesky’s friend list page we see the URL looks like the following (this is true for any page reachable from the profile main menu, such as “about”, “photos”, etc.) :
As you might see, the URL contains a parameter “lst” (underlined) which holds three different integer values separated by the characters %3A (which is simply the URL encoding of character “:”). The second integer is the user ID, in this case 558259929. You can (and should!) double check the id by visiting https://www.facebook.com/558259929 and noticing it redirects to https://www.facebook.com/brianchesky
Find hidden friends with ffff.py
This article does not cover the installation instructions for the tool; some information can be found on the project’s page. In the examples shown in the following sections, the tool is run in a virtual environment on Windows. We have downloaded geckodriver.exe from the official mozilla project page (as described by the instructions on ffff project’s page) and placed it in the same directory as the python script.
We need a valid Facebook account to use for scraping. I suggest not using your own, as it might be blocked by the platform. The best option is probably to create a different account, and from that one create a test account. In the example we will pretend the email address for the account is “firstname.lastname@example.org” and the password “fbpassword”.
We now have all the information we need:
- our Facebook account email (option -fu): email@example.com
- our Facebook account password (option -fp): fbpassword
- our target account id (option -t): 4
- our pivot account id (option -p): 558259929
- if you have used fint.py to retrieve a list of possible pivots, you can use option -P to feed the list as an input
The option -q is used to hide the browser window.
python ffff.py -fu firstname.lastname@example.org -fp fbpassword -d geckodriver.exe -t 4 -p 558259929 -q
To see all available options please run python ffff.py -h
It is quite possible that the program will encounter problems after some time, such as reaching timeouts. You can interrupt the program with Ctrl+C. All results will be saved, together with a session file that can be resumed:
python ffff.py -fu email@example.com -fp fbpassword --resume session-4-2019050812320
After the program terminates, all friends are printed on screen and three different files are created (user ID and datetime will change):
The CSV file contains the list of friends found in the form id, name, profile URL. The .gexf file contains the representation of a graph ready to be open in Gephi (more on that later). Finally, the session file can be used to resume the program run at a later time, for example if any new pivot users are found.
Plot a community graph
We will use the software Gephi to plot a colourful graph of Mark’s friends, possibly somehow reflecting the structure of sub-communities existing within the network. In no way this is intended to be a guide to Gephi, nor the best way to perform community identification.
First of all, let’s open the .gexf file with Gephi. Initially, it will look like a messy, dark bunch of lines and circles.
From there, we can apply some layout algorithms that might be helpful in obtaining a greater insight into the graph seen as a community. Layouts are selectable from the bottom left window in the “Overview tab”. Each layout has different parameters: I suggest playing a bit with them to see their effect.
I have run OpenOrd first, followed by ForceAtlas 2. For the latter, I have set a “Scaling” value of 10. The result is not yet very appealing, but has a different shape.
Now we can try to partition the nodes into different communities by using their “Modularity” class, computable through the section “statistics” on the right of the screen. Here we can provide a value for the “resolution” parameter: the smaller value, the more partitions (i.e.: more communities).
Now we can assign different colors to each modularity class. On top left, in the “Appearance” tab, select “Nodes” > “Partition” and as attribute select “Modularity Class”. Clicking on “Palette…” you can configure the colors to be used. Clicking “Apply” all nodes will be colored based on their modularity class. If you are not satisfied with the communities found, you can go back to the modularity computation and select different parameters.
When you are happy with the graph, go to the “Preview” tab in the top menu and configure how you want your final graph to look like. Finally, you can export it in the format you wish.
If you want an interactive graph, there are a lot of great plugins you can explore that can export the graph in different formats. I like to use Sigma Exporter, by Scott Hale of the Oxford Institute.
You can see an example of our interactive graph at this link.
We have tried to find a portion of Mark Zuckerberg’s Facebook friends, despite his list being hidden. For the sake of example, we have used a single pivot user known to have friends in common with our target and a public friend list. Thanks to this we have found 335 unique, confirmed friends. From our experience, this is not an excellent result.
Furthermore, we were able to plot a community graph of the friendships found. We did not examine in depth this graph, but running the same algorithm on colleagues working for @Mediaservice.net gave some surprisingly accurate results.
In conclusion, despite Facebook privacy settings, it is possible to find a good portion of the friend list of most Facebook users; it is also possible to identify different communities within the friendship graph. In order to automate the procedure, a tool such as ffff can be used.