How far are you from St4ck?
Introduction
I have a deep passion for gaming, particularly in the online games. While waiting in the queue during PvP games, I developed a unique routine where I delve into the Steam profiles of both teammates and opponents to check players country.
I find myself navigating through the friends lists of the profiles I encounter, progressively delving deeper. Eventually, the process leads me to a player known as St4ck, who boasts the highest Steam level achievement.
On a one Sunday morning, I decided to turn this habit into a mini project by automating the entire exploration process and plot graph for fun. I am here to share the journey and insights gained through this endeavor.
Data collecting process
To proceed in locating St4ck within the landscape of Steam profiles, our strategy is inputting a link of a Steam profile and systematically navigating through friends of the profile and friends of friends and go on. The principle is that players with higher levels have a higher probability of being connected with St4ck. Consequently, our goal lies on identifying and prioritizing higher-leveled friends. There is 2 approaches for that.
- Using Steam API
- Data Scraping
Problem with Steam API
Valve provides these APIs so website developers can use data from Steam. They allow developers to query Steam for information that they can present on their own sites. But there is a problem with using Steam API on our case.
The querying and integration capabilities of the Steam API is very limited. For instance, the GetFriendList could have been a potential solution:
http://api.steampowered.com/ISteamUser/GetFriendList/v0001/?key=STEAM_API_KEY&steamid=76561198023414915
Indeed, the GetFriendList is designed to retrieve the friend list of a Steam user. However, the result presents an data of friends that not ordered by Steam profile level. This becomes a limitation, particularly for our specific objective of pinpointing high-leveled friends.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
{
"friendslist": {
"friends": [
{
"steamid": "76561197960272945",
"relationship": "friend",
"friend_since": 1447427037
},
{
"steamid": "76561197960331910",
"relationship": "friend",
"friend_since": 1454512911
},
{
"steamid": "76561197960333443",
"relationship": "friend",
"friend_since": 1550002979
},
...
{
"steamid": "76561198003107402",
"relationship": "friend",
"friend_since": 1447833212
}
]
}
}
Data Scraping
To accomplish this task, we use data scraping technique. Since on a Steam profile, top six friends with the highest level are displayed. To extract this information, we scrape data with the help of BeautifulSoup. The following function is designed to identify all HTML elements with div tags that characterized by the class friendBlock which represents a friend on the Steam profile. Then the function returns a list of user profile links.
1
2
3
4
5
6
7
8
9
10
11
12
13
async def crawl_friends_from_id(self, url) -> list:
async with aiohttp.ClientSession() as session:
async with session.get(url) as response:
html = await response.text()
soup = BeautifulSoup(html, "html.parser")
friends = []
elements = soup.find_all("div", class_="friendBlock")
for element in elements:
user_id = element.find("a", class_="friendBlockLinkOverlay")["href"]
friends.append(user_id)
return friends
Breadth First Search (BFS) for Tree
Here, Breadth First Search algorithm come in handy. Breadth-first search is an algorithm for searching a tree data structure for a node that satisfies a given property. It starts at the tree root and explores all nodes at the present depth prior to moving on to the nodes at the next depth level. In our scenario, the root node is the Steam profile provided as input, and our quest revolves around searching for the node labeled St4ck.
Following function uses Breadth-First Search (BFS) principle to discover the Steam profile labeled St4ck. The algorithm begins by initiating a queue with the input Steam profile URL and its friends. As the exploration unfolds, the function systematically traverses through the friends, updating a visited list to ensure nodes are not revisited. For each friend, it extends the queue with their friends and appends the visited list. The search is constrained to a limit of 100 friends, preventing an exhaustive exploration. If St4ck is found within the queue, the function signals success; otherwise, it concludes with a message indicating the absence of St4ck. Throughout this process, the function builds a DataFrame, recording the relationships between each profile and their respective friends for plotting graph using NetworkX.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
async def find_st4ck(self, steam_url):
friends = await self.scraper.crawl_friends_from_id(steam_url)
visited = []
queue = [steam_url]
queue.extend(friends)
# Convert friends list to pandas Series
friends_series = pd.Series(friends)
self.df_ops.add_data_to_df(
self.df,
[
steam_url.split("/")[-1],
friends_series.map(lambda x: x.split("/")[-1]).tolist(),
],
)
for friend in queue:
if "https://steamcommunity.com/id/St4ck" in queue:
print("St4ck found!")
break
# Limit the number of friends to 100
if len(visited) > 100:
print("St4ck not found!")
break
if friend not in visited:
friend_of_friends = await self.scraper.crawl_friends_from_id(friend)
friend_of_friends_series = pd.Series(friend_of_friends)
queue.extend(friend_of_friends)
visited.append(friend)
self.df_ops.add_data_to_df(
self.df,
[
friend.split("/")[-1],
friend_of_friends_series.map(
lambda x: x.split("/")[-1]
).tolist(),
],
)
queue.pop(0)
self.df.to_csv("./output/data.csv", index=False)
Plotting graph with NetworkX
Following code utilizes the NetworkX library to create a network graph visualizing relationships extracted from the CSV file containing Steam profile data. The data is read into a Pandas DataFrame, and a directed graph (G) is constructed with nodes representing Steam profile IDs and edges representing connections between them. Subsequently, the graph is enhanced by assigning degrees to nodes, adjusting node sizes based on their degrees, and highlighting the target profile, ‘St4ck,’ with increased node size. The graph’s aesthetics are further refined by applying a color palette to represent node sizes. The resulting visualization showcases the interconnectedness of Steam profiles and their proximity to St4ck.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
import networkx as nx
import pandas as pd
from bokeh.io import show, save
from bokeh.models import Range1d, Circle, MultiLine
from bokeh.plotting import figure
from bokeh.plotting import from_networkx
from bokeh.palettes import Blues8
from bokeh.transform import linear_cmap
from includes.data_scrape import Scraper
from includes.df_ops import DataframeOperations
import asyncio
from data_collect import Finder
input = input("Enter steam profile link: ")
async def get_data():
finder = Finder(Scraper(), DataframeOperations())
await finder.find_st4ck(input)
asyncio.run(get_data())
df_ops = DataframeOperations()
user_data = pd.read_csv("./output/data.csv")
# Read the data from the CSV file
df = df_ops.create_source_target_df(user_data)
G = nx.Graph()
G.add_nodes_from(user_data["id"].tolist())
G.add_edges_from(list(df.to_records(index=False)))
# some bug fix according to stackoverflow
mapping = dict((n, i) for i, n in enumerate(G.nodes))
H = nx.relabel_nodes(G, mapping)
# St4ck's key
st4ck_key = mapping["St4ck"]
# Degree
degrees = dict(nx.degree(H))
nx.set_node_attributes(H, name="degree", values=degrees)
# Adjust node size
number_to_adjust_by = 5
adjusted_node_size = dict(
[(node, degree + number_to_adjust_by) for node, degree in nx.degree(H)]
)
# Adjust the size of St4ck's node & one's node
adjusted_node_size[0] += 15
adjusted_node_size[st4ck_key] += 15
nx.set_node_attributes(H, name="adjusted_node_size", values=adjusted_node_size)
size_by_this_attribute = "adjusted_node_size"
color_by_this_attribute = "adjusted_node_size"
color_palette = Blues8
# id
id = dict((i, n) for i, n in enumerate(mapping))
nx.set_node_attributes(H, name="id", values=id)
title = "How far are you from St4ck?"
# Tooltip
HOVER_TOOLTIPS = [
("id", "@id"),
("degree", "@degree"),
]
# Create a plot — set dimensions, toolbar, and title
plot = figure(
tooltips=HOVER_TOOLTIPS,
tools="pan,wheel_zoom,save,reset",
active_scroll="wheel_zoom",
x_range=Range1d(-10.1, 10.1),
y_range=Range1d(-10.1, 10.1),
title=title,
)
# Create a network graph object
network_graph = from_networkx(H, nx.spring_layout, scale=10, center=(0, 0))
# Set node sizes and colors according to node degree (color as spectrum of color palette)
minimum_value_color = min(
network_graph.node_renderer.data_source.data[color_by_this_attribute]
)
maximum_value_color = max(
network_graph.node_renderer.data_source.data[color_by_this_attribute]
)
network_graph.node_renderer.glyph = Circle(
size=size_by_this_attribute,
fill_color=linear_cmap(
color_by_this_attribute, color_palette, minimum_value_color, maximum_value_color
),
)
# Set edge opacity and width
network_graph.edge_renderer.glyph = MultiLine(line_alpha=0.5, line_width=1)
plot.renderers.append(network_graph)
show(plot)
save(plot, filename=f"{title}.html")
Result
You can find full source code in this repository.