Personal blog written from scratch using Node.js, Bootstrap, and MySQL. https://jrtechs.net
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

83 lines
4.9 KiB

  1. <iframe width="100%" height="315" src="https://www.youtube.com/embed/DoDaHmyIPvQ" frameborder="0" allow="autoplay; encrypted-media" allowfullscreen></iframe>
  2. ### Links
  3. - [GitHub](https://github.com/jrtechs/SteamFriendsGraph)
  4. - [Live Site](http://steam.jrtechs.net/)
  5. ### Project Description
  6. This project utilizes the steam API and graph databases to create friend graphs for clients in a
  7. web browser. Currently there are two types of graphs available:
  8. - Friends of Friends Graph:
  9. This graph displays all the steam friends of a single person and all their friend’s friends.
  10. - Common Friends Graph:
  11. This graph will only display your friends; however, it will draw edges between your friends’ if
  12. they are friends with each other.
  13. ### Motivation
  14. While learning about graph databases I thought that it would be awesome to create a massive graph
  15. database with steam friends in it. After a quick google search, I realized that some other people have
  16. done similar things in python, however, nobody has ever made a live website do this. Using my prior
  17. knowledge of the steam API and new knowledge of gremlin, Hadoop and HBase, I figured that this would
  18. be a great weekend project. Although I got most of the functionality done in the first weekend, I
  19. ended up working on it for 3 weekends hammering out all the edges. After ample work, I am satisfied with
  20. the appearance and functionality of the website, but, I am looking forward to making some improvements
  21. in the future.
  22. ![Lucid chart diagram](media/steam/diagram.png)
  23. ### Technical Details
  24. At the core of this project there is a java backend and a JavaScript frontend. I needed a backend for
  25. this project because you cannot distribute your steam API key, plus, this allows me to cache everything
  26. in a local graph database. The second time a player is queried, it is extremely fast since all the steam
  27. friends are cached in a local
  28. [gremlin](http://tinkerpop.apache.org/docs/3.0.1-incubating/) server.
  29. The only time-consuming operation that the backend must carry out is querying the steam API; querying the
  30. gremlin server is relatively fast.
  31. On the front end I used [Sigma-JS](http://sigmajs.org/) with a web socket to render the steam graph.
  32. Since creating a new graph on the back end can take up to 5 minutes (if the player has not been indexed),
  33. I continuously send new nodes to
  34. the client as they arrive from the steam API. This prevents the client from staring at a loading bar
  35. for 5 minutes; plus, it makes great time lapse videos. The only drawback to this is that you cannot
  36. start applying a force to shake the graph until all the nodes are added to the graph.
  37. ![Steam friends graph](media/steam/jrtechs1.png)
  38. ### Uses of Friends Graph Data Base
  39. There are tons of [academic papers](http://infolab.stanford.edu/~ullman/mmds/ch10.pdf)
  40. written on the usages of friends’ graphs. What you may be able to do with friends
  41. graph may shock you. A popular use of friends’ graphs is to view friends’ clusters. Facebook may be able
  42. to identify someone's family, friends, and work groups by simply using their friend’s information. When looking
  43. at steam friends graphs I could identify some people's high school and college friend’s groups. Another
  44. famous example would be for friends’ identification; If a ton of people in your friends list are also
  45. friends with a person, it is likely that they would also know that person. This is how
  46. websites like Snapchat, Instagram, and Facebook recommend people for you to follow. Similarly, you can
  47. use this to identify people’s best friends. If two people share a ton of friends, it is
  48. more likely that they will be good friends vs the person which they only share one friend with.
  49. Simple analysis of graphs like this may seem benign. However, there are tremendous implications when it
  50. comes to privacy and advertising. If websites chose to, they can harvest a ton of data and start
  51. recommending you products based on not just what you like, but what your friends also like. Steam
  52. already does this to a certain extent when they recommend you games that are popular among friends.
  53. ![Steam friends graph](media/steam/ben2.png)
  54. ### Hosting the Project
  55. Hosting the front end of this project is easy, it is just thrown on an apache server. The backend
  56. is trickier since it needs to run both a java app and a gremlin server. Currently I am hosting this
  57. project on an Ubuntu VM with 4gb ram. At idle this project only takes 1gb of ram, however, under heavy
  58. loads it uses around 2 GB of ram and saturates my CPU.
  59. ![Steam friends graph](media/steam/jrtechs2.png)
  60. ### Future Plans for the Project
  61. In the future I would love to add more graphs to this website. Instead of just displaying graphs it would
  62. also be cool to create pages which tries to identify friends’ groups, people you may know, and best
  63. friends. There is a ton of game information available on the steam API, it might be neat to make
  64. graphs based on friends who play the same games.