Personal blog written from scratch using Node.js, Bootstrap, and MySQL. https://jrtechs.net
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

278 lines
8.0 KiB

4 years ago
4 years ago
4 years ago
4 years ago
4 years ago
4 years ago
4 years ago
4 years ago
4 years ago
4 years ago
4 years ago
4 years ago
  1. # The Premis
  2. Last week I started hosting my own git-forge to track sync all of my
  3. git projects. Between school, open-source communities, and personal
  4. projects, I have accumulated a dubious amount of git projects.
  5. Although most of my content gets hosted on Github, I also had a fair
  6. quantity of local projects, stuff on scattered GitLab instances, and
  7. other places. I decided to use Gitea to mirror all of my repositories
  8. and keep them in a central location that I can quickly search for
  9. them.
  10. ![High Level Flow Diagram](media/gitea/giteaFlow.svg)
  11. For simplicity, I decided to host my Gitea instance on a DigitalOcean
  12. droplet using docker and add SSL encryption using a reverse Nginx
  13. proxy using a let's encrypt.
  14. # Installation
  15. The first step was to set up a VM in the cloud. I used a base Debian
  16. server on Digital Ocean. I tend to use Debian for servers because it
  17. is stable and has an extensive collection of packages.
  18. ## Docker and Docker Compose
  19. I included the commands necessary to install docker bellow. These
  20. instructions may get stale, but the gist is going to remain the same
  21. for the foreseeable future. We are adding the docker's certificates to
  22. our package manager and then installing it.
  23. ```bash
  24. apt update
  25. apt upgrade
  26. apt install apt-transport-https ca-certificates curl software-properties-common gnupg2
  27. curl -fsSL https://download.docker.com/linux/debian/gpg | sudo apt-key add -
  28. add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/debian $(lsb_release -cs) stable"
  29. apt update
  30. apt install docker-ce
  31. ```
  32. Install Docker-Compose
  33. ```bash
  34. curl -L https://github.com/docker/compose/releases/download/1.25.0-rc2/docker-compose-`uname -s`-`uname -m` -o /usr/local/bin/docker-compose
  35. chmod +x /usr/local/bin/docker-compose
  36. ```
  37. ## Gitea over Docker Compose
  38. For simplicity, I am running Gitea using docker-compose because it
  39. makes it portable between systems and makes for a straightforward
  40. install. You can find the official Gitea instructions on this
  41. [here](https://docs.gitea.io/en-us/install-with-docker/).
  42. The just of the running entails creating a file called
  43. docker-compose.yml with the following contents:
  44. ```yml
  45. version: "2"
  46. networks:
  47. gitea:
  48. external: false
  49. services:
  50. server:
  51. image: gitea/gitea:latest
  52. environment:
  53. - USER_UID=1000
  54. - USER_GID=1000
  55. restart: always
  56. networks:
  57. - gitea
  58. volumes:
  59. - ./gitea:/data
  60. - /etc/timezone:/etc/timezone:ro
  61. - /etc/localtime:/etc/localtime:ro
  62. ports:
  63. - "3000:3000"
  64. - "222:22"
  65. ```
  66. Once we have our docker-compose.yml saved, we can launch it when we
  67. are in the same directory as it using docker-compose. After this
  68. launches we now have our Gitea server running on port 3000
  69. ```bash
  70. docker-compose build
  71. docker-compose up
  72. ```
  73. ## Nginx and Let's Encrypt
  74. Although our server is running publicly on port 3000, we don't want to
  75. use it yet because, by default, it is running on HTTP, which is not
  76. encrypted. HTTP makes it possible for people sitting in the middle of
  77. your connection to listen to your traffic and see all the passwords
  78. and data that you send to your Gitea server. We are going to be using
  79. a [Nginx](https://nginx.org/) reverse proxy with [Let's
  80. Encrypt](https://letsencrypt.org/) to add HTTPS encryption.
  81. Installing Nginx is easy because it is in most Linux package managers.
  82. ```bash
  83. apt-get install nginx
  84. ```
  85. Next, we tell Systemd to start Nginx on startup.
  86. ```bash
  87. systemctl enable nginx
  88. ```
  89. Next we modify the Nginx config file to add a reverse proxy. This will
  90. forward all traffic on git.jrtechs.net to the localhosts's port 3000.
  91. ```bash
  92. vim /etc/nginx/sites-available/default
  93. ```
  94. Add this content to the very bottom of the default config file
  95. changing "git.jrtechs.net."
  96. ```bash
  97. server
  98. {
  99. listen 80;
  100. server_name git.jrtechs.net;
  101. location / {
  102. proxy_pass http://localhost:3000;
  103. proxy_http_version 1.1;
  104. proxy_set_header Upgrade $http_upgrade;
  105. proxy_set_header Connection 'upgrade';
  106. proxy_set_header Host $host;
  107. proxy_cache_bypass $http_upgrade;
  108. }
  109. }
  110. ```
  111. This command tests the Nginx file you just modified to make sure it is
  112. syntactically correct.
  113. ```bash
  114. nginx -t
  115. ```
  116. Next, we reload Nginx starting the reverse Nginx proxy we just
  117. created.
  118. ```bash
  119. /etc/init.d/nginx reload
  120. ```
  121. Now that Nginx got configured, we can set-up Certbot for encryption.
  122. ```bash
  123. sudo add-apt-repository ppa:certbot/certbot
  124. sudo apt install python-certbot-nginx
  125. ```
  126. This step is beautiful because it has the Let's Encrypt Certbot modify
  127. the Nginx configuration files to make it work over https. When
  128. prompted, select the option that redirects all HTTP traffic to https.
  129. ```bash
  130. systemctl stop nginx
  131. certbot --authenticator standalone --installer nginx -d git.jrtechs.net
  132. systemctl start nginx
  133. ```
  134. Now you should be able to access your Gitea instance using your domain
  135. name. By default, the first user that logs into the Gitea instance
  136. becomes an administrator.
  137. # Results
  138. Now you have a private git-forge that you can access anywhere in the
  139. world over HTTPS and quickly search and display all your git profiles.
  140. ![](media/gitea/teaSearch.png)
  141. The profile for each user you create on Gitea looks shockingly similar
  142. to Github's.
  143. ![](media/gitea/profile.png)
  144. One beautiful thing about Gitea is that you can easily import and
  145. mirror git repositories from other sources.
  146. ![](media/gitea/migration.png)
  147. # Transferring Github Projects to Gitea Instance
  148. Although it is possible to import every single repository you have in
  149. Github manually, this quickly becomes a nuisance if you have 60+
  150. repositories on Github like me.
  151. I found and modified a python script that imports all your Github
  152. repositories into Gitea. What is particularly dandy about this script
  153. is that it uses the Github API to get both your public and private
  154. repositories. If the repository is private on Github, it gets
  155. transferred over as a private repository into Gitea. This script also
  156. mirrors in every repo from your organizations. I posted this script on
  157. my
  158. [Github](https://github.com/jrtechs/dot_files/tree/master/docker/gitea).
  159. To run this script, you need to generate API tokens for both Github
  160. and Gitea. Instructions for the Gitea API tokens are found
  161. [here](https://docs.gitea.io/en-us/api-usage/), and the Github token
  162. can be found
  163. [here](https://help.github.com/en/github/authenticating-to-github/creating-a-personal-access-token-for-the-command-line).
  164. The generated tokens get placed in ".gitea-api" and ".github-token"
  165. ```python
  166. #!/usr/bin/env python -B
  167. from github import Github # https://github.com/PyGithub/PyGithub
  168. import requests
  169. import json
  170. import sys
  171. import os
  172. gitea_url = "http://127.0.0.1:3000/api/v1"
  173. # generage gitea token https://docs.gitea.io/en-us/api-usage/
  174. gitea_token = open(os.path.expanduser("~/.gitea-api")).read().strip()
  175. session = requests.Session() # Gitea
  176. session.headers.update({
  177. "Content-type" : "application/json",
  178. "Authorization" : "token {0}".format(gitea_token),
  179. })
  180. r = session.get("{0}/user".format(gitea_url))
  181. if r.status_code != 200:
  182. print("Cannot get user details", file=sys.stderr)
  183. exit(1)
  184. gitea_uid = json.loads(r.text)["id"]
  185. github_username = "jrtechs"
  186. github_token = open(os.path.expanduser("~/.github-token")).read().strip()
  187. gh = Github(github_token)
  188. for repo in gh.get_user().get_repos():
  189. m = {
  190. "repo_name" : repo.full_name.replace("/", "-"),
  191. "description" : repo.description or "not really known",
  192. "clone_addr" : repo.clone_url,
  193. "mirror" : True,
  194. "private" : repo.private,
  195. "uid" : gitea_uid,
  196. }
  197. if repo.private:
  198. m["auth_username"] = github_username
  199. m["auth_password"] = "{0}".format(github_token)
  200. jsonstring = json.dumps(m)
  201. r = session.post("{0}/repos/migrate".format(gitea_url), data=jsonstring)
  202. if r.status_code != 201: # if not CREATED
  203. if r.status_code == 409: # repository exists
  204. continue
  205. print(r.status_code, r.text, jsonstring)
  206. ```
  207. Sit back and have some tea because this script can take a hot second
  208. to run.