Cleanup hosts map

Fix config loading issue when servers don't resolve
Update README to include check information and code quality
2022-08-18 23:43:58 -04:00 · 2022-08-18 23:42:54 -04:00 · 2022-08-15 02:46:25 -04:00 · 2022-08-15 02:41:31 -04:00 · 2022-08-15 02:37:51 -04:00 · 2022-08-15 02:17:12 -04:00
22 changed files with 1263 additions and 355 deletions
--- a/.drone.yml
+++ b/.drone.yml
@ -3,6 +3,18 @@ name: default
 type: docker

 steps:
+  - name: test
+    image: golang:alpine
+    volumes:
+      - name: build
+        path: /build
+    commands:
+      - go mod download
+      - go install -mod=mod github.com/onsi/ginkgo/v2/ginkgo
+      - ginkgo --randomize-all --p --cover --coverprofile=cover.out .
+      - go tool cover -func=cover.out
+    environment:
+      CGO_ENABLED: '0'
  - name: build
    image: tystuyfzand/goc:latest
    volumes:
@ -16,7 +28,7 @@ steps:
    environment:
      GOOS: linux,windows,darwin
      GOARCH: 386,amd64,arm,arm64
-    depends_on: [ clone ]
+    depends_on: [ test ]
  - name: release
    image: plugins/gitea-release
    volumes:
@ -48,7 +60,7 @@ steps:
        from_secret: docker_password
      repo: registry.meow.tf/tyler/armbian-router
      registry: registry.meow.tf
-    depends_on: [ clone ]
+    depends_on: [ test ]
    when:
      event: tag
 volumes:
--- a/.gitignore
+++ b/.gitignore
@ -2,4 +2,6 @@
 userdata.csv
 dlrouter-apt.yaml
 *.yaml
-!dlrouter.yaml
+!dlrouter.yaml
+*.exe
+cover.out
--- a/2
+++ b/2
@ -1,4 +1,4 @@
-Copyright (c) 2022 Tyler Stuyfzand <admin@meow.tf>
+Copyright (c) 2022 Tyler Stuyfzand <admin@meow.tf>, Armbian Project

 Permission to use, copy, modify, and/or distribute this software for any purpose with or without fee is hereby granted, provided that the above copyright notice and this permission notice appear in all copies.

--- a/README.md
+++ b/README.md
@ -0,0 +1,145 @@
+Armbian Redirector
+==================
+
+This repository contains a redirect service for Armbian downloads, apt, etc.
+
+It uses multiple current technologies and best practices, including:
+
+- Go 1.19
+- Ginkgo v2 and Gomega testing framework
+- GeoIP + Distance routing
+- Server weighting, pooling (top x servers are served instead of a single one)
+- Health checks (HTTP, TLS)
+
+Code Quality
+------------
+
+The code quality isn't the greatest/top tier. Work is being done towards cleaning it up and standardizing it, writing tests, etc.
+
+All contributions are welcome, see the `check_test.go` file for example tests.
+
+Checks
+------
+
+The supported checks are HTTP and TLS.
+
+### HTTP
+
+Verifies server accessibility via HTTP. If the server returns a forced redirect to an `https://` url, it is considered to be https-only.
+
+If the server responds on the `https` url with a forced `http` redirect, it will be marked down due to misconfiguration. Requests should never downgrade.
+
+### TLS
+
+Certificate checking to ensure no servers are used which have invalid/expired certificates. This check is written to use the Mozilla ca certificate list, loaded on start/config load, to verify roots.
+
+OS certificate trusts WERE being used to do this, however some issues with the date validation (which could be user error) caused the move to the ca bundle, which could be considered more usable.
+
+Note: This downloads from github every startup/reload. This should be a reliable process, as long as Mozilla doesn't deprecate their repo. Their HG URL is super slow.
+
+Configuration
+-------------
+
+### Modes
+
+#### Redirect
+
+Standard redirect functionality
+
+#### Download Mapping
+
+Uses the `dl_map` configuration variable to enable mapping of paths to new paths.
+
+Think symlinks, but in a generated file.
+
+### Mirrors
+Mirror targets with trailing slash are placed in the yaml configuration file.
+
+### Example YAML
+```yaml
+# GeoIP Database Path
+geodb: GeoLite2-City.mmdb
+
+# Comment out to disable
+dl_map: userdata.csv
+
+# LRU Cache Size (in items)
+cacheSize: 1024
+
+# Server definition
+# Weights are just like nginx, where if it's > 1 it'll be chosen x out of x + total times
+# By default, the top 3 servers are used for choosing the best.
+# server = full url or host+path
+# weight = int
+# optional: latitude, longitude (float)
+# optional: protocols (list/array)
+servers:
+  - server: armbian.12z.eu/apt/
+  - server: armbian.chi.auroradev.org/apt/
+    weight: 15
+    latitude: 41.8879
+    longitude: -88.1995
+  # Example of a server with additional protocols (rsync)
+  # Useful for defining servers which could be used for rsync sources
+  - server: mirrors.dotsrc.org/armbian-apt/
+    weight: 15
+    protocols:
+      - rsync
+````
+
+## API
+
+`/status`
+
+Meant for a simple health check (nginx/etc can 502 or similar if down)
+
+`/reload`
+
+Flushes cache and reloads configuration and mapping. Requires reloadToken to be set in the configuration, and a matching token provided in `Authorization: Bearer TOKEN`
+
+`/mirrors`
+
+Shows all mirrors in the legacy (by region) format
+
+`/mirrors.json`
+
+Shows all mirrors in the new JSON format. Example:
+
+```json
+[
+  {
+    "available":true,
+    "host":"imola.armbian.com",
+    "path":"/apt/",
+    "latitude":46.0503,
+    "longitude":14.5046,
+    "weight":10,
+    "continent":"EU",
+    "lastChange":"2022-08-12T06:52:35.029565986Z"
+  }
+]
+```
+
+`/mirrors/{server}.svg`
+
+Magic SVG path to show badges based on server status, for use in dynamic mirror lists.
+
+`/dl_map`
+
+Shows json-encoded download mappings
+
+`/geoip`
+
+Shows GeoIP information for the requester
+
+`/region/REGIONCODE/PATH`
+
+Using this magic path will redirect to the desired region:
+
+* NA - North America
+* EU - Europe
+* AS - Asia
+
+`/metrics`
+
+Prometheus metrics endpoint. Metrics aren't considered private, thus are exposed to the public.
--- a/armbianmirror_suite_test.go
+++ b/armbianmirror_suite_test.go
@ -0,0 +1,13 @@
+package redirector
+
+import (
+	"testing"
+
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+)
+
+func TestArmbianMirror(t *testing.T) {
+	RegisterFailHandler(Fail)
+	RunSpecs(t, "ArmbianMirror Suite")
+}
--- a/check.go
+++ b/check.go
@ -0,0 +1,207 @@
+package redirector
+
+import (
+	"crypto/tls"
+	"crypto/x509"
+	"errors"
+	"fmt"
+	log "github.com/sirupsen/logrus"
+	"net"
+	"net/http"
+	"net/url"
+	"runtime"
+	"strings"
+	"time"
+)
+
+var (
+	ErrHttpsRedirect = errors.New("unexpected forced https redirect")
+	ErrHttpRedirect  = errors.New("unexpected redirect to insecure url")
+	ErrCertExpired   = errors.New("certificate is expired")
+)
+
+func (r *Redirector) checkHttp(scheme string) ServerCheck {
+	return func(server *Server, logFields log.Fields) (bool, error) {
+		return r.checkHttpScheme(server, scheme, logFields)
+	}
+}
+
+// checkHttp checks a URL for validity, and checks redirects
+func (r *Redirector) checkHttpScheme(server *Server, scheme string, logFields log.Fields) (bool, error) {
+	u := &url.URL{
+		Scheme: scheme,
+		Host:   server.Host,
+		Path:   server.Path,
+	}
+
+	req, err := http.NewRequest(http.MethodGet, u.String(), nil)
+
+	req.Header.Set("User-Agent", "ArmbianRouter/1.0 (Go "+runtime.Version()+")")
+
+	if err != nil {
+		return false, err
+	}
+
+	res, err := r.config.checkClient.Do(req)
+
+	if err != nil {
+		return false, err
+	}
+
+	logFields["responseCode"] = res.StatusCode
+
+	if res.StatusCode == http.StatusOK || res.StatusCode == http.StatusMovedPermanently || res.StatusCode == http.StatusFound || res.StatusCode == http.StatusNotFound {
+		if res.StatusCode == http.StatusMovedPermanently || res.StatusCode == http.StatusFound {
+			location := res.Header.Get("Location")
+
+			logFields["url"] = location
+
+			switch u.Scheme {
+			case "http":
+				res, err := r.checkRedirect(u.Scheme, location)
+
+				if !res || err != nil {
+					// If we don't support http, we remove it from supported protocols
+					server.Protocols = server.Protocols.Remove("http")
+				} else {
+					// Otherwise, we verify https support
+					r.checkProtocol(server, "https")
+				}
+			case "https":
+				// We don't want to allow downgrading, so this is an error.
+				return r.checkRedirect(u.Scheme, location)
+			}
+		}
+
+		return true, nil
+	}
+
+	logFields["cause"] = fmt.Sprintf("Unexpected http status %d", res.StatusCode)
+
+	return false, nil
+}
+
+func (r *Redirector) checkProtocol(server *Server, scheme string) {
+	res, err := r.checkHttpScheme(server, scheme, log.Fields{})
+
+	if !res || err != nil {
+		return
+	}
+
+	if !server.Protocols.Contains(scheme) {
+		server.Protocols = server.Protocols.Append(scheme)
+	}
+}
+
+// checkRedirect parses a location header response and checks the scheme
+func (r *Redirector) checkRedirect(originatingScheme, locationHeader string) (bool, error) {
+	newUrl, err := url.Parse(locationHeader)
+
+	if err != nil {
+		return false, err
+	}
+
+	if newUrl.Scheme == "https" {
+		return false, ErrHttpsRedirect
+	} else if originatingScheme == "https" && newUrl.Scheme == "http" {
+		return false, ErrHttpRedirect
+	}
+
+	return true, nil
+}
+
+// checkTLS checks tls certificates from a host, ensures they're valid, and not expired.
+func (r *Redirector) checkTLS(server *Server, logFields log.Fields) (bool, error) {
+	var host, port string
+	var err error
+
+	if strings.Contains(server.Host, ":") {
+		host, port, err = net.SplitHostPort(server.Host)
+
+		if err != nil {
+			return false, err
+		}
+	} else {
+		host = server.Host
+	}
+
+	log.WithFields(log.Fields{
+		"server": server.Host,
+		"host":   host,
+		"port":   port,
+	}).Info("Checking TLS server")
+
+	if port == "" {
+		port = "443"
+	}
+
+	conn, err := tls.Dial("tcp", host+":"+port, &tls.Config{
+		RootCAs: r.config.RootCAs,
+	})
+
+	if err != nil {
+		return false, err
+	}
+
+	defer conn.Close()
+
+	err = conn.VerifyHostname(server.Host)
+
+	if err != nil {
+		return false, err
+	}
+
+	now := time.Now()
+
+	state := conn.ConnectionState()
+
+	peerPool := x509.NewCertPool()
+
+	for _, intermediate := range state.PeerCertificates {
+		if !intermediate.IsCA {
+			continue
+		}
+
+		peerPool.AddCert(intermediate)
+	}
+
+	opts := x509.VerifyOptions{
+		Roots:         r.config.RootCAs,
+		Intermediates: peerPool,
+		CurrentTime:   time.Now(),
+	}
+
+	// We want only the leaf certificate, as this will verify up the chain for us.
+	cert := state.PeerCertificates[0]
+
+	if _, err := cert.Verify(opts); err != nil {
+		logFields["peerCert"] = cert.Subject.String()
+
+		if authErr, ok := err.(x509.UnknownAuthorityError); ok {
+			logFields["authCert"] = authErr.Cert.Subject.String()
+			logFields["ca"] = authErr.Cert.Issuer
+		}
+		return false, err
+	}
+
+	if now.Before(cert.NotBefore) || now.After(cert.NotAfter) {
+		logFields["peerCert"] = cert.Subject.String()
+		return false, err
+	}
+
+	for _, chain := range state.VerifiedChains {
+		for _, cert := range chain {
+			if now.Before(cert.NotBefore) || now.After(cert.NotAfter) {
+				logFields["cert"] = cert.Subject.String()
+				return false, ErrCertExpired
+			}
+		}
+	}
+
+	// If https is valid, append it
+	if !server.Protocols.Contains("https") {
+		server.Protocols = server.Protocols.Append("https")
+	}
+
+	return true, nil
+}
--- a/check_test.go
+++ b/check_test.go
@ -0,0 +1,197 @@
+package redirector
+
+import (
+	"crypto/rand"
+	"crypto/rsa"
+	"crypto/tls"
+	"crypto/x509"
+	"crypto/x509/pkix"
+	"encoding/pem"
+	"fmt"
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+	log "github.com/sirupsen/logrus"
+	"math/big"
+	"net"
+	"net/http"
+	"net/http/httptest"
+	"net/url"
+	"strings"
+	"time"
+)
+
+func genTestCerts(notBefore, notAfter time.Time) (*pem.Block, *pem.Block, error) {
+	// Create a Certificate Authority Cert
+	template := x509.Certificate{
+		SerialNumber:          big.NewInt(0),
+		Subject:               pkix.Name{CommonName: "localhost"},
+		SignatureAlgorithm:    x509.SHA256WithRSA,
+		NotBefore:             notBefore,
+		NotAfter:              notAfter,
+		BasicConstraintsValid: true,
+		IPAddresses:           []net.IP{net.ParseIP("127.0.0.1")},
+		KeyUsage:              x509.KeyUsageDigitalSignature | x509.KeyUsageKeyAgreement | x509.KeyUsageKeyEncipherment | x509.KeyUsageDataEncipherment,
+		ExtKeyUsage:           []x509.ExtKeyUsage{x509.ExtKeyUsageServerAuth, x509.ExtKeyUsageClientAuth},
+	}
+
+	// Create a Private Key
+	key, err := rsa.GenerateKey(rand.Reader, 4096)
+	if err != nil {
+		return nil, nil, fmt.Errorf("Could not generate rsa key - %s", err)
+	}
+
+	// Use CA Cert to sign a CSR and create a Public Cert
+	csr := &key.PublicKey
+	cert, err := x509.CreateCertificate(rand.Reader, &template, &template, csr, key)
+	if err != nil {
+		return nil, nil, fmt.Errorf("Could not generate certificate - %s", err)
+	}
+
+	// Convert keys into pem.Block
+	c := &pem.Block{Type: "CERTIFICATE", Bytes: cert}
+	k := &pem.Block{Type: "RSA PRIVATE KEY", Bytes: x509.MarshalPKCS1PrivateKey(key)}
+	return c, k, nil
+}
+
+var _ = Describe("Check suite", func() {
+	var (
+		httpServer *httptest.Server
+		server     *Server
+		handler    http.HandlerFunc
+		r          *Redirector
+	)
+	BeforeEach(func() {
+		httpServer = httptest.NewUnstartedServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+			handler(w, r)
+		}))
+		r = New(&Config{})
+		r.config.SetRootCAs(x509.NewCertPool())
+	})
+	AfterEach(func() {
+		httpServer.Close()
+	})
+	setupServer := func() {
+		u, err := url.Parse(httpServer.URL)
+
+		if err != nil {
+			panic(err)
+		}
+		server = &Server{
+			Host: u.Host,
+			Path: u.Path,
+		}
+	}
+
+	Context("HTTP Checks", func() {
+		BeforeEach(func() {
+			httpServer.Start()
+			setupServer()
+		})
+		It("Should successfully check for connectivity", func() {
+			handler = func(w http.ResponseWriter, r *http.Request) {
+				w.WriteHeader(http.StatusOK)
+			}
+
+			res, err := r.checkHttpScheme(server, "http", log.Fields{})
+
+			Expect(res).To(BeTrue())
+			Expect(err).To(BeNil())
+		})
+	})
+	Context("TLS Checks", func() {
+		var (
+			x509Cert *x509.Certificate
+		)
+		setupCerts := func(notBefore, notAfter time.Time) {
+			cert, key, err := genTestCerts(notBefore, notAfter)
+
+			if err != nil {
+				panic("Unable to generate test certs")
+			}
+
+			x509Cert, err = x509.ParseCertificate(cert.Bytes)
+
+			if err != nil {
+				panic("Unable to parse certificate from bytes: " + err.Error())
+			}
+
+			tlsPair, err := tls.X509KeyPair(pem.EncodeToMemory(cert), pem.EncodeToMemory(key))
+
+			if err != nil {
+				panic("Unable to load tls key pair: " + err.Error())
+			}
+
+			httpServer.TLS = &tls.Config{
+				Certificates: []tls.Certificate{tlsPair},
+			}
+
+			pool := x509.NewCertPool()
+
+			pool.AddCert(x509Cert)
+
+			r.config.SetRootCAs(pool)
+
+			httpServer.StartTLS()
+			setupServer()
+		}
+		Context("HTTPS Checks", func() {
+			BeforeEach(func() {
+				setupCerts(time.Now(), time.Now().Add(24*time.Hour))
+			})
+			It("Should return an error when redirected to http from https", func() {
+				handler = func(w http.ResponseWriter, r *http.Request) {
+					w.Header().Set("Location", strings.Replace(httpServer.URL, "https://", "http://", -1))
+					w.WriteHeader(http.StatusMovedPermanently)
+				}
+
+				logFields := log.Fields{}
+
+				res, err := r.checkHttpScheme(server, "https", logFields)
+
+				Expect(logFields["url"]).ToNot(BeEmpty())
+				Expect(logFields["url"]).ToNot(Equal(httpServer.URL))
+				Expect(err).To(Equal(ErrHttpRedirect))
+				Expect(res).To(BeFalse())
+			})
+		})
+		Context("CA Tests", func() {
+			BeforeEach(func() {
+				setupCerts(time.Now(), time.Now().Add(24*time.Hour))
+			})
+			It("Should fail due to invalid ca", func() {
+				r.config.SetRootCAs(x509.NewCertPool())
+
+				res, err := r.checkTLS(server, log.Fields{})
+
+				Expect(res).To(BeFalse())
+				Expect(err).ToNot(BeNil())
+			})
+			It("Should successfully validate certificates (valid ca, valid date/times, etc)", func() {
+				res, err := r.checkTLS(server, log.Fields{})
+
+				Expect(res).To(BeFalse())
+				Expect(err).ToNot(BeNil())
+			})
+		})
+		Context("Expiration tests", func() {
+			It("Should fail due to not yet valid certificate", func() {
+				setupCerts(time.Now().Add(5*time.Hour), time.Now().Add(10*time.Hour))
+
+				// Check TLS
+				res, err := r.checkTLS(server, log.Fields{})
+
+				Expect(res).To(BeFalse())
+				Expect(err).ToNot(BeNil())
+			})
+			It("Should fail due to expired certificate", func() {
+				setupCerts(time.Now().Add(-10*time.Hour), time.Now().Add(-5*time.Hour))
+
+				// Check TLS
+				res, err := r.checkTLS(server, log.Fields{})
+
+				Expect(res).To(BeFalse())
+				Expect(err).ToNot(BeNil())
+			})
+		})
+	})
+})
--- a/cmd/main.go
+++ b/cmd/main.go
@ -0,0 +1,112 @@
+package main
+
+import (
+	"flag"
+	"github.com/armbian/redirector"
+	"github.com/armbian/redirector/util"
+	log "github.com/sirupsen/logrus"
+	"github.com/spf13/viper"
+	"os"
+	"os/signal"
+	"syscall"
+)
+
+var (
+	configFlag = flag.String("config", "", "configuration file path")
+	flagDebug  = flag.Bool("debug", false, "Enable debug logging")
+)
+
+func main() {
+	flag.Parse()
+
+	if *flagDebug {
+		log.SetLevel(log.DebugLevel)
+	}
+
+	viper.SetDefault("bind", ":8080")
+	viper.SetDefault("cacheSize", 1024)
+	viper.SetDefault("topChoices", 3)
+	viper.SetDefault("reloadKey", redirector.RandomSequence(32))
+
+	viper.SetConfigName("dlrouter")        // name of config file (without extension)
+	viper.SetConfigType("yaml")            // REQUIRED if the config file does not have the extension in the name
+	viper.AddConfigPath("/etc/dlrouter/")  // path to look for the config file in
+	viper.AddConfigPath("$HOME/.dlrouter") // call multiple times to add many search paths
+	viper.AddConfigPath(".")               // optionally look for config in the working directory
+
+	if *configFlag != "" {
+		viper.SetConfigFile(*configFlag)
+	}
+
+	config := &redirector.Config{}
+
+	loadConfig := func(fatal bool) {
+		log.Info("Reading configuration")
+
+		// Bind reload to reading in the viper config, then deserializing
+		if err := viper.ReadInConfig(); err != nil {
+			log.WithError(err).Error("Unable to unmarshal configuration")
+
+			if fatal {
+				os.Exit(1)
+			}
+		}
+
+		log.Info("Unmarshalling configuration")
+
+		if err := viper.Unmarshal(config); err != nil {
+			log.WithError(err).Error("Unable to unmarshal configuration")
+
+			if fatal {
+				os.Exit(1)
+			}
+		}
+
+		log.Info("Updating root certificates")
+
+		certs, err := util.LoadCACerts()
+
+		if err != nil {
+			log.WithError(err).Error("Unable to load certificates")
+
+			if fatal {
+				os.Exit(1)
+			}
+		}
+
+		config.RootCAs = certs
+	}
+
+	config.ReloadFunc = func() {
+		loadConfig(false)
+	}
+
+	loadConfig(true)
+
+	redir := redirector.New(config)
+
+	// Because we have a bind address, we can start it without the return value.
+	redir.Start()
+
+	log.Info("Ready")
+
+	c := make(chan os.Signal)
+
+	signal.Notify(c, syscall.SIGKILL, syscall.SIGTERM, syscall.SIGHUP)
+
+	for {
+		sig := <-c
+
+		if sig != syscall.SIGHUP {
+			break
+		}
+
+		loadConfig(false)
+
+		err := redir.ReloadConfig()
+
+		if err != nil {
+			log.WithError(err).Warning("Did not reload configuration due to error")
+		}
+	}
+}
--- a/config.go
+++ b/config.go
@ -1,96 +1,175 @@
-package main
+package redirector

 import (
+	"crypto/tls"
+	"crypto/x509"
 	lru "github.com/hashicorp/golang-lru"
 	"github.com/oschwald/maxminddb-golang"
+	"github.com/pkg/errors"
 	"github.com/prometheus/client_golang/prometheus"
 	"github.com/prometheus/client_golang/prometheus/promauto"
 	log "github.com/sirupsen/logrus"
-	"github.com/spf13/viper"
 	"net"
+	"net/http"
 	"net/url"
 	"strings"
 	"sync"
+	"time"
 )

-func reloadConfig() {
-	log.Info("Loading configuration...")
+type Config struct {
+	BindAddress string         `mapstructure:"bind"`
+	GeoDBPath   string         `mapstructure:"geodb"`
+	MapFile     string         `mapstructure:"dl_map"`
+	CacheSize   int            `mapstructure:"cacheSize"`
+	TopChoices  int            `mapstructure:"topChoices"`
+	ReloadToken string         `mapstructure:"reloadToken"`
+	ServerList  []ServerConfig `mapstructure:"servers"`
+	ReloadFunc  func()
+	RootCAs     *x509.CertPool
+	checkClient *http.Client
+}

-	err := viper.ReadInConfig() // Find and read the config file
+// SetRootCAs sets the root ca files, and creates the http client for checks
+// This **MUST** be called before r.checkClient is used.
+func (c *Config) SetRootCAs(cas *x509.CertPool) {
+	c.RootCAs = cas

-	if err != nil { // Handle errors reading the config file
-		log.WithError(err).Fatalln("Unable to load config file")
+	t := &http.Transport{
+		TLSClientConfig: &tls.Config{
+			RootCAs: cas,
+		},
 	}

-	// db will never be reloaded.
-	if db == nil {
-		// Load maxmind database
-		db, err = maxminddb.Open(viper.GetString("geodb"))
+	c.checkClient = &http.Client{
+		Transport: t,
+		Timeout:   20 * time.Second,
+		CheckRedirect: func(req *http.Request, via []*http.Request) error {
+			return http.ErrUseLastResponse
+		},
+	}
+}

-		if err != nil {
-			log.WithError(err).Fatalln("Unable to open database")
+type ProtocolList []string
+
+func (p ProtocolList) Contains(value string) bool {
+	for _, val := range p {
+		if value == val {
+			return true
 		}
 	}

-	// Refresh server cache if size changed
-	if serverCache == nil {
-		serverCache, err = lru.New(viper.GetInt("cacheSize"))
-	} else {
-		serverCache.Resize(viper.GetInt("cacheSize"))
+	return false
+}
+
+func (p ProtocolList) Append(value string) ProtocolList {
+	return append(p, value)
+}
+
+func (p ProtocolList) Remove(value string) ProtocolList {
+	index := -1
+
+	for i, val := range p {
+		if value == val {
+			index = i
+			break
+		}
 	}

-	// Set top choice count
-	topChoices = viper.GetInt("topChoices")
+	if index == -1 {
+		return p
+	}
+
+	p[index] = p[len(p)-1]
+	return p[:len(p)-1]
+}
+
+func (r *Redirector) ReloadConfig() error {
+	log.Info("Loading configuration...")
+
+	var err error
+
+	// Load maxmind database
+	if r.db != nil {
+		err = r.db.Close()
+
+		if err != nil {
+			return errors.Wrap(err, "Unable to close database")
+		}
+	}
+
+	// db can be hot-reloaded if the file changed
+	r.db, err = maxminddb.Open(r.config.GeoDBPath)
+
+	if err != nil {
+		return errors.Wrap(err, "Unable to open database")
+	}
+
+	// Refresh server cache if size changed
+	if r.serverCache == nil {
+		r.serverCache, err = lru.New(r.config.CacheSize)
+	} else {
+		r.serverCache.Resize(r.config.CacheSize)
+	}
+
+	// Purge the cache to ensure we don't have any invalid servers in it
+	r.serverCache.Purge()

 	// Reload map file
-	reloadMap()
+	if err := r.reloadMap(); err != nil {
+		return errors.Wrap(err, "Unable to load map file")
+	}

 	// Reload server list
-	reloadServers()
+	if err := r.reloadServers(); err != nil {
+		return errors.Wrap(err, "Unable to load servers")
+	}

 	// Create mirror map
 	mirrors := make(map[string][]*Server)

-	for _, server := range servers {
+	for _, server := range r.servers {
 		mirrors[server.Continent] = append(mirrors[server.Continent], server)
 	}

 	mirrors["default"] = append(mirrors["NA"], mirrors["EU"]...)

-	regionMap = mirrors
+	r.regionMap = mirrors

 	hosts := make(map[string]*Server)

-	for _, server := range servers {
+	for _, server := range r.servers {
 		hosts[server.Host] = server
 	}

-	hostMap = hosts
+	r.hostMap = hosts

 	// Check top choices size
-	if topChoices > len(servers) {
-		topChoices = len(servers)
+	if r.config.TopChoices > len(r.servers) {
+		r.config.TopChoices = len(r.servers)
 	}

 	// Force check
-	go servers.Check()
+	go r.servers.Check(r.checks)
+
+	return nil
 }

-func reloadServers() {
-	var serverList []ServerConfig
-	viper.UnmarshalKey("servers", &serverList)
-
+func (r *Redirector) reloadServers() error {
+	log.WithField("count", len(r.config.ServerList)).Info("Loading servers")
 	var wg sync.WaitGroup

 	existing := make(map[string]int)

-	for i, server := range servers {
+	for i, server := range r.servers {
 		existing[server.Host] = i
 	}

 	hosts := make(map[string]bool)

-	for _, server := range serverList {
+	var hostsLock sync.Mutex
+
+	for _, server := range r.config.ServerList {
 		wg.Add(1)

 		var prefix string
@ -106,11 +185,9 @@ func reloadServers() {
 				"error":  err,
 				"server": server,
 			}).Warning("Server is invalid")
-			return
+			return err
 		}

-		hosts[u.Host] = true
-
 		i := -1

 		if v, exists := existing[u.Host]; exists {
@ -120,19 +197,28 @@ func reloadServers() {
 		go func(i int, server ServerConfig, u *url.URL) {
 			defer wg.Done()

-			s := addServer(server, u)
+			s, err := r.addServer(server, u)
+
+			if err != nil {
+				log.WithError(err).Warning("Unable to add server")
+				return
+			}
+
+			hostsLock.Lock()
+			hosts[u.Host] = true
+			hostsLock.Unlock()

 			if _, ok := existing[u.Host]; ok {
-				s.Redirects = servers[i].Redirects
+				s.Redirects = r.servers[i].Redirects

-				servers[i] = s
+				r.servers[i] = s
 			} else {
 				s.Redirects = promauto.NewCounter(prometheus.CounterOpts{
 					Name: "armbian_router_redirects_" + metricReplacer.Replace(u.Host),
 					Help: "The number of redirects for server " + u.Host,
 				})

-				servers = append(servers, s)
+				r.servers = append(r.servers, s)

 				log.WithFields(log.Fields{
 					"server":    u.Host,
@ -147,24 +233,26 @@ func reloadServers() {
 	wg.Wait()

 	// Remove servers that no longer exist in the config
-	for i := len(servers) - 1; i >= 0; i-- {
-		if _, exists := hosts[servers[i].Host]; exists {
+	for i := len(r.servers) - 1; i >= 0; i-- {
+		if _, exists := hosts[r.servers[i].Host]; exists {
 			continue
 		}

 		log.WithFields(log.Fields{
-			"server": servers[i].Host,
+			"server": r.servers[i].Host,
 		}).Info("Removed server")

-		servers = append(servers[:i], servers[i+1:]...)
+		r.servers = append(r.servers[:i], r.servers[i+1:]...)
 	}
+
+	return nil
 }

 var metricReplacer = strings.NewReplacer(".", "_", "-", "_")

 // addServer takes ServerConfig and constructs a server.
 // This will create duplicate servers, but it will overwrite existing ones when changed.
-func addServer(server ServerConfig, u *url.URL) *Server {
+func (r *Redirector) addServer(server ServerConfig, u *url.URL) (*Server, error) {
 	s := &Server{
 		Available: true,
 		Host:      u.Host,
@ -173,6 +261,15 @@ func addServer(server ServerConfig, u *url.URL) *Server {
 		Longitude: server.Longitude,
 		Continent: server.Continent,
 		Weight:    server.Weight,
+		Protocols: ProtocolList{"http", "https"},
+	}
+
+	if len(server.Protocols) > 0 {
+		for _, proto := range server.Protocols {
+			if !s.Protocols.Contains(proto) {
+				s.Protocols = s.Protocols.Append(proto)
+			}
+		}
 	}

 	// Defaults to 10 to allow servers to be set lower for lower priority
@ -187,11 +284,11 @@ func addServer(server ServerConfig, u *url.URL) *Server {
 			"error":  err,
 			"server": s.Host,
 		}).Warning("Could not resolve address")
-		return nil
+		return nil, err
 	}

 	var city City
-	err = db.Lookup(ips[0], &city)
+	err = r.db.Lookup(ips[0], &city)

 	if err != nil {
 		log.WithFields(log.Fields{
@ -199,7 +296,7 @@ func addServer(server ServerConfig, u *url.URL) *Server {
 			"server": s.Host,
 			"ip":     ips[0],
 		}).Warning("Could not geolocate address")
-		return nil
+		return nil, err
 	}

 	if s.Continent == "" {
@ -211,23 +308,25 @@ func addServer(server ServerConfig, u *url.URL) *Server {
 		s.Longitude = city.Location.Longitude
 	}

-	return s
+	return s, nil
 }

-func reloadMap() {
-	mapFile := viper.GetString("dl_map")
+func (r *Redirector) reloadMap() error {
+	mapFile := r.config.MapFile

 	if mapFile == "" {
-		return
+		return nil
 	}

 	log.WithField("file", mapFile).Info("Loading download map")

-	newMap, err := loadMap(mapFile)
+	newMap, err := loadMapFile(mapFile)

 	if err != nil {
-		return
+		return err
 	}

-	dlMap = newMap
+	r.dlMap = newMap
+
+	return nil
 }
--- a/dlrouter.yaml
+++ b/dlrouter.yaml
@ -34,6 +34,10 @@ servers:
    - server: mirrors.bfsu.edu.cn/armbian/
    - server: mirrors.dotsrc.org/armbian-apt/
      weight: 15
+      protocols:
+        - http
+        - https
+        - rsync
    - server: mirrors.netix.net/armbian/apt/
    - server: mirrors.nju.edu.cn/armbian/
    - server: mirrors.sustech.edu.cn/armbian/
--- a/go.mod
+++ b/go.mod
@ -1,13 +1,17 @@
-module meow.tf/armbian-router
+module github.com/armbian/redirector

-go 1.17
+go 1.19

 require (
 	github.com/chi-middleware/logrus-logger v0.2.0
 	github.com/go-chi/chi/v5 v5.0.7
+	github.com/gwatts/rootcerts v0.0.0-20220501184621-6eac2dff0b8d
 	github.com/hashicorp/golang-lru v0.5.4
 	github.com/jmcvetta/randutil v0.0.0-20150817122601-2bb1b664bcff
+	github.com/onsi/ginkgo/v2 v2.1.4
+	github.com/onsi/gomega v1.20.0
 	github.com/oschwald/maxminddb-golang v1.8.0
+	github.com/pkg/errors v0.9.1
 	github.com/prometheus/client_golang v1.11.0
 	github.com/sirupsen/logrus v1.8.1
 	github.com/spf13/viper v1.10.1
@ -18,6 +22,7 @@ require (
 	github.com/cespare/xxhash/v2 v2.1.2 // indirect
 	github.com/fsnotify/fsnotify v1.5.1 // indirect
 	github.com/golang/protobuf v1.5.2 // indirect
+	github.com/google/go-cmp v0.5.8 // indirect
 	github.com/hashicorp/hcl v1.0.0 // indirect
 	github.com/magiconair/properties v1.8.5 // indirect
 	github.com/matttproud/golang_protobuf_extensions v1.0.1 // indirect
@ -31,9 +36,11 @@ require (
 	github.com/spf13/jwalterweatherman v1.1.0 // indirect
 	github.com/spf13/pflag v1.0.5 // indirect
 	github.com/subosito/gotenv v1.2.0 // indirect
-	golang.org/x/sys v0.0.0-20211210111614-af8b64212486 // indirect
+	golang.org/x/net v0.0.0-20220425223048-2871e0cb64e4 // indirect
+	golang.org/x/sys v0.0.0-20220422013727-9388b58f7150 // indirect
 	golang.org/x/text v0.3.7 // indirect
-	google.golang.org/protobuf v1.27.1 // indirect
+	google.golang.org/protobuf v1.28.0 // indirect
 	gopkg.in/ini.v1 v1.66.2 // indirect
 	gopkg.in/yaml.v2 v2.4.0 // indirect
+	gopkg.in/yaml.v3 v3.0.1 // indirect
 )
--- a/go.sum
+++ b/go.sum
@ -110,6 +110,8 @@ github.com/envoyproxy/protoc-gen-validate v0.6.2/go.mod h1:2t7qjJNvHPx8IjnBOzl9E
 github.com/fatih/color v1.7.0/go.mod h1:Zm6kSWBoL9eyXnKyktHP6abPY2pDugNf5KwzbycvMj4=
 github.com/fatih/color v1.9.0/go.mod h1:eQcE1qtQxscV5RaZvpXrrb8Drkc3/DdQ+uUYCNjL+zU=
 github.com/fatih/color v1.13.0/go.mod h1:kLAiJbzzSOZDVNGyDpeOxJ47H46qBXwg5ILebYFFOfk=
+github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo=
+github.com/fsnotify/fsnotify v1.4.9/go.mod h1:znqG4EE+3YCdAaPaxE2ZRY/06pZUdp0tY4IgpuI1SZQ=
 github.com/fsnotify/fsnotify v1.5.1 h1:mZcQUHVQUQWoPXXtuf9yuEXKudkV2sx1E06UadKWpgI=
 github.com/fsnotify/fsnotify v1.5.1/go.mod h1:T3375wBYaZdLLcVNkcVbzGHY7f1l/uK5T5Ai1i3InKU=
 github.com/ghodss/yaml v1.0.0/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04=
@ -126,6 +128,7 @@ github.com/go-logfmt/logfmt v0.3.0/go.mod h1:Qt1PoO58o5twSAckw1HlFXLmHsOX5/0LbT9
 github.com/go-logfmt/logfmt v0.4.0/go.mod h1:3RMwSq7FuexP4Kalkev3ejPJsZTpXXBr9+V4qmtdjCk=
 github.com/go-logfmt/logfmt v0.5.0/go.mod h1:wCYkCAKZfumFQihp8CzCvQ3paCTfi41vtzG1KdI/P7A=
 github.com/go-stack/stack v1.8.0/go.mod h1:v0f6uXyyMGvRgIKkXu+yp6POWl0qKG85gN/melR3HDY=
+github.com/go-task/slim-sprig v0.0.0-20210107165309-348f09dbbbc0/go.mod h1:fyg7847qk6SyHyPtNmDHnmrv/HOrqktSC+C9fM+CJOE=
 github.com/godbus/dbus/v5 v5.0.4/go.mod h1:xhWf0FNVPg57R7Z0UbKHbJfkEywrmjJnf7w5xrFpKfA=
 github.com/gogo/protobuf v1.1.1/go.mod h1:r8qH/GZQm5c6nD/R0oafs1akxWv10x8SbQlK7atdtwQ=
 github.com/gogo/protobuf v1.3.2/go.mod h1:P1XiOD3dCwIKUDQYPy72D8LYyHL2YPYrpS2s69NZV8Q=
@ -175,8 +178,9 @@ github.com/google/go-cmp v0.5.2/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/
 github.com/google/go-cmp v0.5.3/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
 github.com/google/go-cmp v0.5.4/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
 github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
-github.com/google/go-cmp v0.5.6 h1:BKbKCqvP6I+rmFHt06ZmyQtvB8xAkWdhFyr0ZUNZcxQ=
 github.com/google/go-cmp v0.5.6/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
+github.com/google/go-cmp v0.5.8 h1:e6P7q2lk1O+qJJb4BtCQXlK8vWEO8V1ZeuEdJNOqZyg=
+github.com/google/go-cmp v0.5.8/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
 github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
 github.com/google/martian v2.1.0+incompatible/go.mod h1:9I4somxYTbIHy5NJKHRl3wXiIaQGbYVAs8BPL6v8lEs=
 github.com/google/martian/v3 v3.0.0/go.mod h1:y5Zk1BBys9G+gd6Jrk0W3cC1+ELVxBWuIGO+w/tUAp0=
@ -193,6 +197,7 @@ github.com/google/pprof v0.0.0-20201023163331-3e6fc7fc9c4c/go.mod h1:kpwsk12EmLe
 github.com/google/pprof v0.0.0-20201203190320-1bf35d6f28c2/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=
 github.com/google/pprof v0.0.0-20210122040257-d980be63207e/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=
 github.com/google/pprof v0.0.0-20210226084205-cbba55b83ad5/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=
+github.com/google/pprof v0.0.0-20210407192527-94a9f03dee38/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=
 github.com/google/pprof v0.0.0-20210601050228-01bbb1931b22/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=
 github.com/google/pprof v0.0.0-20210609004039-a478d1d731e9/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=
 github.com/google/pprof v0.0.0-20210720184732-4bb14d4b1be1/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=
@ -203,6 +208,8 @@ github.com/googleapis/gax-go/v2 v2.0.5/go.mod h1:DWXyrwAJ9X0FpwwEdw+IPEYBICEFu5m
 github.com/googleapis/gax-go/v2 v2.1.0/go.mod h1:Q3nei7sK6ybPYH7twZdmQpAd1MKb7pfu6SK+H1/DsU0=
 github.com/googleapis/gax-go/v2 v2.1.1/go.mod h1:hddJymUZASv3XPyGkUpKj8pPO47Rmb0eJc8R6ouapiM=
 github.com/grpc-ecosystem/grpc-gateway v1.16.0/go.mod h1:BDjrQk3hbvj6Nolgz8mAMFbcEtjT1g+wF4CSlocrBnw=
+github.com/gwatts/rootcerts v0.0.0-20220501184621-6eac2dff0b8d h1:Kp5G1kHMb2fAD9OiqWDXro4qLB8bQ2NusoorYya4Lbo=
+github.com/gwatts/rootcerts v0.0.0-20220501184621-6eac2dff0b8d/go.mod h1:5Kt9XkWvkGi2OHOq0QsGxebHmhCcqJ8KCbNg/a6+n+g=
 github.com/hashicorp/consul/api v1.12.0/go.mod h1:6pVBMo0ebnYdt2S3H87XhekM/HHrUoTD2XXb/VrZVy0=
 github.com/hashicorp/consul/sdk v0.8.0/go.mod h1:GBvyrGALthsZObzUGsfgHZQDXjg4lOjagTIwIR1vPms=
 github.com/hashicorp/errwrap v1.0.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4=
@ -232,6 +239,7 @@ github.com/hashicorp/logutils v1.0.0/go.mod h1:QIAnNjmIWmVIIkWDTG1z5v++HQmx9WQRO
 github.com/hashicorp/mdns v1.0.4/go.mod h1:mtBihi+LeNXGtG8L9dX59gAEa12BDtBQSp4v/YAJqrc=
 github.com/hashicorp/memberlist v0.3.0/go.mod h1:MS2lj3INKhZjWNqd3N0m3J+Jxf3DAOnAH9VT3Sh9MUE=
 github.com/hashicorp/serf v0.9.6/go.mod h1:TXZNMjZQijwlDvp+r0b63xZ45H7JmCmgg4gpTwn9UV4=
+github.com/hpcloud/tail v1.0.0/go.mod h1:ab1qPbhIpdTxEkNHXyeSf5vhxWSCs/tWer42PpOxQnU=
 github.com/iancoleman/strcase v0.2.0/go.mod h1:iwCmte+B7n89clKwxIoIXy/HfoL7AsD47ZCWhYzw7ho=
 github.com/ianlancetaylor/demangle v0.0.0-20181102032728-5e5cf60278f6/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc=
 github.com/ianlancetaylor/demangle v0.0.0-20200824232613-28f6c0f3b639/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc=
@ -293,6 +301,21 @@ github.com/modern-go/reflect2 v1.0.1/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3Rllmb
 github.com/modern-go/reflect2 v1.0.2/go.mod h1:yWuevngMOJpCy52FWWMvUC8ws7m/LJsjYzDa0/r8luk=
 github.com/mwitkow/go-conntrack v0.0.0-20161129095857-cc309e4a2223/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U=
 github.com/mwitkow/go-conntrack v0.0.0-20190716064945-2f068394615f/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U=
+github.com/nxadm/tail v1.4.4/go.mod h1:kenIhsEOeOJmVchQTgglprH7qJGnHDVpk1VPCcaMI8A=
+github.com/nxadm/tail v1.4.8/go.mod h1:+ncqLTQzXmGhMZNUePPaPqPvBxHAIsmXswZKocGu+AU=
+github.com/onsi/ginkgo v1.6.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
+github.com/onsi/ginkgo v1.12.1/go.mod h1:zj2OWP4+oCPe1qIXoGWkgMRwljMUYCdkwsT2108oapk=
+github.com/onsi/ginkgo v1.16.4 h1:29JGrr5oVBm5ulCWet69zQkzWipVXIol6ygQUe/EzNc=
+github.com/onsi/ginkgo v1.16.4/go.mod h1:dX+/inL/fNMqNlz0e9LfyB9TswhZpCVdJM/Z6Vvnwo0=
+github.com/onsi/ginkgo/v2 v2.1.3/go.mod h1:vw5CSIxN1JObi/U8gcbwft7ZxR2dgaR70JSE3/PpL4c=
+github.com/onsi/ginkgo/v2 v2.1.4 h1:GNapqRSid3zijZ9H77KrgVG4/8KqiyRsxcSxe+7ApXY=
+github.com/onsi/ginkgo/v2 v2.1.4/go.mod h1:um6tUpWM/cxCK3/FK8BXqEiUMUwRgSM4JXG47RKZmLU=
+github.com/onsi/gomega v1.7.1/go.mod h1:XdKZgCCFLUoM/7CFJVPcG8C1xQ1AJ0vpAezJrB7JYyY=
+github.com/onsi/gomega v1.10.1/go.mod h1:iN09h71vgCQne3DLsj+A5owkum+a2tYe+TOCB1ybHNo=
+github.com/onsi/gomega v1.17.0/go.mod h1:HnhC7FXeEQY45zxNK3PPoIUhzk/80Xly9PcubAlGdZY=
+github.com/onsi/gomega v1.19.0/go.mod h1:LY+I3pBVzYsTBU1AnDwOSxaYi9WoWiqgwooUqq9yPro=
+github.com/onsi/gomega v1.20.0 h1:8W0cWlwFkflGPLltQvLRB7ZVD5HuP6ng320w2IS245Q=
+github.com/onsi/gomega v1.20.0/go.mod h1:DtrZpjmvpn2mPm4YWQa0/ALMDj9v4YxLgojwPeREyVo=
 github.com/oschwald/maxminddb-golang v1.8.0 h1:Uh/DSnGoxsyp/KYbY1AuP0tYEwfs0sCph9p/UMXK/Hk=
 github.com/oschwald/maxminddb-golang v1.8.0/go.mod h1:RXZtst0N6+FY/3qCNmZMBApR19cdQj43/NM9VkrNAis=
 github.com/pascaldekloe/goe v0.0.0-20180627143212-57f6aae5913c/go.mod h1:lzWF7FIEvWOWxwDKqyGYQf6ZUaNfKdP144TG7ZOy1lc=
@ -301,6 +324,7 @@ github.com/pelletier/go-toml v1.9.4 h1:tjENF6MfZAg8e4ZmZTeWaWiT2vXtsoO6+iuOjFhEC
 github.com/pelletier/go-toml v1.9.4/go.mod h1:u1nR/EPcESfeI/szUZKdtJ0xRNbUoANCkoOuaOx1Y+c=
 github.com/pkg/errors v0.8.0/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
 github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
+github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
 github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
 github.com/pkg/sftp v1.10.1/go.mod h1:lYOWFsE0bwd1+KfKJaKeuokY15vzFx25BLbzYYoAxZI=
 github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
@ -368,6 +392,7 @@ github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9de
 github.com/yuin/goldmark v1.1.32/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
 github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
 github.com/yuin/goldmark v1.3.5/go.mod h1:mwnBkeHKe2W/ZEtQ+71ViKU8L12m81fl3OWwC1Zlc8k=
+github.com/yuin/goldmark v1.4.1/go.mod h1:mwnBkeHKe2W/ZEtQ+71ViKU8L12m81fl3OWwC1Zlc8k=
 go.etcd.io/etcd/api/v3 v3.5.1/go.mod h1:cbVKeC6lCfl7j/8jBhAK6aIYO9XOjdptoxU/nLQcPvs=
 go.etcd.io/etcd/client/pkg/v3 v3.5.1/go.mod h1:IJHfcCEKxYu1Os13ZdwCwIUTUVGYTSAM3YSwc9/Ac1g=
 go.etcd.io/etcd/client/v2 v2.305.1/go.mod h1:pMEacxZW7o8pg4CrFE7pquyCJJzZvkvdD2RibOCCCGs=
@ -391,6 +416,7 @@ golang.org/x/crypto v0.0.0-20190923035154-9ee001bba392/go.mod h1:/lpIB1dKB+9EgE3
 golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
 golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
 golang.org/x/crypto v0.0.0-20210817164053-32db794688a5/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
+golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
 golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
 golang.org/x/exp v0.0.0-20190306152737-a1d7652674e8/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
 golang.org/x/exp v0.0.0-20190510132918-efd6b22b2522/go.mod h1:ZjyILWgesfNpC6sMxTJOJm9Kp84zZh5NQWvqDGG3Qr8=
@ -427,8 +453,10 @@ golang.org/x/mod v0.4.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
 golang.org/x/mod v0.4.1/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
 golang.org/x/mod v0.4.2/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
 golang.org/x/mod v0.5.0/go.mod h1:5OXOZSfqPIIbmVBIIKWRFfZjPR0E5r58TLhUjH0a2Ro=
+golang.org/x/mod v0.6.0-dev.0.20220106191415-9b9b3d81d5e3/go.mod h1:3p9vT2HGsQu2K1YbXdKPJLVgG5VJdoTa1poYQBtP1AY=
 golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
 golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
+golang.org/x/net v0.0.0-20180906233101-161cd47e91fd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
 golang.org/x/net v0.0.0-20181114220301-adae6a3d119a/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
 golang.org/x/net v0.0.0-20190108225652-1e06a53dbb7e/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
 golang.org/x/net v0.0.0-20190213061140-3a22650c66bd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
@ -452,6 +480,7 @@ golang.org/x/net v0.0.0-20200324143707-d3edc9973b7e/go.mod h1:qpuaurCH72eLCgpAm/
 golang.org/x/net v0.0.0-20200501053045-e0ff5e5a1de5/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
 golang.org/x/net v0.0.0-20200506145744-7e3656a0809f/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
 golang.org/x/net v0.0.0-20200513185701-a91f0712d120/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
+golang.org/x/net v0.0.0-20200520004742-59133d7f0dd7/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
 golang.org/x/net v0.0.0-20200520182314-0ba52f642ac2/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
 golang.org/x/net v0.0.0-20200625001655-4c5254603344/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA=
 golang.org/x/net v0.0.0-20200707034311-ab3426394381/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA=
@ -465,8 +494,13 @@ golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v
 golang.org/x/net v0.0.0-20210316092652-d523dce5a7f4/go.mod h1:RBQZq4jEuRlivfhVLdyRGr576XBO4/greRjx4P4O3yc=
 golang.org/x/net v0.0.0-20210405180319-a5a99cb37ef4/go.mod h1:p54w0d4576C0XHj96bSt6lcn1PtDYWL6XObtHCRCNQM=
 golang.org/x/net v0.0.0-20210410081132-afb366fc7cd1/go.mod h1:9tjilg8BloeKEkVJvy7fQ90B1CfIiPueXVOjqfkSzI8=
+golang.org/x/net v0.0.0-20210428140749-89ef3d95e781/go.mod h1:OJAsFXCWl8Ukc7SiCT/9KSuxbyM7479/AVlXFRxuMCk=
 golang.org/x/net v0.0.0-20210503060351-7fd8e65b6420/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
 golang.org/x/net v0.0.0-20210813160813-60bc85c4be6d/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
+golang.org/x/net v0.0.0-20211015210444-4f30a5c0130f/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
+golang.org/x/net v0.0.0-20220225172249-27dd8689420f/go.mod h1:CfG3xpIq0wQ8r1q4Su4UZFWDARRcnwPjda9FqA0JpMk=
+golang.org/x/net v0.0.0-20220425223048-2871e0cb64e4 h1:HVyaeDAYux4pnY+D/SiwmLOR36ewZ4iGQIIrtnuCjFA=
+golang.org/x/net v0.0.0-20220425223048-2871e0cb64e4/go.mod h1:CfG3xpIq0wQ8r1q4Su4UZFWDARRcnwPjda9FqA0JpMk=
 golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
 golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
 golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
@ -498,6 +532,7 @@ golang.org/x/sync v0.0.0-20210220032951-036812b2e83c/go.mod h1:RxMgew5VJxzue5/jJ
 golang.org/x/sys v0.0.0-20180823144017-11551d06cbcc/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
 golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
 golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
+golang.org/x/sys v0.0.0-20180909124046-d0be0721c37e/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
 golang.org/x/sys v0.0.0-20181116152217-5ac8a444bdc5/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
 golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
 golang.org/x/sys v0.0.0-20190222072716-a9d3bda3a223/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
@ -509,11 +544,14 @@ golang.org/x/sys v0.0.0-20190507160741-ecd444e8653b/go.mod h1:h1NjWce9XRLGQEsW7w
 golang.org/x/sys v0.0.0-20190606165138-5da285871e9c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20190624142023-c5567b49c5d0/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20190726091711-fc99dfbffb4e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20190904154756-749cb33beabd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20190922100055-0a153f010e69/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20190924154521-2837fb4f24fe/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20191001151750-bb3f8db39f24/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20191005200804-aed5e4c7ecf9/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20191008105621-543471e840be/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20191026070338-33540a1f6037/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20191120155948-bd437916bb0e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20191204072324-ce4227a45e2e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20191224085550-c709ea063b76/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20191228213918-04cbcbbfeed8/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
@ -540,6 +578,7 @@ golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7w
 golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20201201145000-ef89a241ccb3/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20210104204734-6f8348627aad/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20210112080510-489259a85091/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20210119212857-b64e53b001e4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20210124154548-22da62e12c0c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20210220050731-9a76102bfb43/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
@ -564,10 +603,15 @@ golang.org/x/sys v0.0.0-20210823070655-63515b42dcdf/go.mod h1:oPkhp1MJrh7nUepCBc
 golang.org/x/sys v0.0.0-20210908233432-aa78b53d3365/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/sys v0.0.0-20210927094055-39ccf1dd6fa6/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/sys v0.0.0-20211007075335-d3039528d8ac/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.0.0-20211019181941-9d821ace8654/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/sys v0.0.0-20211124211545-fe61309f8881/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
-golang.org/x/sys v0.0.0-20211210111614-af8b64212486 h1:5hpz5aRr+W1erYCL5JRhSUBJRph7l9XkNveoExlrKYk=
 golang.org/x/sys v0.0.0-20211210111614-af8b64212486/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.0.0-20211216021012-1d35b9e2eb4e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.0.0-20220319134239-a9b59b0215f8/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.0.0-20220422013727-9388b58f7150 h1:xHms4gcpe1YE7A3yIllJXP16CMAGuqwO2lX1mTyyRRc=
+golang.org/x/sys v0.0.0-20220422013727-9388b58f7150/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
+golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
 golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
 golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
 golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
@ -627,6 +671,7 @@ golang.org/x/tools v0.0.0-20200904185747-39188db58858/go.mod h1:Cj7w3i3Rnn0Xh82u
 golang.org/x/tools v0.0.0-20201110124207-079ba7bd75cd/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
 golang.org/x/tools v0.0.0-20201201161351-ac6f37ff4c2a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
 golang.org/x/tools v0.0.0-20201208233053-a543418bbed2/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
+golang.org/x/tools v0.0.0-20201224043029-2b0845dc783e/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
 golang.org/x/tools v0.0.0-20210105154028-b0ab187a4818/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
 golang.org/x/tools v0.0.0-20210106214847-113979e3529a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
 golang.org/x/tools v0.1.0/go.mod h1:xkSsbof2nBLbhDlRMhhhyNLN/zl3eTqcnHD5viDpcZ0=
@ -635,10 +680,10 @@ golang.org/x/tools v0.1.2/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk=
 golang.org/x/tools v0.1.3/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk=
 golang.org/x/tools v0.1.4/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk=
 golang.org/x/tools v0.1.5/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk=
+golang.org/x/tools v0.1.10/go.mod h1:Uh6Zz+xoGYZom868N8YTex3t7RhtHDBrE8Gzo9bV56E=
 golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
 golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
 golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
-golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1 h1:go1bK/D/BFZV2I8cIQd1NKEZ+0owSTG1fDTci4IqFcE=
 golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
 google.golang.org/api v0.4.0/go.mod h1:8k5glujaEP+g9n7WNsDg8QP6cUVNI86fCNMcbazEtwE=
 google.golang.org/api v0.7.0/go.mod h1:WtwebWUNSVBH/HAw79HIFXZNqEvBhG+Ra+ax0hx3E3M=
@ -780,8 +825,9 @@ google.golang.org/protobuf v1.24.0/go.mod h1:r/3tXBNzIEhYS9I1OUVjXDlt8tc493IdKGj
 google.golang.org/protobuf v1.25.0/go.mod h1:9JNX74DMeImyA3h4bdi1ymwjUzf21/xIlbajtzgsN7c=
 google.golang.org/protobuf v1.26.0-rc.1/go.mod h1:jlhhOSvTdKEhbULTjvd4ARK9grFBp09yW+WbY/TyQbw=
 google.golang.org/protobuf v1.26.0/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc=
-google.golang.org/protobuf v1.27.1 h1:SnqbnDw1V7RiZcXPx5MEeqPv2s79L9i7BJUlG/+RurQ=
 google.golang.org/protobuf v1.27.1/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc=
+google.golang.org/protobuf v1.28.0 h1:w43yiav+6bVFTBQFZX0r7ipe9JQ1QsbMgHwbBziscLw=
+google.golang.org/protobuf v1.28.0/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqwMG9pJV4I=
 gopkg.in/alecthomas/kingpin.v2 v2.2.6/go.mod h1:FMv+mEhP44yOT+4EoQTLFTRgOQ1FBLkstjWtayDeSgw=
 gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
 gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
@ -789,8 +835,10 @@ gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8
 gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=
 gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q=
 gopkg.in/errgo.v2 v2.1.0/go.mod h1:hNsd1EY+bozCKY1Ytp96fpM3vjJbqLJn88ws8XvfDNI=
+gopkg.in/fsnotify.v1 v1.4.7/go.mod h1:Tz8NjZHkW78fSQdbUxIjBTcgA1z1m8ZHf0WmKUhAMys=
 gopkg.in/ini.v1 v1.66.2 h1:XfR1dOYubytKy4Shzc2LHrrGhU0lDCfDGG1yLPmpgsI=
 gopkg.in/ini.v1 v1.66.2/go.mod h1:pNLf8WUiyNEtQjuu5G5vTm06TEv9tsIgeAvK8hOrP4k=
+gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7/go.mod h1:dt/ZhP58zS4L8KSrWDmTeBkI65Dw0HsyUHuEVlX15mw=
 gopkg.in/yaml.v2 v2.2.1/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
 gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
 gopkg.in/yaml.v2 v2.2.3/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
@ -801,8 +849,9 @@ gopkg.in/yaml.v2 v2.3.0/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
 gopkg.in/yaml.v2 v2.4.0 h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY=
 gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ=
 gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
-gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b h1:h8qDotaEPuJATrMmW04NCwg7v22aHH28wwpauUhK9Oo=
 gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
+gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
+gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
 honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
 honnef.co/go/tools v0.0.0-20190106161140-3f1c8253044a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
 honnef.co/go/tools v0.0.0-20190418001031-e561f6794a2a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
--- a/http.go
+++ b/http.go
@ -1,10 +1,9 @@
-package main
+package redirector

 import (
 	"encoding/json"
 	"fmt"
 	"github.com/jmcvetta/randutil"
-	"github.com/spf13/viper"
 	"net"
 	"net/http"
 	"net/url"
@ -13,13 +12,20 @@ import (
 	"strings"
 )

-func statusHandler(w http.ResponseWriter, r *http.Request) {
+// statusHandler is a simple handler that will always return 200 OK with a body of "OK"
+func (r *Redirector) statusHandler(w http.ResponseWriter, req *http.Request) {
 	w.WriteHeader(http.StatusOK)
-	w.Write([]byte("OK"))
+
+	if req.Method != http.MethodHead {
+		w.Write([]byte("OK"))
+	}
 }

-func redirectHandler(w http.ResponseWriter, r *http.Request) {
-	ipStr, _, err := net.SplitHostPort(r.RemoteAddr)
+// redirectHandler is the default "not found" handler which handles redirects
+// if the environment variable OVERRIDE_IP is set, it will use that ip address
+// this is useful for local testing when you're on the local network
+func (r *Redirector) redirectHandler(w http.ResponseWriter, req *http.Request) {
+	ipStr, _, err := net.SplitHostPort(req.RemoteAddr)

 	if err != nil {
 		http.Error(w, err.Error(), http.StatusInternalServerError)
@ -41,11 +47,13 @@ func redirectHandler(w http.ResponseWriter, r *http.Request) {
 	var server *Server
 	var distance float64

-	if strings.HasPrefix(r.URL.Path, "/region") {
-		parts := strings.Split(r.URL.Path, "/")
+	// If the path has a prefix of region/NA, it will use specific regions instead
+	// of the default geographical distance
+	if strings.HasPrefix(req.URL.Path, "/region") {
+		parts := strings.Split(req.URL.Path, "/")

 		// region = parts[2]
-		if mirrors, ok := regionMap[parts[2]]; ok {
+		if mirrors, ok := r.regionMap[parts[2]]; ok {
 			choices := make([]randutil.Choice, len(mirrors))

 			for i, item := range mirrors {
@ -68,12 +76,20 @@ func redirectHandler(w http.ResponseWriter, r *http.Request) {

 			server = choice.Item.(*Server)

-			r.URL.Path = strings.Join(parts[3:], "/")
+			req.URL.Path = strings.Join(parts[3:], "/")
 		}
 	}

+	// If we don't have a scheme, we'll use http by default
+	scheme := req.URL.Scheme
+
+	if scheme == "" {
+		scheme = "http"
+	}
+
+	// If none of the above exceptions are matched, we use the geographical distance based on IP
 	if server == nil {
-		server, distance, err = servers.Closest(ip)
+		server, distance, err = r.servers.Closest(r, scheme, ip)

 		if err != nil {
 			http.Error(w, err.Error(), http.StatusInternalServerError)
@ -81,25 +97,24 @@ func redirectHandler(w http.ResponseWriter, r *http.Request) {
 		}
 	}

-	scheme := r.URL.Scheme
+	// redirectPath is a combination of server path (which can be something like /armbian)
+	// and the URL path.
+	// Example: /armbian + /some/path = /armbian/some/path
+	redirectPath := path.Join(server.Path, req.URL.Path)

-	if scheme == "" {
-		scheme = "https"
-	}
-
-	redirectPath := path.Join(server.Path, r.URL.Path)
-
-	if dlMap != nil {
-		if newPath, exists := dlMap[strings.TrimLeft(r.URL.Path, "/")]; exists {
+	// If we have a dlMap, we map the url to a final path instead
+	if r.dlMap != nil {
+		if newPath, exists := r.dlMap[strings.TrimLeft(req.URL.Path, "/")]; exists {
 			downloadsMapped.Inc()
 			redirectPath = path.Join(server.Path, newPath)
 		}
 	}

-	if strings.HasSuffix(r.URL.Path, "/") && !strings.HasSuffix(redirectPath, "/") {
+	if strings.HasSuffix(req.URL.Path, "/") && !strings.HasSuffix(redirectPath, "/") {
 		redirectPath += "/"
 	}

+	// We need to build the final url now
 	u := &url.URL{
 		Scheme: scheme,
 		Host:   server.Host,
@ -109,6 +124,7 @@ func redirectHandler(w http.ResponseWriter, r *http.Request) {
 	server.Redirects.Inc()
 	redirectsServed.Inc()

+	// If we used geographical distance, we add an X-Geo-Distance header for debug.
 	if distance > 0 {
 		w.Header().Set("X-Geo-Distance", fmt.Sprintf("%f", distance))
 	}
@ -117,8 +133,15 @@ func redirectHandler(w http.ResponseWriter, r *http.Request) {
 	w.WriteHeader(http.StatusFound)
 }

-func reloadHandler(w http.ResponseWriter, r *http.Request) {
-	token := r.Header.Get("Authorization")
+// reloadHandler is an http handler which lets us reload the server configuration
+// It is only enabled when the reloadToken is set in the configuration
+func (r *Redirector) reloadHandler(w http.ResponseWriter, req *http.Request) {
+	if r.config.ReloadToken == "" {
+		w.WriteHeader(http.StatusUnauthorized)
+		return
+	}
+
+	token := req.Header.Get("Authorization")

 	if token == "" || !strings.HasPrefix(token, "Bearer") || !strings.Contains(token, " ") {
 		w.WriteHeader(http.StatusUnauthorized)
@ -127,30 +150,34 @@ func reloadHandler(w http.ResponseWriter, r *http.Request) {

 	token = token[strings.Index(token, " ")+1:]

-	if token != viper.GetString("reloadToken") {
+	if token != r.config.ReloadToken {
 		w.WriteHeader(http.StatusUnauthorized)
 		return
 	}

-	reloadConfig()
+	if err := r.ReloadConfig(); err != nil {
+		w.WriteHeader(http.StatusInternalServerError)
+		w.Write([]byte(err.Error()))
+		return
+	}

 	w.WriteHeader(http.StatusOK)
 	w.Write([]byte("OK"))
 }

-func dlMapHandler(w http.ResponseWriter, r *http.Request) {
-	if dlMap == nil {
+func (r *Redirector) dlMapHandler(w http.ResponseWriter, req *http.Request) {
+	if r.dlMap == nil {
 		w.WriteHeader(http.StatusNotFound)
 		return
 	}

 	w.Header().Set("Content-Type", "application/json")

-	json.NewEncoder(w).Encode(dlMap)
+	json.NewEncoder(w).Encode(r.dlMap)
 }

-func geoIPHandler(w http.ResponseWriter, r *http.Request) {
-	ipStr, _, err := net.SplitHostPort(r.RemoteAddr)
+func (r *Redirector) geoIPHandler(w http.ResponseWriter, req *http.Request) {
+	ipStr, _, err := net.SplitHostPort(req.RemoteAddr)

 	if err != nil {
 		http.Error(w, err.Error(), http.StatusInternalServerError)
@ -160,7 +187,7 @@ func geoIPHandler(w http.ResponseWriter, r *http.Request) {
 	ip := net.ParseIP(ipStr)

 	var city City
-	err = db.Lookup(ip, &city)
+	err = r.db.Lookup(ip, &city)

 	if err != nil {
 		http.Error(w, err.Error(), http.StatusInternalServerError)
--- a/main.go
+++ b/main.go
@ -1,148 +0,0 @@
-package main
-
-import (
-	"flag"
-	"github.com/chi-middleware/logrus-logger"
-	"github.com/go-chi/chi/v5"
-	lru "github.com/hashicorp/golang-lru"
-	"github.com/oschwald/maxminddb-golang"
-	"github.com/prometheus/client_golang/prometheus"
-	"github.com/prometheus/client_golang/prometheus/promauto"
-	"github.com/prometheus/client_golang/prometheus/promhttp"
-	log "github.com/sirupsen/logrus"
-	"github.com/spf13/viper"
-	"net/http"
-	"os"
-	"os/signal"
-	"syscall"
-)
-
-var (
-	db         *maxminddb.Reader
-	servers    ServerList
-	regionMap  map[string][]*Server
-	hostMap    map[string]*Server
-	dlMap      map[string]string
-	topChoices int
-
-	redirectsServed = promauto.NewCounter(prometheus.CounterOpts{
-		Name: "armbian_router_redirects",
-		Help: "The total number of processed redirects",
-	})
-
-	downloadsMapped = promauto.NewCounter(prometheus.CounterOpts{
-		Name: "armbian_router_download_maps",
-		Help: "The total number of mapped download paths",
-	})
-
-	serverCache *lru.Cache
-)
-
-type LocationLookup struct {
-	Location struct {
-		Latitude  float64 `maxminddb:"latitude"`
-		Longitude float64 `maxminddb:"longitude"`
-	} `maxminddb:"location"`
-}
-
-// City represents a MaxmindDB city
-type City struct {
-	Continent struct {
-		Code      string            `maxminddb:"code" json:"code"`
-		GeoNameID uint              `maxminddb:"geoname_id" json:"geoname_id"`
-		Names     map[string]string `maxminddb:"names" json:"names"`
-	} `maxminddb:"continent" json:"continent"`
-	Country struct {
-		GeoNameID uint              `maxminddb:"geoname_id" json:"geoname_id"`
-		IsoCode   string            `maxminddb:"iso_code" json:"iso_code"`
-		Names     map[string]string `maxminddb:"names" json:"names"`
-	} `maxminddb:"country" json:"country"`
-	Location struct {
-		AccuracyRadius uint16  `maxminddb:"accuracy_radius" json:'accuracy_radius'`
-		Latitude       float64 `maxminddb:"latitude" json:"latitude"`
-		Longitude      float64 `maxminddb:"longitude" json:"longitude"`
-	} `maxminddb:"location"`
-	RegisteredCountry struct {
-		GeoNameID uint              `maxminddb:"geoname_id" json:"geoname_id"`
-		IsoCode   string            `maxminddb:"iso_code" json:"iso_code"`
-		Names     map[string]string `maxminddb:"names" json:"names"`
-	} `maxminddb:"registered_country" json:"registered_country"`
-}
-
-type ServerConfig struct {
-	Server    string  `mapstructure:"server" yaml:"server"`
-	Latitude  float64 `mapstructure:"latitude" yaml:"latitude"`
-	Longitude float64 `mapstructure:"longitude" yaml:"longitude"`
-	Continent string  `mapstructure:"continent"`
-	Weight    int     `mapstructure:"weight" yaml:"weight"`
-}
-
-var (
-	configFlag = flag.String("config", "", "configuration file path")
-	flagDebug  = flag.Bool("debug", false, "Enable debug logging")
-)
-
-func main() {
-	flag.Parse()
-
-	if *flagDebug {
-		log.SetLevel(log.DebugLevel)
-	}
-
-	viper.SetDefault("bind", ":8080")
-	viper.SetDefault("cacheSize", 1024)
-	viper.SetDefault("topChoices", 3)
-	viper.SetDefault("reloadKey", randSeq(32))
-
-	viper.SetConfigName("dlrouter")        // name of config file (without extension)
-	viper.SetConfigType("yaml")            // REQUIRED if the config file does not have the extension in the name
-	viper.AddConfigPath("/etc/dlrouter/")  // path to look for the config file in
-	viper.AddConfigPath("$HOME/.dlrouter") // call multiple times to add many search paths
-	viper.AddConfigPath(".")               // optionally look for config in the working directory
-
-	if *configFlag != "" {
-		viper.SetConfigFile(*configFlag)
-	}
-
-	reloadConfig()
-
-	// Start check loop
-	go servers.checkLoop()
-
-	log.Info("Starting")
-
-	r := chi.NewRouter()
-
-	r.Use(RealIPMiddleware)
-	r.Use(logger.Logger("router", log.StandardLogger()))
-
-	r.Head("/status", statusHandler)
-	r.Get("/status", statusHandler)
-	r.Get("/mirrors", legacyMirrorsHandler)
-	r.Get("/mirrors/{server}.svg", mirrorStatusHandler)
-	r.Get("/mirrors.json", mirrorsHandler)
-	r.Post("/reload", reloadHandler)
-	r.Get("/dl_map", dlMapHandler)
-	r.Get("/geoip", geoIPHandler)
-	r.Get("/metrics", promhttp.Handler().ServeHTTP)
-
-	r.NotFound(redirectHandler)
-
-	go http.ListenAndServe(viper.GetString("bind"), r)
-
-	log.Info("Ready")
-
-	c := make(chan os.Signal)
-
-	signal.Notify(c, syscall.SIGKILL, syscall.SIGTERM, syscall.SIGHUP)
-
-	for {
-		sig := <-c
-
-		if sig != syscall.SIGHUP {
-			break
-		}
-
-		reloadConfig()
-	}
-}
--- a/map.go
+++ b/map.go
@ -1,4 +1,4 @@
-package main
+package redirector

 import (
 	"encoding/csv"
@ -7,7 +7,8 @@ import (
 	"strings"
 )

-func loadMap(file string) (map[string]string, error) {
+// loadMapFile loads a file as a map
+func loadMapFile(file string) (map[string]string, error) {
 	f, err := os.Open(file)

 	if err != nil {
@ -16,6 +17,11 @@ func loadMap(file string) (map[string]string, error) {

 	defer f.Close()

+	return loadMap(f)
+}
+
+// loadMap loads a pipe separated file of mappings
+func loadMap(f io.Reader) (map[string]string, error) {
 	m := make(map[string]string)

 	r := csv.NewReader(f)
--- a/map_test.go
+++ b/map_test.go
@ -0,0 +1,16 @@
+package redirector
+
+import (
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+	"strings"
+)
+
+var _ = Describe("Map", func() {
+	It("Should successfully load the map", func() {
+		m, err := loadMap(strings.NewReader(`bananapi/Bullseye_current|bananapi/archive/Armbian_21.08.1_Bananapi_bullseye_current_5.10.60.img.xz|Aug 26 2021|332M`))
+
+		Expect(err).To(BeNil())
+		Expect(m["bananapi/Bullseye_current"]).To(Equal("bananapi/archive/Armbian_21.08.1_Bananapi_bullseye_current_5.10.60.img.xz"))
+	})
+})
--- a/middleware/middleware.go
+++ b/middleware/middleware.go
@ -1,4 +1,4 @@
-package main
+package middleware

 import (
 	"net"
--- a/mirrors.go
+++ b/mirrors.go
@ -1,23 +1,26 @@
-package main
+package redirector

 import (
 	_ "embed"
 	"encoding/json"
 	"github.com/go-chi/chi/v5"
 	"net/http"
+	"strconv"
 	"strings"
 )

-func legacyMirrorsHandler(w http.ResponseWriter, r *http.Request) {
+// legacyMirrorsHandler will list the mirrors by region in the legacy format
+// it is preferred to use mirrors.json, but this handler is here for build support
+func (r *Redirector) legacyMirrorsHandler(w http.ResponseWriter, req *http.Request) {
 	w.Header().Set("Content-Type", "application/json")

 	mirrorOutput := make(map[string][]string)

-	for region, mirrors := range regionMap {
+	for region, mirrors := range r.regionMap {
 		list := make([]string, len(mirrors))

 		for i, mirror := range mirrors {
-			list[i] = r.URL.Scheme + "://" + mirror.Host + "/" + strings.TrimLeft(mirror.Path, "/")
+			list[i] = req.URL.Scheme + "://" + mirror.Host + "/" + strings.TrimLeft(mirror.Path, "/")
 		}

 		mirrorOutput[region] = list
@ -26,9 +29,10 @@ func legacyMirrorsHandler(w http.ResponseWriter, r *http.Request) {
 	json.NewEncoder(w).Encode(mirrorOutput)
 }

-func mirrorsHandler(w http.ResponseWriter, r *http.Request) {
+// mirrorsHandler is a simple handler that will return the list of servers
+func (r *Redirector) mirrorsHandler(w http.ResponseWriter, req *http.Request) {
 	w.Header().Set("Content-Type", "application/json")
-	json.NewEncoder(w).Encode(servers)
+	json.NewEncoder(w).Encode(r.servers)
 }

 var (
@ -42,10 +46,13 @@ var (
 	statusUnknown []byte
 )

-func mirrorStatusHandler(w http.ResponseWriter, r *http.Request) {
-	serverHost := chi.URLParam(r, "server")
+// mirrorStatusHandler is a fancy svg-returning handler.
+// it is used to display mirror statuses on a config repo of sorts
+func (r *Redirector) mirrorStatusHandler(w http.ResponseWriter, req *http.Request) {
+	serverHost := chi.URLParam(req, "server")

 	w.Header().Set("Content-Type", "image/svg+xml;charset=utf-8")
+	w.Header().Set("Cache-Control", "max-age=120")

 	if serverHost == "" {
 		w.Write(statusUnknown)
@ -54,16 +61,34 @@ func mirrorStatusHandler(w http.ResponseWriter, r *http.Request) {

 	serverHost = strings.Replace(serverHost, "_", ".", -1)

-	server, ok := hostMap[serverHost]
+	server, ok := r.hostMap[serverHost]

 	if !ok {
+		w.Header().Set("Content-Length", strconv.Itoa(len(statusUnknown)))
 		w.Write(statusUnknown)
 		return
 	}

+	key := "offline"
+
 	if server.Available {
+		key = "online"
+	}
+
+	w.Header().Set("ETag", "\""+key+"\"")
+
+	if match := req.Header.Get("If-None-Match"); match != "" {
+		if strings.Trim(match, "\"") == key {
+			w.WriteHeader(http.StatusNotModified)
+			return
+		}
+	}
+
+	if server.Available {
+		w.Header().Set("Content-Length", strconv.Itoa(len(statusUp)))
 		w.Write(statusUp)
 	} else {
+		w.Header().Set("Content-Length", strconv.Itoa(len(statusDown)))
 		w.Write(statusDown)
 	}
 }
--- a/redirector.go
+++ b/redirector.go
@ -0,0 +1,131 @@
+package redirector
+
+import (
+	"github.com/armbian/redirector/middleware"
+	"github.com/chi-middleware/logrus-logger"
+	"github.com/go-chi/chi/v5"
+	lru "github.com/hashicorp/golang-lru"
+	"github.com/oschwald/maxminddb-golang"
+	"github.com/prometheus/client_golang/prometheus"
+	"github.com/prometheus/client_golang/prometheus/promauto"
+	"github.com/prometheus/client_golang/prometheus/promhttp"
+	log "github.com/sirupsen/logrus"
+	"net/http"
+)
+
+var (
+	redirectsServed = promauto.NewCounter(prometheus.CounterOpts{
+		Name: "armbian_router_redirects",
+		Help: "The total number of processed redirects",
+	})
+
+	downloadsMapped = promauto.NewCounter(prometheus.CounterOpts{
+		Name: "armbian_router_download_maps",
+		Help: "The total number of mapped download paths",
+	})
+)
+
+type Redirector struct {
+	config      *Config
+	db          *maxminddb.Reader
+	servers     ServerList
+	regionMap   map[string][]*Server
+	hostMap     map[string]*Server
+	dlMap       map[string]string
+	topChoices  int
+	serverCache *lru.Cache
+	checks      []ServerCheck
+	checkClient *http.Client
+}
+
+type LocationLookup struct {
+	Location struct {
+		Latitude  float64 `maxminddb:"latitude"`
+		Longitude float64 `maxminddb:"longitude"`
+	} `maxminddb:"location"`
+}
+
+// City represents a MaxmindDB city
+type City struct {
+	Continent struct {
+		Code      string            `maxminddb:"code" json:"code"`
+		GeoNameID uint              `maxminddb:"geoname_id" json:"geoname_id"`
+		Names     map[string]string `maxminddb:"names" json:"names"`
+	} `maxminddb:"continent" json:"continent"`
+	Country struct {
+		GeoNameID uint              `maxminddb:"geoname_id" json:"geoname_id"`
+		IsoCode   string            `maxminddb:"iso_code" json:"iso_code"`
+		Names     map[string]string `maxminddb:"names" json:"names"`
+	} `maxminddb:"country" json:"country"`
+	Location struct {
+		AccuracyRadius uint16  `maxminddb:"accuracy_radius" json:'accuracy_radius'`
+		Latitude       float64 `maxminddb:"latitude" json:"latitude"`
+		Longitude      float64 `maxminddb:"longitude" json:"longitude"`
+	} `maxminddb:"location"`
+	RegisteredCountry struct {
+		GeoNameID uint              `maxminddb:"geoname_id" json:"geoname_id"`
+		IsoCode   string            `maxminddb:"iso_code" json:"iso_code"`
+		Names     map[string]string `maxminddb:"names" json:"names"`
+	} `maxminddb:"registered_country" json:"registered_country"`
+}
+
+type ServerConfig struct {
+	Server    string   `mapstructure:"server" yaml:"server"`
+	Latitude  float64  `mapstructure:"latitude" yaml:"latitude"`
+	Longitude float64  `mapstructure:"longitude" yaml:"longitude"`
+	Continent string   `mapstructure:"continent"`
+	Weight    int      `mapstructure:"weight" yaml:"weight"`
+	Protocols []string `mapstructure:"protocols" yaml:"protocols"`
+}
+
+// New creates a new instance of Redirector
+func New(config *Config) *Redirector {
+	r := &Redirector{
+		config: config,
+	}
+
+	r.checks = []ServerCheck{
+		r.checkHttp("http"),
+		r.checkTLS,
+	}
+
+	return r
+}
+
+func (r *Redirector) Start() http.Handler {
+	if err := r.ReloadConfig(); err != nil {
+		log.WithError(err).Fatalln("Unable to load configuration")
+	}
+
+	log.Info("Starting check loop")
+
+	// Start check loop
+	go r.servers.checkLoop(r.checks)
+
+	log.Info("Setting up routes")
+
+	router := chi.NewRouter()
+
+	router.Use(middleware.RealIPMiddleware)
+	router.Use(logger.Logger("router", log.StandardLogger()))
+
+	router.Head("/status", r.statusHandler)
+	router.Get("/status", r.statusHandler)
+	router.Get("/mirrors", r.legacyMirrorsHandler)
+	router.Get("/mirrors/{server}.svg", r.mirrorStatusHandler)
+	router.Get("/mirrors.json", r.mirrorsHandler)
+	router.Post("/reload", r.reloadHandler)
+	router.Get("/dl_map", r.dlMapHandler)
+	router.Get("/geoip", r.geoIPHandler)
+	router.Get("/metrics", promhttp.Handler().ServeHTTP)
+
+	router.NotFound(r.redirectHandler)
+
+	if r.config.BindAddress != "" {
+		log.WithField("bind", r.config.BindAddress).Info("Binding to address")
+
+		go http.ListenAndServe(r.config.BindAddress, router)
+	}
+
+	return router
+}
--- a/servers.go
+++ b/servers.go
@ -1,4 +1,4 @@
-package main
+package redirector

 import (
 	"github.com/jmcvetta/randutil"
@ -6,21 +6,12 @@ import (
 	log "github.com/sirupsen/logrus"
 	"math"
 	"net"
-	"net/http"
-	"net/url"
-	"runtime"
 	"sort"
-	"strings"
 	"sync"
 	"time"
 )

-var (
-	checkClient = &http.Client{
-		Timeout: 20 * time.Second,
-	}
-)
-
+// Server represents a download server
 type Server struct {
 	Available  bool               `json:"available"`
 	Host       string             `json:"host"`
@ -29,110 +20,69 @@ type Server struct {
 	Longitude  float64            `json:"longitude"`
 	Weight     int                `json:"weight"`
 	Continent  string             `json:"continent"`
+	Protocols  ProtocolList       `json:"protocols"`
 	Redirects  prometheus.Counter `json:"-"`
 	LastChange time.Time          `json:"lastChange"`
 }

-func (server *Server) checkStatus() {
-	req, err := http.NewRequest(http.MethodGet, "http://"+server.Host+"/"+strings.TrimLeft(server.Path, "/"), nil)
+type ServerCheck func(server *Server, logFields log.Fields) (bool, error)

-	req.Header.Set("User-Agent", "ArmbianRouter/1.0 (Go "+runtime.Version()+")")
-
-	if err != nil {
-		// This should never happen.
-		log.WithFields(log.Fields{
-			"server": server.Host,
-			"error":  err,
-		}).Warning("Invalid request! This should not happen, please check config.")
-		return
+// checkStatus runs all status checks against a server
+func (server *Server) checkStatus(checks []ServerCheck) {
+	logFields := log.Fields{
+		"host": server.Host,
 	}

-	res, err := checkClient.Do(req)
+	var res bool
+	var err error

-	if err != nil {
+	for _, check := range checks {
+		res, err = check(server, logFields)
+
+		if err != nil {
+			logFields["error"] = err
+		}
+
+		if !res {
+			break
+		}
+	}
+
+	if !res {
 		if server.Available {
-			log.WithFields(log.Fields{
-				"server": server.Host,
-				"error":  err,
-			}).Info("Server went offline")
+			log.WithFields(logFields).Info("Server went offline")

 			server.Available = false
 			server.LastChange = time.Now()
 		} else {
-			log.WithFields(log.Fields{
-				"server": server.Host,
-				"error":  err,
-			}).Debug("Server is still offline")
+			log.WithFields(logFields).Debug("Server is still offline")
 		}
+
 		return
-	}
-
-	responseFields := log.Fields{
-		"server":       server.Host,
-		"responseCode": res.StatusCode,
-	}
-
-	if res.StatusCode == http.StatusOK || res.StatusCode == http.StatusMovedPermanently || res.StatusCode == http.StatusFound || res.StatusCode == http.StatusNotFound {
-		if res.StatusCode == http.StatusMovedPermanently || res.StatusCode == http.StatusFound {
-			location := res.Header.Get("Location")
-
-			responseFields["url"] = location
-
-			log.WithFields(responseFields).Debug("Server responded with redirect")
-
-			newUrl, err := url.Parse(location)
-
-			if err != nil {
-				if server.Available {
-					log.WithFields(responseFields).Warning("Server returned invalid url")
-					server.Available = false
-					server.LastChange = time.Now()
-				}
-				return
-			}
-
-			if newUrl.Scheme == "https" {
-				if server.Available {
-					responseFields["url"] = location
-					log.WithFields(responseFields).Warning("Server returned https url for http request")
-					server.Available = false
-					server.LastChange = time.Now()
-				}
-				return
-			}
-		}
-
+	} else {
 		if !server.Available {
 			server.Available = true
 			server.LastChange = time.Now()
-			log.WithFields(responseFields).Info("Server is online")
-		}
-	} else {
-		log.WithFields(responseFields).Debug("Server status not known")
-
-		if server.Available {
-			log.WithFields(responseFields).Info("Server went offline")
-			server.Available = false
-			server.LastChange = time.Now()
+			log.WithFields(logFields).Info("Server is online")
 		}
 	}
 }

 type ServerList []*Server

-func (s ServerList) checkLoop() {
+func (s ServerList) checkLoop(checks []ServerCheck) {
 	t := time.NewTicker(60 * time.Second)

 	for {
 		<-t.C
-		s.Check()
+		s.Check(checks)
 	}
 }

 // Check will request the index from all servers
 // If a server does not respond in 10 seconds, it is considered offline.
 // This will wait until all checks are complete.
-func (s ServerList) Check() {
+func (s ServerList) Check(checks []ServerCheck) {
 	var wg sync.WaitGroup

 	for _, server := range s {
@ -141,7 +91,7 @@ func (s ServerList) Check() {
 		go func(server *Server) {
 			defer wg.Done()

-			server.checkStatus()
+			server.checkStatus(checks)
 		}(server)
 	}

@ -160,12 +110,12 @@ type DistanceList []ComputedDistance
 // Closest will use GeoIP on the IP provided and find the closest servers.
 // When we have a list of x servers closest, we can choose a random or weighted one.
 // Return values are the closest server, the distance, and if an error occurred.
-func (s ServerList) Closest(ip net.IP) (*Server, float64, error) {
-	choiceInterface, exists := serverCache.Get(ip.String())
+func (s ServerList) Closest(r *Redirector, scheme string, ip net.IP) (*Server, float64, error) {
+	choiceInterface, exists := r.serverCache.Get(scheme + "_" + ip.String())

 	if !exists {
 		var city LocationLookup
-		err := db.Lookup(ip, &city)
+		err := r.db.Lookup(ip, &city)

 		if err != nil {
 			return nil, -1, err
@ -174,7 +124,7 @@ func (s ServerList) Closest(ip net.IP) (*Server, float64, error) {
 		c := make(DistanceList, len(s))

 		for i, server := range s {
-			if !server.Available {
+			if !server.Available || !server.Protocols.Contains(scheme) {
 				continue
 			}

@ -191,9 +141,9 @@ func (s ServerList) Closest(ip net.IP) (*Server, float64, error) {
 			return c[i].Distance < c[j].Distance
 		})

-		choiceCount := topChoices
+		choiceCount := r.config.TopChoices

-		if len(c) < topChoices {
+		if len(c) < r.config.TopChoices {
 			choiceCount = len(c)
 		}

@ -212,7 +162,7 @@ func (s ServerList) Closest(ip net.IP) (*Server, float64, error) {

 		choiceInterface = choices

-		serverCache.Add(ip.String(), choiceInterface)
+		r.serverCache.Add(scheme+"_"+ip.String(), choiceInterface)
 	}

 	choice, err := randutil.WeightedChoice(choiceInterface.([]randutil.Choice))
@ -223,6 +173,13 @@ func (s ServerList) Closest(ip net.IP) (*Server, float64, error) {

 	dist := choice.Item.(ComputedDistance)

+	if !dist.Server.Available {
+		// Choose a new server and refresh cache
+		r.serverCache.Remove(scheme + "_" + ip.String())
+
+		return s.Closest(r, scheme, ip)
+	}
+
 	return dist.Server, dist.Distance, nil
 }

@ -232,9 +189,10 @@ func hsin(theta float64) float64 {
 }

 // Distance function returns the distance (in meters) between two points of
-//     a given longitude and latitude relatively accurately (using a spherical
-//     approximation of the Earth) through the Haversin Distance Formula for
-//     great arc distance on a sphere with accuracy for small distances
+//
+//	a given longitude and latitude relatively accurately (using a spherical
+//	approximation of the Earth) through the Haversine Distance Formula for
+//	great arc distance on a sphere with accuracy for small distances
 //
 // point coordinates are supplied in degrees and converted into rad. in the func
 //
--- a/util.go
+++ b/util.go
@ -1,10 +1,10 @@
-package main
+package redirector

 import "math/rand"

 var letters = []rune("abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ")

-func randSeq(n int) string {
+func RandomSequence(n int) string {
 	b := make([]rune, n)
 	for i := range b {
 		b[i] = letters[rand.Intn(len(letters))]
--- a/util/certificates.go
+++ b/util/certificates.go
@ -0,0 +1,46 @@
+package util
+
+import (
+	"crypto/x509"
+	"github.com/gwatts/rootcerts/certparse"
+	log "github.com/sirupsen/logrus"
+	"net/http"
+)
+
+const (
+	defaultDownloadURL = "https://github.com/mozilla/gecko-dev/blob/master/security/nss/lib/ckfw/builtins/certdata.txt?raw=true"
+)
+
+func LoadCACerts() (*x509.CertPool, error) {
+	res, err := http.Get(defaultDownloadURL)
+
+	if err != nil {
+		return nil, err
+	}
+
+	defer res.Body.Close()
+
+	certs, err := certparse.ReadTrustedCerts(res.Body)
+
+	if err != nil {
+		return nil, err
+	}
+
+	pool := x509.NewCertPool()
+
+	var count int
+
+	for _, cert := range certs {
+		if cert.Trust&certparse.ServerTrustedDelegator == 0 {
+			continue
+		}
+
+		count++
+
+		pool.AddCert(cert.Cert)
+	}
+
+	log.WithField("certs", count).Info("Loaded root cas")
+
+	return pool, nil
+}
Author	SHA1	Message	Date
Tyler	8a4d02c6e2	Cleanup hosts map All checks were successful continuous-integration/drone/push Build is passing Details	2022-08-18 23:43:58 -04:00
Tyler	d22b87da10	Fix config loading issue when servers don't resolve	2022-08-18 23:42:54 -04:00
Tyler	4d7c836810	Update README to include check information and code quality All checks were successful continuous-integration/drone/push Build is passing Details	2022-08-15 02:46:25 -04:00
Tyler	43205337c1	Update README All checks were successful continuous-integration/drone/push Build is passing Details	2022-08-15 02:41:31 -04:00
Tyler	caa0fb43e4	Resolve issues with checks, forced http detection All checks were successful continuous-integration/drone/push Build is passing Details	2022-08-15 02:37:51 -04:00
Tyler	7f27df70ae	Remove dependency on go.mod/go.sum for ginkgo running Some checks failed continuous-integration/drone/push Build is failing Details	2022-08-15 02:17:12 -04:00
Tyler	e7236b13de	Massive refactoring, struct cleanup, supporting more features Some checks failed continuous-integration/drone/push Build is failing Details Features: - Protocol lists (http, https), managed by http responses - Working TLS Checks - Root certificate parsing for TLS checks - Moving configuration into a Config struct, no more direct viper access	2022-08-15 02:16:22 -04:00
tyler	3e7782e5ec	Merge pull request 'Initial testing and improvements of code' (#1 ) from feature/testing into master All checks were successful continuous-integration/drone/push Build is passing Details Reviewed-on: #1	2022-08-14 08:03:49 +00:00
Tyler	2f71e97f2e	Improve readme with proper information and configuration All checks were successful continuous-integration/drone/push Build is passing Details continuous-integration/drone/pr Build is passing Details	2022-08-14 04:02:10 -04:00
Tyler	5ff4aa9fae	go.sum magically removed ginkgo All checks were successful continuous-integration/drone/push Build is passing Details	2022-08-14 03:50:11 -04:00
Tyler	c4bd02485c	Fix tests and add coverage Some checks failed continuous-integration/drone/push Build is failing Details	2022-08-14 03:49:28 -04:00
Tyler	3f71aced93	Improve test coverage and documentation Some checks failed continuous-integration/drone/push Build is failing Details	2022-08-14 03:42:49 -04:00
Tyler	3c5656284c	Fix extra character on cgo environment variable All checks were successful continuous-integration/drone/push Build is passing Details	2022-08-06 16:25:05 -04:00
Tyler	08da75d309	Disable cgo	2022-08-06 16:24:45 -04:00
Tyler	8ea77adee2	Update go.sum Some checks failed continuous-integration/drone/push Build is failing Details	2022-08-06 16:23:50 -04:00
Tyler	91b99572c2	Ensure ginkgo is installed Some checks failed continuous-integration/drone/push Build is failing Details	2022-08-06 16:22:51 -04:00
Tyler	ffcb068ff5	Initial testing implementation of checks Some checks failed continuous-integration/drone Build is failing Details	2022-08-06 16:20:46 -04:00
Tyler	290e5575cc	Initial testing implementation of checks	2022-08-06 16:19:12 -04:00