While a great database, etcd is not devoid of issues. In this post, I'll show how you can use Apache with MySQL.

etcd is an excellent key-value distributed database used internally by Kubernetes and managed by the CNCF. It's a great option, and that's the reason why Apache uses it too. Yet, it's not devoid of issues.

First, some mention scalability, but one can expect this from a distributed data store that values consistency. Another issue may be the need for more familiarity with etcd. It's relatively new, so your Ops team may need help operating it correctly while having decades of operating MySQL or Postgres. Finally, only a few etcd users are aware that it lacks maintainers:

In the last few months, primary maintainers G.L. (Amazon, announcement) and S.B. (Red Hat) have stopped actively participating in the project. This leaves the project with only one active and two occasionally-reviewing maintainers, M.S. (Google), P.T. (Google), both are relatively new to the project (1 month and 1 year of tenure) and S.P.Z. (IBM). Other maintainers are either dormant or have very minimal activity over the last six months. The project is effectively unmaintained (emphasis mine).

-- Google Groups of Kubernetes Steering Committee, March 2022

For all those reasons, you may prefer to use a standard SQL database with Apache . In this post, I'll show how you can use MySQL.

The kine project​

It would be a lot of effort if each product had to introduce an abstraction layer and different adapters for both etcd and other databases. kine is a project that aims to offer a translation step between etcd calls and other implementations:

Kine is an etcdshim that translates etcd API to:

• SQLite
• Postgres
• MySQL
• NATS Jetstream

Features

• Can be ran standalone so any k8s (not just K3s) can use Kine
• Implements a subset of etcdAPI (not usable at all for general purpose etcd)
• Translates etcdTX calls into the desired API (Create, Update, Delete)

-- Kine (Kine is not etcd)

In essence, kine is a Go library that translates etcd calls to the datastore you want (among those implemented).

Yet, using kine directly is a non-trivial effort. Fortunately, api7, the company that gave Apache to the Apache Software Foundation, provides a component already focused on usage.

ETCD adapter​

ETCD adapter wraps kine to be -specific:

ETCD Adapter mimics the ETCD V3 APIs best effort. It incorporates the kine as the Server side implementation, and it develops a totally in-memory watchable backend.

Not all features in ETCD V3 APIs supported, this is designed for Apache , so it's inherently not a generic solution.

-- ETCD adapter

Two things of note:

• At the moment of this writing, the adapter supports either local in-memory storage or MySQL
• It's available as an embeddable library but also as a standalone component

Therefore, we can design our architecture as the following:

Demo​

Let's implement the above architecture with an additional admin UI over MySQL. I'll use Docker Compose:

version: "3"services:  :    image: apache/:3.4.0-debian                         #1    volumes:      - ./config.yaml:/usr/local//conf/config.yaml:ro    ports:      - "9080:9080"      - "9180:9180"    depends_on:      - etcd-adapter    restart: always                                           #2  etcd-adapter:    build: ./etcd-adapter                                     #3    volumes:      - ./adapter.yml:/etcd-adapter/conf/config.yaml:ro       #4    depends_on:      - mysql    restart: always                                           #2  mysql:    image: bitnami/mysql:8.0                                  #5    ports:      - "3306:3306"    environment:      MYSQL_ROOT_PASSWORD: root      MYSQL_USER: etcd      MYSQL_PASSWORD: etcd      MYSQL_DATABASE:   adminer:    image: adminer:standalone                                 #6    ports:      - "8080:8080"    environment:      ADMINER_DEFAULT_SERVER: mysql    depends_on:      - mysql

1. Latest version of Apache , yeah!
2. To avoid any failure with dependencies between containers, restart until it works. Kubernetes's manifests would involve health checks
3. api7.ai still needs to provide a container. We need to build from the source code
4. Override the default configuration file with a context-specific one
5. The regular MySQL image didn't work for me. Let's take the one from Bitnami
6. Adminer, formerly known as PHP myAdmin, will help to visualize the database state

ETCD-adapter's configuration looks like this:

server:  host: 0.0.0.0                 #1  port: 12379log:  level: infodatasource:  type: mysql                   #2  mysql:    host: mysql                 #3    port: 3306                  #3    username: etcd              #3    password: etcd              #3    database: 

1. Bind any IP since Docker will assign a random one
2. Implementation type. The default is btree; we need to change it.
3. As configured in the docker-compose.yml file

Finally, here's Apache configuration:

deployment:  admin:    allow_admin:      - 0.0.0.0/0  etcd:    host:      - "http://etcd-adapter:12379"   #1

1. Use this etcd instance, which is the adapter

Testing​

Now that we are set let's test our system by creating a route:

curl -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -d '{  "methods": ["GET"],  "uris": ["/get"],  "upstream": {    "nodes": {      "httpbin.org:80": 1    }  }}' http://localhost:9180//admin/routes/1

We can now get it:

curl -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' http://localhost:9180//admin/routes/1

We can also check via the Adminer interface that it has been persisted via MySQL:

Unfortunately, we need to stop at this point. Getting all routes doesn't work:

curl -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' http://localhost:9180//admin/routes

{"header":{"revision":"1689689596"},"message":"Key not found"}

Worse, using the route fails:

curl localhost:9080/get

{"error_msg":"404 Route Not Found"</