Feuille de route Odyssey: que voulons-nous d'autre de l'extracteur de connexion. Andrey Borodine

Le rapport Andrei Borodin explique comment ils ont pris en compte l'expérience de mise à l'échelle de PgBouncer dans la conception de l'extracteur de composés Odyssey lors de sa mise en production. De plus, nous discuterons des fonctions de l'extracteur que nous aimerions voir dans les nouvelles versions: il est important pour nous non seulement de couvrir nos besoins, mais aussi de développer la communauté des utilisateurs d' Odyssey .

Vidéo:

salut! Je m'appelle Andrey.

Chez Yandex, je développe des bases de données open source. Et aujourd'hui, nous avons un sujet sur les connexions de pooler de connexions.

Si vous savez comment appeler le pooler de connexion en russe, dites-moi. Je veux vraiment trouver un bon terme technique qui devrait être établi dans la littérature technique.

, connection pooler . - , , , Postgres . ( HighLoad++ 2019) Postgres. , , , , , , . , .

Postgres. . . , Postgres.

– primary , ( ), , .

– Postgres, Postgres connection pooler. Connection pooler fencing .

connection pooler?

Postgres . , – , Postgres. , .

, Postgres , procArray. . procArray , . , . , .

3 :

• .
• .
• , . . .

, pooler . PostgreSQL Professional . , . . application-side pool proxy pool.

Application-side pool – . : .

, , .

, , -. client side pooling . – 10 000 . , .

proxy poolers, poolers, . poolers. poolers + . Pgpool Crunchy-Proxy.

, , . , poolers pooling, . . , .

, PgBouncer, transaction pooling, . . .

– . .

, , . loopback - .

, application_name_add_host. Bouncer IP- application_name. application_name .

, – , – , . – application_name, , .

, Bouncer pool, . . , .

? , C++ - , , . 20 000 , . .

, , Bouncer, , . . pool.

Postgres, . . .

, . PgBouncer , . : , , , , , - . . , , .

, .

, Bouncer . . , , pooler.

, PgBouncer’.

https://lwn.net/Articles/542629/

Bouncer.

, TCP- Bouncers. TCP- round-robin’ .

, . . , Bouncer, idle- Bouncer’.

, 3 Bouncers 100 %. Bouncers. ?

TLS. . Postgres TLS TLS, , , TLS handshake .

, . primary , – . . . - primary , -. TLS.

TLS handshake Bouncer, . - . retry exponential backoff, .

16 PgBouncer, 16 100 %.

PgBouncer. , Bouncer. Bouncers TCP handshake, Bouncers pooling, , .

. 18 Bouncers . . , DevOps , , .

, open source, Bouncer . , , PgBouncers . pull request .

https://www.postgresql.org/docs/current/libpq-cancel.html

https://github.com/pgbouncer/pgbouncer/pull/79

. Postgres , - . TCP-reset, . . . Bouncer? . . , , Bouncer , , .

upstream Bouncer’.

, connection pooler, , , , , .

. TLS-.

, Machinarium, . libpq, , : « . IO, , IO ». . . " N, N ", " SYNC, ". , . , . , , (green thread-). , , .

, TCP accept round-robin' workers TPC-.

. cache-friendly.

, , TCP-stack.

, , Odyssey CANCEL ROLLBACK , . . , Odyssey , , .

. application_name_add_host. , , .

.. managed PostgreSQL connection pooler, , . . , , . Bouncer .

.

, . , , , . , connection pooler Odyssey, .

Odyssey PgBouncer. , . - , pull request, issue GitHub, . PgBouncer .

, , error forwarding. , . , , , .

, 100%- PgBouncer. , Bouncer, .

Odyssey.

https://github.com/yandex/odyssey/pull/66

, «Pause /Resume». . Postgres, connection pooler, pg_upgrade, resume. , . . , . ( )

https://github.com/yandex/odyssey/pull/73 —

, PgBouncer, SCRAM Authentication, , .. – .

Odyssey, .

Odyssey, . Kiwi – Postgres’ . . . proto 3 Postgres – , , . Kiwi.

Machinarium – . Machinarium . , , 15 .

Odyssey. , coroutines. accept TCP- workers.

worker . crone- , pool.

Odyssey Postgres. install-check Bouncer Odyssey, div. , Bouncer Odyssey.

, , . Odyssey.

, - : Postgres + Odyssey, PgBouncer + Odyssey, Odyssey + Odyssey , , , Odyssey - , - , .

Odyssey production. , , . , . . , . , production , PostgreSQL Professional , memory leak. , . .

, connection pooler TLS- TLS-. .

Bouncer Odyssey pcache, pcache , Odyssey . , . , 20 000 .

Pluggable Authentication Method – lunux’ . PgBouncer , PAM PgBouncer, PAM.

. . ?

, , PAM- PAM-, PAM- PAM-. , . , .

, , accept . worker pool, TLS handshake.

, 20 000 , . libpq . 3 .

, , retry.

, PgBouncer , throttling TCP-, accept.

, , , , . , handshake , . - , .

Roadmap

Odyssey? ?

2019 .

roadmap Odyssey :

• SCRAM PAM authentication.
• forward standby.
• online-restart.
• .

roadmap , . . , .

forward read-only queries to standby? , . failover switchover. - - . , -, .

, Postgres, 10- session_attrs. , : . , , session_attrs.

, . - , . , , Odyssey , .

Odyssey primary. , , , , . .

, open source. , , 2,5 PgBouncer. Odyssey.

prepared statement. prepared statement . -, SQL-, «prepared». SQL- SQL Bouncer. overkill, , . . . SQL-.

prepared statement proto3. , , prepared statement, . , - prepared statements. , - .

, - , , prepared statements , , . . prepared statement.

, , prepared statement, . , , , , , , , .

, . , PgBouncer. . – : - , - – . .

, , , , .

– 1.0 ( 1.1). , Odyssey 1.0rc, . . release candidate. , , , memory leak.

1.0? Odyssey . , 1 000 000 , , , 1.0.

, 1.0 SCRAM. , production , SCRAM, . , , .

pull request. , Bouncer. . , - , .

, . !

application_name, , transaction pooling Odyssey?

Odyssey Bouncer?

Odyssey. Bouncer .

.

, ?

, . , application_name. , . . , .

, , ! ! , Odyssey . . multi data-source , Odyssey , . . slave, failover .

, , , . storages. . , , failover, pg_recovery. , . - ? . . , . . , C, .

, adoption . , . . , .

. , . , , , . , , , . , , , , , . . . , , .

, . pcache , , real cache , , , .

- , , , , . . .

. , . , , , 10 . , .

, .

. . , , . , .

Nginx a une telle option slowly startdans le cluster pour le serveur. Et il augmente progressivement la charge.

Oui, bonne idée, nous allons essayer quand nous y arriverons.