Léim go dtí an t-ábhar

Fabhtcheartú etcd

Faisnéis

Ní mór an nós imeachta ar fad a leanúint mar root. Chun do phribhléidí a ardú go root, is féidir leat an t-ordú su - a úsáid.

Is stór eochrach-luacha dáilte é etcd a úsáidtear mar phointe comhordaithe idir nóid na braisle PostgreSQL. Láraíonn agus comhroinneann sé staid na braisle go hiontaofa, lena n-áirítear aitheantas an cheannaire, staid na gcomhaltaí agus na meicníochtaí glasála.

Braitheann patroni ar etcd chun toghchán an nóid cheannaire a stiúradh agus chun cliseadh thar a spreagadh go huathoibríoch nuair a tharlaíonn teagmhas. Suiteáilte ar gach nód, ráthaíonn etcd comhsheasmhacht fhoriomlán na braisle, gan páirt a ghlacadh i stóráil ná i bpróiseáil sonraí PostgreSQL.

Braitheann patroni ar etcd dá bhrí sin, agus ní féidir leis oibriú i gceart mura bhfuil etcd ag obair.

Faisnéis

Stóráiltear cumraíocht etcd sa chomhad /etc/default/etcd ar gach nód.

Logs

Braitheann an nós imeachta ar fad ar logaí na seirbhíse etcd, a fhaigheann tú leis an ordú thíos:

1
journalctl -u etcd

Leid

Chun na logaí a leanúint i bhfíor-am, cuir an rogha -f leis an ordú: 

1
journalctl -fu etcd

Faisnéis

Tá logaí etcd ar fáil in /var/log/syslog freisin.

Mínithe ar logaí earráide

health check for peer <PEER_PSQL_NODE_ID> could not connect: dial tcp <PEER_IP_PSQL_NODE>:2380: i/o timeout

Mura bhfuil an sreabhadh líonra TCP/2380 oscailte idir dhá nód, téann an ceangal thar am agus scríobhann etcd na logaí seo a leanas:

1
2
3
4
5
6
Apr 15 11:48:09 PSQL_1 etcd[1342933]: health check for peer <PEER_PSQL_NODE_ID> could not connect: dial tcp <PEER_IP_PSQL_NODE>:2380: i/o timeout
Apr 15 11:48:09 PSQL_1 etcd[1342933]: health check for peer <PEER_PSQL_NODE_ID> could not connect: dial tcp <PEER_IP_PSQL_NODE>:2380: i/o timeout
Apr 15 11:48:14 PSQL_1 etcd[1342933]: health check for peer <PEER_PSQL_NODE_ID> could not connect: dial tcp <PEER_IP_PSQL_NODE>:2380: i/o timeout
Apr 15 11:48:14 PSQL_1 etcd[1342933]: health check for peer <PEER_PSQL_NODE_ID> could not connect: dial tcp <PEER_IP_PSQL_NODE>:2380: i/o timeout
Apr 15 11:48:19 PSQL_1 etcd[1342933]: health check for peer <PEER_PSQL_NODE_ID> could not connect: dial tcp <PEER_IP_PSQL_NODE>:2380: i/o timeout
Apr 15 11:48:19 PSQL_1 etcd[1342933]: health check for peer <PEER_PSQL_NODE_ID> could not connect: dial tcp <PEER_IP_PSQL_NODE>:2380: i/o timeout

Is é <PEER_PSQL_NODE_ID> an ID a shann etcd don nód sprice, agus is é <PEER_IP_PSQL_NODE> seoladh IP an nóid sprice.

Sampla

Inár sampla, tá an bhraisle PostgreSQL cumraithe mar seo a leanas:

Nód IP an nóid ID etcd
PSQL_1 192.168.1.1 53d2c129945ccb8b
PSQL_2 192.168.1.2 7176dd381f583d83
PSQL_3 192.168.1.3 e3d5ef565a5bb46c

Nuair a ritear an t-ordú journalctl -fu etcd ón bhfreastalaí PSQL_1, faightear na logaí seo a leanas:

1
2
3
4
5
Apr 15 11:50:39 PSQL_1 etcd[1342933]: health check for peer 7176dd381f583d83 could not connect: dial tcp 192.168.1.2:2380: i/o timeout
Apr 15 11:50:44 PSQL_1 etcd[1342933]: health check for peer 7176dd381f583d83 could not connect: dial tcp 192.168.1.2:2380: i/o timeout
Apr 15 11:50:44 PSQL_1 etcd[1342933]: health check for peer 7176dd381f583d83 could not connect: dial tcp 192.168.1.2:2380: i/o timeout
Apr 15 11:50:49 PSQL_1 etcd[1342933]: health check for peer 7176dd381f583d83 could not connect: dial tcp 192.168.1.2:2380: i/o timeout
Apr 15 11:50:49 PSQL_1 etcd[1342933]: health check for peer 7176dd381f583d83 could not connect: dial tcp 192.168.1.2:2380: i/o timeout

Léiríonn na logaí seo nár éirigh leis an gceangal ón bhfreastalaí PSQL_1 leis an bhfreastalaí a bhfuil an seoladh IP 192.168.1.2 (PSQL_2) agus an ID etcd 7176dd381f583d83 aige, ar an bport TCP/2380: chuaigh sé thar am. Is féidir linn a thátal go bhfuil an sreabhadh líonra ón bhfreastalaí PSQL_1 chuig an bhfreastalaí PSQL_2 á chaitheamh i leataobh (DROP) ag balla dóiteáin.

Is féidir go gcaitheann balla dóiteáin logánta an fhearais PostgreSQL an sreabhadh seo i leataobh (DROP): seiceáil cumraíocht an bhalla dóiteáin ar an bhfearas ar na nóid lena mbaineann. Má tá an chumraíocht seo ceart, tá balla dóiteáin de chuid an bhonneagair ag caitheamh an tsreafa líonra TCP/2380 i leataobh (DROP) idir an dá nód.

health check for peer <PEER_PSQL_NODE_ID> could not connect: dial tcp <PEER_IP_PSQL_NODE>:2380: connect: connection refused

Nuair a eitítear ceangal idir dhá nód, tuairiscíonn etcd é leis na logaí seo a leanas:

1
2
3
4
5
6
Apr 15 14:04:30 PSQL_1 etcd[1342933]: health check for peer <PEER_PSQL_NODE_ID> could not connect: dial tcp <PEER_IP_PSQL_NODE>:2380: connect: connection refused
Apr 15 14:04:30 PSQL_1 etcd[1342933]: health check for peer <PEER_PSQL_NODE_ID> could not connect: dial tcp <PEER_IP_PSQL_NODE>:2380: connect: connection refused
Apr 15 14:04:35 PSQL_1 etcd[1342933]: health check for peer <PEER_PSQL_NODE_ID> could not connect: dial tcp <PEER_IP_PSQL_NODE>:2380: connect: connection refused
Apr 15 14:04:35 PSQL_1 etcd[1342933]: health check for peer <PEER_PSQL_NODE_ID> could not connect: dial tcp <PEER_IP_PSQL_NODE>:2380: connect: connection refused
Apr 15 14:04:40 PSQL_1 etcd[1342933]: health check for peer <PEER_PSQL_NODE_ID> could not connect: dial tcp <PEER_IP_PSQL_NODE>:2380: connect: connection refused
Apr 15 14:04:40 PSQL_1 etcd[1342933]: health check for peer <PEER_PSQL_NODE_ID> could not connect: dial tcp <PEER_IP_PSQL_NODE>:2380: connect: connection refused

Is é <PEER_PSQL_NODE_ID> an ID a shann etcd don nód sprice, agus is é <PEER_IP_PSQL_NODE> seoladh IP an nóid sprice.

D'fhéadfadh cúiseanna éagsúla a bheith leis an eiteach:

  • Tá mífheidhm ar an tseirbhís etcd nó tá sí stoptha ar an nód sprice. Sa chás seo, seiceáil stádas (systemctl status etcd) agus logaí na seirbhíse etcd ar an nód sin.
  • Eitíonn balla dóiteáin (REJECT) an sreabhadh líonra TCP/2380 idir an dá nód. D'fhéadfadh gurb é balla dóiteáin logánta an fhearais é nó balla dóiteáin de chuid an bhonneagair.
Sampla

Inár sampla, tá an bhraisle PostgreSQL cumraithe mar seo a leanas:

Nód IP an nóid ID etcd
PSQL_1 192.168.1.1 53d2c129945ccb8b
PSQL_2 192.168.1.2 7176dd381f583d83
PSQL_3 192.168.1.3 e3d5ef565a5bb46c

Nuair a ritear an t-ordú journalctl -fu etcd ón bhfreastalaí PSQL_1, faightear na logaí seo a leanas:

1
2
3
4
5
Apr 15 14:04:15 PSQL_1 etcd[1342933]: health check for peer e3d5ef565a5bb46c could not connect: dial tcp 192.168.1.3:2380: connect: connection refused
Apr 15 14:04:15 PSQL_1 etcd[1342933]: health check for peer e3d5ef565a5bb46c could not connect: dial tcp 192.168.1.3:2380: connect: connection refused
Apr 15 14:04:20 PSQL_1 etcd[1342933]: health check for peer e3d5ef565a5bb46c could not connect: dial tcp 192.168.1.3:2380: connect: connection refused
Apr 15 14:04:20 PSQL_1 etcd[1342933]: health check for peer e3d5ef565a5bb46c could not connect: dial tcp 192.168.1.3:2380: connect: connection refused
Apr 15 14:04:25 PSQL_1 etcd[1342933]: health check for peer e3d5ef565a5bb46c could not connect: dial tcp 192.168.1.3:2380: connect: connection refused

Léiríonn na logaí seo gur eitíodh an ceangal ón bhfreastalaí PSQL_1 leis an bhfreastalaí a bhfuil an seoladh IP 192.168.1.3 (PSQL_3) agus an ID etcd e3d5ef565a5bb46c aige, ar an bport TCP/2380.

Ar an bhfreastalaí PSQL_3, léiríonn an t-ordú systemctl status etcd go bhfuil etcd stoptha:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
○ etcd.service - etcd - highly-available key value store
    Loaded: loaded (/lib/systemd/system/etcd.service; enabled; preset: enabled)
    Active: inactive (dead) since Wed 2026-04-15 16:09:34 CEST; 38s ago
Duration: 1min 36.087s
    Docs: https://etcd.io/docs
            man:etcd
    Process: 1400558 ExecStart=/usr/bin/etcd $DAEMON_ARGS (code=killed, signal=TERM)
Main PID: 1400558 (code=killed, signal=TERM)
        CPU: 49.071s

Apr 15 12:26:32 PSQL_3 etcd[1197556]: finished scheduled compaction at 1665243 (took 583.297µs)
Apr 15 13:26:32 PSQL_3 etcd[1197556]: store.index: compact 1665949
Apr 15 13:26:32 PSQL_3 etcd[1197556]: finished scheduled compaction at 1665949 (took 574.496µs)
Apr 15 14:26:32 PSQL_3 etcd[1197556]: store.index: compact 1666658
Apr 15 14:26:32 PSQL_3 etcd[1197556]: finished scheduled compaction at 1666658 (took 567.196µs)
Apr 15 15:26:32 PSQL_3 etcd[1197556]: store.index: compact 1667368
Apr 15 15:26:32 PSQL_3 etcd[1197556]: finished scheduled compaction at 1667368 (took 568.096µs)
Apr 15 16:09:34 PSQL_3 systemd[1]: etcd.service: Deactivated successfully.
Apr 15 16:09:34 PSQL_3 systemd[1]: Stopped etcd.service - etcd - highly-available key value store.
Apr 15 16:09:34 PSQL_3 systemd[1]: etcd.service: Consumed 49.071s CPU time.

Nuair a thosaítear etcd, réitítear an fhadhb.

rejected connection from "<PEER_IP_PSQL_NODE>:35956" (error "remote error: tls: bad certificate", ServerName "PSQL_1")

Má tá deimhnithe na braisle PostgreSQL imithe in éag, nó mura ndearnadh iad a shuiteáil nó a athnuachan i gceart ar an nód óna bhfuil tú ag léamh na logaí, feictear an earráid seo a leanas i logaí etcd:

1
2
3
4
5
6
Apr 15 16:38:54 PSQL_1 etcd[1407644]: rejected connection from "<PEER_IP_PSQL_NODE>:49274" (error "remote error: tls: bad certificate", ServerName "PSQL_1")
Apr 15 16:38:54 PSQL_1 etcd[1407644]: rejected connection from "<PEER_IP_PSQL_NODE>:49294" (error "remote error: tls: bad certificate", ServerName "PSQL_1")
Apr 15 16:38:54 PSQL_1 etcd[1407644]: rejected connection from "<PEER_IP_PSQL_NODE>:49288" (error "remote error: tls: bad certificate", ServerName "PSQL_1")
Apr 15 16:38:55 PSQL_1 etcd[1407644]: rejected connection from "<PEER_IP_PSQL_NODE>:49316" (error "remote error: tls: bad certificate", ServerName "PSQL_1")
Apr 15 16:38:55 PSQL_1 etcd[1407644]: rejected connection from "<PEER_IP_PSQL_NODE>:49304" (error "remote error: tls: bad certificate", ServerName "PSQL_1")
Apr 15 16:38:55 PSQL_1 etcd[1407644]: rejected connection from "<PEER_IP_PSQL_NODE>:60958" (error "remote error: tls: bad certificate", ServerName "PSQL_1")

Is é <PEER_IP_PSQL_NODE> seoladh IP nóid amháin de chuid na braisle.

Léiríonn an loga seo, nuair a rinne an nód a bhfuil tú ceangailte leis iarracht ceangal a bhunú le nóid eile na braisle (<PEER_IP_PSQL_NODE>), gur eitigh siad é, agus earráid i ndeimhniú an nóid reatha á tuairisciú acu.

Mar sin, ba cheart duit bailíocht na ndeimhnithe braisle a sheiceáil ar an nód a bhfuil tú ceangailte leis.

Rabhadh

Má tá an t-údarás deimhniúcháin atá cumraithe ar nód cianda (in /etc/etcd/ca.crt) mícheart, feictear an earráid freisin ar an nód óna bhfuil tú ag léamh na logaí: léiríonn sí ansin nach féidir leis an nód cianda an deimhniú a cuireadh i láthair a bhailíochtú, toisc nach bhfuil an CA ceart aige.

Sampla

Inár sampla, tá an bhraisle PostgreSQL cumraithe mar seo a leanas:

Nód IP an nóid ID etcd
PSQL_1 192.168.1.1 53d2c129945ccb8b
PSQL_2 192.168.1.2 7176dd381f583d83
PSQL_3 192.168.1.3 e3d5ef565a5bb46c

Nuair a ritear an t-ordú journalctl -fu etcd ón bhfreastalaí PSQL_1, faightear na logaí seo a leanas:

1
2
3
4
5
6
Apr 15 16:38:54 PSQL_1 etcd[1407644]: rejected connection from "192.168.1.2:49274" (error "remote error: tls: bad certificate", ServerName "PSQL_1")
Apr 15 16:38:54 PSQL_1 etcd[1407644]: rejected connection from "192.168.1.3:49294" (error "remote error: tls: bad certificate", ServerName "PSQL_1")
Apr 15 16:38:54 PSQL_1 etcd[1407644]: rejected connection from "192.168.1.2:49288" (error "remote error: tls: bad certificate", ServerName "PSQL_1")
Apr 15 16:38:55 PSQL_1 etcd[1407644]: rejected connection from "192.168.1.2:49316" (error "remote error: tls: bad certificate", ServerName "PSQL_1")
Apr 15 16:38:55 PSQL_1 etcd[1407644]: rejected connection from "192.168.1.3:49304" (error "remote error: tls: bad certificate", ServerName "PSQL_1")
Apr 15 16:38:55 PSQL_1 etcd[1407644]: rejected connection from "192.168.1.3:60958" (error "remote error: tls: bad certificate", ServerName "PSQL_1")

Léiríonn na logaí seo gur eitigh na nóid PSQL_2 agus PSQL_3 an ceangal ón nód PSQL_1, toisc nach bhfuil deimhniú PSQL_1 bailí sa chomhthéacs seo.

Mar sin, seiceálaimid bailíocht na ndeimhnithe ar an nód PSQL_1 tríd an nós imeachta seiceála deimhnithe braisle a leanúint:

1
2
openssl x509 -in /etc/etcd/peer.crt -noout -enddate
openssl x509 -in /etc/etcd/ca.crt -noout -enddate

Léiríonn an t-aschur go bhfuil deimhniú an nóid imithe in éag:

1
2
notAfter=Dec 18 14:04:09 2025 GMT
notAfter=Dec 17 14:03:47 2030 GMT

Sa chás seo, ní mór duit deimhnithe na braisle PostgreSQL a athnuachan.

error listing data dir: /var/lib/etcd/cleanroom

Mura bhfuil na ceadanna cearta ag etcd ar an gcomhadlann /var/lib/etcd/cleanroom, ní thosaíonn an tseirbhís agus scríobhann sí na logaí seo a leanas:

1
2
3
4
5
6
Apr 10 14:15:26 PSQL_1 etcd[454701]: Go OS/Arch: linux/amd64
Apr 10 14:15:26 PSQL_1 etcd[454701]: setting maximum number of CPUs to 1, total number of available CPUs is 1
Apr 10 14:15:26 PSQL_1 etcd[454701]: error listing data dir: /var/lib/etcd/cleanroom
Apr 10 14:15:26 PSQL_1 systemd[1]: etcd.service: Main process exited, code=exited, status=1/FAILURE
Apr 10 14:15:26 PSQL_1 systemd[1]: etcd.service: Failed with result 'exit-code'.
Apr 10 14:15:26 PSQL_1 systemd[1]: Failed to start etcd.service - etcd - highly-available key value store

Chun an fhadhb seo a réiteach, sann na ceadanna cearta don chomhadlann leis an ordú seo a leanas:

1
chown -R etcd:etcd /var/lib/etcd/cleanroom