Bonjour, Habr! Je m'appelle Sasha Kozlov, je développe l'infrastructure et l'administration système à Avito depuis trois ans et demi. Je vais vous dire comment nous avons évolué et modernisé notre travail avec le code d'infrastructure au fil du temps et l'avons porté à un niveau qualitativement nouveau.
Nous sommes responsables du cycle complet de gestion des équipements: de l'approvisionnement et de l'installation dans le DC à la livraison à l'utilisateur final. Il s'agit des leçons que nous avons apprises ces dernières années, en travaillant avec des milliers d'équipements et plusieurs milliers de configurations.
Nous parlerons des outils IaC «première génération» tels que Ansible, Chef, Salt, Puppet. Si vous travaillez avec une infrastructure sur site et que vous ne disposez pas de votre propre cloud sur la virtualisation, vous utilisez probablement déjà l'un de ces outils.
Puppet. - , , Puppetlabs . , , , , . , — .
, . , . - push-, - pull, ansible pull-. — Python, Jinja- YAML-, — DSL Ruby. : , PuppetDB, , .
, . git revert, , . , , . → ’ , 0 → ’. .
, Puppet. - — , , . , :
- , ;
- ;
- CI-, ;
- ;
- , , ;
- «» Configuration Drift.
, , . — . .
. : , , .
Puppet , . , .. pull-. — . 30 , , , Puppet . , event-driven , Salt, .
, . stateful-, , . .
: Puppet stateful-, , . , DNS- Puppet, . , , , API.
, . , . , , git . . .
: control repo . git-. , Puppet-, "The roles and profiles method".
Control repo — . , CI- - . — , . — , « ?». : k8s- , Kafka , ClickHouse- ..
— , . - . Puppet -, , . , , , .
control repo, .
— , , , . control repo, semver . 50 : , , Kubernetes .
, , Puppet. . , , . , control repo?
, , . control repo , . , . , , . Docker, . control repo, . , .
, , :
- Building a Functional Puppet Workflow Part 2: Roles and Profiles
- Roles and Profiles in a Control Repo?
- Workflows Evolved: Even Besterer Practices
- Profiles and the Path to Hiera Data
- Puppetlabs Best Practices Docs
- control repo, . , , , :
External Node Classifier
, , — , ?
node definitions , , . , - :
node /^avi-ceph(2[1-9]|3[0-9]|4[0-9]|5[0-9]|6[0-9]|7[0-9]|8[0-9]|9[0-9])/ {
...
}. — , . , , . Puppetlabs External Node Classifier.
External Node Classifier — , , . , . ENC , top-scope variables. control repo , , , node definition:
node default {
include base # , control repo
if $::role != '' {
notify{ "Node ${::fqdn} has role ${::role}": loglevel => info }
include "role::${role}"
} else {
notify{ "Node ${::fqdn} has no role": loglevel => warning }
}
}ENC , . CMDB, . CMDB netbox Digital Ocean, . razor, , , -.
CMDB - , 15-20 , , .
«» ? , Configuration Drift, . Configuration Drift — , , . — , , , - Puppet.
, , , . , . , k8s-.
, API-. netbox API, . PXE netboot- Debian, preseed' , API, IPMI Redfish API. , , , .
, , . , .
. : , , , , .
, — , . — . . “write-only” , .
3.7 6 Puppet, , . :
- .
- -.
:
- , ..
- , , duplicate resource declaration. : , , .
- Docker-, , inspec.
Kubernetes Puppet "Kubernetes The Hard Way", .
, , smoke-, . , , .
, Beaker , , -, . ~~ smoke-: ~~ , .
, , :
context 'application deployment' do
it 'can deploy an application into a namespace and expose it' do
shell('systemctl restart kubelet')
shell('count=0;
while [[ $(kubectl get pods -n tiller -l name=tiller -o \'jsonpath={..status.conditions[?(@.type=="Ready")].status}\') != "True" ]];
do
if [[ count -gt 180 ]]; then
break;
fi;
sleep 1 && ((count++));
done')
shell('kubectl create -f /tmp/nginx.yaml', acceptable_exit_codes: [0]) do |r|
expect(r.stdout).to match(%r{namespace/nginx created\nconfigmap/my-nginx-config created\ndeployment.apps/my-nginx created\nservice/my-nginx created\n})
end
end
it 'can access the deployed service' do
shell('count=0;
while [[ $(kubectl get pods -n nginx -l run=my-nginx -o \'jsonpath={..status.conditions[?(@.type=="Ready")].status}\') != "True" ]];
do
if [[ count -gt 180 ]]; then
break;
fi;
sleep 1 && ((count++));
done')
shell('curl --connect-timeout 1 --retry-delay 1 --retry-max-time 300 --retry 150 -s --retry-connrefused 10.100.10.5', acceptable_exit_codes: [0]) do |r|
expect(r.stdout).to match %r{Welcome to nginx!}
end
end
end, . CI .
CI PR. PR : . , , , CI PR, , .
, :
- puppet-syntax
- puppet-lint
- rspec-puppet
- puppetlabs-spec-helper
- test-kitchen, Puppet: kitchen-docker-puppet-example
- beaker
-:
Development kit
, CI/CD — , workflow.
- — PDK, . , , . PDK control repo Kitchen Docker. PDK Beaker, .
, :
- , : , CI .
- , , -.
- , -, .
- control repo, , .
- .
- .
development tool, CI , .
(puppet-rspec, puppet-linter, test-kitchen) , . , , .
, , « ». , .
, , . , — . : Forge. Puppet Forge , .
Puppet . , : metadata.json Puppetfile. , - . . librarian-puppet, , r10k, . , . — , : acceptance- -.
. , git-, :
mod 'dba-clickhouse',
:git => 'ssh://git@github.com/iac/dba-clickhouse.git',
:ref => '1.2.2'
mod 'dba-kafka',
:git => 'ssh://git@github.com/iac/dba-kafka.git',
:ref => '1.2.0', , . Puppet Forge, , librarian-puppet.
semver, , . , , — :
# Puppetfile
mod 'arch-puppetserver', '0.20.5' #
mod 'arch-vault', '~> 2.1' #
mod 'si-lxc' # , . , , , . :
[22:39:43] in dba-control on production via ruby-2.5.1 at ️ unstable
$ iack dep show
[] Collecting modules metadata
FULL NAME | CURRENT VERSION | LATEST VERSION | OUT OF DATE?
---------------------|-----------------|----------------|---------------
si-lxc | latest | 0.3.2 | N/A
si-base | latest | 1.3.1 | N/A
petems-hiera_vault | v0.4.1 | | Major
arch-vault | 2.1.0 | 2.1.0 | No
dba-postgresql | 0.1.2 | 0.1.3 | Tiny
dba-pgbouncer | 0.4.0 | 0.5.1 | Minor
si-grub | 0.1.0 | 0.1.0 | No
si-collectd | 0.2.3 | 0.2.4 | Tiny
si-confluent | 0.3.0 | 0.3.0 | No
dba-redis | 0.2.3 | 0.2.3 | No
dba-collectd_plugins | latest | 0.2.0 | N/A
dba-mongodb | 0.2.1 | 0.2.1 | No
dba-patroni | 0.1.4 | 0.2.4 | Minor
dba-cruise_control | 0.1.1 | 0.1.2 | Tiny
dba-lxd | 0.7.0 | 0.7.0 | No
dba-clickhouse | 1.2.1 | 1.2.2 | Tiny
dba-zookeeper | 2.0.0 | 2.0.0 | No
si-td_agent | 0.1.0 | 0.1.0 | No
dba-kafka | 1.1.6 | 1.2.1 | Minor
arch-puppetserver | 0.20.1 | 0.20.2 | Tiny
pcfens-filebeat | 4.1.0 | 4.4.1 | Minor
KyleAnderson-consul | 5.0.3 | 6.0.1 | Major
puppetlabs-apt | 6.3.0 | 7.4.2 | Major
puppetlabs-stdlib | 5.2.0 | 6.3.0 | Major:
- Managing environment content with a Puppetfile
- puppet-forge-server — ruby Sinatra, Puppet Forge
- librarian-puppet
code style
code style ? — , . , , , . review — , .
Puppet . The puppet language style guide . Puppet-lint, CI, .
, . control repo:
development kit, «»:
—
, . Puppet Hiera Vault. hiera-backend, vault hiera_lookup.
-, , - . , :
$token_data = vault::secret_field('tokens.csv', 'data')token_data 'data' tokens.csv, Vault. Vault , , Hiera:
$ vault-util ls puppet/arch/
common/
nodes/
roles/ , , common. — roles/ nodes/.
— , -, , , . TLS- .
, , , . , Puppet:
Canary
Puppet , control repo. control repo , . ENC .
, . , . — - .
, , , .
Puppet
Puppet, , , . DSL — Ruby. Hiera — , . , . Hiera , , .
Puppet — , , . . , : , Hiera, . - , , .
Puppet Ruby , . , pet project. , , — . : , , , .
— . , pull-. , , . , , pull push. Bolt, Puppetlabs , , PuppetDB, .
, , :
- , .
- .
- .
- .
- CI .
- , workflow .
- External Node Classifier .
- .
- .